Testing is Broken. Let's Fix It.

By John Bates | 9/13/17

digital_AI_Hero.jpg

If you’re in the software development and testing realm, you know how rapidly things have changed in a short amount of time. Digital business is here to stay and companies in all sectors are disrupting their business models and undergoing digital transformations to stay relevant. They’re turning to agile development approaches and DevOps to spark innovation and rapidly deliver new, cutting-edge products and solutions that meet evolving customer desires.

Yet, despite these efforts, customer satisfaction isn’t great. Apple’s App Store has more than 2.2 million apps available, and Google Play offers upwards of 2.8 million. Yet nearly 1 in 4 people who download an app only use it once, and 51 percent of users don’t download any apps in a month. Test teams often become the scapegoat for low user engagement and adoption because they can’t keep up with the pace of DevOps and deliver what the business wants.

But sagging customer satisfaction is more than just an execution problem. There are fundamental flaws in the current approaches to testing that were created 10–15 years ago, and that just don’t cut it anymore. And, factors like the digital experience, consumerization, and the internet of things (IoT) have completely changed how software products are architected and produced, what they do, who uses them, and user expectations.

There’s a user experience gap. Testing at many organizations is still more about checking code compliance rather than about delighting users—resulting in low user satisfaction, adoption, conversion, and retention. And this doesn’t just apply to B2C apps. Enterprise software and employee-facing apps suffer from the same low user engagement and adoption.

There’s a productivity gap in time to market and efficiency. Mobile, web, IoT, and technology diversification, for example, mean QA teams have to test on myriad platforms (not just one as in the past), and create thousands (not hundreds) of tests to cover end-to-end user journeys. Simply put, the scope of testing is increasing, while time to delivery is shrinking.

There’s an automation gap. New devices, payment systems, smart TVs, connected cars, two-factor authentication, microservices, rich user interfaces, interactive images, highly responsive web pages, and complex, end-to-end user scenarios. Traditional test automation tools can’t test these, and manual testing is even worse from an efficiency and productivity standpoint.

There’s a confidence and predictability gap. Teams really have no idea what impact a new release will have on user satisfaction until it’s actually released.

There’s a visibility gap. When it comes to user experience, teams are running blind in production, which just increases that user experience gap.

It’s time for a new approach.

Testing is almost an infinite process. You’ve got test objectives, test cases, test scripts, test environments, select test runs, test execution, test results analysis to look for correlations and root causes that get fed back into the code. All of these phases require significant effort and intelligence. Our CTO likens the testing process to someone visiting a city. There are places of interest (functions), different things to do (data), roads between neighborhoods (flow), different modes of transportation (cross-platform), unexpected events (pop-ups), and other variations, and it’s tough to create every possible visitor itinerary.

Lots of people talk about test automation, but if you look at the entire testing process, we’ve really only automated one—albeit critical—aspect: test execution. Which is great because it’s extremely time consuming, but it’s still just one element of the testing process. And teams still perform a significant amount of manual testing. A major time sink is creating test scripts. Another is analyzing test results. We should be able to automate a lot of these processes too.

Modern testing has to be about technology metrics and business metrics, such as user satisfaction and retention. Modern testing needs to handle complex architectures (including the edge of IoT), and ensure complex apps can scale and perform appropriately across platforms, devices, mobile front ends, and cloud or on-premises back ends.

Clearly, organizations need a new approach to testing. One that’s straightforward, fast, and reliable to roll out, enables teams to keep up with DevOps, scales testing efficiently, addresses the automation gap, and drives user satisfaction, adoption, conversion, retention—and revenue.

The future of testing involves artificial intelligence.

According to a recent study by Boston Consulting Group and MIT Sloan Management Review, 84 percent of respondents believe artificial intelligence (AI) will be instrumental in gaining competitive advantage. AI is already transforming the way we work and live, and will have a significant impact on how software is developed.

AI and analytics will be the catalysts to deliver true test automation—recommending tests to carry out, learning continuously, intelligently predicting business impacts, and enabling development teams to fix issues before they occur. The results? Faster time to market, happier customers, better conversion and retention rates, and more revenue and profitability.

Today at Testplant, we announced our new, user-centric approach to testing. We call it Digital Automation Intelligence, and it uses AI, machine learning, and analytics to help teams continuously create amazing digital experiences and predict business and user impacts across different interfaces, platforms, and devices. So you can:

  • Transform testing from a compliance function to a profit center.
  • Shrink time to market by testing new versions, fast.
  • Accelerate productivity by scaling test coverage without blowing your budget.
  • Keep up with the pace and evolution of DevOps.
  • Radically increase productivity by extending automation to test case creation, test execution, and results analysis.

Want to learn even more about how our approach can benefit your business? Check out our most recent webinar, “How AI, Machine Learning, and Analytics Will Transform Testing.”

And, look for our next blog post about the brains behind our Digital Automation Intelligence approach.

Topics: Automation testing, Digital Experience, User Experience, UX, Automated testing tools, UX testing, AI

John Bates

Written by John Bates

Dr. John Bates is the CEO of Testplant, a visionary technologist and highly accomplished business leader. He's also the author of the book “Thingalytics: Smart Big Data Analytics for the Internet of Things.”

Stay up-to-date with the latest in test automation

Lists by Topic

see all