Can You Handle the Load?

By Antony Edwards | 11/22/17

In the past few weeks, I’ve been getting daily emails about early access to retailers’ Black Friday and Cyber Monday sales. Which got me thinking about two things: one, I hope retailers are prepared for the even earlier onslaught of online traffic, and two, the high stakes for site performance on the two busiest shopping days of the year.

Google data reveals that 53 percent of mobile users abandon sites that take longer than 3 seconds to load. According to Kissmetrics, even a one second delay in page response can translate to a seven percent reduction in conversions, and 79 percent of shoppers who are dissatisfied with website performance are less likely to buy from the same site again.

Adobe Digital Insights data shows that online sales for Black Friday last year reached historic numbers: $3.34 billion, including more than a billion from mobile devices. Clearly, a bad user experience could mean a serious hit to retailers’ bottom lines and reputations—and big money in their competitors shopping carts. Companies like Macy’s, Williams Sonoma, and Victoria’s Secret learned these lessons the hard way after their 2016 Black Friday site debacles.

On the positive side, data shows that improving site load speed from 8 seconds to 2 seconds can increase conversion rate by 74 percent. Last year, Neiman Marcus (1.130 seconds), Dell Inc. (1.834), and Apple (2.155) were the big winners with the fastest-loading sites on Black Friday.

Testing teams play a huge role in ensuring that users have great online shopping experiences and that your site performs and loads as it should. But teams are at a disadvantage if they’re using testing methods from 10–15 years ago.

Think about responsiveness testing (time to render and time to interaction) and how much it matters from the user perspective. Current performance testing approaches are fundamentally flawed:

  • A server-centric approach can’t detect a huge percentage of performance issues. And even for those issues that it can detect, it can’t pinpoint the actual impact on the user.
  • A typical GUI test automation tool works entirely at the code level and can prove that an object exists, but it can’t tell you whether a user can see and interact with it.

Think about cross-platform testing, which is key to website performance—ensuring it works across all major browsers, devices, and all their active versions. Traditional testing tools depend on specific hooks within a browser as well as code-level details of your website that differ between browsers. So, you often have to create test automation scripts per browser, and sometimes per browser versions. A necessary but massively time-consuming task.

And think about load testing, which is critical in figuring out whether or not your website or app can handle the load that your active users can put on it. Unfortunately, many QA teams use traditional tools that can’t effectively address all the possible load-testing scenarios.

At Testplant, we take a new approach to testing that we call Digital Automation Intelligence. It really measures time to render and time to interaction through the eyes of the user, not at the code level. It includes the end-to-end flow to detect all performance issues, regardless of root cause. It can simulate multi-user loads and different network conditions. It lets you create a wide variety of virtual users to mimic user activity, including by simulating the HTTP requests that a real user would send while navigating your website. So you really can test the entire user experience—from the user perspective—and catch any issues before the onslaught of online shoppers on Black Friday, Cyber Monday, or any other big shopping day.

In the very near future, we predict that artificial intelligence (AI) will significantly change testing as we know it today. Test teams will be able to do things like auto-generate time-intensive scripts, analyze results to predict bugs, and adjust scripts to improve test coverage—all great capabilities to ensure that you don’t become a Black Friday statistic.

Until that happens, though, consider this thought about Black Friday from a buyer’s perspective: Don’t offer deals if you’re not prepared to handle the load. Sage advice from the masses.

Happy Thanksgiving and happy shopping.

Topics: cross-platform, Load testing, User Experience, Performance testing, User experience testing

Antony Edwards

Written by Antony Edwards

He’s our CTO. He guides where the product goes. Antony likes haiku.

Stay up-to-date with the latest in test automation

Lists by Topic

see all