Testing the performance of your PWA

On the Web with Justin Willis

2018-07-29

Performance, performance, performance. Performance, particularly loading performance, is key to a good PWA. With studies such as this study from DoubleClick by Google showing that sites that load within 5 seconds on the average network (3g) have 70% longer sessions, 35% lower bounce rates, and 25% higher ad viewability than the average website, there is a real economic benefit to making sure your PWA loads fast for your users. I will be covering techniques on how to make your PWA fast in future posts, but an important starting point to performance is measuring the current performance of your PWA. While there are many ways to test the loading performance of your PWA, today I will cover the two easiest ways i’ve found.

What is considered good performance?

Based on studies such as the DoubleClick study mentioned above, 5 - 7 seconds to interactive on a 3g network and an average device (think Moto G) is the standard target that should be met to ensure the best experience for users. Anything over that and you risk losing large amounts of users.

Measurements

While there are a few different standard measurements that are used to measure loading performance, the two I normally pay attention too are called First Meaningful Paint and Time To Interactive. First Meaningful Paint is an internal browser heuristic that measures when the main content of a page has been rendered. In other words, when the First Meaningful Paint point has been reached that means the user can now see the main content of your app. Time To Interactive is a measurement that measures when the main thread is able to start handling user input, webfonts are loaded and visible and the layout has stabilized. In simple terms this means the point when your app has loaded enough to be usable.

Lighthouse

Ligthouse is a tool from the Google Chrome team that, among many other things, allows you to easily test the performance of your PWA in an emulated environment that matches the average network speed and device that a real user will have. One of the most convenient things about Lighthouse is that it is built right into Chrome Dev Tools. Because of this, testing with Lighthouse is incredibly easy!

To get started with Lighthouse you first need to make sure you have the Chrome Dev Tools open on a tab where you have your PWA open. If you dont already, typing Ctrl + Shift + i will open the devtools for you. Once you have the devtools open you need to navigate to the Audits tab in devtools. On the Audits tab you will see a button that says Perform an audit. Before clicking this button I recommend closing any other chrome tabs you have open along with stopping any intensive programs currently running on your computer. Once you have clicked the Perform an audit button I recommend making sure only the Performance section is checked before starting the audit. While Ligthouse can audit many very important things such as Accessibility, when only testing performance it will make your test go faster if you only run the Performance section.

Once ligthouse has succesfully run you will see an audit report displayed. This report will include alot of info, but I find the most important numbers are the first meaningful paint and time to interactive scores. If these times are under the 5 second mark then you are good to go.

WebPageTest

WebPageTest is a more advanced tool that can give you detailed info on your loading performance. WebPageTest lets you run tests on actual devices with actual 3G networks which gives you very accurate numbers. WebPageTest also integrates with Lighthouse! This combines the best parts of Ligthouse with the accurate, real world test that WebPageTest enables.

To get started with WebPageTest go to https://www.webpagetest.org/easy. This url will get you started with the preferred testing setup, a Moto G4 on a 3g connection. Before starting the test make sure the Run Lighthouse Audit checkbox is checked. Once that is checked you can run the test.

One the test has succesfully run you should see a new page that includes a full perf report on your PWA. Because the full report can be a little overwhelming I normally use the lighthouse report that WebPageTest generates. This can be reached by clicking the square labeled Lighthouse PWA Score towards the top of the page. This lighthouse audit will give you the same information as running lighthouse in Chrome Dev Tools but with the added advantage of these numbers being from a test on a real average device and real 3g network instead of emulated.