Over the last few months I have been working with Nathan Wall finding out how agencies manage their web analytics reporting and what their needs are. We’ve now created some easy to understand dashboard reports. It’s a solution that:
- can be automated
- will help agencies with routine web analytics reporting
- will encourage consistent and common reporting across government
- will reduce the feeling some people have of being overwhelmed by data.
Creating easier to understand dashboard reports
Dashboards! Everything and everyone needs a dashboard! But what should dashboards show on them? How do we provide content owners with easy to understand information about what’s happening on their sites so they can use that information to make informed decisions and prioritise improvements.
After speaking to a number of agencies we identified 3 basic types of dashboard needed:
- "traffic" dashboards
- "engagement" dashboards
- "experience" dashboards.
We’ve built examples of each of the 3 dashboard types to find out:
- if the concept of common web analytics reporting will be useful to agencies
- if there is a common approach to web analytics reporting we can use across different websites
- what’s needed to automate the collection of data to build the dashboards
- if there is an opportunity for public release of web analytics data.
A common approach to reporting
We think it’s possible to use a common approach to report basic analytics about our websites.
For the proof of concept, we created dashboards for 11 different websites across government. All websites have a purpose: some require the user to complete a task; others require the user to find information. No matter what the purpose, we found all the dashboards we built could have a common layout.
Traffic dashboards provide basic stats such as:
- pageviews for the entire site
- the total number of visits to the site
- top 10 pages visited
- the top country of origin where visitors came from
- the types of browsers and devices visitors to the site are using.
Sound familiar? Well it should. A lot of agencies report on these high-level stats in their monthly and quarterly reports. We think we can make that reporting a bit easier, giving people more time to look a little deeper into what the data reveals.
Engagement dashboards and what they report on varies from site to site, however the way in which the engagement is presented on a dashboard can be consistent. We’ve measured engagement by looking at key user journeys and measuring the proportion of users that did tasks on a site, such as:
- downloading certain documents
- working step-by-step through collections of content pages
- clicking on external links needed to complete a specific task
- returning to the site within a set number of days.
Working out what to report on means site managers and content owners need to be clear about what success means for their site. Not such a bad thing if you ask me!
Is “engagement” the “right” word to use for these reports? Well to be honest we’re not completely sure, but haven’t been able to come up with something else. If you can think of a word that says “Did your users actually do what you wanted them to do?” in a simpler way, let us know.
We think “Experience” dashboards are where the real value starts to be shown. Using a feature in Google Analytics called segments, we map out specific user journeys and then measure the completion rate – meaning, how many users got to the end of the user journey.
This technique doesn’t confirm things like users were able to find what they were looking for, or that they understood they were reading, but it’s an indicator at least in broad-terms that the site was meeting at least some of their needs.
In addition to an overall completion rate, we can segment users by things like the type of device they used, and see if that had any impact on their ability to complete the task or journey.
Automating the dashboards
We’ve built the proof of concept dashboard interface using the Common Web Platform (CWP). If the dashboards are turned into a live service, this does not mean your website needs to be on CWP to take advantage of the easier reporting. You just need to have Google Analytics running over your site and you’d need to give the dashboard tool we’ve built access to your analytics data. We’ll use Google’s API to pull in the data to the dashboard, so once the initial set up is complete the dashboard will update automatically.
We have used the free Google Analytics tool, and the Google Analytics API for the proof-of-concept. We are looking at what the extra features Google Analytics Premium would give us, including:
- access to the raw data
- guaranteed retention of data
- the ability to analyse user journeys across government websites.
What agencies are telling us
When showing website owners the reports we created, they could quickly and clearly see whether users are finding their site easy to use. They could also see if their content (not just their transactions) was being used effectively. The dashboards would give them knowledge and evidence to help prioritise and measure site improvements.
The automation of reporting is extremely appealing to agencies. They can’t wait to spend less time copying and pasting numbers into spreadsheets, and more time actually improving the quality of their websites.
How much time do you spend collating data rather than analysing it? Would an automated dashboard by useful to you?
Public release of web analytics data
We think our dashboards can help us create better public services for everyone. The dashboards would also promote more openness in government. It’s not about pointing fingers and blaming agencies for things that aren’t working. It’s about finding the things we’re doing right, and doing more of it.
If we keep building these dashboards and offering a service to agencies, we don’t yet know where the dashboards would be published. We could publish them on:
- psi.govt.nz so the dashboards remain accessible to government employees, but not to the general public (psi.govt.nz is not accessible outside the public sector).
- a site such as data.govt.nz making the data open to all.
Where would you feel happy having your website data published? What would stop you from releasing the data in a more open way?
There are a few things we need to do to get the show on the road
We need to:
- refine the dashboard designs to ensure they are accessible
- update some of our code to ensure we comply with web standards
- show our prototypes to more agencies and get feedback so we can make the reports even easier to understand
- decide where we should publish the dashboards
- investigate what other tools are needed, including the possibility of Google Analytics Premium, to support an all-of-government service
- define an operating model that would support agencies long-term – and this includes working out how to fund the service, and what agencies would (probably) need to pay to use it.
And finally… a big thank you
A huge shout out to the team at the Digital Transformation Office in Australia. They’re on a similar journey and the conversations we’re having with them, the information they’re sharing, and their awesome collaborative approach to working with us is really going beyond the call. From us here in New Zealand, a huge thank you!