We’ve just completed the ‘alpha’ phase of the project to redevelop newzealand.govt.nz by creating a prototype website. Prototyping is a method used to get feedback from users about future designs. In recent times, many governments around the world have used prototyping as a way of increasing the user-centred focus of the site they’re building.
We built our prototype site knowing we’d throw it away to make something that was better. This is kind of scary stuff, and not how most of the project team have been used to working, but it made us focus on what we needed to learn, instead of rushing to build a finished product.
We kept the prototype simple and did just enough to give us a site we could test. To get the site ready we:
- published about 150 pages of content, across a range of topics
- created some landing pages to help people navigate the site
- installed a local search engine
- left the site deliberately looking a bit rough and the design wasn’t polished
The team worked on the prototype site between September and the end of January. We actually delivered two prototypes, each one with a slightly different layout. Both versions of the prototype were finished on schedule too.
We spent several days in January and February testing the prototype with real New Zealanders. During the testing we made some tweaks, changed some content, modified the design for a few key templates and tested again. We now have pages and pages of research notes and hours of video to digest.
Once again, we worked with Optimal Usability. They helped up recruit test participants, and they ran the test sessions.
Here are the main documents that cover this round of research:
- The testing scenarios and test script (PDF 207KB) (DOC 76KB)
- The user testing report (PDF 6.69MB) (DOC 8MB)
- Implications for ‘beta’ (PDF 80KB) (DOC 92KB)
Sometimes Putting Yourself in Your User’s Shoes Is an Uncomfortable Experience
Watching user testing can be a very humbling experience and sometimes it can make you really uncomfortable too.
During our most recent testing, we asked people to seek out information relating to various situations like “having trouble paying your rent,” “find out if you can get out of jury service,” and more common tasks like “find out where you can pay a speeding fine”. Several participants didn’t always feel they could easily complete the task we had set them.
People think about tasks in different ways, and their different life experiences influence the language they use. Watching what people did on the site pointed to a few problems we could easily fix. The labels for some of the links in the prototype content didn’t quite make sense, contact details for services were still hard to find. I had expected we’d see these sorts of problems, but I was also pleased to see that our overall approach seemed to be working.
Then something happened during the testing that I didn’t expect. Some of the people we tested said it was their fault
that they couldn’t find the right content, that they were not good enough using computers
, and worse, that it was just something they had to live with
. People were apologising to us for not being able to finish some of the tasks. You could see their self-confidence plummet, and this was just a test, they should have been blaming the site for making the test hard to finish, but they were blaming themselves.
A group of us were observing the tests from another room, and it was hard to watch. I thought to myself, “This is not acceptable. No one should feel like this.”
Thankfully, most of the users we tested felt more empowered, and even when they didn’t complete their tasks easily they thought the site would help them deal with “government stuff” more easily. Several users said once they got more familiar with the site it would be exactly what they needed. Here’s what a few of the users had to say:
The consistent layout on each page was good.
The info that was needed was right there, so there was no need to search further.
I like the cleaner layout compared to some of the sites it links through to.
The Lessons We’ve Learned Have Influenced Our ‘Beta’ Plans
We received great feedback on a number of key features and we observed consistent patterns in user behaviour, so if we focus on these areas first, we should end up with a more usable website when the site goes live.
We will need to continue development and adding new features after the site is available to the public, launching the site before it’s finished just helps us collect more feedback along the way.
A variety of themes emerged from the user testing, I’ll run through the highlights here. If you want more detail be sure to download the full set of research documents.
Keep Search as Simple as Possible
The ‘alpha’ site enabled users to search both published content and to search the whole of the government domain. 100% of users we tested said they would start their search for information using Google. Most of them also said they would go back to Google if they couldn’t find something. Local search was useful but only to look for content on the site they were already on.For ‘beta’: The search engine will only include ‘beta’ content.
Make Content Easy to Find Using Different Pathways
On ‘alpha’ we implemented:- automatic suggestions on search queries
- featured links on the homepage and topic pages
- content tags
- a ‘fat footer’ across the site
- filters that could be used on search results
Ensure Content Supports a Complete User Journey
Users responded really well to pages that summarised the full-range of services that were available around a particular topic, but they still struggled a bit when looking for contact information.For ‘beta’: We’re calling these ‘sign-post pages’. Linking together services around a specific task is the biggest opportunity for the ‘beta’ site. This is content that government doesn’t really provide now, especially where the full range of services are provided by multiple departments.
We also need to do some more work on how contact details will be displayed. We’ve got some ideas, but this is an area that will need some more testing. Making the ‘beta’ live to the public is probably the best way to discover what users really need. We don’t need to solve these tricky design questions upfront.
Create the Consistent Experience Users Expect
The ‘alpha’ was, as one user put it,A bit messy, and that was fine with us. Showing users something that wasn’t quite finished helped them feel like they could give us ideas on how to make it better. They did, however, like the consistency of the design. It didn’t matter which task they were completing, the basic layout and structure was the same. By the time users had completed two or three testing tasks they knew how the site worked and later tasks were easier to complete.
Each time they were linked out to another website, the experience wasn’t so smooth. In many cases, the inconsistency in the designs slowed users down as they had to ‘relearn’ how each worked. Content on some of the sites forced users to jump back and forth in and out of pages, some of the sites used layouts that were so unique and unexpected they blocked users from completing their task completely.
For ‘beta’: The site will be a publicly available site so it has to look good. We’ve already started work on the first set of templates and we’re experimenting with content from different topics to see if we can create the consistent experience users expect. Early feedback from people who’ve seen the beta mock-ups is, The pages look stunning
. I can’t wait to show you more about what we’re building, but that will have to wait for another blog post.
Come on the Journey With Us
We’ve got a busy few months ahead of us here at Internal Affairs. There’s content, designs, templates, and systems to build and test. There’ll be lots of testing to do, and not just technical testing. We’ve got plenty of opportunities planned to show off our ‘beta’ site to users and get their feedback, and we’ll continue to share the lessons we learn along the journey.
Have you got any user research you can share? Let me know and I can help you get it published here on the Web Toolkit.
We’ll publish a sneak preview of the ‘beta’ design as soon as we’ve got something ready to share, but until then, if you’ve got any questions about ‘beta’ feel free to leave a comment.
