In this phase you develop and refine your service based on feedback, and prepare it to become a live and fully functional system. Your focus now is working with users to move from a prototype towards a viable, sustainable service. Depending on your product, there may be a lot to consider to lay a proper foundation for a production service. Don't underestimate the amount of effort you'll need to invest in this phase.

Even though you're in beta, it's essential to have assurance from the outset that the service you're building meets the standards and policies required on the .govt.nz domain – privacy, security, accessibility, good practice in information management and strategic alignment with government priorities.

Questions to answer in the beta phase

As your beta progresses and is refined, you should consider and answer these questions.

User feedback and ongoing research

• What can you learn from the evolution of Govt.nz?
  • Govt.nz set an example for some of the issues you'll face in moving from a prototype to a sustainable service. You should review how user feedback shaped its evolution.
• Does your organisation have a service design team who can provide advice?
  • If not, can you partner with another agency that can provide advice?
• How can service design methods help shape your service?
  • User experience design is complicated. Review the methods and tools you have available to decide what can best inform your design decisions.
• How are you testing your design choices?
  • By now you should be undertaking a rigorous analysis of how users experience your service, and you should have a method for dealing with feedback that can inform design decisions and project priorities.
    User testing analysis
• How is your content performing?
  • Do you have an ongoing programme of user testing in place? Does it test users' comprehension of your content? Does it offer any insights to the effectiveness of your information architecture and labelling?
• Is your analytics solution giving you enough evidence?
  • By now your analytics should be well tuned to provide detailed insights into how the service is used, and its performance against business requirements.
  • Are there ways in which it could be better tuned before you move into production? If you're just tracking visits and page views you may want to consider additional metrics. Can you track completion rates for your service? How are you measuring bounce rates, pathways through the service and user satisfaction?

Privacy

• Does the project team understand the Information Privacy Principles?
  • Project members and managers must be certain the service is designed and built to appropriately handle and store any personally identifying information. The service needs to follow all the Information Privacy Principles.
• Will the users of your service be advised of their rights in regard to information they provide?
  • The NZ Government Web Usability Standard describes your obligations to advise users of their rights over their personal information.
    Web Usability Standard
• Do you need to review your service design with your organisation’s Privacy Officer?
  • If your service requires users to disclose any personally identifying information – or even if it doesn’t but deals with potentially sensitive information – is your Privacy Officer comfortable with your approach?

Security

• Have you documented your security testing and assurance plans?
  • Security testing must be an integral part of your project planning. You need to ensure that enough time, budget and resource is available to provide adequate levels of assurance during development. At some time your system will need to go through a security accreditation process.
  • Is it in accordance with the requirements of the NZISM?
• Does your development team have expertise and process in place to assure you that your security requirements will be met?
  • Can they articulate the OWASP Top Ten, for example, and how they are relevant to your project? Your developers and project team must be aware of your security requirements and be able to demonstrate how they are embedded in the development process.
  • Is there a security champion on the team?
  • Are they reviewing their approach with your security advisers from time to time?
• Have you developed policies and processes in place to preserve the ongoing security of the service?
  • While a service may be adequately secured when it’s launched, its security profile can be eroded over time without adequate management. All services can be expected to meet a minimum baseline; services dealing with more sensitive information will need more extensive measures to safeguard their security.
  • What would be the consequences if the confidentiality, integrity or availability of the service were compromised?

Accessibility

• Have you put in place appropriate tools and quality assurance processes to ensure your content can be understood?
  • What specific design research has been done around issues like literacy levels and the readability of your content? Can your team help content owners write in plain English?
• Are you planning an accessibility audit, and checking the basics during each sprint or phase of development?
  • In the desire to build the latest best features into your service, sometimes things slip through the design process. Simple tests like ensuring your design meets the minimum colour contrast required, that your service can be navigated using only a keyboard, how forms handle erroneous input and testing using different-sized screens and devices are easy things to do. Errors on simple things like these can be indicators of deeper issues.
• How are accessibility issues being handled?
  • How do your developers and content authors go about testing for and identifying accessibility issues? What are they doing to deal with them? Are fixes being prioritised?
  • Are accessibility checks part of each sprint?
  • Does your user testing include people with disabilities?

Information and data management

• How will the service meet basic Information Management principles?
  • You need to be sure your users know who they're dealing with, that the information they're being given is reliable, trustworthy, timely and accurate, and that it's being actively managed.
• How will content versioning or archiving be handled as the service evolves?
  • If needed, how will you recreate a point-in-time record of the service?
  • Are versioning and archive facilities adequate to recreate a previous snapshot? Are logging and record-keeping facilities adequate to provide a reliable history of user interaction with the service? Would they be sufficiently robust to withstand a potential legal challenge?
• How do you plan to make sure that users can continue to have confidence and trust in the service you'll provide?
  • What measures will you take to ensure the ongoing integrity, authority and reliability of your service?
• Is it clear to users what they can and cannot do with your service?
  Web Usability Standard
  
  • Does your service carry clear licensing and/or terms of use statements? Does it describe how it can be used or re-used, and what to do if they want to suggest improvements (or discover a flaw)? Do users have a safe and reliable way to provide feedback?
  • Web content should generally be licensed for re-use under a NZGOAL licence.

Governance

• Who will be the owner of the service when it moves into a live phase?
  • The owner of the service will be accountable for its performance against business objectives and KPIs, and for ensuring that it meets required standards for operating on the government web domain.
  • The service owner must be able to provide ongoing assurance that the service continues to meet user needs, meets security, privacy and accessibility obligations and that day-to-day operations are adequately resourced and managed.
  • The service owner should ensure that the service has a roadmap, and that the roadmap is maintained and supported by adequate budget.
• What governance structures and processes will be needed to oversee the strategic direction of the service when it moves into production?
  • What level and frequency of reporting will the governance bodies need for effective oversight of the service? What metrics will be needed to inform the reporting against the service's KPIs?
• What resourcing will be needed to manage the day-to-day operations of the service when it moves into production?
  • How will funding be secured? What skills will be needed to manage site operations? What procedures need to be set up to support the service?
• What level and extent of documentation will be needed to support the service in production?
  • At the very least, there should be an operations guide or manual, a product roadmap and a system security plan. More complex services will need a more extensive suite of documentation.

Output from the beta phase

At the end of this phase your service should be either approved to move into a production environment or closed down with lessons learned. Approval to proceed should be straightforward if you have the following:

1. A functional service ready to move into a production environment.
2. Operating budget including allocation for periodic testing and ongoing enhancement, including a backlog of further enhancements to be developed.
3. Service roadmap outlining the technical direction the service will evolve in.
4. A detailed content roadmap or content strategy, including a style guide for the service (language and visual style).
5. An updated user research plan that reflects learnings from the discovery, alpha and beta phases.
6. Documented support processes, change management process, operating guide or operator training materials.
7. Security documentation and system security plan as required by the Protective Security Requirements.
8. Analytics data collection mechanism to allow measurement of service uptake and KPIs, and ideally a plan for making that data open and publicly available.
9. Documented governance structure and responsibilities.
10. Formal signoff by your chief executive that the service meets required government standards and policies, and acceptance of residual risks, before going into live phase.