Website user testing results – June 2021
Our website revamp project is well underway. To make sure we’re on the right track, we did more user testing over the winter to see if our proposed new website structure really is more user-friendly. To make sure the evidence supports the new structure, we compared it to baseline testing of our current website done in October 2020. The data proves that our new website structure will be much more user-centric and task-focused.
Method
Out of the ten tasks chosen to represent what people would come to the Social Security Tribunal (SST) website for, we gave participants five randomly assigned tasks to complete in round one, and six tasks to complete in round two.
With each task, participants got scores in three categories.
Correct – self-serve
Applies when the user can figure out the correct solution to the task
Correct – contact/help
Applies when the user chooses the ‘Contact the SST’ link or chooses pages related to getting help for the specific appeal
Wrong
Applies when the user can’t figure out how to complete the task and moves on to the next one
Results
Correct – self-serve
Increased from 27% to 58%.
More than doubled self-serve success rate (215% increase).
Correct – contact/help
Decreased from 17% to 14%
Shows greater confidence in ability to self-serve because less people choose this option and more people are able to figure out the correct solution.
Wrong
Decreased from 56% to 28%
Halved the number of wrong answers and identified clear fixes to most remaining ones.
Improvements by task
We measured task improvement by comparing our current website structure to the new one we’re working on.
| Task | Total correct |
|---|---|
| Find the list of documents you need to give the SST to appeal a decision. | 40 63 58% improvement |
| Find out how long it will take to get a decision about your Canada Pension Plan appeal at the General Division. | 62 71 15% improvement |
| Find out how to change representatives during the Old Age Security appeal process at the Appeal Division. | 7 92 1183% improvement |
| Find out who can help with an Employment Insurance appeal in Victoria. | 43 80 87% improvement |
| The General Division dismissed a Canada Pension Plan Disability appeal but you received a new medical report that could affect the decision. Find out how to ask the SST to reconsider your appeal. | 27 62 132% improvement |
| Find out if there are any fees involved with submitting an Employment Insurance appeal. | 43 27 36% decrease |
| Find the form you need to fill out when you file an Old Age Security appeal to the General Division. | 63 88 40% improvement |
| Your hearing for your Canada Pension Plan (non-disability) appeal with the Appeal Division is next week. Find out who will be at the hearing. | 40 79 96% improvement |
| Find out whether an appellant who doesn’t speak English or French can get an interpreter at their Old Age Security hearing with the General Division. | 64 90 41% improvement |
| Find out how to send your updated contact information to the SST. | 55 69 27% improvement |
Who did we talk to?
Feedback
During both rounds of testing, we asked participants how we could improve the current website. Here are some suggestions we heard.
