Inside the Maze — Unmoderated Usability test
Aka: A small guide about Maze.
Context
This month I used Maze to conduct an unmoderated usability test for the project OneHealth. The main reasons that made my decision to use it were:
- Learning a new tool (I like to learn new things)
- Reach more people (I also needed more data)
- Have some data pre-analysed (Maze does it greatly, saves time!)
- Generate data quickly (No need for scheduling, or taking long-hours to analyse the data)
Preparing the prototype (Intro)
Maze supports prototypes from Figma, Adobe XD, Sketch, InVision and Marvel.
My prototype was created on Figma. The first issue that I had was about the file size… To solve this, I did some adjustments and only left the necessary frames on the prototype.
Tip: Create a file that will only have your prototype, without any other pages. Decreases the file size as much as possible. If you do this, it will decreases the loading time, thus reducing the users drop-off.
Preparing the Maze
After uploading the prototype. You are able to set the test(missions).
Maze uses paths to determine success rate, the direct success will only be counted if the user follows the path(s) that you pre-configurated on Maze. If the user reaches the end-screen of a path by any other mean it will count as indirect success.
Tip: Be smart! Use conditionals to get more insights. But don’t overload the user!
Tip 2: Plan your test beforehand, document it!
The pictures below show the Maze current test set-up interface.
Report
Maze report is generated automatically. It measures the following parameters:
- Total testers (for each task, even if someone drops the test before the end, it saves what the tester completed)
- Misclick rate (clicks outside of the path)
- Avg. duration
- Avg. success
- Avg. Bounce
- Usability score (based on bounce rate, misclick and avg. duration. Maze has a formula to calculate it)
They also prepare a Heatmap and clickMap both are useful to understand the user interaction with the prototype.
Maze limits (test your test before releasing it)
I tested the test before releasing the official test. Doing this was crucial to get most of the Maze, understanding it’s limits and potentials. I could validate some assumptions and common knowledge about unmoderated usability testing:
- Testers internet connection might interfere
- Testers will not have the same attention level as in an moderated usability test
- The device model might interfere with the test (low-end models might not run properly the test)
- The animations might not work properly
- The user might not distinguish between prototype and fully operational live version.
- The user might try to navigate randomly just for curiosity.
I also noticed that some iPhone users were expecting an error message. To understand this I contacted Maze support team (great support by the way!). They quickly replied and answered all my questions.
Maze currently do not have a feature to upload an image and set a target zone on it (click test). I really missed this feature because it is useful to test the interface concept. (I told to support about this, hopefully this feature might be implemented on the future)
Conclusion
Always test your test! This will improve the quality of the result, decrease bias, avoid mistakes, and save time!
Break the test into small logical steps like, sign-up, upload your medical test, choose where you want to do it, pick a date, choose the patient… Instead of book a medical test.
Plan your test in advance, use conditionals and don’t make a long test.
Maze is great to test about user perception and small interactions (avoid animations or large prototypes).
Keep in mind and understand the limits and capabilities of the test to get the most of it.
If you want to read more about moderated vs unmoderated usability tests this article might be useful to you.
Hope this article could help you about this amazing tool called Maze. If you want to talk further or want more details you can reach me out on my Linkedin.
