Measuring success
To track progress towards our goals we are focusing on a number of metrics grouped under two headings: Adoption and coverage and Quality and Usability. FT staff can see their current values in this report.
Adoption and Coverage
Team’s need to use Origami for benefits of our design system to be realised.
Primary metric
Our North Star Metric is adoption of Origami by products; We’re measuring this by counting the total number of component uses by projects via the npm package manager – the most popular way to install Origami components. E.g. two projects using 3 components each would add 6 to this metric.
We’re able to use our adoption metric for both legacy Origami and our latest components, so it will also help us to manage the transition.
Secondary metrics
These metrics help us break the headline figure down into its constituent parts, allowing us to determine if more projects are using Origami, if it’s greater usage by existing users, and where in the product development lifecycle Origami was first used.
- Number of Origami dependencies declared via npm
- Figma insertion rate: The number of times a designer has used a representation of Origami in Figma design tooling
2. Quality and Usability
If a design system is hard to use then even if it has great adoption, it will still have failed because one of the key aims of using a design system is to make it more efficient to deliver consistent designs.
Primary metric
We are measuring usability with a short System Usability Score survey, circulated to designers and engineers approximately quarterly. This asks users to rate a few generic statements, such as “I’m confident I’m using Origami correctly”, from “Strongly Disagree” to “Strongly Agree”. Mapping these answers from -2 to 2 and averaging over all respondents we get a reasonably objective measure of Origami’s usability.
Our baseline figure (from October 2023) is 0.6, which is exactly where we want to be; positive but with plenty of room for improvement
Secondary metrics
- Community engagement: We are measuring how many people attend our design system guild meetings. While not a goal in itself, we need an engaged community to help us make the right decisions, and an engaged community is also a sign that we’re solving problems that people want solved! So far the Origami project has seen a huge uptick in attendance, which is very encouraging.
- Figma detachment rate: This is an indicator of whether the components we provide via our new figma libraries are usable “as is”, without designers needing to make their own non-standard modifications
- Build time: This is one potential measure of how easy it is for developers to work with origami components
- Built file size: While the end user’s experience is in many ways subjective, smaller built files should translate into faster page loads