A Mental Model to Assess Existing SLIs and Evaluate New Ones. We discuss 2 dimension: fidelity shows how closely your SLI is measuring the user experience, granularity shows how many parameter are measured by your SLI
They are different things that try to solve the same problem.
Integration tests usually run before the code goes to prod. SLI measures reliability after the code goes to prod. The same tests could be used. I've seen, for example, setups where the same Apica tests were run in development and production.
Apart from that, tests usually verify known flows and are synthetic. According to the SLI compass, organic tests have higher fidelity (e.g. analyzing RUM traces to deduce how many user flows break or detect anomalies in the user journey funnel).
This framing makes the real problem visible. Most SLIs fail because they are cheap stand-ins for the user experience. The fidelity and granularity compass makes the tradeoff explicit and shows how signals can stay coherent while drifting away from what users actually feel.
Hi, Alex. It seems like a high fidelity SLI is essentially an QA integration test. Is that accurate to say?
They are different things that try to solve the same problem.
Integration tests usually run before the code goes to prod. SLI measures reliability after the code goes to prod. The same tests could be used. I've seen, for example, setups where the same Apica tests were run in development and production.
Apart from that, tests usually verify known flows and are synthetic. According to the SLI compass, organic tests have higher fidelity (e.g. analyzing RUM traces to deduce how many user flows break or detect anomalies in the user journey funnel).
Thank you
This framing makes the real problem visible. Most SLIs fail because they are cheap stand-ins for the user experience. The fidelity and granularity compass makes the tradeoff explicit and shows how signals can stay coherent while drifting away from what users actually feel.