This playbook is distilled from our work with hundreds of data teams, from 5-person startups to the Fortune 500.
Inside you’ll find:
- The 5-step data observability workflow and clear examples of how top data teams use this in practice
- A testing strategy to help you go from basic to strategic testing combining static tests with anomaly monitors
- Proven frameworks to level up your DataOps – from incident management and ownership best practices, to SLA and data quality frameworks
Get the free guide
By the numbers
Outlining the clear relationship between data reliability and business impact.
A framework to define data products to manage tiers of data assets and SLAs for maintenance.
We build a testing and monitoring framework that maximises errors caught and minimizes alerts.
Developing scalable ownership and efficient incident management processes to quickly resolve issues.
Establishing feedback loops and learning processes to continuously enhance data reliability practices.
Putting the workflow into practice faster with MCPs, prompts, skills, and playbooks
The Problem
Data broke. Now what?
Data is now business critical. A stale model just emailed the wrong offer to 40,000 customers. The regulator report is wrong. The forecast didn’t run.
As a result, data teams become overwhelmed by alerts, stakeholders are often the first to uncover issues, and confidence in the value of testing begins to erode.
This guide provides a framework to help solve it.
The Solution
Define
data use cases as products
Clarify
ownership and severity
Deploy
testing & monitoring strategy
Manage
and resolve incidents
Review
quality metrics
About the authors
Petr Janda · VP, Engineering, Coalesce
Petr has built large-scale data and software systems at scaleups such as Pleo and GW, and now leads the software engineering teams at Coalesce.
Mikkel Dengsøe · Director, Data Strategy, Coalesce
Mikkel led data teams at Google and Monzo, and has worked with hundreds of teams on optimizing their data reliability workflows. Today he leads data strategy at Coalesce.
Get the guide
70+ pages of frameworks, SLIs, defined metrics, and real-world case studies. Explore the interactive web version + downloadable PDF.
Most data teams aren’t equipped to take a strategic approach to building reliable data — tests are placed sporadically without much consideration of the use case, and ownership is unclear, creating “broken windows” of tests left failing too long. As a consequence, teams drown in alerts, stakeholders are the first to discover issues, and belief in the impact of testing erodes. Chapter 1 explains the pattern; the rest of the guide is the framework that reverses it.
A data product is a group of data assets structured based on their use case. If you define your data products well, everything follows: ownership becomes clear, it guides your testing and monitoring strategy, and you can manage SLAs in a way that’s consistent with data consumers’ expectations.
An SLA (Service Level Agreement) is a contract that defines the expected service level between a provider and consumer, including remedies if expectations aren’t met. A P1 SLA for a critical customer-facing dashboard might guarantee 99.9% uptime — with a 30-minute response time and two-hour resolution if data quality issues arise.
One that maximizes the errors caught while minimizing the alerts your team receives. The guide slices the data platform into four layers — pipelines, sources, transformations, and marts — to set a common standard and remove redundancies, like re-testing that customer_id is not_null in every model.
Three approaches are widely adopted across the industry but not optimal: redundant testing (re-checking the same fields at every layer), disconnected testing and monitoring tools, and defaulting to testing every model. Eliminating redundant tests is easier to reason about, generates less alert noise, and costs less compute — one team cut warehouse costs 10% by removing redundant uniqueness tests.
In five steps: integrate metadata, define data products where the stakes are highest, assign ownership to responsible teams — ideally using existing structures such as Google Groups — deploy data controls informed by owners’ domain knowledge, and activate ownership through alerting and escalation.
Set benchmarks tied to priority. The guide’s MTTD/MTTR targets: detect P1 issues within 1 hour and resolve ASAP (hours); P2 within 12 hours, resolved in a day; P3 and P4 within 24 hours, resolved in 3 and 7 days. Triage every alert with three questions first: scope, impact, severity.
Track coverage and quality score together — quality score alone gives a false sense of security if only a fraction of assets have controls. The guide defines 18 metrics across four groups, chosen with four principles: fit the business outcome, lead to action, segment by key dimensions, and trend over time.
AI doesn’t change the need for reliable data — the fundamentals still hold — but it changes how fast you go from theory to practice. Teams have turned this guide’s framework into Claude Skills that deploy a testing strategy to a new data product with a single command, and use MCPs to build bespoke weekly data quality reports.
Ad-hoc prompts work well for speeding up individual workflows, but they create inconsistency across a team. The guide maps three maturity levels — exploratory prompts, reproducible skills, and deterministic team-standard playbooks — with the guardrails to roll them out safely.
Yes. Fill in the form to explore the interactive web version, and receive the full 70+ page PDF in your inbox instantly.