Data Quality Framework: How to Build One, With Tests and Examples

TLDR: A data quality framework turns "this data feels wrong" into a set of repeatable tests: not null, unique, valid date, correct format, within range, no orphan keys. You write the checks once, run them on every file or export, and fix what fails instead of discovering it in a report. This guide shows how to build, run, and maintain that framework, with examples you can reuse.

The fastest way to reduce your time on bad data is to catch problems at the input instead of discovering them in the output. A data quality framework does that with a small, maintained set of rules that every file must pass before anyone analyzes it.

This guide covers what the 6 dimensions of data quality actually mean in practice, how to turn them into concrete tests, how to run those tests on messy CSVs and exports, and how to keep the framework from turning into a wall of ignored alerts.

The 6 dimensions as testable rules

The classic framework has six dimensions. Each one maps to a test you can run:

You do not need all six tests on every column. You need the right subset on the columns that matter, usually the ones that feed metrics, joins, and billing.

How to write a data quality test

A quality test is a tiny, repeatable check with three parts: a column, a rule, and an expected result. Here are concrete examples:

Not null (completeness):

column: customer_id
rule:   must not be null
fail:   any row where customer_id is empty

Unique key (uniqueness):

column: order_id
rule:   must be unique
fail:   any order_id appearing more than once

Valid date (validity):

column: order_date
rule:   must parse as YYYY-MM-DD
fail:   values that do not parse, or are in the future

Within range (validity):

column: amount
rule:   must be between 0 and 1,000,000
fail:   negatives and implausible values

Reference check (accuracy):

column: diagnosis_code
rule:   must be a valid ICD-10 pattern (e.g., J45.909)
fail:   codes that do not match the pattern

Write these as a table before you write any tooling. The table is the framework; the tooling is just how you run it.

A starter rule catalog

Here is a general-purpose catalog you can adapt in minutes:

Start with 5-10 rules on your most-used exports. Quality frameworks fail when they grow to hundreds of rules nobody reviews.

Run the tests at the right moment

The framework should run at three points:

1. On the raw file before cleaning, which tells you what needs fixing and how bad the input is.
2. After cleaning, which proves the file now passes and nothing got worse.
3. On every future export of the same structure, the "same file next month" scenario. This is where a saved recipe or repeatable check pays for itself.

Load the file, run the test set, get a count of failures per rule. A good cleaning pass then fixes the failures, and the same test set run again shows zero. That before/after comparison is your audit trail.

Real example: a weekly sales export

A sales team exports a weekly CSV with order_id, customer_id, order_date, amount, currency. The maintenance nightmare is not the volume; it's that the file breaks reports silently.

A 5-rule framework catches it:

1. order_id not null and unique.
2. customer_id not null and present in the customer reference.
3. order_date parses to YYYY-MM-DD and its column has one format.
4. amount numeric, >= 0, currency in the allowed set (USD, EUR).
5. order_date not in the future.

Week one, the framework flags 3,100 rows with mixed date formats and 58 null customer_ids. After normalization (dates to ISO, trim on IDs) and a fill rule, the same tests return zero. Next week, run the saved recipe and the test set again. The whole thing takes seconds, and the report stops breaking.

Where data cleaning fits in the framework

Data quality tests tell you that something is wrong. Data cleaning is what fixes it. The two belong in one loop:

1. Test → 2. Fix → 3. Re-test → 4. Save the fix as a recipe → 5. Replay + re-test next month.

Mungr bundles both: run transform steps (trim, change case, dates to ISO, regex replace, fill nulls, deduplicate, validate) with a real-time column-stat preview, see exact counts of what changed, and re-run the same workflow on the next export. Because processing happens in the browser, the framework works on files you could never upload: patient data, customer PII, financial records.

Frequently asked questions

What is a data quality framework?

A maintained set of rules, such as not null, unique, valid date, within range, and reference checks, that every file or export must pass before it is analyzed. It turns vague concerns into repeatable, auditable tests.

What are the 6 dimensions of data quality?

Accuracy, completeness, consistency, validity, timeliness, and uniqueness. Each dimension maps to a concrete test you can run on columns: reference checks, not-null tests, format and range tests, freshness checks, consistency rules, and duplicate-key checks.

How do I test data quality in SQL?

Count rows that violate a rule: not-null via WHERE col IS NULL, uniqueness via GROUP BY col HAVING COUNT(*) > 1, range via WHERE col < 0 OR col > 100, and reference checks via a LEFT JOIN counting nulls. Each query returns a failure count you can compare before and after cleaning.

What is a data quality test example?

A check that a column must not be null, must be unique, must parse as a valid date, or must fall within a defined range. For example, "order_date must parse as YYYY-MM-DD" or "amount must be between 0 and 1,000,000".

How do I build a data quality framework for CSV files?

Write a rule catalog (column + rule + expected result), load the file, run the checks, fix what fails with a repeatable cleaning recipe, then re-run the checks. Save both the tests and the cleaning steps so next month's export runs through the same gate.

Bottom line

A data quality framework is a small, living set of tests that run before, during, and after cleaning, so failures show up in the intake check instead of a board meeting. Start with the 6 dimensions, write 5-10 rules for your most important columns, and pair the tests with a saveable cleaning recipe.

Run quality checks and fixes locally — try Mungr free

Related: Data Quality Guide: What It Is, How to Measure It, and How to Fix It · Clean Dataset: How to Get a Truly Clean Dataset From a Messy File · Data Normalization: What It Means, When to Use It, and How to Do It