Master Patient Index: How to Clean Duplicate Patient Records Without Uploading PHI

TLDR: A master patient index (MPI) should hold one record per patient, but real files hold the same person twice because " Maria garcia" doesn't match "Maria Garcia", birthdates come in five formats, and phones are written differently. To clean duplicates safely: normalize names, dates, phones, and IDs first, deduplicate on exact matches, then review near-matches with a human in the loop. Because this is PHI, do it with a tool that never uploads the file.

An enterprise healthcare system can carry hundreds of thousands of duplicate patient records. Each one is a billing claim that might not pay, a record that might get filed under the wrong chart, and a consent or coordination case that fails silently. Cleaning those duplicates without touching PHI is a specific, repeatable job, and it is very doable.

This guide shows the safe workflow: what makes two patient rows look different, how to normalize them so duplicates become visible, how to deduplicate without deleting real people, and how to review near-matches correctly.

Why duplicate patient records happen

Almost every duplicate is a formatting difference, not a different patient:

"JOHN SMITH"        vs  "John Smith"
"  Maria Garcia "   vs  "Maria Garcia"
"02/15/1989"        vs  "February 15, 1989"
"(212) 555-0148"    vs  "212-555-0148"
"J45909"            vs  "J45.909"

Until values are normalized, every one of these compares as a different patient. Two records that are really the same person are filed under two identities, and exact-match deduplication cannot see it.

The root causes are the same as in any messy healthcare export: free-text entry, multiple source systems (EHR, billing, registration, lab), and re-exports that recreated a row.

Step 1: Normalize name fields

Names are patient identity. Normalize them before anything else:

Trim whitespace, including doubled internal spaces.
Fix casing, title case for names (Maria Garcia), never all-caps or all-lower.
Normalize suffixes and nicknames to a standard set: Jr., Sr., II, avoiding "Junior" vs "Jr." differences when reasonable.
Decide on middle names: for matching, compare full names normally, but flag cases where a middle initial is present in one row and absent in another.

Rule: standardize, don't delete. A different spelling is not a different patient, but it is also not proof of a duplicate.

Step 2: Normalize dates of birth

A date of birth mismatch quietly prevents the match that all your other fields would produce.

Convert every date column to ISO 8601 (YYYY-MM-DD): 1989-02-15.
Flag rows where the value is obviously wrong (birth year in the future, age over 120) and route them to review instead of guessing.

Two rows with the same name but a one-digit-off DOB need a human decision, not an automated merge.

Step 3: Normalize phone numbers and addresses

Phones and addresses are the strongest matchable fields after name and DOB:

Phones → E.164 (+12125550148): strip spaces, dashes, dots, parentheses, then add country code.
Addresses → strip punctuation and standardize abbreviations (St. vs Street, Ave vs Avenue), and collapse extra spaces. Keep the original in a raw column.

Address normalization is exactly the job of an address cleansing step: parse street, city, state, ZIP into separate fields where possible, so "New York" matches "New York, NY 10001" on the state and city parts.

Step 4: Clean and standardize identifiers

Partial identifiers are the danger zone:

MRN / patient ID: trim and standardize leading zeros. 001234 and 1234 must not look different.
SSN: strip spaces and dashes, or better, mask it entirely. You rarely need the full number for cleaning.
Diagnosis codes: a regex fix turns J45909 into J45.909 (ICD-10). Pattern errors like this break joins and matching.

Keep one column as the source of truth identifier after cleaning, and retain the original MRN so records can be traced back.

Step 5: Deduplicate on exact matches

Once every field is normalized, run an exact-row deduplication. This removes rows that are identical in every significant column. Real duplicate patients, created by re-exports and double-entry, mostly look like this: same name, DOB, phone, and address.

What you keep:

Two different patients who both happen to be named "John Smith", with a different DOB or phone.
The same patient with a genuinely changed phone (one record pre-move, one post-move), when the new data can't be reconciled automatically.

Exact-match dedup on normalized data removes the bulk of duplicates with zero risk.

Step 6: Review near-matches with a human

The remaining duplicates are near-matches: same name, slight DOB or phone difference. Never merge these automatically.

Group by a key you trust (normalized last name + first three digits of DOB + phone area code).
For each group of 2+ records that "almost" match, show a comparison side by side.
Only a human decides merge, keep, or flag. A wrong merge here costs real harm.

A good cleaning workflow produces exactly this hand-off: bulk exact dedup by machine, near-match review by a person, and a clear report of what each side changed.

Why this must not upload patient data

Everything above operates on protected health information. HIPAA requires a Business Associate Agreement for any vendor that receives PHI, and GDPR adds the same constraint for European data. In practice, that means healthcare analysts cannot paste a patient file into an online cleaner.

The safe pattern is local processing: the file never leaves the machine. Tools that run entirely in the browser (via WebAssembly) do the cleaning, normalization, and exact dedup locally, with no server ever receiving a row, so no BAA is needed for the cleaning step, and you can audit it with a network monitor.

This is the core reason Mungr processes everything on-device: healthcare teams get a real data-cleaning tool on data that cloud products can't touch.

Frequently asked questions

What is a master patient index (MPI)?

A database that maps every patient to one canonical record and links the identifiers each source uses to that record. It is the layer that prevents one patient from having three charts. It is only as good as the cleanliness of the patient data feeding it.

How do I find duplicate patient records?

Normalize names, dates, phones, and IDs first, then run exact-match deduplication to remove identical rows, and finally review near-match groups (same name, slightly different DOB or phone) manually. Matching on raw, unnormalized data hides duplicates.

Can I clean patient records without violating HIPAA?

Yes, if the cleaning happens locally and the file is never uploaded. Browser-based tools that process via WebAssembly keep PHI on-device, which avoids the cloud vendor BAA requirement for the cleaning step. Avoid any tool that transmits or logs the file.

What causes duplicate patient records?

Multiple source systems exporting the same patient, free-text entry variations (case, spacing, missing middle names), different date formats, and re-exports recreating rows. Identifier formats like 001234 vs 1234 also break matching.

Should I merge or flag near-match patient records?

Flag, don't merge. Automated merging of near-matches can combine two different patients. Let exact-match dedup run automatically and leave near-match decisions to a human with a side-by-side review.

Bottom line

Cleaning a master patient index is a discipline, not a one-off: normalize names, DOBs, phones, addresses, and IDs; deduplicate exact matches; route near-matches to a human; and do it all locally so PHI stays on the device. Save the steps as a recipe and your next audit-safe clean takes minutes.

Clean patient records locally — try Mungr free