u-Probability Calculations
calculations/outputs/u_probabilities.md and calculations/README.md in the cms-hte-patient-matching repository. That output file is generated by the tool on each run, so it is always in sync with the code; this page summarizes it.
What u means
u is the probability that two randomly chosen, distinct people agree on a given field.
Two estimators
For each field, the tool reports two estimates, where nv is the count of people with value v and N is the population:
| Estimator | Formula | Role |
|---|---|---|
u_simple | Σ (n_v / N)² | Plug-in estimator |
u_unbiased | Σ n_v (n_v − 1) / (N (N − 1)) | Finite-population pairs estimator. The headline number. It answers "probability that two distinct people agree", rather than "probability that two independent draws agree". |
Per-field methods
| Field | Method | Kind of estimate |
|---|---|---|
| Last name, first name (exact) | Census name frequencies. Two bounds are reported (see below). | Frequency-based, with a suppression caveat |
| Last name, first name (fuzzy) | Exact-match term plus the mass of all other listed names within Damerau-OSA edit distance 1. Names shorter than 5 characters fall back to exact-match probability only. | Frequency-based |
| Middle name | Reuses the first-name distribution. Census publishes no middle-name table. | Proxy |
| Year of birth | Single-year-of-age population estimates, converted to birth year. | Frequency-based |
| Date of birth (full) | Year-of-birth u divided by 365.25, assuming birthdates are uniform within a year. | Derived |
| ZIP (5-digit) | ACS 5-year ZCTA populations, used as a proxy for USPS ZIP codes. | Frequency-based, with a geography caveat |
| State | Census state population estimates (50 states + DC headline). | Frequency-based |
| City | Census incorporated places plus Census Designated Places. Two bounds are reported. | Frequency-based, with a coverage caveat |
| Street line (with and given ZIP) | Co-resident floor from ACS household-size data. | Floor |
| Phone | Co-resident landline-sharing floor: P(co-resident) × P(shared household has a landline). | Floor |
| SSN / ITIN last 4 | Closed form 1/9999. | Closed form (post-2011 SSN cohort only) |
| MBI | 1/N, with N the CMS-reported total Medicare enrollment. | Namespace-size floor |
Name fields: two bounds
Census name files list only names above an occurrence threshold. To keep the effect of that suppression visible, every name field reports two bounds:
- Bound (a), the headline: listed names renormalized to their own total.
- Bound (b): a lower bound that treats the unlisted remainder as all-singleton names.
The coverage (listed population divided by total) is reported with each field.
Fuzzy name matching
The tool uses a SymSpell-style deletion-neighborhood index to find all other listed names within one Damerau-OSA edit (insert, delete, substitute, or adjacent transposition). Every candidate is then verified exactly with rapidfuzz, not trusted to the approximate index. Names under 5 characters are excluded from fuzzy matching, matching the specification's rule that fuzzy matching is not applied to strings shorter than 5 characters (sections IV.C and V.E).
Variants beyond what the specification uses are also computed for pricing proposed rule changes, such as a 4-character minimum, an initial-only first name, and several date-of-birth widenings. They are labeled as such in the output and are not part of the specification's comparison.
Date of birth
Full date of birth assumes a uniform distribution within a birth year. The tool also computes date-level variants, including a ±1-day variant (fuzzy_pm1day). These are widened variants computed for pricing proposed rule changes against the threshold. They have no conservative value to compare against and are not part of the specification comparison. Table 2 allows a ±1-day tolerance for date of birth where it is marked with an asterisk, but the specification's Table 3 lists no fuzzy u-value for date of birth ("Not used"), and section IV.C says fuzzy matching is not applied to structured fields such as DOB.
Where a number comes from
Each result in the generated output names its source file. A result flagged proxy, floor, or closed form is not a direct frequency measurement. See Known Limitations before using any of them.