Matching

FerroPIX decides that two records are one patient in three tiers. An identifier in an authoritative domain links two records on its own. The probabilistic scorer on this page compares demographics and grades a pair. A data steward decides the pairs the scorer is unsure of. No IHE profile governs how a Manager cross-references: PIXm leaves it to the Manager, and PDQm asks only that a match carry a score and a grade. The rules below are FerroPIX’s own.

The scorer and the feed transaction that runs both tiers are built, and the PDQm Supplier answers ITI-119 $match with the scorer’s score and grade (The PDQm Supplier). The PIXm Manager hands each ITI-104 message to the feed transaction (The PIXm Manager), and the review queue is planned in #107.

When a feed arrives

Each ITI-104 message is one database transaction:

• Add or revise. The record of the identifier is created or gets a new version. FerroPIX then looks for every other record whose current version carries one of the same identifiers in an authoritative domain, such as the BSN. Those records are one patient with this one, so the record joins the patient that survives (a master identity first, then the one created first) and any other is merged into it, unless the two records disagree on the birth year or on a stated administrative gender. Such a hard conflict links nothing: the pair becomes a candidate for review, with the conflict in its evidence. A value in a local domain never links across domains, even when two domains share it. Two master identities are never merged by matching: they become a candidate with the master_identities conflict, and only their Source merges them.
• Scoring. The new version is scored against every record in another patient that shares a blocking key with it. A pair graded possible or better becomes an open candidate, or refreshes the open candidate of the pair. With auto_link = true, a certain pair merges the two patients instead.
• Resolve duplicate. When the Source says that one of its identifiers is replaced by another, the subsumed record stops answering for itself: a PIX query of it answers the survivor’s identifiers, its patient is merged into the survivor’s, and the cross-referencing runs again over the survivor. A survivor FerroPIX does not hold is refused.
• Remove. The identifier is answered as not found from then on, and the versions it had stay for the audit trail.

Every link, merge, candidate and removal is recorded as an action in the audit trail the operator console reads. Two feeds that join the same patients at the same moment both commit and end as one patient.

How a pair is scored

The scorer follows Fellegi and Sunter. For each field you configure two probabilities: m, that the field agrees when the two records are one patient, and u, that it agrees for two records picked at random. A field that agrees adds log2(m / u) bits to the total, a field that differs adds log2((1 - m) / (1 - u)) (a negative number), a partial agreement adds half the agreement weight, and a field missing on either side adds nothing.

The total maps to a score between 0 and 1:

score = 1 / (1 + 2^(10 - total))

A total of 10 bits scores 0.5: the evidence for the pair has to outweigh odds of one in 1,024 that two records sharing a blocking key are one patient. The mapping is the same in every deployment, so a score means the same thing everywhere.

The fields and their defaults

Names and addresses are folded before they are compared: accents removed, lowercased, punctuation read as a space. Where a field holds several values (several names, addresses or numbers), the best pair counts. Override a field under [matching.fields.<field>] in the configuration; each override must hold 0 < u < m < 1.

Grades

Change a threshold in [matching] and the grade of every later score moves with it; no code changes.

ITI-119 $match answers each person with the best score of its records against the Patient you send, as search.score, and its grade as the match-grade extension, whose codes are the grades above (PDQm 3.2.0 §2:3.119.4.2.2.4). A person graded certainly-not is no match and is left out of the answer.

Automatic linking

[matching] auto_link is false by default, so a certain pair is a candidate at the top of the review queue and a steward decides it. Turn it on only once your own data shows the scorer has earned it: load your records, run ferropix matching estimate (below), set the weights it prints, and read the scores of the candidates your stewards link and hold apart against link_at.

With auto_link = true, a certain pair links in the feed that scored it: the younger patient merges into the older, and each record that moves gets a link of kind probabilistic that keeps the score and the evidence the pair was graded on. The merge is recorded as an auto-link action of the Source’s client and can be reverted like any other merge. The dashboard’s auto-link rate counts these links alone: the records created in the last 24 hours that the scorer linked, over all the records created in that time.

Blocking

A pair is scored only when the two records share a blocking key, so a feed scores the few records that could be the same patient and never the whole table:

• birth_year_family_phonetic: the birth year and the Soundex code of the last word of the family name, so a prefix such as van der does not decide the block;
• postal_code_birth_date: the postal code and the full birth date;
• shared_identifier: an identifier both records carry.

Evidence

Every score keeps its evidence: one entry per field with the comparator, the two values as compared, masked to their last three characters (at most half the value), how the field agreed, the comparator’s reading (such as the Jaro-Winkler similarity) and the weight it added, plus the blocking key that produced the pair. The operator console draws its score breakdown from it. No log line and no error shows a compared value.

Estimating u

u depends on your population: a common family name agrees by chance more often than a rare one. Once the store holds your records, run

ferropix matching estimate

It counts how often each value of each field occurs over the current version of every record and prints, for each field that at least 100 records state, the [matching.fields.<field>] table that would set its u, beside the m your configuration holds. Paste the tables you agree with into your configuration. A field whose estimate reaches its m is named in a comment and left out, since [matching] refuses it. The output holds counts and probabilities, never a value.