Knowledge kept with its time and its source

Sources disagree about a fact, and one source changes its answer over time. A register lists a company under one legal form, the company’s own site states another, and later the register corrects itself. A store that keeps only the current value cannot say who claimed what, or when. The second part of my research question is how knowledge about an object is kept with its time and its source. My approach is to keep every fact as one immutable assertion.

One row is one assertion

Every property value of every object is one row. The row holds the value, the account that wrote it, the source it cites, a license, a language, a precision and a trust rank. A row is never updated. A correction inserts a new row and hides the old one, so the store still shows what it said before and who said it. Removing a value needs a written reason.

The trade-off is volume, because the table grows with every claim and every correction. I accept that cost. An overwritten value cannot be recovered, and a hidden row can.

Two kinds of time

Valid time says when the fact was true. It is stored as partial dates, because a source often knows the year a contract began and nothing more exact. Entry time says when the store learned the fact. This is bitemporal modeling applied to single property values. The store answers what was true on a given day, and what it believed on that day.

The editor keeps one more date apart, the day the editor saw the source. A register entry read today may describe a change that took effect long ago.

The trust rank

Every account has a rank and every source has a rank. The rank of a row is the lower of the two when the row cites a source, and the account rank when it cites none.

The rank is stored evidence, and it does not decide which value shows. If the higher-ranked row keeps its slot, an old row that no editor can see blocks every later correction. The last editor save wins. The price is that a careless save can displace a well-sourced value. The old row is still there, so the save can be reviewed and reversed.

Rows are the truth, flat columns are a projection

Many systems keep flat copies of the same values on their entity tables, because a flat column is fast to read. It holds one value without its qualifier. A page can then show a product’s gross weight where the net weight belongs, and nothing in the column says so.

The rows are the source of truth, and flat tables, hierarchy tables and search documents are derived snapshots that can be rebuilt. A projection never writes back. A gap in the rows can be deliberate. An editor may have retired a row as wrong or as a duplicate, and refilling it from an old flat value would overrule a human judgment.

Immutable rows under versions

The canonical fields of a row cannot change. A correction is a replacement assertion, and each complete mutation of an object creates at most one version, stored as the set of assertions it adds or removes against its predecessor. Row identity, source, writer, rank and language count as provenance, so resubmitting an unchanged value keeps the existing row and creates no version.

Where the design is going

The assertion rows, both kinds of time and the trust rank are in production use on a client’s information platform. Versions are a design I am developing. Until then a selection flag on the row marks the value that shows.

One open question is how a reader picks among competing assertions when the last editor save and the trust rank point to different rows. Another is how assertions follow an object through a merge or a split, which is part of my work on object identity.

I use W3C PROV-O and the ISO standard on data provenance as references and do not call the store compliant with either.