Why lineage is an economic variable, not an ornamental field
A dataset buyer does not purchase rows alone. The buyer purchases a defensible ability to use those rows for an identified decision, repeatedly and within a defined rights boundary. If source identity, transformations and update history cannot be reconstructed, an otherwise valuable data product creates verification work at every renewal. This is the economic distinction between a file with metadata and a licensed, evidence-bearing data asset. The W3C PROV-O Recommendation, published on 30 April 2013, provides vocabulary for describing entities, activities and agents involved in provenance. DCAT 3, a W3C Recommendation dated 22 August 2024, describes datasets and data services for interoperable catalog discovery. Neither standard sets a commercial price. They make possible a traceable inventory whose production and validation costs can actually be measured.
Define a minimum provenance receipt
Our proposed receipt has seven fields: an immutable delivery version, source identifier, collection time, transformation pipeline revision, responsible agent or organisation, rights reference, and a verification result. Each revision should point to its immediate parent; a buyer must be able to distinguish a genuine correction from an unexplained replacement. The supplier should attach a manifest hash and keep raw sensitive records behind access controls rather than inserting personal information into an openly indexed metadata catalog. Provenance is not proof that every source assertion is true. It is evidence of what was processed, by whom and under which procedure. Record the remaining epistemic uncertainty separately, with sampling results and limitations.
A reproducible unit-cost model
Consider a hypothetical supplier delivering 200,000 validated records each month. The assumed monthly lineage workload is 12 engineering hours at $60 per hour, six compliance hours at $75, $180 in storage and compute, and $250 in external sampling. These illustrative inputs total $1,600, or $0.008 per delivered record. A 20% operating reserve adds $320, giving $1,920 monthly before margin. The figures are not industry benchmarks or evidence that any particular data market pays this price. The important feature is the disclosed denominator: accepted records, not records originally ingested. If only 160,000 records meet the agreed quality specification, the same $1,920 becomes $0.012 per accepted record. Failing to declare the acceptance denominator quietly shifts the risk to the buyer.
Separate baseline delivery from enhanced assurance
A seller can offer three clearly distinguished service tiers. Baseline supplies a dated schema, source and licence list, plus a version identifier. Audited adds sampled transformation logs, change histories and a discrepancy escalation process. Regulated-workflow adds retention rules, evidence export, a designated reviewer and pre-negotiated response intervals. Avoid marketing these tiers as automatic regulatory compliance. Industry-specific duties depend on the jurisdiction, content and intended use. Charge for the actual workload in each tier. A buyer who only needs aggregate trend monitoring should not subsidise a costly investigative evidence package requested by a higher-risk customer.
Price risk, not merely stored bytes
Suppose a buyer estimates $8,000 in annual avoided rework from reliable lineage, but an unverified delivery introduces a 15% probability of a $10,000 remediation project. Expected remediation exposure is $1,500 before considering extreme losses, so an additional $1,000 annual verification package could be rational if it demonstrably reduces both probability and investigation time. Those figures are scenario inputs, not measured population probabilities. Present a sensitivity table for 5%, 15% and 25% error rates. Show separately the supplier’s direct verification cost, the buyer’s avoided-loss estimate and the negotiated price. A price higher than the buyer’s incremental value needs a separate benefit; cost-plus alone is not a sufficient commercial argument.
An auditable procurement experiment
Before enterprise rollout, run a thirty-day paired evaluation on two comparable delivery streams. Require the same acceptance rules, sample rate, source documentation and incident classification. Measure median minutes to trace a disputed record to a source, the percentage of transformations with reproducible versions, the number of unresolved rights questions, and reviewer labour per one thousand accepted records. Publish failures and ambiguous cases alongside successes. A post-hoc comparison of only clean records will flatter the system. The evaluation should also record provenance coverage by source family, because one well-documented supplier can mask gaps elsewhere. For example, if 80% of records come from a mature feed and 20% arrive through manual spreadsheets, report the pass rate for each source rather than only their weighted average. Ask reviewers to replay at least ten randomly selected transformations using the documented pipeline revision. Preserve the selection seed so that an independent auditor can reproduce the sample without seeing unrelated personal records. When a provenance hop is missing, distinguish a harmless documentation defect from a missing licence or a materially altered value. Finally, publish the number of records excluded from the exercise and why. The price premium can only be defended if the pilot’s measurement scope and rejection rules are visible to both parties. In procurement, that clarity may be more valuable than an elaborate technical graph that no buyer can independently challenge. Buyers should reserve a right to inspect receipts without gaining access to unrelated customers’ sensitive information. The resulting price negotiation can then reference observed workload and a transparent uncertainty interval instead of treating ‘lineage’ as an unlimited premium feature.
Sources
- W3C PROV-O Recommendation, 30 April 2013
- W3C DCAT Version 3 Recommendation, 22 August 2024
- NIST AI Risk Management Framework, 26 January 2023
Editorial methodology: illustrative commercial assumptions are not reported market prices. Updated 19 November 2026.
