Parse, validate and index the evidence

A CSV reader supplies fields, not trusted records

CSV parsing separates fields and handles quoted delimiters. It does not decide whether an asset identifier is valid, whether a key is unique or whether the collector returned every asset. Our contract requires the header asset_id,owner_id,site in that order. This deliberately strict first version makes schema drift explicit. Supporting reordered columns is a reasonable extension, but it needs its own uniqueness and missing-column checks.

Read the source bytes once and calculate the hash of those bytes. Decode them with the selected encoding and parse that snapshot. If you read once for parsing and read again for hashing, a source file could change between reads and the reported hash could describe different evidence. Reading a small fixture into memory makes the relationship clear; a large export would need an appropriate streaming or immutable-snapshot design.

DictReader maps fields to header names. Extra fields and missing fields can be represented in ways a careless loop ignores. Our row validator rejects both rather than accepting a damaged record. The parser's strict mode helps identify some CSV quoting errors, but it is not a universal assurance that any accepted file has the intended schema or business meaning.

Reject duplicate normalized identities

A common shortcut builds a dictionary with {row['asset_id']: row for row in rows}. If the same identity appears twice, the later row can overwrite the earlier one. The result looks unique precisely because the evidence of duplication was discarded.

This program checks for an existing key before inserting a row. A-001 and a-001 collide after normalization. Even identical duplicate rows are rejected because the one-record-per-asset contract has been violated. The program does not guess which row is current: neither source includes a per-asset modification time or an authoritative revision to justify that choice.

In some systems duplicate observations are legitimate—for example, one row per asset per timestamp. In that case the row grain is different, and an asset-only key is insufficient. Change the data model instead of pretending a multi-observation export is a unique current inventory.

Use sets for population questions and dictionaries for field questions

A set of registered identities and a set of observed identities let you ask three separate questions: intersection for shared identities, register-minus-observed for register-only identities, and observed-minus-register for observed-only identities. Set operations concern membership. Dictionaries let you retrieve the owner and site fields for a shared identity.

The example below is a small independent practice step. Predict its three outputs before executing it:

registered = {'A-001', 'A-002', 'A-003', 'A-004'}
observed = {'A-001', 'A-002', 'A-003', 'A-005'}
print(sorted(registered & observed))
print(sorted(registered - observed))
print(sorted(observed - registered))

Expected outputs are the shared list A-001, A-002, A-003; then A-004; then A-005. Sorting makes the display deterministic because membership sets are not a report ordering contract. Stable ordering helps a reviewer distinguish an actual changed finding from an arbitrary row reorder.

Swapping the operands changes the directional difference. Add a test that catches this mistake. A test containing identical source populations cannot distinguish register-only from observed-only logic, even if it exercises the comparison function.

Completeness is an evidence requirement

After parsing and rejecting duplicates, compare the number of accepted records with the manifest's expected count. If it differs, stop the report. If complete is false, stop the report even when the file is readable. Do not publish missing-device findings based on an export its own collector declares incomplete.

A matching count and a true completion flag are still declarations, not cryptographic proof of the real fleet. The upstream collector can be wrong. The program records provenance so an analyst can inspect the source and collection process. It does not fabricate certainty by labeling the summary “complete inventory truth.”

A declared complete zero-record inventory is allowed by this data contract. That can be legitimate for an empty practice source. In an operational system with thousands of expected assets, an unexpected empty export should trigger an additional population-change review. A machine cannot infer that expectation from an isolated file. Your final extension will design that review gate without making the first report destructive.

Practice: diagnose before comparing

Create four copies of the observed source: one with a duplicate A-001, one with an extra fourth field, one without an owner value, and one with a manifest count of 5. Predict a rejected batch in all four cases and no newly created result directory. Explain the distinct reason for each rejection.

For the missing owner case, an empty field and a physically missing field both violate this contract, but they are different parser observations. Your error note should identify the field contract and offending record. Avoid copying every sensitive row into an error log when a record number and safe identity are enough for investigation.

Evidence laboratory — attempt before reading the solution

A synthetic inventory feed contains (asset-1,Desk), (asset-2,Laptop), (asset-1,Desk), (asset-2,Tablet). The contract says one current type per asset; identical duplicate records may be collapsed, but conflicting types must stop publication. The old published report is valid and must remain intact if the new batch fails. Design collection, validation, conflict detection and publication stages. Predict whether this batch can be published.

Official reference for further study