The CSV validation checklist
for recurring partner files.

Layer 1 · Envelope — is this the file we expected?

• Name matches the pattern, including the date part — and that date is the expected business date, not a resend of last week.
• Size is plausible: within a band of recent deliveries. A 2 KB file where 40 MB is normal is an incident, not an input.
• Arrival is on schedule; a late file triggers the late-file rule from your spec, not an improvised load.
• No duplicate processing: the same filename (or same content hash) hasn't been loaded already.

Layer 2 · Structure — is it shaped like the contract?

• Encoding is the agreed one; flag an unexpected BOM or a non-UTF-8 byte sequence immediately.
• Delimiter and quoting: the declared delimiter, balanced quotes, agreed escape rule. One unbalanced quote can swallow a thousand rows into a single field.
• Header row matches exactly — spelling, order, count. Treat renamed, reordered or extra columns as failures, not warnings.
• Uniform column count on every row. Ragged rows are the signature of embedded delimiters or truncation.
• Line endings consistent; a mid-file switch usually means two systems concatenated output.

Layer 3 · Content — are the values usable?

• Types parse: dates in the one agreed format, amounts as plain decimals (no symbols, no thousands separators).
• Required fields present on every row; empty-string vs NULL handled the way the spec says.
• Enums within the code list; an unknown status code is a conversation with the supplier, not a guess.
• Keys unique per the spec's duplicate policy.
• Control totals reconcile: row count and sum against the trailer or sidecar. This is the only reliable defense against silent truncation.

Order matters: structure before content. When a column has shifted, every content check downstream of it is noise.

The drift taxonomy — what actually changes

Recurring files rarely fail randomly; they drift in predictable ways. Name them and you can watch for them: header renames and reorders, added columns (the "harmless" one that shifts positional loaders), delimiter or quoting changes after a supplier-side export rewrite, encoding switches, date-format changes, new enum codes, and truncation from an interrupted transfer. Every one of these is cheap to detect at the structural layer and expensive to discover in a dashboard three days later.

What to automate first

If you automate nothing else, automate layer 2. Structural checks are deterministic, run in milliseconds, and catch the majority of real incidents. Layer 1 is a handful of scripting around your delivery channel; layer 3 grows with your spec. The cadence that works: every file, every time for layers 1–2, and layer 3 at minimum on every load plus a weekly review of rejects with the supplier.