Ledger & Audit Trail Design

The revenue ledger is the single source of truth that every reconciliation and settlement decision writes to, and this page covers how to design it as an append-only, tamper-evident record inside the Revenue Reconciliation & Settlement pipeline. Once a fare, a transfer credit, or an inter-agency payout is committed here, it must never be silently edited: an auditor six months later has to be able to replay the exact sequence of entries and arrive at the same balance, and a disputing operator has to be shown the untouched original rather than a reconstruction. This page treats the ledger not as a database table but as a cryptographically chained log — the substrate that makes reconciliation trustworthy rather than merely convenient.

The scope here is the write path and its integrity guarantees: how an entry is shaped, how it is hash-chained to its predecessor, how a deterministic idempotency key stops a redelivered tap from being booked twice, and how a verifier walks the chain to prove nothing was mutated after the fact. The mechanics of one specific chaining scheme are worked end-to-end in the guide on designing append-only fare ledgers with hash chaining, which this component builds on.

The reason to invest in a tamper-evident ledger rather than a plain audit table is that transit revenue is contested revenue. An operator in a shared-corridor agreement will dispute a proration; a card processor will dispute a chargeback; an FTA reviewer will question a fare figure that funds a formula grant. In every case the defensible answer is the same: here is the original entry, here is the cryptographic proof it has not changed since it was booked, and here is the compensating entry that adjusted it, in order. A conventional table with UPDATE privileges cannot make that claim, because no one can prove a privileged user did not quietly rewrite history before the audit. The append-only, hash-chained design closes that gap by construction.

Architecture: The Append-Only Write Path

An entry does not become part of the ledger the moment code constructs it. It is committed only after its idempotency key is checked against what is already recorded and its prev_hash is bound to the current chain head. The diagram below shows the write path from a priced decision through the chain to a periodic snapshot and the verifier that both an audit and a dispute rely on.

Each entry’s hash is a one-way function of both its own contents and the hash of the entry before it. Writing entry $n$ computes

$$h_n = \text{SHA256}\big(h_{n-1} \parallel \text{canonical}(e_n)\big)$$

where $\parallel$ is byte concatenation and $\text{canonical}(e_n)$ is a stable serialization of the entry body. Because $h_n$ depends transitively on every earlier entry, altering the amount on entry 4 of a million-entry ledger changes $h_4$ and therefore every hash after it — the tamper cannot be localized or hidden, which is the entire point.

Prerequisites & Environment

This component targets Python 3.11+ and depends only on the standard library for its integrity core — hashlib, decimal, datetime, and json — so the same code runs on a cloud reconciliation worker and inside a verification job an auditor executes independently. Persistence is deliberately abstracted behind an append interface: the reference implementation writes to an in-process list, but production backs it with an append-only table (a Postgres table with no UPDATE/DELETE grant) or an object-store log.

The entry_key is the seam between this ledger and the components that feed it. Taps arrive already authenticated and canonicalized after clearing the AFC System Security Boundaries, so the natural identity the key is derived from — media hash, validator, and instant — is trustworthy by the time it reaches the append path.

Core Implementation

The class below is an append-only ledger with per-entry SHA-256 chaining, deterministic idempotency, Decimal amounts, and a verify() that recomputes the entire chain. It never exposes a mutator: the only write operation is append, and the only correction primitive is compensate, which itself appends.

from __future__ import annotations

import hashlib
import json
import logging
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
from decimal import Decimal
from typing import Iterator, Optional

logger = logging.getLogger("transit.ledger")

GENESIS_HASH = "0" * 64  # prev_hash of the very first entry


class LedgerIntegrityError(Exception):
    """Raised when the hash chain fails verification."""


class DuplicateEntryError(Exception):
    """Raised when an entry_key has already been booked."""


@dataclass(frozen=True)
class LedgerEntry:
    entry_key: str            # deterministic idempotency key
    account: str              # operator, fund, or clearing account
    amount_minor: Decimal     # signed minor units; credits +, debits -
    currency: str             # ISO 4217, e.g. "USD"
    kind: str                 # 'fare' | 'transfer_credit' | 'payout' | 'compensation'
    occurred_utc: datetime    # when the underlying event happened
    booked_utc: datetime      # when this entry was appended
    prev_hash: str            # hash of the preceding entry
    entry_hash: str = ""      # SHA-256 over (prev_hash || canonical body)
    corrects: Optional[str] = None  # entry_key this compensates, if any
    meta: dict = field(default_factory=dict, repr=False)

    def canonical_body(self) -> bytes:
        """Stable, machine-independent serialization used for hashing."""
        body = {
            "entry_key": self.entry_key,
            "account": self.account,
            "amount_minor": str(self.amount_minor),  # str(Decimal) is exact
            "currency": self.currency,
            "kind": self.kind,
            "occurred_utc": self.occurred_utc.isoformat(),
            "corrects": self.corrects,
            "meta": self.meta,
        }
        return json.dumps(body, sort_keys=True, separators=(",", ":")).encode("utf-8")

    def compute_hash(self) -> str:
        h = hashlib.sha256()
        h.update(self.prev_hash.encode("ascii"))
        h.update(b"\x1f")  # unit separator: prevents prev_hash/body ambiguity
        h.update(self.canonical_body())
        return h.hexdigest()


class AppendOnlyLedger:
    """Tamper-evident, append-only revenue ledger with SHA-256 chaining."""

    def __init__(self) -> None:
        self._entries: list[LedgerEntry] = []
        self._keys: set[str] = set()
        self._head: str = GENESIS_HASH

    @staticmethod
    def derive_entry_key(source: str, natural_id: str) -> str:
        """Deterministic idempotency key. The same source event always maps
        to the same key, so redelivery books the entry exactly once."""
        return hashlib.sha256(f"{source}:{natural_id}".encode("utf-8")).hexdigest()

    def append(
        self,
        *,
        entry_key: str,
        account: str,
        amount_minor: Decimal,
        currency: str,
        kind: str,
        occurred_utc: datetime,
        corrects: Optional[str] = None,
        meta: Optional[dict] = None,
    ) -> LedgerEntry:
        if not isinstance(amount_minor, Decimal):
            raise TypeError("amount_minor must be Decimal, not float/int")
        if occurred_utc.tzinfo is None:
            raise ValueError("occurred_utc must be timezone-aware UTC")
        if entry_key in self._keys:
            # Idempotent: the same booking arriving twice is not an error to
            # the caller, but it must never extend the chain a second time.
            raise DuplicateEntryError(entry_key)

        draft = LedgerEntry(
            entry_key=entry_key,
            account=account,
            amount_minor=amount_minor,
            currency=currency,
            kind=kind,
            occurred_utc=occurred_utc.astimezone(timezone.utc),
            booked_utc=datetime.now(timezone.utc),
            prev_hash=self._head,
            corrects=corrects,
            meta=meta or {},
        )
        sealed = LedgerEntry(**{**asdict(draft), "entry_hash": draft.compute_hash()})
        self._entries.append(sealed)
        self._keys.add(entry_key)
        self._head = sealed.entry_hash
        logger.info("booked %s %s %s minor=%s", sealed.kind, sealed.account,
                    sealed.entry_key[:12], sealed.amount_minor)
        return sealed

    def compensate(self, target_key: str, reason: str) -> LedgerEntry:
        """Reverse a booked entry by appending its inverse. The original is
        left untouched; correction is additive, never destructive."""
        target = self._by_key(target_key)
        if target is None:
            raise KeyError(f"cannot compensate unknown entry {target_key}")
        rev_key = self.derive_entry_key("compensation", target_key)
        return self.append(
            entry_key=rev_key,
            account=target.account,
            amount_minor=-target.amount_minor,
            currency=target.currency,
            kind="compensation",
            occurred_utc=datetime.now(timezone.utc),
            corrects=target_key,
            meta={"reason": reason},
        )

    def verify(self) -> bool:
        """Walk the chain, recomputing each hash. Returns True if intact,
        raises LedgerIntegrityError at the first divergence."""
        prev = GENESIS_HASH
        for idx, entry in enumerate(self._entries):
            if entry.prev_hash != prev:
                raise LedgerIntegrityError(
                    f"broken link at index {idx}: prev_hash mismatch")
            if entry.compute_hash() != entry.entry_hash:
                raise LedgerIntegrityError(
                    f"tampered body at index {idx}: hash mismatch")
            prev = entry.entry_hash
        return True

    def balance(self, account: str) -> Decimal:
        return sum((e.amount_minor for e in self._entries if e.account == account),
                   Decimal("0"))

    def _by_key(self, key: str) -> Optional[LedgerEntry]:
        return next((e for e in self._entries if e.entry_key == key), None)

    def __iter__(self) -> Iterator[LedgerEntry]:
        return iter(self._entries)

Three properties carry the guarantees. The entry_hash binds each row to its predecessor through prev_hash, so verify() detects any post-hoc edit as a hash divergence. The entry_key set gives exactly-once booking even under stream redelivery, which matters because a double-booked fare is a real overstatement of revenue. And compensate proves that the only way to reverse money is to add a signed-inverse row — the audit trail records both the mistake and the fix, in order.

Two details in the LedgerEntry are worth dwelling on because they are the difference between a ledger that replays and one that only appears to. First, the entry carries both occurred_utc and booked_utc. These are almost never equal in practice — a tap captured offline on a bus and forwarded hours later occurred at capture time but is booked at ingestion time — and downstream reporting needs both: the economic period keys on when the fare happened, while the audit timeline keys on when the agency recorded it. Collapsing the two into a single timestamp is a subtle way to make a period report unreproducible. Second, the amount is a signed Decimal: credits are positive, debits and reversals negative. Signing the amount rather than storing a separate direction flag means balance() is a plain sum and a compensating entry is a literal negation, so there is no branch where a sign could be applied inconsistently between the booking path and the reporting path.

Schema Validation & Edge Cases

A ledger inherits authenticated events, but “authenticated” does not mean “well-ordered” or “singular.” The edges that corrupt a naive ledger are all concurrency, correction, and serialization edges.

• Concurrent appends. The chain head is a shared mutable pointer; two writers reading the same prev_hash and both appending would fork the chain. Serialize appends behind a single writer per ledger partition (an application-level lock, or a database SERIALIZABLE transaction with the head selected FOR UPDATE). Never let two processes extend the same head — a forked chain fails verify() and is unrecoverable without a rebuild.
• Back-dated corrections. A settlement dispute resolved in March that concerns a January fare is booked as a compensating entry dated now, with occurred_utc referencing the original event and corrects pointing at its key. The January entry is never touched. Period reports handle the timing split by reading occurred_utc versus booked_utc explicitly, which is exactly the restatement problem the Compliance Reporting Automation layer resolves.
• Idempotent redelivery. derive_entry_key is a pure function of the source event’s natural identity, so a tap replayed by a broker maps to the same key and append raises DuplicateEntryError rather than booking twice. Callers treat that exception as success, not failure.
• Canonical serialization drift. The hash is only reproducible if canonical_body is byte-stable across Python versions and platforms. Sorting keys, fixing JSON separators, and serializing Decimal via str() (which is exact) rather than float() are load-bearing, not stylistic.
• Amount typing. append rejects non-Decimal amounts at the door. A single float amount that slips in poisons every downstream sum and, worse, produces a hash that a re-derivation from clean data will not reproduce.

Integration Pattern

The ledger is the shared spine of the settlement pipeline, not a leaf service. Its correctness depends on one upstream contract and it feeds two downstream consumers.

Upstream, every priced decision that reaches append has already cleared authentication and canonicalization at the AFC System Security Boundaries, so the natural identity the entry_key is derived from is trustworthy — the ledger proves that a booking is singular and immutable, not that the underlying tap was genuine. Keeping those concerns separate is deliberate: the boundary decides authenticity, the ledger decides integrity.

Downstream, Daily Variance Reconciliation reads the ledger’s per-account balances as the authoritative internal figure it matches settlement files against — a variance is only meaningful because the ledger side of the comparison is tamper-evident. And Compliance Reporting Automation aggregates the same entries by period and category into regulatory filings, asserting its totals tie back to balance() before a report is signed. Both consumers depend on the same invariant: the numbers they read cannot have been edited after the fact.

Performance & Scale Considerations

At production volume — hundreds of millions of entries a year across a metro’s fare, transfer, and inter-agency accounts — the ledger’s cost model is dominated by the append serialization point and the verification walk.

• Partition the chain. A single global chain serializes all writes through one head. Partition by settlement account or service day so each partition is an independent chain with its own head, then verify partitions in parallel. Never partition mid-journey in a way that splits related entries across chains a dispute has to join.
• Snapshot and truncate the verify window. Full-chain verification is $O(n)$ in total entries and becomes slow at scale. Emit a signed snapshot of the head hash and running balances every N entries (or per closed service day); routine verification then walks only from the last trusted snapshot forward, while a full walk runs on a schedule and after any restore.
• Stream, never materialize. Both verify() and balance aggregation should iterate the persistence layer in bounded chunks rather than loading a year of entries into RAM. The reference list is illustrative; the production append target is a cursor over an append-only table.
• Hashing cost is real but linear. SHA-256 over a compact canonical body is fast, but it is per-entry and unavoidable on the write path. Keep the canonical body minimal — identity, amount, kind, and a small meta — and push bulky payloads to a side store referenced by key.

Operational Checklist

1. Revoke mutation at the storage layer. Grant the ledger role INSERT and SELECT only; no UPDATE or DELETE. Integrity that depends on application discipline alone is not integrity.
2. Single writer per partition. Enforce one append path per chain head, via an application lock or a SELECT ... FOR UPDATE on the head row, so concurrent appends cannot fork the chain.
3. Pin the canonical serialization. Freeze the canonical_body format and cover it with a golden-hash test so a library upgrade cannot silently change a hash.
4. Correct only by compensation. Wire every reversal path through compensate; alert on any code path that attempts to edit a booked entry.
5. Snapshot on a fixed cadence. Sign and archive the head hash plus running balances per closed service day, and store snapshots off the primary so a compromised primary cannot rewrite its own history.
6. Verify continuously. Run incremental verification from the last snapshot on every batch and a full-chain walk nightly; page on the first LedgerIntegrityError, freeze writes, and escalate rather than auto-repair.
7. Keep amounts Decimal end to end. Assert the type on ingestion and forbid float in the ledger module; a lint rule is cheaper than a restated quarter.
8. Retain entry keys with their settlement batch. Store each entry_key alongside the batch it settled so any booking is forensically traceable to its source event during a dispute.

Related deep-dive guides

• Designing Append-Only Fare Ledgers with Hash Chaining — building the chain step by step, with a verifier that pinpoints exactly which entry was tampered with.

Related

• Daily Variance Reconciliation — matches settlement files against the ledger balances this component guarantees.
• Compliance Reporting Automation — aggregates ledger entries into regulatory filings that tie back to ledger totals.
• Inter-Agency Settlement Clearing — the payout events that book compensating and clearing entries into this ledger.
• AFC System Security Boundaries — the authentication gate every entry clears before it reaches the append path.
• Transfer Window Logic — the source of transfer-credit decisions written to the ledger as entries.

Part of Revenue Reconciliation & Settlement.