Architecture

How the Medici Protocol is built — from Canton ledger to frontend.

System Diagram

The protocol is organised as eight layers. The bottom three (Canton, Ledger Service, Agent SDK) are shared infrastructure — every user, agent, and frontend routes through them. The middle three (Agents, Runtime, Intent Expression) are where autonomous execution lives. The top two (On-Chain Intents, Frontend Interfaces) face users and the ledger. Twelve specialised agents form the execution fleet.

Layer Descriptions

Layer 1 -- Canton Ledger (DAML)

The foundation. All contracts live on the Canton Network, a privacy-preserving blockchain from Digital Asset. DAML contracts define the protocol's rules: what a vault is, how P and N tokens work, how settlement happens. Once deployed, DAML contracts are immutable (upgradable via SCU -- Smart Contract Upgrade for minor changes, or migration for major ones).

Key properties: Sub-transaction privacy (only parties to a contract see its details), no public mempool (no front-running or MEV), ledger time for maturity checks, causality enforcement via DAML's contract model.

Privacy model: a contract is visible only to its signatories and observers (plus parties whose token readAs includes a stakeholder). Exactly four designed-public templates carry the dedicated public party (medici-price-feed) as publicReader observer — PublishedPrice (the consensus price), MaturityPriceCommitment (settlement prices at maturity), SystemSummary (aggregate system stats), and IntentAnnouncement (coarse intent buckets). All four contain aggregates or deliberately coarse data, never per-user positions; no other template may add the public party as a stakeholder, or that data would leak to every user.

Layer 2 — Ledger Service (Go)

A single Go binary that absorbs all 8 Canton JSON API quirks and exposes a clean REST + WebSocket API. This is the recommended interface for all application code. It normalises created-event payloads (created.payload ?? created.createArguments), handles token refresh (5-min lifespan with retry on 401/403), wraps commands into the proper Canton format with the caller's actAs claims, and manages the full party/user lifecycle including the self-service onboarding flow (POST /api/v1/onboard — party allocation, user creation, GrantUserRights with the correct IdP, Keycloak attribute write-back). The /auth/login endpoint supports both confidential (client_credentials) and public (password grant, no secret) Keycloak clients.

For non-custodial external parties, it also owns the Path C prepare/execute flow: the service prepares transactions, relays prepared hashes to the wallet for signing, and submits the signed transaction — the service never sees a private key.

Layer 3 -- Agent SDK (TypeScript)

Shared library providing:

• LedgerClient -- typed wrapper around the Ledger Service API
• StateStore -- SQLite-backed persistent state, survives restarts and replays missed events
• EventBus -- publish/subscribe for agent coordination
• Agent Base Class -- lifecycle (start/stop/tick), health checks, structured logging
• Safety -- per-agent constraint enforcement (circuit breaker, rate limits, coherence checks)

Layer 4 — Specialized Agents

Twelve independent, single-purpose agents form the execution fleet. Each is a separate k8s pod with its own identity, party, and least-privilege Canton rights (per Brief 28, design recorded but not yet implemented):

Layer 5 -- Agent Runtime

The runtime orchestrator manages the fleet: tick scheduling (every N seconds), singleton enforcement (one instance per agent type), health checks with auto-restart, process supervision, and structured logging. Agents register with the runtime, which handles the lifecycle.

Layer 6 -- Intent Expression (Off-Chain)

Users express what they want through structured intents. A StrategyIntent might say "Track $50k BTC/USDC with standard strategy, max 15 bps/roll slippage." The StrategyEngine validates constraints, applies parameter defaults, and configures agents. A constraintHash links the off-chain intent to on-chain verification.

Layer 7 -- On-Chain Intents (DAML)

Intents recorded on the ledger provide an immutable audit trail. Templates include IntentAnnouncement (public signal of user intent), VaultComplianceRegistry (per-vault compliance tracking), and AgentMessage (inter-agent coordination). The DAML layer enforces lifecycle: an intent is created, agents reference it, and settlement verifies against it.

Intent Execution Flow (Brief 29)

When a user creates an intent via POST /api/v1/intents, the Ledger Service simultaneously creates an on-chain SplitRequest proposal signed by the user's own token. The StrategyAgent later exercises OperatorExecute on the stored proposal cid as operator-svc — its own narrow per-agent identity, with no canActAs grants across users (mechanism A).

1. User calls POST /api/v1/intents with their RS256 token.
2. Ledger Service validates the intent, extracts the user's party from their token's actAs claim, and submits a CreateCommand for a SplitRequest with depositor = user and operator = operator-svc (observer). The user's token signs the create — the user is the depositor signatory.
3. Ledger Service stores the resulting contract ID as intent.runtime.proposalCid and returns the intent to the user.
4. StrategyAgent polls the intents DB, finds the intent with proposalCid, and exercises OperatorExecute on the SplitRequest as vaultAdmin (the operator's own party). No extraActAs, no cross-user canActAs grants.
5. Canton creates the CollateralVault, PToken, and NToken — signed by (depositor, oracle, admin) where oracle == admin == vaultAdmin (the shared-oracle model).

Delegation (Mechanism B — Phase 4)

For recurring actions (rolls, rebalances, stop-loss), the operator needs authority beyond the initial split. Mechanism (B) introduces a StrategyDelegation DAML template — a power-of-attorney contract where the user delegates scoped authority to the operator for a bounded time window. See strategy-delegation-design.md.

Layer 8 -- Frontend Interfaces

Three interface paths:

• Chat / NL Interface -- natural language intent expression ("Track $50k BTC/USDC")
• Strategy Dashboard -- drift tracking, roll history, agent status, M2M values
• Manual Trading -- existing Trade, Earn, Swap, Admin pages as escape hatch

Data Flow

Every read and write passes through the Ledger Service. The service is the single chokepoint where Canton's JSON quirks are absorbed, the caller's identity is injected, and the token lifecycle is managed. All five flow patterns share the same four-hop path, shown below, with the specific endpoint and result at each hop.

The READ paths fetch current state (a bounded snapshot of active contracts) or open a real-time WebSocket stream (seeded from the current offset, only new events are pushed). In both cases the service normalises Canton's payload quirk — created.payload with fallback to created.createArguments — so downstream code sees clean Contract objects.

The WRITE custodial path is the default: the service holds the signing key and can submit unilaterally. It wraps the logical command into CreateAndExerciseCommand format, injects the caller's actAs claims into the body, and forwards the token unchanged. submit-and-wait blocks until the transaction commits.

The WRITE non-custodial path (Path C) splits into prepare and execute for users who hold their own keys. Prepare: the service asks Canton to build the transaction and returns a preparedTransactionHash — the service never sees a private key. The wallet decodes the base64 hash, signs it with Ed25519, and encodes the signature. Execute: the signed transaction is relayed back; Canton verifies the signature against the user's public key before executing.

Auth Flow

The auth flow is a standard OAuth2 sequence with one critical Canton detail: audience-based auth. The token's actAs and readAs claims are used for addressing (which party the Ledger Service puts in the submit body), but Canton authorises by looking up the user by the token's sub claim and checking that user's rights — the claims themselves are ignored for authorisation. This is why the two-leg model (Canton rights at onboard + token claims at re-mint) is necessary.

Step 1–2: Human users authenticate via password grant against the public medici-app client (no secret). Agents use client_credentials. Keycloak returns an RS256 JWT with a sub claim and 5-minute expiry. At this point the token has no actAs claim — that appears only after onboarding, when the canton_party attribute is written to Keycloak and the dynamic mapper emits it on the next token request.

Step 3–4: The caller sends the token as a Bearer header. The Ledger Service extracts sub, actAs, and readAs from the JWT claims, then forwards the same token unchanged to Canton — it does not mint its own. Canton therefore sees the original sub and resolves the Canton user from it.

Step 5: Canton performs five checks in sequence: JWKS signature validation against the issuer's /certs endpoint, issuer match against the participant's IdP configs, token expiry, user existence (id == sub), and rights (canActAs / canReadAs grants on that user). Any failure returns a masked 401/403 — "A security-sensitive error has been received" — regardless of the real cause.

Step 6–7: On success, the result is returned to the caller. On failure, the Ledger Service invalidates its cached token, fetches a fresh one from Keycloak, and retries once. A second failure is returned as a real error to the caller — the retry is a hedge against expiry, not a retry loop.

Self-Service Onboarding

Human users authenticate and onboard through a fully automatic flow with no manual admin steps and no approval. The flow is two-leg: rights in Canton (what the user can do — authorisation) and claims in the token (which party commands address — addressing). Both legs are set up at onboard time.

Leg 1 — Canton User Rights (Authorisation)

At onboard, the Ledger Service calls Canton's GrantUserRights to grant:

• CanActAs — the user's own party
• CanReadAs — own party + medici-price-feed (the shared public price feed party)

Under audience-based auth, Canton resolves the user by the token's sub claim, looks up that user's rights, and enforces them — the token's actAs/readAs claims are ignored for authorisation. Rights live on the Canton user, not in the token. Onboard is idempotent — repeating it returns the same party with onboarded:false and re-grants the same rights (the grant is a no-op because the rights already exist).

Leg 2 — Token Claims (Addressing)

The Ledger Service builds command bodies from the caller's token claims (actAs/readAs). After onboard, the service writes the user's canton_party attribute to Keycloak. A dynamic OIDC mapper reads that attribute and emits it into the token's actAs and readAs claims on the next token request — so the caller must re-mint their token after a successful onboard before they can act as their party.

Agents (Brief 28 — Implemented)

Each agent (Layer 4) follows the same model: its own Canton user + party + least-privilege rights, provisioned at deploy time by auth-setup. Unlike human users, agents use client_credentials (not password grant) and are provisioned server-side rather than calling /api/v1/onboard. Per the recorded Brief 28 decision (option b), agents pass their party explicitly in the command body rather than relying on per-agent Keycloak mappers — removing a class of deploy-time config staleness and keeping a single source of truth for the party.

Multi-Party Propose/Operator-Execute Pattern

The protocol uses a two-phase pattern for operations that require multiple parties to authorize. This is Canton's equivalent of a multi-sig wallet (like Gnosis Safe).

Identity Reconciler (Refactor-Auth)

The identity reconciler (deploy/identity-reconciler in medici-dev and medici) continuously converges Keycloak and Canton state against a declarative identity manifest (k8s/identity/dev.yaml, k8s/identity/prod.yaml). It replaces the imperative auth-setup Job with a declarative control plane: observe, diff, converge, verify with synthetic probes, and report drift.

Each cycle (every ~60s) the reconciler:

1. Observes current Keycloak and Canton state (clients, users, parties, rights).
2. Diffs against the manifest. The manifest lists every principal (fleet agent, operator service, bootstrap user), its Keycloak client, its Canton party, and its actAs/readAs rights matrix -- as data, not bash loop bodies.
3. Converges by creating or updating Keycloak clients and Canton users/rights via their admin APIs, absorbing all Keycloak 26 and Canton API quirks in one Go library.
4. Verifies each principal with a synthetic probe -- a benign Canton read as that principal -- to prove the rights actually work end-to-end.
5. Reports drift via Prometheus metrics (identity_drift, identity_probe_failures, identity_converge_cycle), a status ConfigMap, and structured logs.

Drift monitoring: A Grafana dashboard (Identity Reconciler) surfaces converge cycle health, per-principal drift status, and probe failure rates. Three PrometheusRule alerts -- IdentityDrift, ProbeFail, ReconcileStalled -- fire when a rights gap persists, a probe consistently fails, or the reconciler itself stalls. Runbook: see operations-runbook.md. Full design: refactor-auth design.

Key Design Decisions

• No contract keys -- templates use ContractId for cross-contract references, avoiding key-based lookup complexity
• Pure verification -- invariant checks are closed-form math (verifyInvariant etc.), not runtime assertions
• Upgrade-transparent package IDs -- using the #package-name form instead of content hashes so references never go stale across DAR rebuilds
• Singleton price feed -- the oracle archives the prior PublishedPrice in the same transaction as each new publish, so at most one is active per ticker
• Audience-based auth (two-leg model) — authorisation (Canton rights) lives on the Canton user by sub, not in the token's actAs/readAs claims. Addressing (which party a command acts as) comes from the re-minted token's claims after onboarding, or from the party passed explicitly by the SDK (per Brief 28 option b).
• Bearer token gates service; signature authorizes ledger — the RS256 token proves who is calling the Ledger Service; the wallet's Ed25519 signature over preparedTransactionHash authorizes the on-ledger action
• Coherence-gated multi-attestation — divergent oracle observations trigger fallback, not bad settlements