Fault Injection

Failure taxonomy

Every bundle failure is classified into one of five canonical classes. Each class has a distinct recovery policy that the agent applies automatically.

Class	Cause	Agent recovery
`fee_too_low`	Tip below current auction floor	`getRawTx` → requeue → submit at escalated percentile
`expired_blockhash`	Slot exceeded `lastValidBlockHeight`	`getRawTx` → requeue → submit (sidecar fetches fresh blockhash)
`compute_exceeded`	`InstructionError: ComputationalBudgetExceeded`	`writeTrace` + hold — the instruction itself is broken
`bundle_failed`	On-chain execution error or post-send RPC error	Inspect `raw_error` → retry or hold based on error content
`slot_skip`	Leader produced no block (`SLOT_DEAD` event)	`getRawTx` → requeue → submit at same tip

Injecting a fault

The fault injection system lets you trigger any failure class on demand — without real network conditions. Register a one-shot fault before enqueueing a transaction:

import { Sinker } from 'sinker-sdk';

const sinker = new Sinker();

// Register a fee_too_low fault
await sinker.debug.injectFault('fee_too_low');

// Enqueue a real pre-signed transaction
const handle = await sinker.enqueue(rawTxBase64);

// The agent will:
//   1. Call submit → sidecar intercepts → returns fee_too_low lifecycle entry
//   2. Receive bundle_settled(failed) SSE
//   3. Run retry cycle with escalated tip
//   4. Submit the real bundle to Jito

The fault is consumed after a single use. The next submit after the faulted one goes to real Jito.

How it works

When a fault is registered via POST /debug/fault, the sidecar stores it in a next_fault field. The next call to POST /internal/submit:

Drains the tx queue normally (bytes are retained for retry)
Detects the pending fault
Synthesizes a failed lifecycle entry with the specified failure_class
Emits TxStatusChanged + BundleSettled SSE events
Returns HTTP 200 with a structured result — no Jito call is made

The agent receives the SSE events and executes the correct retry policy as if the failure happened on mainnet.

Running the full demo

The fault_demo.ts script in the agents/ directory automates the full sequence:

cd sinker/agents
npx tsx src/fault_demo.ts --failure fee_too_low

Expected output:

[FAULT ] Registering fee_too_low fault...
[QUEUE ] Enqueued tx: 346e5b0e-...
[AGENT ] Cycle 1 fired (tx_enqueued)
[AGENT ] → submit at p75=5000
[SIDECAR] FAULT INJECTED: bundle_id=fault-1748... failure=fee_too_low
[AGENT ] bundle_settled(failed) received — pendingRetry queued
[AGENT ] Cycle 1 finally drain → RETRY cycle
[AGENT ] [RETRY:fee_too_low] getRawTx → requeue → submit at p95=100000
[AGENT ] → bundle_id: 20a071a0d48f...
✅ PASS — retry bundle detected in 14s

Valid fault classes

string

required

One of the five canonical classes:

fee_too_low
expired_blockhash
bundle_failed
compute_exceeded
slot_skip

string

Optional custom error string stored in the lifecycle entry’s raw_error field. Defaults to "Injected fault: {failure_class} (demo)".

Overview

Concepts

Fault Injection

Failure taxonomy

Injecting a fault

How it works

Running the full demo

Valid fault classes

​Failure taxonomy

​Injecting a fault

​How it works

​Running the full demo

​Valid fault classes

Failure taxonomy

Injecting a fault

How it works

Running the full demo

Valid fault classes