FPGA Streaming Pipeline

FPGA Market Data Processor

A five-stage SystemVerilog pipeline (parser, an order book holding eight price levels per side, a fixed-point signal engine, a stateful pre-trade risk gate) that turns a packed 38-bit market-data message into an accept-or-reject order decision in a fixed five clock cycles, one message per cycle.

SystemVerilog

cocotb

Icarus Verilog

Python

FPGA

Live: Simulation onlyCode

A waveform from the integration testbench: one message asserted on the input, the decision five clock cycles later, and the design's own latency counter reading five

Employer signal

What This Project Shows

This is the project where the specification is a timing contract rather than a feature list: every stage has a stated cycle count and a stated throughput, and the testbench fails if the hardware misses either. It shows I can design a streaming pipeline with real backpressure and a genuine read-modify-write hazard in it, and then build the independent Python model that proves the RTL correct field by field instead of eyeballing a waveform.

Problem

What Needed To Be Solved

A market-data hot path written in software has latency you cannot bound. A cache miss, a scheduler decision or a branch misprediction moves the tail, so the honest answer to "how long does a message take" is a distribution rather than a number. A pipelined hardware datapath has a fixed cycle count per message by construction, which is what makes wire-to-decision latency in cycles the metric worth designing against, and which means the design has to be able to state that number and then prove it, not assert it in a README.

Approach

How I Built The Solution

I built the hot path as four handshaked modules that add up to five register stages (parser 1, order book 2, signal engine 1, risk gate 1), with a valid/ready handshake between every module and a purely combinational order-generation step between the signal engine and the risk gate, so after fill one message enters and one decision leaves every cycle. Verification is a mirror rather than a set of spot checks: an independent Python model of each module, composed into a model of the whole pipeline, with every output beat compared as a complete ten-field record so a mismatch in any field fails the test. The stimulus is deliberately hostile (malformed frames, reserved opcodes, modifies and deletes against prices that are not in the book, crossing aggressions, zero and near-saturation quantities) and the generator models how a real feed behaves rather than only its shape: a crossed book is usually uncrossed by the very next message, so crossed episodes in the stimulus are short and transient instead of persisting for hundreds of beats.

Outcome

What It Demonstrates

In simulation the pipeline is held to its contract by the suites themselves: five cycles wire-to-decision asserted two independent ways, one message per cycle asserted as a cycle bound rather than logged, 300 back-to-back messages must complete within n + 5 + 2 cycles, and a randomly stalling consumer scoreboarded beat for beat with nothing lost. That is 47 self-checking tests across five suites, including a 2000-message pinned-seed end-to-end run, a 600-message run against that stalling consumer, a 600-operation weighted-random order-book soak and 500-beat boundary-heavy scoreboards, carried by roughly twice as much testbench code as RTL (1,639 lines of tests and a 154-line golden-model library against 856 lines of SystemVerilog) run on Icarus Verilog by GitHub Actions on every push. Nothing has been synthesised: the Quartus directory holds a placeholder and the repo's own measurement tables carry fmax, logic elements and critical path as explicit TBD cells, so the five cycles and the one message per cycle are simulation cycle counts at an unconstrained clock with no wall-clock figure behind them. The board half (a UART byte-framing front-end, a DE10-Lite top level, pin assignments, LED and HEX mapping) is unwritten, and is the next block of work. The risk gate's notional check is likewise an honest stand-in: a cumulative accepted-notional cap where the plan asked for a daily-loss limit, because a true profit-and-loss gate needs marked-to-market state the design does not carry.

Technical Signals

SystemVerilog RTL

Pipeline hazard resolution

Valid/ready backpressure

Fixed-point arithmetic

cocotb + Icarus verification

Golden-model scoreboarding

Implementation Notes

Committed the risk gate's accept decision and its position and notional state update on the same clock edge, with all four limits evaluated combinationally against the currently registered state, so order N+1 is judged against post-order-N state at a sustained one order per cycle with no forwarding path and no stall.
Reported every failing risk check at once in a four-bit reject_reasons vector instead of short-circuiting on the first failure (four parallel guards cost the same in hardware as one, which is not true of a priority chain in software).
Re-registered the signal-stage context (signal, book validity, midprice, spread and imbalance) in lockstep with the risk gate's input accept, so the output beat carries the decision together with the exact snapshot it was computed from. Without that register the decision and its context leave the pipeline one beat apart.
Stored the eight price levels per side in a register file rather than BRAM, because the price match and the best-of comparator tree need every entry visible in the same cycle and BRAM would serialise that into multiple reads. The match is a combinational compare across all levels with priority encoders for the hit and the free slot.
Scoped the book to a single symbol on purpose and wrote the reason into the RTL: order_book.sv has no symbol_id port at all, because symbol filtering is an upstream feed-handler concern, and the top level leaves the parser's symbol_id unconnected with a comment pointing at that decision.
Used two different backpressure policies on purpose: the single-register stages take a pass-through ready (s_ready = !m_valid || m_ready) that sustains one message per cycle with no bubbles at the cost of a combinational ready chain, while the order book, which carries a token across two internal stages, instead freezes the whole module whenever its output is held so nothing is accepted and nothing is lost.
Kept the midprice as bid + ask in half-tick units rather than shifting right by one: the shift truncates a genuine half tick, and a 16-bit plus 16-bit sum fits exactly in 17 bits, so refusing the shift removes the rounding error and the overflow question at the same time.
Made the Python feed generator the single owner of the 38-bit wire format (the testbench models import and re-export its field table, pack and unpack, and the message width is summed from that table rather than hard-coded) so the host tool and every testbench cannot drift apart on the packing.

Evidence From Source

One order per cycle, and stateful

The risk gate is a read-modify-write hazard in the same shape as a pipelined accumulator, and what makes the same-edge fix credible rather than merely claimed is the verification around it. Every limit is pinned at the boundary and one step past it, with "==" proven legal on all four: walking the position to exactly the limit still accepts, and so does the order that lands cumulative notional exactly on the cap. Rejects are asserted not to move position or consume notional budget, which is the natural bug if the state update were driven off the order request instead of the accept. A 500-beat boundary-heavy random stream is scoreboarded field by field against the independent model, and 100 back-to-back stateful orders must complete within 102 cycles: n + 2, the one-order-per-cycle claim written as a bound the test can fail.

Red testbench before the module existed

The scaffold commit ships parser.sv as a deliberate stub, its header says the stub only keeps the module legal so the testbench compiles and runs RED, alongside a five-test contract testbench written from the module's stated contract. The next commit implements the module and hardens that suite from five tests to nine. Four later RTL headers end with the command that has to go green, and the parser's own header ends with the same command under "Implement until the suite is green". The contract was written before the hardware, and the commit order shows it.

Latency proved twice, independently

The integration test measures the accept-to-output-valid gap itself and separately requires the design's own on-die latency counter to report the identical number, with both compared against 1 + 2 + 1 + 1 = 5. That counter arms only when a message enters an empty pipeline, tracked with an in-flight credit counter, because arming mid-stream would time the wrong message: the next output pulse belongs to an older message still in flight. A separate test proves the arming logic recovers: burst 50 messages, drain to empty, then send one message and it must still measure exactly five.

Tests named for the failure they guard against

The end-to-end test enforces coverage floors as hard assertions on a pinned-seed 2000-message run: it fails unless the stream actually produced both BUY and SELL signals, at least one accepted order, at least one reject, at least one one-sided or crossed episode, and specifically a price-band reject, a position reject and a notional reject, so a regression that quietly stopped exercising a limit fails the suite instead of passing while testing nothing. The directed tests are written the same way, each pinning one failure mode: a quantity merge must saturate at 4095 rather than wrap, a ninth distinct price on a full eight-level side must drop while a merge at an existing price still works and a delete frees the slot, a touching book counts as crossed, valid=0 frames never reach the book, an idle-gap test asserts an exact output beat count so a duplicate or a loss fails, reset is exercised with an output beat parked mid-stream, and the signal threshold is pinned on both sides so a strict inequality cannot silently become >=.

The bugs the harness hid

cocotb 2.0's runner does not raise on failing tests, so the suite runner exited 0 with tests red: CI would have reported green on red. Fixed by capturing the run results and propagating them into the exit code, then applying that to every suite. A second one mimicked an RTL bug outright: suites sharing one build directory reused each other's compiled binary, because the timestamp cache ignores both the toplevel and RTL parameter overrides, and the symptom was a wrong notional limit that looked exactly like a design fault. Fixed by giving every suite its own build directory and forcing a rebuild.