moonraft

A production-grade Raft consensus library in MoonBit.

Home · Guide · API · Live demo · mooncakes

Raft keeps a cluster of nodes agreeing on the order of a command log even when nodes crash and the network drops, delays and reorders messages — the foundation of the replicated state machines behind distributed stores, schedulers and coordination services. This library implements the full protocol — election, replication, membership, persistence and reads — as a self-contained MoonBit package you drive from one of two fronts.

It ships two ways to drive one consensus core:

• a synchronous driver (run_election, replicate) that composes the RPC handlers into whole rounds — small and easy to read or embed; and
• a message-driven server (RaftNode) that speaks only in Messages through tick and step, so a real transport — or the bundled deterministic simulator — can drive it.

Install

moon add moonbitstack/moonraft

Quick example

Elect a leader, replicate a command, crash the leader and watch the cluster re-elect — then drive the same core by hand through RawNode:

// Drive a five-node cluster through the deterministic simulator.
let cluster = @raft.Cluster::new(["a", "b", "c", "d", "e"], seed=1)
let leader = cluster.run_until_leader(200)          // elect a leader
let _ = cluster.propose(b"set x = 1")               // replicate a command
let _ = cluster.run_until_committed(2, 200)         // wait for commit

cluster.crash(leader.unwrap())                       // inject a fault
let _ = cluster.run_until_leader(400)               // a new leader takes over
assert_true(cluster.one_leader_per_term())          // safety still holds
assert_true(cluster.committed_agrees())

That snippet is not a copy in a README: it is compiled and run by moon test from src/readme/README.mbt.md, together with the membership, RawNode and logging examples. The prose stays here; the code that has to keep working lives there. The guide is executable the same way — every step of it, and every line it prints, comes from src/readme/GUIDE.mbt.md.

Or run the bundled example — a five-node cluster elects, replicates, loses its leader and re-elects, printing the safety invariants at each step:

git clone https://github.com/moonbitstack/moonraft && cd moonraft
moon run src/cmd/example

Deterministic output (same seed → same transcript)

cluster of 5 nodes, seed 1
elected leader: c
committed 'set x = 1' on a majority
crashed the leader
new leader: a
one leader per term : true
committed prefixes agree : true
safety invariants hold : true

Source: src/cmd/example/main.mbt. The lower-level Node / RaftNode APIs are used directly in the tests — see raftnode_wbtest.mbt (message-driven) and cluster_wbtest.mbt (synchronous).

New to the library? src/examples/ is a guided tour of short, runnable programs — each one showcasing a different feature, from a first election up to a replicated key-value store and the lower-level RawNode deployment loop. They are meant to overlap on purpose: some show the same idea through the high-level Cluster simulator, others through the message-driven RawNode, so you can pick the layer you care about.

# The high-level, deterministic Cluster simulator — the easiest place to start
moon run src/examples/00-helloworld      # a minimal system: elect a leader
moon run src/examples/01-replicate       # propose commands, confirm every node commits
moon run src/examples/02-fault-tolerance # crash, partition, drop packets — safety holds
moon run src/examples/03-operations      # progress, leadership transfer, compaction
moon run src/examples/04-kvstore         # a complete replicated key-value store on a StateMachine

# Membership and the two roles a node can play
moon run src/examples/05-membership      # add and remove voters (ConfChange) at runtime
moon run src/examples/06-learner         # a non-voting learner, then promote it to a voter

# Fault handling and log management
moon run src/examples/07-checkquorum     # check-quorum: a leader steps down when it loses its majority
moon run src/examples/08-snapshot        # compact the log; a restarted node catches up from a snapshot

# The message-driven RawNode — the API you use to deploy Raft on a real transport
moon run src/examples/09-rawnode         # the RawNode tick → Ready → Advance loop, by hand
moon run src/examples/10-storage         # persist Raft state to MemoryStorage and recover it
moon run src/examples/11-readindex       # a linearizable (ReadOnlySafe) read, confirmed by quorum

The Cluster simulator (src/examples/00–08) gives you a seeded, deterministic cluster in a single process. RawNode (src/examples/09–11) is the same consensus core driven by your own ticks and messages — that is the layer you wire to a network and a real storage backend.

How it stays correct

Raft's value is its safety properties, and the library checks them as it runs rather than only on paper. The deterministic simulator (Cluster) re-checks, on every step, that there is at most one leader per term, that every committed prefix agrees across the cluster, and that the logs stay consistent — the same three invariants the in-browser demo re-checks on every frame. Each feature below is exercised by a runnable example, and every public type is documented in the API reference.

Verification

Every public behaviour is exercised by the test suite — 787 tests, 100% line coverage of the library — and the code is reconciled against a pinned upstream (etcd-io/raft at 26647d5). The upstream datadriven corpus is replayed by an in-repo interpreter: all 41 etcd-io/raft scripts (raft/testdata/, confchange/testdata/, quorum/testdata/) are archived verbatim, embedded at build time, and driven through the same RawNode / Changer / quorum code the library ships — in CI. The scripts' expected blocks are compared byte for byte: all 41 match, including every one of the 558 cases in the 28 raft scripts (an exact count, not a floor). the porting notes record how the replay works and what it took to converge.

the ledger audits the other direction: each of upstream's 258 Test* functions against the MoonBit test that carries it.

The library also runs as one voter of a cluster whose other voters are the reference Go implementation. integration/etcd-peer builds a MoonBit node and a harness that drives it and two etcd-io/raft nodes over one wire: the MoonBit node wins an election and replicates to them, follows an etcd leader, is killed and repaired from a new leader, and is brought up to date by a snapshot after the leader compacts its log. Every committed entry is compared, not just the index.

Benchmarks

src/cmd/benchmark ports upstream's BenchmarkStatus, BenchmarkRawNode and BenchmarkMajorityConfig_CommittedIndex case for case — including the member sweep (1, 3, 5, 100 voters), the two voter configurations of the RawNode loop, and the 1-to-11-voter majority sweep. MoonBit ships no go test -bench, so the executable times each loop with core/bench's monotonic clock and prints ns/op.

moon run --release src/cmd/benchmark

Representative figures (release build, one x86-64 machine; your numbers will differ):

Benchmark	ns/op
BenchmarkStatus/members=1/Status	~280
BenchmarkStatus/members=100/Status	~31
BenchmarkStatus/members=100/WithProgress	~5300
BenchmarkRawNode/single-voter	~33000
BenchmarkRawNode/two-voters	~27000
BenchmarkMajorityConfig_CommittedIndex/voters=3	~220
BenchmarkMajorityConfig_CommittedIndex/voters=11	~980

Live demo — real consensus in your browser

▶ https://moonbitstack.github.io/moonraft/demo.html

Five nodes, five Web Workers. Each worker instantiates its own copy of this consensus core compiled to WebAssembly, ticks on its own wall-clock timer, and talks to peers only by postMessage. The main thread is the network — drop packets, add delay, split the cluster, isolate or crash the leader — and it holds no Raft state of its own. Elections race, messages reorder, nothing about the schedule is deterministic; a panel re-checks the safety invariants (one leader per term, committed prefixes agree) on every frame.

Click Split 2 | 3 and you can watch two nodes lead different terms at once - and Election Safety still holds, because two leaders only contradict Raft if they share a term, and the stale one cannot reach a majority, so it cannot commit. Heal the partition and it steps down.

It is not a JavaScript re-implementation — messages cross the boundary as flat integers, node state is a JSON string read straight out of the wasm module's linear memory, and every transition happens inside the same MoonBit code the tests exercise (src/examples/browser). Five workers on one machine model concurrency, not a distributed deployment, and a restarted node catches up from the leader since the workers have no persistent storage.

Build and run the site locally

moon build --target wasm --release src/examples/browser
cp _build/wasm/release/build/moonbitstack/moonraft/examples/browser/browser.wasm docs/moonraft.wasm
python3 -m http.server 8099 --directory docs    # then open http://localhost:8099/

Workers fetch the wasm, so a file:// URL will not work.

Features

A complete Raft, not a sketch — click to collapse

• Leader election with the Follower / Candidate / Leader roles and the up-to-date-log voting restriction (§5.2, §5.4.1).
• Pre-vote so a partitioned node cannot inflate the cluster term, plus randomized election timeouts and heartbeats off a per-node deterministic PRNG.
• Log replication through AppendEntries: log-matching, conflicting-suffix truncation, and majority commit within the current term (§5.3, §5.4.2). Replies carry a conflict_index hint for one-jump backoff and a reject_index that keeps a reordered rejection from driving a spurious back-off; per-follower Progress (probe / replicate / snapshot) drives repair, including from heartbeat acks.
• Snapshots and log compaction (§7): compact, the InstallSnapshot RPC, and automatic snapshot fallback for a follower whose next entry was already compacted away.
• Membership changes (§4, §6): single-server add/remove and full joint consensus with ConfChangeV2 and auto-leave — C(old,new) needs a majority of both halves, and the leader appends the leave entry itself once it commits. A committed change reconfigures the running node: quorums resize, a leader that removed itself steps down, and an in-flight transfer to a removed target aborts.
• Learners (§4.2.1): non-voting members that receive the log, never campaign and never count toward a quorum, with promotion to voter and learners_next-staged demotion across a joint change.
• Flow control: a sliding-window Inflights limit, a byte cap per batch (MaxSizePerMsg), and a bound on the uncommitted tail (MaxUncommittedEntriesSize).
• raftLog split into stable storage and an unstable tail with in-progress bookkeeping and byte-level pagination, so a caller knows exactly what to persist and what to apply.
• RawNode with Ready / Advance: ask whether there is work, take a batch (entries to persist, HardState/SoftState if changed, messages, committed entries, read states), do it, acknowledge — no threads, no async.
• Persistence and crash recovery (§5.3): a HardState, an append-only write-ahead log (WalStore) with replay, and a MemoryStorage engine. Storage reads report Compacted, Unavailable and SnapOutOfDate as distinct errors, so a caller can tell "send a snapshot" from "wait".
• Linearizable reads in both modes — ReadOnlySafe (fresh quorum round-trip) and lease-based — plus check-quorum, which steps a leader down when it loses a majority and makes followers refuse disruptive votes.
• Leadership transfer (TimeoutNow, §3.10): the target is caught up first, proposals are blocked mid-transfer, and it aborts on timeout, step-down or removal.
• Pluggable StateMachine, Transport, LogStore, RaftStorage and Logger traits, with a replicated key-value store as the worked example and a moonlog adapter for the logger.
• Deterministic simulation harness (Cluster): a single-seed discrete-time network that drops, delays, reorders, partitions and crashes/restarts nodes, with built-in safety-invariant checks and a suite of scenario and chaos tests.

Architecture

The public surface is a pure facade: the root package re-exports the symbols so consumers write @raft.X regardless of where a type lives. The implementation is split into focused packages, each owning one concern:

Package	Responsibility
quorum/	Majority and joint-configuration vote counting
tracker/	Per-follower Progress (Probe / Replicate / Snapshot) and the Inflights window
raftpb/	On-the-wire types: Entry, Message, RPCs, HardState, Snapshot, ConfState, entry sizing
confchange/	Configurations, joint consensus, ConfChange and the config Changer
storage/	MemoryStorage, the write-ahead log, and the LogStore / RaftStorage traits
log/	RaftLog, the unstable tail, term lookup and bounded slices
core/	The consensus engine: RaftNode / Node step & dispatch, election, replication, snapshots, ReadIndex, leader lease, check-quorum, RawNode / Ready, Config, and the simulator
moonlog_bridge/	The one adapter that turns the core's Logger seam into a moonlog logger

The consensus core names no logging backend: it logs through a one-method Logger trait and defaults to silence, and moonlog_bridge is the only package that depends on moonlog. Everything else — the protocol, storage, membership, the simulator — has no third-party dependency.

Licence

Apache-2.0. See LICENSE and NOTICE.