Proofload

What it is

Proofload is a load testing library for Kotlin and the JVM. You describe a user journey as a value (a scenario), say how many users arrive and when (a load profile), run it from an ordinary test, and get back a typed result you can assert on, plus an HTML report.

A load test is only useful if two things are true: the generator delivered the load you intended, and the latency it reports is the system's rather than the generator's. Proofload measures both, and tells you when the first was not true.

Use it when you want to know how a service behaves at a stated rate: whether the p99 of a step stays under a limit, what share of requests both succeeded and came back in time, or the highest rate the service sustains while holding its goals. Scenarios can mix HTTP, server-sent events, WebSockets, gRPC, Kafka, JDBC, Pelican clients and your own code, and run under JUnit, Kotest, zio-test or a plain main.

The coordinated-omission problem

Ask a load generator for 200 requests a second and it has to start a request every 5 ms. When it cannot (its threads are busy, the machine is loaded, or it waits for each response before sending the next request) the requests queue inside the generator.

A generator that times a request from the moment it is finally sent hides that queue. A request due at 10.000 s that leaves at 10.300 s and gets its answer at 10.310 s is reported as taking 10 ms. A user who arrived at 10.000 s waited 310 ms.

Worse, a generator that waits for responses sends less while the target is slow, so the slow period is sampled least. The tail looks healthy, the test passes, and the target never received the load you asked for. This is called coordinated omission: the measurement coordinates with the system under test and omits exactly the samples that matter. Nothing in the numbers says it happened.

Proofload deals with it in three ways:

• Departures are scheduled in advance. In an open workload each user's start time is computed from the load profile and the user's index alone. A slow response does not delay the next user, because each user runs on its own virtual thread.
• Every step is timed on two clocks. Service time is what the step took once it started. Response time adds how late the user's departure was, so the wait a user would have felt is kept. Goals read response time by default.
• Every run checks its own schedule. Departure lateness is recorded for every request and reduced to two checks, fellBehind() and lostGround(). The report, the JSON output and the CLI exit code all show the verdict.

So when a run fails, you can tell whether the target missed its objective or the generator could not sustain the workload.

When not to use it

• As a latency gate on a shared CI runner. The project's own advice is to fail a job on failed requests and errors, and to report latency rather than gate on it. Shared runners measure the same commit differently between runs.
• As a security boundary. proofload.toml can limit hosts, rate and duration, but it is a fence against mistakes, not a sandbox. Use a network policy or firewall for a guarantee.
• For protocols it does not cover. There is no queue client beyond Kafka. Anything else works through your own step body or emit, where you write the client.
• When a closed population is the only model you want. Proofload supports fixed looping populations, but a closed run sends less when the target slows down, which is coordinated omission by construction. The open model is the default for that reason.
• For more load than one JVM can send, unless you split the run across injectors. See what it costs.

Install

Proofload is on Maven Central under the group io.github.matthewjones372. All modules share one version. 0.1.0-rc4 is the current release; 0.1.0 is not out yet, so the API may still change between release candidates.

Requirements:

• JDK 21 or later. Users run on virtual threads. The build is tested on JDK 21, 23 and 25.
• Kotlin for the examples on this page. The library is compiled with Kotlin 2.4.20. Java and Scala 3 have their own facades (proofload-java, proofload-scala).
• A target to send load to. A JDK HttpServer started in the same test is enough to try it (see the recipes).

Most projects need three modules: a protocol, a test framework and a report.

dependencies {
    testImplementation("io.github.matthewjones372:proofload-http:0.1.0-rc4")
    testImplementation("io.github.matthewjones372:proofload-junit5:0.1.0-rc4")
    testImplementation("io.github.matthewjones372:proofload-report-html:0.1.0-rc4")
}

tasks.test {
    useJUnitPlatform()
}

proofload-core arrives through any of them, and proofload-engine through the test-framework modules. Core depends on the Kotlin standard library and nothing else, and each module has a test that checks its own runtime classpath.

Modules

Your first load test

This is the complete getting-started test from the repository. It sends 3,000 users over a minute, one every 20 ms. Each browses, places an order, captures the order id from a header and fetches the order.

import io.github.matthewjones372.proofload.at
import io.github.matthewjones372.proofload.engine.Proofload
import io.github.matthewjones372.proofload.expecting
import io.github.matthewjones372.proofload.http.http
import io.github.matthewjones372.proofload.junit5.LoadTest
import io.github.matthewjones372.proofload.keptSchedule
import io.github.matthewjones372.proofload.lostGround
import io.github.matthewjones372.proofload.p99
import io.github.matthewjones372.proofload.perSecond
import io.github.matthewjones372.proofload.report.writeHtmlReport
import io.github.matthewjones372.proofload.scenario
import io.github.matthewjones372.proofload.sessionKey
import io.github.matthewjones372.proofload.step
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Assertions.assertFalse
import org.junit.jupiter.api.Assertions.assertTrue
import java.nio.file.Path
import kotlin.time.Duration.Companion.milliseconds
import kotlin.time.Duration.Companion.minutes

private val orderId = sessionKey<String>("orderId")
private val browse = step("browse")
private val placeOrder = step("place order")
private val confirm = step("confirm")

private val api = http.baseUrl("https://orders.internal")

private val checkout = scenario("checkout") {
    exec(browse, api.get("/products"))
    exec(
        placeOrder,
        api.post("/orders")
            .body("""{"cart":"1 anvil"}""")
            .expecting(201)
            .capture(orderId) { it.header("x-order-id") },
    )
    exec(confirm, api.get("/orders/{orderId}"))
}

class CheckoutLoadTest {

    @LoadTest
    fun `checkout holds up at fifty a second`(proofload: Proofload) {
        val result = proofload.run(
            checkout.at(50.perSecond, over = 1.minutes)
                .expecting(p99(placeOrder) under 200.milliseconds, keptSchedule),
        )

        result.writeHtmlReport(Path.of("build/reports/proofload/checkout.html"))

        assertFalse(result.lostGround(), "the generator did not deliver the planned load")
        assertTrue(result.metEveryGoal, result.verdicts.joinToString("\n"))
        assertEquals(0L, result.failed)
    }
}

What each part does:

• step(...) and sessionKey<String>(...) declare names once. The same values build the scenario and read the result, so renaming a step is a compile error rather than an assertion about a step that no longer runs.
• scenario { ... } is a value. Nothing is sent when it is built.
• expecting(201), capture(...) and {orderId} belong to the requests. A request with the wrong status fails with HttpStatus(code) as its reason, and that user's journey stops there.
• at(50.perSecond, over = 1.minutes) is an open workload: each user departs on time whether or not earlier users have finished.
• expecting(...) on the simulation states the goals. keptSchedule is the goal that the generator itself kept up. It is not added for you.
• lostGround() checks that departures were not late by more than the planned gap between them, which keptSchedule does not check.

Running it

./gradlew test runs it like any other test. While it runs it prints its plan and a progress line every few seconds:

proofload: 00:05  departed 250  in flight 3  behind 88.033us  00:55 left

behind is how late the most recent departures were. If it keeps growing, the generator is not offering the rate you asked for. Past the window the line says draining: the target is finishing the users it was given.

A run takes the whole machine while it lasts. Two @LoadTest methods that start together run one after the other, and a second JVM on the same host waits for the first, because two runs sharing a host measure each other.

Reading the report

Open build/reports/proofload/checkout.html. It is one self-contained file. Read it in this order:

1. Did the generator keep up?

   A "behind schedule" banner, or a missed "the generator keeps its schedule" goal, means response times include time spent queued inside the generator. Fix that first: lower the rate, give the generator more machine, or split the run across injectors.

2. Did requests fail?

   Failures are listed by reason. A step whose requests failed fast can have a deceptively low p99, because a step's percentiles include failed requests.

3. Did the goals hold?

   Each goal shows what was measured and, for a miss, by how much.

4. Is the run long enough?

   p999 is reported only with at least 1,000 samples. The steady-state section says whether the run settled.

The same reading as a decision table:

How Proofload measures

Read this before you put a Proofload number in front of someone who will act on it.

Service time and response time

                scheduled          actually
                departure          started          answer
                    |------ late ------|--- service ---|
                    |------------ response ------------|

Service time includes everything the step body does: building and sending the request, waiting, reading the response, expecting, checking and capture, followed redirects, and any code you wrote in an exec(step) { ... } block. A pause is not a step and is in neither clock.

Departure lateness is measured once per user, when its virtual thread starts, and the same value is added to every step that user records. For a first step, response time is exactly "from the scheduled departure to the answer". For later steps it corrects for the user starting late, but does not give each later step a schedule of its own.

Goals read response time unless you ask for the other clock with of = Clock.ServiceTime. A goal written against service time can be met by a generator that never sent the load it was asked to send.

Some modules report their own figure: an HTTP step with retrying(...) records the last attempt; JDBC records the pool wait apart as waitedForPool; streams (WebSocket, SSE, gRPC streams) record one sample per message.

The schedule checks

result.behind is a histogram of departure lateness, one sample per recorded request. Two checks read it:

They can disagree. At 150 requests a second the planned interval is 6.67 ms, so a behind.p99 of 40 ms is lost ground. Against a target answering in microseconds, fellBehind() is true at almost any rate, because almost any lateness is large compared with the response. That is a statement about relative size, not proof the generator struggled.

result.fellBehind()          // generator lateness is a material part of the tail
result.lostGround()          // departures were late by more than the planned gap
result.behind.p99            // how late the late departures were
result.refused               // arrivals refused at an inFlightAtMost cap, never sent
result.generatorSaturated()  // refused, ran out of room, or lost ground

result.offered?.asked        // what the plan asked for, e.g. 2500/s
result.offered?.left         // what actually left, e.g. 1923/s
result.heldScheduleFor       // how long before the first second that lost ground

keptSchedule is met when fellBehind() is false. It does not consult lostGround(). Nothing fails a run for falling behind unless you ask it to:

When the generator fell behind, the run is not wasted. Service times are still measurements of the target, at the load that actually reached it (offered.left). Proofload does not lower the rate for you mid-run.

A cap on users in flight

Without a cap, every arrival that falls due starts a virtual thread, whatever is already running. When the target falls behind, the backlog grows on the generator's side. With a cap, an arrival that falls due while that many users are in flight is refused at once (never queued) and counted in result.refused, apart from the target's failures.

checkout.at(50_000.perSecond, over = 2.minutes)
    .inFlightAtMost(20_000)

A cap a little above rate times the slowest response you expect keeps a saturated run cheap enough to report what happened. A closed workload ignores it.

Percentiles and failures

Latencies are counted into histograms with HdrHistogram's bucket layout. A percentile is the upper bound of the bucket holding the sample at that rank. Nothing is interpolated, so a reported percentile is never below the true value and at most 0.78% above it.

• p50, p95, p99 and max are always present. p999 is absent below 1,000 samples.
• The per-second timeline and the steady segment use coarser buckets: 6.25%.
• Each step keeps ok and failed requests in separate histograms (ok and failed). serviceTime and responseTime merge the two, so a fast failure pulls a step's percentiles down. goodput reads only the ok side.
• Results from several runs or shards are combined by adding bucket counts. Percentiles are never averaged.

A failed step is counted, not thrown. It carries a Reason such as HttpStatus(503), TimedOut, Threw("ConnectException"), CheckFailed("has an id") or Said(...). A failed step ends that user's journey: the steps after it are not run and not counted.

Steady state and Little's law

The first seconds of a run include class loading, JIT and cold caches. result.steadyState answers SteadyState.From(offset) or NeverSettled, and result.steady is the run from that point (or the whole run where it never settled). It needs at least ten seconds of timeline.

import io.github.matthewjones372.proofload.steady
import io.github.matthewjones372.proofload.steadyState

result.steadyState        // SteadyState.From(offset) or NeverSettled
result.steady[placeOrder].responseTime.p99
result.concurrency        // Little's law: observed, predicted, ratio, agrees

result.concurrency compares the users counted in flight with what arrival rate times latency predicts. A ratio that drifts from 1 means something is queueing where the measurement does not see it, usually in the generator.

What a run does not establish

Clean schedule checks tell you the generator started users on time. They do not prove every number is the target's:

• A stalled virtual thread reads as a slow target. If a user's thread is descheduled mid-step (busy carriers, a JDBC driver that pins), the wait is inside service time and nothing flags it.
• Lateness after departure is not modelled. Delays between later steps are not added.
• Short and empty runs are not flagged as invalid. A percentile from a handful of samples is reported like any other.
• A user abandoned after a failure is not counted for the steps it did not reach. The HTML report states the shortfall between planned and sent requests.
• A run cannot be cancelled. There is no deadline, and a killed run produces no result.

Shaping the workload

A simulation is a scenario plus a load profile. Most profiles are open: they say when each user arrives, and a user arrives on time whether or not earlier users have finished. The arithmetic is exact and done before anything is sent:

• at(50.perSecond, over = 1.minutes) is 3,000 users. The count is rate times window, rounded down, so 1.perSecond over 500.milliseconds is zero users.
• User i departs at i / rate, computed from its index so the schedule does not drift.

checkout.at(50.perSecond, over = 1.minutes).userCount()   // 3000, before anything is sent
checkout.stepNames                                         // the steps it will run

Flat, ramped and staged

import io.github.matthewjones372.proofload.constantRate
import io.github.matthewjones372.proofload.hold
import io.github.matthewjones372.proofload.injecting
import io.github.matthewjones372.proofload.rampRate
import io.github.matthewjones372.proofload.then
import io.github.matthewjones372.proofload.thenRampTo

checkout.at(50.perSecond, over = 10.minutes)                     // flat

checkout.injecting(rampRate(from = 0.perSecond, to = 200.perSecond, over = 5.minutes))

checkout.injecting(
    hold(10.perSecond, over = 1.minutes)                         // warm the caches
        .thenRampTo(200.perSecond, over = 5.minutes)             // climb from 10
        .then(hold(200.perSecond, over = 10.minutes)),           // and sit there
)

thenRampTo reads the rate the shape was already running at, so the ramp starts at 10 without writing 10 twice. The profile is still a value:

val profile = hold(10.perSecond, over = 1.minutes).thenRampTo(200.perSecond, over = 5.minutes)

profile.over          // 6m
profile.userCount()   // 32100
profile.endRate       // 200.0/s

Poisson arrivals

Evenly spaced departures understate queueing. Real arrivals are close to Poisson, and queueing delay grows with the variability of arrivals, not only their mean.

import io.github.matthewjones372.proofload.randomized

checkout.injecting(constantRate(50.perSecond, over = 10.minutes).randomized(seed = 7))

Each one-second window gets the number of users the rate owes it, at random positions inside the window. There is no default seed, because an unseeded run cannot be reproduced. The report prints the seed.

If you have a log of real arrival times, replay it instead: arrivalsFrom(csv(path), column = "at") reads it, and replaying(from, window, scaled) sends a slice of it, scaling every gap by the same factor.

val friday = arrivalsFrom(csv(Path.of("friday-peak.csv")), column = "at")
friday.count      // known before anything is sent
friday.cov        // how bursty that capture was

val peak = checkout.injecting(friday.replaying(from = 40.minutes, window = 10.minutes, scaled = 2.0))

Think time

pause is the gap while a real user reads the page. It has no name and no row in the report.

val checkout = scenario("checkout") {
    exec(browse, api.get("/products"))
    pause(2.seconds)
    exec(placeOrder, api.post("/orders").body("""{"cart":"1 anvil"}"""))
}

A constant pause puts the metronome back: two hundred users that reach it together leave it together. Draw the wait instead, and give the run a seed:

val checkout = scenario("checkout") {
    exec(browse, api.get("/products"))
    pause(exponential(mean = 2.seconds))
    exec(placeOrder, api.post("/orders").body("""{"cart":"1 anvil"}"""))
}

val soak = checkout.at(50.perSecond, over = 10.minutes).thinkingFrom(seed = 20260826)

The shapes are constant, exponential, lognormal(median, sigma) and uniform(from, until). A scenario that draws is refused without a seed.

Closed populations

A closed workload is a fixed population, each user starting again when it finishes:

import io.github.matthewjones372.proofload.users

val soak = looping.at(users(50, over = 10.minutes))

Loops, conditions and mixes

val browsing = scenario("browsing") {
    exec(signIn, api.post("/session").body(credentials))

    repeat(10) {
        exec(browse, api.get("/products"))
        pause(2.seconds)
    }

    during(5.minutes) {
        exec(poll, api.get("/notifications"))
        pause(30.seconds)
    }

    doIf({ it[tier] == "gold" }) {
        exec(concierge, api.get("/concierge"))
    }
}

result[browse].count      // requests, e.g. 300
result[browse].reached    // users that reached it, e.g. 30

repeat deliberately shadows kotlin.repeat inside a scenario: it builds one step that runs ten times, so the report has one row with ten times the samples. doIf reads only the session.

For a mix of journeys, give each an arm with its own rate. The arms are merged into one departure schedule; two arms may not share a step name.

import io.github.matthewjones372.proofload.Arm
import io.github.matthewjones372.proofload.Simulation

val simulation = Simulation(
    arms = listOf(
        Arm(browsing, constantRate(500.perSecond, over = 10.minutes)),
        Arm(checkout, constantRate(20.perSecond, over = 10.minutes)),
    ),
)

proofload.run(simulation)

When one JVM cannot send the load, sharded(index, of, startingAt) splits the departures across hosts so that N injectors offer exactly what one would.

Scenarios and data

Ten thousand users sending one identical request measure whatever the target does with a duplicate: a cache hit, a deduplicated write, a unique-index collision. Give each user its own data.

Sessions, keys and captures

Each virtual user has a session: an immutable map from typed keys to values. Values flow along one user's journey and never between users. {name} in a request path or body is filled from the session key of that name.

val orderId = sessionKey<String>("orderId")

val checkout = scenario("checkout") {
    exec(
        placeOrder,
        api.post("/orders")
            .body("""{"cart":"1 anvil"}""")
            .expecting(201)
            .capture(orderId) { response -> response.header("location") },
    )
    exec(confirm, api.get("/orders/{orderId}/confirmation"))
}

A capture runs only when the status matched and every check passed. A capture that finds nothing fails the step it was declared on. A {name} the session has nothing under fails the step rather than sending the braces. Only an identifier between braces is a placeholder, so a JSON body's own braces arrive as written.

A step body reads and writes the session directly:

exec(pay) {
    val order = get(orderId)
    send(api.get("/pay").header("x-order", order.orEmpty()))
}

Feeders, lists and CSV

A feeder gives each user its starting session. It is a function of the user's number, not a cursor over a source: nothing is locked on the request path, nothing runs out, and user 4,001 gets the same data on every run.

import io.github.matthewjones372.proofload.feed
import io.github.matthewjones372.proofload.fedBy
import io.github.matthewjones372.proofload.sessionKey

val customer = sessionKey<String>("customer")

val checkout = scenario("checkout") {
    exec(browse, api.get("/customers/{customer}/products"))
}

checkout.at(50.perSecond, over = 1.minutes)
    .fedBy(feed(customer) { user -> "customer-$user" })

import io.github.matthewjones372.proofload.feedFrom
import io.github.matthewjones372.proofload.plus

val sku = sessionKey<String>("sku")

checkout.at(50.perSecond, over = 1.minutes)
    .fedBy(feedFrom(sku, listOf("anvil", "rocket", "birdseed")))   // wraps round at the end

checkout.at(50.perSecond, over = 1.minutes)
    .fedBy(feed(customer) { "customer-$it" } + feedFrom(sku, skus))  // the later one wins on a shared key

A CSV file is read once, before the run, and held in memory, because a file read between a departure and its request would be measured as the target's latency.

import io.github.matthewjones372.proofload.csv
import io.github.matthewjones372.proofload.feeding

val customer = sessionKey<String>("customer")
val tier = sessionKey<String>("tier")
val account = sessionKey<Long>("account")

val accounts = csv(Path.of("src/test/resources/accounts.csv"))

checkout.at(50.perSecond, over = 1.minutes).fedBy(accounts.feeding(customer, tier))

accounts.feeding(account) { it.toLong() }   // a non-String key names its conversion

Each key fills from the column with the same name. A key naming a column the file lacks fails when the feeder is built, listing the columns that are there. Rows wrap round. The parser handles quoted fields and doubled quotes, a small part of RFC 4180.

Generators with a stated cardinality and skew

A thousand-row file cycled for a million users keeps every row in every cache; drawing uniformly at random misses every cache. What decides the number is how many distinct keys there are and how unevenly they are asked for. proofload-arbs says both out loud.

dependencies {
    testImplementation("io.github.matthewjones372:proofload-arbs:0.1.0-rc4")
}

import io.github.matthewjones372.proofload.arbs.map
import io.github.matthewjones372.proofload.arbs.oneOf
import io.github.matthewjones372.proofload.arbs.zipf
import io.github.matthewjones372.proofload.drawing
import io.github.matthewjones372.proofload.feed
import io.github.matthewjones372.proofload.plus

val customerId = zipf(keys = 1_000_000, skew = 1.1).map { "customer-$it" }
val basket = oneOf("anvil", "rocket", "birdseed")

checkout.at(50.perSecond, over = 1.minutes)
    .fedBy(feed(customer) { customerId at it } + feed(sku) { basket at it })
    .drawing(customerId.shape, basket.shape)

A generator is read with at(userNumber), never next(), so a run replays exactly. drawing records the shapes on the report and in a baseline, and a comparison refuses two runs that drew from different ones. Other generators: uniform, digits, uuids and weighted.

Protocols

Each protocol is a module that adds steps of its own kind, and one journey can mix them. What a step times is part of what its numbers mean:

HTTP

proofload-http uses the JDK's java.net.http client, one client shared by the whole run so connection and TLS setup are not re-measured on every request. get, post, put, patch, delete and head build a request.

import io.github.matthewjones372.proofload.http.http

val api = http.baseUrl("https://orders.internal")

exec(
    placeOrder,
    api.post("/orders")
        .header("content-type", "application/json")
        .body("""{"cart":"1 anvil"}""")
        .expecting(201)
        .checking("has an order id") { it.body.contains("\"id\"") }
        .timeout(Duration.ofSeconds(2)),   // java.time.Duration
)

• Without expecting(...), a request succeeds only on 200. Any other status fails as HttpStatus(code).
• declaring(404) marks a status the target documents. It still fails, but as DeclaredStatus.
• The default timeout is 30 seconds; exceeding it fails as TimedOut. Other I/O failures are Threw(class).
• Cookies (http.baseUrl(...).withCookies(), a jar per user) and redirects (.following()) are off unless asked for.
• Retries are off by default. .retrying(times = 2, on = { it.status == 503 }, backingOff = 100.milliseconds) records the last attempt and counts the rest in attempts.
• .traced() adds a W3C traceparent and a synthetic-traffic baggage entry to every request.
• .over(transport) swaps the JDK client for your own Transport.

Server-sent events

import io.github.matthewjones372.proofload.http.cadence
import io.github.matthewjones372.proofload.http.firstEvent
import io.github.matthewjones372.proofload.http.open
import io.github.matthewjones372.proofload.http.sse
import io.github.matthewjones372.proofload.http.stopReading

val fills = sse.baseUrl("https://feeds.internal").traced().at("/fills")

val watching = scenario("watching") {
    open(opened, fills)
    firstEvent(first, within = 5.seconds)
    cadence(each, count = 99, within = 60.seconds)
    stopReading(done)
}

A hundred events is firstEvent plus cadence(count = 99). A heartbeat comment line satisfies no wait, nothing reconnects, and a far end that lets go fails the waiting step with StreamEnded.

WebSocket

import io.github.matthewjones372.proofload.Correlation
import io.github.matthewjones372.proofload.websocket.awaiting
import io.github.matthewjones372.proofload.websocket.close
import io.github.matthewjones372.proofload.websocket.open
import io.github.matthewjones372.proofload.websocket.send
import io.github.matthewjones372.proofload.websocket.ws

val feed = ws.baseUrl("wss://prices.internal")

val streaming = scenario("streaming") {
    open(connect, feed.at("/prices"))
    send(subscribe, feed.text("SUB ACME"), keyedBy = Correlation { it[order] ?: 0L })
    awaiting(ticks, count = 100, within = 30.seconds)
    close(disconnect)
}

send times the write alone; awaiting records one sample per answer, each from the send it answers. There is one socket per user, so expect file descriptors to bound your user count before the scheduler does.

gRPC

import io.github.matthewjones372.proofload.grpc.exec
import io.github.matthewjones372.proofload.grpc.grpc

val orders = grpc.target("orders.internal:8443").traced().deadline(2.seconds)
val stub = OrdersGrpc.newBlockingStub(orders.channel)

val checkout = scenario("checkout") {
    exec(orders.call(OrdersGrpc.getPlaceOrderMethod()) { stub.placeOrder(anvil) })
}

result[placeOrder].failedWith(GrpcStatus(Status.Code.UNAVAILABLE))

The row is named from the method descriptor (orders.v1.Orders/PlaceOrder). The run uses one channel, not one per user. No transport is bundled: grpc-netty-shaded or grpc-okhttp is your choice. Without generated code, proofload-grpc-dynamic makes unary calls with a JSON body from a descriptor set or server reflection.

Kafka

A broker acknowledging a record is not a consumer having done the work. A produce is an emit, and the answer read off another topic is a completing:

import io.github.matthewjones372.proofload.completing
import io.github.matthewjones372.proofload.kafka.Acks
import io.github.matthewjones372.proofload.kafka.Header
import io.github.matthewjones372.proofload.kafka.completions
import io.github.matthewjones372.proofload.kafka.emit
import io.github.matthewjones372.proofload.kafka.kafka

val broker = kafka.brokers("localhost:9092").acks(Acks.All)

val trades = scenario("trades") {
    emit(
        submitted,
        broker.topic("trades")
            .keyed { it[account]?.toString()?.toByteArray() }
            .value { avro.serialize(it[trade]) }        // your serializer
            .correlatedBy(Header("trade-id")),
        keyedBy = { it[account] ?: 0L },
    )
}

trades.at(5_000.perSecond, over = 5.minutes)
    .completing(
        settled,
        from = broker.topic("settlements").correlatedBy(Header("trade-id")).completions(),
        drainingFor = 30.seconds,
    )

No serializer or schema registry is included. linger.ms is set to 0 unless you say otherwise. A registry-backed serializer's first schema fetch lands in the produce step's own latency, not in behind; read result.steady for the run without it. result.unmatched counts work that was never answered and result.inFlight work that departed too late to be.

JDBC

import io.github.matthewjones372.proofload.jdbc.exec
import io.github.matthewjones372.proofload.jdbc.jdbc
import io.github.matthewjones372.proofload.jdbc.rows
import io.github.matthewjones372.proofload.jdbc.waitedForPool

val orderId = sessionKey<Int>("orderId")
val byId = step("select an order")

val orders = jdbc.on(dataSource)

val reading = scenario("reading") {
    exec(byId, orders.query("select id, sku from orders where id = ?").binding { listOf(it[orderId]) })
}

result[byId].serviceTime.p99    // what the database took
result[byId].waitedForPool.p99  // what users spent queueing for a connection
result[byId].rows               // rows returned, counted

The driver and the pool are yours, so the run measures the pool your service actually runs. Rows are counted, never mapped into objects. A failure is a SqlState. There are no transactions across steps and no batches.

Pelican

proofload-pelican provides a Pelican ClientTransport on the JDK HTTP client, so a generated Pelican client runs inside a load test without Pekko. Call it from a step body:

import io.github.matthewjones372.proofload.pelican.proofloadTransport

val transport = proofloadTransport(templates = listOf("/orders/{id}"))
// `orders` stands for your generated Pelican client, built on `transport`.

val reading = scenario("reading") {
    exec(readOrder) {
        val outcome = orders.get(OrderId(42))
        if (outcome.isFailure) fail("order lookup failed")
    }
}

Your own client

exec takes a block as readily as a request, and the whole block is timed. Use it for a cache, a queue, or any client you already have.

val settle = step("settle")

val ledger = scenario("ledger") {
    exec(settle) {
        val account = this[customer] ?: return@exec fail("no customer fed")
        val outcome = ledgerClient.settle(account)
        if (!outcome.ok) fail(outcome.reason)
    }
}

fail marks the step failed; it does not throw. An exception thrown from the block fails the step as Threw(class). Keep preparation out of the block: use a feeder or refreshing for values made in advance. For work that completes on another channel, emit with InMemoryCompletions and call observe(id) from your consumer.

Goals, results and reports

There are three places to say what a run must achieve: on the request (expecting, checking, capture), as goals on the simulation, and as assertions on the RunResult that run returns.

Goals and verdicts

Assert when one number decides the test:

result[placeOrder].responseTime.p99 shouldBeLessThan 200.milliseconds

Declare goals when several do, or when you want the report to say which one missed and by how much:

import io.github.matthewjones372.proofload.expecting
import io.github.matthewjones372.proofload.failureRate
import io.github.matthewjones372.proofload.goodput
import io.github.matthewjones372.proofload.keptSchedule
import io.github.matthewjones372.proofload.p99
import io.github.matthewjones372.proofload.percent

val result = proofload.run(
    checkout.at(50.perSecond, over = 10.minutes).expecting(
        p99(placeOrder) under 200.milliseconds,
        failureRate under 0.1.percent,
        goodput(placeOrder, under = 200.milliseconds) atLeast 99.percent,
        keptSchedule,
    ),
)

Each goal becomes a Verdict in result.verdicts with met, measured and, for a miss, overBy. goodput is worth reaching for over a bare percentile: a target that gets fast by failing does not pass it.

checkout.injecting(hold(100.perSecond, over = 2.minutes).thenRampTo(1000.perSecond, over = 5.minutes))
    .expecting(
        (p99(placeOrder) under 300.milliseconds).inEveryStage,
        p99(placeOrder) under 300.milliseconds,
    )

Failure reasons

import io.github.matthewjones372.proofload.Said
import io.github.matthewjones372.proofload.Threw
import io.github.matthewjones372.proofload.TimedOut
import io.github.matthewjones372.proofload.http.CheckFailed
import io.github.matthewjones372.proofload.http.HttpStatus

result[placeOrder].failedWith(HttpStatus(503))        // the target said no
result[placeOrder].failedWith(TimedOut)               // it was up and too slow
result[placeOrder].failedWith(Threw("ConnectException"))
result[placeOrder].failedWith(CheckFailed("has an order id"))

val serverErrors = result[placeOrder].failed.reasons
    .filterKeys { it is HttpStatus && it.code >= 500 }
    .values.sum()

Declare your own reason by implementing Reason with a data class or object. Past twenty distinct reasons on one step, the rest are counted under Other.

HTML reports and job summaries

import io.github.matthewjones372.proofload.report.appendToStepSummary
import io.github.matthewjones372.proofload.report.writeHtmlReport
import io.github.matthewjones372.proofload.report.writePagesIndex

result.writeHtmlReport(Path.of("build/reports/proofload/checkout.html"))   // proofload-report-html
result.appendToStepSummary()                                               // proofload-report-github
writePagesIndex(Path.of("build/pages"))                                   // an index.html for GitHub Pages

The HTML page is one file with data, stylesheet and script inline, so it opens from disk and uploads as a CI artifact unchanged. It shows the goals and verdicts first, then the plan and what was sent, any schedule banner, goodput, the injector's own stalls, failures by reason, a per-step table on either clock, distributions and the timeline. appendToStepSummary writes to the Actions job summary and returns StepSummary.NotOnActions off Actions, so the same code runs on a laptop.

Baselines and comparisons

import io.github.matthewjones372.proofload.against
import io.github.matthewjones372.proofload.baseline.readBaseline
import io.github.matthewjones372.proofload.baseline.writeBaseline

result.writeBaseline(Path.of("build/proofload/checkout.proofload"))

val comparison = result.against(readBaseline(Path.of("build/proofload/checkout.proofload")))

against compares p99 of response time per step and answers Comparison.Compared (each step Better, Worse, Indistinguishable, Added or Gone) or Comparison.NotComparable, for example when the runs were not asked to do the same thing. Runs from different machines get a caveat.

A single run cannot bound its own noise. Runs compares populations of at least five runs a side by bootstrap, and answers Better, Worse or CannotTell:

import io.github.matthewjones372.proofload.Runs
import io.github.matthewjones372.proofload.against
import io.github.matthewjones372.proofload.baseline.readAll
import io.github.matthewjones372.proofload.junit5.assertNotWorseThan

val before = Runs.readAll(Path.of("baselines/checkout"))
val now = Runs(List(5) { proofload.run(checkout.at(50.perSecond, over = 2.minutes)) })

now.against(before, p99(placeOrder), acceptable = 10.percent)
    .assertNotWorseThan(10.percent)

proofload.calibrate() measures what the machine can resolve (up to 30 seconds, once per JVM), and result.calibratedBy(floor) lets a comparison say "cannot tell" when a difference is below that floor.

Capacity search

Rather than guessing a rate, ask for the highest one that holds your goals:

import io.github.matthewjones372.proofload.engine.run
import io.github.matthewjones372.proofload.sustainable

val capacity = checkout.sustainable(
    upTo = 500.perSecond,
    holding = 2.minutes,
    expecting = listOf(p99(placeOrder) under 200.milliseconds, failureRate under 1.percent),
).run()

capacity.curve      // every rung that ran, with the run behind it
capacity.voided     // true if a rung was void: the answer is about this machine

capacity.writeHtmlReport(Path.of("build/reports/proofload/capacity.html"))

It climbs a ladder of up to ten rungs, continues two rungs past the first miss, then bisects. A rung is void when the generator lost ground, the process ran short of file descriptors or ports, or it refused arrivals at its cap. Check voided before quoting the answer. search.warmingUp(over) warms each rung at its own rate.

Exporting

import io.github.matthewjones372.proofload.export.Density
import io.github.matthewjones372.proofload.export.json
import io.github.matthewjones372.proofload.export.writeHistogramLog
import io.github.matthewjones372.proofload.export.writeJson
import io.github.matthewjones372.proofload.export.writeOpenMetrics
import io.github.matthewjones372.proofload.otel.Sent
import io.github.matthewjones372.proofload.otel.sendOtlp

println(result.json(Density.Summary))                        // the answer, under two kilobytes
result.writeJson(Path.of("build/run.json"))                  // Density.Full, every step
result.writeHistogramLog(Path.of("build/proofload/run.hlog"))
result.writeOpenMetrics(Path.of("/var/lib/node_exporter/proofload.prom"))

when (val sent = result.sendOtlp("http://collector:4318/v1/metrics")) {
    Sent.Accepted -> println("the collector took them")
    is Sent.Refused -> println("it did not: ${sent.why}")
}

Verdicts travel only in the JSON run document. Its verdict is the first of behind, nothingAsked, missed, cannotTell and met that applies, with a one-sentence remedy beside it. The metrics exports carry measurements but not goals or verdicts.

Running it

A simulation is a value. Running it needs an engine, and proofload-engine runs it on virtual threads. The scenario and the result are the same whichever runner you use.

JUnit

@LoadTest is a JUnit Jupiter @Test with a Proofload parameter and a one-hour timeout. There is no base class and nothing to register. The first test above is a complete example.

Kotest

Add proofload-kotest. proofload() is a suspend function that returns a runner; it prints no progress lines.

import io.github.matthewjones372.proofload.kotest.proofload
import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.comparables.shouldBeLessThan

class CheckoutSpec : StringSpec({

    "checkout holds up at fifty a second" {
        val result = proofload().run(checkout.at(50.perSecond, over = 1.minutes))

        result[placeOrder].responseTime.p99 shouldBeLessThan 200.milliseconds
    }
})

zio-test

proofload-zio-test gives Scala 3 a ProofloadSpec, a ZIOSpecDefault that already carries TestAspect.sequential and TestAspect.withLiveClock. measured runs a simulation and writes its report under reportsTo (default target/proofload).

object CheckoutSpec extends ProofloadSpec:

  def spec = suite("checkout")(
    test("holds up at fifty a second"):
      for
        result <- measured("checkout"):
          checkout.at(50.perSecond, over = 1.minute).expecting(p99(placeOrder) under 200.millis)
      yield assert(result)(failedNone && metEveryGoal),
  )

A plain main

import io.github.matthewjones372.proofload.engine.run
import io.github.matthewjones372.proofload.report.writeHtmlReport
import java.nio.file.Path

fun main() {
    val result = checkout.at(50.perSecond, over = 10.minutes).run()

    result.writeHtmlReport(Path.of("build/reports/soak.html"))
}

To quieten or slow the progress lines, build the runner yourself: Proofload(Progress.silent) or Proofload(Progress.lines(every = 30.seconds)).

YAML plans and the CLI

proofload-plan reads a plan/1 file describing a scenario, its load and its goals:

proofload:  plan/1
baseUrl:  https://orders.internal
scenario: checkout
steps:
  - name: browse
    get:  /products
  - name: place order
    post: /orders
    body: '{"cart":"1 anvil"}'
    expecting: 201
load:
  rate: 50/s
  over: 1m
goals:
  - step: place order
    p99:  200ms

import io.github.matthewjones372.proofload.plan.asSimulation
import io.github.matthewjones372.proofload.plan.readPlan

val simulation = readPlan(Path.of("checkout.yaml")).asSimulation()
val result = proofload.run(simulation)

An unknown key is refused and every error names its line. A plan has no keptSchedule goal, and cannot name a gRPC, WebSocket, SSE or JDBC step. Kafka steps need proofload-plan-kafka.

proofload-cli runs plans without a Kotlin build. The published jar names its main class, so a launcher that resolves Maven coordinates can start it. java -jar does not work, because the jar carries no classpath.

jbang io.github.matthewjones372:proofload-cli:0.1.0-rc4 validate checkout.yaml   # parse and resolve, send nothing
jbang io.github.matthewjones372:proofload-cli:0.1.0-rc4 preview checkout.yaml    # say what it would send
jbang io.github.matthewjones372:proofload-cli:0.1.0-rc4 run checkout.yaml        # run it; exit code is the verdict
jbang io.github.matthewjones372:proofload-cli:0.1.0-rc4 emit checkout.yaml --package com.acme.load

run --json prints the JSON run document instead of a summary. emit prints the plan as the equivalent Kotlin, to edit and commit when a plan outgrows the format.

OpenAPI and HAR

proofload from-openapi (module proofload-openapi) writes a first-draft plan from an OpenAPI document: one step per GET, the first documented 2xx as expecting, 4xx and 5xx as declared, bounded parameters as draws, a load of 1 a second for 10 seconds and a placeholder p99 goal of one second per step.

jbang io.github.matthewjones372:proofload-cli:0.1.0-rc4 from-openapi orders.yaml --base-url https://orders.internal

proofload-record reads a HAR recording into Kotlin source: one exec per request, values one response produced and a later request used become a capture, static assets are excluded, and credentials are removed. Its published jar does not name a main class; from a clone of the repository:

./gradlew :proofload-record:run --args="checkout.har --package com.acme.load --out src/test/kotlin"

Recipes

Things a new user tends to need in the first week.

Load test a server started in the same test

The repository's own example starts a JDK HttpServer on a free port, so a test measures the tool rather than a network. Trimmed:

import com.sun.net.httpserver.HttpServer
import java.net.InetSocketAddress

class CheckoutLoadTest {

    private lateinit var server: HttpServer

    @BeforeEach
    fun start() {
        server = HttpServer.create(InetSocketAddress(0), 0)
        server.createContext("/products") { exchange ->
            val bytes = """{"items":2}""".toByteArray()
            exchange.sendResponseHeaders(200, bytes.size.toLong())
            exchange.responseBody.use { it.write(bytes) }
        }
        server.start()
    }

    @AfterEach
    fun stop() = server.stop(0)

    @LoadTest
    fun `browsing holds up`(proofload: Proofload) {
        val api = http.baseUrl("http://localhost:${server.address.port}")
        val browsing = scenario("browsing") { exec(browse, api.get("/products")) }

        val result = proofload.run(browsing.at(50.perSecond, over = 1.seconds))

        assertEquals(50L, result[browse].count)
    }
}

Walk one user through before spending ten minutes of load

import io.github.matthewjones372.proofload.engine.trace

checkout.trace()

trace prints what each step did for one user, including the step where the user was abandoned and why. It schedules nothing and returns nothing, so it cannot be read as a measurement.

Warm up without recording it

A cold JVM, pool and JIT inflate the first seconds. warmingUp sends load before the measurement and records none of it. There is no default.

import io.github.matthewjones372.proofload.warmingUp

proofload.run(shopping.at(200.perSecond, over = 10.seconds).warmingUp(2.seconds))

A token that expires mid-run

Fetching a token inside a step makes the p99 a measurement of your identity provider. refreshing keeps it off the measured path:

import io.github.matthewjones372.proofload.refreshing

val token = refreshing(every = 4.minutes) { fetchToken() }

val checkout = scenario("checkout") {
    exec(placeOrder) {
        send(api.post("/orders").header("authorization", "Bearer ${token.current}"))
    }
}

// afterwards
token.stop()

Download large responses without holding them

exec(download, api.get("/exports/{id}").discardingBody())

Response.bytes is what came back. Check it: a download that returned 4 KB instead of 200 MB reads as a very fast target. A check or capture on a discarding step is refused when the scenario is built.

Retry a 503 the way a real client would

exec(
    placeOrder,
    api.post("/orders")
        .expecting(201)
        .retrying(times = 2, on = { it.status == 503 }, backingOff = 100.milliseconds),
)

result[placeOrder].count      // requests
result[placeOrder].attempts   // trips to the target

Keep load tests out of ./gradlew test

Load tests busy the machine. The petshop service keeps them in their own source set and runs them only through a dedicated task, starting the service in-process and holding it at 200 requests a second:

// Only run as `./gradlew loadTest`.
tasks.test { enabled = false }

tasks.register<Test>("loadTest") {
    description = "Runs the load tests against the whole service, started in-process."
    group = "verification"
    testClassesDirs = sourceSets.test.get().output.classesDirs
    classpath = sourceSets.test.get().runtimeClasspath
    // A cached pass says nothing about this machine now.
    outputs.upToDateWhen { false }
    maxHeapSize = "4g"
    jvmArgs("-XX:+ExitOnOutOfMemoryError")
}

Stop a typo from flooding a target

A proofload.toml beside the build says what runs on this machine may do. The CLI and MCP server read it from their working directory; from Kotlin, preview checks a simulation against it before anything leaves.

# what a run on this machine may do
hosts       = ["localhost", "*.staging.internal"]
maxRate     = "500/s"
maxDuration = "10m"
maxRequests = 2_000_000

import io.github.matthewjones372.proofload.Allowance
import io.github.matthewjones372.proofload.Preview
import io.github.matthewjones372.proofload.preview

when (val asked = checkout.at(50.perSecond, over = 1.minutes).preview(Allowance.fromFile())) {
    is Preview.Allowed -> println("${asked.users} users to ${asked.hosts}")
    is Preview.Refused -> println(asked.reason.described)
}

Report latency in CI, gate on failures

On a shared runner, fail the job on failed requests, errors and steps that stopped running. Report latency in the job summary, against the last baseline, and let a reviewer decide.

fun main() {
    val baseline = Path.of("build/proofload/paying.proofload")
    val proofload = Proofload()

    val floor = proofload.calibrate()   // before the load
    val result = proofload.run(paying.at(200.perSecond, over = 2.minutes)).calibratedBy(floor)

    val previous = baseline.takeIf { Files.exists(it) }?.let(::readBaseline)
    val comparison = result.against(previous)

    result.appendToStepSummary(comparison, floor)
    result.writeBaseline(baseline)

    if (result.failed > 0L) exitProcess(1)
}

Keep the baseline file between runs with actions/cache, an uploaded artifact, or an orphan branch changed by pull request. The repository's AgainstTheBaseline.kt and baseline.yml are a working copy.

Costs, gotchas and limits

What it costs

The project measures its own overhead with the benchmarks in benchmarks/, on named machines, one run per rate. Every target answered immediately, so these are rates at near-zero concurrency, not what Proofload costs holding thousands of users open.

At 5,000 and 10,000 a second over loopback the failures were ephemeral ports recycling too slowly, not descriptors and not the target. Nothing here runs in ./gradlew build: run the benchmark tasks on your own quiet machine and read your own numbers. Every run carries result.hiccups; where it is the same size as the target's tail, the tail is the machine.

Gotchas

• Nothing fails for falling behind unless you ask. Declare keptSchedule, assert metEveryGoal, and assert lostGround() separately.
• No goals means nothing was checked. metEveryGoal is true, the JSON verdict is nothingAsked and the CLI exits 0.
• User counts round down. 1.perSecond over 500.milliseconds is zero users, and some goals are then met. Check result[step].count.
• Fast failures flatter a p99. A step's percentiles include failed requests; prefer goodput.
• fellBehind() fires easily against very fast targets, because any lateness is large next to a sub-millisecond response.
• Different surfaces judge goals over different windows. result.verdicts uses the steady segment; JSON, CLI and the Java facade use the whole run.
• The first run in a JVM is cold. Warm up, or discard it before keeping a baseline.
• Closed runs pass every schedule check by construction. Read Little's law instead.
• Cookies, redirects and retries are off by default for HTTP. A 302 is a failure until you add following().
• Two containers on one host do not share a temp directory, so the one-run-per-machine lock cannot queue them against each other.
• A run cannot be cancelled, and a killed run leaves no result.

Status and versions

0.1.0-rc4 is the current release on Maven Central, and every coordinate on this page is pinned to it. 0.1.0 has not been released; until 1.0 a breaking change can come without a major version bump and is recorded in the CHANGELOG. Every published module carries an API dump, and the build fails where the public surface moved and the dump did not.

The project's documents describe main. Where rc4 differs:

• result.against(previous, of = Clock.ServiceTime) (choosing the clock for a comparison) is on main only. In rc4, against always compares response time.
• On main, a long run's timeline stops growing after 3,600 slots by widening them.
• On main, cs launch on the proofload-mcp coordinate starts the MCP server; with rc4, pass -M io.github.matthewjones372.proofload.mcp.ServerKt.

Release candidates after rc4 go to Maven Central's snapshot repository, which drops them after 90 days. To use main, build it with ./gradlew publishToMavenLocal.

Where next

The repository's documents go further on every topic here.