Benchmarks

OasDelta against oasdiff, measured on GitHub-hosted runners by the Benchmarks & site workflow on every push to main and weekly. The numbers on this page come from the run that built it.

Linux 4 cores, GitHub-hosted runner

Speed-up, real-world APIs

2.8×

geometric mean, GitHub, Stripe and Kubernetes releases

Speed-up, all cases

2.6×

range 1.1× – 5.9×

Cases where oasdelta is faster

23 / 23

median wall time, start-up included

Identical output

22 / 23*

stdout and exit code, byte for byte

* One case (k8s-core changelog -f json) prints the same entries as oasdiff in a different order. oasdiff’s own order there changes from run to run (5 orders in 5 runs); OasDelta’s does not. oasdiff sorts changes with a comparison that ignores some of their fields, and leaves changes that differ only in those fields in Go’s random map order.

Windows 4 cores, GitHub-hosted runner

Speed-up, real-world APIs

2.4×

geometric mean, GitHub, Stripe and Kubernetes releases

Speed-up, all cases

2.2×

range 1.1× – 5.2×

Cases where oasdelta is faster

23 / 23

median wall time, start-up included

Identical output

22 / 23*

stdout and exit code, byte for byte

* One case (k8s-core changelog -f json) prints the same entries as oasdiff in a different order. oasdiff’s own order there changes from run to run (5 orders in 5 runs); OasDelta’s does not. oasdiff sorts changes with a comparison that ignores some of their fields, and leaves changes that differ only in those fields in Go’s random map order.

Results

oasdelta (C#, NativeAOT)oasdiff (Go)

Median wall time per invocation, process start-up included. Shorter is better. Hover or focus a row for details.

Small YAML (openapi-test1 → test3, 10 KB)

diff

2.7× faster

summary

2.4× faster

breaking

2.4× faster

changelog -f json

1.9× faster

changelog -f html

2.3× faster

0 ms10 ms20 ms30 ms

Large YAML (synthetic, ~1.5 MB)

diff

3.2× faster

summary

2.9× faster

breaking

2.5× faster

changelog -f json

2.5× faster

changelog -f html

2.6× faster

0 ms500 ms1 s1.5 s2 s

Large JSON (synthetic, ~1.1 MB)

diff

2.6× faster

summary

2.7× faster

breaking

2.0× faster

changelog -f json

1.9× faster

changelog -f html

2.0× faster

0 ms250 ms500 ms750 ms1 s

GitHub REST API (YAML, ~10 MB, Aug → Nov 2026)

diff

2.5× faster

breaking

2.4× faster

changelog -f json

2.4× faster

0 ms2 s4 s6 s8 s

Stripe API (JSON, ~8 MB, v2500 → v2598)

breaking

5.4× faster

changelog -f json

4.6× faster

0 ms5 s10 s15 s20 s

Kubernetes core/v1 (JSON, ~2 MB, 1.36 → 1.37)

diff

5.9× faster

breaking

1.8× faster

changelog -f json

1.1× faster

0 ms2 s4 s6 s

What is measured

• Binaries. oasdelta is the NativeAOT build that dotnet publish produces from this repository. oasdiff is the official release binary downloaded from oasdiff/oasdiff releases, with its checksum verified. Neither is rebuilt or tuned for the benchmark.
• Inputs.
  • Small: openapi-test1.yaml → openapi-test3.yaml from the upstream test data, about 10 KB. This case is dominated by process start-up.
  • Large YAML / JSON: a synthetic API from tools/gen_bench_spec.py with 600 resources (1,200 paths, 2,400 operations, 650 schemas). The revision changes about one resource in ten: removed and added properties, type and enum changes, new required fields, removed endpoints and responses. So every stage, from parsing through diffing and checks to rendering, has work to do.
  • Real-world: two consecutive releases each of the GitHub REST API (YAML, ~10 MB), Stripe (JSON, ~8 MB) and Kubernetes core/v1 (JSON, ~2 MB), from OASDelta-TestData. Each source's license is recorded there.
• Commands. diff, summary, breaking, changelog -f json and changelog -f html on the small and synthetic inputs. diff, breaking and changelog -f json on the real-world pairs, except Stripe's diff, whose YAML output is over 5 GB.
• Timing. hyperfine 2.0 measures the wall time of the whole process, from spawn to exit, start-up included, because that is what a CI step pays. It starts each binary directly, with no shell in between, and discards its output. Each synthetic case gets 3 untimed warm-up runs, then 15 timed runs per binary. The real-world cases take seconds each, so they get 1 warm-up and 5 timed runs. hyperfine runs one binary's runs and then the other's, and which binary goes first alternates from case to case. The chart shows the median; the table adds the standard deviation.
• Memory. On Linux, peak resident set size as hyperfine measures it, the highest over the timed runs. A process that hyperfine starts inherits hyperfine's own peak of a few MB, so no reading is lower than that. The environment table shows the reading for a program that does nothing. hyperfine cannot measure memory on Windows, so there each binary runs 3 more times, untimed, and the peak working set comes from GetProcessMemoryInfo, which counts only the process itself.
• Correctness. Before timing, each binary runs once with its output saved to a file, and the two outputs and exit codes are compared byte for byte. A speed-up only counts if the answer is the same. Where the bytes differ but the lines, or the JSON array entries, are the same in another order, the case is reported as same entries, other order. Both binaries then run 5 times each, and the report records how many different outputs each one printed.

The harness is tools/bench.py. It runs the output check and drives hyperfine, measuring no time itself.

Environment

	Linux	Windows
OS	Linux 7.0.0-1019-azure (x86_64)	Windows 2025Server (AMD64)
CPU	INTEL(R) XEON(R) PLATINUM 8573C, 4 cores	AMD EPYC 9V74 80-Core Processor, 4 cores
Go	oasdiff version 1.35.0, 18.4 MB binary	oasdiff version 1.35.0, 18.9 MB binary
C#	oasdelta version 1.0.0, 10.3 MB binary	oasdelta version 1.0.0, 10 MB binary
Runs	3 warm-up + 15 timed, per case and binary	3 warm-up + 15 timed, per case and binary
Timing	hyperfine 2.0.0	hyperfine 2.0.0
Memory	peak resident set size (hyperfine), highest of the timed runs; 1.4 MB for a program that does nothing	peak working set (GetProcessMemoryInfo), highest of 3 untimed runs
Real-world	1 warm-up + 5 timed, from OASDelta-TestData 15b2cb4	1 warm-up + 5 timed, from OASDelta-TestData 15b2cb4
Measured	Sat, 14 Nov 2026 13:56 UTC	Sat, 14 Nov 2026 13:57 UTC
Source	278608a · workflow run	278608a · workflow run

All numbers

Linux

Input	Command	oasdiff (Go)	oasdelta (C#)	Speed-up	Go peak	C# peak	Output
small	diff	22.2 ms ± 1.6 ms	8.3 ms ± 1.1 ms	2.7×	21.7 MB	16.3 MB	identical
small	summary	18.9 ms ± 0.6 ms	8.1 ms ± 1.1 ms	2.4×	20.3 MB	16.2 MB	identical
small	breaking	21.1 ms ± 1.8 ms	8.9 ms ± 1.1 ms	2.4×	21.9 MB	16.8 MB	identical
small	changelog -f json	21.4 ms ± 1.1 ms	11.3 ms ± 0.7 ms	1.9×	21.8 MB	21.9 MB	identical
small	changelog -f html	21.5 ms ± 1.1 ms	9.4 ms ± 0.7 ms	2.3×	21.8 MB	17.3 MB	identical
large-yaml	diff	1.95 s ± 21.4 ms	603 ms ± 5.4 ms	3.2×	317 MB	201 MB	identical
large-yaml	summary	1.69 s ± 14.7 ms	577 ms ± 6.0 ms	2.9×	302 MB	182 MB	identical
large-yaml	breaking	1.77 s ± 13.7 ms	696 ms ± 10.6 ms	2.5×	311 MB	209 MB	identical
large-yaml	changelog -f json	1.78 s ± 20.0 ms	711 ms ± 7.7 ms	2.5×	315 MB	214 MB	identical
large-yaml	changelog -f html	1.78 s ± 12.2 ms	691 ms ± 8.1 ms	2.6×	313 MB	210 MB	identical
large-json	diff	970 ms ± 7.3 ms	372 ms ± 6.8 ms	2.6×	223 MB	193 MB	identical
large-json	summary	734 ms ± 4.7 ms	271 ms ± 5.1 ms	2.7×	103 MB	184 MB	identical
large-json	breaking	799 ms ± 16.3 ms	398 ms ± 6.9 ms	2.0×	109 MB	193 MB	identical
large-json	changelog -f json	835 ms ± 7.8 ms	436 ms ± 4.9 ms	1.9×	113 MB	198 MB	identical
large-json	changelog -f html	830 ms ± 9.9 ms	421 ms ± 6.1 ms	2.0×	113 MB	193 MB	identical
github	diff	7.38 s ± 83.5 ms	2.96 s ± 18.1 ms	2.5×	1136 MB	481 MB	identical
github	breaking	6.81 s ± 50.1 ms	2.86 s ± 10.3 ms	2.4×	1003 MB	481 MB	identical
github	changelog -f json	6.94 s ± 55.5 ms	2.90 s ± 38.3 ms	2.4×	1118 MB	481 MB	identical
stripe	breaking	17.26 s ± 58.0 ms	3.18 s ± 39.9 ms	5.4×	457 MB	543 MB	identical
stripe	changelog -f json	17.55 s ± 88.7 ms	3.80 s ± 21.3 ms	4.6×	857 MB	661 MB	identical
k8s-core	diff	4.11 s ± 129 ms	691 ms ± 3.5 ms	5.9×	2060 MB	221 MB	identical
k8s-core	breaking	1.36 s ± 10.8 ms	780 ms ± 30.4 ms	1.8×	218 MB	241 MB	identical
k8s-core	changelog -f json	1.96 s ± 17.2 ms	1.73 s ± 12.6 ms	1.1×	593 MB	458 MB	same entries, other order (distinct in 5 runs: oasdiff 5, oasdelta 1)

Windows

Input	Command	oasdiff (Go)	oasdelta (C#)	Speed-up	Go peak	C# peak	Output
small	diff	30.6 ms ± 1.3 ms	13.7 ms ± 0.2 ms	2.2×	20.0 MB	13.1 MB	identical
small	summary	28.2 ms ± 1.1 ms	13.4 ms ± 0.2 ms	2.1×	18.7 MB	12.9 MB	identical
small	breaking	29.8 ms ± 1.2 ms	14.5 ms ± 0.5 ms	2.1×	19.8 MB	14.3 MB	identical
small	changelog -f json	29.5 ms ± 0.6 ms	15.2 ms ± 0.4 ms	1.9×	19.8 MB	14.5 MB	identical
small	changelog -f html	30.5 ms ± 0.6 ms	15.2 ms ± 1.1 ms	2.0×	20.5 MB	15.0 MB	identical
large-yaml	diff	1.67 s ± 19.9 ms	685 ms ± 4.3 ms	2.4×	325 MB	193 MB	identical
large-yaml	summary	1.48 s ± 21.7 ms	717 ms ± 58.1 ms	2.1×	307 MB	193 MB	identical
large-yaml	breaking	1.56 s ± 20.3 ms	717 ms ± 14.9 ms	2.2×	303 MB	194 MB	identical
large-yaml	changelog -f json	1.56 s ± 24.2 ms	724 ms ± 5.1 ms	2.2×	262 MB	195 MB	identical
large-yaml	changelog -f html	1.57 s ± 16.1 ms	720 ms ± 6.8 ms	2.2×	299 MB	196 MB	identical
large-json	diff	831 ms ± 9.5 ms	344 ms ± 12.5 ms	2.4×	236 MB	165 MB	identical
large-json	summary	679 ms ± 11.0 ms	321 ms ± 3.3 ms	2.1×	104 MB	165 MB	identical
large-json	breaking	724 ms ± 6.0 ms	366 ms ± 2.8 ms	2.0×	103 MB	172 MB	identical
large-json	changelog -f json	729 ms ± 7.0 ms	376 ms ± 2.6 ms	1.9×	106 MB	179 MB	identical
large-json	changelog -f html	735 ms ± 7.6 ms	378 ms ± 4.5 ms	1.9×	107 MB	183 MB	identical
github	diff	6.29 s ± 34.6 ms	3.26 s ± 32.7 ms	1.9×	978 MB	519 MB	identical
github	breaking	6.50 s ± 421 ms	3.06 s ± 18.0 ms	2.1×	1080 MB	519 MB	identical
github	changelog -f json	5.91 s ± 31.1 ms	3.07 s ± 32.1 ms	1.9×	1063 MB	519 MB	identical
stripe	breaking	14.79 s ± 194 ms	2.87 s ± 17.1 ms	5.2×	457 MB	505 MB	identical
stripe	changelog -f json	15.09 s ± 84.1 ms	3.79 s ± 13.4 ms	4.0×	848 MB	746 MB	identical
k8s-core	diff	2.77 s ± 49.7 ms	717 ms ± 7.5 ms	3.9×	2164 MB	216 MB	identical
k8s-core	breaking	1.26 s ± 5.1 ms	736 ms ± 7.1 ms	1.7×	209 MB	233 MB	identical
k8s-core	changelog -f json	1.74 s ± 17.6 ms	1.66 s ± 7.4 ms	1.1×	576 MB	458 MB	same entries, other order (distinct in 5 runs: oasdiff 5, oasdelta 1)

Reproduce it

dotnet publish src/OasDelta -c Release -r linux-x64 -o out
mkdir specs && python3 tools/gen_bench_spec.py specs 600
python3 tools/bench.py --go "$(command -v oasdiff)" --cs out/oasdelta --specs specs --runs 15

This needs hyperfine 2.0 or later on PATH, or passed with --hyperfine. On Windows, use -r win-x64 and out\oasdelta.exe. Add --json results.json to write a file in the format this site reads; put it in docs/data/benchmarks/ and run npm run dev in docs/ to see it here.

Caveats

• Shared CI runners are noisy. The medians are stable from run to run, but treat a difference under about 10% as a tie.
• Until November 2026 the harness timed runs itself in Python and read each binary's output through a pipe, so those numbers included reading the output. On Linux its memory readings were too high: they included memory held by the Python process that started each binary, which kept every output it had read.
• The synthetic inputs are built to exercise every stage of the pipeline, and the real-world pairs show three large public APIs. Your API's shape may still favour one implementation more or less.
• Memory is a trade-off, not a free win. oasdelta peaks lower on small specs and on large YAML, but on large JSON oasdiff's peak is usually lower. .NET's GC is tuned here for throughput on a short-lived process (see Why C# and .NET), so check the memory view before running oasdelta in a tight container.