Benchmarks Public benchmark suites generated from reference environments and verified against live runtime truth. benchmarks benchmarks benchmarks benchmarks/_index.md

Benchmarks

Benchmarks

Public benchmark suites generated from reference environments and verified against live runtime truth.

GraphJin benchmarks are frozen, oracle-verified suites generated from reference environments. Models run through the same GraphJin interface, while publication gates preserve the suite fingerprint, reject stale scoring contracts, flag suspicious answer/method divergence, and record the exact GraphJin build used for every result.

The family grows as real reference environments are added. We list only benchmarks that exist and have a published, reproducible suite.

BenchmarkStatusModels on boardLast updated
DeepContoso — The Retail & Supply Chain BenchmarkLiveGemini 3.8 Flash2026-09-24
DeepORG — The Organizational Agent BenchmarkLiveDeepSeek V4 Flash, Gemini 3.5 Flash, gemini-3.5-flash-lite, gemini-3.7-flash, gemini-4-argon, Gemma 4 26B A4B IT (Vertex AI MaaS), Gemma 4 31B (Cerebras), GPT-5.4 mini, meta-models/Muse-Glimmer-30B, OpenAI GPT-4o mini2026-10-15
Docs