Benchmarks
Thirty prompts, ten categories, human-labeled with the cheapest tier that should handle each. Run any of them against all three tiers.
Prompts30
Categories10
Router accuracy30/30router agrees with human labels
coding
3 promptsFix a broken regex
humanLunarouterLuna
This regex is supposed to match ISO dates but also matches '2024-13-45'. Fix it: /\d{4}-\d{2}-\d{2}/Write a debounce hook
humanTerrarouterTerra
Write a React useDebouncedValue hook in TypeScript. It should accept a value and a delay in ms, update only after the value stops changing, clean up its timer on unmount, and be fully typed with generics.
Migrate callback API to async/await
humanTerrarouterTerra
Refactor this Node.js module from nested callbacks to async/await, preserving error handling semantics and adding proper TypeScript types:
```js
function loadUser(id, cb) {
db.query('SELECT * FROM users WHERE id = ?', [id], (err, rows) =…architecture
3 promptsDesign a distributed rate limiter
humanSolrouterSol
Design a distributed rate limiter for an API gateway serving 50k requests/sec across 12 edge regions. Compare token bucket vs sliding window at this scale, explain where state should live, how you handle clock skew, and what happens during …
Pick a database for a leaderboard
humanLunarouterLuna
I'm building a real-time gaming leaderboard with 2M daily active users, frequent score updates, and global top-100 reads. Should I use Postgres, Redis sorted sets, or something else? One paragraph on why.
Monolith to event-driven migration plan
humanSolrouterSol
We have a Rails monolith processing 30k orders/day. I need a phased migration plan to event-driven microservices: what to carve out first, how to keep the monolith authoritative during transition, how to avoid dual-write inconsistencies, an…
extraction
3 promptsPull contact info from a signature
humanLunarouterLuna
Extract the name, title, company, phone, and email from this signature as JSON: Best, Priya Ramanathan Senior Director, Platform Engineering Hexawave Systems +1 (415) 555-0173 priya.r@hexawave.io
Extract invoice line items
humanLunarouterLuna
From this invoice text, extract each line item with description, quantity, unit price, and total as a JSON array: INVOICE #4482 Web hosting (annual) — 1 × $240.00 — $240.00 SSL certificate — 2 × $45.00 — $90.00 Overage bandwidth 40GB — 1 ×…
Normalize messy dates
humanLunarouterLuna
Convert every date in this text to ISO 8601 and list them: 'The lease starts March 3rd, 2025, with a review on 9/15/2025 and termination no later than 2026-02-28.'
summarization
3 promptsOne-line support ticket summary
humanLunarouterLuna
Summarize this support ticket in one sentence for a triage dashboard: 'Hi, since the update yesterday around 2pm our team of 14 people cannot log in via SSO, we get redirected in a loop. Regular password login works. This is blocking all wo…
Summarize a long ops report
humanTerrarouterTerra
Summarize the following operations report into 5 bullet points for an executive readout: Quarterly operations review — Northbridge Logistics. Revenue grew 8% quarter over quarter, driven by same-day delivery expansion in three metro areas.…
Meeting notes to action items
humanLunarouterLuna
Turn these meeting notes into action items with owners: 'Ana: launch moved to the 14th because payments aren't ready. Boris will finish Stripe webhook retries by Friday. Priya to confirm the email copy with legal — they flagged the discoun…
translation
3 promptsTranslate UI strings to Spanish
humanLunarouterLuna
Translate these UI strings to Spanish, keeping placeholders untouched: 'Save changes', 'Discard draft', '{count} items selected', 'Your trial ends in {days} days'.Translate an error message to Japanese
humanLunarouterLuna
Translate to natural, polite Japanese suitable for an end-user error dialog: 'We couldn't process your payment. Your card was declined. Please try a different payment method or contact your bank.'
Translate marketing copy to German
humanLunarouterLuna
Translate this product blurb to German, preserving the confident tone and adapting idioms naturally rather than literally: 'Stop babysitting your deploys. Ship on Friday at 5pm and sleep like a baby — our rollback button has your back.'
reasoning
3 promptsClassic word problem
humanLunarouterLuna
A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? Answer with the amount only.
Weigh two product options
humanTerrarouterTerra
We can either ship SSO this quarter (requested by 4 enterprise prospects worth ~$220k ARR combined) or rebuild onboarding (current activation is 31%, industry median ~45%). Engineering capacity: one team, one quarter. Which do we pick and w…
Multi-constraint logic puzzle
humanTerrarouterTerra
Five engineers (Ada, Babbage, Church, Dijkstra, Hopper) each own exactly one of five deploy slots (Mon–Fri) and one language (Go, Rust, Python, TS, Haskell). Clues: Ada deploys before Church. The Rust user deploys on Wednesday. Dijkstra use…
math
3 promptsSplit a bill
humanLunarouterLuna
Three people split a $86.40 dinner with 18% tip. Two had equal shares, the third paid exactly double. What did each pay? Round to cents.
Compound growth projection
humanTerrarouterTerra
Our MRR is $48k growing 7% month over month. Churn is 3.5% of MRR monthly, applied to the prior month's base. Model the next 12 months month-by-month and tell me the month we cross $100k MRR. Show the formula you used.
Multi-stage probability proof
humanSolrouterSol
A monitoring pipeline has three independent stages with false-negative rates 2%, 5%, and 1%. An incident must pass all three checks undetected to be fully missed. (a) Prove the probability of a fully missed incident. (b) We can halve exactl…
long-context
3 promptsFind the contradiction in a long report
humanTerrarouterTerra
Read the following operations report carefully. Somewhere in it, two statements contradict each other about the Reno facility. Quote both and explain the contradiction: Quarterly operations review — Northbridge Logistics. Revenue grew 8% q…
Executive brief from a long document
humanTerrarouterTerra
Write a 150-word executive brief from this report, focusing only on financial risks and the two most urgent operational problems: Quarterly operations review — Northbridge Logistics. Revenue grew 8% quarter over quarter, driven by same-day…
Cross-reference claims across a document
humanTerrarouterTerra
Analyze this report end to end. Identify every claim about cost savings, cross-reference each against the stated risks and overruns, and produce a net assessment of whether the reported savings are credible. Structure the answer as: claims …
tool-calling
3 promptsSingle tool lookup
humanLunarouterLuna
What's the weather in Lisbon right now, and should I bring a jacket tonight? Use the weather tool.
Chained tool workflow
humanTerrarouterTerra
Look up the customer with email deniz@acme.co, find their last 3 invoices, check if any are overdue, and if so draft a polite payment reminder email referencing the specific invoice numbers. Use the available tools in the right order.
Parallel tool orchestration
humanTerrarouterTerra
For our top 10 accounts by MRR: pull current usage metrics, compare against their plan limits, flag any within 15% of a limit, then for each flagged account draft an upsell note customized to their usage pattern and file it as a CRM task as…
writing
3 promptsShort Slack status update
humanLunarouterLuna
Write a 3-line Slack update for #eng: the search reindex finished overnight, latency is back under 200ms, and we're monitoring for 24h before closing the incident.
Product changelog entry
humanLunarouterLuna
Write a changelog entry for: new CSV import with column mapping, 3x faster dashboard loads, and a fix for the timezone bug in scheduled reports. Tone: clear, a little confident, no hype words. Include a 'Why it matters' line per item.
Launch announcement blog post
humanTerrarouterTerra
Write a 900-word launch post for our new usage-based pricing. Cover: why we switched (customers hated seat math), how the migration works (grandfathering for 12 months), a worked cost example for a 20-person team, and an FAQ. Voice: direct,…
Run history
No runs yet. Hit Run on any benchmark above.