DAYJOB: A Benchmark for Long-Horizon Professional Work
Quick summary
arXiv:2610.01306v1 Announce Type: new Abstract: Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds. We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80). The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance. Each task is a containerized Harbor environment with an expert rubric of binary criteria (median 47.5 and 57.5 per task) that an agentic judge applies to the delivered
Key takeaways
- arXiv:2610.01306v1 Announce Type: new Abstract: Professional work often starts with a brief request that leaves the professional to work out what is needed, which documents matter, and whether the request's premise holds.
- We introduce DAYJOB, a benchmark of 130 tasks built by professionals in healthcare (50) and finance (80).
- The tasks are estimated to take a professional 13.6 hours on average in healthcare and 16.6 in finance.
Why it matters
“DAYJOB: A Benchmark for Long-Horizon Professional Work” highlights the need for repeatable measurement rather than a single impressive demonstration. Independent validation across datasets and clearly stated limitations determine whether a result can guide product decisions.
Member comments