Researcher · co-author
Benjamin Davis
Co-author of the CrucibleBench technical report and maintainer of the project's implementation and public research materials through Folcright LLC.
About
A different failure surface
Most agent evals measure one of three things: knowledge, task completion, or tool use. Some newer benchmarks measure social intelligence or deception in bounded settings. CrucibleBench exists to measure something else: whether an agent can remain coherent, adaptive, and trustworthy when relationships persist, feedback accumulates, and actions change the future state of the world.
The instrument is deliberately old technology: a battle-tested MUD substrate repurposed into a persistent social benchmark for frontier AI agents.
How we work
Open methods, private evaluations
What is public
- Benchmark methodology and scoring rubrics
- Full results from validation runs, all 650 transcripts
- Statistical framework and confidence intervals
- Known limitations and failure modes
- Research paper with all supporting data and the billing export
- The itemized Phase 2 planning budget, with actuals published as spent
What is private / paid
- Evaluations of your models and agents
- Custom scenario design for your use case
- Deployment diagnostics and failure analysis
- Regression testing across model versions
- Confidential results and detailed transcripts
Background
An independent applied research effort
CrucibleBench is built by Folcright LLC, an independent research effort focused on behavioral evaluation for AI agents operating in persistent social environments.
Nintendo's "lateral thinking with withered technology" describes using mature, inexpensive, well-understood technology in a new way. CrucibleBench applies that idea to AI evaluation: the old MUD substrate is useful precisely because it is text-native, constrained, replayable, and inspectable.
The work draws on experience in AI test & evaluation, adversarial scenario design, simulation design, and product deployment. We publish benchmark methods, results, and limitations openly, and use the same framework for private research-stage evaluations with teams shipping agentic systems.
Researchers
The people named on the work
CrucibleBench's Phase 1 report is authored by Benjamin Davis and Philip Mims. The descriptions below stay within what the published paper and project history support; additional profile information will be linked when the authors publish it.
Researcher · co-author
Philip Mims
Co-author of the published Phase 1 report. The project does not currently publish a verified individual profile, so the paper and citation record are the authoritative links.
Contact
Get involved in Phase 2
Phase 2 is in active build. The instrument-validation direction is defined; environment completion, calibration, preregistration, and final allocation remain in progress. Fund the provisional envelope, help build the instrument, or join the post-calibration pilot cohort.