Benchmarking Knowledge in Legal Work

The real test of AI is not simply how well it answers isolated questions, but how well it can understand and reason about the information those answers depend on. In legal and enterprise work, knowledge is rarely contained in a single document. It is distributed across thousands of communications, events, and decisions, with relationships that evolve over time.

Fireworks’ SII recognizes that meaningful progress must be measured against the demands of real work. Our contribution focuses on one of its hardest challenges: constructing coherent knowledge from vast collections of fragmented, unstructured information.

What CorporateBench measures

CorporateBench evaluates how well models can reconstruct and reason over organizational knowledge across four synthetic companies, with document collections ranging from 353 to more than 230,000 records. Its underlying temporal knowledge bases provide verifiable ground truth for testing whether models can recover facts, connect evidence across documents, and track relationships as they change over time.

Beyond isolated retrieval

By joining SII, we’re helping push AI evaluation beyond retrieval and question answering toward a more fundamental measure of intelligence: the ability to construct, connect, and reason over knowledge at scale.

For us, this is a central frontier of enterprise AI, and one that better benchmarks can help advance.