Scoreboard

Split, Classify and Extract: text models compared

How each text model cuts a file into documents, names each one and reads its values, with what it costs.

Its key points, its tables, how it was scored and its log load from /api/public/scoreboards/split-classify-extract.

Test documents

The test sets are not published: Claude wrote their ground truth from the pages, and no person has checked it yet. The documents are public filings: SpaceX's prospectus (37 of its pages), second-quarter earnings release and bond offering press release, and, in the states' case against Paramount, the complaint (31 of its pages), the consent decree (22 of its pages) and the order entering it; and, in Musk v. Altman, the orders on a preliminary injunction and on summary judgment, the jury instructions, the verdict form, the post-trial order and ten trial exhibits, filed as docket entry 601 (the first).

Term	Definition
OCR	Optical character recognition: reading the text, tables and marks of a page from its image or PDF.
engine	An OCR product or model that reads a page and returns its text, such as Mistral OCR, Landing AI or Reducto.
block	One part of a page as an engine returns it: a paragraph, a table, a heading or a figure.
bounding box	The rectangle that marks where something is printed on the page.
table cell	One value of a table, where a row and a column meet.
form field	A label on a form and the value written for it.
grounding	Linking each value of a reading to its bounding box on the page.
ground truth	The correct answers for a page, one fact per value, checkbox, choice or signature. Every reading is scored against them.
test set	The pages a scoreboard is measured on, with their ground truth.
grader	The program or model that compares a reading with the ground truth and counts what is right.