Sample task, 10-Q
Blackline, Inc.
Was gross margin at least 80 percent for the quarter ended 30 June 2026?
- Gross profit
- $142,678,000
- Revenue
- $187,822,000
- Gross margin
- 75.96%
Verdict no. The claim misses the filing.
Tasks from
the filing.
Core references trace to cells in the filing.
grade.py scores Core. Verifiers format.
Get the sampleCore references trace to cells in the filing.
Tickmark Bench checks answers against SEC EDGAR bulk XBRL. In the Core set, every reference traces to cells in the filing. Core is scored entirely by grade.py. Numeric answers are scored within the stated tolerance, and claim checks on the verdict alone. The official score is score = passed/149. grade.py's total is the official score.
Tasks ship in the Prime Intellect verifiers format. Built for AI labs and RL environment vendors. An agent can read this page. Core is $299 through Stripe Checkout on Pricing. The grading account is on How it's graded.
Core is 149 verified tasks, the real_v1 batch. The record of the pack is the datasheet.
Four task types
What the pack checks today. A count of each type is not published.
Single-number. One figure, often a computed ratio or margin, from one filing period. Code checks it against a reference value taken from, or computed from, the filed XBRL facts, within a tolerance.
Multi-period and segment. A change across periods, or a figure from a segment. Code checks it against the figure computed from the filed XBRL, within a tolerance.
Claim checks. A yes or no on a statement about the filing. grade.py scores the verdict alone.
Comparisons across companies. The same metric for more than one issuer. Code checks it against reference values taken from, or computed from, the filed XBRL facts, within a tolerance.
Solve rates
Solve rates: not yet published. We publish only measured numbers, with model, attempts and date.
First, the filing.
Every question is generated from bulk XBRL and kept only when the filing is from Q2 2026 or later.
Then the question.
Single-number, multi-period, segment, claim check, or a comparison across companies.
Numbers, in code.
Single-number, multi-period, segment, and cross-company answers are checked against a reference value taken from, or computed from, the filed XBRL facts. A model is not in that loop.
Then the verdict.
A claim check is a yes or no. grade.py scores the verdict alone. The explanation is not scored.
Worked checks, with the references from the sample, are on How it's graded.
Sources, the filing window, and the grading method are on the datasheet.
Who it is for
AI labs. Eval and RL on questions whose references come from filings dated Q2 2026 or later.
RL environment vendors. Tasks in the Prime Intellect verifiers format. grade.py scores the number within the stated tolerance and the claim check on the verdict alone.
Agents. The catalog, the free sample, and the prices are in llms.txt.
Get the pack
Five tasks of each type, in an earlier sample format. They are not part of Core.
Core is 149 verified tasks in the real_v1 batch. Prices are on the pricing page.
Facts an agent can read
The machine-readable catalog is llms.txt. A JSON copy of these facts is in the script with id sec-task-pack.
- Product
- Tickmark Bench
- Page status
- Home page. The Blackline filing in the hero is a real task from the committed sample. Solve rates: not yet published.
- Source
- SEC EDGAR bulk XBRL. Not a product of the SEC.
- Filing rule
- Filings from Q2 2026 or later.
- Solve rates
- Solve rates: not yet published. We publish only measured numbers, with model, attempts and date.
- Datasheet
- datasheet.html
- Task types
- Single-number calculations. Multi-period or segment. Claim checks, with a yes or no verdict. Comparisons across companies.
- Numeric grading
- Code against a reference value taken from, or computed from, the filed XBRL facts, within a tolerance.
- Claim checks
- grade.py scores the verdict alone. The explanation is not scored. The official score is score = passed/149.
- Format
- Prime Intellect verifiers. Not a Prime Intellect product.
- Free sample
- 20 tasks, five of each type, in an earlier sample format. They are not part of Core. The sample page is sample.html.
- Pricing
- pricing.html
- Catalog
- llms.txt
Pricing
Core, and what it costs.