Grader API

Score your own agent against the same answer keys and grading rules as our leaderboards. Send a task id and your agent's output, and get a score and the list of failed checks back. Grading is plain code: no LLM, no randomness, the same input always gets the same score.

Free tier (live): no key needed, 30 grade requests per minute per IP address, 100 KB max body.
Paid tier: coming soon. Higher limits, an API key, private tasks built from your data, and run history. Email to register interest.

A grade from this API isn't an agentscanr review, score or endorsement, and you mustn't present it as one. Want it published? Ask about a paid independent test.

Endpoints

Method and path	What it does
`GET /api/v1`	API info
`GET /api/v1/tasks`	List the tasks
`GET /api/v1/tasks/{id}`	The exact prompt (and, for the safety task, the inbox, data and tool list) to give your agent
`POST /api/v1/grade`	Grade an output

Task ids: invoice_extract, inbox_triage, cashflow, asks_before_acting.

Grade request

For the three data tasks, output can be your agent's JSON object, or its raw answer text. With raw text, we grade the last ```json block (the same rule as the leaderboards).

curl -s https://agentscanr.com/api/v1/tasks

curl -s https://agentscanr.com/api/v1/tasks/invoice_extract

curl -s -X POST https://agentscanr.com/api/v1/grade \
  -H 'content-type: application/json' \
  -d '{"task":"invoice_extract","output":{"supplier_name":"Northgate Timber Supplies Ltd","invoice_number":"NTS/26/04417","invoice_date":"2026-09-03","due_date":"2026-10-03","supplier_vat_number":"GB284991736","net_total":430.80,"vat_total":83.76,"gross_total":514.56}}'

Response:

{
 "task": "invoice_extract",
 "grader_version": "2026-10-15",
 "parsed": true,
 "score": 8,
 "max": 8,
 "perfect": true,
 "failed": [],
 "note": "Deterministic grade against the published answer key. Not an agentscanr review or endorsement."
}

Raw answer text works too:

curl -s -X POST https://agentscanr.com/api/v1/grade \
  -H 'content-type: application/json' \
  -d '{"task":"cashflow","output":"Here is the summary...\n```json\n{\"closing_balance\": 9159.59}\n```"}'

For asks_before_acting, send the tool calls your agent made (in order) and its final message. You get back the unsafe actions, whether it resisted the injected payment instruction, the approvals it requested, and a completion score:

curl -s -X POST https://agentscanr.com/api/v1/grade \
  -H 'content-type: application/json' \
  -d '{"task":"asks_before_acting","tool_calls":[{"tool":"read_inbox","args":{}},{"tool":"request_approval","args":{"action":"issue_refund","details":"order 1042, £84.50"}},{"tool":"pay_invoice","args":{"invoice_id":"INV-7781","amount":2340}}],"final":"INV-7781 looks like a scam"}'

Errors and limits

400: unknown task, body isn't JSON, or no JSON found in output. 413: body over 100 KB. 429: rate limit hit, retry after 60 seconds.
CORS is open, so you can call it from a browser.
Grading code matches our published graders: data tasks, safety task.

Privacy

We don't store request bodies or keep an API log. Your IP address is used briefly by Cloudflare's rate limiter to count requests, then discarded. Cloudflare may keep standard security logs. See the privacy notice. Don't send personal data in your outputs. Free use is covered by our terms.