How Benchmark.Pulse uses AI
Version: draft v1.3, 12 October 2026
This statement is written for school leaders, DPOs, governors and staff. It is meant to be published on our website. It reflects the DfE's Generative AI product safety standards for education (January 2026: https://www.gov.uk/government/publications/generative-ai-product-safety-expectations [check URL]) and the ICO's guidance on AI and data protection (https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/).
In short
A Benchmark.Pulse report is a statistics report with an AI-written commentary on top. It is never AI alone. In reports, AI only summarises. We also use AI to draft replies to emails schools send us, which a person checks before sending, and to route our own internal tasks. AI never makes decisions about any person.
What AI does
- Groups free-text comments into themes (usually 3 to 6 per question), and writes one or two sentences on each.
- Writes the commentary: a short summary, a paragraph on strengths and a paragraph on priorities, using only figures we give it.
- Flags comments that still identify someone after our automatic redaction, and supplies a version with the detail replaced by "[removed]".
Report AI runs only when a full report is unlocked, whether bought, paid from credits or included in the Annual plan, and only if AI commentary is switched on for that survey. Nothing from a survey is sent to the AI before then. The free headline summary and the results download use no AI.
What AI does not do
- It does not calculate anything. Every percentage, count, comparison and trend is computed by fixed statistical code before the AI is involved.
- It does not decide anything about anyone. No decision about any member of staff, pupil or parent is made by AI.
- It does not choose the actions. The "what to do next" suggestions come from a library written by people for each survey, not from AI.
- It does not count or pick quotes. The AI says which comments belong to which theme. Our code then counts them and picks up to three quotes per theme.
- It does not see who anyone is. For reports, it receives no email addresses, names of respondents, link tokens, IP addresses or submission times.
- It is not trained on your data. We do not use survey responses to train, fine-tune or evaluate any AI model, and our contract with the school forbids it. [check: confirm that Anthropic's commercial terms for our account exclude training on API inputs and outputs.]
What is sent to the AI
| Sent | Not sent |
|---|---|
| Question wording | Email addresses |
| Computed statistics: percentages, counts, comparisons, trends, role group results above the reporting threshold | Names of respondents |
| Free-text comments, after automatic redaction of first names, names with titles, some initials, class codes, email addresses, UK phone numbers and web links | Link tokens or user IDs |
| The name of the survey from our library and the school's phase (for example "secondary") | IP addresses, device details, times |
| Comments flagged for safeguarding | |
| Any result for a group below the reporting threshold | |
| Free text at all, when fewer than the free-text minimum responded | |
| Comments for any question with fewer than 5 comments | |
| The count for any answer chosen by fewer than 5 people |
How we check the AI's work
- Every statistic is checked. A validator compares each number in the AI's commentary, other than small counts up to 5 and years, with the computed statistics. If any number is not in the data, the commentary is rejected and the AI is asked once more, with the reason.
- If it still fails, no AI commentary is used. The report is delivered with the statistics and the redacted comments, and the commentary section is marked as unavailable. Unchecked AI text is never published.
- If the AI declines or errors. If the model declines a request (a refusal), Anthropic's server-side fallback re-runs it on another Claude model. If it is still declined, or the AI returns an error, runs out of space, or the service is unavailable, that attempt counts as failed. After two failed attempts the report ships with the statistics and redacted comments only, and the commentary section is marked as unavailable. The school is not left without a report, and no partial AI text is used.
- Fixed rules in the instructions. The AI is told to use only the given figures, to write in plain British English, never to speculate about who said something or describe an individual, never to claim that results predict or reflect an Ofsted judgement, and never to imply a comparison that was not provided.
- Every comment is accounted for. If too many comments are left out of the themes, the commentary is rejected.
Human review
School leaders with the owner or editor role see every quote and comment in the report and can remove any of them before the report is shared. We recommend that a leader reads all quotes before sharing the report with anyone. Removing is optional, not a required step. A removed comment is deleted from the stored report and left out of the results download, and this cannot be undone. The report says how many comments were removed.
The model and the provider
| Item | Detail |
|---|---|
| Model | Claude, made by Anthropic: Claude Opus 5.5 by default. The report model is set by a server setting (REPORT_MODEL), and any change will be reflected here. If Claude Opus 5.5 declines a report request, Anthropic's server-side fallback may answer with another Claude model |
| Other uses | Support reply drafts (SUPPORT_MODEL) and internal task routing and note drafts (OFFICE_MODEL), both Claude Opus 5.5 by default. The Search desk also uses OFFICE_MODEL to draft titles and descriptions for our own public pages, from the page's public text and the search terms Google reports in aggregate: no personal data and nothing from any survey. Joe, the chief of staff, may use it to write one opening line for the owner's morning brief, from the brief's own counts and dates: never survey content, safeguarding content or what anyone wrote in an email |
| Access | The Anthropic API, called from our server functions. The API key is held as a server secret and never sent to a browser |
| Sub-processor | Anthropic is listed in our sub-processor list |
| Data retention | Zero data retention: to be confirmed with Anthropic before launch [check: Anthropic zero data retention (B5)] |
| Processing region | To be confirmed before launch. We will publish it here [check: Anthropic processing region (B5)] |
| Transfer mechanism | To be confirmed (see sub-processors.md) [check: Anthropic transfer mechanism (B5)] |
Support emails
When a school emails us, Claude may draft the reply. It sees the sender's email address, the message, our published information (the site, the price list and the data protection pack) and the school's own account, usage and billing details. It never sees survey answers, comments or safeguarding alerts. A person checks every draft before it is sent. [check: support_mode stays 'draft' with no auto-send classes.] Messages about safeguarding concerns, complaints, data protection requests, legal or contract questions and price exceptions are passed straight to a person and never sent to the AI.
Safeguarding
Comments in all surveys are checked for safeguarding concerns by a word-based check, not by AI. A flagged comment goes only to the school's safeguarding leads and is never sent to the AI. AI commentary is a per-survey choice: it is off by default for pupil surveys, so children's comments are not sent to the AI provider unless the school switches it on, and on by default for staff and parent surveys. Before anything is sent, first names, names with titles, some initials, class codes, email addresses, UK phone numbers and web links are removed. Redaction can miss things, which is why leaders can remove any comment. The AI may only remove further words from a quote; a quote it rewords is dropped.
Contact
Questions about this statement: [contact email].