AI analysis governance
How to Keep AI Crosstab Analysis Evidence-Linked, Approvable, and Undoable
Give the model only a bounded, disclosed analytical context; bind every factual cell claim to a stable evidence ID and independently resolve it against the computed table; fail closed when a citation does not match; keep answers read-only; express requested project changes as typed, previewable proposals; require a named human to approve or reject them; recheck the project hash before atomic apply; record the evidence, model, prompt version, actors, decisions, and hashes; and allow exact undo only while it cannot erase later work.
Published by crosstabs.com · Named statistical review pending · Last updated
Run this on your own data — free, no signup
Upload a CSV or XLSX. Everything runs in your browser; your file never leaves your device.
Open the workspace →Treat evidence, decisions, and state transitions as different objects
A polished answer is not evidence, and an evidence-linked answer is not permission to change a project. Keep the computed result, generated prose, proposed operations, human decision, applied state, and recovery record separate so each can be verified.
| Stage | Controlled object | Enforcement | Retained evidence |
|---|---|---|---|
| Bound context | Computed table, filters, weight, wave, table ID | No respondent rows for aggregate analysis | Context hash and disclosed payload |
| Generate | Model receives the bounded analytical context | Read-only output | Provider/model boundary and prompt contract |
| Validate | Resolve every cell pointer against the computed result | Fail closed on invalid link, label, row, or column | Accepted evidence IDs or withheld issues |
| Propose | Convert a requested mutation into allowed typed operations | No direct project write | Evidence-linked before/after diff and dependencies |
| Approve | A named human selects or rejects operations | Human decision required | Reviewer, time, selected and rejected operations |
| Apply | Recompute the approved base hash and apply atomically | Reject a stale approval | Before hash, after hash, actor, time |
| Undo | Restore the recorded pre-apply snapshot | Refuse if later work changed the post-apply state | Undo actor, time, and retained plan history |
Use application-enforced evidence pointers, not decorative citations
A cell pointer should resolve to row and column coordinates plus labels, counts, percentages, expected values, residuals, base, warnings, filters, weight, wave, and table identity under a stable context hash. Validate the claim beside each pointer so one valid citation cannot lend credibility to a different sentence.
NIST's Generative AI Profile recommends reviewing and verifying sources and citations during pre-deployment measurement and ongoing monitoring. For crosstabs, that review can be executable: a bad pointer, wrong label, wrong row, or wrong column causes the model wording to be withheld rather than merely flagged.
Worked example
Worked example
An analyst asks an external agent about a Region × Satisfaction table. The agent gives the public MCP only the aggregate frequency matrix and declared labels. Crosstabs returns the calculated table, statistical warnings, provenance, limitations, and method links; it never receives the respondent rows or stores the request as a project.
If the analyst wants the agent to work with a private project, they use the local MCP instead. The file and project stay on the same device. Adding cuts is a separate typed operation: the analyst previews the change and applies it against the current project revision rather than treating prose as permission to mutate the project.
The resulting project history retains deterministic evidence, revisions, timestamps, and the local audit record. The engine refuses exact undo when later edits would be overwritten; the safe recovery is another reviewed forward change.
How to interpret it
Rule of thumb
Governance is strongest when the model has the least authority needed. Provider failover can keep a request available, but every provider remains behind the same deterministic evidence and approval contract. Model agreement, confidence, fluency, or brand reputation is not a substitute for statistical evidence or a human decision.
Make approval selective, dependency-safe, and stale-state aware
Every proposed operation should name its evidence, risk, affected object, before state, after state, and dependencies. Partial approval is legitimate only when every required dependency is also approved; otherwise the preview must refuse the selection.
Record the approved base hash. At apply time, recompute it. If the project changed after approval, the previous decision does not authorize the new state: generate a fresh preview and obtain a new approval. Apply an accepted set atomically so the project cannot be left half-mutated.
Preserve recourse without overwriting later work
Store a pre-apply snapshot and the exact post-apply integrity hash. Exact undo is allowed only while the current project matches that post-apply hash. If another edit occurred, the engine refuses exact undo because restoring the snapshot would silently destroy work.
This guard implements recourse without pretending every state can be rewound safely. Retain the original plan and its decisions, then use a new evidence-linked forward change when later work exists.
Current Crosstabs product boundary
Crosstabs does not run a hosted model or hosted project service. The deterministic Workspace analyzes files in the browser, and the local MCP keeps respondent files and complete projects on the user's device.
The public MCP exposes exactly five no-account tools for fixed evidence lookup and deterministic analysis of bounded aggregate frequency matrices. It accepts no file paths or respondent-level records and stores no rows, projects, identities, prompts, or generated artifacts.
The conversational model belongs to the agent the user chose, not to Crosstabs. Giving that agent a file is subject to the agent's own data boundary. Use the local MCP when private rows must stay on the device, and treat prose as interpretation rather than computed evidence unless it remains attached to the Crosstabs result.
Measure the control system, not only model quality
Track citation-integrity failures, withheld answers, proposal validation failures, approvals, rejections, partial approvals, stale-state conflicts, applies, undo attempts, refused undos, user corrections, and provider/model versions. Evaluate representative tables and risky edge cases whenever prompts, models, routing, or statistical output contracts change.
NIST's AI RMF makes human oversight, systematic documentation, monitoring, and risk response lifecycle activities rather than a one-time checklist. Its Playbook also recommends histories, audit logs, override statistics, and documented go/no-go decisions.
Frequently asked questions
- What makes an AI crosstab answer evidence-linked?
- Each factual cell claim should point to a stable evidence record containing the table context, row and column coordinates, labels, count, percentages, base, warnings, filters, weight, wave, and table identity. The application—not the model—must resolve that pointer back to the computed result and reject mismatches.
- Should an AI answer be allowed to edit a research project directly?
- No. Read-only analysis and project mutation should be separate capabilities. A requested change should become a typed proposal with a previewable before/after diff, evidence, dependencies, risk, and allowed operation set. A human can approve selected operations, reject the plan, or leave it unapplied.
- How do conflict checks prevent stale AI changes?
- Hash the governed project scope when the proposal is created and again at preview, approval, and apply time. If the project changed after approval, the approved diff is stale and must be previewed and approved again instead of being applied to a different state.
- When is an AI-created change safely undoable?
- An exact snapshot restore is safe only while the current governed project state still matches the recorded post-apply hash. If later work changed that state, exact undo must be refused because it would erase subsequent edits; use a new forward change instead.
- Does using several model providers make an answer more trustworthy?
- No. Provider routing and fallbacks can improve availability, cost, or latency, but they do not validate statistical claims. Evidence resolution, deterministic checks, human review, audit history, and product-specific evaluation remain necessary regardless of which model served the request.
References & further reading
Try it on your own data — free, no signup
Upload a CSV or XLSX. Everything runs in your browser; your file never leaves your device.
Open the workspace →