# AI Bias Narrative Auditor

> Document risk, bias, and transparency for Algorithmic Impact Assessment (AIA). Critique Human-in-the-loop descriptions against EU AI Act Art. 14 and Canadian Directive thresholds; verify dataset provenance and copyright/privacy compliance.



Tags: EU AI Act, Algorithmic Impact Assessment, Compliance, Bias, Human oversight, Canadian Directive, Data governance


## Example Prompts

- Audit our human-in-the-loop section for EU AI Act Art. 14
- Check that our AIA documents training data sources and copyright
- Critique the human oversight narrative for Canadian Directive compliance
- Run dataset provenance check on our technical documentation

URL: https://rakenne.app/skills/ai-bias-narrative-auditor/index.md

Try this skill: https://rakenne.app/a/?skill=ai-bias-narrative-auditor



## Overview

Document risk, bias, and transparency for Algorithmic Impact Assessment (AIA). Critique Human-in-the-loop descriptions against EU AI Act Art. 14 and Canadian Directive thresholds; verify dataset provenance and copyright/privacy compliance.

## Getting Started

Add your source materials to the project workspace (e.g. technical documentation, Instructions for Use, or an AIA draft), then activate the *AI Bias Narrative Auditor* skill. The agent will scope the document, run the validation tools on the relevant sections, and help you fix gaps until the narrative meets the required thresholds.

## Example Conversation

> **You:** We're preparing for EU AI Act conformity. Please audit the human oversight section in `docs/instructions-for-use.md` and the data governance section in `docs/technical-file.md`.

> **Agent:** I'll run both tools. First, the human-in-the-loop narrative audit on the Instructions for Use...

> **Agent:** Here are the results of `hitl_narrative_audit` on `docs/instructions-for-use.md`:
> - **Summary:** 2 ERROR(s), 1 WARNING
> - [ERROR] [Art. 14(4)(b)] Awareness of automation bias: not found or insufficient.
> - [ERROR] [Art. 14(4)(e)] Ability to intervene or interrupt (e.g. stop mechanism): not found or insufficient.
> - [WARNING] [Canadian Directive] Meaningful human review: not found or insufficient.
>
> I recommend adding a short subsection on automation bias (e.g. training for deployers, checklists) and explicitly describing the "stop" or override control. I'll run the dataset provenance check next.

> **Agent:** `dataset_provenance_check` on `docs/technical-file.md` found:
> - 0 ERROR(s), 1 WARNING — Retention or deletion policy for data: not found. I've added a retention subsection to the data governance chapter; you can adjust the periods to match your policy.

## What the Tools Validate

The skill includes two validation tools that run against your documentation:

**`dataset_provenance_check`** verifies that training and data sources are documented and compliant:
- **Training/data source** listing or description (EU AI Act Art. 10(2) — origin and acquisition)
- **Copyright or licensing** of data sources (rights, attribution, clearance for training)
- **Privacy / lawful basis** for personal data (GDPR Art. 6)
- **Retention or deletion** policy (WARNING if missing)
- **Anonymisation or pseudonymisation** where personal data is used (WARNING; Art. 10(5) when special categories are processed)
- **Special categories** (Art. 9 GDPR) — INFO if processed without explicit safeguards
- Unfilled placeholders (`[TODO]`, `[INSERT]`, `[SPECIFY]`) reported as INFO

**`hitl_narrative_audit`** critiques the human oversight / Human-in-the-loop description:
- **Oversight modality** (HITL, HOTL, or HIC) — Art. 14
- **Understand capabilities and limitations** — Art. 14(4)(a)
- **Awareness of automation bias** — Art. 14(4)(b)
- **Ability to interpret the AI output correctly** — Art. 14(4)(c)
- **Ability to disregard, override, or reverse the output** — Art. 14(4)(d)
- **Ability to intervene or interrupt** (e.g. stop mechanism) — Art. 14(4)(e)
- **Meaningful human review** (Canadian Directive) — WARNING
- **Explanation and recourse** (Canadian Directive) — WARNING

Address all ERRORs before finalising; resolve or document WARNINGs as appropriate for your jurisdiction.

## Output Excerpt

After edits, a compliant human oversight section might read like this (the tools would report no errors for Art. 14(4)):

```markdown
## Human oversight (Art. 14)

**Modality:** Human-in-the-loop (HITL). Each recommendation is reviewed and approved by a qualified officer before any decision is applied.

**Capabilities and limitations (Art. 14(4)(a)).** Deployers receive training on the system’s intended purpose, capabilities, and known limitations; these are also set out in this document and in the user interface.

**Automation bias (Art. 14(4)(b)).** Training materials and checklists remind operators to treat the system’s output as advisory and to apply independent judgment, especially when the outcome affects individuals.

**Interpretation of output (Art. 14(4)(c)).** The interface displays confidence scores and explanatory text; deployers are trained to interpret these and to recognise edge cases.

**Override and reversal (Art. 14(4)(d)).** The deployer may disregard or override any recommendation and record the reason in the system.

**Intervention and stop (Art. 14(4)(e)).** Each screen includes a "Stop / Do not use this output" control that immediately halts use of the recommendation and logs the event for review.
```

The dataset provenance section would similarly document sources, licensing, lawful basis, and retention so that `dataset_provenance_check` returns zero ERRORs.


---

Back to [Skill Library](https://rakenne.app/skills/index.md)