Responsible AI Governance and Assurance
Learn to govern AI uses responsibly, from classifying a use before you assess it to checking fairness and oversight. No coding: you work on the data of Raket Parts, a fictional maker of rocket parts.
You will be able to
- record an AI use and classify it before you assess it
- rate failure and misuse scenarios and set stop rules
- compare outcomes by group and read small groups with care
- design a contest route that can change the outcome
What kind of AI use is this?
Record an ai use in a register and classify it in the right order before you assess it.
Who could be harmed?
Describe failure and misuse scenarios, rate them, choose proportionate controls and write stop rules someone else could test.
Fair for every group?
Compare flag rates by group against the highest rate, read small groups with care and spot columns that stand in for a protected trait.
Can people understand it and push back?
Say what affected people and staff must be told about an ai-supported decision and design a contest route that can change the outcome.
Bought in, still our responsibility
Decide what evidence a bought-in ai feature needs, set contract conditions that close the gaps and recommend accept, accept with conditions or refuse.
Human oversight of AI in use
Make sure people can check and stop an AI system in daily use.
Mid-Path project: an AI incident
Handle an incident, put things right and reassess the use.
Final assessment
AI governance decisions on an unseen case.
Responsible AI / Module 1
What kind of AI use is this?
Noor is building a register of proposed AI uses at Raket Parts. Learn to classify the exact use before assessing its risk.
Sections
Responsible AI / Module 1 / Free practice
Try it on your own question
Run it in the practice editor
The practice editors have the Raket Parts tables loaded. Type a query or a formula, run it and change it as often as you like. Nothing is marked.
Responsible AI / Module 2
Who could be harmed?
The credit-limit helper is said to be 94% accurate. Noor asks what the other 6% means, and for whom: a wrong cut for one university is not the same as one for a rocket builder.
This module builds on module 1. If you are new to this, start with module 1.
Sections
Responsible AI / Module 2 / Free practice
Try it on your own question
Run it in the practice editor
The practice editors have the Raket Parts tables loaded. Type a query or a formula, run it and change it as often as you like. Nothing is marked.
Responsible AI / Module 3
Fair for every group?
A practice rule sends large invoices for a manual credit check. Overall it looks modest. Jelmer computes the rate for each segment and country, and finds where the checks land.
This module builds on module 2. If you are new to this, start with module 2.
Sections
Responsible AI / Module 3 / Free practice
Try it on your own question
Run it in the practice editor
The practice editors have the Raket Parts tables loaded. Type a query or a formula, run it and change it as often as you like. Nothing is marked.
Responsible AI / Module 4
Can people understand it and push back?
The credit-limit helper will suggest lower limits for some customers. Noor asks what a customer is told, what credit staff are told, and what a customer can do if the suggestion is wrong.
This module builds on module 3. If you are new to this, start with module 3.
Sections
Responsible AI / Module 4 / Free practice
Try it on your own question
Run it in the practice editor
The practice editors have the Raket Parts tables loaded. Type a query or a formula, run it and change it as often as you like. Nothing is marked.
Responsible AI / Module 5
Bought in, still our responsibility
The accounts payable supplier adds an AI invoice-matching feature, and a second supplier offers a tool that ranks customers for credit review. Noor decides what evidence each needs before Raket Parts relies on it.
This module builds on module 4. If you are new to this, start with module 4.
Sections
Responsible AI / Module 5 / Free practice
Try it on your own question
Run it in the practice editor
The practice editors have the Raket Parts tables loaded. Type a query or a formula, run it and change it as often as you like. Nothing is marked.
Responsible AI / Module 1 / 1.1
1.1 A register of AI uses
Section 1.1 complete
Next: 1.2 Is it AI and is it forbidden?
Responsible AI / Module 1 / 1.2
1.2 Is it AI and is it forbidden?
Section 1.2 complete
Next: 1.3 What it decides about people
Responsible AI / Module 1 / 1.3
1.3 What it decides about people
Section 1.3 complete
Next: 1.4 Provider or deployer
Responsible AI / Module 1 / 1.4
1.4 Provider or deployer
Section 1.4 complete
Next: Module 1 check
Continue to the module check Recap this section Back to module 1
Responsible AI / Module 1 / Check
Module 1 check
Module 1 complete
Next: Module 2, Who could be harmed?
Continue to module 2 Try it on your own question Recap module 1 Back to module 1
Responsible AI / Module 2 / 2.2
2.2 Scenarios
Section 2.2 complete
Next: 2.3 Controls in proportion
Responsible AI / Module 2 / 2.4
2.4 Stop rules
Section 2.4 complete
Next: Module 2 check
Continue to the module check Recap this section Back to module 2
Responsible AI / Module 2 / Check
Module 2 check
Module 2 complete
Next: Module 3, Fair for every group?
Continue to module 3 Try it on your own question Recap module 2 Back to module 2
Responsible AI / Module 3 / 3.1
3.1 Overall hides groups
Section 3.1 complete
Next: 3.2 Ratios to the highest
Responsible AI / Module 3 / 3.4
3.4 Stand-in columns
Section 3.4 complete
Next: Module 3 check
Continue to the module check Recap this section Back to module 3
Responsible AI / Module 3 / Check
Module 3 check
Module 3 complete
Next: Module 4, Can people understand it and push back?
Continue to module 4 Try it on your own question Recap module 3 Back to module 3
Responsible AI / Module 4 / 4.1
4.1 Accurate is not the same as usable
Section 4.1 complete
Next: 4.2 Who must be told what
Responsible AI / Module 4 / 4.2
4.2 Who must be told what
Section 4.2 complete
Next: 4.3 Explanations staff can act on
Responsible AI / Module 4 / 4.3
4.3 Explanations staff can act on
Section 4.3 complete
Next: 4.4 A route that changes outcomes
Responsible AI / Module 4 / 4.4
4.4 A route that changes outcomes
Section 4.4 complete
Next: Module 4 check
Continue to the module check Recap this section Back to module 4
Responsible AI / Module 4 / Check
Module 4 check
Module 4 complete
Next: Module 5, Bought in, still our responsibility
Continue to module 5 Try it on your own question Recap module 4 Back to module 4
Responsible AI / Module 5 / 5.1
5.1 A good demonstration is not enough
Section 5.1 complete
Next: 5.2 Evidence for this risk and this role
Responsible AI / Module 5 / 5.2
5.2 Evidence for this risk and this role
Section 5.2 complete
Next: 5.3 Test the claim on our records
Responsible AI / Module 5 / 5.3
5.3 Test the claim on our records
Section 5.3 complete
Next: 5.4 Conditions that close gaps
Responsible AI / Module 5 / 5.4
5.4 Conditions that close gaps
Section 5.4 complete
Next: Module 5 check
Continue to the module check Recap this section Back to module 5
Responsible AI / Module 5 / Check
Module 5 check
Module 5 complete
Next: Module 6, coming later
Back to the Path Try it on your own question Recap module 5 Back to module 5
Responsible AI / Module 1 / Recap
Recap card
1.1 A register of AI uses
A register of AI uses
- A register entry names the use, purpose, who it affects and an owner.
- Raket Parts data holds no AI use; the three uses are practice.
Common mistake
The mistake. Rating risk before the use is written down.
Responsible AI / Module 1 / Recap
Recap card
1.2 Is it AI and is it forbidden?
Is it AI and is it forbidden?
- First: is it AI or a fixed rule? Then: is it on the banned list?
- Hiring is a high-risk area, not banned outright.
Common mistake
The mistake. Treating every AI use in hiring as banned.
Responsible AI / Module 1 / Recap
Recap card
1.3 What it decides about people
What it decides about people
- Decisions about people's jobs or access to essential services are high-risk areas.
- Customers are organisations: the credit helper is an open legal question.
Common mistake
The mistake. Settling a legal classification without the legal adviser.
Responsible AI / Module 1 / Recap
Recap card
1.4 Provider or deployer
Provider or deployer
- Provider builds or sells; deployer uses under its authority.
- Buying a tool keeps the deployer's duties.
Common mistake
The mistake. Assuming the supplier carries every duty.
Responsible AI / Module 1 / Recap
Recap card
Module 1: What kind of AI use is this?
A register of AI uses
- A register entry names the use, purpose, who it affects and an owner.
- Raket Parts data holds no AI use; the three uses are practice.
Common mistake
The mistake. Rating risk before the use is written down.
Is it AI and is it forbidden?
- First: is it AI or a fixed rule? Then: is it on the banned list?
- Hiring is a high-risk area, not banned outright.
Common mistake
The mistake. Treating every AI use in hiring as banned.
What it decides about people
- Decisions about people's jobs or access to essential services are high-risk areas.
- Customers are organisations: the credit helper is an open legal question.
Common mistake
The mistake. Settling a legal classification without the legal adviser.
Provider or deployer
- Provider builds or sells; deployer uses under its authority.
- Buying a tool keeps the deployer's duties.
Common mistake
The mistake. Assuming the supplier carries every duty.
Responsible AI / Module 2 / Recap
Recap card
2.1 Accuracy is not the whole risk
Accuracy is not the whole risk
- Accuracy says how often; harm says how badly, and to whom.
- 6 rocket builders hold 72.8% of sales.
Common mistake
The mistake. Signing off on an accuracy figure alone.
Responsible AI / Module 2 / Recap
Recap card
2.2 Scenarios
Scenarios
- Failure, misuse, over-reliance: name who is affected and who would notice.
- Over-reliance needs a record to be seen.
Common mistake
The mistake. Listing only the failure scenario.
Responsible AI / Module 2 / Recap
Recap card
2.3 Controls in proportion
Controls in proportion
- Match the control to the rating.
- Too much control is a cost too.
Common mistake
The mistake. One control for every case.
Responsible AI / Module 2 / Recap
Recap card
2.4 Stop rules
Stop rules
- Measure, threshold, data source, who acts.
- Practice thresholds are labelled.
Common mistake
The mistake. Writing "stop if it causes problems".
Responsible AI / Module 2 / Recap
Recap card
Module 2: Who could be harmed?
Accuracy is not the whole risk
- Accuracy says how often; harm says how badly, and to whom.
- 6 rocket builders hold 72.8% of sales.
Common mistake
The mistake. Signing off on an accuracy figure alone.
Scenarios
- Failure, misuse, over-reliance: name who is affected and who would notice.
- Over-reliance needs a record to be seen.
Common mistake
The mistake. Listing only the failure scenario.
Controls in proportion
- Match the control to the rating.
- Too much control is a cost too.
Common mistake
The mistake. One control for every case.
Stop rules
- Measure, threshold, data source, who acts.
- Practice thresholds are labelled.
Common mistake
The mistake. Writing "stop if it causes problems".
Responsible AI / Module 3 / Recap
Recap card
3.1 Overall hides groups
Overall hides groups
- Report each group's rate next to the overall.
- 6.84% overall, 21.02% for rocket builders.
Common mistake
The mistake. Reporting only the overall rate.
Responsible AI / Module 3 / Recap
Recap card
3.2 Ratios to the highest
Ratios to the highest
- Ratio: a group's rate divided by the highest rate.
- A gap starts a review; it is not proof.
Common mistake
The mistake. Calling a rule unfair from a gap alone.
Responsible AI / Module 3 / Recap
Recap card
3.3 Small groups
Small groups
- Give counts with rates.
- X-CUS-0001 holds 480 of 801 rocket-builder flags.
Common mistake
The mistake. Ranking small groups on rate alone.
Responsible AI / Module 3 / Recap
Recap card
3.4 Stand-in columns
Stand-in columns
- A stand-in carries a group's information under another name.
- Invoice size stands in for segment here.
Common mistake
The mistake. Thinking a rule is neutral because it leaves a column out.
Responsible AI / Module 3 / Recap
Recap card
Module 3: Fair for every group?
Overall hides groups
- Report each group's rate next to the overall.
- 6.84% overall, 21.02% for rocket builders.
Common mistake
The mistake. Reporting only the overall rate.
Ratios to the highest
- Ratio: a group's rate divided by the highest rate.
- A gap starts a review; it is not proof.
Common mistake
The mistake. Calling a rule unfair from a gap alone.
Small groups
- Give counts with rates.
- X-CUS-0001 holds 480 of 801 rocket-builder flags.
Common mistake
The mistake. Ranking small groups on rate alone.
Stand-in columns
- A stand-in carries a group's information under another name.
- Invoice size stands in for segment here.
Common mistake
The mistake. Thinking a rule is neutral because it leaves a column out.
Responsible AI / Module 4 / Recap
Recap card
4.1 Accurate is not the same as usable
Accurate is not the same as usable
- A usable notice: what happened, why, what to do, by when.
- Publishing a document is not telling the customer.
Common mistake
The mistake. Sending a correct but technical explanation.
Responsible AI / Module 4 / Recap
Recap card
4.2 Who must be told what
Who must be told what
- Customer, controller and managing director need different things.
- Label each item duty, policy or good practice; legal questions to the legal adviser.
Common mistake
The mistake. One message for every audience.
Responsible AI / Module 4 / Recap
Recap card
4.3 Explanations staff can act on
Explanations staff can act on
- Uses, limits, when not to rely, how to override.
- Overrides go back to the owner.
Common mistake
The mistake. A staff guide that only gives accuracy.
Responsible AI / Module 4 / Recap
Recap card
4.4 A route that changes outcomes
A route that changes outcomes
- 0 of 30 late-delivery cases record an outcome.
- A contest must reach someone who can change the result.
Common mistake
The mistake. Routing a contest back to the same queue.
Responsible AI / Module 4 / Recap
Recap card
Module 4: Can people understand it and push back?
Accurate is not the same as usable
- A usable notice: what happened, why, what to do, by when.
- Publishing a document is not telling the customer.
Common mistake
The mistake. Sending a correct but technical explanation.
Who must be told what
- Customer, controller and managing director need different things.
- Label each item duty, policy or good practice; legal questions to the legal adviser.
Common mistake
The mistake. One message for every audience.
Explanations staff can act on
- Uses, limits, when not to rely, how to override.
- Overrides go back to the owner.
Common mistake
The mistake. A staff guide that only gives accuracy.
A route that changes outcomes
- 0 of 30 late-delivery cases record an outcome.
- A contest must reach someone who can change the result.
Common mistake
The mistake. Routing a contest back to the same queue.
Responsible AI / Module 5 / Recap
Recap card
5.1 A good demonstration is not enough
A good demonstration is not enough
- A demo shows the supplier's case, not ours.
- 79 of our supplier invoices are not in euros.
Common mistake
The mistake. Treating a demo or certificate as evidence for our use.
Responsible AI / Module 5 / Recap
Recap card
5.2 Evidence for this risk and this role
Evidence for this risk and this role
- Evidence follows the risk to people.
- Do not over-control a low-risk feature.
Common mistake
The mistake. One evidence list for every feature.
Responsible AI / Module 5 / Recap
Recap card
5.3 Test the claim on our records
Test the claim on our records
- "Every" needs known cases and a count of misses.
- Our records hold one caught duplicate, by today's rule.
Common mistake
The mistake. Taking one example as proof of "every".
Responsible AI / Module 5 / Recap
Recap card
5.4 Conditions that close gaps
Conditions that close gaps
- Each condition closes a named gap.
- Recommend, and name who decides.
Common mistake
The mistake. Conditions that close no gap.
Responsible AI / Module 5 / Recap
Recap card
Module 5: Bought in, still our responsibility
A good demonstration is not enough
- A demo shows the supplier's case, not ours.
- 79 of our supplier invoices are not in euros.
Common mistake
The mistake. Treating a demo or certificate as evidence for our use.
Evidence for this risk and this role
- Evidence follows the risk to people.
- Do not over-control a low-risk feature.
Common mistake
The mistake. One evidence list for every feature.
Test the claim on our records
- "Every" needs known cases and a count of misses.
- Our records hold one caught duplicate, by today's rule.
Common mistake
The mistake. Taking one example as proof of "every".
Conditions that close gaps
- Each condition closes a named gap.
- Recommend, and name who decides.
Common mistake
The mistake. Conditions that close no gap.