Lesson 1

When an AI system needs an impact assessment

Video
2 min
Reading
10 min
Exercise
7 min
Total
19 min

Educational content, not legal advice.

In this lesson you will

  1. Tell which rules ask for an impact assessment of a given use of an AI system.
  2. Explain who carries each duty: the department, the Quebec enterprise or the EU deployer.
  3. Use the NIST AI Risk Management Framework as a voluntary method under those duties.

Video lesson

Length: 1:19
Read the transcript

This is lesson one of AI Risk and Impact Assessment in practice. We follow Quillmere Decision Systems, our fictional software company, and its claims triage model, which three customers want to use.

A federal department wants it to sort benefit applications. The Directive on Automated Decision-Making covers systems that make or assist administrative decisions about clients TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1, and the duty sits with the department, not with Quillmere.

A Quebec insurer wants to decide small claims automatically. The project needs a privacy impact assessment P-39.1 s.3.3, s.17, and a decision made by the model alone brings the automated decision rule. P-39.1 s.12.1

A life insurer in Europe wants it for pricing. From December 2, 2027, that use calls for a fundamental rights impact assessment by the insurer before first use AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a) Regulation (EU) 2024/1689 Art. 2(2), Art. 111(4), Art. 113, as amended by Regulation (EU) 2026/1744, and often a data protection impact assessment too. GDPR Art. 35(1), (3)(a)

The NIST framework is voluntary NIST AI RMF 1.0 (NIST AI 100-1), so we use it as a method, not a duty.

Recap: one system, three assessments, three owners, one shared description. Next: describing the system.

Notes

One system, several rulebooks

An AI system rarely meets only one set of rules. The same model can sort applications for a government, make decisions for a business in Quebec and set prices for an insurer in Europe, and each of those uses brings its own assessment duty. The first job is not to fill in a form. It is to work out which assessments the uses call for, and who has to do them.

Meet Quillmere and its claims triage model

Quillmere Decision Systems Inc. is a fictional Montreal software company with one product: a model that reads a claim or an application and sorts it into fast track, standard review or detailed review. Three customers want it this year:

  • a federal department, to sort applications for a benefit program;
  • a Quebec insurer, to decide small claims automatically;
  • a life insurer in the European Union, to help price policies.

Quillmere will be asked for help by all three. This course follows its privacy and AI governance lead through one assessment, from screening to monitoring.

The federal Directive: the department's duty

The Treasury Board Directive on Automated Decision-Making applies to institutions subject to the Policy on Service and Digital, and to any automated decision system in production used to make an administrative decision, or a related assessment, about a client. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 An administrative decision is one made by an authorized official under powers from an Act of Parliament or a prerogative order, and it affects legal rights, privileges or interests. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 Systems used only for research and experimentation, or in test environments, are excluded. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1

Sorting benefit applications is an assessment related to an administrative decision, so the department's use is covered. The duty sits with the department, not with Quillmere. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 In practice the department will ask Quillmere for most of what the assessment needs: how the model works, what data trained it, how it was tested.

The current version of the Directive was modified on June 24, 2025, and systems built or bought before that date had until June 24, 2026 to meet the new requirements. TBS Directive on Automated Decision-Making ss.1.1, 1.2.1, 1.3 The Directive is reviewed every two years. TBS Directive on Automated Decision-Making ss.1.1, 1.2.1, 1.3

Quebec: the project assessment and the automated decision rule

In Quebec, a privacy impact assessment is required for any project to acquire, develop or overhaul an information system or electronic service delivery system involving personal information. P-39.1 s.3.3, s.17 The person in charge of the protection of personal information is consulted from the outset of the project. P-39.1 s.3.3 Sensitivity sets the depth of the assessment, not whether one is needed. P-39.1 s.3.3, s.17

When the Quebec insurer approves a claim on the model's output alone, a second rule applies. For a decision based exclusively on automated processing of personal information, the enterprise informs the person, on request explains the information used and the principal factors and parameters, and gives the person an opportunity to submit observations to a staff member who can review the decision. P-39.1 s.12.1 Lesson 5 works through that rule.

The European Union: deployers, and the fundamental rights assessment

Under the EU AI Act, deployers of high risk AI systems carry their own duties, among them using the system according to the instructions for use, assigning human oversight to competent people, and monitoring its operation. AI Act Art. 26(1), (2), (4), (5) (both subparagraphs), (6) (both sentences), (7), (8) and (11); recital 158

In this course, Quillmere places the model on the EU market under its own name, which makes it the provider: the company that develops a system and places it on the market under its own name or trademark. AI Act Art. 3(1), (3) and (4) Its own provider duties are outside this course.

Some deployers also perform a fundamental rights impact assessment before first use. That includes bodies governed by public law, private entities providing public services, and deployers of systems used for life and health insurance risk assessment and pricing. AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a) The European life insurer's pricing use falls in that last group, so the assessment is the insurer's to perform, and it notifies the market surveillance authority of the results. AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a)

These high risk duties apply from December 2, 2027 for systems listed in Annex III, such as life and health insurance pricing. Regulation (EU) 2024/1689 Art. 2(2), Art. 111(4), Art. 113, as amended by Regulation (EU) 2026/1744 A high risk system placed on the market or put into service before that date is covered only if its design changes significantly from then on, except that a high risk system intended for use by public authorities must comply by August 2, 2030 in any case. AI Act Art. 111(1), (2) and (3), Art. 111(2) as replaced by Regulation (EU) 2026/1744; Regulation (EU) 2026/1744, recital 39 The insurer still plans the assessment now, because the system will outlive the date and the duty returns with the first significant change.

Where the same processing of personal data is likely to result in a high risk to people's rights and freedoms, the GDPR also calls for a data protection impact assessment, in particular for a systematic and extensive automated evaluation of personal aspects that leads to decisions with legal or similarly significant effects. GDPR Art. 35(1), (3)(a) The two can be joined: the fundamental rights assessment may cross refer to the data protection impact assessment where it already covers an obligation. AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a)

NIST: a method, not a duty

The NIST AI Risk Management Framework, released on January 26, 2023, is intended for voluntary use. NIST AI RMF 1.0 (NIST AI 100-1) None of the rules in this course requires any of Quillmere's customers to follow it. NIST AI RMF 1.0 (NIST AI 100-1) It is useful because it gives one vocabulary for mapping, measuring and managing risk that fits under all three customers' duties, and this course borrows from it as a method.

The screening table

The template for this lesson is a one page screening table: one row per use, the rule that applies, who carries the duty, which assessment it calls for, and what Quillmere supplies. Filled in for the three customers, it shows that one system needs three assessments, owned by three different organizations, built from one shared description of the system.

Worked example

Quillmere's screening table

Fictional organization. Quillmere Decision Systems Inc. is invented for this course. Any resemblance to a real organization is a coincidence.

The sales team asked one question: "Do we need an impact assessment?" The governance lead answered with a table instead of a yes or no.

UseRuleWho carries the dutyAssessmentWhat Quillmere supplies
Federal department sorts benefit applicationsDirective on Automated Decision-Making TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1The departmentAlgorithmic impact assessmentModel description, training data, test results
Quebec insurer decides small claims automaticallyQuebec privacy impact assessment and automated decision rule P-39.1 s.3.3, s.17 P-39.1 s.12.1The insurerPrivacy impact assessmentSame description, plus the factors behind a decision
EU life insurer prices policiesAI Act fundamental rights assessment, from December 2, 2027 AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a) Regulation (EU) 2024/1689 Art. 2(2), Art. 111(4), Art. 113, as amended by Regulation (EU) 2026/1744, GDPR assessment GDPR Art. 35(1), (3)(a)The insurerBoth, cross referenced AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a)Instructions for use, oversight design
Quillmere's own testing on synthetic dataNone of the above: test use is excluded from the Directive TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1QuillmereInternal review onlyNot applicable

What the table showed. Three customers, three owners, three assessments. Every one of them needs the same core description of the system, so Quillmere decided to write that description once and keep it current.

Scenario exercise · Allow 7 min

Screen three new uses

Three more requests reach Quillmere this quarter:

  1. A provincial health insurer in Ontario wants to test the model on last year's anonymized claims, in a test environment, before deciding whether to buy.
  2. A Quebec credit union wants the model to approve credit limit increases with no person involved.
  3. A federal department wants it to rank inspection reports by urgency, so inspectors visit the worst sites first.

For each, say which rule from this lesson is most likely to apply, who would carry the duty, and what Quillmere should prepare.

Show the model answer
  1. Probably none of this lesson's assessment rules yet. The Directive does not apply to a provincial body, and even for a federal one, test environments are excluded. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 Quillmere still documents the test, so the description is ready if the insurer buys.
  2. Quebec rules, the credit union's duty. The project needs a privacy impact assessment P-39.1 s.3.3, s.17, and approving limits with no person involved is a decision based exclusively on automated processing. P-39.1 s.12.1 Quillmere prepares the description and the principal factors and parameters the credit union will have to explain.
  3. The Directive, the department's duty. Ranking reports to decide where inspectors go is an assessment related to administrative decisions. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 Quillmere prepares the material for the department's algorithmic impact assessment.

Why this is the strongest answer

Start from the use, not the model: the same model is in scope for one customer and out of scope for another. Then name the owner of the duty, because the vendor supplies information but does not carry the customer's assessment. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a)

Template to download

AI impact screening table

One row per use of the system: the rule that applies, who carries the duty, which assessment it calls for, and what the vendor supplies, with the exclusions to check first.

Check your understanding

Scenario questions. Answer them all, then check: you will see why each option is right or wrong.

Question 1 of 5

A federal department uses a model to rank applications for review. Is the Directive on Automated Decision-Making engaged?

Question 2 of 5

Quillmere runs its model on synthetic data in a test environment. Does the Directive apply to that work?

Question 3 of 5

A Quebec insurer's project to add the model to its claims system involves personal information. What does Quebec law ask for?

Question 4 of 5

An EU life insurer will use Quillmere's model to help price policies. Who performs the fundamental rights impact assessment?

Question 5 of 5

A manager says the NIST framework is the law for the company. What is the accurate reply?

Sources cited in this lesson

  1. TBS Directive on Automated Decision-Making ss.5.1, 5.2, 8.1, 8.2; Appendix A (administrative decision); Policy on Service and Digital s.6.1 official text current to 2025-06-24, checked on 2026-11-01
  2. TBS Directive on Automated Decision-Making ss.1.1, 1.2.1, 1.3 official text current to 2025-06-24, checked on 2026-10-29
  3. P-39.1 s.3.3, s.17 official text current to 2026-10-15, checked on 2026-11-01
  4. P-39.1 s.3.3 official text current to 2026-10-15, checked on 2026-10-26
  5. P-39.1 s.12.1 official text current to 2026-10-15, checked on 2026-10-24
  6. AI Act Art. 26(1), (2), (4), (5) (both subparagraphs), (6) (both sentences), (7), (8) and (11); recital 158 official text current to 2026-07-27, checked on 2026-11-03
  7. AI Act Art. 3(1), (3) and (4) official text current to 2026-07-27, checked on 2026-10-29
  8. AI Act Art. 27(1) to (4); Art. 27(4) as replaced by Regulation (EU) 2026/1744, Art. 1(13)(a) official text current to 2026-07-27, checked on 2026-11-01
  9. Regulation (EU) 2024/1689 Art. 2(2), Art. 111(4), Art. 113, as amended by Regulation (EU) 2026/1744 official text current to 2026-07-27, checked on 2026-11-01
  10. AI Act Art. 111(1), (2) and (3), Art. 111(2) as replaced by Regulation (EU) 2026/1744; Regulation (EU) 2026/1744, recital 39 official text current to 2026-07-27, checked on 2026-11-01
  11. GDPR Art. 35(1), (3)(a) official text current to 2016-05-04, checked on 2026-10-26
  12. NIST AI RMF 1.0 (NIST AI 100-1) checked on 2026-10-29

Get the full course

One payment of 59 USD. Every lesson, the knowledge checks, the final exam and the certificate open on your account. The first lesson is free to read.

Prices are in US dollars. Terms of sale · Privacy notice

You can ask for a full refund of a course within 14 days of buying it, if you have not passed its final exam.