Artificial Intelligence Safety Checklist

This tool responds to the GC AI Strategy for the Federal Public Service 2025-2027, the GC Data Strategy for the Federal Public Service (2023-2026) Mission 3.3 for responsible, transparent and ethical data stewardship to maintain trust, the GC Directive on automated decision-making , the GC Guide on the use of generative artificial intelligence , the GC Algorithmic impact assessment tool , NIST AI Risk Management Framework , the International Scientific Report on the Safety of Advanced AI , and the EU Artificial Intelligence Act.

This tool is suitable for a first screening level estimation of the AI safety of a particular AI application.

You may also find the following companion tools useful:

Do you work with Artificial Intelligence (AI)? Are you looking to make it future proof? The FAIRER data principles will help you!

This tool helps you to assess the AI safety level of an AI application and get tips on how you could increase the AI safety.

The tool is discipline-agnostic, making it relevant to any field.

The checklist will take 15-30 minutes to complete, after which you will receive a quantitative summary of the AI safety level your application, and tips on how you can improve the level of AI safety. No information is saved on our servers, but you will be able to save the results of the assessment, including tips for improvement, to your local computer and add notes for future reference.

Version 1.0

CRediT Author statement

ARTIFICIAL INTELLIGENCE (AI) BACKGROUND

Artificial Intelligence (AI) simulates human-like intelligence in machines, enabling them to learn, reason, solve problems, perceive environments, and make autonomous or semi-autonomous decisions. AI is a machine-based system that, for explicit or implicit objectives, infers from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. AI systems differ in their autonomy (ability to operate without human intervention) and adaptiveness (capacity to improve or modify behavior after deployment). (OECD 2024)

AI includes knowledge-based systems (which use human-curated domain knowledge, rules, facts, and relationships to simulate expert decision-making) and machine learning systems (which learn patterns from data and generalize to new tasks without being explicitly programmed). The latter include neural networks (algorithms inspired by the structure and function of the human brain and used for pattern recognition and classification tasks) and deep learning (a subfield of neural networks that uses many layers of representation to learn complex features from large datasets).

AI application areas include natural language processing (NLP), computer vision, speech recognition, intelligent decision support systems, intelligent robotic systems, predictive analytics, and recommendation systems.

Generative AI is a subfield of artificial intelligence focused on creating new content (e.g., text, images, audio, video, music, speech, computer code, or synthetic data) based on patterns learned from input data.

Automation (not AI) uses technology to execute predefined, often repetitive tasks with minimal human intervention. Automation systems do not learn, adapt, or infer; they follow static rules or programmed sequences. Examples include robotic process automation (RPA), scripting and macros, workflow engines and batch processing, industrial robotics and control systems and algorithmic tasks that do not involve learning or adaptation.

NOTE: This checklist is not designed for assessment of automation applications that do not use AI.

Checklist Questions

1. I am:
A data steward	Yes No
A policy advisor	Yes No
A researcher	Yes No
A scientist	Yes No
An AI developer	Yes No
An AI end-user	Yes No
Other
2. I use this AI application during one or more stages of the AI lifecycle (NIST):
View Description Lifecycle and Key Dimensions of an AI System. The two inner circles show AI systems’ key dimensions and the outer circle shows AI lifecycle stages: Plan & Design, Collect & Process Data, Build & Use Model, Verify & Validate, Deploy & Use, Operate & Monitor. Ideally, risk management efforts start with the Plan & Design function in the application context and are performed throughout the AI system lifecycle. SOURCE: NIST, modified from OECD (2022) Framework for the Classification of AI systems.
Plan and Design	Yes No Don't know
Collect and Process Data	Yes No Don't know
Build and Use Model	Yes No Don't know
Verify and Validate	Yes No Don't know
Deploy and Use	Yes No Don't know
Operate and Monitor	Yes No Don't know
Use or Impacted by	Yes No Don't know
3. This AI application is used in:
Administrative decision-making	Yes No Don't know
Policy formulation	Yes No Don't know
Project prioritization	Yes No Don't know
Scenario analysis	Yes No Don't know
Science based operations	Yes No Don't know
Research	Yes No Don't know
A system operating in a test environment	Yes No Don't know
Other
4. I have completed an Algorithmic Impact Assessment and the score was:
0-25% (level I, Little to no impact).
26-50% (Level II, moderate impact).
51-75% (Level III, high impact).
76%-100% (Level IV, very high impact).
Not completed
5. The practical use of this AI application is for:
Art and creativity	Yes No Don't know
Autonomous vehicles	Yes No Don't know
Finance	Yes No Don't know
Healthcare	Yes No Don't know
Image and video recognition	Yes No Don't know
Natural Language Processing (NLP)	Yes No Don't know
Generative AI	Yes No Don't know
Other
6. This AI application embodies principles characteristic of trustworthy AI:
View Description Trustworthiness: nine principles characteristic of trustworthy AI systems. Valid & Reliable is a necessary condition of trustworthiness and is shown as the base for the principles: safe, secure & resilient, explainable & interpretable, privacy-enhanced, and fair & managed bias. Accountable & Transparent is shown as a vertical box relating to those six principles. Altogether, 150 properties of trustworthiness have been identified across all seven principles (Newman, 2023) . The 8th and 9th principles (Human-centred values and Inclusive growth, sustainable development, and well-being) are shown as underlying all the other principles. Figure adapted from NIST, European Commission, and OECD.
Safe	Yes No Don't know
Secure & resilient	Yes No Don't know
Explainable & interpretable	Yes No Don't know
Privacy-enhanced	Yes No Don't know
Fairness and management of bias	Yes No Don't know
Valid & Reliable	Yes No Don't know
Accountable & Transparent	Yes No Don't know
Human-centred values	Yes No Don't know
Inclusive growth, sustainable development, and well-being	Yes No Don't know
7. This AI application was designed taking into account:
The positive and negative impacts on end users	Yes No Don't know
All foreseeable use cases of the AI application	Yes No Don't know
8. Designated humans have the ultimate responsibility for all decisions and outcomes from this AI application:
Responsibilities are explicitly defined between the AI system and human(s), and how they are shared.	Yes No Don't know
Human responsibility will be preserved for final decisions that affect a person’s life, quality of life, health, or reputation.	Yes No Don't know N/A
Humans are always able to monitor, control, and deactivate systems.	Yes No Don't know
Significant decisions made by the AI system are explained.	Yes No Don't know N/A
Significant decisions made by the AI system are able to be overridden.	Yes No Don't know N/A
Significant decisions made by the AI system are appealable.	Yes No Don't know N/A
Significant decisions made by the AI system are reversible.	Yes No Don't know N/A
9. The design and use of this AI application embodies transparency and engenders trust:
The purpose, limitations, and biases of the AI system are explained in plain language.	Yes No Don't know
Data sources have unambiguous respected sources, and biases are known and explicitly stated.	Yes No Don't know
Data used for training are updated to suit the appropriate use cases	Yes No Don't know
Algorithms and models are appropriate and verifiable.	Yes No Don't know
Confidence level and context are presented for humans to base decisions on.	Yes No Don't know
Transparent justification for recommendations and outcomes are provided.	Yes No Don't know
Straightforward and interpretable monitoring systems are provided.	Yes No Don't know
Humans are aware when they are being monitored or surveilled for the purpose of data collection or performance	Yes No Don't know N/A
Humans can easily discern when they are interacting with the AI system vs. a human.	Yes No Don't know N/A
Humans can easily discern when and why the AI system is taking action and/or making decisions.	Yes No Don't know N/A
Improvements are made regularly to meet human needs and technical standards.	Yes No Don't know
10. The most relevant method used for this AI application is (ordered from safest to least safe):
Rule-based expert system
Machine Learning
Neural Network
Deep Learning
Computer Vision
Natural Language Processing (NLP) or Large Language Model (LLM)
Don't know
11. This AI application uses tools and methods that are inherently designed for AI tasks to build, train, and deploy AI models:
AI Development and Deployment Platforms	Yes No Don't know
AI Frameworks for Edge and Mobile	Yes No Don't know
AI Research and Experimentation Platforms	Yes No Don't know
AI-Specific Hardware	Yes No Don't know
Automated Machine Learning (AutoML)	Yes No Don't know
Computer Vision Libraries	Yes No Don't know
Deep Learning Libraries	Yes No Don't know
Machine Learning Frameworks	Yes No Don't know
Natural Language Processing (NLP) Tools	Yes No Don't know
Reinforcement Learning Libraries	Yes No Don't know
Simple AI tasks	Yes No Don't know
Recommendation System	Yes No Don't know
Robotics	Yes No Don't know
Speech recognition	Yes No Don't know
Other
12. The capability vs. inherent limitation of this AI application is properly managed to mitigate negative impacts:
Data-Driven Decision Making vs. Human Judgment	Yes No Don't know N/A
Efficiency vs. Ethical Considerations	Yes No Don't know N/A
Personalization vs. Privacy	Yes No Don't know N/A
Automation vs. Human Interaction	Yes No Don't know N/A
Common Sense Reasoning vs. Causality	Yes No Don't know N/A
Data Processing vs. Data Quality	Yes No Don't know N/A
Creativity vs. Hallucination	Yes No Don't know N/A
Knowledge vs. Inconsistency	Yes No Don't know N/A
Optimization vs. Context Awareness	Yes No Don't know N/A
Predictability vs. Innovation	Yes No Don't know N/A
Other (Choices 11-26)	Yes No Don't know N/A
13. The data used in this AI application are available:
Raw input dataset(s) are openly available.	Yes No Upon request only
Input data provenance is identified	Yes No Upon request only
AI generated dataset(s) are openly available	Yes No Upon request only
14. The data used in this AI application comply with FAIRER principles as described by the FAIRER-Aware Data Assessment Tool
FAIRER = Findable, Accessible, Interoperable, Ethical, and Reproducible	Yes No Partially Don't know
15. I have completed FAIRER-Aware Reproducibility Checklist and the score was:
0-25% (Poor)
26-50% (Low)
51-75% (Good)
76%-100% (High)
Not completed
16. This AI application has been tested for model performance using:
Benchmark measurements	Yes No Don't know
Adversarial attack	Yes No Don't know
Auditing	Yes No Don't know
Field testing	Yes No Don't know
Human evaluation	Yes No Don't know
Other	Yes No Don't know
Not tested
17. I have read AI legislation, international and government references, and other resources:
International	Yes No Some
Government of Canada	Yes No Some
Europe	Yes No Some
U.S. Government	Yes No Some
National Institute of Standards and Technology (NIST)	Yes No Some
National Academies of Sciences, Engineering, and Medicine (NASEM)	Yes No Some
Other	Yes No Some

Your Notes

Add any notes you may have here. These notes will be included when you print and save your results to your local computer. No information will be saved to our server. Feel free to capture any thoughts or insights that you'd like to remember or revisit later.

Your use of this AI application is 0% safe

Author Statement

Emily Chu (Data curation, Software, Visualization) ;and,
Claire C. Austin (Conceptualization, Supervision, Visualization, Writing).

All authors reviewed, discussed, and agreed to all aspects of the final work.

All views and opinions expressed are those of the co-authors, and do not necessarily reflect the official policy or position of their respective employers, or of any government, agency or organization.

Cite as: Chu E, and Austin CC (2025). FAIRER Aware AI Safety Checklist. https://fairerdata.github.io/FAIRER-Aware-AI-SAFETY-Assessment/

Art and creativity

Examples:

Generating art
Music composition
Style transfer

Autonomous vehicles

Examples:

Navigation
Obstacle detection
Decision-making for self-driving cars

Finance

Examples:

Fraud detection
Algorithmic trading
Credit scoring

Healthcare

Examples:

Disease diagnosis
Drug discovery
Personalized medicine

Image and video recognition

Examples:

Facial recognition
Object detection
Medical imaging analysis
Identifying pedestrians and obstacles

Natural Language Processing (NLP)

Examples:

Language translation
Sentiment analysis
Chatbot
Virtual assistants

Generative AI

Generative AI refers to a category of artificial intelligence techniques and algorithms that are designed to generate new content, such as text, images, audio, video, computer code, or other types of data by learning from existing data.

Examples:

Art
Images
Text
Stories
Music
Synthetic data
Computer code

Rule-based Expert System

It relies on predefined rules and logical statements created by experts (usually with if-then statements) to make decisions and perform tasks. It is deterministic: (i.e. produces the same output for a given input), and is effective for well-defined problems with clear rules.

Machine Learning

It focuses on developing algorithms that allow computers to learn from and make predictions or decisions based on data

Examples:

Supervised learning
Unsupervised learning
Reinforcement learning

Deep Learning

It uses neural networks with many layers (deep neural networks) to analyze complex patterns in large datasets

Computer Vision

It enables computers to interpret and understand the visual world. By using digital images from cameras, videos, and deep learning models to identify and process objects, environments, and actions.

AI Development and Deployment Platforms

Examples:

AI Frameworks for Edge and Mobile

Examples:

AI Research and Experimentation Platforms

Examples:

Google Colab

AI-Specific Hardware

Examples:

Automated Machine Learning (AutoML)

Examples:

Computer Vision Libraries

Examples:

Deep Learning Libraries

Examples:

Machine Learning Frameworks

Examples:

Natural Language Processing (NLP) Tools

Examples:

Reinforcement Learning Libraries

Examples:

Simple AI tasks

Examples:

Recommendation System

Examples:

Personalized content delivery in streaming service
E-commerce
Social media

Robotics

Examples:

Robot vision
Manipulation
Interaction

Speech recognition

Examples:

Voice-controlled device
Transcription service
Virtual assistant
Language translator

Unanswered Questions

Improvement Suggestions

All capability vs. inherent limitation choices

Administrative decision-making is any decision that has the potential to affect legal rights, privileges or interests of individuals or businesses. For example,

Likely use cases include:

Claims processing
Government benefits
Healthcare administration
HR management
Immigration
Permit approvals
Regulatory investigations

Possible use cases include:

Environmental impact assessments
Environmental risk assessment
Finance and accounting
Monitoring compliance
Regulatory compliance
Supply chain management

Not usually involving administrative decision-making:

Budget management
Citizen feedback analysis
Customer service
Data-driven insights
Economic regulation
Forecasting and resource allocation
Inventory management
Logistics planning

Checklist Questions

View Description

View Description

Your Notes

1. Data-Driven Decision Making vs. Human Judgment

2. Efficiency vs. Ethical Considerations

3. Personalization vs. Privacy

4. Automation vs. Human Interaction

5. Common Sense Reasoning vs. Causality

6. Data Processing vs. Data Quality

7. Creativity vs. Hallucination

8. Knowledge vs. Inconsistency

9. Optimization vs. Context Awareness

10. Predictability vs. Innovation

11. Formal Reasoning vs. Compositionality

12. Forecasting vs. Novel Concepts

13. Consistency vs. Adaptability

14. Precision vs. Generalization

15. Integration vs. Autonomy

16. Simulation vs. Embodiment

17. Automation vs. Flexibility

18. Scalability vs. Real-time Processing

19. Adaptability vs. Stability

20. Scalability vs. Interpretability

21. Objective Analysis vs. Empathy

22. Accessibility vs. Complexity

23. Multi-tasking vs. Specialization

24. Cost Reduction vs. Implementation Cost

25. Consistency vs. Creativity

26. Speed vs. Accuracy