The Compass QuestionnaireTake this questionnaire to turn complex detailsinto tailored manageable guides for actions and decisions.COMPASS FRAMEWORK INSTRUCTION Assessing Streghts and Weaknesses The questions are designed to help assess and understand various aspects of the AI system such as ethical implications and societal impacts. For each of the seven components, there are close-ended questions that can be rated on a scale from 0 to 5. The closed-ended questions will result in a radar chart. This provides direct insight into the areas where the team scored well and where they did not perform so well. Date of evaluation: Name/working title of AI system under evaluation: Short description of AI system: (e.g., type of AI/ML application, main appliation area, etc.) Stage of development: (e.g., idea phase, design, early development, late-stage development, complete, legacy system, or Technology Readiness Level) CONTEXT To what extent have you described the problem definition and use case for your AI system? NA012345 To what extent have you determined what machine learning domain the AI system will fall under and what consequences this may bring about in its application(s)? NA012345 To what extent AI system is required for the job or task in question and whether it should be used instead of traditional engineering approaches? NA012345 To what extent have you comprehensively outlined the roles and responsibilities of all current or potential stakeholders? NA012345 To what extent have you identified who will be impacted by the AI system and in what ways, throughout its whole life cycle? NA012345 To what extent have you specified selection criteria for the data(sets) used in the training of your model? NA012345 To what extent have you included considerations for disadvantages, inaccuracies, potential biases, and other concerns related to including certain data(sets) in the definitions of these selection criteria? NA012345 OPENNESS To what extent is it possible to obtain information about the basic structure and nature of the data used? NA012345 To what extent have you designed the AI system documentation to be accessible and usable? NA012345 How well are stakeholders informed about data collection, processing, and storage during the AI lifecycle? NA012345 To what extent does the AI system provide meaningful and understandable explanations and justifications regarding its output? NA012345 Is there a mechanism for human intervention when it is necessary such as the operator taking control in situations where artificial intelligence encounters uncertainties? NA012345 To what extent is the AI system transparent about human intervention (e.g., HITL, HOTL, Human-in-Command)? NA012345 To what extent have you shared relevant performance indicators (e.g. accuracy, precision,…) for your models with your stakeholders? NA012345 To what extent do you make research findings open and do you interact with expert communities about the AI system? NA012345 To what extent can you rate the suitability and openness of the AI system for external review and auditing? NA012345 MEASURES To what extent have you established a method for assessing how well the AI system operates, including which aspects you'll evaluate and the metrics you'll use? NA012345 To what extent have you defined the criteria for success and failure of the AI system? NA012345 To what extent have you considered standardizing these criteria? NA012345 To what extent have you determined if the AI system will function differently in other contexts, for various (groups of) people or different applications, and whether this different functioning is appropriate or acceptable? NA012345 To what extent have you integrated measures to prevent overdependence on the AI system and deskilling of the workforce? NA012345 To what extent have you integrated (potentials for) human intervention (e.g., HiTL, HoTL, Human in Command)? NA012345 PRIVACY POTENTIALS To what extent does the AI system align with current privacy regulations or standards such as GDPR, AI ACT, ISO, IEC standards, or other relevant regulations? NA012345 How thoroughly do you review third-party data sources, tools, libraries, or otherwise for potential privacy risks before integrating them into your AI system? NA012345 To what extent does the system allow stakeholders to control their data (e.g., data review request, data deletion request)? NA012345 To what extent are privacy protection techniques incorporated in the AI system’s design? NA012345 To what extent have you identified risk scenarios that could harm privacy, including direct and indirect causes? NA012345 To what extent have you ensured that the AI system notifies relevant parties of changes and adjustments, especially users? NA012345 To what extent does the AI system prioritize data minimization and anonymization practices? NA012345 How effective is the level of the AI system at obtaining necessary permissions and preventing unauthorized access to sensitive information? NA012345 To what extent have you considered the possibility that the output of the AI system could lead to privacy violations? NA012345 To what extent do you ensure privacy (preservation) beyond regulatory compliance? NA012345 Have you tested the resilience of your system against state of the art privacy attacks such as membership inference attacks? Is your system yielding high-enough robustness? NA012345 ACCOUNTABILITY To what extent do you have methods in place to ensure traceability and/or explainability of the AI system throughout its life cycle? NA012345 To what extent can you explain and justify the operations and decision-making processes of your AI system to different stakeholder groups? NA012345 To what extent can you justify the responsible use of your training/testing/validation datasets? NA012345 To what extent is your development process auditable; how open/prepared is your development process for external auditing? NA012345 To what extent have you established internal structures to ensure ethical and responsible development and deployment of the AI system? NA012345 How well can you ensure that the AI-system is accountable in the event of failures or undesirable consequences? NA012345 To what extent have you discussed the cost of predictive errors against different groups? Are they asymmetric (i.e. certain groups can suffer more from specific errors?)? NA012345 SAFETY AND SECURITY To what extent is security considered throughout the life cycle of the AI system, e.g., access and control, resilience, data security/protection and breach management? NA012345 To what extent is responsibility for security of the AI system determined, documented, and communicated, considering the complete life cycle and all stakeholders involved? NA012345 To what extent have you defined how security is balanced with other characteristics of the AI system, e.g., openness, transparency, and performance? NA012345 To what extent have you recognized and defined the (safety) risks with the current and potential future use cases of the AI system? NA012345 To what extent have you determined potential individual and societal harms stemming from use or misuse of the AI system? NA012345 To what extent have you established a procedure for managing security or safety incidents? NA012345 To what extent have you identified and analyzed potential threats to your AI-based system and defined corresponding mitigation strategies? NA012345 To what extent have you evaluated the robustness of your AI systems against state-of-the-art adversarial ML attacks (e.g. data poisoning attacks, evasion attacks)? NA012345 SUSTAINABILITY To what extent does your system development actively acquire and use feedback from different stakeholder groups? NA012345 To what extent is your organization responsible for quality assurance and maintenance of the system during deployment and beyond? NA012345 To what extent do you consider futureproofing of the AI system in your development process? NA012345 To what extent do you remain updated on future developments and changes in standards relevant to your AI system? NA012345 To what extent have you formulated detailed short-term plans for updates to the AI system? NA012345 To what extent have you developed detailed long-term plans for updates to the AI system? NA012345 To what extent do you consider the environmental impact of the development and deployment of your AI system? NA012345 Exploring Insights on AI Sytems The open-ended questions are not graded, but they help evaluators identify areas needing more extensive work. It provides an opportunity to delve deeper into specific challenges or considerations related to AI systems. CONTEXT Briefly describe the worst-case scenario related to the usage of your AI system. Who are the main affected stakeholders? Who is responsible for addressing this situation? What are potential implications your identified machine learning domain has on your potential user base or stakeholders, and what are potential mitigation strategies you can implement? What dataset(s) will be used, and what are the advantages and disadvantages for using the selected option(s)? OPENNESS What strategies are you developing to encourage a more human-centered approach to AI system openness? How can a non-technical third party understand the functioning of your system? What steps would they have to take? What do you think about the adequacy of technical descriptions, documents, or browsers of your AI system for individuals with different backgrounds and knowledge levels to access? MEASURES Briefly describe how you (intend to) assess the AI system, in different scenarios and for different types of stakeholders. How will the collaboration with humans and AI system look? What are potential risks for this collaboration, and what training needs can be identified for people? PRIVACY POTENTIALS How do you stay compliant with changes in privacy laws and regulations? How do you approach and prioritize data minimization and anonymization principles, even when using third-party data sources? Briefly describe the main privacy risk scenarios and how you identified these risks. In case you are reusing pretrained models by others, how did you consider their privacy protection level? ACCOUNTABILITY Briefly describe which methods have you deployed to achieve explainability? Also briefly explain how this relates to justifying your decision-making process to different stakeholder groups. Briefly describe the responsibility chain in case of adverse functioning by or failure of the AI system and/or its components. If you are making use of or producing open-source systems, describe your own role of accountability in the impact and re-use of these systems. SAFETY AND SECURITY Briefly describe potential security risks for the AI system itself, and for the stakeholders using the AI system. What mechanisms do you (plan to) use to detect, mitigate, and protect against these threats? How do you ensure the confidentiality and availability of data specifically sensitive and personal data within the AI life cycle ? SUSTAINABILITY Briefly describe how the AI system will be maintained over time, who will (need to) be involved, and include environmental considerations. Reflect on the short-term and long-term plans for keeping your AI system up-to-date in regards to compatibility, standards, security etc. Briefly describe mitigation actions that are (or will be) deployed to mitigate the environmental impact of the system. Briefly describe what actions are required to update the AI system during its operational phase and which potential (side) effects may occur. The Compass Framework Results: Strengths and Weaknesses The results are intended to direct actions. Those serve as a navigation to turn complex details into tailored manageable guides for actions and decisions. COMPASS Component Close-ended Questions Question score Overall score COMPASS Component Open-ended Questions AI System data Context Openness Measures Privacy Potentials Accountability Safety and Security Sustainability
