How should a quality management system for high-risk AI systems be established and implemented? The Spanish Agency for the Supervision of Artificial Intelligence (AESIA) has published its fourth guideline, Quality Management System. While Guide No. 3 focuses on the conformity assessment of high-risk AI systems, Guide No. 4 addresses the organisational and technical measures that providers must establish in order to comply with Article 17 of the AI Act. The quality management system is intended to ensure that the development, placing on the market and operation of high-risk AI systems are carried out according to documented, controlled and traceable processes throughout the system’s lifecycle. AESIA identifies Guide No. 4 as one of its technical guides developed within the Spanish AI regulatory sandbox. The guides are non-binding and provide practical recommendations aligned with the requirements of the AI Act.

‍

Article 17 requires providers of high-risk AI systems to put in place a quality management system that ensures compliance with the AI Act. The system must be documented in a systematic and orderly manner through written policies, procedures and instructions and must cover the different stages and activities involved in the development and operation of the AI system. The Guide therefore treats quality management not as a single document or certification, but as an organisational framework integrating the different compliance processes applicable to the AI system.

‍

The quality management system must include a regulatory compliance strategy. This strategy must identify the applicable regulatory requirements and explain how the provider will ensure compliance with the AI Act and other relevant Union or national legislation. It must also take into account the use of standards, technical specifications and other relevant compliance instruments.

‍

The Guide places particular emphasis on the definition of responsibilities within the organisation. The provider must establish an appropriate accountability and governance structure and clearly allocate responsibilities among management, technical personnel and other persons involved in the development and operation of the AI system. The organisational structure must make it possible to determine who is responsible for each relevant activity and who has authority to take decisions concerning compliance, risks and corrective measures.

‍

The quality management system must also establish procedures for the design, development and testing of the AI system. These procedures should ensure that the development process is controlled and documented and that the system is subject to appropriate testing and validation before being placed on the market or put into service. Design and development activities must take account of the requirements applicable to high-risk AI systems and must be linked to the risk management process.

‍

Another element concerns the examination, testing and validation procedures that must be applied before and during the development of the AI system. The provider must establish appropriate methodologies and criteria for verifying the system's performance and compliance. Testing should be carried out under conditions that are representative of the environment in which the system is intended to operate, taking into account reasonably foreseeable uses and potential risks.

‍

The Guide also addresses data management. The quality management system must include procedures concerning the acquisition, collection, analysis, labelling, storage and management of data used by the AI system. These procedures must be coordinated with the requirements on data governance established by Article 10 of the AI Act. Data-related processes must therefore be documented and controlled as part of the overall quality management framework.

‍

The QMS must incorporate the risk management system required by Article 9. Risk management is therefore not treated as a separate activity but as an integral component of the provider's quality management structure. The provider must establish processes for identifying, analysing, evaluating and mitigating risks associated with the AI system and must ensure that the results of the risk management process are reflected in the design, development, testing and operation of the system.

‍

The Guide further considers post-market monitoring. The quality management system must contain procedures for collecting and analysing information concerning the performance of the AI system after it has been placed on the market or put into service. Information obtained through post-market monitoring must be used to identify problems, emerging risks or non-conformities and, where necessary, to trigger corrective or preventive measures.

‍

The management of changes to the AI system is another important element of the QMS. Providers must establish procedures for identifying, assessing, documenting and controlling modifications to the system. Changes must be evaluated to determine whether they may affect compliance with the AI Act or require a new conformity assessment. This is particularly important for AI systems that may evolve during their lifecycle.

‍

The quality management system must also establish procedures for dealing with non-conformities. Where the provider identifies that an AI system does not comply with an applicable requirement, appropriate corrective action must be taken. The procedures must allow the provider to identify the nature and cause of the non-conformity, determine its consequences and prevent its recurrence.

‍

The Guide addresses record-keeping and document management as part of the quality management system. Relevant information must be documented and maintained in a manner that ensures traceability and allows the provider to demonstrate how compliance has been achieved. Documentation must be appropriately controlled, updated and made available to the relevant persons and authorities where required.

‍

The QMS must also establish procedures for communication with competent authorities and other relevant stakeholders. The provider must be able to provide the information and documentation required under the AI Act and to cooperate with market surveillance authorities and other competent bodies. Communication procedures must therefore form part of the organisational compliance framework.

‍

The Guide also considers the allocation of resources necessary to operate the quality management system. The provider must ensure that sufficient personnel, technical resources and organisational arrangements are available to carry out the activities required by the AI Act. The resources devoted to quality management should be appropriate to the nature, size and complexity of the organisation and to the risks associated with the AI system.

‍

An important principle is that the quality management system should be proportionate to the size of the provider's organisation. The AI Act does not require every provider to establish an identical or equally complex organisational structure. The system must nevertheless cover all the elements required by Article 17, while its implementation can take account of the provider's size, organisational structure, resources and the complexity of the AI systems concerned. This proportionality is particularly relevant for SMEs and start-ups.

‍

The Guide also addresses the relationship between the quality management system and other AI Act requirements. The QMS must provide an organisational framework within which risk management, data governance, technical documentation, record-keeping, human oversight, accuracy, robustness, cybersecurity, conformity assessment and post-market monitoring can operate in a coordinated manner. Compliance with Article 17 therefore requires the provider to integrate these different activities rather than treating them as isolated obligations.

‍

The quality management system must remain operational throughout the entire lifecycle of the AI system. It therefore begins with the regulatory and organisational planning of the system and continues through design, development, testing, validation, conformity assessment, placing on the market or putting into service, post-market monitoring, management of changes and corrective actions.

‍

The Guide also connects the QMS with continuous improvement. Information obtained through testing, monitoring, incidents, complaints, non-conformities and other sources should feed back into the quality management system. Where weaknesses or new risks are identified, the provider should update its processes and, where necessary, modify the AI system or introduce additional controls.

‍

The QMS must also address the management of suppliers and third parties involved in the development or operation of the AI system. Where activities are outsourced, the provider remains responsible for ensuring compliance with the AI Act and must establish appropriate procedures for controlling and monitoring the activities carried out by external organisations.

‍

The Guide places particular emphasis on the integration of accuracy and cybersecurity into the development process. These aspects should not be addressed only after the AI system has been completed but should be incorporated into design, development, testing and validation procedures. The QMS must therefore ensure that technical requirements are translated into documented organisational processes.

‍

The quality management system is also closely connected with conformity assessment. The processes and documentation established under Article 17 provide the organisational basis for demonstrating compliance with the AI Act. The QMS must therefore be structured so that the provider can demonstrate that the AI system has been developed and controlled in accordance with the applicable requirements and can provide the necessary evidence during conformity assessment.

‍

The Guide ultimately presents the QMS as a comprehensive governance framework composed of interconnected policies, procedures, responsibilities and controls. Its main elements include regulatory compliance, accountability and governance, design and development control, testing and validation, data management, risk management, technical documentation, record-keeping, post-market monitoring, change management, non-conformity management and communication with authorities and stakeholders.

‍

Overall, Guide No. 4 explains how Article 17 can be translated into an operational organisational system. The provider must establish a documented framework that assigns responsibilities, controls the design and development of the AI system, manages data and risks, verifies performance, monitors the system after deployment, manages changes and non-conformities, maintains the necessary records and ensures continuing compliance throughout the AI system's lifecycle.

‍