REPORT on artificial intelligence: questions of interpretation and application of international law in so far as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

4.1.2021 - (2020/2013(INI))

Committee on Legal Affairs
Rapporteur: Gilles Lebreton
Rapporteur for the opinion (*):
Urmas Paet, Committee on Foreign Affairs
(*) Associated committees – Rule 57 of the Rules of Procedure


Procedure : 2020/2013(INI)
Document stages in plenary
Document selected :  
A9-0001/2021
Texts tabled :
A9-0001/2021
Debates :
Votes :
Texts adopted :

MOTION FOR A EUROPEAN PARLIAMENT RESOLUTION

on artificial intelligence: questions of interpretation and application of international law in so far as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

(2020/2013(INI))

The European Parliament,

 having regard to the preamble to the Treaty on European Union (TEU), and to Articles 2, 3, 10, 19, 20, 21, 114,167, 218, 225 and 227 thereof,

 having regard to the right to petition enshrined in Articles 20 and 227 of the Treaty on the Functioning of the European Union (TFEU),

 having regard to the Charter of Fundamental Rights of the European Union,

 having regard to Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin (Racial Equality Directive)[1],

 having regard to Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation (Equal Treatment in Employment Directive)[2],

 having regard to Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)(GDPR)[3], and to Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and repealing Council Framework Decision 2008/977/JHA[4],

 having regard to Council Regulation (EU) 2018/1488 of 28 September 2018 establishing the European High Performance Computing Joint Undertaking[5],

 having regard to the proposal for a regulation of the European Parliament and of the Council of 6 June 2018 establishing the Digital Europe programme for the period 2021-2027 (COM(2018)0434),

 having regard to its resolution of 16 February 2017 setting out recommendations to the Commission on civil law rules on robotics[6],

 having regard to its resolution of 1 June 2017 on digitising European industry[7],

 having regard to its resolution of 12 September 2018 on autonomous weapon systems[8],

 having regard to its resolution of 11 September 2018 on language equality in the digital age[9],

 having regard to its resolution of 12 February 2019 on a comprehensive European industrial policy on artificial intelligence and robotics[10],

 having regard to the communication of 11 December 2019 from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on the European Green Deal (COM/2019/640),

 having regard to the Commission White Paper of 19 February 2020 on Artificial Intelligence – A European approach to excellence and trust (COM(2020)0065),

 having regard to the Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions of 19 February 2020 on a European Strategy for data (COM(2020)0066),

 having regard to the Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions of 19 February 2020 on Shaping Europe’s digital future (COM(2020)0067),

 having regard to the report of 8 April 2019 of the High-Level Expert Group on Artificial Intelligence set up by the Commission in June 2018, entitled ‘Ethics Guidelines for Trustworthy AI’,

 having regard to the Council of Europe’s Framework Convention for the Protection of National Minorities, Protocol No. 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms, and the European Charter for Regional or Minority Languages,

 having regard to the European ethical Charter on the use of Artificial Intelligence in judicial systems and their environment adopted by the Council of Europe Working Group on quality of justice (CEPEJ-GT-QUAL) in December 2018,

 having regard to the OECD Council Recommendation on Artificial Intelligence adopted on 22 May 2019,

 having regard to Rule 54 of its Rules of Procedure,

 having regard to the opinions of the Committee on Foreign Affairs, the Committee on the Internal Market and Consumer Protection, the Committee on Transport and Tourism and the Committee on Civil Liberties, Justice and Home Affairs,

 having regard to the report of the Committee on Legal Affairs (A9-0001/2021),

Introduction

A. whereas artificial intelligence (AI), robotics and related technologies are being developed quickly, and have a direct impact on all aspects of our societies, including basic social and economic principles and values;

B. whereas AI is causing a revolution in military doctrine and equipment through a profound change in the way armies operate, owing mainly to the integration and use of new technologies and autonomous capabilities;

C. whereas the development and design of so-called ‘artificial intelligence’, robotics and related technologies are done by humans, and their choices determine the potential of technology to benefit society;

D. whereas a common Union framework must cover the development, deployment and use of AI, robotics and related technologies, and must ensure respect for human dignity and human rights, as enshrined in the Charter of Fundamental Rights of the European Union;

E.  whereas the Union and its Member States have a particular responsibility to make sure that AI, robotics and related technologies – as they can be used cross borders – are human-centred, i.e. basically intended for use in the service of humanity and the common good, in order to contribute to the well-being and general interest of their citizens; whereas the Union should help the Member States to achieve this, in particular those which begun to reflect on the possible development of legal standards or legislative changes in this field;

F. whereas European citizens could benefit from an appropriate, effective, transparent and coherent regulatory approach at Union level that defines sufficiently clear conditions for companies to develop applications and plan their business models, while ensuring that the Union and its Member States retain control over the regulations to be established, so that they are not forced to adopt or accept standards set by others;

G. whereas ethical guidance, such as the principles adopted by the High Level Expert Group on Artificial Intelligence, provides a good starting point but is not enough to ensure that businesses act fairly and guarantee the effective protection of individuals;

H. whereas this particular responsibility implies a need to examine questions of interpretation and application of international law related to the active participation of the EU in international negotiations, in so far as the EU is affected by the civil and military uses of this kind of AI, robotics and related technologies, and questions of state authority over such technologies lie outside the scope of criminal justice;

I. whereas it is essential to provide an appropriate and comprehensive legal framework for the ethical aspects of these technologies as well as for liability, transparency and accountability (in particular for AI, robotics and related technologies considered to be high risk); whereas this framework must reflect that the intrinsically European and universal humanist values are applicable to the entire value chain in the development, implementation and uses of IA; whereas this ethical framework must apply to the development (including research and innovation), deployment and use of IA, in full respect of Union law and the values set out in the Charter of Fundamental Rights of the European Union;

J. whereas the purpose of this examination is to determine to what extent the rules of international public and private law and EU law are geared to dealing with these technologies, and to highlight the challenges and risks which the latter pose for state authority, so that they can be properly and proportionately managed;

K. whereas the European Commission does not consider the military aspects of the use of artificial intelligence in its White Paper;

L. whereas a harmonised European approach to these problems calls for a common definition of AI, and for steps to ensure that the fundamental values of the European Union, the principles of the Charter of Fundamental Rights and international human rights legislation are upheld;

M. whereas AI is providing unprecedented opportunities to enhance performance in the transport sector by addressing the challenges of increasing travel demand, safety and environmental concerns, while making all transport modes smarter, more efficient and more convenient;

N. whereas addressing AI in defence at the EU level is indispensable for the development of EU capabilities in this sector;

Definition of artificial intelligence

1. Considers that it is necessary to adopt a common European legal framework with harmonised definitions and common ethical principles, including the use of AI for military purposes; calls on the Commission, therefore, to adopt the following definitions:

- ‘AI system’ means a system that is either software-based or embedded in hardware devices, and that displays behaviour simulating intelligence by, inter alia, collecting and processing data, analysing and interpreting its environment, and by taking action, with some degree of autonomy, to achieve specific goals;

- ‘autonomous’ means an AI system that operates by interpreting certain input, and by using a set of predetermined instructions, without being limited to such instructions, despite the system’s behaviour being constrained by and targeted at fulfilling the goal it was given and other relevant design choices made by its developer;

2. Highlights that the security and defence policies of the European Union and its Member States are guided by the principles enshrined in the European Charter of Fundamental Rights and the UN Charter – the latter calling on all states to refrain from the threat or use of force in their relations with each other – as well as by international law, by the principles of human rights and respect for human dignity, and by a common understanding of the universal values of the inviolable and inalienable rights of the human person, of freedom, of democracy, of equality and of the rule of law; highlights that all defence-related activities within the Union framework must respect these universal values while promoting peace, stability, security and progress in Europe and in the world;

International public law and military uses of artificial intelligence

3. Considers that AI used in a military and a civil context must be subject to meaningful human control, so that at all times a human has the means to correct, halt or disable it in the event of unforeseen behaviour, accidental intervention, cyber-attacks or interference by third parties with AI-based technology or where third parties acquire such technology;

4. Considers that the respect for international public law, in particular humanitarian law, which applies unequivocally to all weapons systems and their operators, is a fundamental requirement with which Member States must comply, especially when protecting the civilian population or taking precautionary measures in the event of an attack such as military aggression or cyberwarfare;

5. Highlights that AI and related technologies can also play a part in irregular or unconventional warfare; suggests that research, development, and use of AI in such cases should be subject to the same conditions as use in conventional conflicts;

6. Emphasises that the use of AI provides an opportunity to strengthen the security of the European Union and its citizens, and that it is essential for the EU to adopt an integrated approach in future international debates on this topic;

7. Calls on the AI research community to integrate this principle into all the aforementioned AI-based systems intended for military use; considers that no authority may establish any exception to those principles or certify such a system;

8. Reiterates that autonomous decision-making should not absolve humans from responsibility, and that people must always have ultimate responsibility for decision-making processes so that the human responsible for the decision can be identified;

9. Stresses that during the use of AI a military context, Member States, parties to a conflict and individuals must at all times comply with their obligations under applicable international law and take responsibility for actions resulting from the use of such systems; underlines that under all circumstances the anticipated, accidental or undesirable actions and effects of AI-based systems must be considered the responsibility of Member States, parties to a conflict and individuals;

10. Welcomes the possibilities for using artificial intelligence systems for training and exercises, whose potential should not be underestimated, especially given that the EU conducts exercises of a dual civilian and military nature;

11. Highlights that during the design, development, testing, deployment and use phases of AI-enabled systems, due account must be taken of potential risks at any time, with particular regard to accidental civilian casualties and injury, accidental loss of life, and damage to civilian infrastructure, as well as risks related to unintended engagement, manipulation, proliferation, cyber-attacks, interference by third parties with AI-based autonomous technology, or where third parties acquire such technology;

12. Recalls that according to the Advisory Opinion of the International Court of Justice of 8 July 1996, the principle of originality cannot be cited in support of any derogation regarding compliance with current norms of international humanitarian law;

13. Considers that, in addition to supporting operations, AI will also benefit service staff of the armed forces through the mass processing of their health data and expanding health monitoring, will identify risk factors related to their environment and working conditions and propose appropriate safeguards to limit health impacts on service personnel;

14. Reiterates that regulatory efforts must be supported by meaningful certification and surveillance schemes, as well as by clear auditability, explainability, accountability, and traceability mechanisms, so that the regulatory framework does not become outdated as a result of technological developments.

15. Stresses the importance, in a hyper-connected world, of European Union involvement in the creation of an international legal framework for the use of artificial intelligence: urges the EU to take the lead and assume, with the United Nations and the international community, an active role in promoting this global framework governing the use of AI for military and other purposes, ensuring that this use remains within the strict limits set by international law and international humanitarian law , in particular the Geneva Conventions of 12 August 1949; stresses that this framework must never breach. or permit breaches of, the dictates of public conscience and humanity as stated in the Martens clause, and be in line with safety rules and consumer protection requirements; urges the EU and the Member States to define robust surveillance and evaluation systems for the development of AI technologies, particularly those used for military purposes in authoritarian states;

16. Highlights that robotics will not only enable military personnel to stay at a distance, but also provide better self-protection, for example in operations in contaminated environments, fire-fighting, mine clearance on land or at sea, and defence against drone swarms;

17. Stresses the fact that the development, deployment, use and management of AI must respect the fundamental rights, values and freedoms enshrined in the EU Treaties, and calls on Member States to refrain from deploying high-risk AI systems that pose threats to fundamental rights; takes note of the publication of the Commission’s White Paper on Artificial Intelligence, and encourages more in-depth research into the potential risk to fundamental rights resulting from the use of AI by state authorities and agencies, bodies and institutions of the European Union;

18. Calls on the Commission to facilitate research into and discussion on the opportunities for using AI in disaster relief, crisis prevention and peacekeeping;

19. Welcomes the creation of a UN Group of Governmental Experts (GGE) on Advancing responsible State behaviour in cyberspace in the context of international security, and calls for the EU to fully participate in its work;

20. Calls on the Vice President of the Commission / High Representative for Foreign Affairs and Security Policy (VP/HR) to pave the way for global negotiations with a view to putting in place an AI arms control regime and updating all existing treaty instruments on arms control, disarmament and non-proliferation so as to take into account AI-enabled systems used in warfare; calls for the Council Common Position defining common rules governing control of exports of military technology and equipment to fully take into account and cover AI-enabled weapons systems;

21. Reiterates that these rules must always be consistent with the principles referred to in the Rome Convention of 17 July 1998 regarding the prohibition of crimes of genocide, crimes against humanity and war crimes;

22. Points to the clear risks involved in decisions made by humans if they rely solely on the data, profiles and recommendations generated by machines; points out that the overall design of AI systems should also include guidelines on human supervision and oversight; calls for an obligation to be imposed regarding transparency and explainability of AI applications and the necessity of human intervention, as well as other measures, such as independent audits and specific stress tests to facilitate and enforce compliance; stresses that such audits should be conducted periodically by an independent authority that would supervise high-risk AI applications used by state authorities or the military;

23. Emphasises the importance of verifying how high-risk AI technologies arrive at decisions; recalls that the principles of non-discrimination and proportionality need to be respected, and that questions of causality, liability and responsibility, as well as transparency, accountability and explainability, need to be clarified to determine whether, or to what extent, the state as an actor in public international law, but also in exercising its own authority, can act with the help of AI-based systems with a certain autonomy, without breaching obligations stemming from international law, such as due process

24. Insists on the importance of investing in human skills, including digital skills, in order to adapt to scientific progress involving AI-driven solutions, for individuals exercising regulated professions, including activities connected with the exercise of state authority, such as the administration of justice; calls on the Member States and the Commission to duly take this into account as part of the implementation of Directive 2005/36/EC;

25. Insists that AI systems must always comply with the principles of responsibility, equity, governability, precaution, accountability, attributability, predictability, traceability, reliability, trustworthiness, transparency, explainability, the ability to detect possible changes in circumstances and operational environment, the distinction between combatants and non-combatants, and proportionality; stresses that the latter principle makes the legality of a military action conditional on a balance between the objective pursued and the means used, and that the assessment of proportionality must always be made by a human being;

26. Stresses that in the use of AI-enabled systems in security and defence, the comprehensive situational understanding of the human operator, the predictability, reliability, and resilience of the AI-enabled system, as well as the human operator’s ability to detect possible changes in circumstances and operational environment, and their ability to intervene in or discontinue an attack are needed to ensure that international humanitarian law principles, in particular distinction, proportionality and precaution in attack, are fully applied across the entire chain of command and control; stresses that AI-enabled systems must allow the humans in charge to exert meaningful control, to assume full responsibility over the systems, and be accountable for all of their uses; calls on the Commission to foster dialogue, closer cooperation and synergies between Member States, researchers, academics, civil society actors, the private sector, in particular leading companies, and the military, to ensure that policy-making processes for defence-related AI regulations are inclusive;

27. Stresses that Parliament has called for the drafting and urgent adoption of a common position on lethal autonomous weapon systems (LAWS), preventing the development, production and the use of LAWS capable of attack without meaningful human control, as well as the initiation of effective negotiations for their prohibition; recalls in this regard its resolution of 12 September 2018 on autonomous weapon systems; recalls that the term ‘Lethal Autonomous Weapons Systems’ (LAWS) refers to weapons systems without meaningful human control over the critical functions of targeting and attacking individual targets; emphasises that the decision to select a target and to take lethal action by means of weapons systems with a certain degree of autonomy must always be made by human operators exercising meaningful control, oversight and the necessary judgment in line with the principles of proportionality and necessity; stresses that AI-enabled systems can under no circumstances be allowed to replace human decision-making in this field;

28. Notes, moreover, that autonomous weapons systems, as a particular category of AI in the military domain, should be discussed and agreed internationally, specifically in the UN Convention on Certain Conventional Weapons forum; draws attention to the ongoing international debate on LAWS to regulate emerging military technologies, which has so far failed reach agreement; points out that the EU has only recently agreed to discuss the effects of AI developments and digitalisation on the defence sector; believes that the EU can play a crucial role in helping Member States to harmonise their approach to military AI, in order to lead international discussions;

29. Insists on the need for an EU-wide strategy against LAWS and a ban on so-called ‘killer robots’;

30. Emphasises that the AI used in a military context must meet a minimum set of requirements, namely it should be able to distinguish between combatants, non-combatants, and combatants on the battlefield, recognise when a combatant surrenders or is hors de combat, not have indiscriminate effects, not cause unnecessary human suffering, not be biased or trained on intentionally incomplete data, and comply with the principles of international humanitarian law, proportionality in the use of force and precaution before intervention;

31. Considers that the use of lethal autonomous weapon systems raises fundamental ethical and legal questions about the ability of humans to control these systems, and requires that AI-based technology should not be able to make autonomous decisions involving the legal principles of distinction, proportionality and precaution;

32. Calls for transparent risk-reduction measures at international level for the development and use of military AI, in particular with regard to the principles of territorial integrity, non-intervention and the use of force; stresses the importance of taking into account military aspects when addressing legal and ethical issues in the European framework on AI; recalls its position on a ban on the development, production and use of LAWS; regrets that no explicit global conventions exist on the use of these weapons;

33. Acknowledges that the modern arms-race dynamics resulting from major military nation states developing LAWS are outpacing the progress on and effective universal application and enforcement of common rules and legal frameworks because information on the development and deployment of these systems is classified, and nation states have an inherent interest in creating the fastest and most effective offensive capabilities, irrespective of current or potential future legal frameworks or principles;

34. Considers that LAWS should be used only as a last resort, and are lawful only if they are subject to strict human control, with a human able to take over command at any time, as meaningful human intervention and supervision are essential in the process of making lethal decisions, and since human beings should always be responsible when deciding between life and death; believes that systems without any human control (‘human off the loop’) and human oversight must be banned with no exceptions and under all circumstances;

35. Calls on the VP/HR, the Member States and the European Council to develop and adopt, as a matter of urgency, a common position on autonomous weapons systems that ensures meaningful human control over the critical functions of weapons systems, including during deployment, to speak with one voice in relevant forums and act accordingly; calls, in this context, on the VP/HR, the Member States and the Council to share best practices and garner input from experts, academics and civil society, as reflected in the 12 September 2018 position on autonomous weapons systems, which states that attacks should always be carried out with significant human intervention;

36. Encourages all states to carry out an assessment of whether and how autonomous military devices have contributed to their national security, and what their national security could gain from AI-enabled weapon systems, in particular from the potential of such technologies to support and enhance human decision-making in compliance with international humanitarian law and its principles; recalls that any LAWS or weapon with a high degree of autonomy can malfunction because of badly written code or a cyber-attack perpetrated by an enemy state or a non-state actor;

37. Stresses that LAWS should be used only in clearly defined cases and in accordance with authorisation procedures laid down in detail in advance in documents to which the state concerned — whether or not it is a member of the North Atlantic Treaty Organisation — guarantees public access, or at least access for its national parliament;

38. Considers that LAWS must comply with the provisions of the Convention of 10 October 1980 on Certain Conventional Weapons, including the prohibition of weapons deemed ‘excessively injurious’;

39. Suggests, in order to prevent their uncontrolled spread, that LAWS should be included in the list of weapons subject to the provisions of the Arms Trade Treaty of 2 April 2013, listed under Article 2 of this Treaty;

40. Calls for the anthropomorphisation of LAWS to be prohibited in order to rule out any possibility of confusing humans with robots;

41. Welcomes the agreement between the Council and Parliament to exclude lethal autonomous weapons ‘without the possibility for meaningful human control over the selection and engagement decisions when carrying out strikes’ from actions funded under the European Defence Fund (EDF);recalls its position that the use, the development or the production of LAWS without meaningful human control is not eligible for funding under the EDF;

42. Calls on the Commission to support the research, development, deployment and use of AI for preserving peace and preventing conflicts;

43. Notes that the global AI ecosystem is dominated by American and Chinese digital giants, which are developing domestic capabilities and buying many promising companies; is of the firm opinion, therefore, that in order to avoid lagging behind in artificial intelligence technology, the EU needs to move towards a better balance between basic research and industrial applications, while developing comparative strategic advantages by further building its own potential and resources;

44. Stresses that, insofar as they fall under the definition of machinery set out in Directive 2006/42/EC, robots should be designed and assembled in compliance with the standards and safety measures provided for therein;

45. Recalls the EU’s ambition to be a global actor for peace, and calls for the expansion of its role in global disarmament and non-proliferation efforts, and for its actions and policies to strive for the preservation of international peace and security, ensuring respect for international humanitarian and human rights law and the protection of civilians and civilian infrastructure;

46. Stresses the need to examine the potential impact of AI as a strategic factor for the EU’s Common Security and Defence Policy (CSDP), especially in military and civilian missions and operations, and the development of EU capabilities;

47. Recalls that our allies within national, NATO or EU frameworks are themselves in the process of integrating AI into their military systems; believes that interoperability with our allies must be preserved by means of common standards, which are essential for the conduct of operations in coalition; recalls that, apart from that, cooperation on AI should occur within a European framework, which is the only relevant framework for truly generating powerful synergies, as proposed by the EU’s AI strategy;

48. Considers that the EU needs to carefully monitor and consider the implications of advances in AI for defence and warfare, including potentially destabilising developments and deployments, and guide ethical research and design, ensuring the integrity of personal data and individual access and control, as well as taking into account economic and humanitarian issues;

49. Recalls its position of 12 September 2018 on autonomous weapons systems, which states that strikes must not be carried out without meaningful human intervention; calls on the VP/HR, the Member States and the European Council to adopt a common position on autonomous weapons systems that ensures meaningful human control over the critical functions of weapons systems, including during deployment; reaffirms its support for the work on LAWS of the UN GGE of the High Contracting Parties to the Convention on Certain Conventional Weapons, which remains the relevant international forum for discussions and negotiations on the legal challenges posed by autonomous weapons systems; calls for all current multilateral efforts to be accelerated so that normative and regulatory frameworks are not outpaced by technological developments and new methods of warfare; calls on the VP/HR, in the framework of the ongoing discussions on the international regulation of LAWS by the states parties to the CCW, to remain engaged and help to advance, without delay, the effort to develop a new global regulatory framework and a legally binding instrument focused on definitions, concepts and characteristics of emerging technologies in the area of LAWS, ethical and legal questions of human control, in particular with regard to their critical functions, such as target selection and engagement, the maintenance of human responsibility and accountability and the necessary degree of human-machine interaction, including the concept of human control and human judgment; calls for these efforts to ensure compliance with international humanitarian and human rights law during the different stages of the lifecycle of AI-enabled weapons, with a view to agreeing specific recommendations for the clarification, consideration and development of aspects of the normative framework relating to emerging technologies in the area of LAWS;

50. Believes that an effective mechanism for enforcing the rules on non-proliferation of LAWS and any future offensive AI-enabled technologies is of paramount importance for global security;

State authority: examples from civil areas, including health and justice

51. Stresses that Member States must act effectively to reduce their reliance on foreign data and, without significantly distorting the market, ensure that the possession of highly sophisticated AI technologies by powerful private groups does not result in the authority of the state being challenged or even usurped by private entities, especially if these private groups are owned by a third country outside the European Union;

52. Stresses that the use of AI systems in the decision-making process of public authorities can result in biased decisions that negatively affect citizens, and therefore should be subject to strict control criteria regarding their security, transparency, accountability, non-discrimination, social and environmental responsibility, among others; urges Member States to assess the risks related to AI-driven decisions connected with the exercise of State authority, and to provide for safeguards such as meaningful human supervision, transparency requirements and the possibility to contest such decisions;

53. Urges the Member States to assess the risks related to AI-driven technologies before automating activities connected with the exercise of state authority, such as the administration of justice; calls on the Member States to consider the need to provide for safeguards such as supervision by a qualified professional and strict rules on professional ethics;

54. Stresses the importance of taking action at European level to help promote much-needed investment, data infrastructure, research, including research into the use of artificial intelligence by public authorities and a common ethical framework;

55. Stresses that the European Union needs to strive for strategic resilience so that it never again finds itself unprepared in the event of a crisis, and underlines that this is of crucial significance, especially for artificial intelligence and its military applications; emphasises that supply chains for military AI systems which can lead to technological dependence should be reviewed, and that such dependencies should be phased out; calls for increased investment in European AI for defence and in the critical infrastructure that sustains it;

56. Invites the Commission to assess the consequences of a moratorium on the use of facial recognition systems, and, depending on the results of this assessment, to consider a moratorium on the use of these systems in public spaces by public authorities and in premises meant for education and healthcare, as well as on the use of facial recognition systems by law enforcement authorities in semi-public spaces such as airports, until the technical standards can be considered fully fundamental rights-compliant, the results derived are non-biased and non-discriminatory, and there are strict safeguards against misuse that ensure the necessity and proportionality of using such technologies;

57. Emphasises the importance of cybersecurity for AI, in both offensive and defensive scenarios; notes in this regard the importance of international cooperation and of the publication and sharing of IT security vulnerabilities and remedies; calls for international cybersecurity cooperation for effective AI use and deployment, and for safeguards against misuse of AI and cyber-attacks; notes, furthermore, the dual-use nature of IT systems (i.e. use for civil and military purposes) and of AI, and calls for its effective regulation;

58. Believes that Member States should promote AI technologies that work for people, and that persons who have been the subject of a decision taken by a public authority based on the information from an AI system should be informed thereof, should receive the information referred to in the preceding paragraph without delay, be offered the possibility of contesting that decision, and to opt for this appeal to be resolved without the intervention of an AI system; calls on the Member States to consider the need to establish safeguards, as provided for in Directive (EU) 2018/958, such as supervision by a qualified professional and rules on professional ethics;

59. Underlines that making predictions based on sharing data, access to data, or its use, must be governed by the requirements of quality, integrity, transparency, security, privacy and control; stresses the need, throughout the development, deployment and use of AI, robotics and related technologies, to respect the EU legal framework on data protection and privacy, in order to increase citizens’ security and their trust in those technologies;

60. Observes the rapid development of AI applications that recognise unique characteristic elements, such as facial characteristics, movements and attitudes; warns of issues of invasion of privacy, non-discrimination and the protection of personal data related to the use of automated recognition applications;

61. Underlines that any decision about a natural person that is based solely on automated processing, including profiling, and which produces an adverse legal effect on the data subject or significantly affects that person, is prohibited under the GDPR unless authorised by Union or Member State law, subject to appropriate measures to safeguard the data subject’s rights, freedoms and legitimate interests;

62. Calls for the explainability of algorithms, for transparency and regulatory oversight when artificial intelligence is used by public authorities, and for impact assessments to be conducted before tools using AI technologies are deployed by state authorities; calls on the Commission and the European Data Protection Board to issue guidelines and recommendations and develop best practices in order to further specify the criteria and conditions applicable to decisions based on profiling and the use of AI by public authorities;

63. Notes that artificial intelligence is playing an increasingly fundamental role in healthcare, in particular through algorithms to assist diagnosis, robot-assisted surgery, smart prostheses, personalised treatments based on the three-dimensional modelling of an individual patient’s body, social robots to help elderly people, digital therapies designed to improve the independence of some mentally ill people, predictive medicine and epidemic response software;

64. Insists, nevertheless, that all uses of AI in the area of public health must guarantee the protection of patients’ personal data and prevent the uncontrolled dissemination of those data;

65. Call for all uses of AI in public health to uphold the principle of the equal treatment of patients in terms of access to treatment, preserve the patient-doctor relationship, and be consistent with the Hippocratic Oath at all times, so that the doctor is always able to deviate from the solution suggested by AI, thereby maintaining responsibility for any decision;

66. Notes that the use of AI in fighting crime and cybercrime could bring a wide range of possibilities and opportunities; affirms, at the same time, that the principle that what is illegal offline is illegal online should continue to prevail;

67. Notes that AI is increasingly being used in the field of justice in order to take decisions which are more rational, more in keeping with the law in force, and quicker; welcomes the fact that the use of AI is expected to speed up judicial proceedings;

68. Considers that it is necessary to clarify whether it is appropriate for law enforcement decisions to be partially delegated to AI, while maintaining human control over the final decision;

69. Stresses that the use of AI in justice could improve the analysis and collection of data and the protection of victims, and that this could be explored in research and development and accompanied by impact assessments, in particular regarding safeguards for due process and against bias and discrimination, with the precautionary principle being applied; recalls, however, that this is no substitute for human involvement in sentencing or decision-making;

70. Recalls the importance of the principles of governance, transparency, impartiality, accountability, fairness and intellectual integrity in the use of AI in criminal justice;

71. Urges the Member States to assess the risks related to AI-driven technologies before automating activities connected with the exercise of state authority, especially in the area of justice; calls on them to consider the need to provide safeguards, such as supervision by a qualified professional and rules on professional ethics;

72. Notes that certain AI technologies enable the automation of information processing and action on an unprecedented scale, such as mass civil and military surveillance, which poses a threat to fundamental rights, and paves the way for unlawful intervention in state sovereignty; calls for the scrutiny of mass surveillance activities under international law, including as regards questions of jurisdiction and enforcement; expresses serious concerns about some highly intrusive social scoring applications that have been developed, as they seriously endanger the respect of fundamental rights; calls for an explicit ban on the use of mass social scoring by public authorities as a way to restrict the rights of citizens; calls for the accountability of private actors under international law to be enhanced, given the decision-making hegemony and control of certain private actors over the development of these technologies; calls, in this context, on the Commission, the Council and the Member States to pay particular attention when negotiating, concluding and ratifying international agreements related to cross-border family cases, such as international child abductions, and to ensure that in this context AI systems are always used under effective human verification, and respect due process within the EU and countries which are signatories of these agreements;

73. Requests that the public is kept informed about the use of AI in the field of justice, and that such uses do not give rise to discrimination resulting from programming biases; stresses that the right of every individual to have access to a public official must be respected, as well as the right of the responsible official to personally take the decision and deviate from the information received from the AI when they deem it necessary in the light of the details of the matter in question; highlights the right of the defendant to appeal the decision in accordance with national legislation, without ever eliminating the final responsibility of the judiciary;

74. Calls, therefore, for all these public and administrative uses be deemed information in the public domain, and for discrimination due to programming biases to be avoided;

75. Stresses the importance of enabling the proper deployment and use of AI; calls on the Member States to provide their civil and military personnel with appropriate training in order to allow them to accurately identify and avoid discrimination and bias in datasets;

76. Is deeply concerned about deepfake technologies that allow increasingly realistic photo, audio and video forgeries to be produced that could be used to blackmail, generate fake news reports, or erode public trust and influence public discourse; believes such practices have the potential to destabilise countries, spreading disinformation and influencing elections; calls, therefore, for an obligation for all deepfake material or any other realistically made synthetic videos to be labelled as ‘not original’ by the creator, with strict limits on their use for electoral purposes and robust enforcement; calls for adequate research in this field to ensure that technologies to counter these phenomena keep pace with the malicious use of AI;

Transport

77. Takes note of the significant economic potential of AI applications, including for the optimisation of long-term performance, maintenance, failure prediction and construction planning in transport infrastructure and buildings, as well as for safety, energy efficiency and costs; calls on the Commission, therefore, to continue promoting AI research and the exchange of good practices in transport;

78. Stresses the need to promote artificial intelligence to foster the multimodality, interoperability and energy efficiency of all modes of transport, to enhance efficiency in the organisation and management of goods and passenger traffic flows, to make better use of infrastructure and resources along the Trans-European Transport Network (TEN‑T), and to address the obstacles to the creation of a true single European transport area;

79. Recalls the benefits of the European Rail Traffic Management System (ERTMS), a seamless automatic train protection system, and supports the development and international standardisation of the automation of train operations;

80. Welcomes the work of the Single European Sky Air Traffic Management Research project (SESAR) on unmanned aircraft systems and air traffic management systems, both civil and military;

81. Recalls that autonomous vehicles have great potential to improve mobility, safety, and bring environmental benefits, and calls on the Commission and the Member States to ensure cooperation among regulators and all stakeholders relevant to the deployment of automated road vehicles in the EU;

82. Points out that the global shipping industry has greatly changed thanks to the integration of AI in recent years; recalls the current comprehensive discussions in the International Maritime Organization on effectively integrating new and emerging technologies, such as autonomous ships, in its regulatory framework;

83. Stresses that intelligent transport systems mitigate traffic congestion, increase safety and accessibility and contribute to improving the management of traffic flows, efficiency and mobility solutions; draws attention to the increased exposure of traditional transport networks to cyber threats; recalls the importance of sufficient resources and further research in security risks in ensuring the safety of automated systems and their data; welcomes the Commission’s intention to include cybersecurity as a regular agenda item for discussion within transport-related international organisations;

84. Welcomes the efforts to introduce AI systems in the public sector, and will support further discussions on AI deployment in transport; calls on the Commission to carry out an evaluation of the use of AI and similar technologies in the transport sector, and to compile a non-exhaustive list of high-risk segments in AI systems replacing decisions within the framework of prerogatives of public power in this area;

85. Underlines that the European Defence Fund and Permanent Structured Cooperation should stimulate cooperation between Member States and European defence industries to develop new European defence capabilities in AI, and ensure security of supply, taking ethical considerations into account; emphasises the need to avoid fragmentation by building bridges between various actors and application domains, by promoting compatibility and interoperability at all levels, and by focusing on joint work on architecture and platform solutions; recalls, moreover, that the next Connecting Europe Facility, which also promotes smart infrastructure, will provide for a fund for the adaptation and the development of civilian or military dual-use transport infrastructure in the TEN‑T in order to increase synergies between civil and defence needs, and with a view to improving civil and military mobility within the Union; emphasises, therefore, the need for further European investment, research, and leadership in technologies with both high economic growth impact as well as significant dual-use potential;

86. Stresses that many investments in new technologies in transport and mobility are market-driven, but that dual-use commercial off-the-shelf technologies and products are often used in innovative ways for military purposes; highlights, therefore, that the dual-use potential of AI-enabled solutions needs to be taken into account when drafting standards for the use of AI in various areas of the commercial and military sectors; calls for high ethical standards and policy to be included in the development of defence technologies, products and operating principles;

87. Points out that the effective transportation of goods, ammunition, armaments and troops is an essential component of successful military operations; stresses that AI is expected to play a crucial role and create numerous possibilities in military logistics and transport; points out that countries throughout the world, including EU Member States, are embedding AI weapons and other systems in land, naval, airborne platforms; recalls that AI applications in the transport sector could provide new capabilities and allow new forms of tactics, such as the combination of many systems such as drones, unmanned boats or tanks in an independent and coordinated operation;

International private law

88. Notes that, given that an increasing number of disputes under international private law are arising from the internationalisation of human activities, either online or in the real world, AI can help to resolve them by creating models to identify the competent jurisdiction and applicable law for each case, and also to identify the most sensitive conflicts of laws and propose ways of resolving them;

89. Considers, however, that the public must be properly informed about the uses of AI in international private law, that these uses must not lead to discrimination through programming, which would result in one nation’s laws being systematically favoured over another’s, must respect the rights of the court predetermined by law, permit appeals in accordance with the applicable law, and allow any judge to disregard the solution suggested by AI;

90. Stresses that the circulation of autonomous vehicles in the European Union, which is liable to give rise to a particularly high number of disputes under international private law, must be the subject of specific European rules stipulating the legal regime applicable in the event of cross-border damage;

91. Points out that with the increasing importance of research and development in the private sector, and massive investments from third countries, the EU is facing strong competition; supports, therefore, the EU’s efforts to further develop its competitive advantages, and believes that the EU should aim to act as a norm-setter for AI in a hyper-connected world by adopting an effective strategy towards its external partners, stepping up its efforts to set global ethical norms for AI at international level in line with safety rules and consumer protection requirements, as well as with European values and citizens’ rights, including fundamental rights; considers that this is also key for the competitiveness and sustainability of European companies; calls on the Commission and Member States to strengthen cooperation with third countries and international organisations, such as the UN, OECD, G7 and G20, and to engage in a broader dialogue to address challenges arising from the development of this rapidly changing technology; considers that these efforts should seek, in particular, to establish common standards and improve the interoperability of AI-enabled systems; calls on the Commission to foster dialogue, closer cooperation and synergies between Member States, researchers, academics, civil society actors, the private sector, in particular leading companies, and the military, in order to ensure that policy-making processes for defence-related AI regulations are inclusive;

Guiding principles

92. Considers that AI technologies and network systems should aim to provide legal certainty for citizens; underlines, therefore, that rules on conflict of laws and jurisdictions should continue to apply, while taking into account citizens’ interests, as well as the need to reduce the risk of forum-shopping; recalls that AI cannot replace humans in the judicial process when it comes to passing sentence or taking a final decision of any kind, as such decisions must always be taken by a human, and be strictly subject to human verification and due process; insists that when using evidence provided by AI-assisted technologies, the judicial authorities should have the obligation to provide reasons for their decisions;

93. Recalls that AI is a scientific advance which must not undermine the law, but must on the contrary always be governed by it — in the European Union by the law emanating from its institutions and its Member States — and that under no circumstances can AI, robotics and related technologies contravene fundamental rights, democracy and the rule of law;

94. Stresses that AI used for defence purposes should be: responsible, equitable, traceable, reliable and governable;

95. Considers that artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies, regardless of the field in which they are used, should be developed in a secure and technically rigorous manner;

°

° °

96. Instructs its President to forward this resolution to the Council and the Commission.


EXPLANATORY STATEMENT

The impressive advances in artificial intelligence (AI) pose a challenge for international law, both public and private, and more broadly for the authority of states. As the framing of legal provisions to govern AI has become one of the EU’s priorities, the European Parliament cannot ignore this issue. This report seeks to address the main aspects concerned.

International public law: military uses of AI

In keeping with the working document of 29 April 2020, a significant portion of the report is devoted to the military uses of AI, the development of which for their armed forces is regarded as a priority by all the main global powers. The values on which the EU is founded justify its desire to address the need to establish a legal framework, with due regard for the powers of the Member States and the United Nations.

The report approaches the issue from three different angles. First of all, it reiterates the principles which should govern all military uses of AI, whatever form they may take: processing of information for military purposes, military logistics, ‘collaborative combat’ and real-time support for decision-making, defensive systems, all weapons using AI including lethal autonomous weapons systems (LAWS).

Secondly, specific principles are set out for LAWS, given the threat level they pose as highlighted by the European Parliament in its resolution of 12 September 2018.

Finally, the report addresses the role of the European Defence Fund, which already finances military research projects implementing AI, evidence that the EU is already directly involved in the military use of AI.

State authority

AI also has implications for the authority of states in non-military matters. Following the debate in the JURI Committee on 16 June 2020, during which a number of members considered that they should also be discussed, it was decided to address these implications in two particularly sensitive areas — health and justice.

International private law

During that same debate, some members also called for the impact of the use of AI on international private law to be considered. It is in response to this that the final part of the report seeks to address this issue, albeit briefly in the light of Parliament’s constraints regarding the length of this type of report.

Guiding principle

Beyond the diversity of the issues it deals with, what shapes the report is the realisation that AI can pose a threat to democratic principles and the rule of law, and a clear-sighted approach must be taken to addressing this threat. For as Martin Heidegger warned when speaking of technology, ‘we are delivered over to it in the worst possible way when we regard it as something neutral’.


 

 

OPINION OF THE COMMITTEE ON FOREIGN AFFAIRS (7.7.2020)

for the Committee on Legal Affairs

on artificial intelligence: questions of interpretation and application of international law insofar as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

(2020/2013(INI))

Rapporteur for opinion (*): Urmas Paet

(*) Associated committee – Rule 57 of the Rules of Procedure

 

 

SUGGESTIONS

The Committee on Foreign Affairs calls on the Committee on Legal Affairs, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1. Highlights that the security and defence policies of the European Union and its Member States are guided by the principles enshrined in the European Charter of Fundamental Rights and the UN Charter – the latter of which calls upon all states to refrain from the threat or use of force in their relations with each other – as well as by international law, by the principles of human rights and respect for human dignity and by a common understanding of the universal values of the inviolable and inalienable rights of the human person, of freedom, of democracy, of equality and of the rule of law; highlights that all defence-related efforts within the Union framework must respect these universal values while promoting peace, stability, security and progress in Europe and in the world;

2. Calls on the UN and the wider international community to undertake all necessary regulatory efforts to ensure that the development and application of artificial intelligence (AI) for military uses and to increase law enforcement civil capacities, such as those of police and border control forces, as well as the study, development and use of AI-enabled systems by the military, stay within the strict limits set by international law, including international humanitarian law (IHL) and human rights law; stresses that the EU should pursue the international adoption of its technical and ethical standards for AI-enabled military systems, and, in close cooperation with like-minded partners, strive for an international regulatory framework outlining common norms based on democratic values, adequately framed so as to prevent their use for espionage, mass, targeted and political surveillance, disinformation and data manipulation, and a cyber arms race; calls for increased cooperation with NATO in order to establish common standards and improve the interoperability of AI-enabled systems; calls on the Commission to foster dialogue, closer cooperation and synergies between Member States, researchers, academics, civil society actors and the private sector, in particular leading companies, and the military, to ensure that policymaking processes for defence-related AI regulations are inclusive;

3. Considers in particular that the design and development for and use of AI-enabled systems in armed conflicts must, as provided for in the Martens Clause, abide by the general principles of IHL and never breach or be permitted to breach the dictates of the public conscience and humanity; considers that compliance with IHL is the minimum admissibility standard for the use of AI-enabled systems in warfare; calls on the AI research community to integrate this principle in all AI-enabled systems intended to be used in warfare; considers that no authority can issue a derogation from those principles or certify an AI-enabled system that breaches them; therefore invites developers of lethal autonomous weapons systems (LAWS) to halt all design, prototyping and production of such systems;

4. Welcomes the creation of a UN Group of Governmental Experts (GGE) on advancing responsible State behaviour in cyber space in the context of international security and calls on the EU to fully participate in its work;

5. Calls on the VP/HR to pave the way for global negotiations with a view to putting in place an AI arms control regime and updating all existing treaty instruments dedicated to arms control, disarmament and non-proliferation so as to take AI-enabled systems used in warfare into account; calls for the Council Common Position defining common rules governing control of exports of military technology and equipment to fully take into account and cover AI-enabled weapons systems;

6. Stresses that states, parties to a conflict and individuals, when employing AI-enabled systems in warfare, must at all times adhere to their obligations and liability under the applicable international law and must remain accountable for actions resulting from the use of such systems; recalls that humans remain accountable for intended, unintended or undesirable effects caused by AI-enabled systems on the battlefield; emphasises that the decision to take lethal action by means of weapons systems with a high degree of autonomy must always be made by human operators exercising meaningful control and oversight and the necessary level of judgment in line with the principles of proportionality and necessity; stresses that AI-enabled systems can under no circumstances be allowed to replace human decision-making;

7. Highlights the need to take duly into account, during the design, development, testing and deployment phases of an AI-enabled system, potential risks with particular regard to incidental civilian casualties and injury, accidental loss of life, and damage to civilian infrastructure, but also risks related to unintended engagement, manipulation, proliferation, cyber-attacks or interference and acquisition by organised crime and terrorist groups, leading to escalatory destabilising effects that aim to make our societies, militaries and institutions vulnerable;

8. Is concerned by the fact that not all members of the international community would follow a regulatory human-centric AI approach; urges the EU and the Member States to assess the development of AI technologies, particularly those used for military and surveillance purposes, in authoritarian states that avoid compliance with EU-led regulations;

9. Stresses the need for robust testing, evaluation, certification, monitoring and verification systems based on clear legal and democratic norms as well as on clear safety and security provisions to ensure that during the entire life cycle of AI-enabled systems in the military domain, in particular during the phases of human-machine interaction, machine learning and adjusting and adapting to new circumstances, the systems and their effects do not go beyond the intended limits and that at all times they are used in a manner that is consistent with the applicable international law; considers that, in the event of these limits being overstepped in a theatre of operations, the civilian or military authorities that deployed AI systems should be held to account;

10. Highlights that any AI-enabled system used in the military domain must, as a minimum set of requirements, be able to distinguish between combatants and non-combatants on the battlefield, between military and civilian targets, recognise when a combatant surrenders or is hors de combat, not have indiscriminate effects, individuate the use of force and not target a certain category of people, not cause unnecessary suffering to persons, nor cruel or degrading treatment, not be biased or be trained on biased data, and be used in compliance with the IHL general principles of humanity, distinction, proportionality, precaution and the principle of military necessity prior to engagement and during attacks; underlines the importance of the quality of algorithms, original data and ex ante review of decision-making processes;

11. Stresses that in the use of AI-enabled systems in security and defence, the comprehensive situational understanding of the human operator, the predictability and reliability of the AI-enabled system, as well as the human operator’s ability to detect possible changes in circumstances and operational environment and their ability to intervene in or discontinue an attack are needed to ensure that IHL principles, in particular distinction, proportionality and precaution in attack, are fully applied across the entire chain of command and control; stresses that AI-enabled systems must allow the military leadership to exert meaningful control, to assume its full responsibility and be accountable for all of their uses;

12. Encourages states to carry out an assessment of whether and how autonomous military devices have contributed to their national security and what their national security could gain from AI-enabled weapon systems, in particular as regards the potential of such technologies to support and enhance human decision-making in compliance with IHL and its principles; recalls that any LAW or weapon with a high degree of autonomy could malfunction on account of badly written code or a cyber-attack perpetrated by an enemy state or a non-state actor;

13. Recalls its resolution of 12 September 2018 on autonomous weapon systems; welcomes in this respect the agreement between the Council and Parliament to exclude lethal autonomous weapons ‘without the possibility for meaningful human control over the selection and engagement decisions when carrying out strikes’ from actions funded under the European Defence Fund;

14. Calls for the establishment of a European agency for artificial intelligence, the mandate of which would cover common standards, certification and monitoring frameworks, as well as strong bilateral cooperation with NATO when it comes to the deployment, development and use of AI in the military field;

15. Recalls its position of 12 September 2018 on autonomous weapons systems, which states that strikes must not be carried out without meaningful human intervention; calls on the VP/HR, the Member States and the European Council to adopt a common position on autonomous weapons systems that ensures meaningful human control over the critical functions of weapons systems, including during deployment; reaffirms its support for the work on LAWS of the UN GGE of the High Contracting Parties to the Convention on Certain Conventional Weapons, which remains the relevant international forum for discussions and negotiations on the legal challenges posed by autonomous weapons systems; calls for all existing multilateral efforts to be accelerated so that normative and regulatory frameworks are not outpaced by technological developments and new methods of warfare; calls on the VP/HR, in the framework of the ongoing discussions on the international regulation of LAWS by the states parties to the CCW, to remain engaged and help to advance, without delay, the effort to develop a new global regulatory framework and a legally binding instrument focused on definitions, concepts and characteristics of emerging technologies in the area of LAWS, ethical and legal questions of human control, in particular with regard to their critical functions, such as target selection and engagement, the maintenance of human responsibility and accountability and the necessary degree of human-machine interaction, including the concept of human control and human judgment, to ensure compliance with international humanitarian and human rights law during the different stages of the lifecycle of an AI-enabled weapon, with a view to agreeing tangible recommendations for the clarification, consideration and development of aspects of the normative framework relating to emerging technologies in the area of LAWS;

16. Stresses that the European Union needs to strive for strategic resilience so that it never again finds itself unprepared in times of crisis, and underlines that, especially where artificial intelligence and its military applications are concerned, this is of crucial significance; emphasises that supply chains for military AI systems which can lead to technological dependence should be recalibrated and that such dependencies should be phased out; calls for increased investment in European AI for defence and in the critical infrastructure that sustains it;

17. Acknowledges that modern arms-race dynamics between major military nation states in the development of LAWS are outpacing the advancement and effective universal application and enforcement of common rules and legal frameworks because information on the development and deployment of these systems is classified and nation states have an inherent interest in creating the fastest and most effective offensive capabilities, irrespective of current or potential future legal frameworks or principles;

18. Believes that it is paramount to global security to have an effective mechanism to enforce the rules on non-proliferation of LAWS and any future offensive AI-enabled technologies.

 


INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

6.7.2020

 

 

 

Result of final vote

+:

–:

0:

60

6

4

Members present for the final vote

Alviina Alametsä, Alexander Alexandrov Yordanov, Maria Arena, Petras Auštrevičius, Traian Băsescu, Lars Patrick Berg, Anna Bonfrisco, Reinhard Bütikofer, Fabio Massimo Castaldo, Susanna Ceccardi, Włodzimierz Cimoszewicz, Katalin Cseh, Tanja Fajon, Anna Fotyga, Michael Gahler, Kinga Gál, Sunčana Glavak, Raphaël Glucksmann, Klemen Grošelj, Bernard Guetta, Márton Gyöngyösi, Sandra Kalniete, Karol Karski, Dietmar Köster, Andrius Kubilius, Ilhan Kyuchyuk, David Lega, Miriam Lexmann, Nathalie Loiseau, Antonio López-Istúriz White, Lukas Mandl, Thierry Mariani, David McAllister, Vangelis Meimarakis, Sven Mikser, Francisco José Millán Mon, Javier Nart, Gheorghe-Vlad Nistor, Urmas Paet, Kostas Papadakis, Tonino Picula, Manu Pineda, Kati Piri, Giuliano Pisapia, Diana Riba i Giner, Jérôme Rivière, María Soraya Rodríguez Ramos, Nacho Sánchez Amor, Isabel Santos, Jacek Saryusz-Wolski, Andreas Schieder, Radosław Sikorski, Sergei Stanishev, Tineke Strik, Hermann Tertsch, Hilde Vautmans, Harald Vilimsky, Idoia Villanueva Ruiz, Viola Von Cramon-Taubadel, Thomas Waitz, Witold Jan Waszczykowski, Charlie Weimers, Isabel Wiseler-Lima, Salima Yenbou, Željana Zovko

Substitutes present for the final vote

Nikos Androulakis, Vladimír Bilčík, Marisa Matias, Juozas Olekas, Mick Wallace

 


FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

60

+

EPP

Alexander Alexandrov Yordanov, Traian Băsescu, Michael Gahler, Kinga Gál, Sunčana Glavak, Sandra Kalniete, Andrius Kubilius, David Lega, Miriam Lexmann, Antonio López-Istúriz White, David McAllister, Lukas Mandl, Vangelis Meimarakis, Francisco José Millán Mon, Gheorghe-Vlad Nistor, Radosław Sikorski, Isabel Wiseler-Lima, Željana Zovko, Vladimír Bilčík

S&D

Maria Arena, Włodzimierz Cimoszewicz, Tanja Fajon, Raphaël Glucksmann, Sven Mikser, Tonino Picula, Kati Piri, Giuliano Pisapia, Nacho Sánchez Amor, Isabel Santos, Andreas Schieder, Sergei Stanishev, Nikos Androulakis Nikos, Juozas Olekas

RENEW

Petras Auštrevičius, Katalin Cseh, KLemen Grošelj, Bernard Guetta, Ilhan Kyuchyuk, Nathalie Loiseau, Javier Nart, Urmas Paet, María Soraya Rodríguez Ramos, Hilde Vautmans

ID

Anna Bonfrisco, Susanna Ceccardi

VERTS

Alviina Alametsä, Reinhard Bütikofer, Diana Riba i Giner, Tineke Strik, Viola von Cramon-Taubadel, Thomas Waitz, Salima Yenbou

ECR

Anna Fotyga, Karol Karski, Jacek Saryusz-Wolski, Hermann Tertsch, Witold Jan Waszczykowski, Charlie Weimers

NI

Fabio Massimo Castaldo, Márton Gyöngyösi

 

6

-

S&D

Dietmar Köster

GUE

Manu Pineda, Idoia Villanueva Ruiz, Marisa Matias, Mick Wallace

NI

Kostas Papdakis

 

4

0

ID

Lars Patrick Berg, Thierry Mariani, Jérôme Rivière, Harald Vilimsky

 

Key to symbols:

+ : in favour

- : against

0 : abstention

 


 

 

 

OPINION OF THE COMMITTEE ON THE INTERNAL MARKET AND CONSUMER PROTECTION (7.7.2020)

for the Committee on Legal Affairs

on artificial intelligence: questions of interpretation and application of international law in so far as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

(2020/2013(INI))

Rapporteur for opinion: Andreas Schwab

 

SUGGESTIONS

The Committee on the Internal Market and Consumer Protection calls on the Committee on Legal Affairs, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1. Recalls that it follows from Directive (EU) 2018/958 that humans must always bear ultimate responsibility for decision-making that involves risks to the achievement of public interest objectives; underlines that any AI technology should be subject to human oversight; stresses that, in order to direct the use of artificial intelligence towards the good of individual consumers or groups of consumers and society as a whole, the human-centric approach to artificial intelligence should always be the ultimate pattern of control when interpreting and applying the law on artificial intelligence;

2. Believes that the EU should aim to act as a norm-setter for AI in a hyper-connected world by adopting an efficient strategy towards its external partners, fostering its efforts to set global ethical norms for AI at international level in line with safety rules and consumer protection requirements, as well as with European values and fundamental rights; considers that this is also key for the competitiveness and sustainability of European companies; calls on the Commission and Member States to strengthen cooperation with third countries and international organisations and to engage in a broader dialogue to address challenges arising from the development of this rapidly changing technology;

3. Recalls that the principles of non-discrimination and proportionality need to be respected and that questions of causality, liability and responsibility, as well as transparency, accountability and explainability, need to be clarified to determine if, or to what extent, the State as an actor in public international law, but also in exercising its own authority, can act with the help of systems based on AI, which have a certain autonomy, without breaching obligations stemming from international law, such as due process;

4. Urges, therefore, the Member States to assess the risks related to AI-driven technologies, such as, for example, the risks related to the protection of consumers and recipients of services, before automating activities or professional services connected with the exercise of State authority, such as the proper administration of justice; calls on the Member States to consider the need to provide for safeguards, foreseen in Directive (EU) 2018/958, such as supervision by a qualified professional and rules on professional ethics; emphasises the need for proper transposition and implementation of this Directive by the Member States, and encourages the Commission to closely monitor the matter;

5. Insists on the importance of investing in human skills, including digital skills, in order to adapt to scientific progress involving AI-driven solutions for individuals exercising regulated professions, including activities connected with the exercise of the powers of State authority, such as the administration of justice; calls on the Member States and the Commission to take this duly into account as part of the implementation of Directive 2005/36/EC;

6. Considers that, with regard to smart products or devices, the development of AI also entails opportunities to improve global market surveillance and address product safety and consumer protection in a much more effective way and on a large scale; calls, consequently, on the Commission to establish a governance and market surveillance mechanism for AI-based systems to issue guidance, opinions and expertise to Member States’ authorities;

7. Stresses that, insofar as they fall under the definition of machinery set out in Directive 2006/42/EC, robots should be designed and assembled in compliance with the standards and safety measures provided therein;

8. Recalls that the NIS Directive provides for a common European approach to cybersecurity and the resilience of network and information systems in order to strengthen the single market and promote consumer confidence; urges the Member States and the Commission to take seriously the threat of cyberattacks based on AI technologies, for example when they undermine the EU’s economy, and to invest heavily to achieve a high level of overall digital literacy and reduce dependency on third-country vendors;

9. Believes that Member States’ authorities, including their regional and local authorities, and the Commission should promote AI technologies that work for people; calls on the Member States, in close cooperation with the Commission, to develop AI applications aimed at automating and facilitating e-government services, for example in the areas of tax administration, customs and consumer protection or the notification of cross-border business activities; urges Member States to use public procurement as an instrument to support the responsible development of AI for the benefit of new market entrants, citizens and consumers; underlines that explainable and unbiased algorithms that meet the obligation of sufficient transparency, as well as the use of open data in line with Directive (EU) 2019/1024 on open data and the re-use of public sector information, are essential to ensure that businesses and consumers can trust in, and benefit from, better, accessible, non-discriminatory and reliable public services at a fair cost.


INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

7.7.2020

 

 

 

Result of final vote

+:

–:

0:

39

1

4

Members present for the final vote

Alex Agius Saliba, Andrus Ansip, Alessandra Basso, Brando Benifei, Adam Bielan, Hynek Blaško, Biljana Borzan, Vlad-Marius Botoş, Markus Buchheit, Dita Charanzová, Deirdre Clune, David Cormand, Petra De Sutter, Carlo Fidanza, Evelyne Gebhardt, Alexandra Geese, Sandro Gozi, Maria Grapini, Svenja Hahn, Virginie Joron, Eugen Jurzyca, Arba Kokalari, Marcel Kolaja, Kateřina Konečná, Andrey Kovatchev, Jean-Lin Lacapelle, Maria-Manuel Leitão-Marques, Adriana Maldonado López, Antonius Manders, Beata Mazurek, Leszek Miller, Kris Peeters, Anne-Sophie Pelletier, Christel Schaldemose, Andreas Schwab, Tomislav Sokol, Ivan Štefanec, Kim Van Sparrentak, Marion Walsmann, Marco Zullo

Substitutes present for the final vote

Pascal Arimont, Maria da Graça Carvalho, Edina Tóth, Stéphanie Yon-Courtin

 

 


 

 

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

39

+

EPP

S&D

RENEW

GREENS/EFA

ECR

EUL/NGL

NI

Pascal Arimont, Maria da Graça Carvalho, Deirdre Clune, Arba Kokalari, Andrey Kovatchev, Antonius Manders, Kris Peeters, Andreas Schwab, Tomislav Sokol, Ivan Štefanec, Edina Tóth, Marion Walsmann

Alex Agius Saliba, Brando Benifei, Biljana Borzan, Evelyne Gebhardt, Maria Grapini, Maria‑Manuel Leitão‑Marques, Adriana Maldonado López, Leszek Miller, Christel Schaldemose

Andrus Ansip, Vlad‑Marius Botoş, Dita Charanzová, Sandro Gozi, Svenja Hahn, Stéphanie Yon‑Courtin

David Cormand, Petra De Sutter, Alexandra Geese, Marcel Kolaja, Kim Van Sparrentak

Adam Bielan, Carlo Fidanza, Eugen Jurzyca, Beata Mazurek

Kateřina Konečná, Anne‑Sophie Pelletier

Marco Zullo

 

1

-

ID

Hynek Blaško

 

4

0

ID

Alessandra Basso, Markus Buchheit, Virginie Joron, Jean‑Lin Lacapelle

 

Key to symbols:

+ : in favour

- : against

0 : abstention

 

 


 

 

 

15.7.2020

 

OPINION OF THE COMMITTEE ON TRANSPORT AND TOURISM

for the Committee on Legal Affairs

on artificial intelligence: questions of interpretation and application of international law in so far as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

(2020/2013(INI))

Rapporteur for opinion: Ondřej Kovařík

 

SUGGESTIONS

The Committee on Transport and Tourism calls on the Committee on Legal Affairs, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A. whereas artificial intelligence (AI) is providing unprecedented opportunities to enhance the performance of the transport sector by addressing the challenges of increasing travel demand, safety and environmental concerns, while making all transport modes smarter, more efficient and more convenient;

B. whereas addressing AI in defence at the EU level is indispensable for the development of EU capabilities;

1. Welcomes the Commission’s White Paper on AI and supports the EU ambition to be one of the global AI leaders while strengthening cooperation with like-minded actors;

2. Points out that with the increasing importance of research and development in the private sector and massive investments from third countries such as the United States and China, the EU is facing strong competition; supports, therefore, the EU’s efforts to further develop its competitive advantages in the AI sector and believes that it should act as a global AI norm-setter in the areas of civil and military use and of state authority, and increase efforts towards an effective strategy aimed at strengthening its influence on shaping global standards and diminishing its reliance on foreign data, which is essential for algorithm-based technologies; calls on the Commission and the Member States to advocate for broader cooperation within the UN, OECD, G7, G20 and other international fora in order to promote the EU approach to AI, emphasising the fundamental rights, freedoms and values that are enshrined in the EU Treaties, the Charter of Fundamental Rights of the European Union and international human rights law, and taking into account ethical standards and liability issues;

3. Takes note of the significant economic potential of AI applications and therefore calls on the Commission to continue promoting AI research and the exchange of good practices in the field of transport; highlights the need for innovative cross-sectoral uses of data and cooperation between different ecosystem players in order to strengthen the EU’s AI industrial base; stresses the importance of taking into consideration not only AI technologies, but also other next-generation technologies, e.g. quantum computing, which are already being considered and represent the next leap in cross-sectoral technological advancements;

4. Supports the use of machine learning AI using big data for the optimisation of long-term performance, maintenance, failure prediction and construction planning of transport infrastructure and buildings, including factors such as safety, energy efficiency or costs; notes that this will require enablers and supporting infrastructure, including energy to hardware, software, network resources and services as well as ensuring the high quality and quantity of data; notes, moreover, that the deployment of AI in transport, in particular where civil and military uses are interlinked, should be compliant with EU data protection and privacy law; calls on the Commission to continue working with the Member States on data security and protection;

5. Believes that AI has changed and has helped the development and modernisation of the transport sector through increasing automation and greater integration and connectivity of transport networks; underlines that automation and the integration of AI vary between transport modes and infrastructure in place throughout the Union; stresses the need to boost artificial intelligence to foster the multimodality, interoperability and energy efficiency of all modes of transport to enhance efficiency in the organisation and management of goods and passenger traffic flows, to make better use of infrastructure and resources along the Trans-European Transport Network (TEN-T) and to address the obstacles to the creation of a true Single European Transport Area;

6. Highlights that one of the key outcomes of the Commission’s White Paper on Artificial Intelligence is to have a human-centric approach to AI systems; recalls the continued importance of the human factor towards full automation; underlines the need for sufficient transparency and predictability, reliable technology, high-quality deployment and proper training, as well as the upskilling and reskilling of personnel using AI-based systems; underlines that the use of AI for military purposes should always lead back to a natural or legal person, with responsibility and accountability being assigned to the responsible decision makers taking into account the military chain of command;

7. Recalls the benefits of the European Rail Traffic Management System (ERTMS), a seamless automatic train protection system replacing incompatible national ones, as regards reliability, capacity, costs, safety, speed and maintenance, the full deployment of which is key to the creation of the Single European Railway Area; supports the full deployment of ERTMS and its continuous establishment as a global automatic train protection system; notes the work of the Shift2Rail Joint Undertaking;

8. Supports the development and international standardisation of the automation of train operations, also in order to promote interoperability, transport efficiency and safety; encourages the development of automated slot allocation in various transport modes and the use of AI in logistics and other areas of transport;

9. Welcomes the work of the Single European Sky ATM Research project (SESAR) in the area of unmanned aircraft systems and air traffic management systems, both civil and military, and its contribution to overcoming the fragmentation of the Single European Sky; recalls that both civil and military use of Unmanned Aerial Vehicles (UAVs), commonly referred to as drones, has increased; highlights the potential, among other, that drone deliveries, drone inspections, and drone surveillance hold for citizens and society; stresses that global interoperability and harmonisation constitute a sine qua non for a safe, functional and secure global air traffic management system; encourages the Commission and the Member States to promote SESAR internationally, to contribute to the work of international organisations such as the International Civil Aviation Organization or the International Air Transport Association in this regard, and to cooperate to set international norms for the civil and military use of drones;

10. Calls on the Commission and the Member States to participate in the international regulatory activities and discussions on autonomous vehicles, especially in the area of safety, while ensuring cooperation among regulators and all stakeholders relevant to the deployment of automated vehicles in road traffic in the EU; calls for standardised and interoperable geographic input data and stresses the need for accuracy of such data;

11. Recalls that autonomous vehicles have great potential to improve mobility, safety, and bring environmental benefits; welcomes, in this regard, the adoption by the United Nations Economic Commission for Europe’s World Forum for Harmonization of Vehicle Regulations of the framework document on automated/autonomous vehicles; notes the work of the new AI technical committee of the International Organization for Standardization; calls on the Commission to propose an effective approach to ensure a clear international legal framework for the driver concept, and relevant issues of responsibility and liability, within the meaning of the UN Conventions and other international traffic laws;

12. Points out that the global shipping industry has greatly changed thanks to AI integration in the most recent years; recalls the current comprehensive discussions in the International Maritime Organization on effectively integrating new and emerging technologies, such as autonomous ships, in the regulatory framework;

13. Stresses how intelligent transport systems mitigate traffic congestion, increase safety and accessibility and contribute to improving the management of traffic flows, efficiency and mobility solutions; draws attention to the increased exposure of traditional transport networks to cyber threats; recalls the importance of resources and further research on security risks in ensuring the safety of automated systems and their data; welcomes the Commission’ s intention to include cybersecurity as a regular agenda item for discussion within transport-related international organisations;

14. Welcomes the efforts to introduce AI systems in the public sector and will support further discussions on AI deployment in transport; calls on the Commission to carry out an evaluation of the use of AI and similar technologies in the transport sector and to compile a non-exhaustive list of high-risk segments in the context of AI systems replacing decisions within the framework of prerogatives of public power in this area;

15. Underlines that the European Defence Fund and Permanent Structured Cooperation should stimulate cooperation between Member States and European defence industries to develop new European defence capabilities in the field of AI and ensure security of supply, taking ethical considerations into account; emphasises the need to avoid fragmentation by building bridges between various actors and application domains, by promoting compatibility and interoperability at all levels and by focusing on common architectural-level work and platform solutions; recalls, moreover, that the next Connecting Europe Facility, which also encourages smart infrastructure, will provide for a fund for the adaptation and the development of civilian/military dual-use transport infrastructure on the TEN-T in order to increase synergies between civil and defence needs and with a view to improving civil and military mobility within the Union; emphasises, therefore, the need for further European investments, research, and leadership in technologies with both high economic growth impact as well as significant dual-use potential;

16. Stresses that many investments in new technologies in the field of transport and mobility are market-driven, but dual-use commercial off-the-shelf technologies and products are often used in innovative ways for military purposes; highlights, therefore, that the dual use potential of AI-enabled solutions needs to be taken into account when drafting standards for the use of AI in various areas of the commercial and military sectors; calls for high ethical standards and policy to be included in developing defence technologies, products and operating principles;

17. Points out that the effective transportation of goods, ammunition, armaments and troops is an essential component of successful military operations; stresses that AI is expected to play a crucial role and have numerous possibilities in the field of military logistics and transport; points out that different countries across the world, including EU Member States, are embedding AI weapons and other systems used on land, naval, airborne platforms; recalls that AI applications in the transport sector could provide for new capabilities and allow new forms of tactics, like the combination of many systems such as drones, unmanned boats or tanks in an independent and coordinated operation;

18. Notes, moreover, that autonomous weapons systems, as a particular category of AI in the military domain, should be discussed and agreed internationally, specifically in the UN Convention on Certain Conventional Weapons forum; draws attention to the ongoing international debate on lethal autonomous weapons systems, to regulate emerging military technologies, has so far failed to be reached and points out that the EU, as a whole, has only recently agreed to discuss the effects of AI developments and digitalisation on the defence sector; believes that the EU can play a crucial role in helping Member States in harmonising their approach to military AI, in order to lead the international discussions.


INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

14.7.2020

 

 

 

Result of final vote

+:

–:

0:

36

8

5

Members present for the final vote

Magdalena Adamowicz, Andris Ameriks, José Ramón Bauzá Díaz, Izaskun Bilbao Barandica, Marco Campomenosi, Ciarán Cuffe, Jakop G. Dalunde, Johan Danielsson, Andor Deli, Karima Delli, Anna Deparnay-Grunenberg, Ismail Ertug, Gheorghe Falcă, Giuseppe Ferrandino, Mario Furore, Søren Gade, Isabel García Muñoz, Jens Gieseke, Elsi Katainen, Kateřina Konečná, Elena Kountoura, Julie Lechanteux, Bogusław Liberadzki, Benoît Lutgen, Elżbieta Katarzyna Łukacijewska, Marian-Jean Marinescu, Tilly Metz, Giuseppe Milazzo, Cláudia Monteiro de Aguiar, Caroline Nagtegaal, Jan-Christoph Oetjen, Philippe Olivier, Rovana Plumb, Dominique Riquet, Dorien Rookmaker, Massimiliano Salini, Sven Schulze, Barbara Thaler, István Ujhelyi, Elissavet Vozemberg-Vrionidi, Lucia Vuolo, Roberts Zīle, Kosma Złotowski

Substitutes present for the final vote

Leila Chaibi, Markus Ferber, Carlo Fidanza, Maria Grapini, Roman Haider, Alessandra Moretti

 

 



 

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

36

+

ECR

Angel Dzhambazki, Carlo Fidanza, Roberts Zīle, Kosma Złotowski

NI

Mario Furore, Dorien Rookmaker

PPE

Magdalena Adamowicz, Andor Deli, Gheorghe Falcă, Markus Ferber, Jens Gieseke, Benoît Lutgen, Marian‑Jean Marinescu, Giuseppe Milazzo, Cláudia Monteiro de Aguiar, Massimiliano Salini, Barbara Thaler, Elissavet Vozemberg‑Vrionidi, Elżbieta Katarzyna Łukacijewska

Renew

José Ramón Bauzá Díaz, Izaskun Bilbao Barandica, Søren Gade, Elsi Katainen, Caroline Nagtegaal, Jan‑Christoph Oetjen, Dominique Riquet

S&D

Andris Ameriks, Johan Danielsson, Ismail Ertug, Giuseppe Ferrandino, Isabel García Muñoz, Maria Grapini, Bogusław Liberadzki, Alessandra Moretti, Rovana Plumb, István Ujhelyi

 

8

-

GUE/NGL

Leila Chaibi, Kateřina Konečná, Elena Kountoura

Verts/ALE

Ciarán Cuffe, Jakop G. Dalunde, Karima Delli, Anna Deparnay‑Grunenberg, Tilly Metz

 

5

0

ID

Marco Campomenosi, Roman Haider, Julie Lechanteux, Philippe Olivier, Lucia Vuolo

 

Key to symbols:

+ : in favour

- : against

0 : abstention

 

 


 

 

 

OPINION OF THE COMMITTEE ON CIVIL LIBERTIES, JUSTICE AND HOME AFFAIRS (23.11.2020)

for the Committee on Legal Affairs

on artificial intelligence: questions of interpretation and application of international law in so far as the EU is affected in the areas of civil and military uses and of state authority outside the scope of criminal justice

(2020/2013(INI))

Rapporteur for opinion: Patryk Jaki

 

SUGGESTIONS

The Committee on Civil Liberties, Justice and Home Affairs calls on the Committee on Legal Affairs, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

 having regard to Articles 2 and 3 of the Treaty on European Union (TEU),

 having regard to Articles 10, 19, 21 and 167 of the Treaty on the Functioning of the European Union (TFEU),

 having regard to the right to petition enshrined in Articles 20 and 227 of the TFEU and Article 44 of the Charter of Fundamental Rights of the European Union (EUCFR),

 having regard to Articles 21 and 22 of the EUCFR,

 having regard to the preamble to the TEU,

 having regard to the Council of Europe’s Framework Convention for the Protection of National Minorities, Protocol No 12 to the Convention for the Protection of Human Rights and Fundamental Freedoms, and the European Charter for Regional or Minority Languages,

 having regard to Council Directive 2000/43/EC of 29 June 2000 implementing the principle of equal treatment between persons irrespective of racial or ethnic origin[11] (Racial Equality Directive),

 having regard to Council Directive 2000/78/EC of 27 November 2000 establishing a general framework for equal treatment in employment and occupation[12] (Equal Treatment in Employment Directive),

 having regard to Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation)[13] (GDPR), and to Directive (EU) 2016/680 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, and on the free movement of such data, and repealing Council Framework Decision 2008/977/JHA[14],

 having regard to the communication of 11 December 2019 from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on the European Green Deal,

 having regard to its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics[15],

 having regard to the OECD Council Recommendation on Artificial Intelligence adopted on 22 May 2019,

 having regard to its resolution of 12 September 2018 on autonomous weapon systems[16],

A. whereas the development and design of so-called ‘artificial intelligence’, robotics and related technologies are done by humans, and their choices determine the potential of technology to benefit society;

B. whereas ethical guidance, such as the principles adopted by the High-Level Expert Group on Artificial Intelligence, provides a good starting-point but is not enough to ensure that businesses act fairly and guarantee the effective protection of individuals;

1. Acknowledges the potentials and the risks offered by the development, deployment and use of artificial intelligence (AI) for security both within the EU and in its external relations; draws attention to the fact that many Member States use AI in both the civil and the military field, and underlines the importance of the EU playing a significant and adequate role in the future international discussions on this topic;

2. Notes the regulatory advances that some Member States have already made in the AI area; emphasises the importance of a common European approach which harmonises the legal situation across Member States and gives legal certainty to those developing and deploying AI;

3. Stresses the fact that the development, deployment, use and management of AI must respect the fundamental rights, values and freedoms expressed in the EU Treaties, and calls on Member States to refrain from deploying high-risk AI systems that pose threats to fundamental rights; takes note of the publication of the Commission’s White Paper on Artificial Intelligence and encourages deeper research into the potential risk to fundamental rights of the use of AI by state authorities and by agencies, bodies and institutions of the European Union;

4. Stresses that in order to harness the opportunities offered by AI the EU should adopt an appropriate framework to mitigate the risks, ensure its ethical use, and prevent its use for malicious purposes; considers that such a framework should clearly determine appropriate liability, accountability, security and traceability regimes; underlines the importance of the EU helping to develop much-needed investment, data infrastructure, research and common ethical norms;

5. Stresses that the EU must be in the forefront of the creation of an international legal and ethical framework governing research and development, creation, use and maintenance in the area of AI that is rooted in the principles of human rights and counteracts and remedies potential risks; considers it necessary to create a clear and fair international regime for assigning legal responsibility for adverse consequences produced by these advanced digital technologies; underlines that first and foremost the aim must be to pre‑empt such consequences;

6. Calls on the Commission to facilitate research into and discourse on opportunities of using AI in disaster relief, crisis prevention, and upholding peace;

7. Points to the clear risks in decisions made by humans if solely relying on the data, profiles and recommendations generated by machines; points out that the overall design of AI systems should also include guidelines on human supervision and oversight; calls for an obligation to be imposed regarding transparency and explainability of AI applications and the necessity of human intervention, as well as other measures, such as independent audits and specific stress tests to assist and enforce compliance; stresses that such audits should be conducted periodically by an independent authority, to supervise high-risk AI applications used by state authorities or the military;

8. Notes concerns that AI, when improperly designed and developed, may lead to bias and discrimination; commits to finding regulatory and policy solutions to ensure that discrimination, rather than being reinforced, is eliminated, through the use of existing and emerging technologies;

9. Underlines that making predictions from the processing of sharing of, access to and use of data must be governed in accordance with the requirements of quality, integrity, transparency, security, privacy and control; stresses the need, throughout the development, deployment and use of AI, robotics and related technologies, to respect the EU legal framework on data protection and privacy in order to increase citizens’ safety and their trust in those technologies;

10. Observes the rapid development of AI applications to recognise unique characteristic elements, such as facial characteristics, movements and attitudes; warns about issues of invasion of privacy, non-discrimination and the protection of personal data related to the use of automated recognition applications;

11. Underlines that any decision about a natural person that is based solely on automated processing, including profiling, and which produces an adverse legal effect concerning the data subject or significantly affects that person, is prohibited under the GDPR unless authorised by Union or Member State law, subject to suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests;

12. Calls for explainability of algorithms, transparency and regulatory oversight when artificial intelligence is used by public authorities, and for impact assessments to be conducted before tools resorting to AI technologies are deployed by state authorities; calls on the Commission and the European Data Protection Board to issue guidelines, recommendations and best practices in order to further specify the criteria and conditions for decisions based on profiling and the use of AI by public authorities;

13. Calls for transparent and risk-reducing measures at international level regarding the development and use of military AI, in particular with regard to the principles of territorial integrity and non-intervention and concerning the use of force; stresses the importance of taking account of the military aspects when addressing legal and ethical issues in the European framework on AI; recalls its position on a ban on the development, production and use of lethal autonomous weapons systems; regrets that no explicit conventions exist on a global scale on the use of such weapons;

14. Notes that certain AI technologies enable the automation of information processing and action to an unprecedented scale, as in the case of mass surveillance in the civil and military domains, which poses a threat to fundamental rights and paves the way for unlawful interference with national sovereignty; calls for the scrutiny of mass surveillance activities under international law jurisdiction and enforcement standards; expresses strong concerns about some highly intrusive social scoring applications that have been developed, as being highly dangerous for respect for fundamental rights; calls for an explicit ban on the use of mass-scale social scoring by public authorities to restrict the rights of citizens; considering the decisional hegemony and control of certain private actors over the development of such technologies, calls for strengthening the accountability of such private actors under international law;

15. Invites the Commission to assess the consequences of a moratorium on the use of facial recognition systems, and, depending on the results of such an assessment, to consider a moratorium on the use of those systems by public authorities in the public space and in premises meant for education and healthcare, and on the use of facial recognition systems by law enforcement authorities in semi-public spaces such as airports, until the technical standards can be considered fully fundamental rights-compliant, the results derived are non-biased and non-discriminatory, and there are strict safeguards against misuse and that ensure the necessity and proportionality of using such technologies;

16. Considers that artificial intelligence, robotics and related technologies, including the software, algorithms and data used or produced by such technologies, regardless of the field in which they are used, should be developed in a secure and technically rigorous manner;

17. Stresses the importance of enabling the proper deployment and use of AI; calls on the Member States to provide their civil and military personnel with appropriate training in order to allow them to accurately identify and avoid discrimination and bias in datasets;

18. Notes that the use of AI in fighting crime and cybercrime could bring a wide range of possibilities and opportunities, while at the same time the principle that what is illegal offline is illegal online should continue to prevail;

19. Is deeply concerned about the deepfake technologies that allow increasingly realistic photo, audio and video forgeries to be produced that could be used to blackmail, to generate false news reports, or to erode public trust and influence public discourse; believes such practices have the potential of destabilising countries, spreading disinformation and influencing elections; calls, therefore, for an obligation for all deepfake material or any other realistically made synthetic videos to be labelled as ‘not original’ by the creator, with strict limits on their use for electoral purposes and strong enforcement; calls for adequate research in this regard to ensure that countering technologies keep pace with the malicious use of AI;

20. Stresses that the use of AI in justice could improve the analysis and collection of data and the protection of victims, and that this could be explored in research and development and accompanied by impact assessments, in particular regarding safeguards for due process and against bias and discrimination, with the precautionary principle being applied; recalls, however, that this is no substitute for human beings in terms of sentencing or decision-making;

21. Emphasises the importance of cybersecurity for AI, in both offensive and defensive scenarios; notes in this regard the importance of international cooperation and of the publication and sharing of IT security vulnerabilities and remedies; calls for international cybersecurity cooperation for effective AI use and deployment, and for safeguards against misuse of AI and cyberattacks; notes in addition the dual-use nature of IT systems (i.e. use for civil and military purposes), and of AI, and calls for its effective regulation;

22. Calls on the Member States to consider the need to provide safeguards for the use of AI, such as rules on professional ethics; calls on the Commission to establish a clear set of criteria to determine the various levels of risk in domains for which AI technologies are designed or to which they are deployed, taking into consideration existing EU legislation;

23. Emphasises the importance of verifying how high-risk AI technologies arrive at a decision; recalls that the principle of proportionality needs to be respected and that questions of causality and liability need to be clarified;

24. Recalls the importance of the principles of governance, transparency, impartiality, accountability, fairness and intellectual integrity in the use of AI in criminal justice;

25. Urges the Member States to assess the risks related to AI-driven technologies before automating activities connected with the exercise of state authority, especially in the area of justice; calls on them to consider the need to provide safeguards, such as supervision by a qualified professional and rules on professional ethics.


INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

17.11.2020

 

 

 

Result of final vote

+:

–:

0:

56

4

0

Members present for the final vote

Malik Azmani, Pietro Bartolo, Nicolas Bay, Vladimír Bilčík, Vasile Blaga, Ioan‑Rareş Bogdan, Patrick Breyer, Saskia Bricmont, Jorge Buxadé Villalba, Damien Carême, Caterina Chinnici, Clare Daly, Anna Júlia Donáth, Lena Düpont, Cornelia Ernst, Laura Ferrara, Nicolaus Fest, Jean‑Paul Garraud, Maria Grapini, Sylvie Guillaume, Andrzej Halicki, Evin Incir, Patryk Jaki, Marina Kaljurand, Assita Kanko, Fabienne Keller, Peter Kofod, Łukasz Kohut, Alice Kuhnke, Moritz Körner, Jeroen Lenaers, Juan Fernando López Aguilar, Nuno Melo, Nadine Morano, Javier Moreno Sánchez, Maite Pagazaurtundúa, Nicola Procaccini, Emil Radev, Terry Reintke, Diana Riba i Giner, Ralf Seekatz, Birgit Sippel, Tineke Strik, Ramona Strugariu, Annalisa Tardino, Tomas Tobé, Dragoş Tudorache, Tom Vandendriessche, Bettina Vollath, Jadwiga Wiśniewska, Elena Yoncheva, Javier Zarzalejos, Sophia in ’t Veld, Michal Šimečka

Substitutes present for the final vote

Malin Björk, Delara Burkhardt, Klára Dobrev, Rasa Juknevičienė, Kris Peeters, Karlo Ressler

 


 

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

56

+

PPE

Vladimír Bilčík, Vasile Blaga, Ioan‑Rareş Bogdan, Lena Düpont, Andrzej Halicki, Rasa Juknevičienė, Jeroen Lenaers, Nuno Melo, Kris Peeters, Emil Radev, Karlo Ressler, Ralf Seekatz, Tomas Tobé, Javier Zarzalejos

S&D

Pietro Bartolo, Delara Burkhardt, Caterina Chinnici, Klára Dobrev, Maria Grapini, Sylvie Guillaume, Evin Incir, Marina Kaljurand, Łukasz Kohut, Juan Fernando López Aguilar, Javier Moreno Sánchez, Birgit Sippel, Bettina Vollath, Elena Yoncheva

Renew

Malik Azmani, Anna Júlia Donáth, Fabienne Keller, Moritz Körner, Maite Pagazaurtundúa, Ramona Strugariu, Dragoş Tudorache, Sophia in ’t Veld, Michal Šimečka

ID

Nicolas Bay, Nicolaus Fest, Jean‑Paul Garraud, Peter Kofod, Annalisa Tardino, Tom Vandendriessche

Verts/ALE

Patrick Breyer, Saskia Bricmont, Damien Carême, Alice Kuhnke, Terry Reintke, Diana Riba i Giner, Tineke Strik

ECR

Jorge Buxadé Villalba, Patryk Jaki, Assita Kanko, Nicola Procaccini, Jadwiga Wiśniewska

NI

Laura Ferrara

 

4

-

PPE

Nadine Morano

GUE/NGL

Malin Björk, Clare Daly, Cornelia Ernst

 

0

0

 

Key to symbols:

+ : in favour

- : against

0 : abstention

 


INFORMATION ON ADOPTION IN COMMITTEE RESPONSIBLE

Date adopted

10.12.2020

 

 

 

Result of final vote

+:

–:

0:

16

4

0

Members present for the final vote

Gunnar Beck, Geoffroy Didier, Angel Dzhambazki, Ibán García Del Blanco, Jean-Paul Garraud, Esteban González Pons, Sergey Lagodinsky, Gilles Lebreton, Karen Melchior, Jiří Pospíšil, Franco Roberti, Marcos Ros Sempere, Stéphane Séjourné, Raffaele Stancanelli, Adrián Vázquez Lázara, Axel Voss, Marion Walsmann, Tiemo Wölken

Substitutes present for the final vote

Daniel Buda, Pascal Durand, Heidi Hautala, Emmanuel Maurel, Sabrina Pignedoli, Bettina Vollath

Substitutes under Rule 209(7) present for the final vote

Juan Ignacio Zoido Álvarez

 


FINAL VOTE BY ROLL CALL IN COMMITTEE RESPONSIBLE

16

+

EPP

Daniel Buda, Geoffroy Didier, Esteban González Pons, Jiří Pospíšil, Axel Voss, Marion Walsmann, Juan Ignacio Zoido Álvarez

RENEW

Karen Melchior, Adrián Vázquez Lázara

ID

Gunnar Beck, Jean-Paul Garraud, Gilles Lebreton

VERTS/ALE

Heidi Hautala

ECR

Angel Dzhambazki, Raffaele Stancanelli

NI

Sabrina Pignedoli

 

4

-

RENEW

Pascal Durand, Stéphane Séjourné

GREENS/EFA

Sergey Lagodinsky

GUE/NGL

Emmanuel Maurel

 

0

0

 

 

 

Key to symbols:

+ : in favour

- : against

0 : abstention

 

 

Last updated: 6 January 2021
Legal notice - Privacy policy