EU AI 규칙안상의 허용할 수 없는 위험의 AI 시스템에 대한 고찰 = Consideration of AI Systems for Unacceptable Risks under the Proposed EU AI Regulation

다국어 초록 (Multilingual Abstract)

The proposed EU AI rules categorize AI risks into unacceptable, high, low or minimal according to the level, and propose risk management proportional to the level of risk(regulations by level of risk). The risk level of the AI system is evaluated by taking into account the purpose of intentional use of the AI system, its degree of use, its impact on health, safety, and basic rights, its intensity and scope of damage, and its likelihood of recovery. Therefore, the risks are divided into four stages, prohibiting the use of AI systems that can lead to unacceptable risks, and proposing the principle that high risk AI systems allow when meeting requirements.
Since the EU's proposed AI rule sets a geographical scope that can be applied to Korean companies, it is required to clarify whether any AI systems that it is researching and developing or selling are included in this scope.
The concept of the so-called unacceptable and dangerous AI system in the proposed EU AI Regulation is expected to have a significant impact on our AI-related industrial and legal policies. In particular, countermeasures against the risks posed by AI to fundamental human rights, namely the protection of personal data and privacy and non-discrimination, must be completed as soon as possible.
In addition, the following risks must be addressed: (i) AI must be used by government agencies, etc. for large-scale monitoring purposes; (ii) AI must be used by employers to monitor employee behavior; (iii) AI must be used to reacknowledge anonymized data; (iv) online intermediaries (platforms) to prioritize user company information or adjust content.

국문 초록 (Abstract)

EU AI 규칙안에서는 AI 위험을 수준에 따라 unacceptable, high, low or minimal로 구분하고 위험수준에 비례하는 위험관리를 제안하고 있다(위험수준별(Risk-based) 규제).
AI 시스템의 의도된 사용목적, 사용정도, 건강·안전·기본권에 미치는 영향, 피해의 강도 및 범위, 피해복원 가능성 정도 등을 고려하여 AI 시스템의 위험수준을 평가하고 있다. 그에 따라 그 위험은 4단계로 구분하고, 허용할 수 없는 위험(unacceptable risk)을초래할 수 있는 AI 시스템의 활용은 금지하고, 고위험(high risk) AI 시스템은 요구사항충족 시 허용하는 원칙을 제안하고 있다.
EU의 AI 규칙안은 우리나라 기업에도 적용될 수 있는 지리적 적용범위를 정하고 있기 때문에 자사가 연구개발 또는 판매하고 있는 AI 시스템 중에 이 적용범위에 포함될수 있는 것이 있는지를 밝혀내는 것이 요구된다.
EU AI 규칙안상의 이른바 허용할 수 없는 위험의 AI 시스템이라는 개념은 AI와 관련한 산업 및 법정책에 미치는 영향이 지대할 것으로 기대된다. 특히, AI가 초래할 위험과관련하여 기본적 인권에 대한 위험(개인 데이터와 프라이버시 보호, 비차별을 포함)의 문제에 대해서는 빠른 시일 내에 이에 대한 대응책이 완비되지 않으면 안 될 것이다. AI가정부기관 등에 의해서 대규모 감시의 목적으로 이용되거나 고용주에 의해서 종업원의행동을 감시하는 목적으로 이용되는 것, AI가 익명화된 데이터(de-anonymise data)의 재인식에 이용되는 것, 온라인 중개 사업자(플랫포머)에 의해서 사용자 기업의 정보를 우선시키거나 컨텐츠의 조정을 실시하기 위해서 이용되는 것, 인간의 의사결정과 같은 편견이 AI에 존재할 수 있는 것 등의 위험에 대해서 대응하지 않으면 안 될 것이다.

참고문헌 (Reference)

1 経済産業省, "｢我が国のAIガバナンスの在り方 ver. 1.0(AI社会実装アーキテク チャー検討会 中間報告書)｣の意見公募手続(パブリックコメント)を開始しました"

2 이경선, "초점 EU 인공지능 규제안의 주요 내용과 시사점" 정보통신정책연구원 2021

3 김현수, "주요국의 혁신성장 정책과 제도: 미국, 유럽, 일본을 중심으로 제2편 주요국의 혁신성장 제도" 대외경제정책연구원 2019

4 김자회, "자율무기시스템(AWS)에 대한 국제법적 규제 : 새로운 협약도입의 필요성을 중심으로" 2019

5 이현종, "인공지능시대의 민사소송 - 온라인분쟁해결 시스템의 도입을 중심으로 -" 한국민사소송법학회 25 (25): 1-51, 2021

6 김한균, "인공지능기반 범죄피해자보호의 정책적 가능성과 법적 과제" 한국피해자학회 29 (29): 23-42, 2021

7 오시진, "인공지능과 국제인권법 - 표현의 자유, 차별, 프라이버시를 중심으로 -" 법학연구소 (91) : 239-281, 2021

8 김기중, "인공지능(AI) 개발과 활용에서의 인권 가이드라인 연구(최종보고서)" 국가인권위원회 2021

9 정종구, "인공지능 시대의 개인정보 개념에 대한 연구 — 자연어 처리를 중심으로 —" 법학연구소 38 (38): 213-241, 2021

10 강미영 ; 정혜진, "인공지능 기반의 CCTV에 의한 범죄예측 시스템 도입에 따른 법적 과제와 활용 방안" 비교법학연구소 66 : 195-232, 2022

11 박혜성 ; 김법연 ; 권헌영, "인공지능 규제에 대한 연구 ― 유럽연합의 입법안을 중심으로 ―" 한국공법학회 49 (49): 349-374, 2021

12 김진우, "유럽연합의 인공지능 백서에 관한 고찰" 법학연구소 44 (44): 129-161, 2020

13 이보연, "유럽연합의 인공지능 관련 입법 동향을 통해 본 시사점" 법학연구원 43 (43): 5-35, 2019

14 손민숙, "유럽연합, 인공지능법 초안 발표" KIRI 2021

15 박현정 ; 서정범, "감시카메라설치에 관한 경찰법적 문제- 공공건물에서의 감시카메라설치의 문제를 중심으로 -" 경찰대학 14 (14): 31-50, 2014

16 JETRO(日本貿易振興機構), "欧州委、AIに関する方針を発表"

17 小泉雄介, "欧州AI規則案の概要"

18 European Commission, "White Paper On Artificial Intelligence - A European approach to excellence and trust"

19 European Commission, "White Paper On Artificial Intelligence - A European approach to excellence and trust"

20 EUROPEAN COMMISSION, "WHITE PAPER On Artificial Intelligence - A European approach to excellence and trust"

21 EUROPEAN COMMISSION, "WHITE PAPER On Artificial Intelligence - A European approach to excellence and trust"

22 Joy Buolamwini, "Timnit Gebru; Proceedings of the 1st Conference on Fairness, Accountability and Transparency" 81 : 77-91, 2018

23 Dee, Michael J., "The use of CCTV to police public spaces : a case of big brother or big friend?" 2000

24 Ottavio Marzocchi, "The protection of Article 2 TEU values in the EU, European Parliament"

25 Elizabeth E. Joh, "The New Surveillance Discretion: Automated Suspicion, Big Data, and Policing" 10 : 15-, 2016

26 European Commission, "The GDPR: new opportunities, new obligations" European Union 2018

27 Carlos Alvarenga, "The EU’s new rules will completely rewrite the AI playbook"

28 TOM SIMONITE, "The Best Algorithms Struggle to Recognize Black Faces Equally"

29 CIARA MURPHY, "The Advent of Gender Neutral Insurance Premiums"

30 EUROPEAN COMMISSION, "THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS" 19 (19): 2020

31 European Commission, "Speech by Executive Vice-President Vestager at the press conference on fostering a European approach to Artificial Intelligence"

32 Yamin Durrani, "Smart Surveillance Cameras Bring Human Intelligence to the Workplace and Home"

33 EUROPEAN COMMISSION, "Shaping Europe’s digital future"

34 IFSEC GLOBAL, "Role of CCTV Cameras: Public, Privacy and Protection"

35 Samantha Besson, "Reconstructing International Law starting from Regional Organizations, Revue europeenne du droit"

36 EUROPEAN COMMISSION, "REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL AND THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE Report on the safety and liability implications of Artificial Intelligence, the Internet of Things and robotics"

37 EUROPEAN COMMISSION, "Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS"

38 EUROPEAN COMMISSION, "Proposal for a REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS"

39 Ministry of Justice, "Press release : New act to increase opportunities for camera surveillance"

40 Brian Clifton, "Predicting Financial Crime: Augmenting the Predictive Policing Arsenal"

41 Mark MacCarthy, "Machines learn that Brussels writes the rules:The EU’s new AI regulation"

42 CJEU, "Joined Cases C-92/09 and C-93/09, Volker und Markus Schecke and Eifert GbR and Hartmut Eifert, Opinion 51 of Advocate General Sharpston"

43 European Commission, "Impact Assessment of the Regulation on Artificial intelligence"

44 "IT セキュリティ対策, 社会信用システムと信用スコア"

45 Aharon Barak, "Human dignity as a framework right (mother-right)" Cambridge University Press 2015

46 "Harry Brignull's 90 Percent of Everything Dark Patterns: dirty tricks designers use to make people do stuff"

47 FRA, "Handbook on European data protection law" 52 Publications Office 2021

48 European Data Protection Board, "Guidelines 3/2019 on processing of personal data through video devices(EDPB Plenary meeting), 09-10 July 2019, European Digital Rights, Use cases: Impermissible AI and fundamental rights breaches"

49 Human Rights Committee, "General comment No. 37 (2020) on the right of peaceful assembly (article 21)"

50 個人情報保護委員会, "GDPR（General Data Protection Regulation：一般データ保護規則）"

51 FRA, "Facial recognition technology: fundamental rights considerations in the context of law enforcement" Publications Office 2021

52 Theodore Christakis, "Facial Recognition - Related Provisions in the Draft AI Regulation"

53 European Union Agency for Fundamental Rights, "FRA Europa" Publications Office of the European Union 2016

54 The White House, "Executive Order on the Establishment of the White House Office of Faith-Based and Neighborhood Partnerships"

55 Spyraand Kranzkow, "European Court of Human Rights, Guide on Article 8 of the European Convention on Human Rights"

56 Ivan Marc, "Europe: Artificial Intelligence Act must protect free speech and privacy"

57 小泉雄介, "EU・英国・米国におけるAI倫理動向"

58 지능정보원(NIA), "EU 인공지능법(안) 의 주요 내용과 시사점" 2021

59 손영화, "EU AI 규칙안에 대한 일고찰" 인하대학교 법학연구소 1 (1): 2021

60 Frederik Zuiderveen Borgesius, "Discrimination, Artificial Intelligence, and Algorithmic" Directorate General of Democracy 2018

61 "Directive 2006/54/EC of the European Parliament and of the Council of 5 July 2006 on the implementation of the principle of equal opportunities and equal treatment of men and women in matters of employment and occupation (recast)"

62 European Union Agency for Fundamental Rights, "Data quality and artificial intelligence – mitigating bias and error to protect fundamental rights"

63 Vienna, Austria, "Can Surveillance Cameras Be Successful in Preventing Crime and Controlling Anti-Social Behaviours?" 2001

64 Haley Samsel, "California Becomes Third State to Ban Facial Recognition Software in Police Body Cameras"

65 EUROPEAN COMMISSION, "COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS" 19 (19): 2020

66 Nikolas DeCosta-Klipa, "Brookline becomes 2nd Massachusetts community to ban facial recognition, Cambridge Votes To Ban Face Surveillance Technology" WGBH

67 Sara Merken, "Berkeley Bans Government Face Recognition Use"

68 Eve Gaumond Friday, "Artificial Intelligence Act: What Is the European Approach for AI?"

69 Eve Gaumond, "Artificial Intelligence Act: What Is the European Approach for AI?"

70 Jeffrey Dastin, "Amazon scraps secret AI recruiting tool that showed bias against women"

71 Aleš Završnik, "Algorithmic justice: Algorithms and big data in criminal justice settings" 18 (18): 2021

72 Alexander Ottosson, "Accessing Justice in the Age of AI, Digital Freedom Fund"

73 김송옥, "AI 법제의 최신 동향과 과제 - 유럽연합(EU) 법제와의 비교를 중심으로 -" 한국비교공법학회 22 (22): 115-148, 2021

74 Ian Johnston, "AI Robots Learning Racism, Sexism and other Prejudices from Humans, Study Finds"

75 Eleanor Spaventa, "9 Fundamental rights in the European Union"

76 Tamara K. Hervey, "7 - Rights: health rights as human rights, Part II - EU internal health law: the individual focus" Cambridge University Press 2015