Much has been said about the importance of new technologies in solidifying the impacts of the Fourth Industrial Revolution during the transition to the Internet of Things (IoT), and one of the most prominent topics is the development of autonomous vehicles, which is being explored through various innovative solutions. An interesting example is LiDAR technology (an acronym for Light Radar), which is based on tracking light reflected by objects in the vicinity of an autonomous vehicle. Undoubtedly, this is an exciting technology that is being adopted for the development of autonomous cars and ships—and even drones—that do not require a driver or pilot (1).
To properly contextualize the subject, it is important to recall that it was the widely publicized testing of a self-driving Toyota Prius—an initiative carried out by Google, Inc. (the “Waymo” project (2)), as well as the Chevrolet Bolt autonomous vehicle, developed by Cruise, LLC (3), and the pilot project by the American manufacturer Tesla, Inc., that such discussions began to be conceived as commercial projects. The goal, in fact, is to achieve “Level 5” vehicle autonomy, which is considered ideal for offering autonomous vehicles to the consumer market (4). In any case, the lack of concrete definition regarding autonomy raises significant concerns about the risks (and failures) that machines classified at lower levels (ranging from “0” to “4”) may present.
It is true that the possibility of creating sophisticated machines capable of advancing societal development raises a series of ethical questions, related both to the need to ensure that such machines do not cause harm to humans and other morally relevant beings, as well as to aspects concerning the various stages of design and development in which they find themselves (5), and—of particular interest to this brief essay—to the civil liability applicable due to failures in these algorithmic systems.
The proposals from the “Waymo” and “Cruise” projects are based on the aforementioned LiDAR technology to perform mathematical calculations that estimate the distance between the vehicle and the objects surrounding it (6). In milliseconds, data is collected and processed to create a three-dimensional model of the environment (7). Based on this model, the algorithm determines whether or not to accelerate or brake the vehicle, while also controlling its average speed and any maneuvers (whether abrupt or subtle).
Naturally, one must consider a series of parameters that vary from one manufacturer to another, and among car models with different dimensions, mass, aerodynamic structure, etc. Furthermore, the fact that it is a luminance-based technology, that is, one that measures light density based on beams projected by the vehicle, reflected by the environment, and then returned and recaptured by the sensor that projected them—which uses mathematical formulas to calculate the distance between the original projection and the captured reflection— its primary use has always been the measurement of perpendicular axes (x- and y-coordinates) for morphological mapping, indicating the terrain’s relief structure based on beams emitted by geostationary artificial satellites dedicated to topographic mapping. Only now is this technology being tested for projects involving autonomous vehicles. It is in this context that questions arise regarding the safety of these systems.
Images and data collected from luminance and radar sensors make up the initial dataset that feeds the system and triggers the corresponding algorithm. From there, a 3D model is generated, initiating the actual vehicle navigation phase. It is based on this three-dimensional model (width, height, and depth)—and on new data that will be continuously collected and used to identify elements in the surroundings (the road itself, any objects, their colors, contours, etc.) —that the second stage, “interaction,” begins, during which the vehicle will process the data and start moving. At this point, horizontal and vertical traffic signs will be taken into account, as well as any objects and obstacles, such as pedestrians and other vehicles. This is the most critical stage of the process, as it is during this stage that collisions and pedestrian strikes may occur.
In the third stage, described as “reasoning” (8), cyclical decision-making routines are structured. The data reaching this stage has already been processed and filtered so that the contextual assessment of the consequences of any decision (accelerating, braking, changing course, issuing a warning, etc.) is feasible. And the cycle repeats continuously, with new data, new contexts, and new decisions. The vehicle and its physical components are merely objects, controlled by a complex algorithm (software), which will consider space, time, speed, context, the nature of other objects in the surroundings, collision risks, and potential damage (9)—and which, ultimately, will “decide” how to “react” to all of this.
It is at this stage of the process that algorithmic decisions, based on statistical predictions, are implemented (10). It is also within these decision-making cycles that potential errors can cause damage! And, between the predictable and the unpredictable, the weighable and the unweighable, there are decisions that go beyond mere heuristics… These are moral decisions.
Initiatives such as the Moral Machine, from the Massachusetts Institute of Technology (MIT), propose testing decisions made by humans in extreme “harm-or-harm” scenarios (11). An illustrative autonomous car is presented in situations in which, for example, a pedestrian crosses the road and a choice must be made between maintaining the course—which will result in the pedestrian being struck and killed—or, alternatively, swerving to save the pedestrian, but causing a collision or loss of control of the vehicle and the certain death of the passenger inside (12).
Obviously, if an error occurs—whether due to poor data collection, faulty processing, or the lack of a viable algorithmic solution to the problem—there is a risk that the decision made will be tainted by biases. In simple terms, algorithmic bias (13) refers to a flaw “in the consequence,” which can cause harm. However, it is necessary to consider the “antecedent,” that is, to investigate the causal pathway of the heuristic process so that it is possible to determine whether the flawed decision was made as a result of a failure that occurred in a previous stage, which ultimately tainted the subsequent processing stages.
In 2019, mainstream media reported that self-driving cars based on LiDAR technology were more likely to strike Black people, suggesting a possible discriminatory nature of the algorithm (14). In fact, it is common for debates involving the topic of “artificial intelligence” to generate heated controversy, including from a terminological standpoint, since we have not yet reached the speculative and dystopian moment of “technological singularity” (15) described by Vernor Vinge and Ray Kurzweil. Furthermore, the topic is riddled with inconsistencies and uncertainties, which make any regulatory effort an even greater challenge.
On December 12, 2017, the “Fundamentally Understanding the Usability and Realistic Evolution of Artificial Intelligence Act,” or simply “Future of AI Act” (16), was introduced in the United States. It is closely tied to the correlation between the concept of AI and the functioning of the human brain, indicating a conceptual proximity to the idea of “technological singularity.” This document also sets forth ethical guidelines for promoting algorithmic development but does not directly address civil liability.
Some more recent documents, such as the 2021 European Artificial Intelligence Act (17) (2021 EU AIA) and the very recent 2022 U.S. Algorithmic Accountability Act (18) (2022 US AAA)—which updated the previous 2019 version (19)—avoid terminological debates regarding the semantic scope of the term “intelligence,” preferring instead to refer to “automated decision-making systems” (20) (Automated Decision Systems, or ADSs) to make explicit the need to establish a civil liability regime applicable to harmful events caused by such systems, and even to reaffirm the importance of establishing ethical parameters for algorithm development.
According to authoritative legal scholarship (21), the cited documents possess qualities that can serve as mutual inspiration, highlighting the importance of proper semantic assimilation (in addition to other topics) for the evolution of discussions even at the global level.
In Brazil, Bills Nos. 5.051/2019, 21/2020, and 872/2021 aim to regulate the issue in general terms (and not just in the context of autonomous vehicles), prioritizing the establishment of a liability system based on the anachronistic theory of fault, which simply does not make sense for regulating such a complex matter.
However, in February 2022, the Federal Senate established the commendable “Commission of Jurists Responsible for Providing Input on the Drafting of a Substitute Bill on AI” (CJSUBIA). Several meetings and public hearings have already been held, and the deadline for drafting the substitute bill—originally set for August 9, 2022—has been extended to December 7, 2022 (22). Undoubtedly, the commission’s activities deserve high praise, and its members are extremely competent. Therefore, it is hoped that a substitute bill will be presented that more closely aligns with the solution adopted by the European Union, which has been discussing the matter proactively for years, striving to categorize possible solutions for each context based on the level of risk involved in the activity.
The example of LiDAR technology illustrates this well, since—being based on light beams and the measurement of luminance to map obstacles—its fallibilities and its predictability (foreseeability) make it possible, with absolute certainty, to identify the causal link, thereby enabling civil liability to be imposed on the manufacturer that places a vehicle based on such technology on the market, grounded in the theory of risk.
Here are some of the reasons: (i) these sensors are commonly used for topographic mapping and have only recently been applied to autonomous vehicles; (ii) the sensors’ reliance on luminance measurement carries risks of data bias in post-collection processes; (iii) the algorithm’s dependence on data quality reduces the system’s redundancy, leading to distorted results during decision-making stages and making it impossible to monitor errors and audit data; (iv) there is a lack of clarity regarding the inclusion of backdoors in LiDAR-based autonomous vehicles (such as “emergency brakes” by design), automatic shutdown features (“shut down”), or features that allow operators or users to “turn off the AI” via manual commands, or render it “non-intelligent” by pressing a panic button.
Therefore, rather than simply adopting a general regime of fault-based civil liability for events involving AI system failures, it would be more prudent for Brazil to recognize the wide range of situations with varying degrees of propensity to cause harm arising from such systems, enabling solutions tailored to the specific characteristics of each situation, as defined in the recent European document (2021 EU AIA), which opted for regulation through a risk-based approach. In the case of LiDAR, given the evident risks, liability would be strict. And this could also be the case in Brazil.
On this point, we align with the views of Nelson Rosenvald (23), who highlights, with solid arguments, the need for the substitute bill to the Brazilian legislation to move beyond the simple subjectivist model and advance this debate, broadening the understanding of risk theory to encompass multiple layers that catalyze it, such as accountability and answerability.
(1) Some initial reflections on the topic that served as inspiration for this column were drawn from an article I wrote in 2020, namely: FALEIROS JÚNIOR, José Luiz de Moura. Discriminação por algoritmos de Inteligência Artificial: a responsabilidade civil, os vieses e o exemplo das tecnologias baseadas em luminância. Revista de Direito da Responsabilidade, Coimbra, ano 2, p. 1007-1043, 2020.
(2) FINGAS, Jon. Waymo launches its first commercial self-driving car service. Engadget, 5 dez. 2018. Disponível em: https://www.engadget.com/2018-12-05-waymo-one-launches.html Acesso em: 19 set. 2022.
(3) OHNSMAN, Alan. GM's Cruise Poised To Add 1,100 Silicon Valley Self-Driving Car Tech Jobs. Forbes, 04 abr. 2017. Disponível em: https://www.forbes.com/sites/alanohnsman/2017/04/04/gms-cruise-poised-to-add-1100-silicon-valley-autonomous-car-tech-jobs/ Acesso em: 19 set. 2022.
(4) GOH, Brenda; SUN, Yilei. Tesla 'very close' to level 5 autonomous driving technology, Musk says. Reuters, 09 jul. 2020. Disponível em: https://reut.rs/3ebPPEm Acesso em: 19 set. 2022.
(5) BOSTROM, Nick; YUDKOWSKY, Eliezer. The ethics of Artificial Intelligence. In: FRANKISH, Keith; RAMSEY, William (ed.). The Cambridge Handbook of Artificial Intelligence. Cambridge: Cambridge University Press, 2014, p. 316.
(6) See NEFF, Todd. The laser that's changing the world: the amazing stories behind LiDAR for 3D mapping to self-driving cars. Nova York: Prometheus, 2018. E-book.
(7) ÖZGÜNER, Ümit; ACARMAN, Tankut; REDMILL, Keith. Autonomous ground vehicles. Boston: Artech House, 2011, p. 86-87. They explain: “A scanning laser range finder system, or LIDAR, is a popular system for obstacle detection. A pulsed beam of light, usually from an infrared laser diode, is reflected from a rotating mirror. Any nonabsorbing object or surface will reflect part of that light back to the LIDAR, which can then measure the time of flight to produce range distance measurements at multiple azimuth angles”.
(8) CHENG, Hong. Autonomous intelligent vehicles: theory, algorithms, and implementation. Londres: Springer, 2011, p. 13.
(9) SIEGWART, Roland; NOURBAKHSH, Illah. Introduction to autonomous mobile robots. Cambridge: The MIT Press, 2004, p. 90.
(10) FLASINSKI, Mariusz. Introduction to Artificial Intelligence. Cham: Springer, 2016, p. 15-22.
(11) BELAY, Nick. Robot ethics and self-driving cars: how ethical determinations in software will require a new legal framework. The Journal of the Legal Profession, Tuscaloosa, v. 40, n. 1, p. 119-130, 2015, p. 119-120. He notes: “Perhaps most notably, machines will have to make decisions regarding whom to save or protect in the event of a collision or unforeseen obstacle.”
(12) MASSACHUSETTS INSTITUTE OF TECHNOLOGY. Moral Machine. Disponível em: https://www.moralmachine.net/ Acesso em: 19 set. 2022.
(13) DANKS, David; LONDON, Alex John. Algorithmic bias in autonomous systems. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), Viena, p. 4691-4697, 2017. Disponível em: https://www.ijcai.org/Proceedings/2017/ Acesso em: 19 set. 2022.
(14) KIM, Theodore. Op-Ed: AI flaws could make your next car racist. Los Angeles Times, 7 out. 2021. Disponível em: https://www.latimes.com/opinion/story/2021-10-07/op-ed-ai-flaws-could-make-your-next-car-racist Acesso em: 19 set. 2022; HERN, Alex. The racism of technology - and why driverless cars could be the most dangerous example yet. The Guardian, 13 mar. 2019. Disponível em: https://www.theguardian.com/technology/shortcuts/2019/mar/13/driverless-cars-racist Acesso em: 19 set. 2022; VIEIRA, Laís. Carros autônomos podem atropelar mais pessoas negras do que brancas. R7, 11 mar. 2019. Disponível em: https://noticias.r7.com/tecnologia-e-ciencia/fotos/carros-autonomos-podem-atropelar-mais-pessoas-negras-do-que-brancas-11032019 Acesso em: 19 set. 2022.
(15) The “technological singularity” would be the theoretical, future moment at which the advancement and sophistication of algorithmic systems would lead to true symbiosis—and possibly indistinguishability—between the “biological” and the “technological.” For more on this topic, see VINGE, Vernor. The coming technological singularity: How to survive in the post-human era. In: Interdisciplinary Science and Engineering in the Era of Cyberspace. NASA John H. Glenn Research Center at Lewis Field, Cleveland, 1993, p. 11-22. Disponível em: https://ntrs.nasa.gov/search.jsp?R=19940022856 Acesso em: 19 set. 2022; KURZWEIL, Ray. The age of spiritual machines: when computers exceed human intelligence. Nova York: Viking, 1999. p. 213; BARBOSA, Mafalda Miranda. Inteligência artificial, e-persons e direito: desafios e perspectivas. Revista Jurídica Luso-Brasileira, Lisboa, ano 3, n. 6, p. 1475-1503, 2017, p. 1501-1502.
(16) ESTADOS UNIDOS DA AMÉRICA. House of Representatives. House Resolution No. 4625, Dec. 12, 2017. FUTURE of Artificial Intelligence Act. Disponível em: https://www.congress.gov/115/bills/hr4625/BILLS-115hr4625ih.pdf Acesso em: 19 set. 2022.
(17) EUROPA. European Commission. Artificial Intelligence Act. 2021/0106(COD), abr. 2021. Disponível em: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A52021PC0206 Acesso em: 19 set. 2022.
(18) ESTADOS UNIDOS DA AMÉRICA. House of Representatives. House Resolution No. 6580, Feb. 3, 2022. Algorithmic Accountability Act of 2022. Disponível em: https://www.congress.gov/bill/117th-congress/house-bill/6580/text Acesso em: 19 set. 2022.
(19) ESTADOS UNIDOS DA AMÉRICA. House of Representatives. House Resolution No. 2231, Apr. 10, 2019. Algorithmic Accountability Act of 2019. Disponível em: https://www.congress.gov/116/bills/hr2231/BILLS-116hr2231ih.pdf Acesso em: 19 set. 2022.
(20) See SELBST, Andrew. An institutional view of algorithmic impact assessments. Harvard Journal of Law & Technology, Cambridge, v. 35, 2021. Disponível em: https://ssrn.com/abstract=3867634 Acesso em: 19 set. 2022.
(21) MÖKANDER, Jakob; JUNEJA, Prathm; WATSON, David S.; FLORIDI, Luciano. The US Algorithmic Accountability Act of 2022 vs. The EU Artificial Intelligence Act: what can they learn from each other? Minds and Machines, Cham: Springer, v. 22, p. 1-9, jun. 2022. Disponível em: https://doi.org/10.1007/s11023-022-09612-y Acesso em: 19 set. 2022.
(22) BRASIL. Senado Federal. Atividade Legislativa. Comissão de Juristas responsável por subsidiar elaboração de substitutivo sobre IA (CJSUBIA). Disponível em: https://legis.senado.leg.br/comissoes/comissao?codcol=2504 Acesso em: 19 set. 2022.
(23) ROSENVALD, Nelson. A falácia da responsabilidade subjetiva na regulação da IA. Migalhas, 13 maio 2022. Disponível em: https://www.migalhas.com.br/coluna/migalhas-de-protecao-de-dados/365913/a-falacia-da-responsabilidade-subjetiva-na-regulacao-da-ia Acesso em: 19 set. 2022.
Article originally written on September 19, 2022