Algorithmic Explainability and Civil Liability

There is no doubt that the transitional period in which the heyday of the so-called “information society” is taking shape reveals nuances that, for legal science, necessitate the restructuring of certain theoretical foundations. That said, it is well known that algorithms are not “intelligent.” Quite the contrary: in order to function, they still rely on the past to generate heuristic predictions of a probabilistic nature (since, in short, all they do is process data), and for this simple and utterly obvious reason, they are a far cry from the intuitive, imaginative, and creative way humans act.

There are legal institutes permeated by interesting discussions arising from this. This is the case with civil liability, whose functions have long been investigated by specialized legal scholarship. Certainly, it is in this field of study that the great concern lies regarding the limits of attributing the duty to provide compensation arising from a potential failure caused by complex algorithms which, although parameterized and adequately developed according to the current state of the art, may be vectors for causing harm due to factors that are not known to be—or not to be—totally or partially foreseeable.

Accountability is being addressed in the context of algorithmic development (and its underlying risks), pointing to an undeniable tendency toward the establishment of duties that restructure the specific framework of classical legal doctrine and reinforce proactive or ex ante investigative efforts—which prevent harm—as a key element in addressing activities whose risk is inherent in the very complexity of the algorithms. To this end, in analyzing the topic, I have drawn upon (1) the proposals of three prominent researchers, namely:

• Frank Pasquale’s theoretical constructs surrounding the principle of explainability and “data-informed duties”;

• Mark Coeckelbergh’s expanded concept of explainability, which opens the door to the proposal of “relational justifications”;

• Giovanni Comandé’s multilayered accountability, based on ethical parameters legally incorporated into the regulatory debate.

Let’s take a look at them.

The term “artificial intelligence” is used by convention, although the expression itself is confusing, since it has come to encompass various meanings, resembling a multidisciplinary scientific field (2) more than a technology per se. For this reason, the definition of more precise (and, consequently, more contingent) duties leads to the notion of “foreseeability,” which—in the current state of the art—best aligns with the preventive function of civil liability.

Foreign legal scholarship uses the term “foreseeability” (3) to encapsulate this very element even in contexts where the theory of fault might make more sense (such as in investigating negligent conduct by the developer of an algorithmic system). Nevertheless, it is also recognized that we must go further in the search for an appropriate criterion to fulfill the precautionary function of civil liability in contexts of total unpredictability.

It is expected that complex algorithms will be designed through collaborative structures, in which trust permeates the relationships among professionals and corporations directly involved in all stages of development, highlighting the importance of trust for trustworthy AI, grounded in ethics, and which can be studied in light of the concept Giddens refers to as “reliability” when specifically addressing abstract principles of technical knowledge (4).

The essence of the three postulates coined by Jack Balkin (5) shows that the expected degree of exceptional diligence on the part of the agent who programs or develops an algorithm stems not only from the expectation of compliance with legislation and risk management (compliance), but also from proactive action regarding risk mitigation (ex ante accountability), in accordance with the well-known “principle of responsibility,” long advocated by Hans Jonas (6). This does not, however, diminish the importance of formulating rules of best practices and governance aimed at addressing potential harm and its mitigation (ex post accountability), taking into account the nature, scope, purpose, probability, and severity of the risks and benefits arising from the adoption of algorithmically structured systems.

The cooperative nature identified in Balkin’s proposals led Frank Pasquale to propose a “fourth law”: a robot must always indicate the identity of its creator, controller, or owner (7). This new postulate gives concrete form to what legal doctrine had already established as the “principle of explainability,” aimed at enabling a given machine to identify its creator and, where applicable, also reveal the identity of its owner or user/operator.

If risk is the central element of all these proposals, yet the notion of danger—and precaution itself—also prompts reflection on the consequences of unbridled technological development, it remains to be emphasized that “the concept, the typology, and the severity of the harms that have shaped the development of civil liability systems over time have varied in a manner proportional to the very transformation of society” (8).

Undeniably, whether risk is proven or potential does not negate the relevance of the principles of prevention and precaution, precisely because every type of “new harm” generates a certain amount of excitement and, as Ulrich Beck warns, leads to assumptions of social acceptance of new technologies9 —even if not fully tested—due to the fact that the risk, to some extent, becomes inherent in the various activities of daily life.

Pasquale’s proposal is further supported by observations made in the classic work by Stuart Russell and Peter Norvig, who already spoke of the “quantification of uncertainties”: “Agents may need to deal with uncertainty, whether due to partial observability, the non-determinism of uncertainty, or a combination of the two” (10). In summary, it can be stated that the assumptions upon which “data-informed duties” are based align with the aforementioned “fourth law of robotics” proposed by Pasquale (principle of explainability) (11), as this idea reinforces the need to overcome a problem also described by the author, in another work (12): that of algorithmic “black boxes,” typically identified by the use of machine learning techniques that are opaque or non-transparent to users (13).

In civil liability, a very close connection to the principle of explainability can be traced based on numerous proposals. One of these, by Mark Coeckelbergh, draws on “relational justifications” to broaden the notion of explainability to the point of requiring experts—given that the development of algorithms for AI systems is essential—to do so clearly, indicating that they are capable of developing them, that they wish to do so, and explaining the reasons that led them to make each decision during the development process, including during the training and testing stages of datasets (14).

An alternative proposal to the structuring of data-informed duties involves recognizing the preventive function and its operationalization based on the desirable accountability (15) for algorithm development. This idea is drawn from the writings of Giovanni Comandé, which emphasize the need to transition from traditional strict liability to a model of accountability that also addresses the preventive and precautionary functions, imposing on those who assume a higher hierarchical position with regard to assuming “data-informed” duties the responsibility to make choices and justify them to those affected by their effects.

In the information society, accountability must represent a “culture” (16) (rather than merely a “duty”), and its effects must go beyond those of mere “rendering of accounts” (as one might infer from a literal translation of the term) for the choices made. It is, therefore, the legitimate expectation that the agent will be held accountable, in the appropriate spheres (political, civil, criminal, administrative, ethical, social), for their possible failures and for the shortcomings of their choices (especially if they are informed choices) (17).

With a focus on protecting trust, the discussion on accountability addresses ethical aspects that are reinterpreted for the application of artificial intelligence systems in the field of research concerning the need for specific regulation. Rather than prescribing “how” algorithms should be created, applied, and monitored, this more open model creates specific niches in which it makes more sense to establish broader guidelines, with a sufficient degree of generality to guide technological development without depriving it of the environment conducive to free testing and pivoting.

Here, finally, is a summary of considerations to help us understand the appropriate legal framework for explainability in relation to civil liability:

1. A first observation on the subject is that, regarding the requirements imposed on algorithmic operators (18), compliance with duties of care is required, which pertain to: a.1) the choice of technology, particularly in light of the tasks to be performed and the operator’s own competencies and skills; a.2) the established organizational framework, especially with regard to adequate oversight; a.3) and maintenance, including any security checks and repairs. Failure to comply with these obligations could give rise to liability based on fault (even if presumed, with the burden of proof reversed), regardless of whether the operator is also strictly liable for the risk created by the implementation of a particular technology, since one does not deal with uncertainty or with high or excessive levels of risk.

2. With regard to producers and manufacturers, in consumer relations—including those who incidentally act as operators—the following standards of conduct must be observed, according to Benhamou and Ferland (19): b.1) design, describe, and market products in a way that allows them to fulfill their “data-informed duties,” making risks more predictable (with an emphasis on expected foreseeability); and adequately monitor the product after it has been placed on the market, in light of the characteristics of emerging digital technologies, particularly their openness and dependence on the broader digital environment, including obsolescence (whether planned or unplanned), the emergence of malware, or even their vulnerability to potential external attacks.

3. So-called supervision, understood in the context of the duty to monitor (“superior” or “hierarchical,” which may even stem from state police powers, in what Pasquale refers to as “oversight” (20)), could be achieved by conducting audits and studies of the specific algorithm, even after its release to the market. Thus, as a result of the implementation of supervised risk monitoring systems (risk assessment), one would expect the identification of anomalies and the prior configuration of systems to “alert” users to the occurrence of unexpected behaviors, as well as the observation of specific evolutionary trends derived from machine learning to “predict” anomalous behaviors. Once such monitoring is implemented, the obligation to inform potential victims arises as a duty inherent in objective good faith (21), intertwined with the principle of transparency, which is closely correlated with the principle of explainability.

4. Where feasible, it is argued that producers, manufacturers, or developers should be required to include mandatory backdoors (22) in their AI systems. Other terms for this include “emergency brakes by design,” “automatic shutdown features” (shutdown), or features that allow operators or users to “turn off the AI” via manual commands, or render it “non-intelligent” by pressing a “panic button” (23). Failure to provide such tools and options could be considered a design defect sufficient to justify liability for breach of the general duty of safety that would be imposed on them, which would open the door to claims for damages under civil liability for product defects, recognizing the algorithm itself as defective. In fact, depending on the circumstances, manufacturers or developers could also be required to “shut down” their own AI systems as part of their algorithmic monitoring and auditing tasks.

5. Similar to the post-sale duties already established in the consumer market—which consist of warnings and instructions for recalling defective products—producers/manufacturers may also assume duties of support and correction—corollaries of auditability and transparency (24)—in line with other recent developments regarding the potential obligation of software developers to update unsafe algorithms for as long as the technology remains on the market (i.e., beyond any contractual warranty period) (25).

---

1. FALEIROS JÚNIOR, José Luiz de Moura. Responsabilidade por falhas de algoritmos de inteligência artificial: ainda distantes da singularidade tecnológica, precisamos de marcos regulatórios para o tema? Revista de Direito da Responsabilidade, Coimbra, v. 4, p. 906-933, 2022. p. 907.
2. In 2020, with the publication of the 4th edition of their seminal work, Stuart J. Russell and Peter Norvig further elaborated on this conceptual problem. On this point, see RUSSELL, Stuart J.; NORVIG, Peter. Artificial Intelligence: a modern approach. 4. ed. Londres: Pearson, 2020. p. 19-20. For an even more comprehensive historical overview, a must-read is the recent and thoroughly illuminating work by WOOLDRIDGE, Michael. The road to conscious machines: The story of AI. Los Angeles: Pelican, 2020.
3. CALO, Ryan. Robotics and the lessons of cyberlaw. California Law Review, Berkeley, v. 103, p. 513-563, 2015. p. 555. The author comments: “Foreseeability remains a necessary ingredient even where liability is otherwise ‘strict’ (i.e., where no showing of negligence by the plaintiff is necessary to recovery). There will be situations, particularly as emergent systems interact with one another, wherein otherwise useful technology will legitimately surprise all involved. Should these systems prove deeply useful to society, as many envision, some other formulation than foreseeability may be necessary to assess liability”
4. According to the author, “(….) confidence in the reliability of a person or system, regarding a given set of outcomes or events, where that confidence expresses faith in the probity or love of another, or in the correctness of abstract principles (technical knowledge).” GIDDENS, Anthony. The consequences of modernity. Stanford: Stanford University Press, 1990. p. 34.
5. The central focus of this new concern must be—as it always has been—the human aspect. This conclusion follows from a reading, for example, of the classic three laws of robotics described by Isaac Asimov in the short story “Vicious Circle” from his famous collection I, Robot (ASIMOV, Isaac. Eu, Robô. Trad. Aline Storto Pereira. São Paulo: Aleph, 2014), which, in turn, inspired the American Jack Balkin to also formulate three postulates for the legal framework of this complex discussion. These have been termed by legal scholars the “laws of robotics in the age of Big Data” and are summarized as follows: (a) algorithmic operators must act as fiduciaries of information with respect to their clients and end users; (b) algorithmic operators have duties toward the general public; (c) algorithmic operators have a public duty not to engage in algorithmic nuisance. (BALKIN, Jack M. The three laws of robotics in the age of Big Data. Ohio State Law Journal, Columbus, v. 78, p. 1-45, ago. 2017. Disponível aqui. Acesso em: 02 fev. 2024.)
6. JONAS, Hans. Le principe responsabilité: une éthique pour la civilisation technologique. Trad. Jean Greisch. 3. ed. Paris: Éditions du Cerf, 1992. p. 225.
7. PASQUALE, Frank. Toward a fourth law of robotics: Preserving attribution, responsibility, and explainability in an algorithmic society. University of Maryland Legal Studies Research Papers, Baltimore, n. 21, p. 1-13, 2017. Disponível aqui. Acesso em: 02 fev. 2024.
8. VENTURI, Thaís G. Pascoaloto. Responsabilidade civil preventiva: a proteção contra a violação dos direitos e a tutela inibitória material. São Paulo: Malheiros, 2014. p. 248.
9. BECK, Ulrich. Risk society: towards a new modernity. Trad. Mark Ritter. Londres: Sage Publications, 1992. p. 6.
10. RUSSELL, Stuart J.; NORVIG, Peter. Artificial Intelligence: a modern approach. 4. ed. Londres: Pearson, 2020. p. 403, tradução livre. No original: "Agents in the real world may need to handle uncertainty, whether due to partial observability, nondeterminism, or adversaries. An agent may never know for sure what state it is in now or where it will end up after a sequence of actions".
11. PASQUALE, Frank. Toward a fourth law of robotics: Preserving attribution, responsibility, and explainability in an algorithmic society. University of Maryland Legal Studies Research Papers, Baltimore, n. 21, p. 1-13, 2017. Disponível aqui. Acesso em: 02 fev. 2024.
12. PASQUALE, Frank. The black box society: the secret algorithms that control money and information. Cambridge: Harvard University Press, 2015. p. 6-7.
13. ASARO, Peter M. A body to kick, but still no soul to damn: legal perspectives on robotics. In: LIN, Patrick; ABNEY, Keith; BEKEY, George A. (ed.). Robot ethics: the ethical and social implications of robotics. Cambridge: The MIT Press, 2011. p. 169-186.
14. COECKELBERGH, Mark. Artificial intelligence, responsibility attribution, and a relational justification of explainability. Science and Engineering Ethics, Cham, v. 26, p. 2051-2068, 2020. p. 2066.
15. The term “responsibility” does not have a single meaning. Its scope is even broader in languages such as French or Spanish, in which “responsibility” is used in relation to a very broad range of legal, political, and economic relationships and, within them, to their respective dimensions. In English, however, the existence of different terms to refer to the various dimensions of responsibility—responsibility, accountability, liability—allows for a more precise application of the concept. On this point, see ROSENVALD, Nelson. A polissemia da responsabilidade civil na LGPD. Migalhas de Proteção de Dados, 06 nov. 2020. Disponível aqui. Acesso em: 02 fev. 2024.
16. NISSENBAUM, Helen. Accountability in a computerized society. Science and Engineering Ethics, Cham, v. 2, n. 1, p. 5-42, mar. 1996. p. 7.
17. COMANDÉ, Giovanni. Intelligenza Artificiale e responsabilità tra liability e accountability: il carattere trasformativo dell'IA e il problema della responsabilità. In: NUZZO, Antonio; OLIVIERI, Gustavo (a cura di). Analisi giuridica dell'Economia. Studi e discussioni sul diritto dell'impresa. Bolonha: Il Mulino, 2019, v. 1. p. 185.
18. BALKIN, Jack M. The path of robotics law. California Law Review Circuit, Berkeley, v. 6, 45-60, jun. 2015. p. 52. The author is emphatic: “We might hold many different potential actors liable, including the owner, operator, retailer, hardware designer, operating system designer, or programmer(s), to name only a few possibilities.”
19. BENHAMOU, Yaniv; FERLAND, Justine. Artificial Intelligence & damages: assessing liability and calculating the damages. In: D'AGOSTINO, Pina; PIOVESAN, Carole; GAON, Aviv (ed.). Leading legal disruption: Artificial Intelligence and a toolkit for lawyers and the law. Toronto: Thomson Reuters Canada, 2021. p. 196-197.
20. PASQUALE, Frank. New laws of robotics: defending human expertise in the age of AI. Cambridge: Harvard University Press, 2020. p. 99.
21. WISCHMEYER, Thomas. Artificial Intelligence and transparency: Opening the black box. In: WISCHMEYER, Thomas; RADEMACHER, Timo (ed.) Regulating Artificial Intelligence. Cham: Springer, 2020. p. 76.
22. LIAO, Cong; ZHONG, Haoti; SQUICCIARINI, Anna et al. Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation. Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy, p. 97-108, mar. 2020. Disponível aqui. Acesso em: 02 fev. 2024.
23. BENHAMOU, Yaniv; FERLAND, Justine. Artificial Intelligence & damages: assessing liability and calculating the damages. In: D'AGOSTINO, Pina; PIOVESAN, Carole; GAON, Aviv (ed.). Leading legal disruption: Artificial Intelligence and a toolkit for lawyers and the law. Toronto: Thomson Reuters Canada, 2021. p. 196-197.
24. PASQUALE, Frank. Data-informed duties in AI development. Columbia Law Review, Nova York, v. 119, p. 1917-1940, 2019. p. 1937.
25. In fact, although no law clearly contains a clear and explicit obligation to do so, there are already judicial indications that interpret existing legal norms in a way that creates such an obligation for developers; this is the case in the Dutch precedent Consumentenbond v. Samsung. For a detailed study of the case and its implications, see: WOLTERS, Pieter T. J. The obligation to update insecure software in the light of Consumentenbold/Samsung. Computer Law & Security Review, Londres, v. 35, n. 3, p. 295-305, maio 2019.