Roger Clarke's Web-Site
© Xamax Consultancy Pty Ltd, 1995-2025
Infrastructure
& Privacy
Matilda
Roger Clarke's Web-Site© Xamax Consultancy Pty Ltd, 1995-2025 |
|
|||||
| HOME | eBusiness |
Information Infrastructure |
Dataveillance & Privacy |
Identity Matters | Other Topics | |
| What's New |
Waltzing Matilda | Advanced Site-Search | ||||
PrePrint of 6 April 2025
Forthcoming in Computer Law & Security Review
© Xamax Consultancy Pty Ltd, 2024-25
Available under an AEShareNet 
licence or a Creative
Commons 
licence.
This document is at http://rogerclarke.com/EC/RGAI-C.html
The quest for Artificial Intelligence (AI) has comprised successive waves of excessive enthusiasm followed by long, dispirited lulls. Most recently, during the first 3-4 years of public access to Generative Artificial Intelligence (GenAI), many authors have bought into the bullish atmosphere, replaying consultancies' predictions about gold mines of process efficiency and innovation. A more balanced approach to the technology is needed. Instances of apparently positive results need calm analysis, firstly to distinguish mirages from genuine contributions; secondly, to identify ways to effectively exploit the new capabilities; and thirdly, to formulate guidance for the avoidance and mitigation of negative consequences.
This article's first contribution is to ground the evaluation of GenAI's pathway, applications, impacts, implications and risks in a sufficiently deep appreciation of the technology's nature and key features. A wide range of sources is drawn on, in order to present descriptions of the processes involved in text-based GenAI. From those processes, 20 key characteristics are abstracted that together give rise to the promise and the threats GenAI embodies.
The effects of GenAI derive not from the technological features alone, but also from the patterns within which it is put to use. By mapping usage patterns across to domains of application, the phenomenon's impacts and implications can be more reliably delineated. The analysis provides a platform whereby the article's final contribution can be made. Previously-formulated principles for the responsible application of AI of all kinds are applied in the particular context of GenAI.
After a long gestation period, widespread availability of a new form of Artificial Intelligence (AI) became publicly available during 2022-23. The most common term currently used for it is 'Generative AI'. This article adopts the short form 'GenAI'. The term 'GenAI artefact' refers to a product, generally software, embedded in hardware, but commonly accessible remotely, which embodies GenAI technology. Viewed as a black box, a GenAI artefact receives textual input from a person, in the form of a question or request, and provide a synthetically-generated textual response to it. Several architectures exist and more are emerging. The dominant form during the early period is that called Generative Pre-trained Transformer (GPT). GPT is therefore a useful generic descriptor of contemporary GenAI artefacts. It also also, however, strongly associated with the most prominent instance, Open-AI's ChatGPT.
Textual responses from ChatGPT evidence apparent authenticity, most commonly in the form of smoothly constructed sentences of moderate complexity, in communications styles requested or implied by the requestor. From the time of ChatGPT's release, the quality of its expression startled people. This encouraged users to impute considerable authority and even veracity to the content. The believability of its output in response to simple requests resulted in many users suspending disbelief, and assuming that responses were generally reliable for both further simple questions and less trivial requests.
A wide variety of potential benefits of the technology have been asserted by its proponents (e.g. McAfee et al. 2023, Holmstr?m & Carroll 2024). This has excited 'goldrush', 'bandwagon' behaviours. Media reports abound conveying that ChatGPT and some other GenAI artefacts have been experimented with and very rapidly adopted by many organisations and individuals and for a wide range of tasks. Some reports also convey that their use is accompanied by a substantial lack of scepticism, and that the use of their outputs commonly lacks quality assurance. The work reported here is motivated by the need to identify and address the risks arising from uncritical adoption of GenAI.
The standpoint adopted by the author is that of an Information Systems Analyst whose intention is to provide an accessible description and analysis of the technology, in a form reasonably accessible to readers whose interests are in responsible application of technology or the design of appropriate regulatory measures to address consequences of the technology.
The technologies underlying GenAI are being applied to data in a wide variety of forms, including text, programming code, audio/sound including the human voice, static and animated diagrams, computer-generated images (CGI), and series of CGI images intended for rapid display as synthesised video. The focus of this article, however, is solely on source materials that are textual, giving rise to responses that are also in textual form. Textual applications are particularly crucial, because output from GenAI artefacts is being casually substituted for the results of human efforts to understand problems and formulate solutions. To date, however, little insight is being sought or gained in relation to the very different strengths and weaknesses of GenAI technology in comparison with human capabilities and frailties that are well understood and risk-managed.
Concerns have been expressed about the direct impacts of specific uses, and the indirect implications of various categories of application. Issues include bias, discrimination, disinformation and privacy (Baldassarre et al. 2023), but also broader matters such as cultural values, trustworthiness and autonomy, inequality, marginalisation, violence, concentration of authority, and ecosystem and environment (Solaiman et al. 2023). A key concern is the undermining of established mechanisms for righting wrongs, and for assigning legal and financial responsibility for harm caused. Many organisations and individuals are working on ways in which societies and economies can deal with the GenAI phenomenon. This paper is a contribution to that process. It draws on prior work in related areas in order to propose a comprehensive set of principles for the responsible application of GenAI.
Section 2 presents a sufficiently rich model of the category of artefacts, based on descriptions of the technologies and resources underlying them. Section 2.1 provides a brief overview of the technology, sufficient to enable readers who wish to do so to move directly to the main body of the paper in sections 3-6.
In section 2.2, the technology is presented and discussed in considerably greater detail. The purpose of doing so is to provide a firm foundation for the analysis, in the form of a comprehensive appreciation of the technology's context, data-sources and techniques. Section 3 applies the insights gained from that analysis to identify characteristics of GenAI that are likely to have harmful consequences. In section 4, patterns of use of GenAI are reviewed, revealing additional risks of the technology-in-use. Consequences of the application of GenAI are summarised in section 5. Prior work on the responsible application of AI generally is drawn upon in section 6, in order to propose a set of principles. It is argued that, if respected, or if codified and enforced, these principles would address the negative impacts, implications and risks, while enabling delivery of the technology's achievable benefits.
Many discussions of GenAI give quite cursory consideration to the underlying technology. This article, on other hand, takes up the challenge of establishing a sufficient depth of understanding of the mechanics of GenAI to support reliable assessment of its impacts. This section draws on academic, technical and, where appropriate, populist sources in order to present accessible explanations of what GenAI does, and how it does it. Section 2.1 provides an intentionally simplistic overview of the whole, sufficient for the reader to follow through the analyses and proposals in sections 3-6. Section 2.2. then delves into greater depth on each element, in order to establish a platform for the subsequent analysis.
Many authors have endeavoured to explain GenAI in a manner that will convey adequately to whatever audience they have chosen to target. Many explanations deliver only impressionistic or superficial senses of the artefacts and what they do, failing to distinguish definitional aspects of the technology and instead describing its application, or emphasising its claimed or anticipated effects. Some are so heavily imbued with aspirational and/or inspirational marketing-speak that such value as they contain is masked. Here are three definitions that achieve a degree of understandability by a layperson while remaining reasonably consistent with the underlying technology:
"generative models encode a simplified representation of their training data and draw from it to create a new work that's similar, but not identical, to the original data" (IBM 2023)
"Generative AI models use neural networks to identify the patterns and structures within existing data to generate new and original content" (Nvidia 2024)
"Generative AI uses a computing process known as deep learning to analyze patterns in large sets of data and then replicates this to create new data that appears human-generated" (Crouse 2024)
A GenAI artefact can be regarded as a combination of two inter-related components, as depicted in Figure 1:
In Figure 1, the boundaries of the two components are depicted with curved rather than straight-line forms, because the internal architectures of existing GenAI artefacts vary, and continue to mature and change. For example, a response elicited from ChatGPT in October 2024 evidenced overlap between the functions of "generating coherent text" and "generating responses":
[ ChatGPT handles ] the conversation with you, interpreting your questions and generating responses. The underlying LLM does the heavy lifting of understanding language patterns, context, and generating coherent text based on the input it receives. [ ChatGPT's ] role is to facilitate interaction and provide responses, while the LLM provides the language processing capabilities.
To generate the resource, the LLM processes considerable, and even vast, quantities of source-texts to produce a pre-stored, structured and highly-compressed representation of each text. This requires a great deal of storage, a great deal of computing power - and hence uses a great deal of energy, plus technology and materials to cool the heavily-worked and very dense silicon componentry. As a result, this function is currently performed in specialised installations, and that arrangement may well continue. At present, much of the foreground function also runs remotely from the user. However, that code requires less storage and is less computationally intensive, so it is feasible for more of it to run locally to the user.
The previous sub-section provides sufficient information to enable the reader to follow the analysis and argument presented in later sections. The purpose of this sub-section is to provide a comprehensive but nonetheless reasonably accessible rendition of the technical characteristics of GenAI artefacts relevant to the potential negative consequences. It draws on a wide range of sources, including OICT (2023), ASD (2024), Al-Amin et al. (2024) and WIPO (2024), to present an explanation at a deeper level, seeking to provide a bridge between superficial and technical descriptions. This is used in section 3 to identify characteristics of GenAI that have significance for the analysis of the technology's impacts and implications. The functions of a GenAI artefact are outlined under six sub-headings, starting with the two functions of an LLM that establish the resource needed by the Chatbot, and then describing the sequence of functions performed by the Chatbot when a user submits a request.
Language modelling is the underlying means whereby GenAI is able to deliver responses. Effective modelling depends on the acquisition of suitable and suitably large collections of source-texts. A model may be devised to address a particular problem-domain, such as corporate financial information (as the Bloomberg GenAI does). A specialist GenAI artefact in, say, the diagnosis and treatment of melanomas, would of course need to be built from scientific and clinical information in that field; and one on environmental regulation would depend on access to statutory and case law, codes, and commentaries in that field. To date, however, the focus of activity has been general-purpose GenAI artefacts, which draw on a very wide catchment of documents.
The nature of the existing data collections on which GenAI depends is a key factor underpinning the quality or effectiveness of the result. General-purpose GenAI artefacts draw on the vast pools of data that have become available during the 'big data' era of the last few decades (Clarke 2016). This includes such collections as web-pages, research publications, media collections and social media (Kaubre 2024, Turing 2024). The responses provided to users may be fairly similar to one or some existing sources in the data collections, with differences arising variously from the use of multiple sources, the expression of the request, output-styling, and arbitrary changes made to avoid repetition in responses.
The acquisition process described above is in effect indiscriminate rather than targeted. An important outcome of this approach to acquiring source-texts is that no quality safeguards exist other than any relevant filters that may have been applied to each collection that is being appropriated. For example, an open-source refereed journal publishes only those articles submitted to it after they have been passed by one or more layers of reviewers and editors, and it withdraws or qualifies any articles whose appropriateness is subsequently brought into material question. Publicly-accessible news outlets exercise varying degrees of self-control, from what were called in the 20th century 'quality broadsheets' to 'tabloid' / 'yellow press' outlets. The content of social media on 'tech platforms' such as those operated by Meta, on the other hand, is largely devoid of any meaningful quality control.
The acquisition process has inherited a feature from the data mining / data analytics field, which is most usefully described as 'data scrubbing'. The intention of this step is to identify and correct errors in a cost-effective manner, entirely or at least substantially by automated means. The point of reference of data scrubbing, however, rather than being a lack of correspondence with reality, is the existence of anomalies within the data-set. For example, Chu & Ilyas (2016) are concerned only with duplicate records, and with integrity constraints, by which those authors mean data quality rules that the database should conform with (which are breached by, for example, missing values in mandatory data-items, and values in data-items that are not in the set of permitted values). Some methods discard or adjust outliers -- which, in many contexts, deprives the collection of some of its most valuable content. The circumspect term 'scrubbing' has been largely displaced in recent literature by the terms 'data cleaning' and 'data cleansing', and this despite the technique's limited scope, and the substantial technical challenges it involves. The bold new terms mislead users into an expectation that the desired outcome of the 'scrubbing' has been achieved. It seldom has been, not least because the techniques are self-referential, with no linkage to real-world phenomena.
Moreover, the large majority of guidance in relation to data scrubbing relates to structured data. When text is embedded in row-and-column data-structures, it is commonly limited to constrained vocabularies, and the tests applied to it are entirely syntactic. Techniques developed for text of that nature is if limited value when dealing with large text-collections. Further, some of the 'cleansing' actions taken in relation to text-sources remove or substitute content on the basis of words or expressions judged on some grounds to be objectionable (OpenAI 2024). Such 'moderation' processes may (but may not) achieve the intentions of the designer. They also deny access to information or warp the intention of content-utterer, and they may not be consistent with user needs. This censorship of content has become more apparent as LLMs and GenAI artefacts have proliferated. It was prominent in the responses of the predominantly US first-mover service-providers when the Chinese GenAI artefact DeepSeek launched. DeepSeek was quickly found to exclude content on topics and views that the PRC government treats as illegitimate (Booth & Milmo 2025).
A further consideration is the degree of opaqueness rather than transparency to users as to the text-sources that are in the collection as a whole, and the specific text-sources that have been drawn on in preparing any particular response. At least during its early iterations, ChatGPT studiously avoided declaring its sources, even when express requests were made to do so. Some other GenAI artefacts have addressed that deficiency, notably Perplexity, not by adapting the underlying LLM, but by complementing the LLM with content separately acquired through web-searches (Hobson 2024).
Since GenAI became publicly available in 2022, an additional factor has arisen. Because text-sources are commonly loaded into LLMs in an indiscriminate manner, it is inevitable that some text synthesised by GenAI artefacts is played back into the corpus, as source-material for the generation of future responses. (Empirical support for this speculation is emergent, with Cheng et al. 2024 presenting an assessment of the proportion of AI-generated content in preprint platforms).
These factors alone make it unwise to assume that the data collection from which a GenAI artefact produces responses is fit for the user's specific purpose when they make any particular request.
The term 'encoding' is used to refer to the production of structured representations of source-texts. An understanding of key aspects of the underlying LLM is essential, as a basis for appreciating some attributes of chatbot responses that are variously helpful to and threatening to the interests of stakeholders in the process.
The nature of LLMs can be best appreciated by first considering an underlying AI technique called artificial neural networks (ANNs) (Hardesty 2017). The technique was inspired by the structure of biological networks within animal brains, which feature nodes inter-connected by pathways. These may be simple networks with only an input layer connected forward to an output layer, or may feature one or more intervening layers. These are commonly used for what are referred to as 'deep learning' models (Doane 2024).
A popular application of the technique is to Machine Learning (ML or AI/ML). AI/ML generates inferences about whether a particular phenomenon, as represented by data, belongs to some category of phenomena. It involves two steps:
In one particular business application of AI/ML, the instances comprise data relating to a loan application, and the resulting classification is a Yes/No decision about whether to lend or not, or a risk-rating applied to a Yes decision. A (somewhat sceptical) expression of the nature of the classification process is that AI/ML based on neural networks involves feeding in lots of pictures of cats, to produce an artefact that (in some sense) learns to distinguish images of cats from images of non-cats (Clark 2012).
GenAI also applies ANNs, but with a different kind of outcome. Rather than the classification of instances, GenAI's purpose is the generation of a new instance that has characteristics that match a user's request. Table 1 summarises key differences between AI/ML's and GenAI's uses of ANN techniques.
|
AI/ML
|
GenAI
|
Purpose
|
Classification of an Instance
|
Generation of an Instance
|
Stimulus
|
A Further Instance
|
A Request
|
Output
|
A Category or a Rating
|
Textual Statements
|
Training-Set
|
Curated
|
Gathered Indiscriminately
|
?Learning
|
Primarily Supervised
|
Primarily Unsupervised
|
One key difference that has significance for the analysis being conducted here relates to the approach adopted to ingesting source-data, in order to establish the values within the neural network. The approach adopted in AI/ML has commonly been to ingest and pre-process a more or less curated 'training-set'. This comprises a sufficiently large volume of data representing instances of a general class of a thing. GenAI, on the other hand, is based on data collections that are very large, and are not curated but instead gathered indiscriminately.
In AI/ML, it is common to apply ANNs using what is referred to as 'supervised learning'. With this approach, the designer imposes a moderate degree of structure on the nodes and connections within the network of nodes. An alternative approach is 'unsupervised learning', whereby the software infers or invents nodes, based on measures of similarity among the instances fed to it as training data. This requires large numbers of instances to be processed, and results in structures that may be unknown and maybe largely meaningless even to the 'designers' of the product. According to its proponents, however, this approach "can find previously unknown patterns in data using unlabelled datasets" (Nvidia 2023).
In GenAI, it appears that, in the 'encoding' phase, developers at least initially adopt unsupervised, automated learning approaches to cope with the vast volumes of data. It also appears that new models perform poorly until they have been subjected to refinements using 'supervised' approaches. These are inevitably somewhat human-intensive, experimental and iterative. Both the unsupervised and supervised phases appear to have the tendency to have unforeseen consequences.
The category of GenAI artefact in focus in this article is designed to synthesise text. Text-oriented GenAI draws on a line of research in the field of natural language processing (NLP) (Karanikolas et al. 2023), and a technical development called a language model (LM). An LM is a system "trained on string prediction tasks: that is, predicting the likelihood of a token (character, word or string) given either its preceding context or (in bidirectional and masked LMs) its surrounding [textual] context" (Bender et al. 2021, p.611). The capacity of contemporary computational and storage devices has enabled the firehose of 'big data' (Clarke 2016). That term refers to very large quantities of data, in the present context textual data, that has been appropriated, in some cases with dubious legality, from a wide array of sources, often with little consideration given to data quality and to incompatibilities among the sources. The combination of technological capacity with vast quantities of raw data-resources has encouraged the recognition of patterns that enable the creation of fast ways of choosing an appropriate next word.
Based on such limited lingual archaeology as has been published (e.g. Foote 2023) and scans of Google Scholar, the notion Large Language Model (LLM) appears to have emerged during the early-mid-2010s. It appears to have been a form of re-badging of a developing field, much like the re-birth of 'data mining' as 'data analytics', or the overriding of the original, client-oriented 'World Wide Web' with organisation-empowering 'Web 2.0' technologies. LLM does, however, have the advantage of being rather more descriptive than many other business-fashion terms. From mid-2022, the particular LLM that has dominated public perceptions has been OpenAI's GPT. This is available as a number of materially different versions. Initially at least, some versions have been accessible gratis, with further developed versions available for-fee. Other LLM services with a significant public profile in up to the first quarter of 2025 include LLaMA (by Meta), Gemini (by Google), Grok (by xAI), Claude, Mistral and Ernie, and most recently DeepSeek.
Various approaches have been taken to the reduction of text into a form that can be readily manipulated. An approach associated with the explosion of GenAI from 2022 onwards has been the 'encoding' of source-text into a dense representation, such that similarities are closely associated with one another. Use of an Encoder alone - of which Bidirectional Encoder Representations from Transformers (BERT) was an important early exemplar - may be of value for functions such as the classification of text-sources and sentiment analysis (Talaat 2023). To support GenAI, however, the need is for representations that will efficiently support the identification of suitable needles in a very large haystack, and subsequent 'decoding' of the 'dense representation' into a form from which convincing-looking text can be generated.
A mainstream architecture for the encoding element of GenAI is transformers, a technique that was introduced by a team of researchers at Google (Vaswani et al. 2017). "In natural language processing, a transformer encodes each word in a corpus of text as a token [ a numerical representation of a chunk of data ] and then generates an attention map, which captures each token's relationships with all other tokens. This attention map helps the transformer understand [ internal, lingual ] context when it generates new text" (Zewe 2023).
One explanation of how the process delivers outcomes is that "In this huge corpus of text, words and sentences appear in sequences with [reasonably consistent] dependencies. This recurrence helps the model understand how to cut text into statistical chunks that have some predictability. It learns the patterns of these blocks of text and uses this knowledge to propose what might come next" (Zewe 2023).
Another interpretation of the encoding techniques used in GenAI is that it is appropriate to "Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way, that a JPEG retains much of the information of a higher-resolution image, but, if you're looking for an exact sequence of bits, you won't find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it's usually acceptable" (Chiang 2023).
A further aspect of LLMs is the re-purposing of vast general-purpose models for more specific purposes: "A foundation [large language] model is an AI neural network - trained on mountains of raw data, generally with unsupervised learning - that can be adapted to accomplish a broad range of tasks" (Merritt 2023). See also Jones (2023). Bommasani et al. (2023) noted "the rise of models ... trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character". One result of this technically-lauded approach is that GenAI artefacts that are applied to reasonably specific purposes are generating responses from a vast resource, a large proportion of whose content is not semantically relevant to the specific purpose, but which is assumed to have relevance in a structural-linguistic sense. These origins may influence the expression and even the apparent semantic content of the response.
GenAI technology is still new, and maturation continues. Small Language Models (SLM) are emergent (Schick & Sch?tze 2021, Javaheripi 2023). A natural development is for a foundation LLM to provide lingual structure, but to be combined with a specialised and perhaps curated SLM containing semantic content relevant to, for example, melanoma diagnosis or environmental regulation. A further refinement might then be the abstraction of generic lingual models from LLMs, and their expression in a compressed form, enabling combination with curated collections in particular domains in a manner that is more convenient, and less profligate of processing-power and energy.
The LLM's encoding phase results in a resource that can be accessed by a Chatbot in order to generate responses to users. OED dates the term 'chatbot' to 1994, and defines it as "A computer program designed to simulate conversation with a human user, usually over the [I]nternet ... ". Other terms in use include virtual assistant, conversational AI, and (more restrictively) web-interface. The notion (but not yet the term) originated around the time of the publication of the imitation game, popularly referred to as the Turing Test (Turing 1950). In Weizenbaum (1966), a paper on Eliza ("a program which makes natural language conversation with a computer possible", p.36), that author examined the challenges involved in human language interactions with computers. Its effect was to demonstrate human users' inherent gullibility and willingness to suspend disbelief (Dodgson 2023). Although the origins of the term 'chatbot' are conventionally traced to the use of 'chatterbot' in a paper title by Michael Mauldin in 1994, the first mention of 'chatbot' located in the Google Scholar archive is as the name of a particular 'chatterbot' mentioned in de Angeli et al. (2001). This was built using the same technology as Wallace's Alice (Artificial Linguistic Internet Computer Entity) of 1995.
Core processes within many early chatbots were developed using procedural (i.e. genuinely algorithmic) languages. Logic programming and rule-based approaches have also been adopted, and most recently LLM-based GenAI (Adamopoulou & Moussiades 2020). An example of a chatbot that is specifically designed as a component of a GenAI artefact is OpenAI's ChatGPT, accessible at https://chatgpt.com/. This accepts and interprets natural-language text, and in more languages than just English. Microsoft offers Copilot as an alternative chatbot to ChatGPT, invoking OpenAI's GPT-4 LLM. Perplexity is a chatbot that uses both the GPT 3.5 LLM and web-searching. The service-names Claude, Ernie, Gemini and DeepSeek each apply to the amalgam of the relevant provider's LLM+chatbot. Playing catch-up even moreso than other major tech platforms, Apple announced in mid-2024 its intention to integrate OpenAI's ChatGPT into iOS and macOS.
The process whereby GenAI creates new data commences with interpretation of the task to be performed. The specialised chatbots that provide human users with their means of interfacing with a GenAI artefact commonly use established techniques of natural language processing (NLP) to analyse the words and phrases in the request, in some cases using inbuilt synonym tables (Kucherbaev et al. 2018). This generally involves identifying verbs and verb-forms that are related to the requestor's intent, for example distinguishing a question (seeking information) from an instruction to conduct a transaction; and recognising the nouns that define the subject-matter. The end-result is a set of tokens, compatible with those used within the LLM, which the chatbot passes to the relevant component of the LLM. In Figure 1, the encoder (discussed in sub-section (2) above) is depicted as part of the LLM only, whereas the decoder is depicted as the engine running at the intersection of the LLM and the Chatbot, at the heart of the extraction and decoding function.
The 'decoding' process involves selection and decompression of the tightly structured data in the model (Shi et al. 2024). Because of the high compression-factor during encoding, the stored data is a highly 'lossy' interpretation of the source. The decompression process consequently delivers decoded data that is unlikely to result in text closely similar to any of the text that was encoded. However, LLM designers seek to compensate for this by applying supervised learning and reinforcement learning from human feedback (RLHF) in order to deliver a workable 'decoding' function (Kaufmann et al. 2024).
The decoded new data is not in a form intelligible to users. The intent of the designer of a GenAI artefact is generally the production of synthetic text that is likely to appear authoritative, authentic and convincing to the person who asked the question. Prior work in the field of natural language generation (NLG) has resulted in tools for the production of understandable texts from non-lingual data (Gatt & Krahmer 2018). As a result, the presentation quality of responses delivered by contemporary NLGs is, in syntactic terms at least, very high. Integrating an LLM to an existing NLG tool may, however, involve conversion of the particular token scheme into an intermediate form.
Given the scale of the source-texts represented in current LLMs, it appears likely that responses will generally differ from each of the selected source-texts in at least style and potentially also content. The reasons for the differences include the approaches adopted to selecting the data from the LLM, to filling in for the lossyness of the compression, to the filtering and merger of multiple sources, to the expression of the request, and to the styling of the output.
Inconsistencies in presentation and content among the sources may contribute to the existence of strangeness in some responses. Instances of this are commonly referred as 'hallucination'. This expression refers to content that appears nonsensical or unfaithful to the sources, or to be a fabrication, even including nonexistent literature (Nah et al. 2023, pp.287-88). The judgement that any particular passage is a hallucination may be based on inconsistency with the user's perception of the real world, but might be based on internal consistency within the response. GenAI artefacts may also be designed to generate arbitrary differences in responses, in order to establish and sustain an aura of intelligence and creativity and avoid giving the appearance of being a boring automaton.
A single Request-Response pair can deliver (perceived) value in some contexts, but not in others. Circumstances in which a succession of cumulative Request-Response pairs is necessary include where the intention of the request was misunderstood, an aspect of the response needs clarification, the response stimulates an alternative or a deeper follow-on request, the response 'raises more questions than it answers', or the user had asked a preliminary question with the intention of probing further or more deeply afterwards. Published reports suggest users have had highly varied experiences with regard to the effectiveness of GenAI artefacts in relation to dialogues. A literature is emergent. See, for example, Pan et al. (2023) and Labruna et al. (2023).
In experiments conducted by the author, after the response was provided, the ChatGPT page remained open for further interactions. It appears that further input was assumed to be on the same topic as the previous input, at least if there was some commonality in the expressions used and/or only a relatively short time-lapse between one set of user input and the next. This retention of the thread-context is referred to as 'state-preservation', and enables some degree of cumulativeness of interactions, with the interactions intended to resemble a conversation between humans. A dialogue containing more than a few exchanges may, however, require the user to perform the 'state management' task, for example by including a summary of the previous conversation with follow-on communications (Bellow 2023). This is an area in which maturation can be reasonably anticipated.
It appears, however, that intended continuations may be misinterpreted. Some responses to follow-on input appear bizarre, including apparently unreasoned or poorly-reasoned apologies, and new statements inconsistent with previous ones. This may reflect limits on the number of words, tokens or linguistic constructs the particular GenAI artefact can deal with in a single session. It is also possible that input that the user intends to commence a new request may be misinterpreted due to the previous interaction(s) being treated as though part of a still-relevant context.
The descriptions in this section establish sufficient insight into GenAI technology to enable key characteristics to be identified that are relevant to its consequences, and in particular those impacts and implications that may harm the interests of stakeholders.
There is widespread enthusiasm about the quality and usefulness of GenAI artefacts' responses. There is, however, a growing collection of more thoughtful publications based on careful analysis. Some of them call for caution, e.g. concluding that responses are "generic, not specific; are misleading in parts; and do not provide confidence that ChatGPT could be used, as suggested, to guide clinical imaging" (Currie 2023, p.260). Others are highly critical, for example describing AI as brittle, opaque, greedy, shallow and tone-deaf, manipulative and hackable, biased, invasive and "faking it" (Willcocks et al. 2023. See also Walsh 2023, pp.43-55 and Jazwinska & Chandrasekar 2024 for specific examples of ChatGPT blunders).
A more abstract interpretation of GenAI is that it is empiricism unguided by theory. This represents abandonment of the philosophy of science of the last 500 years, which carefully blends directed observation with disciplined conceptual work. GenAI accepts whatever data it is provided, and lacks any network of conceptual understanding. A bold and entrancing rendition of this viewpoint, enunciated during an earlier phase of the 'big data' era, is that " ... faced with massive data, [the old] approach to science -- hypothesize, model, test -- is becoming obsolete ... There is now a better way. ... We can stop looking for models. We can analyze the data without hypotheses about what it might show. We can throw the numbers into the biggest computing clusters the world has ever seen and let statistical algorithms find patterns where science cannot" (Anderson 2008).
This section draws on the descriptions of GenAI artefacts in section 2.2 in order to identify features that create the likelihood of negative consequences for stakeholders. In Table 2, 20 features are allocated into four groups as a reference-point for the subsequent analysis.
___________________
The source-texts that a GenAI artefact encodes into an LLM evidence issues that significantly affect the quality of responses and hence outcomes. The comments made here relate to current LLMs reflecting very-large-scale source-text collections. Some qualifications are likely to be appropriate in the case of SLMs, whose designers intend a narrower scope of application than the universalist idea of 'a response to any question', and are selective in their incorporation of source-texts into the data collection and hence into the LM.
Generally, there has been to date a strong leaning towards indiscriminate acquisition of source-texts, with little attention paid to the nature of the utterances (Entry 1.1 in Table 2). This reflects the bold assumptions of the 'big data' movement about 'more is better' and 'bad data gets overwhelmed by the maelstrom of good data' (Chen et al. 2012, Wang et al. 2018). The decoding that underlies response generation is blind to the question of data quality, and its selection criteria may be such that bad data may have a disproportionate impact on the response and hence on the requestor's interpretation of the outcome. In the present context of written text, 'bad data' encompasses not only mis- and dis-information, but also fantasy, intentionally fictional, imagined and non-empirical content.
A related feature is inadequacies in the quality assurance of source-texts (1.2). The IS field is well-acquainted with quality factors in the context of structured data. These comprise data quality factors, which are assessable both at the time of data-creation and subsequently, and information quality factors, which are assessable only at the time of use (Clarke 2018). Textual data presents different challenges (Clarke 2024). Editorial standards are very low in social media, but have also plummeted within formal media as advertising revenues have been lost to tech platforms (ACCC 2023).
The text sources that are fed into LLMs may vary enormously in the extent to which those quality factors are addressed. The terms 'disinformation' (for content that is misleading and is demonstrably intentionally so) and 'misinformation' (for other misleading content) have been prominent in discussions about both the source-texts in tech platforms supporting various forms of social media, and in responses produced by GenAI artefacts. The analyses conducted by 'fact-checkers' suggest that a high degree of both forms of malinformation exist in populist sources (Aimeur et al. 2023) (1.3).
A further concern is that presentation quality (including grammar, style and ambiguity-avoidance) is highly varied, but often low. This gives rise to ambiguities, which are encoded into LLMs, and are then extracted and re-expressed in responses, resulting in misunderstandings of the intentions of the original source (1.4). It remains unclear whether circumstances arise under which legal recourse may exist where malinformation or malperformance of a GenAI artefact leads to loss (Herbosch 2024, Arcila 2024).
Source-texts bring with them intellectual baggage, or dominant perspectives, that reflect both the period and the cultural context in which they were expressed, and the value-set of the utterer. Some of the more apparent areas in which preconceptions are inbuilt into a great deal of text are race / skin-colour / ethnicity (associated with colonialism), and gender (as re-evaluated in feminist literature). Content bias arising from embedded values comes in a great many other flavours, including age, class, caste, religion, wealth and educational attainment (1.5). In the words of Bender et al. (2021), "the tendency of training data ingested from the Internet to encode hegemonic worldviews [and] the tendency of LMs to amplify biases and other issues in the training data [ are key features of GenAI artefacts] ..." (p.616).
Some aspects of source-text quality have been addressed by GenAI developers. For example, some may perform de-duplication prior to encoding, and others say that they delete 'toxic' content, such as "removing documents which contain any word on the 'List of Dirty, Naughty, Obscene, or Otherwise Bad Words'" (Dodge et al. 2021, p.2). The descriptive term 'data scrubbing' conveys that such processes are endeavours to, in some sense, clean or cleanse the source-text(s), but also that those endeavours may or may not achieve their purpose, and may or may not have unintended consequences for other quality attributes (1.6).
The removal of source-texts from a collection prior to the encoding step represents historical revisionism. It also constitutes yet another form of bias, and one that may be detrimental to the values that the action may be intended to support. For example, the removal of misogynistic works from a corpus assists misogynists in arguing that claims about misogyny are exaggerated.
On the other hand, almost all of the literature on bias management assumes that bias-removal is desirable, that it is achievable, and that it has no negative consequences. That impression is borne out by the absence of any recognition of the issue in a recent survey article (Siddique et al. 2024). Some authors do, however, evidence awareness of the issue, e.g. DeMartini et al. (2021), Lattimer (2023). Further issues arise from the difficulties involved in formulating operational criteria for the filtering, amendment or qualification of source-texts.
A recently-emerged problem is 'Pollution by Synthetic Texts' (1.7). As responses have been produced by GenAI artefacts, some of the synthetic text has inevitably been fed back into collections of source-texts, whether directly by the LLM or Chatbot provider, or indirectly by requestors republishing it as though it were an original contribution. This creates the risk of an echo-chamber in which variants of the same, partly information, partly misinformation and partly disinformation, accumulate and alter the balance in the collection. This entrenches errors and bias inherent in the original collection, and may result in the creation an 'alternative, authoritative source of truth'. It also creates an opportunity for devotees of causes to shift future responses in a particular direction, by 'gaming the system' through the submission of large numbers of customised requests. For example, an experimental study by Sharma et al. (2024) "found that participants engaged in more biased information querying with LLM-powered conversational search, and an opinionated LLM reinforcing their views exacerbated this bias".
A second cluster of issues at the heart of GenAI arises from the compression of source-texts into an 'encoded' form that reflects only the structural aspects of the original material. The results of large numbers of encodings are then inter-related to form a cut-down, syntax-level-only model of language. Table 2 at 2 summarises this as limited reflection of language structure.
When compressed forms are 'decoded' as a basis for generating responses, they primarily reflect the structures of the selected sources, and only indirectly reflect their content. The process dissociates the text from the human context within which it originated: "Our human understanding of coherence derives from our ability to recognize interlocutors' beliefs ... and intentions ... within context ... That is, human language use takes place between individuals who share common ground and are mutually aware of that sharing (and its extent), who have communicative intents which they use language to convey, and who model each others' mental states as they communicate" (Bender et al. 2021, p.616). In contrast, "an LM is a system for haphazardly stitching together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning: a stochastic parrot" (p.617). Even a researcher strongly supportive of LLMs tentatively concludes that " ... BERT is not learning inference through semantic relations between premisses and conclusions. Instead it appears to be identifying certain lexical and structural patterns in the inference pairs" (Lappin 2024).
A cluster of the most serious concerns arises from the absence from the GenAI universe of any real-world referents for the 'tokens' that are generated during the process of encoding (Table 1 at 3). The resulting Language Model, and the responses it generates, lack any connection with real-world phenomena (Hovorka 2012, p.2):
GenAI has an emptiness within. It has no world from which to draw knowledge within and is limited to constructing texts based on semantic probabilities regarding "facts" that people or other GenAI mechanisms have produced. ... It has never inhabited a world, never been wrong, never made a conceptual leap, challenged an assumption, or recognized a different worldview. It has only probabilities derived from what has been previously written about these phenomena
Human language emerged and evolved among individuals seeking to convey meaning to one another. Most of the primary concerns in the kinds of societies in which language originated were physical things, particularly safety, food and water, and actions to achieve, acquire or safeguard each of those things. As 'subjective' 'mind-stuff' became more prominent, referents still existed. Notions such as categories and counts represent abstractions beyond individual physical phenomena. Even more abstract expressions, such as those about feelings, sentiments and aesthetics presume some commonality of experience between utterer and audience.
Semiotics is the study of the use of symbolic communication, including written and spoken languages. It encompasses three segments (Morris, 1938):
Even at the syntactic level, GenAI is limited to merely syntactical associations, dominated by sequence and proximity -- which are low-order aspects of a language's grammar. It also makes the inherent assumption either that a single, coherent and consistent grammar exists for whatever language is being used, or that there are multiple grammars but the artefact can identify and select which variant the conversation is in. (In response to the request "What did a flower-seller in Covent Garden really think of the aristocracy? Could you please express the response in the style of late 19th century Cockney", ChatGPT began to display a response in the requested style -- featuring cheekiness -- but that text was very quickly overwritten with "This content may violate our usage policies". This might have related to the expression 'Not Pygmalion likely'; but a perusal of the policies identified only "donÅft circumvent safeguards", suggesting the purpose of the safeguard may be to protect ChatGPT's reputation by blocking what some feature of the underlying ANN interpreted as 'a trick question').
The technology has almost nothing to offer in the other two segments of semiotics. For example, among humans, the words 'flower', 'rose', 'smell', 'sweet' and 'bunch' have associations that are semantic (relating the word-clusters to real-world objects and contexts), and pragmatic (relating word-clusters to feelings). These elements of common contextual understanding are crucial to achieving understanding among humans. There is no sense in which content is 'understood' by a GenAI artefact (3.1). In Bender et al. (2021), concern is expressed about "the tendency of researchers and other people to mistake LM-driven performance gains for actual natural language understanding" (p.616).
Human language is generally used within some context, and frequently in a very rich context. Individuals interpret, or create, meaning in utterances, and the meanings may vary somewhat and even greatly between people; but the meanings are imputed within a context that is subject to at least some degree of sharing among those individuals (3.2). This enables many misunderstandings to be avoided, because of each individual's tendency to apply 'common sense' or 'normal understanding' (Weizenbaum 1967), which involves great complexity and subtlety of contextual appreciation (Dreyfus 1992, pp.214-221). No such moderating influence exists within the GenAI realm.
A further shortfall in GenAI is that the response is generated without any appreciation of the nature of the requestor, or of any broader audience to which the response is intended to be communicated (3.3). A weak proxy may be used, for example, where the request includes an expression such as 'in the style of a tabloid newspaper / secondary school essay / scientific journal article'. Such 'styles' are, however, generated probabilistically, based on narrow, syntactical-level-only modelling of sources. A human providing a textual response has the opportunity to also consider the nature and interests of the requestor, of any intended audience, and of third parties. A GenAI artefact has no such opportunity, and may produce text that is bland, or bias-reinforcing, or brutal, depending on the particular stakeholder's perspective. It also has no capacity to consider the consequences that may flow from the response, whether for the requestor, any intended audience, or any particular category of third party (3.4). This recklessness may be transmitted further by the requestor's unthinking application or publication of the response.
Group 4 in Table 2 identifies potentially harmful attributes of the generative function itself. The mis- and dis-information that is caught up in indiscriminately gathered, uncurated source-text collections finds its way into responses, and in many cases may affect inferences, decisions and actions, or be projected further afield (4.1). An inference is "a conclusion drawn from data [and/]or premisses" (OED 2), and hence represents new data (Custers & Vrabec 2024). Unreasonable inferences may have serious consequences, and unreasonable decisions and actions are even more likely to do so. Because the components used to express responses are reasonably mature, and have been the subject of further investment by GenAI developers, responses have the appearance of authenticity, and hence are liable to lull requestors into assuming the response is reliable.
A further feature of GenAI is its limited ability to express a rationale to go with what many requestors perceive as its pronouncements (4.2). This is a general problem with many currently-popular forms of machine learning. The topic of 'eXplainable AI' (XAI) has become popular in recent years. XAI is, however, not a category of theory, technology or tool, but merely an aspirational research-domain. Explainability is commonly understood, consistently with EDPS (2023), as the ability of a GenAI artefact to provide clear and understandable explanations for its inferences, decisions and actions. The XAI literature appears, instead, to have the intention of reducing the notion of explainability to interpretability (e.g. Linardatos et al. 2020), or perhaps to some even weaker proxy. See for example TRS (2023, p.13) and Panigutti et al. (2023). LLMs fail the criteria for transparency (Lipton 2018), and no means exists to generate 'post hoc' explanations of the operation of the 'black box' (Adamson 2022). Wigan et al. (2022) underline the significance of GenAI's inability to satisfy the explainability test.
The inadequacy is compounded by the inability of some GenAI artefacts to provide quotations (4.3) - because LLMs inherently abstract lingual structure and abandon the original text. The problem is further exacerbated in the all-too-common circumstances in which the artefact is incapable of provision of citations of the source-texts on which it has drawn, or is even precluded by design from doing so, e.g. to avoid exposure of the fact that the provider has breached other parties' rights in relation to access to and/or use of the source-texts. Some providers, including Perplexity, have addressed this by complementing access to an LLM with web-searches for specific text-passages.
Another limiting factor is the set of criteria used in selecting modes of expression of the response. Rather than reflecting an understanding of the requestor's purpose and the nature of the intended audience (e.g. their educational background, interests and values), the primary driver of expression is word-combination probabilities (4.4). This may then be filtered through mechanisms whose purpose is to convey authenticity and avoid boring repetitions (4.5).
A final concern (4.6) is inconsistencies or hallucinations in responses (Ji et al. 20123). These are referred to by ChatGPT's developer as "plausible-sounding but incorrect or nonsensical answers" (OpenAI 2022). For humans, who are used to patterns of human error, it is challenging to recognise error patterns in GenAI behaviour (Schneier & Sanders 2025). To many users, in many circumstances, even inconsistent and hallucinatory responses "are plausible enough that identifying them requires comparing them against the originals, which in this case means either the Web or our knowledge of the world ... [I]f a compression algorithm is designed to reconstruct text after ninety-nine percent of the original has been discarded, we should expect that significant portions of what it generates will be entirely fabricated" (Chiang 2023).
This section has outlined 20 aspects of GenAI that appear to be the underlying causes of difficulties. The next section considers the use of responses provided by GenAI artefacts, as a prelude to an assessment of impacts, implications and risks.
The previous sections have adopted a narrow conception of GenAI technology as artefact. This section considers the broader notion of GenAI technology-in-use. Important aspects of its use include the categories of users, requests, data-formats and responses, the individual and organisational behaviours of users, the capacity of the GenAI artefact to provide a rationale for its statements, and the perception that users have of the artefact.
GenAI artefacts may be used in a wide variety of contexts. The user may be an individual human, for their personal purposes, gaining access to content of the nature of a digest of the multiple sources that might be delivered by using a search-engine. The user may be an individual as an agent for a small, informal group in a social setting or may be acting in a more structured environment, as an employee of an enterprise, or in a formalised community group, and may apply the responses to more business-like purposes. The results may then be then integrated into the business processes of an incorporated organisation. The user might, however, be another artefact, and that artefact may be capable of acting in the real world.
In Figure 2, a user is depicted as situated within a context, composing a request, receiving a response, and applying it to one or more of a range of purposes. Some purposes are casual, and merely add to the user's body of knowledge. Other purposes are instrumentalist, in that they purposively change a state of mind or a state of reality. A change in the state of mind may be in a general attitude of the user, or may relate to an inference about to a particular instance or circumstance. Alternatively, through communication, the user may influence a decision or action of another party. Finally, a change in the state of the real world may arise from the user making a decision or taking an action.
To this point in the discussion, the term 'request' has been used to describe any form of input by a user to a GenAI artefact. However, requests take various forms. As depicted in Table 3, some are framed as questions, some as requests for text to be composed, and yet others involve multiple, successive questions, requests and statements making up a dialogue between human(s) and the GenAI artefact. The degree of reliability of responses, and the nature of response errors, may vary depending on the Request category, e.g. it may be that the frequency of hallucinations is greatest during a prolonged interrogation.
As depicted in Table 4, the request may be in any of a series of modes, some concerning knowledge in a particular domain and others relating to a particular case, instance or circumstance to which knowledge may be applied. The form of the response that is sought may be an exposition of knowledge, a description, an explanation, or an anticipation of future states.
Which of the modes in Table 4 is in play is likely to be most readily inferred from the choices of interrogative adverbs, verbs, moods and tenses. For example, the six categories might be indicated by questions of the nature, respectively, of 'What is?', 'What are?', 'How does?', 'Describe what', 'Explain how' and 'Project scenarios'. Some languages, notably English and German, have a very rich set of tenses. This enables great precision of expression, but at the cost of complexity of grammatical structure and scope for ambiguity. For example, a dialogue in a criminal investigation might involve a question along the lines of 'What might have been the motivations of the suspect?', and someone managing the search for a missing person might ask 'Which would have been the person's most likely directions and speed of movement when they discovered they were lost at what we postulate to have been the following coordinates ...?'.
From a process perspective, the delivery by a GenAI artefact of responses of satisfactory quality depends on some key factors:
The analysis in section 3 concludes that these requirements are generally not able to be delivered by GenAI technology.
Interpretation of the response is a matter for the person(s) or artefact(s) that gain access to it. The overall design of a system that incorporates GenAI does, however, need to encompass means to ensure that users of responses are generally informed about their nature. They also need to be provided with means to interrogate and/or collaborate with the AI artefact in order to have sufficient understanding of the strengths and weaknesses of the technique and its outputs, and how to ensure that they place only as much reliance on the quality of the Response as is justifiable. As Heersmink et al. (2024) expressed it, GenAI artefacts' "data and algorithmic opacity and their phenomenological and informational transparency can make it difficult for users to calibrate their trust correctly".
Various classification schemes for use-cases may be useful in particular circumstances. For example, ChatGPT provided one in response to the question "How can lecturers use ChatGPT to teach Nuclear Medicine?" (Currie 2023, pp.256-257). The result was a coherent list of 24 "ways that ChatGPT can be useful", including "Providing feedback for assignments", "Question generation", "Generating lecture notes", "Developing course syllabi" and delivering "summaries of research findings and clinical reports".
Interrogation and collaboration are very difficult functions to deliver with current tools. GenAI, like AI/ML, is merely empirical, based on large heaps of previous data. Inferences from them are a-rational, in the sense that there is no express algorithm, problem-solution or problem-definition underlying the output. This means that no humanly-understandable explanation of the reasoning underlying the response can be produced, and hence interactions by a human user with a GenAI artefact are limited in form, and stilted. Further, in the absence of explanations, human decision-makers cannot fulfil any obligations they may have to explain to interested parties (such as their managers, auditors, people affected by decisions, regulators and courts) why they made a decision or took an action. If it is infeasible to review decisions and actions. Procedural fairness, due process and natural justice cannot be respected. The end-result is that personal and organisational accountability is undermined.
Depending on contexts and patterns of use, the interpretation of a GenAI artefact's response may be subject to moderation processes within and beyond an organisation. A first level of reflection and filtering may arise where the individual conducting or leading the interaction applies critical thought to the responses. On the other hand, the promptness, and the contrived fluidity and authoritativeness of expression of the responses may in many cases cause the individual to suspend their disbelief, and uncautiously embrace the responses as being reliable. Subsequent levels of moderation may arise through the dynamics of organisational processes; but enthusiasm and momentum might still result in inferences, decisions and actions that are inappropriate or harmful.
Any consideration of GenAI-in-use has to confront the issue of the perception that the user has of the artefact. Responses that are well-expressed sound to a human as though they are authoritative. This tends to numb the mind, and is likely to lull into a false sense of security individuals who are less well educated about the technology, busy, tired or lazy. Even the naturally sceptical and the trained professional may slide into perceiving a GenAI artefact not as a tool but as an entity that draws inferences, makes decisions and even acts in the real world to implement them. From the perspective of the Information Systems discipline and profession, users need to rid themselves of the notion that a GenAI artefact is a 'decision system', and engrain the perception of it as a 'decision support system', designed to work with humans in solving semi-structured problems (Er 1988).
A more contemporary approach is adopted in Clarke (2023, section XI, pp.29-31). The conception of AI as 'artificial intelligence' is criticised as being demonstrably inappropriate and harmful. Re-conception is proposed, such that the intellectual behaviour of AI-impregnated devices is designated as 'complementary artefact intelligence'. This firstly avoids the implication of the adjective 'artificial' that the intelligence is a copy of human intelligence, and secondly builds in the notion of the device as tool, or perhaps diverse workmate. To fulfil the requirement of complementariness, the device must be conceived and engineered to interface conveniently with humans, and with other artefacts. The combination of human and complementary artefact intelligences can then deliver the desirable phenomenon of 'augmented intelligence', a 'symbiot' that can deliver more than either 'artificial' or 'human' intelligence can deliver alone (Engelbart 1962).
The last century has seen a substantial increase in the social distance between organisations and the people with whom they deal. This is typified by the disappearance of local bank managers and 'case managers' interacting directly with clients, and IT providers' abandonment of customer support and its replacement by reliance on self-help among user communities. The current era of digitalisation involves the replacement of interpretation and management of the world through human perception and cognition with processes that are almost entirely dependent on digital data (Brennen & Kreiss 2016).
Digitalisation exacerbates the longstanding problems of social distance, in that organisations no longer relate to individuals, and instead make their judgements based on the digital personae that arise from the large pools of data associated with individual identities that the organisation has access to (Clarke 1994, 2019a). One of the effects of GenAI is further amplification of the social distance between organisations and the people who are directly and indirectly affected by the organisations' actions. The inferences are drawn, and the decisions are made, in isolation from the affected parties. Further, the basis on which inferences are drawn and decisions are made are obscured even from those participating in the process, let alone from affected individuals who are remote from the point of decision.
The preceding discussions of GenAI technology and key aspects of its patterns of use have laid the foundation for the assessment of effects that are likely to result in harm to the interests of various parties.
GenAI's fallibility is acknowledged in text appearing at times in the footer of the ChatGPT input page at https://chatgpt.com/: "ChatGPT can make mistakes. Check important info". Further, OpenAI's Terms of Use for ChatGPT, as at January 2025, included the following (all emphases added):
Given the probabilistic nature of machine learning, use of our Services may, in some situations, result in Output that does not accurately reflect real people, places, or facts. ... you understand and agree:
The purpose of this section is to delve into the consequences that follow from the use of GenAI. The focus is not on the potential benefits, which are acknowledged here, and which are addressed (although often with insufficiently tempered enthusiasm) by many authors (e.g. McAfee et al. 2023). The focus is on downsides. Distinctions are drawn between first-round direct impacts, second-round less direct implications, and contingent outcomes, or risks, that arise from "mistakes" of the kinds referred to by OpenAI, and other features of the technology-in-use. Issues also arise from responses that, while perhaps not "mistakes", are misleading, and responses that appropriately reflect the content of the text sources relied upon, but give rise to materially negative consequences for some parties, particularly those who, or that, lack the power, resources or competencies to defend their interests.
Many scholars have offered frameworks within which consequences can be evaluated. For example, Figure 3 displays categorisations and inter-relationships among various 'social' impacts and implications, extending into the polity and environment, arising from workshops with "thirty experts across industry, academia, civil society, and government" Solaiman et al. (2023, p.3).
Reproduction of Figure 1 of Solaiman et al. (2023)
Bold and optimistic promotional statements about GenAI and its use have stimulated a flood of counter-statements of concern. Table 5 provides a checklist of concerns that have been expressed, drawing on such publications as Wach et al. (2023), Weidinger et al. (2023), pp.30-31, Zohny et al. (2023), Gupta et al. (2023), Schlagwein & Willcocks (2023), Hagendorff (2024). Distinctions are made between first-round, relatively direct Impacts, second-round, less direct Implications, and contingent outcomes or Risks.
Direct Impacts on Individuals
Direct Impacts on Society and the Polity
Indirect Implications
Risks
______________
In relation to the various forms of malinformation that can arise, the question has been raised as to whether providers of GenAI artefacts and services have, or should have, an obligation to 'tell the truth' (Wachter et al. 2024). The proposition is hamstrung by its invocation of the notion that an accessible absolute truth exists, in contexts in which many value-sets are applied and many interpretations are possible. A more relativistic approach may, on the other hand, address the issues. For example, the responsibility might be operationalised as a duty of care to avoid the republication of misinformation and disinformation, and to risk-manage hallucinations (Clarke 2024).
Many stakeholders may be involved. A stakeholder in any particular use of a GenAI artefact is any party whose interests are, or may be, affected by that use. Users, as participants in the process, and the organisations for which they perform their actions, are stakeholders. The usefully descriptive term 'usees' refers to categories of stakeholders affected by the process, but not participants in it (Berleur & Drumm 1991, Clarke 1992, Fischer-H?bner & Lindskog 2001, Baumer 2015).
In each particular domain, many different parties may be stakeholders. The remainder of this section narrows the focus to individuals, such as loan applicants in credit evaluations, students in educational institutions, patients in health care contexts, and voters in political processes. This is partly because the majority of discussion to date has been about impacts on organisations, economies, societies and polities; and partly because individuals generally suffer from a lack of institutional or market power, and are likely to find it particularly difficult to defend their interests when confronted by GenAI-based inferences, decisions and actions.
In (Clarke 2023), a set of impacts of AI generally was identified, which affect the economic and social interests of individuals. These, reproduced in Table 6 below, appear to also be impacts of GenAI artefacts in particular.
Adaptation of Table 2 of (Clarke 2023, p.29)
_____________________
The opaqueness of inferencing techniques used in GenAI has potentially serious consequences for some stakeholders, in particular those who rely on the protections referred to in various contexts as due process and procedural fairness. GenAI artefacts commonly exhibit a-rationality. No description exists of how and why an outcome came about, and an ex post facto rationalisation of the decision criteria may not be able to be constructed. In the fairly common circumstance in which a GenAI artefact changes over time, it is also likely to exhibit 'unreplicability', i.e. the process cannot be repeated. This undermines the scope for investigation, because a reconstruction of the sequence of events is infeasible. Inferencing and decision processes therefore become 'unauditable', in that an independent party such as an auditor, judge or coroner is precluded from identifying initial, intermediate and final states, and triggers for transitions between states. Further, because errors are undiscoverable, the GenAI's impact becomes 'uncorrectable'. The end result is that organisations and individuals become 'unaccountable', and escape responsibility for harmful decisions and actions (Clarke 2019b).
Sections 2-4 described the nature and mechanics of GenAI artefacts, identified characteristics relevant to the technology's impacts, and considered the users and contexts of use. This section has outlined impacts, implications and risks arising from the application of GenAI artefacts, with an emphasis on consequences for individuals. On these foundations, the final section outlines a proposal for a key element of a regulatory regime to address GenAI's downsides.
The preceding sections catalogued consequences of the application of GenAI. Regulatory measures are necessary to provide protections, at one level for individual users and usees, and at another for communities, societies, economies, polities and the environment. This section draws on existing sources to propose a set of principles that can underpin regulatory mechanisms designed to achieve responsible behaviour.
A wide array of approaches to regulation exist, from natural controls, and infrastructural mechanisms to reinforce them, via various forms of self-regulation, and co- and meta-regulation, to the stringencies of formal regulation through statute, enforced through regulatory agencies and sanction schemes (Clarke & Bennett Moses 2014, Clarke2019d, 2020b, 2021, 2022a, 2022b). Regulatory measures may be applied early in the life of an innovative technology or held back until deployment and adoption have occurred and harm has arisen.
A great deal of literature exists considering formal regulation (or 'hard law') and the extent to which existing and new laws prevent and mitigate harm and provide recourse where harm arises. Revolutionary technologies challenge existing laws (Bennett Moses 2007, Brownsword 2008), and changes in law may occur at glacial speed, slowed and/or materially diluted by lobbying on behalf of financial and economic interests in innovation (summed up by such maxims as 'minimum viable product', 'permanent beta', 'ask for forgiveness, not permission' and 'move fast and break things'). Protections may emerge at lower levels in the regulatory hierarchy, commonly in weak and largely ineffective forms. Some organisations pay some respect to notions such as 'corporate social responsibility' and exercise at least a degree of self-restraint, but whole industry sectors are dragged down by 'cowboy' operators who flout the unenforceable 'norms'.
Nomatter at what level protections emerge, their designers will benefit from predefined principles that assist in the formulation of safeguards against negative impacts and implications, and means for deterring, preventing, detecting and mitigating risks.
The presentation of principles in this section draws on prior work which began with a set of principles for responsible data analytics (Clarke 2018), and an explanation of how to embed them in business processes (Clarke & Taylor 2019). The surge of interest in AI/ML during the decade 2010-2019 resulted in many organisations uttering 'principles for responsible AI', with varying degrees of superficiality and analytical depth, some of them generic, and some specific to AI/ML. Several researchers drew on those publications to abstract comprehensive meta-sets, including Zeng et al. (2019). and Jobin et al. (2019).
This article uses the meta-set published in Clarke (2019c). This consolidated the content of 30 sets published by a diverse set of organisations, resulting in 50 Principles clustered into 10 Themes. The meta-set's 10 Themes are reproduced in Table 7, substituting 'GenAI' for 'AI'. The Appendix reproduces the 50 Principles, with a second column containing suggestions for re-casting them in a form directly applicable to GenAI.
The meta-set has subsequently been applied to evaluate the comprehensiveness of the original source-sets, and multiple further sets. The primary scoring mechanism used has been a simple count of Principles to which the particular set makes at least material contributions. Experiments have been conducted with the assignment of a subjective fractional score for each Principle depending on the extent of the contribution, and heavier weightings to those Principles that can be argued to be of greatest importance.
All laws, proposals and codes evaluated to date have been found to be seriously inadequate, covering significantly less than 50% of the meta-set. The EC's much-publicised Artificial Intelligence Act (AIA 2024) is a serious failure when measured against that yardstick, scoring about 30%. See also Wachter (2024). This stands in stark contrast to the sole example of a strongly performing set, which scored about 75%. That was the 'Ethics Guidelines for Trustworthy AI', published the by the EC based on the work of its own High-Level Expert Group on Artificial Intelligence (EC 2019). Those Guidelines were essentially ignored by the Commission when it developed the AIA in 2021-24.
Adapted from
(Clarke
2019c), Table 4
The 50 Principles are reproduced in the
Appendix,
grouped under the 10 Themes
The following apply to each entity responsible for each of the five phases of
AI:
research, invention, innovation, dissemination and application.
GenAI offers prospects of considerable benefits and disbenefits. All entities involved in creating and applying GenAI have obligations to assess its short-term impacts and longer-term implications, to demonstrate the achievability of the postulated benefits, to be proactive in relation to disbenefits, and to involve stakeholders in the process.
Considerable public disquiet exists in relation to the replacement of human inferencing, decision-making and action by GenAI artefacts, and displacement of human workers by GenAI artefacts. All entities have obligations in relation to recognising and addressing the reasons underlying this disquiet.
Considerable public disquiet exists in relation to the prospect of humans being subjected to obscure GenAI-based processes, and ceding power to GenAI-based artefacts and systems. All entities have obligations in relation to recognising and addressing the reasons underlying this disquiet.
All entities involved in creating and applying GenAI have obligations to provide safeguards for all human stakeholders, whether as users of GenAI artefacts, or as usees affected by them, and to contribute to human stakeholders' wellbeing.
All entities involved in creating and applying GenAI have obligations to avoid, prevent and mitigate negative impacts on individuals, and to promote the interests of individuals.
All entities have obligations in relation to due process, procedural fairness and natural justice. These obligations can only be fulfilled if all entities involved in creating and applying GenAI ensure that humanly-understandable explanations are provided to the relevant entities at the time, and are available to the people affected by GenAI-based inferences, decisions and actions.
All entities involved in creating and applying GenAI have obligations in relation to the quality of business processes, products and outcomes.
All entities involved in creating and applying GenAI have obligations to ensure resistance to malfunctions (robustness) and recoverability when malfunctions occur (resilience), commensurate with the significance of the benefits, the data's sensitivity, and the potential for harm.
All entities involved in creating and applying GenAI have obligations in relation to due process, procedural fairness and natural justice. The obligations include ensuring that the relevant entity is discoverable, and addressing problems as they arise.
Each entity's obligations in relation to due process, procedural fairness and natural justice include the implementation and operation of systematic problem-handling processes, and respect for and compliance with external problem-handling processes.
This paper has contributed to the literature on the design and use of Generative AI in several ways. It has provided a description of the technology and each of its elements at sufficient depth to support critical analysis. Secondly, it has abstracted from that description 20 key characteristics of the technology that create the likelihood of negative impacts, implications and risks for individuals, communities, societies, economies, polities and the environment. Thirdly, it has extended the analysis into the patterns of use of the technology, and thereby ensured that the full range of sociotechnical considerations are able to be surfaced.
Fourthly, it has provided a structured list of impacts, implications, and contingent effects. In addition to the broader social aspects that have already been the subject of considerable discussion in the literature, the analysis emphasises effects on individuals. Finally, it has utilised the insights generated in the analysis in order to apply and adapt an existing meta-set of Principles for Responsible AI, to provide guidance in relation to the application of GenAI specifically.
The analyses and proposals in this article require scrutiny, evaluation and refinement. Particularly after adjustment for feedback arising from early uses, the Themes and Principles will provide a firm foundation in many contexts, and at all levels of the regulatory hierarchy.
Policy-makers can apply the 50 Principles to the design of new guidelines, voluntary and enforceable codes, and formal laws. Alternatively, they can use them as a checklist to support the evaluation of regulatory proposals and existing regimes. Where a co-regulatory approach is adopted, the Themes and Principles can provide a framework for negotiations among the stakeholder representatives. For a specific proposal in relation to co-regulation of AI, see Clarke (2019d,. s.7 and Table 3, pp.406-407).
Similarly, in the self-regulatory space, industry associations, organisations developing GenAI artefacts, and organisations using them, can utilise the Themes and Principles to establish guidelines, evaluation templates, and training. Tertiary institutions can use Table 7, the Appendix and the associated text as reading and project materials for their students, variously in law, computer science, information systems and management.
The primary focus of the work reported here has been the exercise of control over negative impacts and implications and the management of risks. It is also contended that application of the meta-set of Themes and Principles will significantly contribute to both the image and the reality of trustworthiness of GenAI artefacts, and hence to successful exploitation of the technology's promise.
Abbas R., Michael K., Michael M.G., Perakslis C. & Pitt, J. (2022) 'Machine learning, convergence digitalization, and the concentration of power: Enslavement by design using techno-biological behaviors' IEEE Transactions on Technology and Society 3,2 (2022) 76-88, at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9801492
ACCC (2021) 'Digital advertising services inquiry: Final report' Australian Competition and Consumer Commission, August 2021, at https://www.accc.gov.au/system/files/Digital%20advertising%20services%20inquiry%20-%20final%20report.pdf
Adamopoulou E. & Moussiades L. (2020) 'Chatbots: History, Technology, and Applications' Machine Learning with Applications 2 (Dec. 2020), at https://doi.org/10.1016/j.mlwa.2020.100006
Adamson G. (2022) 'Explainable artificial intelligence (XAI): A reason to believe?' Law in Context 37,3 (Apr. 2022), at https://search.informit.org/doi/pdf/10.3316/informit.553002286408436
Adensamer A, Gsenger R. & Klausner L.D. (2021) '"Computer says no": Algorithmic decision support and organisational responsibility' Journal of Responsible Technology 7-8 (October 2021) 100014, at https://www.sciencedirect.com/science/article/pii/S266665962100007X
AIA (2024) AI Act (Regulation (EU) 2024/1689), at https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
Aimeur E., Amri S. & Brassard G. (2023) 'Fake news, disinformation and misinformation in social media: a review' Social Network Analysis and Mining 13,30 (2023), at https://link.springer.com/content/pdf/10.1007/s13278-023-01028-5.pdf
Al-Amin M., Ali M.S., Salam A., Khan A., Ali A., Ullah A., Alam N. & Chowdhury S.K. (2024) 'History of generative Artificial Intelligence (AI) chatbots: Past, present, and future development' arXiv, 2024, at https://arxiv.org/pdf/2402.05122
Anderson C. (2008) 'The End of Theory: The Data Deluge Makes the Scientific Method Obsolete' Wired Magazine 16:07, 23 June 2008, at https://www.wired.com/2008/06/pb-theory/
Arcila B.B. (2024) 'AI liability in Europe: How does it complement risk regulation and deal with the problem of human oversight?' Computer Law & Security Review 54 (2024) 106012
Asaro P.M. (2019) 'AI Ethics in Predictive Policing: From Models of Threat to an Ethics of Care' IEEE Technology and Society Magazine 38,2 (June 2019) 40-53, at https://peterasaro.org/writing/Asaro_PredictivePolicing.pdf
ASD (2023) 'An AI Primer' Australian Signals Directorate, November 2023, at https://www.cyber.gov.au/sites/default/files/2023-11/%28OFFICIAL%29_Convoluted_layers_-_an_artificial_intelligence_%28AI%29_primer.pdf
Baldassarre M.T., Caivano D., B.F. Nieto, Gigante D. & Ragone A. (2023) 'The Social Impact of Generative AI: An Analysis on ChatGPT' Proc. GoodIT, September 2023, Lisbon, at https://dl.acm.org/doi/pdf/10.1145/3582515.3609555
Baumer E.P.S. (2015) 'Usees' Proc. 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15), April 2015
Bellow P. (2023) 'Maintaining conversation continuity with the ChatGPT API' OpenAI Community, December 2023, at https://community.openai.com/t/how-can-i-maintain-conversation-continuity-with-chat-gpt-api/574819/3
Bender E.M., Gebru T., McMillan-Major A. & Shmitchell S. (2021) 'On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?' Proc. FAccT'21, March 3-10, 2021, Virtual Event, Canada, pp.610-623, at https://dl.acm.org/doi/pdf/10.1145/3442188.3445922
Bennett Moses L. (2007) 'Recurring Dilemmas: The Law's Race to Keep Up with Technological Change' University of Illinois Journal of Law, Technology and Policy 7 (2007) 239-285, at http://www.jltp.uiuc.edu/archives/moses.pdf
Bennett Moses L. & Chan J. (2018) 'Algorithmic prediction in policing: assumptions, evaluation, and accountability' Policing and Society 28, 7 (2018) 806-822, at https://www.tandfonline.com/doi/full/10.1080/10439463.2016.1253695
Berleur J. & Drumm J. (Eds.) (1991) 'Information Technology Assessment' Proc. 4th IFIP-TC9 International Conference on Human Choice and Computers, Dublin, July 8-12, 1990, Elsevier Science Publishers (North-Holland), 1991
Bommasani Ret al. (2023) 'On the Opportunities and Risks of Foundation Models' arXiv, at https://arxiv.org/pdf/2108.07258
Booth R. & Milmo D. (2025) 'We tried out DeepSeek. It worked well, until we asked it about Tiananmen Square and Taiwan' The Guardian, 29 Jan 2025, at https://www.theguardian.com/technology/2025/jan/28/chinese-ai-chatbot-deepseek-censors-itself-in-realtime-users-report
Borgesius F.J.Z. (2020) 'Strengthening legal protection against discrimination by algorithms and artificial intelligence' Intl J. of Human Rights 24, 10 (2020) 1572-1593, at https://www.tandfonline.com/doi/pdf/10.1080/13642987.2020.1743976
Brennen S. & Kreiss D. (2016) 'Digitalization and Digitization' International Encyclopedia of Communication Theory and Philosophy, October 2016, PrePrint at http://culturedigitally.org/2014/09/digitalization-and-digitization/
Brownsword R. (2008) 'Rights, Regulation and the Technological Revolution' Oxford University Press, 2008
Chen H., Chiang R.H.L. & Storey V.C. (2012) 'Business Intelligence and Analytics: From Big Data to Big Impact' MIS Quarterly 36,4 (December 2012) 1165-1188
Cheng H.-Z., et al. (2024) 'Have AI-Generated Texts from LLM Infiltrated the Realm of Scientific Writing? A Large-Scale Analysis of Preprint Platforms' arXiv, 26 March 2024, at https://www.biorxiv.org/content/biorxiv/early/2024/03/26/2024.03.25.586710.full.pdf
Chiang T. (2023) 'ChatGPT Is a Blurry JPEG of the Web' The New Yorker, 9 February 2023, at https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web#:~:text=Think%20of%20ChatGPT%20as%20a,ever%20get%20is%20an%20approximation
Chu X. & Ilyas I.F. (2016) 'Qualitative Data Cleaning' Proc. VLDB Endowment 9,13 (2016) at https://dl.acm.org/doi/pdf/10.14778/3007263.3007320
Clark L. (2012) 'Google's Artificial Brain Learns to Find Cat Videos' Wired Magazine, 26 June 2012, at https://www.wired.com/2012/06/google-x-neural-network/
Clarke R. (1992) 'Extra-Organisational Systems: A Challenge to the Software Engineering Paradigm' Proc. IFIP World Congress, Madrid, September 1992, PrePrint at https://www.rogerclarke.com/SOS/PaperExtraOrgSys.html
Clarke R. (1994) 'The Digital Persona and its Application to Data Surveillance' The Information Society 10,2 (June 1994) 77-92, PrePrint at http://www.rogerclarke.com/DV/DigPersona.html
Clarke R. (2014) 'The Regulation of Civilian Drones' Impacts on Public Safety' Computer Law & Security Review 30, 3 (June 2014) 263-285, PrePrint at http://www.rogerclarke.com/SOS/Drones-PS.html
Clarke R. (2016) 'Big Data, Big Risks' Information Systems Journal 26, 1 (January 2016) 77-90, PrePrint at http://www.rogerclarke.com/EC/BDBR.html
Clarke R. (2018) 'Guidelines for the Responsible Application of Data Analytics' Computer Law & Security Review 34,3 (May-Jun 2018) 467- 476, PrePrint at http://www.rogerclarke.com/EC/GDA.html
Clarke R. (2019a) 'Risks Inherent in the Digital Surveillance Economy: A Research Agenda' Journal of Information Technology 34,1 (Mar 2019) 59-80, PrePrint at http://www.rogerclarke.com/EC/DSE.html
Clarke R. (2019b) 'Why the World Wants Controls over Artificial Intelligence' Computer Law & Security Review 35, 4 (2019) 423-433, PrePrint at http://www.rogerclarke.com/EC/AII.html
Clarke R. (2019c) 'Principles and Business Processes for Responsible AI' Computer Law & Security Review 35,4 (2019) 410-422, PrePrint at http://www.rogerclarke.com/EC/AIP.html
Clarke R. (2019d) 'Regulatory Alternatives for AI' Computer Law & Security Review 35, 4 (2019) 398-409, PrePrint at http://www.rogerclarke.com/EC/AIR.html
Clarke R. (2020a) 'Responsible AI Technologies, Artefacts, Systems and Applications: 50 Principles' Xamax Consultancy Pty Ltd, 15 January 2020, at https://www.rogerclarke.com/EC/RAIC-50Ps-200115.pdf
Clarke R. (2020b) 'RegTech Opportunities in the Platform-Based Business Sector' Proc. Bled eConference, June 2020, pp. 79-106, PrePrint at http://rogerclarke.com/EC/RTFB.html
Clarke R. (2021) 'A Comprehensive Framework for Regulatory Regimes as a Basis for Effective Privacy Protection' Proc. 14th Computers, Privacy and Data Protection Conference (CPDP'21), Brussels, 27-29 January 2021, PrePrint at http://rogerclarke.com/DV/RMPP.html
Clarke R. (2022a) 'Research Opportunities in the Regulatory Aspects of Electronic Markets' Electronic Markets 32, 1 (Jan-Mar 2022) 179-200, PrePrint at http://rogerclarke.com/EC/RAEM.html
Clarke R. (2022b) 'Responsible Application of Artificial Intelligence to Surveillance: What Prospects?' Information Polity 27, 2 (Jun 2022) 175-191, PrePrint at http://rogerclarke.com/DV/AIP-S.html
Clarke R. (2023) 'The Re-Conception of AI: Beyond Artificial, and Beyond Intelligence' IEEE Transactions on Technology & Society 4,1 (March 2023) 24-33, PrePrint at http://rogerclarke.com/EC/AITS.html
Clarke R. (2024) 'The Authentication of Textual Assertions' Proc. 10th IS Foundations Workshop, ANU, 10-11 September 2024, PrePrint at http://rogerclarke.com/ID/ATS-ISF.html
Clarke R. & Bennett Moses L. (2014) 'The Regulation of Civilian Drones' Impacts on Public Safety' Computer Law & Security Review 30, 3 (June 2014) 263-285, PrePrint at http://www.rogerclarke.com/SOS/Drones-PS.html
Clarke R. & Taylor K. (2018) 'Towards Responsible Data Analytics: A Process Approach' Proc. Bled eConference, 17-20 June 2018, PrePrint at http://www.rogerclarke.com/EC/BDBP.html
Crouse M. (2024) 'Generative AI Defined: How it Works, Benefits and Dangers' Tech Republic, 21 June 2024, at https://www.techrepublic.com/article/what-is-generative-ai/
Currie G. (2023) 'A Conversation with ChatGPT' J Nucl Med Technol 51 (2023) 255-260, at https://tech.snmjournals.org/content/jnmt/51/3/255.full.pdf
Custers B. & Vrabec H. (2024) 'Tell me something new: data subject rights applied to inferred data and profiles' Computer Law & Security Review 52 (2024)
De Angeli A., Johnson G.I. & Coventry L. (2001) 'The unfriendly user: exploring social reactions to chatterbots' Proc. Int'l Conf. on Affective Human Factors Design, Asean Academic Press, London, 2001, at https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=030e394decf7c07cc358d0f24fc3d5e46bc55219
DeMartini G., Roitero K. & Mizzaro S. (2021) 'Managing Bias in Human-Annotated Data: Moving Beyond Bias Removal' arXiv, 26 October 2021, at https://arxiv.org/pdf/2110.13504https://arxiv.org/pdf/2110.13504
Doane J. (2024) 'AI/ML Learning Essentials 10: A Primer on Deep Learning and Neural Networks' Medium, March 2024, at h https://medium.com/@jonathandoanephd/ai-ml-learning-essentials-10-a-primer-on-deep-learning-and-neural-networks-27521021b91f
Dodge J. et al. (2021) 'Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus' arXiv.org, 30 Sep 2021, at https://arxiv.org/pdf/2104.08758
Dodgson N. (2023) 'Artificial intelligence: ChatGPT and human gullibility' Policy Quarterly 19,3 (August 2023) 19-24, at https://ojs.victoria.ac.nz/pq/article/download/8308/7374/11943
Dreyfus H.L. (1992) 'What Computers Still Can't Do: A Critique of Artificial Reason' MIT Press, 1972, 1979, 1992
EC (2019) 'Ethics Guidelines for Trustworthy AI' High-Level Expert Group on Artificial Intelligence, European Commission, April 2019, at https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=58477
EDPS (2023) 'TechDispatch #2/2023 - Explainable Artificial Intelligence' European Data Protection Supervisor, 16 November 2023, at https://www.edps.europa.eu/data-protection/our-work/publications/techdispatch/2023-11-16-techdispatch-22023-explainable-artificial-intelligence_en
Engelbart D.C. (1962) 'Augmenting Human Intellect: A Conceptual Framework' SRI Summary Report AFOSR-3223, Stanford Research Institute, October 1962, at https://dougengelbart.org/pubs/augment-3906.html
Engstrom D.F., Ho D.E., Sharkey C.M. & Cuellar N.-F. (2020) 'Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies' NYU School of Law, Public Law Research Paper No. 20-54, April 2020, at https://ssrn.com/abstract=3551505
Er M.C. (1988) 'Decision Support Systems: A Summary, Problems, and Future Trends ' Decision Support Systems 4 (1988) 355-363, at https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=b46fe8ecb8ee899aa23fda35422cbdd1645a31fc
Fischer-H?bner, S. & Lindskog, H. (2001). 'Teaching Privacy-Enhancing Technologies,' Proceedings of the IFIP WG 11.8 2nd World Conference on Information Security Education, Perth, Australia, 2001
Jones E. (2023) 'What is a foundation model?' Lovelace Institute, 17 July 2023, at https://www.adalovelaceinstitute.org/resource/foundation-models-explainer/
Foote K.D. (2023) 'A Brief History of Large Language Models' Dataversity, 28 December 2023, at https://www.dataversity.net/a-brief-history-of-large-language-models/
Gatt A. & Krahmer E. (2018) 'Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation' Journal of Artificial Intelligence Research 61 (2018) 65-170, at https://www.jair.org/index.php/jair/article/download/11173/26378
Gupta M., Akiri C., Aryal K., Parker E. & Praharaj L. (2023) 'From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy' IEEE Access 11 (2023) 80218-80254, at https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=10198233
Hagendorff T. (2024) 'Mapping the ethics of generative ai: A comprehensive scoping review' arXiv, 2024, at https://arxiv.org/pdf/2402.08323
Hardesty L. (2017) 'Explained: Neural networks' MIT News, 14 April 2017, at https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414
Heersmink R., de Rooij B., Vazquez M.J.C. & Colombo M. (2024) 'A phenomenology and epistemology of large language models: transparency, trust, and trustworthiness' Ethics and Information Technology 26,41 (2024), at https://link.springer.com/article/10.1007/s10676-024-09777-3?fromPaywallRec=false
Herbosch M. (2024) 'Fraud by generative AI chatbots: On the thin line between deception and negligence' Computer Law & Security Review 52 (April 2024) 105941
Hobson N. (2024) 'The Future of Online Search: A Closer Look at Perplexity AI' NevilleHobson.com, 5 February 2024, at https://www.nevillehobson.com/2024/02/05/the-future-of-online-search-a-closer-look-at-perplexity-ai/
Holmstr?m L. & Carroll N. (2024) 'How organizations can innovate with generative AI' Business Horizons, online 23 February 2024, at https://www.sciencedirect.com/science/article/pii/S0007681324000247
IBM (2023) 'What is generative AI?' IBM, 20 April 2023, at https://research.ibm.com/blog/what-is-generative-AI
Javaheripi M. (2023) 'Phi-2: The surprising power of small language modelsMicrosoft Research, 12 December 2023, at https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/
Jazwinska K. & Chandrasekar A. (2024) 'How ChatGPT Search (Mis)represents Publisher Content' Columbia Journalism Review, 27 November 2024, at https://www.cjr.org/tow_center/how-chatgpt-misrepresents-publisher-content.php
Ji Z. et al. (2023) 'Survey of Hallucination in Natural Language Generation' ACM Comput. Surv. 55,12 (March 2023) 248:1-38, at https://dl.acm.org/doi/pdf/10.1145/3571730
Jobin A., Ienca M. & Vayena E. (2019) 'The global landscape of AI ethics guidelines' Nature Machine Intelligence 1 (September 2019) 389-399, at https://doi.org/10.1038/s42256-019-0088-2
Karanikolas N., Manga E., Samaridi N., Tousidou E. & Vassilakopoulos N. (2023) 'Large Language Models versus Natural Language Understanding and Generation' Proc. PCI, 24?26 November 2023, Lamia, Greece, pp.278-290, at https://dl.acm.org/doi/pdf/10.1145/3635059.3635104
Kattnig M., Angerschmid A., Reichel T. & Kern R. (2024) 'Assessing trustworthy AI: Technical and legal perspectives of fairness in AI' Computer Law & Security Review 55 (2024) 106053
Kaubre (2024) 'LLM Training Data: The 8 Main Public Data Sources' Oxylabs, 27 September 2024, at https://oxylabs.io/blog/llm-training-data
Kaufmann T., Weng P., Bengs V. & H?llermeier E. (2024) 'A Survey of Reinforcement Learning from Human Feedback' arXiv, 30 April 2024, at https://arxiv.org/pdf/2312.14925
Kucherbaev P., Bozzon A. & Houben G.-J. (2018) 'Human-aided bots' IEEE Internet Comput. 22 (2018) 36-43, at https://scholar.archive.org/work/3uuziqg7qzagppltbhjpz7aoqa/access/wayback/https://s3-hzgw.figshare.com/pstorage-ams-784855595/17710232/08371205.pdf
Labruna T., Brenna S., Zaninello A. & Magnini B. (2023) 'Unraveling ChatGPT: A Critical Analysis of AI-Generated Goal-Oriented Dialogues and Annotations' arXiv, 23 May 2023, at https://arxiv.org/pdf/2305.14556
Lappin S. (2024) 'Assessing the Strengths and Weaknesses of Large Language Models' J of Log Lang and Inf 33 (2024) 9-20, at https://doi.org/10.1007/s10849-023-09409-x
Lattimer K. (2023) 'Dealing with casual bias in textual corpuses' Medium, 2 May 2023, at https://medium.com/@karllattimer/dealing-with-casual-bias-in-textual-corpuses-20351cf37998
Levantino F.P. (2023) 'Generative and AI-powered oracles: ÅgWhat will they say about you?Åh' Computer Law & Security Review 51 (November 2023) 105898
Linardatos P., Papastefanopoulos V. & Kotsiantis S. (2021) 'Explainable AI: A Review of Machine Learning Interpretability Methods' Entropy 23,18 (2021), at https://www.mdpi.com/1099-4300/23/1/18/pdf
Lipton Z.C. (2017) 'The Mythos of Model Interpretability' Queue 16,3 (May-June 2018) 31-57, at https://dl.acm.org/doi/pdf/10.1145/3236386.3241340
McAfee A., Rock D, & Brynjolfsson E. (2023) 'How to Capitalize on Generative AI' Harvard Business Review (Nov-Dec 2023), at https://hbr.org/2023/11/how-to-capitalize-on-generative-ai
Merritt R. (2023) 'What Are Foundation Models?' Nvidia, 13 March 2023, at https://blogs.nvidia.com/blog/what-are-foundation-models/
Morris C. (1938) 'Foundations of the theory of signs' University of Chicago Press, 1938
Nah F.F.-H., Zheng R., Cai J., Siau K. & Chen L. (2023) 'Generative AI and ChatGPT: Applications, challenges, and AI-human collaboration' Journal of Information Technology Case and Application Research 25,3 (2023) 277-304, at https://www.tandfonline.com/doi/epdf/10.1080/15228053.2023.2233814
Nvidia (2023) 'Large Language Models Explained' Nvidia, 31 May 2023, at https://www.nvidia.com/en-us/glossary/large-language-models/
Nvidia (2024) 'What is Generative AI?' Nvidia, 1 May 2024, at https://www.nvidia.com/en-gb/glossary/generative-ai/
OICT (2023) 'Generative AI Primer' UN Office of Information and Communications Technology, 29 Aug 2023, at https://unite.un.org/sites/unite.un.org/files/generative_ai_primer.pdf
OpenAI (2022) 'Introducing ChatGPT' OpenAI, 30 November 2022, at https://openai.com/index/chatgpt/, accessed 27 October 2024
OpenAI (2024) 'Moderation' OpenAI, undated, apparently of 2024, at https://platform.openai.com/docs/guides/moderation, accessed 27 October 2024
Pan W., Chen Q., Xu X., Che W. & Qin L. (2023) 'A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understanding' arXive, 9 April 2023, at https://arxiv.org/pdf/2304.04256
Panigutti C. et al. (2023) 'The role of explainable AI in the context of the AI Act' Proc. ACM Conference on Fairness, Accountability, and Transparency, June 2023, pp. 1139-1150, at https://tinyurl.com/e7fkz76y
Sabherwal R. & Grover V. (2024) 'The Societal Impacts of Generative Artificial Intelligence: A Balanced Perspective' Journal of the Association for Information Systems, 25,1 (2024) 13-22
Schick T. & Sch?tze H. (2021) 'It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners' arXiv, 12 April 2021, at https://arxiv.org/pdf/2009.07118
Schlagwein D. & Willcocks L. (2023) ''ChatGPT et al.': The ethics of using (generative) artificial intelligence in research and science' Journal of Information Technology 38,3 (2023) 232-238
Schneier B. & Sanders N.E. (2025) 'AI Mistakes Are Very Different From Human Mistakes: We need new security systems designed to deal with their weirdness' IEEE Spectrum, 13 Jan 2025, at https://spectrum.ieee.org/ai-mistakes-schneier
Sharma N., Liao Q.V. & Xiao Z. (2024) 'Generative Echo Chamber? Effects of LLM-Powered Search Systems on Diverse Information Seeking' Proc. CHI'24, May 2024, Honolulu, at https://dl.acm.org/doi/pdf/10.1145/3613904.3642459
Shi C., Yang H., Cai D., Zhang Z., Wang Y., Yang Y. & Lam W. (2024) 'A Thorough Examination of Decoding Methods in the Era of LLMs' arXiv, 8 October 2024, at https://arxiv.org/pdf/2402.06925
Siddique S., Haque M.A., George G., Gupta K.D., Gupta D. & Gupta M.J.H. (2024) 'Survey on Machine Learning Biases and Mitigation Techniques' Digital 4 (2024) 1-68, at https://www.mdpi.com/2673-6470/4/1/1/pdf
Smith A. (2021) 'Face scanning and 'social scoring' AI can have 'catastrophic effects' on human rights, UN says' The Independent, 16 September 2021, at https://www.independent.co.uk/tech/artificial-intelligence-united-nations-face-scan-social-score-b1921311.html
Solaiman I. et al. (2023) 'Evaluating the Social Impact of Generative AI Systems in Systems and Society' arXiv, June 2023, at https://arxiv.org/pdf/2306.05949
Talaat A.S. (2023) 'Sentiment analysis classification system using hybrid BERT models' Journal of Big Data 10,110 (2023), at https://journalofbigdata.springeropen.com/articles/10.1186/s40537-023-00781-w
TRS (2019) 'Explainable AI: the basics' The Royal Society, November 2019, at https://ec.europa.eu/futurium/en/system/files/ged/ai-and-interpretability-policy-briefing_creative_commons.pdf
Turing (2024) 'Prominent Data Collection Methods and Tools for LLMs' Turing Institute, 2 May 2024, at https://www.turing.com/resources/data-collection-methods-and-tools-for-llms
Turing A.M. (1950) 'Computing Machinery and Intelligence' Mind LIX,236 (October 1950) 433-460
Vaswani A., Jones L., Shazeer N., Parmar N., Uszkoreit J., Gomez A.N., Kaiser L. & Polosukhin I. (2017) 'Attention Is All You Need' Proc. 31st Conf. Neural Information Processing Systems (NIPS 2017), Long Beach, CA, at https://user.phil.hhu.de/~cwurm/wp-content/uploads/2020/01/7181-attention-is-all-you-need.pdf
Vinge V. (1981) 'True Names' Dell Binary Star, 1981
Wach K., Duong C.D., Ejdys J., Kazlauskait? R., Korzynski P., Mazurek G., Paliszkiewicz J. & Ziemba E. (2023) 'The dark side of generative artificial intelligence: A critical analysis of controversies and risks of ChatGPT' Entre- preneurial Business and Economics Review 11,2 (2023) 7-30, at https://doi.org/10.15678/EBER.2023.110201
Wachter S. (2024) 'Limitations and Loopholes in the EU AI Act and AI Liability Directives: What This Means for the European Union, the United States, and Beyond' Yale Journal of Law & Technology 26,3 (2024) 671-718, at https://ora.ox.ac.uk/objects/uuid:0525099f-88c6-4690-abfa-741a8c057e00/files/sht24wm314
Wachter S., Mittelstadt B. & Floridi L. (2017) 'Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation' International Data Privacy Law 7, 2 (May 2017) 76-99, at https://academic.oup.com/idpl/article/7/2/76/3860948
Wachter S., Mittelstadt B. & Russell C. (2024) 'Do large language models have a legal duty to tell the truth?' R. Soc. Open Sci. 11:240197, at https://royalsocietypublishing.org/doi/pdf/10.1098/rsos.240197
Walsh T. (2023) 'Faking It' LaTrobe University Press, 2023
Wang Y., Kung L. & Byrd T.A. (2018) 'Big data analytics: Understanding its capabilities and potential benefits for healthcare organizations' Technological Forecasting and Social Change 126 (January 2018) 3-13
Weidinger L. et al. (2023) 'Sociotechnical Safety Evaluation of Generative AI Systems' arXiv, 2023, at https://arxiv.org/pdf/2310.11986
Weizenbaum J. (1966) 'Eliza' Commun. ACM 9,1 (Jan 1966) 36-45, at https://dl.acm.org/doi/pdf/10.1145/365153.365168
Weizenbaum J. (1967) 'Contextual Understanding by Computers' Communications of the ACM l0,8 (August 1967) 474-480, at https://dl.acm.org/doi/pdf/10.1145/363534.363545
Wigan M., Adamson G., Ranim P., Dyson N. & Horton F. (2022) 'Chatbots and Explainable Artificial Intelligence' Proc. IEEE Int'l Symposium on Technology and Society (ISTAS), 2022, at https://ieeexplore.ieee.org/document/10227122
Willcocks L., Hindle J., Stanton M. & Smith J. (2023) 'Maximizing Value with Automation and Transformation: A Realist's Guide' Palgrave Macmillan, London, 2023
WIPO (2024) 'Patent Landscape Report - Generative Artificial Intelligence (GenAI)' World Intellectual Property Organization, 27 June 2024, at https://doi.org/10.34667/tind.49740
Zeng Y., Lu E. & Huangfu C. (2019) 'Linking Artificial Intelligence Principles' Proc. AAAI Workshop on Artificial Intelligence Safety (AAAI-Safe AI 2019), 27 January 2019, at https://arxiv.org/abs/1812.04814
Zewe A. (2023) 'Explained: Generative AI' MIT News 23 November 2023, at https://news.mit.edu/2023/explained-generative-ai-1109
Zohny H., McMillan J. & King M. (2023) 'Ethics of generative AI' Journal of Medical Ethics 49 (2023) 79-80, at https://jme.bmj.com/content/medethics/50/2/75.full.pdf
Adapted from Clarke (2020a)
Column 1 reproduces the 50 Principles for Responsible AI Technologies, Artefacts, Systems and Applications. These are intended to be applied by the entities responsible for all phases of AI research, invention, innovation, dissemination and application. They were derived by consolidating elements from 30 international sources on 'Ethical Analysis and IT' and 'Principles for AI'. Column 2 suggests wording relevant to the use of GenAI artefacts.
The 10 Themes and 50 Principles |
Interpretation in the Context of GenAI Artefacts |
1. Assess Positive and Negative Impacts and Implications |
|
1.1 Conceive and design only after ensuring adequate understanding of purposes and contexts |
Think before making a request |
1.2 Justify objectives |
Ask yourself whether the action you're taking is 'the right thing to do' |
1.3 Demonstrate the achievability of postulated benefits |
Recognise that, the greater the negative consequences, the more confidence is needed in the realistic chances of the intended positive outcomes being achieved |
1.4 Conduct impact assessment, including risk assessment from all stakeholders' perspectives |
Take account of your own interests, but also those of other stakeholders |
1.5 Publish sufficient information to stakeholders to enable them to conduct impact assessment |
Enable other stakeholders to consider the consequences for themselves |
1.6 Conduct consultation with stakeholders and enable their participation in design |
Gain insights into stakeholders' views |
1.7 Reflect in the design the concerns of stakeholders |
Find suitable ways to reflect stakeholders' views, balance all parties' interests, and mitigate unavoidable harm |
1.8 Justify negative impacts on individuals ('proportionality') |
Ensure that harm done is not disproportionate to the benefits achieved |
1.9 Consider alternative, less harmful ways of achieving the same objectives |
Remain awake to better ways to achieve much the same outcomes |
2. Complement Humans |
|
2.1 Design as an aid for people, to augment their capabilities, and support collaboration and inter-operability |
Treat GenAI artefacts as tools for yourself as author, analyst, decision-maker and/or real-world actor, and not as autonomous devices that you delegate work to |
2.2 Avoid design for replacement of people by independent artefacts or systems, except in circumstances in which those artefacts or systems are demonstrably more capable than people, and even then ensuring that the result is complementary to human capabilities |
Recognise GenAI's limitations, and avoid thinking of it as an author, an analyst, a decision-maker, or an actor. Sign off on inferences, decisions and actions, to make clear that they are your own responsibility, not that of a GenAI artefact |
3. Ensure Human Control |
|
3.1 Ensure human control over AI-based technology, artefacts and systems |
Think hard before using a response |
3.2 In particular, ensure control over autonomous behaviour of AI-based technology, artefacts and systems |
Avoid uncritical adoption of a response, and uncritical use use of it to draw inferences, make decisions and/or take actions. Sign off on inferences, decisions and actions, to make clear that they are your own responsibility, not that of a GenAI artefact |
3.3 Respect people's expectations in relation to personal data protections, including their awareness of data-usage, their consent, data minimisation, public transparency, design consultation and participation, and the relationship between data-usage and the data's original purpose |
When using GenAI, go beyond the basics of data protection law. People are threatened by the intrusion of unfeeling machines into organisations' decision-making about human beings. |
3.4 Respect each person's autonomy, freedom of choice and right to self-determination |
When using GenAI, recognise that you are responsible for respecting these higher-order values, because technological artefacts are incapable of doing so |
3.5 Ensure human review of inferences and decisions prior to action being taken |
When using GenAI, recognise that responsibility rests with you and your organisation |
3.6 Avoid deception of humans |
When using GenAI, provide explanations, not pretexts cover stories. If you can't explain clearly to yourself why you're doing something, don't do it. |
3.7 Avoid services being conditional on the acceptance of obscure AI-based artefacts and systems and opaque decisions |
When using GenAI, ensure any stakeholder who may be negatively affected isa aware of its use, and is able to question the rationale for the resulting inferences, decisions and/or actions |
4. Ensure Human Safety and Wellbeing |
|
4.1 Ensure people's physical health and safety ('nonmaleficence') |
When using GenAI, ensure you avoid doing bad things to other people |
4.2 Ensure people's psychological safety, by avoiding negative effects on their mental health, emotional state, inclusion in society, worth, and standing in comparison with other people |
When using GenAI, ensure you avoid doing bad things to other people |
4.3 Ensure people's wellbeing ('beneficence') |
When using GenAI, set out to do good things for people |
4.4 Implement safeguards to avoid, prevent and mitigate negative impacts and implications |
When using GenAI, ensure that your intentions to avoid doing bad things are carried through |
4.5 Avoid violation of trust |
When using GenAI, make sure your behaviour is trustworthy |
4.6 Avoid the manipulation of vulnerable people, e.g. by taking advantage of individuals' tendencies to addictions such as gambling, and to letting pleasure overrule rationality |
When using GenAI, avoid exercising power over people whose ability to protect their own interests is impaired |
5. Ensure Consistency with Human Values and Human Rights |
|
5.1 Be just / fair / impartial, treat individuals equally, and avoid unfair discrimination and bias, not only where they are illegal, but also where they are materially inconsistent with public expectations |
Recognise that GenAI, particularly when using very large and indiscriminate sources of text, provides biassed responses and discriminates against people in unacceptable ways |
5.2 Ensure compliance with human rights laws |
When using GenAI, recognise that human rights responsibility rests with you and your organisation |
5.3 Avoid restrictions on, and promote, people's freedom ofmovement |
When using GenAI, recognise that human rights responsibility rests with you and your organisation |
5.4 Avoid interference with, and promote privacy, family, home orreputation |
When using GenAI, recognise that human rights responsibility rests with you and your organisation |
5.5 Avoid interference with, and promote, the rights of freedom of information, opinion and expression, of freedom of assembly, of freedom of association, of freedom to participate in public affairs, and of freedom to access public services |
When using GenAI, recognise that human rights responsibility rests with you and your organisation |
5.6 Where interference with human values or human rights is outweighed by other factors, ensure that the interference is no greater than is justified ('harm minimisation') |
When using GenAI, minimise the harm to people, and design and implement measures to mitigate unavoidable harm |
6. Deliver Transparency and Auditability |
|
6.1 Ensure that the fact that a process is AI-based is transparent to all stakeholders |
When using GenAI, be open and honest about it, to all stakeholders |
6.2 Ensure that data provenance, and the means whereby inferences are drawn from it, decisions are made, and actions are taken, are logged and can be reconstructed |
Do not use GenAI artefacts that cannot provide clear explanations, because the business process and the rationale must be able to be communicated to those affected, and to those exercising regulatory powers |
6.3 Ensure that people are aware of inferences, decisions and actions that affect them, and have access to humanly-understandable explanations of how they came about |
When using GenAI, be open and honest to affected people about what it is used for, and be ready and able to explain the business process and the rationale to those affected, and to those exercising regulatory powers |
7. Embed Quality Assurance |
|
7.1 Ensure effective, efficient and adaptive performance ofintended functions |
Do not use GenAI if how it works is opaque, because responsibility rests with you and your organisation, and in the absence of a transparent business process and rationale, you cannot fulfil that responsibility |
7.2 Ensure data quality and data relevance |
Do not use GenAI if the data it uses is opaque, because responsibility rests with you and your organisation, and in the absence of data quality and data relevance, you cannot fulfil that responsibility |
7.3 Justify the use of data, commensurate with each data-item'ssensitivity |
When using GenAI, review the input provided to ensure that the process has used only information that is relevant and whose use is justifiable |
7.4 Ensure security safeguards against inappropriate data access, modification and deletion, commensurate with its sensitivity |
Do not use GenAI if it appropriates data that you provide to it, or may apply it to other purposes |
7.5 Deal fairly with people ('faithfulness', 'fidelity') |
Do not use GenAI responses without human review of the provisional inferences drawn, recommendations, decisions or actions, or without being convinced that the use is fair to all stakeholders |
7.6 Ensure that inferences are not drawn from data using invalid or unvalidated techniques |
When using GenAI, do not rely on responses without applying an understanding of the context, using your common sense, andapplying 'the pub test' |
7.7 Test result validity, and address the problems that are detected |
When using GenAI, do not rely on responses without checking their reasonableness in comparison with norms, cases, precedents, archetypes and your own knowledge of the real world In designing tests, take into account that patterns of LLM errors and hallucinations are very different from those of humans |
7.8 Impose controls in order to ensure that the intended safeguards are in place and effective |
When using GenAI, ensure your intentions to avoid doing bad things are carried through |
7.9 Conduct audits of controls |
When using GenAI, check that all controls are functioning as intended |
8. Exhibit Robustness and Resilience |
|
8.1 Deliver and sustain appropriate security safeguards against the risk of compromise of intended functions arising from both passive threats and active attacks, commensurate with the significance of the benefits and the potential to cause harm |
When using GenAI, make sure service threats and vulnerabilities are managed |
8.2 Deliver and sustain appropriate security safeguards against the risk of inappropriate data access, modification and deletion, arising from both passive threats and active attacks, commensurate with the data's sensitivity |
When using GenAI, make sure data threats and vulnerabilities are managed |
8.3 Conduct audits of the justification, the proportionality, the transparency, and the harm avoidance, prevention and mitigation measures and controls |
When using GenAI, check that all safeguards and controls are functioning as intended |
8.4 Ensure resilience, in the sense of prompt and effective recovery from incidents |
When using GenAI, check that fallback arrangements are in place and functioning as intended |
9. Ensure Accountability for Legal and Moral Obligations |
|
9.1 Ensure that the responsible entity is apparent or can be readily discovered by any party |
When using GenAI, be open about where the responsibility lies |
9.2 Ensure that effective remedies exist, in the forms of complaints processes, appeals processes, and redress where harmful errors have occurred |
When using GenAI, operate well-documented processes to receive, address and manage incidents and cases |
10. Enforce, and Accept Enforcement of, Liabilities and Sanctions |
|
10.1 Ensure that complaints, appeals and redress processes operate effectively |
When using GenAI, check that all controls are functioning as intended |
10.2 Comply with external complaints, appeals and redress processes and outcomes, including, in particular, provision of timely, accurate and complete information relevant to cases |
When using GenAI, cooperate constructively with regulatory agencies |
This article benefited from feedback from colleagues, and from the CLSR reviewers.
Roger Clarke is Principal of Xamax Consultancy Pty Ltd, Canberra. He is also a Visiting Professorial Fellow associated with UNSW Law & Justice, and a Visiting Professor in the Research School of Computer Science at the Australian National University.
| Personalia |
Photographs Presentations Videos |
Access Statistics |
 |
The content and infrastructure for these community service pages are provided by Roger Clarke through his consultancy company, Xamax. From the site's beginnings in August 1994 until February 2009, the infrastructure was provided by the Australian National University. During that time, the site accumulated close to 30 million hits. It passed 75 million in late 2024. Sponsored by the Gallery, Bunhybee Grasslands, the extended Clarke Family, Knights of the Spatchcock and their drummer |
Xamax Consultancy Pty Ltd ACN: 002 360 456 78 Sidaway St, Chapman ACT 2611 AUSTRALIA Tel: +61 2 6288 6916 |
Created: 13 June 2024 - Last Amended: 6 April 2025 by Roger Clarke - Site Last Verified: 15 February 2009
This document is at www.rogerclarke.com/EC/RGAI-C.html
Mail to Webmaster - © Xamax Consultancy Pty Ltd, 1995-2024 - Privacy Policy