Before I get into Anthropic’s Responsible Scaling Policy (RSP), it’s worth explaining some of the unique challenges around measuring AI risks that led us to develop our RSP. The most important thing to understand about AI is how quickly it is moving. A few years ago, AI systems could barely string together a coherent sentence. Today they can pass medical exams, write poetry, and tell jokes. This rapid progress is ultimately driven by the amount of available computation, which is growing by 8x per year and is unlikely to slow down in the next few years. The general trend of rapid improvement is predictable, however, it is actually very difficult to predict when AI will acquire specific skills or knowledge. This unfortunately includes dangerous skills, such as the ability to construct biological weapons¹. We are thus facing a number of potential AI-related threats which, although relatively limited given today’s systems, are likely to become very serious at some unknown point in the near future. This is very different from most other industries: imagine if each new model of car had some chance of spontaneously sprouting a new (and dangerous) power, like the ability to fire a rocket boost or accelerate to supersonic speeds.

We need both a way to frequently monitor these emerging risks, and a protocol for responding appropriately when they occur. Responsible scaling policies—initially suggested by the Alignment Research Center—attempt to meet this need. Anthropic published its RSP in September, and was the first major AI company to do so. It has two major components:

• First, we’ve come up with a system called AI safety levels (ASL), loosely modeled after the internationally recognized BSL system for handling biological materials. Each ASL level has an if-then structure: if an AI system exhibits certain dangerous capabilities, then we will not deploy it or train more powerful models, until certain safeguards are in place.
• Second, we test frequently for these dangerous capabilities at regular intervals along the compute scaling curve. This is to ensure that we don’t blindly create dangerous capabilities without even knowing we have done so.

In our system, ASL-1 represents models with little to no risk—for example a specialized AI that plays chess. ASL-2 represents where we are today: models that have a wide range of present-day risks, but do not yet exhibit truly dangerous capabilities that could lead to catastrophic outcomes if applied to fields like biology or chemistry. Our RSP requires us to implement present-day best practices for ASL-2 models, including model cards, external red-teaming, and strong security.

ASL-3 is the point at which AI models become operationally useful for catastrophic misuse in CBRN areas, as defined by experts in those fields and as compared to existing capabilities and proofs of concept. When this happens we require the following measures:

• Unusually strong security measures such that non-state actors cannot steal the weights, and state actors would need to expend significant effort to do so.
• Despite being (by definition) inherently capable of providing information that operationally increases CBRN risks, the deployed versions of our ASL-3 model must never produce such information, even when red-teamed by world experts in this area working together with AI engineers. This will require research breakthroughs, but we believe it is a necessary condition of safety.
• ASL-4 must be rigorously defined by the time ASL-3 is reached.

ASL-4 represents an escalation of the catastrophic misuse risks from ASL-3, and also adds a new risk: concerns about autonomous AI systems that escape human control and pose a significant threat to society. Roughly, ASL-4 will be triggered when either AI systems become capable of autonomy at a near-human level, or become the main source in the world of at least one serious global security threat, such as bioweapons. It is likely that at ASL-4 we will require a detailed and precise understanding of what is going on inside the model, in order to make an “affirmative case” that the model is safe.

 

Next, I’ll briefly mention some of our key practices and lessons learned, which we hope are helpful to others in crafting an RSP. First, deep executive involvement is critical. As CEO, I personally spent 10-20% of my time on the RSP for 3 months—I wrote multiple drafts from scratch, in addition to devising and proposing the ASL system. One of my co-founders devoted 50% of their time to developing the RSP for 3 months. Together, this sent a meaningful signal to employees that Anthropic’s leadership team takes the matter of AI safety seriously and is firmly committed to responsible scaling at the frontier.

Second, make the protocols outlined in the RSP into product and research requirements, such that they become baked into company planning and drive team roadmaps and expansion plans. Set the expectation that missing RSP deadlines directly impacts the company’s ability to continue training models and ship products on time. At Anthropic, teams such as security, trust and safety, red teaming, and interpretability, have had to greatly ramp up hiring to have a reasonable chance of achieving ASL-3 safety measures by the time we have ASL-3 models.

Third, accountability is necessary. Anthropic’s RSP is a formal directive of its board, which ultimately is accountable to our Long Term Benefit Trust, an external panel of experts with no financial stake in Anthropic. On the operational side, we will put in place a whistleblower policy before we reach ASL-3 and already have an officer responsible for ensuring compliance with the RSP and reporting to our Long Term Benefit Trust. As risk increases, we expect that stronger forms of accountability will be necessary.

Finally, I’d like to discuss the relationship between RSPs and regulation. RSPs are not intended as a substitute for regulation, but rather a prototype for it. I don’t mean that we want Anthropic’s RSP to be literally written into laws—our RSP is just a first attempt at addressing a difficult problem, and is almost certainly imperfect in a bunch of ways. Importantly, as we begin to execute this first iteration, we expect to learn a vast amount about how to sensibly operationalize such commitments. Our hope is that the general idea of RSPs will be refined and improved across companies, and that in parallel with that, governments from around the world—such as those in this room—can take the best elements of each and turn them into well-crafted testing and auditing regimes with accountability and oversight. We’d like to encourage a “race to the top'' in RSP-style frameworks, where both companies and countries build off each others’ ideas, ultimately creating a path for the world to wisely manage the risks of AI without unduly disrupting the benefits.

Generative artificial intelligence (generative AI, GenAI,^[1] or GAI) is a subset of artificial intelligence that uses generative models to produce text, images, videos, or other forms of data.^[2][3][4] These models learn the underlying patterns and structures of their training data and use them to produce new data^[5][6] based on the input, which often comes in the form of natural language prompts.^[7][8]

Improvements in transformer-based deep neural networks, particularly large language models (LLMs), enabled an AI boom of generative AI systems in the early 2020s. These include chatbots such as ChatGPT, Copilot, Gemini, and LLaMA; text-to-image artificial intelligence image generation systems such as Stable Diffusion, Midjourney, and DALL-E; and text-to-video AI generators such as Sora.^{[9][10][11][12]} Companies such as OpenAI, Anthropic, Microsoft, Google, and Baidu as well as numerous smaller firms have developed generative AI models.^[7][13][14]

Generative AI has uses across a wide range of industries, including software development, healthcare, finance, entertainment, customer service,^[15] sales and marketing,^[16] art, writing,^[17] fashion,^[18] and product design.^[19] However, concerns have been raised about the potential misuse of generative AI such as cybercrime, the use of fake news or deepfakes to deceive or manipulate people, and the mass replacement of human jobs.^[20][21] Intellectual property law concerns also exist around generative models that are trained on and emulate copyrighted works of art.^[22]

Since its inception, researchers in the field have raised philosophical and ethical arguments about the nature of the human mind and the consequences of creating artificial beings with human-like intelligence; these issues have previously been explored by myth, fiction and philosophy since antiquity.^[23] The concept of automated art dates back at least to the automata of ancient Greek civilization, where inventors such as Daedalus and Hero of Alexandria were described as having designed machines capable of writing text, generating sounds, and playing music.^[24][25] The tradition of creative automations has flourished throughout history, exemplified by Maillardet's automaton created in the early 1800s.^[26] Markov chains have long been used to model natural languages since their development by Russian mathematician Andrey Markov in the early 20th century. Markov published his first paper on the topic in 1906,^[27][28] and analyzed the pattern of vowels and consonants in the novel Eugeny Onegin using Markov chains. Once a Markov chain is learned on a text corpus, it can then be used as a probabilistic text generator.^[29][30]

The academic discipline of artificial intelligence was established at a research workshop held at Dartmouth College in 1956 and has experienced several waves of advancement and optimism in the decades since.^[31] Artificial Intelligence research began in the 1950s with works like Computing Machinery and Intelligence (1950) and the 1956 Dartmouth Summer Research Project on AI. Since the 1950s, artists and researchers have used artificial intelligence to create artistic works. By the early 1970s, Harold Cohen was creating and exhibiting generative AI works created by AARON, the computer program Cohen created to generate paintings.^[32]

The terms generative AI planning or generative planning were used in the 1980s and 1990s to refer to AI planning systems, especially computer-aided process planning, used to generate sequences of actions to reach a specified goal.^[33][34] Generative AI planning systems used symbolic AI methods such as state space search and constraint satisfaction and were a "relatively mature" technology by the early 1990s. They were used to generate crisis action plans for military use,^[35] process plans for manufacturing^[33] and decision plans such as in prototype autonomous spacecraft.^[36]

Since its inception, the field of machine learning used both discriminative models and generative models, to model and predict data. Beginning in the late 2000s, the emergence of deep learning drove progress and research in image classification, speech recognition, natural language processing and other tasks. Neural networks in this era were typically trained as discriminative models, due to the difficulty of generative modeling.^[37]

In 2014, advancements such as the variational autoencoder and generative adversarial network produced the first practical deep neural networks capable of learning generative models, as opposed to discriminative ones, for complex data such as images. These deep generative models were the first to output not only class labels for images but also entire images.

In 2017, the Transformer network enabled advancements in generative models compared to older Long-Short Term Memory models,^[38] leading to the first generative pre-trained transformer (GPT), known as GPT-1, in 2018.^[39] This was followed in 2019 by GPT-2 which demonstrated the ability to generalize unsupervised to many different tasks as a Foundation model.^[40]

The new generative models introduced during this period allowed for large neural networks to be trained using unsupervised learning or semi-supervised learning, rather than the supervised learning typical of discriminative models. Unsupervised learning removed the need for humans to manually label data, allowing for larger networks to be trained.^[41]

In 2021, the release of DALL-E, a transformer-based pixel generative model, followed by Midjourney and Stable Diffusion marked the emergence of practical high-quality artificial intelligence art from natural language prompts.

In 2022, the public release of ChatGPT popularized the use of generative AI for general-purpose text-based tasks.^[42]

In March 2023, GPT-4 was released. A team from Microsoft Research argued that "it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system".^[43] Other scholars have disputed that GPT-4 reaches this threshold, calling generative AI "still far from reaching the benchmark of ‘general human intelligence’" as of 2023.^[44] In 2023, Meta released an AI model called ImageBind which combines data from text, images, video, thermal data, 3D data, audio, and motion which is expected to allow for more immersive generative AI content.^[45][46]

According to a survey by SAS and Coleman Parkes Research, China is leading the world in adopting generative AI, with 83% of Chinese respondents using the technology, surpassing the global average of 54% and the U.S. at 65%. A UN report revealed China filed over 38,000 GenAI patents from 2014 to 2023, far exceeding the U.S.^[47]

A generative AI system is constructed by applying unsupervised machine learning (invoking for instance neural network architectures such as GANs, VAE, Transformer, ...) or self-supervised machine learning to a data set. The capabilities of a generative AI system depend on the modality or type of the data set used.

Generative AI can be either unimodal or multimodal; unimodal systems take only one type of input, whereas multimodal systems can take more than one type of input.^[48] For example, one version of OpenAI's GPT-4 accepts both text and image inputs.^[49]

Generative AI systems trained on words or word tokens include GPT-3, GPT-4, GPT-4o, LaMDA, LLaMA, BLOOM, Gemini and others (see List of large language models). They are capable of natural language processing, machine translation, and natural language generation and can be used as foundation models for other tasks.^[51] Data sets include BookCorpus, Wikipedia, and others (see List of text corpora).

In addition to natural language text, large language models can be trained on programming language text, allowing them to generate source code for new computer programs.^[52] Examples include OpenAI Codex.

Producing high-quality visual art is a prominent application of generative AI.^[53] Generative AI systems trained on sets of images with text captions include Imagen, DALL-E, Midjourney, Adobe Firefly, FLUX.1, Stable Diffusion and others (see Artificial intelligence art, Generative art, and Synthetic media). They are commonly used for text-to-image generation and neural style transfer.^[54] Datasets include LAION-5B and others (see List of datasets in computer vision and image processing).

Generative AI can also be trained extensively on audio clips to produce natural-sounding speech synthesis and text-to-speech capabilities, exemplified by ElevenLabs' context-aware synthesis tools or Meta Platform's Voicebox.^[55]

Generative AI systems such as MusicLM^[56] and MusicGen^[57] can also be trained on the audio waveforms of recorded music along with text annotations, in order to generate new musical samples based on text descriptions such as a calming violin melody backed by a distorted guitar riff.

Audio deepfakes of lyrics have been generated, like the song Savages, which used AI to mimic rapper Jay-Z's vocals. Music artist's instrumentals and lyrics are copyrighted but their voices aren't protected from regenerative AI yet, raising a debate about whether artists should get royalties from audio deepfakes.^[58]

Many AI music generators have been created that can be generated using a text phrase, genre options, and looped libraries of bars and riffs.^[59]

Generative AI trained on annotated video can generate temporally-coherent, detailed and photorealistic video clips. Examples include Sora by OpenAI,^[12] Gen-1 and Gen-2 by Runway,^[60] and Make-A-Video by Meta Platforms.^[61]

Generative AI can also be trained on the motions of a robotic system to generate new trajectories for motion planning or navigation. For example, UniPi from Google Research uses prompts like "pick up blue bowl" or "wipe plate with yellow sponge" to control movements of a robot arm.^[62] Multimodal "vision-language-action" models such as Google's RT-2 can perform rudimentary reasoning in response to user prompts and visual input, such as picking up a toy dinosaur when given the prompt pick up the extinct animal at a table filled with toy animals and other objects.^[63]

Artificially intelligent computer-aided design (CAD) can use text-to-3D, image-to-3D, and video-to-3D to automate 3D modeling.^[64] AI-based CAD libraries could also be developed using linked open data of schematics and diagrams.^[65] AI CAD assistants are used as tools to help streamline workflow.^[66]

Generative AI models are used to power chatbot products such as ChatGPT, programming tools such as GitHub Copilot,^[67] text-to-image products such as Midjourney, and text-to-video products such as Runway Gen-2.^[68] Generative AI features have been integrated into a variety of existing commercially available products such as Microsoft Office (Microsoft Copilot),^[69] Google Photos,^[70] and the Adobe Suite (Adobe Firefly).^[71] Many generative AI models are also available as open-source software, including Stable Diffusion and the LLaMA^[72] language model.

Smaller generative AI models with up to a few billion parameters can run on smartphones, embedded devices, and personal computers. For example, LLaMA-7B (a version with 7 billion parameters) can run on a Raspberry Pi 4^[73] and one version of Stable Diffusion can run on an iPhone 11.^[74]

Larger models with tens of billions of parameters can run on laptop or desktop computers. To achieve an acceptable speed, models of this size may require accelerators such as the GPU chips produced by NVIDIA and AMD or the Neural Engine included in Apple silicon products. For example, the 65 billion parameter version of LLaMA can be configured to run on a desktop PC.^[75]

The advantages of running generative AI locally include protection of privacy and intellectual property, and avoidance of rate limiting and censorship. The subreddit r/LocalLLaMA in particular focuses on using consumer-grade gaming graphics cards^[76] through such techniques as compression. That forum is one of only two sources Andrej Karpathy trusts for language model benchmarks.^[77] Yann LeCun has advocated open-source models for their value to vertical applications^[78] and for improving AI safety.^[79]

Language models with hundreds of billions of parameters, such as GPT-4 or PaLM, typically run on datacenter computers equipped with arrays of GPUs (such as NVIDIA's H100) or AI accelerator chips (such as Google's TPU). These very large models are typically accessed as cloud services over the Internet.

In 2022, the United States New Export Controls on Advanced Computing and Semiconductors to China imposed restrictions on exports to China of GPU and AI accelerator chips used for generative AI.^[80] Chips such as the NVIDIA A800^[81] and the Biren Technology BR104^[82] were developed to meet the requirements of the sanctions.

There is free software on the market capable of recognizing text generated by generative artificial intelligence (such as GPTZero), as well as images, audio or video coming from it.^[83] Potential mitigation strategies for detecting generative AI content include digital watermarking, content authentication, information retrieval, and machine learning classifier models.^[84] Despite claims of accuracy, both free and paid AI text detectors have frequently produced false positives, mistakenly accusing students of submitting AI-generated work.^[85][86]

In the United States, a group of companies including OpenAI, Alphabet, and Meta signed a voluntary agreement with the Biden administration in July 2023 to watermark AI-generated content.^[87] In October 2023, Executive Order 14110 applied the Defense Production Act to require all US companies to report information to the federal government when training certain high-impact AI models.^[88][89]

In the European Union, the proposed Artificial Intelligence Act includes requirements to disclose copyrighted material used to train generative AI systems, and to label any AI-generated output as such.^[90][91]

In China, the Interim Measures for the Management of Generative AI Services introduced by the Cyberspace Administration of China regulates any public-facing generative AI. It includes requirements to watermark generated images or videos, regulations on training data and label quality, restrictions on personal data collection, and a guideline that generative AI must "adhere to socialist core values".^[92][93]

Generative AI systems such as ChatGPT and Midjourney are trained on large, publicly available datasets that include copyrighted works. AI developers have argued that such training is protected under fair use, while copyright holders have argued that it infringes their rights.^[94]

Proponents of fair use training have argued that it is a transformative use and does not involve making copies of copyrighted works available to the public.^[94] Critics have argued that image generators such as Midjourney can create nearly-identical copies of some copyrighted images,^[95] and that generative AI programs compete with the content they are trained on.^[96]

As of 2024, several lawsuits related to the use of copyrighted material in training are ongoing. Getty Images has sued Stability AI over the use of its images to train Stable diffusion.^[97] Both the Authors Guild and The New York Times have sued Microsoft and OpenAI over the use of their works to train ChatGPT.^[98][99]

A separate question is whether AI-generated works can qualify for copyright protection. The United States Copyright Office has ruled that works created by artificial intelligence without any human input cannot be copyrighted, because they lack human authorship.^[100] However, the office has also begun taking public input to determine if these rules need to be refined for generative AI.^[101]

The development of generative AI has raised concerns from governments, businesses, and individuals, resulting in protests, legal actions, calls to pause AI experiments, and actions by multiple governments. In a July 2023 briefing of the United Nations Security Council, Secretary-General António Guterres stated "Generative AI has enormous potential for good and evil at scale", that AI may "turbocharge global development" and contribute between $10 and $15 trillion to the global economy by 2030, but that its malicious use "could cause horrific levels of death and destruction, widespread trauma, and deep psychological damage on an unimaginable scale".^[102]

From the early days of the development of AI, there have been arguments put forward by ELIZA creator Joseph Weizenbaum and others about whether tasks that can be done by computers actually should be done by them, given the difference between computers and humans, and between quantitative calculations and qualitative, value-based judgements.^[104] In April 2023, it was reported that image generation AI has resulted in 70% of the jobs for video game illustrators in China being lost.^[105][106] In July 2023, developments in generative AI contributed to the 2023 Hollywood labor disputes. Fran Drescher, president of the Screen Actors Guild, declared that "artificial intelligence poses an existential threat to creative professions" during the 2023 SAG-AFTRA strike.^[107] Voice generation AI has been seen as a potential challenge to the voice acting sector.^[108][109]

The intersection of AI and employment concerns among underrepresented groups globally remains a critical facet. While AI promises efficiency enhancements and skill acquisition, concerns about job displacement and biased recruiting processes persist among these groups, as outlined in surveys by Fast Company. To leverage AI for a more equitable society, proactive steps encompass mitigating biases, advocating transparency, respecting privacy and consent, and embracing diverse teams and ethical considerations. Strategies involve redirecting policy emphasis on regulation, inclusive design, and education's potential for personalized teaching to maximize benefits while minimizing harms.^[110]

Generative AI models can reflect and amplify any cultural bias present in the underlying data. For example, a language model might assume that doctors and judges are male, and that secretaries or nurses are female, if those biases are common in the training data.^[111] Similarly, an image model prompted with the text "a photo of a CEO" might disproportionately generate images of white male CEOs,^[112] if trained on a racially biased data set. A number of methods for mitigating bias have been attempted, such as altering input prompts^[113] and reweighting training data.^[114]

Deepfakes (a portmanteau of "deep learning" and "fake"^[115]) are AI-generated media that take a person in an existing image or video and replace them with someone else's likeness using artificial neural networks.^[116] Deepfakes have garnered widespread attention and concerns for their uses in deepfake celebrity pornographic videos, revenge porn, fake news, hoaxes, health disinformation, financial fraud, and covert foreign election interference.^{[117][118][119][120][121][122][123]} This has elicited responses from both industry and government to detect and limit their use.^[124][125]

In July 2023, the fact-checking company Logically found that the popular generative AI models Midjourney, DALL-E 2 and Stable Diffusion would produce plausible disinformation images when prompted to do so, such as images of electoral fraud in the United States and Muslim women supporting India's Hindu nationalist Bharatiya Janata Party.^[126][127]

In April 2024, a paper proposed to use blockchain (distributed ledger technology) to promote "transparency, verifiability, and decentralization in AI development and usage".^[128]

Instances of users abusing software to generate controversial statements in the vocal style of celebrities, public officials, and other famous individuals have raised ethical concerns over voice generation AI.^{[129][130][131][132][133][134]} In response, companies such as ElevenLabs have stated that they would work on mitigating potential abuse through safeguards and identity verification.^[135]

Concerns and fandom have spawned from AI-generated music. The same software used to clone voices has been used on famous musicians' voices to create songs that mimic their voices, gaining both tremendous popularity and criticism.^{[136][137][138]} Similar techniques have also been used to create improved quality or full-length versions of songs that have been leaked or have yet to be released.^[139]

Generative AI has also been used to create new digital artist personalities, with some of these receiving enough attention to receive record deals at major labels.^[140] The developers of these virtual artists have also faced their fair share of criticism for their personified programs, including backlash for "dehumanizing" an artform, and also creating artists which create unrealistic or immoral appeals to their audiences.^[141]

Generative AI's ability to create realistic fake content has been exploited in numerous types of cybercrime, including phishing scams.^[142] Deepfake video and audio have been used to create disinformation and fraud. Former Google fraud czar Shuman Ghosemajumder has predicted that while deepfake videos initially created a stir in the media, they would soon become commonplace, and as a result, more dangerous.^[143] Additionally, large-language models and other forms of text-generation AI have been at a broad scale to create fake reviews on e-commerce websites to boost ratings.^[144] Cybercriminals have created large language models focused on fraud, including WormGPT and FraudGPT.^[145]

Recent research done in 2023 has revealed that generative AI has weaknesses that can be manipulated by criminals to extract harmful information bypassing ethical safeguards. The study presents example attacks done on ChatGPT including Jailbreaks and reverse psychology. Additionally, malicious individuals can use ChatGPT for social engineering attacks and phishing attacks, revealing the harmful side of these technologies.^[146]

Training frontier AI models requires an enormous amount of computing power. Usually only Big Tech companies have the financial resources to make such investments. Smaller start-ups such as Cohere and OpenAI end up buying access to data centers from Google and Microsoft respectively.^[147]

Scientists and journalists have expressed concerns about the environmental impact that the development and deployment of generative models are having: high CO₂ emissions,^{[148][149][150]} large amounts of freshwater used for data centers,^[151][152] and high amounts of electricity usage.^{[153][149][154]} There is also concern that these impacts may increase as these models are incorporated into widely used search engines such as Google Search and Bing;^[153] as chatbots and other applications become more popular;^[153][152] and as models need to be retrained.^[153]

Proposed mitigation strategies include factoring potential environmental costs prior to model development or data collection,^[148] increasing efficiency of data centers to reduce electricity/energy usage,^{[151][153][149][152][154][150]} building more efficient machine learning models,^{[151][149][152]} minimizing the number of times that models need to be retrained,^[150] developing a government-directed framework for auditing the environmental impact of these models,^[151][150] regulating for transparency of these models,^[150] regulating their energy and water usage,^[151] encouraging researchers to publish data on their models' carbon footprint,^[153][150] and increasing the number of subject matter experts who understand both machine learning and climate science.^[150]

The New York Times defines slop as analogous to spam: "shoddy or unwanted A.I. content in social media, art, books and ... in search results."^[155] Journalists have expressed concerns about the scale of low-quality generated content with respect to social media content moderation,^[156] the monetary incentives from social media companies to spread such content,^[156][157] false political messaging,^[157] spamming of scientific research paper submissions,^[158] increased time and effort to find higher quality or desired content on the Internet,^[159] the indexing of generated content by search engines,^[160] and on journalism itself.^[161]

A paper published by researchers at Amazon Web Services AI Labs found that over 57% of sentences from a sample of over 6 billion sentences from Common Crawl, a snapshot of web pages, were machine translated. Many of these automated translations were seen as lower quality, especially for sentences were translated across at least three languages. Many lower-resource languages (ex. Wolof, Xhosa) were translated across more languages than higher-resource languages (ex. English, French).^[162][163]

In September 2024, Robyn Speer, the author of wordfreq, an open source database that calculated word frequencies based on text from the Internet, announced that she had stopped updating the data for several reasons: high costs for obtaining data from Reddit and Twitter, excessive focus on generative AI compared to other methods in the natural language processing community, and that "generative AI has polluted the data".^[164]

The adoption of generative AI tools led to an explosion of AI-generated content across multiple domains. A study from University College London estimated that in 2023, more than 60,000 scholarly articles—over 1% of all publications—were likely written with LLM assistance.^[165] According to Stanford University's Institute for Human-Centered AI, approximately 17.5% of newly published computer science papers and 16.9% of peer review text now incorporate content generated by LLMs.^[166]

Visual content follows a similar trend. Since the launch of DALL-E 2 in 2022, it’s estimated that an average of 34 million images have been created daily. As of August 2023, more than 15 billion images had been generated using text-to-image algorithms, with 80% of these created by models based on Stable Diffusion.^[167]

If AI-generated content is included in new data crawls from the Internet for additional training of AI models, defects in the resulting models may occur.^[168] Training an AI model exclusively on the output of another AI model produces a lower-quality model. Repeating this process, where each new model is trained on the previous model's output, leads to progressive degradation and eventually results in a "model collapse" after multiple iterations.^[169] Tests have been conducted with pattern recognition of handwritten letters and with pictures of human faces.^[170] As a consequence, the value of data collected from genuine human interactions with systems may become increasingly valuable in the presence of LLM-generated content in data crawled from the Internet.

On the other side, synthetic data is often used as an alternative to data produced by real-world events. Such data can be deployed to validate mathematical models and to train machine learning models while preserving user privacy,^[171] including for structured data.^[172] The approach is not limited to text generation; image generation has been employed to train computer vision models.^[173]

This section may contain unverified or indiscriminate information in embedded lists. Please help clean up the lists by removing items or incorporating them into the text of the article. (July 2024)

In January 2023, Futurism.com broke the story that CNET had been using an undisclosed internal AI tool to write at least 77 of its stories; after the news broke, CNET posted corrections to 41 of the stories.^[174]

In April 2023, the German tabloid Die Aktuelle published a fake AI-generated interview with former racing driver Michael Schumacher, who had not made any public appearances since 2013 after sustaining a brain injury in a skiing accident. The story included two possible disclosures: the cover included the line "deceptively real", and the interview included an acknowledgment at the end that it was AI-generated. The editor-in-chief was fired shortly thereafter amid the controversy.^[175]

Other outlets that have published articles whose content and/or byline have been confirmed or suspected to be created by generative AI models – often with false content, errors, and/or non-disclosure of generative AI use - include: