Artificial General Intelligence

Comments · 33 Views

Artificial general intelligence (AGI) is a type of synthetic intelligence (AI) that matches or goes beyond human cognitive capabilities across a large variety of cognitive tasks.

Artificial general intelligence (AGI) is a kind of expert system (AI) that matches or exceeds human cognitive abilities across a wide variety of cognitive jobs. This contrasts with narrow AI, which is limited to particular tasks. [1] Artificial superintelligence (ASI), on the other hand, refers to AGI that considerably surpasses human cognitive capabilities. AGI is thought about among the meanings of strong AI.


Creating AGI is a primary goal of AI research study and of companies such as OpenAI [2] and Meta. [3] A 2020 survey recognized 72 active AGI research study and development tasks across 37 nations. [4]

The timeline for accomplishing AGI stays a topic of continuous debate among researchers and professionals. As of 2023, some argue that it might be possible in years or decades; others preserve it may take a century or longer; a minority think it might never be attained; and another minority declares that it is already here. [5] [6] Notable AI researcher Geoffrey Hinton has actually expressed issues about the quick development towards AGI, suggesting it could be achieved sooner than lots of expect. [7]

There is debate on the precise meaning of AGI and regarding whether contemporary large language designs (LLMs) such as GPT-4 are early forms of AGI. [8] AGI is a typical topic in science fiction and futures research studies. [9] [10]

Contention exists over whether AGI represents an existential threat. [11] [12] [13] Many professionals on AI have stated that mitigating the threat of human termination postured by AGI must be a global concern. [14] [15] Others discover the advancement of AGI to be too remote to provide such a threat. [16] [17]

Terminology


AGI is also called strong AI, [18] [19] complete AI, [20] human-level AI, [5] human-level intelligent AI, or general intelligent action. [21]

Some scholastic sources book the term "strong AI" for computer programs that experience sentience or consciousness. [a] On the other hand, weak AI (or narrow AI) is able to solve one specific problem but lacks basic cognitive abilities. [22] [19] Some academic sources utilize "weak AI" to refer more broadly to any programs that neither experience awareness nor have a mind in the very same sense as humans. [a]

Related principles consist of synthetic superintelligence and transformative AI. An artificial superintelligence (ASI) is a theoretical kind of AGI that is much more usually intelligent than people, [23] while the notion of transformative AI relates to AI having a big effect on society, for example, similar to the farming or industrial transformation. [24]

A framework for classifying AGI in levels was proposed in 2023 by Google DeepMind researchers. They define five levels of AGI: emerging, skilled, specialist, virtuoso, and superhuman. For instance, a skilled AGI is specified as an AI that surpasses 50% of knowledgeable adults in a wide variety of non-physical tasks, and a superhuman AGI (i.e. an artificial superintelligence) is likewise defined but with a threshold of 100%. They consider large language models like ChatGPT or LLaMA 2 to be circumstances of emerging AGI. [25]

Characteristics


Various popular definitions of intelligence have actually been proposed. Among the leading propositions is the Turing test. However, there are other popular meanings, and some scientists disagree with the more popular techniques. [b]

Intelligence qualities


Researchers usually hold that intelligence is required to do all of the following: [27]

reason, use technique, akropolistravel.com resolve puzzles, and make judgments under unpredictability
represent knowledge, consisting of common sense knowledge
strategy
learn
- interact in natural language
- if necessary, integrate these abilities in conclusion of any provided objective


Many interdisciplinary approaches (e.g. cognitive science, computational intelligence, and choice making) think about additional traits such as imagination (the capability to form unique psychological images and concepts) [28] and autonomy. [29]

Computer-based systems that show a number of these abilities exist (e.g. see computational creativity, automated thinking, decision assistance system, robot, evolutionary computation, smart agent). There is debate about whether modern-day AI systems have them to a sufficient degree.


Physical characteristics


Other capabilities are considered preferable in smart systems, as they may impact intelligence or help in its expression. These include: [30]

- the capability to sense (e.g. see, hear, and so on), and
- the capability to act (e.g. relocation and control items, change place to check out, etc).


This consists of the ability to detect and respond to danger. [31]

Although the ability to sense (e.g. see, hear, etc) and the capability to act (e.g. relocation and manipulate things, modification location to check out, etc) can be preferable for some intelligent systems, [30] these physical capabilities are not strictly needed for an entity to certify as AGI-particularly under the thesis that big language models (LLMs) may already be or end up being AGI. Even from a less positive point of view on LLMs, there is no company requirement for an AGI to have a human-like form; being a silicon-based computational system suffices, provided it can process input (language) from the external world in location of human senses. This analysis lines up with the understanding that AGI has never ever been proscribed a particular physical embodiment and thus does not require a capability for mobility or traditional "eyes and ears". [32]

Tests for human-level AGI


Several tests meant to validate human-level AGI have actually been considered, consisting of: [33] [34]

The concept of the test is that the device has to try and pretend to be a guy, by answering concerns put to it, wiki.rrtn.org and it will just pass if the pretence is reasonably persuading. A considerable portion of a jury, who must not be expert about makers, must be taken in by the pretence. [37]

AI-complete issues


A problem is informally called "AI-complete" or "AI-hard" if it is thought that in order to resolve it, one would require to execute AGI, because the solution is beyond the abilities of a purpose-specific algorithm. [47]

There are many issues that have been conjectured to need general intelligence to resolve along with human beings. Examples include computer system vision, natural language understanding, and handling unforeseen circumstances while fixing any real-world problem. [48] Even a particular job like translation needs a device to read and compose in both languages, follow the author's argument (reason), understand the context (understanding), and faithfully recreate the author's original intent (social intelligence). All of these problems require to be resolved all at once in order to reach human-level maker efficiency.


However, a number of these tasks can now be performed by modern-day large language models. According to Stanford University's 2024 AI index, AI has actually reached human-level performance on lots of benchmarks for checking out understanding and visual reasoning. [49]

History


Classical AI


Modern AI research study began in the mid-1950s. [50] The first generation of AI researchers were encouraged that artificial basic intelligence was possible which it would exist in just a few years. [51] AI leader Herbert A. Simon wrote in 1965: "devices will be capable, within twenty years, of doing any work a man can do." [52]

Their predictions were the motivation for Stanley Kubrick and Arthur C. Clarke's character HAL 9000, who embodied what AI scientists believed they might produce by the year 2001. AI pioneer Marvin Minsky was a specialist [53] on the job of making HAL 9000 as realistic as possible according to the consensus predictions of the time. He said in 1967, "Within a generation ... the issue of creating 'synthetic intelligence' will considerably be fixed". [54]

Several classical AI tasks, such as Doug Lenat's Cyc task (that began in 1984), and Allen Newell's Soar task, were directed at AGI.


However, in the early 1970s, it became apparent that researchers had grossly ignored the problem of the job. Funding companies became doubtful of AGI and put scientists under increasing pressure to produce useful "used AI". [c] In the early 1980s, Japan's Fifth Generation Computer Project revived interest in AGI, setting out a ten-year timeline that included AGI objectives like "continue a casual discussion". [58] In action to this and the success of specialist systems, both industry and federal government pumped money into the field. [56] [59] However, confidence in AI marvelously collapsed in the late 1980s, and the objectives of the Fifth Generation Computer Project were never ever satisfied. [60] For the second time in 20 years, AI scientists who anticipated the impending achievement of AGI had been misinterpreted. By the 1990s, AI scientists had a reputation for making vain guarantees. They ended up being unwilling to make forecasts at all [d] and avoided reference of "human level" expert system for fear of being labeled "wild-eyed dreamer [s]. [62]

Narrow AI research


In the 1990s and early 21st century, mainstream AI accomplished business success and scholastic respectability by focusing on specific sub-problems where AI can produce proven outcomes and commercial applications, such as speech acknowledgment and suggestion algorithms. [63] These "applied AI" systems are now utilized thoroughly throughout the innovation market, and research in this vein is greatly moneyed in both academia and industry. As of 2018 [upgrade], advancement in this field was thought about an emerging pattern, and a fully grown phase was anticipated to be reached in more than ten years. [64]

At the turn of the century, lots of traditional AI researchers [65] hoped that strong AI could be established by combining programs that solve numerous sub-problems. Hans Moravec composed in 1988:


I am positive that this bottom-up route to expert system will one day meet the traditional top-down route majority way, ready to supply the real-world proficiency and the commonsense knowledge that has been so frustratingly evasive in reasoning programs. Fully smart devices will result when the metaphorical golden spike is driven joining the 2 efforts. [65]

However, even at the time, this was disputed. For instance, Stevan Harnad of Princeton University concluded his 1990 paper on the symbol grounding hypothesis by stating:


The expectation has frequently been voiced that "top-down" (symbolic) approaches to modeling cognition will somehow meet "bottom-up" (sensory) approaches somewhere in between. If the grounding considerations in this paper are valid, then this expectation is hopelessly modular and there is truly only one feasible route from sense to signs: from the ground up. A free-floating symbolic level like the software level of a computer will never be reached by this route (or vice versa) - nor is it clear why we need to even attempt to reach such a level, considering that it looks as if getting there would just amount to uprooting our signs from their intrinsic significances (therefore merely lowering ourselves to the practical equivalent of a programmable computer system). [66]

Modern synthetic general intelligence research


The term "synthetic basic intelligence" was used as early as 1997, by Mark Gubrud [67] in a discussion of the ramifications of completely automated military production and operations. A mathematical formalism of AGI was proposed by Marcus Hutter in 2000. Named AIXI, the proposed AGI representative increases "the capability to please objectives in a broad variety of environments". [68] This kind of AGI, characterized by the capability to maximise a mathematical definition of intelligence rather than show human-like behaviour, [69] was also called universal expert system. [70]

The term AGI was re-introduced and promoted by Shane Legg and Ben Goertzel around 2002. [71] AGI research activity in 2006 was described by Pei Wang and Ben Goertzel [72] as "producing publications and preliminary results". The first summer season school in AGI was organized in Xiamen, China in 2009 [73] by the Xiamen university's Artificial Brain Laboratory and OpenCog. The first university course was given up 2010 [74] and 2011 [75] at Plovdiv University, Bulgaria by Todor Arnaudov. MIT provided a course on AGI in 2018, organized by Lex Fridman and featuring a number of visitor speakers.


As of 2023 [upgrade], a little number of computer scientists are active in AGI research, and lots of contribute to a series of AGI conferences. However, significantly more scientists have an interest in open-ended learning, [76] [77] which is the concept of allowing AI to constantly find out and innovate like people do.


Feasibility


Since 2023, the advancement and possible achievement of AGI remains a topic of extreme debate within the AI neighborhood. While traditional agreement held that AGI was a remote goal, current advancements have led some scientists and industry figures to declare that early forms of AGI might already exist. [78] AI leader Herbert A. Simon hypothesized in 1965 that "makers will be capable, within twenty years, of doing any work a male can do". This forecast failed to come real. Microsoft co-founder Paul Allen thought that such intelligence is unlikely in the 21st century because it would need "unforeseeable and basically unforeseeable developments" and a "clinically deep understanding of cognition". [79] Writing in The Guardian, roboticist Alan Winfield declared the gulf between modern-day computing and human-level expert system is as broad as the gulf in between present space flight and practical faster-than-light spaceflight. [80]

An additional obstacle is the lack of clarity in specifying what intelligence requires. Does it require consciousness? Must it display the capability to set objectives as well as pursue them? Is it purely a matter of scale such that if design sizes increase adequately, intelligence will emerge? Are facilities such as preparation, reasoning, and causal understanding required? Does intelligence need clearly reproducing the brain and its particular faculties? Does it require emotions? [81]

Most AI researchers believe strong AI can be accomplished in the future, but some thinkers, like Hubert Dreyfus and Roger Penrose, reject the possibility of achieving strong AI. [82] [83] John McCarthy is amongst those who think human-level AI will be accomplished, however that the present level of progress is such that a date can not properly be forecasted. [84] AI experts' views on the expediency of AGI wax and subside. Four surveys carried out in 2012 and 2013 suggested that the average estimate among professionals for when they would be 50% positive AGI would show up was 2040 to 2050, depending on the poll, with the mean being 2081. Of the experts, 16.5% answered with "never" when asked the exact same question however with a 90% confidence rather. [85] [86] Further current AGI development considerations can be discovered above Tests for verifying human-level AGI.


A report by Stuart Armstrong and Kaj Sotala of the Machine Intelligence Research Institute found that "over [a] 60-year time frame there is a strong bias towards anticipating the arrival of human-level AI as between 15 and 25 years from the time the forecast was made". They analyzed 95 forecasts made between 1950 and 2012 on when human-level AI will come about. [87]

In 2023, Microsoft researchers published a detailed evaluation of GPT-4. They concluded: "Given the breadth and depth of GPT-4's abilities, we believe that it could fairly be seen as an early (yet still insufficient) version of an artificial general intelligence (AGI) system." [88] Another research study in 2023 reported that GPT-4 exceeds 99% of people on the Torrance tests of imaginative thinking. [89] [90]

Blaise Agüera y Arcas and Peter Norvig wrote in 2023 that a substantial level of basic intelligence has currently been accomplished with frontier models. They composed that unwillingness to this view comes from four primary factors: a "healthy hesitation about metrics for AGI", an "ideological dedication to alternative AI theories or methods", a "dedication to human (or biological) exceptionalism", or a "concern about the financial ramifications of AGI". [91]

2023 also marked the introduction of big multimodal designs (big language designs capable of processing or producing multiple techniques such as text, audio, and images). [92]

In 2024, OpenAI released o1-preview, the very first of a series of designs that "invest more time thinking before they react". According to Mira Murati, this ability to think before reacting represents a brand-new, additional paradigm. It improves design outputs by spending more computing power when creating the answer, whereas the design scaling paradigm enhances outputs by increasing the design size, training data and training compute power. [93] [94]

An OpenAI staff member, Vahid Kazemi, declared in 2024 that the business had attained AGI, mentioning, "In my opinion, we have actually already accomplished AGI and it's much more clear with O1." Kazemi clarified that while the AI is not yet "better than any human at any job", it is "better than a lot of humans at a lot of tasks." He likewise resolved criticisms that big language designs (LLMs) merely follow predefined patterns, comparing their knowing procedure to the scientific approach of observing, assuming, and verifying. These statements have triggered debate, as they rely on a broad and unconventional definition of AGI-traditionally comprehended as AI that matches human intelligence throughout all domains. Critics argue that, while OpenAI's models show amazing versatility, they might not completely meet this standard. Notably, Kazemi's remarks came soon after OpenAI eliminated "AGI" from the regards to its collaboration with Microsoft, triggering speculation about the company's strategic objectives. [95]

Timescales


Progress in synthetic intelligence has actually historically gone through durations of quick progress separated by durations when progress appeared to stop. [82] Ending each hiatus were basic advances in hardware, software application or both to create space for further development. [82] [98] [99] For example, the hardware offered in the twentieth century was not sufficient to implement deep learning, which needs great deals of GPU-enabled CPUs. [100]

In the introduction to his 2006 book, [101] Goertzel says that estimates of the time needed before a truly flexible AGI is constructed differ from ten years to over a century. As of 2007 [upgrade], the consensus in the AGI research neighborhood seemed to be that the timeline discussed by Ray Kurzweil in 2005 in The Singularity is Near [102] (i.e. in between 2015 and 2045) was possible. [103] Mainstream AI researchers have actually offered a large range of viewpoints on whether progress will be this fast. A 2012 meta-analysis of 95 such opinions discovered a predisposition towards predicting that the beginning of AGI would happen within 16-26 years for modern-day and historic predictions alike. That paper has actually been criticized for how it classified opinions as specialist or non-expert. [104]

In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton established a neural network called AlexNet, which won the ImageNet competitors with a top-5 test mistake rate of 15.3%, considerably better than the second-best entry's rate of 26.3% (the traditional approach used a weighted sum of scores from different pre-defined classifiers). [105] AlexNet was regarded as the initial ground-breaker of the existing deep knowing wave. [105]

In 2017, scientists Feng Liu, Yong Shi, and Ying Liu performed intelligence tests on openly offered and freely available weak AI such as Google AI, Apple's Siri, and others. At the optimum, these AIs reached an IQ value of about 47, which corresponds roughly to a six-year-old kid in very first grade. A grownup pertains to about 100 typically. Similar tests were carried out in 2014, with the IQ score reaching an optimum value of 27. [106] [107]

In 2020, OpenAI developed GPT-3, a language model capable of performing lots of varied tasks without particular training. According to Gary Grossman in a VentureBeat short article, while there is consensus that GPT-3 is not an example of AGI, it is thought about by some to be too advanced to be classified as a narrow AI system. [108]

In the exact same year, Jason Rohrer used his GPT-3 account to establish a chatbot, and supplied a chatbot-developing platform called "Project December". OpenAI requested changes to the chatbot to comply with their security guidelines; Rohrer disconnected Project December from the GPT-3 API. [109]

In 2022, DeepMind developed Gato, a "general-purpose" system efficient in performing more than 600 various tasks. [110]

In 2023, Microsoft Research released a research study on an early variation of OpenAI's GPT-4, contending that it showed more basic intelligence than previous AI models and demonstrated human-level performance in tasks covering numerous domains, such as mathematics, coding, and law. This research study stimulated a dispute on whether GPT-4 could be considered an early, insufficient variation of synthetic basic intelligence, highlighting the need for further expedition and assessment of such systems. [111]

In 2023, the AI researcher Geoffrey Hinton stated that: [112]

The concept that this things might in fact get smarter than people - a few individuals believed that, [...] But many people believed it was way off. And I believed it was way off. I thought it was 30 to 50 years and even longer away. Obviously, I no longer believe that.


In May 2023, Demis Hassabis likewise stated that "The progress in the last few years has been pretty unbelievable", which he sees no reason it would decrease, anticipating AGI within a decade or perhaps a few years. [113] In March 2024, Nvidia's CEO, Jensen Huang, stated his expectation that within five years, AI would be capable of passing any test a minimum of as well as people. [114] In June 2024, the AI researcher Leopold Aschenbrenner, a former OpenAI staff member, estimated AGI by 2027 to be "strikingly possible". [115]

Whole brain emulation


While the advancement of transformer designs like in ChatGPT is thought about the most promising course to AGI, [116] [117] whole brain emulation can work as an alternative technique. With whole brain simulation, a brain design is constructed by scanning and mapping a biological brain in information, and then copying and imitating it on a computer system or another computational device. The simulation design must be sufficiently faithful to the initial, so that it behaves in practically the same way as the original brain. [118] Whole brain emulation is a type of brain simulation that is talked about in computational neuroscience and neuroinformatics, and for medical research study functions. It has actually been discussed in synthetic intelligence research study [103] as a technique to strong AI. Neuroimaging technologies that might deliver the essential detailed understanding are improving quickly, and futurist Ray Kurzweil in the book The Singularity Is Near [102] forecasts that a map of adequate quality will appear on a comparable timescale to the computing power needed to emulate it.


Early approximates


For low-level brain simulation, an extremely effective cluster of computer systems or GPUs would be needed, given the huge amount of synapses within the human brain. Each of the 1011 (one hundred billion) neurons has on average 7,000 synaptic connections (synapses) to other nerve cells. The brain of a three-year-old kid has about 1015 synapses (1 quadrillion). This number declines with age, stabilizing by their adult years. Estimates vary for an adult, varying from 1014 to 5 × 1014 synapses (100 to 500 trillion). [120] A price quote of the brain's processing power, based upon a simple switch design for neuron activity, is around 1014 (100 trillion) synaptic updates per second (SUPS). [121]

In 1997, Kurzweil looked at different estimates for the hardware required to equal the human brain and embraced a figure of 1016 computations per 2nd (cps). [e] (For comparison, if a "calculation" was equivalent to one "floating-point operation" - a step utilized to rate present supercomputers - then 1016 "calculations" would be comparable to 10 petaFLOPS, attained in 2011, while 1018 was accomplished in 2022.) He utilized this figure to predict the essential hardware would be offered sometime between 2015 and 2025, if the exponential growth in computer system power at the time of writing continued.


Current research study


The Human Brain Project, an EU-funded effort active from 2013 to 2023, has actually established an especially in-depth and openly accessible atlas of the human brain. [124] In 2023, scientists from Duke University carried out a high-resolution scan of a mouse brain.


Criticisms of simulation-based methods


The artificial nerve cell model assumed by Kurzweil and used in lots of existing synthetic neural network applications is basic compared with biological neurons. A brain simulation would likely have to record the detailed cellular behaviour of biological neurons, presently comprehended just in broad overview. The overhead presented by full modeling of the biological, chemical, and physical information of neural behaviour (specifically on a molecular scale) would require computational powers numerous orders of magnitude bigger than Kurzweil's price quote. In addition, the quotes do not represent glial cells, which are understood to contribute in cognitive processes. [125]

A basic criticism of the simulated brain approach stems from embodied cognition theory which asserts that human embodiment is an important aspect of human intelligence and is needed to ground significance. [126] [127] If this theory is correct, any fully functional brain design will require to encompass more than simply the nerve cells (e.g., a robotic body). Goertzel [103] proposes virtual embodiment (like in metaverses like Second Life) as an alternative, but it is unknown whether this would be enough.


Philosophical viewpoint


"Strong AI" as specified in approach


In 1980, philosopher John Searle created the term "strong AI" as part of his Chinese room argument. [128] He proposed a distinction in between 2 hypotheses about expert system: [f]

Strong AI hypothesis: A synthetic intelligence system can have "a mind" and "consciousness".
Weak AI hypothesis: An expert system system can (just) act like it thinks and has a mind and awareness.


The very first one he called "strong" since it makes a stronger declaration: it presumes something unique has happened to the machine that exceeds those abilities that we can check. The behaviour of a "weak AI" maker would be exactly similar to a "strong AI" machine, however the latter would also have subjective mindful experience. This use is also typical in scholastic AI research study and books. [129]

In contrast to Searle and mainstream AI, some futurists such as Ray Kurzweil utilize the term "strong AI" to indicate "human level synthetic basic intelligence". [102] This is not the like Searle's strong AI, unless it is presumed that awareness is necessary for human-level AGI. Academic philosophers such as Searle do not think that holds true, and to most synthetic intelligence researchers the concern is out-of-scope. [130]

Mainstream AI is most interested in how a program behaves. [131] According to Russell and Norvig, "as long as the program works, they don't care if you call it genuine or a simulation." [130] If the program can act as if it has a mind, then there is no requirement to understand if it actually has mind - undoubtedly, there would be no chance to inform. For AI research, Searle's "weak AI hypothesis" is equivalent to the statement "synthetic basic intelligence is possible". Thus, according to Russell and Norvig, "most AI scientists take the weak AI hypothesis for given, and do not care about the strong AI hypothesis." [130] Thus, for academic AI research study, "Strong AI" and "AGI" are 2 different things.


Consciousness


Consciousness can have various meanings, and some aspects play considerable functions in sci-fi and the principles of artificial intelligence:


Sentience (or "sensational consciousness"): The capability to "feel" perceptions or feelings subjectively, instead of the capability to factor about perceptions. Some philosophers, such as David Chalmers, utilize the term "awareness" to refer solely to extraordinary awareness, which is roughly equivalent to life. [132] Determining why and how subjective experience emerges is understood as the tough issue of consciousness. [133] Thomas Nagel described in 1974 that it "feels like" something to be conscious. If we are not conscious, then it does not feel like anything. Nagel uses the example of a bat: we can smartly ask "what does it feel like to be a bat?" However, we are unlikely to ask "what does it feel like to be a toaster?" Nagel concludes that a bat appears to be conscious (i.e., has consciousness) however a toaster does not. [134] In 2022, a Google engineer claimed that the company's AI chatbot, LaMDA, had actually accomplished life, though this claim was widely challenged by other specialists. [135]

Self-awareness: To have conscious awareness of oneself as a separate person, especially to be knowingly aware of one's own ideas. This is opposed to simply being the "topic of one's thought"-an operating system or debugger has the ability to be "knowledgeable about itself" (that is, to represent itself in the very same method it represents whatever else)-but this is not what people normally imply when they utilize the term "self-awareness". [g]

These qualities have an ethical measurement. AI sentience would trigger issues of well-being and legal defense, likewise to animals. [136] Other aspects of awareness associated to cognitive abilities are likewise relevant to the concept of AI rights. [137] Finding out how to incorporate sophisticated AI with existing legal and social frameworks is an emergent concern. [138]

Benefits


AGI might have a wide range of applications. If oriented towards such goals, AGI might help mitigate numerous problems in the world such as cravings, hardship and health problems. [139]

AGI might improve performance and effectiveness in many jobs. For instance, in public health, AGI could accelerate medical research study, especially against cancer. [140] It might look after the senior, [141] and democratize access to fast, premium medical diagnostics. It might offer fun, inexpensive and personalized education. [141] The need to work to subsist might end up being obsolete if the wealth produced is effectively redistributed. [141] [142] This also raises the concern of the place of human beings in a drastically automated society.


AGI could likewise help to make logical choices, and to prepare for and prevent disasters. It might also help to reap the advantages of possibly disastrous technologies such as nanotechnology or climate engineering, while avoiding the associated dangers. [143] If an AGI's primary goal is to prevent existential disasters such as human termination (which could be difficult if the Vulnerable World Hypothesis ends up being real), [144] it might take steps to dramatically reduce the dangers [143] while minimizing the effect of these steps on our quality of life.


Risks


Existential threats


AGI might represent multiple kinds of existential risk, which are risks that threaten "the premature termination of Earth-originating smart life or the permanent and drastic destruction of its potential for preferable future advancement". [145] The danger of human termination from AGI has actually been the topic of numerous debates, but there is likewise the possibility that the development of AGI would lead to a permanently flawed future. Notably, it could be used to spread out and protect the set of values of whoever establishes it. If mankind still has moral blind areas similar to slavery in the past, AGI might irreversibly entrench it, avoiding moral progress. [146] Furthermore, AGI could help with mass monitoring and indoctrination, which could be used to produce a steady repressive worldwide totalitarian regime. [147] [148] There is likewise a danger for the machines themselves. If machines that are sentient or otherwise worthwhile of moral factor to consider are mass created in the future, taking part in a civilizational path that forever neglects their well-being and interests could be an existential disaster. [149] [150] Considering how much AGI might enhance humankind's future and help in reducing other existential threats, Toby Ord calls these existential threats "an argument for continuing with due caution", not for "abandoning AI". [147]

Risk of loss of control and human termination


The thesis that AI presents an existential threat for people, which this threat needs more attention, is controversial however has actually been endorsed in 2023 by numerous public figures, AI researchers and CEOs of AI companies such as Elon Musk, Bill Gates, Geoffrey Hinton, Yoshua Bengio, Demis Hassabis and Sam Altman. [151] [152]

In 2014, Stephen Hawking criticized widespread indifference:


So, dealing with possible futures of enormous benefits and risks, the professionals are undoubtedly doing whatever possible to ensure the best result, right? Wrong. If a remarkable alien civilisation sent us a message saying, 'We'll show up in a couple of years,' would we simply reply, 'OK, call us when you get here-we'll leave the lights on?' Probably not-but this is more or less what is happening with AI. [153]

The possible fate of humanity has in some cases been compared to the fate of gorillas threatened by human activities. The comparison mentions that higher intelligence allowed mankind to dominate gorillas, which are now susceptible in manner ins which they could not have anticipated. As an outcome, the gorilla has ended up being a threatened species, not out of malice, but just as a security damage from human activities. [154]

The skeptic Yann LeCun considers that AGIs will have no desire to dominate humankind which we should be careful not to anthropomorphize them and interpret their intents as we would for human beings. He said that people will not be "wise sufficient to create super-intelligent machines, yet ridiculously silly to the point of giving it moronic objectives with no safeguards". [155] On the other side, the principle of instrumental merging suggests that almost whatever their objectives, intelligent agents will have reasons to attempt to make it through and acquire more power as intermediary steps to attaining these goals. Which this does not need having emotions. [156]

Many scholars who are worried about existential danger advocate for more research study into resolving the "control issue" to answer the question: what types of safeguards, algorithms, or architectures can programmers carry out to increase the probability that their recursively-improving AI would continue to behave in a friendly, instead of devastating, way after it reaches superintelligence? [157] [158] Solving the control issue is made complex by the AI arms race (which might result in a race to the bottom of security preventative measures in order to launch items before competitors), [159] and using AI in weapon systems. [160]

The thesis that AI can pose existential risk likewise has detractors. Skeptics normally say that AGI is not likely in the short-term, or that concerns about AGI sidetrack from other issues related to existing AI. [161] Former Google fraud czar Shuman Ghosemajumder thinks about that for lots of individuals beyond the innovation industry, existing chatbots and LLMs are already viewed as though they were AGI, causing additional misunderstanding and fear. [162]

Skeptics often charge that the thesis is crypto-religious, with an illogical belief in the possibility of superintelligence changing an unreasonable belief in a supreme God. [163] Some scientists think that the communication campaigns on AI existential threat by particular AI groups (such as OpenAI, Anthropic, DeepMind, and Conjecture) may be an at attempt at regulative capture and to pump up interest in their products. [164] [165]

In 2023, the CEOs of Google DeepMind, OpenAI and Anthropic, along with other market leaders and scientists, provided a joint declaration asserting that "Mitigating the danger of termination from AI must be a global concern together with other societal-scale threats such as pandemics and nuclear war." [152]

Mass unemployment


Researchers from OpenAI estimated that "80% of the U.S. labor force could have at least 10% of their work tasks impacted by the intro of LLMs, while around 19% of workers might see at least 50% of their tasks affected". [166] [167] They think about office employees to be the most exposed, for instance mathematicians, accounting professionals or web designers. [167] AGI could have a better autonomy, ability to make choices, to user interface with other computer tools, however likewise to control robotized bodies.


According to Stephen Hawking, the result of automation on the lifestyle will depend on how the wealth will be redistributed: [142]

Everyone can enjoy a life of elegant leisure if the machine-produced wealth is shared, or most individuals can end up badly bad if the machine-owners effectively lobby versus wealth redistribution. So far, the pattern appears to be towards the 2nd choice, with technology driving ever-increasing inequality


Elon Musk considers that the automation of society will need governments to embrace a universal basic earnings. [168]

See likewise


Artificial brain - Software and hardware with cognitive abilities comparable to those of the animal or human brain
AI result
AI safety - Research area on making AI safe and beneficial
AI alignment - AI conformance to the intended objective
A.I. Rising - 2018 movie directed by Lazar Bodroža
Expert system
Automated maker knowing - Process of automating the application of maker knowing
BRAIN Initiative - Collaborative public-private research study effort revealed by the Obama administration
China Brain Project
Future of Humanity Institute - Defunct Oxford interdisciplinary research centre
General video game playing - Ability of artificial intelligence to play various video games
Generative expert system - AI system capable of creating material in reaction to triggers
Human Brain Project - Scientific research study project
Intelligence amplification - Use of infotech to enhance human intelligence (IA).
Machine principles - Moral behaviours of man-made devices.
Moravec's paradox.
Multi-task knowing - Solving several device learning tasks at the very same time.
Neural scaling law - Statistical law in artificial intelligence.
Outline of expert system - Overview of and topical guide to synthetic intelligence.
Transhumanism - Philosophical motion.
Synthetic intelligence - Alternate term for or kind of expert system.
Transfer knowing - Machine learning technique.
Loebner Prize - Annual AI competition.
Hardware for expert system - Hardware specifically designed and optimized for synthetic intelligence.
Weak artificial intelligence - Form of artificial intelligence.


Notes


^ a b See below for the origin of the term "strong AI", and see the scholastic definition of "strong AI" and weak AI in the post Chinese space.
^ AI founder John McCarthy composes: "we can not yet identify in basic what type of computational procedures we wish to call smart. " [26] (For a conversation of some definitions of intelligence utilized by synthetic intelligence researchers, see viewpoint of expert system.).
^ The Lighthill report particularly slammed AI's "grandiose objectives" and led the taking apart of AI research study in England. [55] In the U.S., DARPA ended up being determined to money only "mission-oriented direct research study, instead of basic undirected research study". [56] [57] ^ As AI creator John McCarthy composes "it would be a great relief to the rest of the employees in AI if the innovators of new basic formalisms would reveal their hopes in a more safeguarded type than has actually often held true." [61] ^ In "Mind Children" [122] 1015 cps is used. More just recently, in 1997, [123] Moravec argued for 108 MIPS which would approximately represent 1014 cps. Moravec talks in regards to MIPS, not "cps", which is a non-standard term Kurzweil introduced.
^ As specified in a basic AI textbook: "The assertion that machines could perhaps act intelligently (or, possibly much better, act as if they were smart) is called the 'weak AI' hypothesis by thinkers, and the assertion that devices that do so are actually thinking (instead of simulating thinking) is called the 'strong AI' hypothesis." [121] ^ Alan Turing made this point in 1950. [36] References


^ Krishna, Sri (9 February 2023). "What is artificial narrow intelligence (ANI)?". VentureBeat. Retrieved 1 March 2024. ANI is created to carry out a single task.
^ "OpenAI Charter". OpenAI. Retrieved 6 April 2023. Our objective is to ensure that artificial basic intelligence advantages all of humanity.
^ Heath, Alex (18 January 2024). "Mark Zuckerberg's new objective is developing artificial basic intelligence". The Verge. Retrieved 13 June 2024. Our vision is to construct AI that is much better than human-level at all of the human senses.
^ Baum, Seth D. (2020 ). A Survey of Artificial General Intelligence Projects for Ethics, Risk, and Policy (PDF) (Report). Global Catastrophic Risk Institute. Retrieved 28 November 2024. 72 AGI R&D projects were recognized as being active in 2020.
^ a b c "AI timelines: What do specialists in expert system expect for the future?". Our World in Data. Retrieved 6 April 2023.
^ Metz, Cade (15 May 2023). "Some Researchers Say A.I. Is Already Here, Stirring Debate in Tech Circles". The New York City Times. Retrieved 18 May 2023.
^ "AI leader Geoffrey Hinton stops Google and warns of danger ahead". The New York Times. 1 May 2023. Retrieved 2 May 2023. It is tough to see how you can prevent the bad actors from utilizing it for bad things.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric (2023 ). "Sparks of Artificial General Intelligence: Early explores GPT-4". arXiv preprint. arXiv:2303.12712. GPT-4 shows sparks of AGI.
^ Butler, Octavia E. (1993 ). Parable of the Sower. Grand Central Publishing. ISBN 978-0-4466-7550-5. All that you touch you change. All that you change changes you.
^ Vinge, Vernor (1992 ). A Fire Upon the Deep. Tor Books. ISBN 978-0-8125-1528-2. The Singularity is coming.
^ Morozov, Evgeny (30 June 2023). "The True Threat of Expert System". The New York Times. The real threat is not AI itself however the way we release it.
^ "Impressed by synthetic intelligence? Experts state AGI is coming next, and it has 'existential' risks". ABC News. 23 March 2023. Retrieved 6 April 2023. AGI might pose existential risks to humankind.
^ Bostrom, Nick (2014 ). Superintelligence: Paths, Dangers, Strategies. Oxford University Press. ISBN 978-0-1996-7811-2. The very first superintelligence will be the last development that humanity requires to make.
^ Roose, Kevin (30 May 2023). "A.I. Poses 'Risk of Extinction,' Industry Leaders Warn". The New York Times. Mitigating the danger of termination from AI should be a global concern.
^ "Statement on AI Risk". Center for AI Safety. Retrieved 1 March 2024. AI experts warn of danger of extinction from AI.
^ Mitchell, Melanie (30 May 2023). "Are AI's Doomsday Scenarios Worth Taking Seriously?". The New York Times. We are far from developing machines that can outthink us in general ways.
^ LeCun, Yann (June 2023). "AGI does not provide an existential risk". Medium. There is no reason to fear AI as an existential threat.
^ Kurzweil 2005, p. 260.
^ a b Kurzweil, Ray (5 August 2005), "Long Live AI", Forbes, archived from the original on 14 August 2005: Kurzweil explains strong AI as "maker intelligence with the full series of human intelligence.".
^ "The Age of Expert System: George John at TEDxLondonBusinessSchool 2013". Archived from the original on 26 February 2014. Retrieved 22 February 2014.
^ Newell & Simon 1976, This is the term they use for "human-level" intelligence in the physical sign system hypothesis.
^ "The Open University on Strong and Weak AI". Archived from the original on 25 September 2009. Retrieved 8 October 2007.
^ "What is artificial superintelligence (ASI)?|Definition from TechTarget". Enterprise AI. Retrieved 8 October 2023.
^ "Artificial intelligence is changing our world - it is on everybody to make sure that it works out". Our World in Data. Retrieved 8 October 2023.
^ Dickson, Ben (16 November 2023). "Here is how far we are to accomplishing AGI, according to DeepMind". VentureBeat.
^ McCarthy, John (2007a). "Basic Questions". Stanford University. Archived from the initial on 26 October 2007. Retrieved 6 December 2007.
^ This list of intelligent characteristics is based upon the subjects covered by significant AI books, including: Russell & Norvig 2003, Luger & Stubblefield 2004, Poole, Mackworth & Goebel 1998 and Nilsson 1998.
^ Johnson 1987.
^ de Charms, R. (1968 ). Personal causation. New York: Academic Press.
^ a b Pfeifer, R. and Bongard J. C., How the body forms the method we think: a brand-new view of intelligence (The MIT Press, 2007). ISBN 0-2621-6239-3.
^ White, R. W. (1959 ). "Motivation reevaluated: The concept of skills". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ White, R. W. (1959 ). "Motivation reevaluated: The principle of skills". Psychological Review. 66 (5 ): 297-333. doi:10.1037/ h0040934. PMID 13844397. S2CID 37385966.
^ Muehlhauser, Luke (11 August 2013). "What is AGI?". Machine Intelligence Research Institute. Archived from the original on 25 April 2014. Retrieved 1 May 2014.
^ "What is Artificial General Intelligence (AGI)?|4 Tests For Ensuring Artificial General Intelligence". Talky Blog. 13 July 2019. Archived from the original on 17 July 2019. Retrieved 17 July 2019.
^ Kirk-Giannini, Cameron Domenico; Goldstein, Simon (16 October 2023). "AI is closer than ever to passing the Turing test for 'intelligence'. What happens when it does?". The Conversation. Retrieved 22 September 2024.
^ a b Turing 1950.
^ Turing, Alan (1952 ). B. Jack Copeland (ed.). Can Automatic Calculating Machines Be Said To Think?. Oxford: Oxford University Press. pp. 487-506. ISBN 978-0-1982-5079-1.
^ "Eugene Goostman is a real kid - the Turing Test says so". The Guardian. 9 June 2014. ISSN 0261-3077. Retrieved 3 March 2024.
^ "Scientists dispute whether computer 'Eugene Goostman' passed Turing test". BBC News. 9 June 2014. Retrieved 3 March 2024.
^ Jones, Cameron R.; Bergen, Benjamin K. (9 May 2024). "People can not differentiate GPT-4 from a human in a Turing test". arXiv:2405.08007 [cs.HC]
^ Varanasi, Lakshmi (21 March 2023). "AI designs like ChatGPT and GPT-4 are acing whatever from the bar test to AP Biology. Here's a list of difficult examinations both AI variations have passed". Business Insider. Retrieved 30 May 2023.
^ Naysmith, Caleb (7 February 2023). "6 Jobs Expert System Is Already Replacing and How Investors Can Profit From It". Retrieved 30 May 2023.
^ Turk, Victoria (28 January 2015). "The Plan to Replace the Turing Test with a 'Turing Olympics'". Vice. Retrieved 3 March 2024.
^ Gopani, Avi (25 May 2022). "Turing Test is undependable. The Winograd Schema is outdated. Coffee is the answer". Analytics India Magazine. Retrieved 3 March 2024.
^ Bhaimiya, Sawdah (20 June 2023). "DeepMind's co-founder suggested testing an AI chatbot's capability to turn $100,000 into $1 million to measure human-like intelligence". Business Insider. Retrieved 3 March 2024.
^ Suleyman, Mustafa (14 July 2023). "Mustafa Suleyman: My new Turing test would see if AI can make $1 million". MIT Technology Review. Retrieved 3 March 2024.
^ Shapiro, Stuart C. (1992 ). "Expert System" (PDF). In Stuart C. Shapiro (ed.). Encyclopedia of Artificial Intelligence (Second ed.). New York City: John Wiley. pp. 54-57. Archived (PDF) from the original on 1 February 2016. (Section 4 is on "AI-Complete Tasks".).
^ Yampolskiy, Roman V. (2012 ). Xin-She Yang (ed.). "Turing Test as a Defining Feature of AI-Completeness" (PDF). Artificial Intelligence, Evolutionary Computation and Metaheuristics (AIECM): 3-17. Archived (PDF) from the original on 22 May 2013.
^ "AI Index: State of AI in 13 Charts". Stanford University Human-Centered Artificial Intelligence. 15 April 2024. Retrieved 27 May 2024.
^ Crevier 1993, pp. 48-50.
^ Kaplan, Andreas (2022 ). "Artificial Intelligence, Business and Civilization - Our Fate Made in Machines". Archived from the original on 6 May 2022. Retrieved 12 March 2022.
^ Simon 1965, p. 96 quoted in Crevier 1993, p. 109.
^ "Scientist on the Set: An Interview with Marvin Minsky". Archived from the initial on 16 July 2012. Retrieved 5 April 2008.
^ Marvin Minsky to Darrach (1970 ), estimated in Crevier (1993, p. 109).
^ Lighthill 1973; Howe 1994.
^ a b NRC 1999, "Shift to Applied Research Increases Investment".
^ Crevier 1993, pp. 115-117; Russell & Norvig 2003, pp. 21-22.
^ Crevier 1993, p. 211, Russell & Norvig 2003, p. 24 and see likewise Feigenbaum & McCorduck 1983.
^ Crevier 1993, pp. 161-162, 197-203, 240; Russell & Norvig 2003, p. 25.
^ Crevier 1993, pp. 209-212.
^ McCarthy, John (2000 ). "Reply to Lighthill". Stanford University. Archived from the original on 30 September 2008. Retrieved 29 September 2007.
^ Markoff, John (14 October 2005). "Behind Expert system, a Squadron of Bright Real People". The New York Times. Archived from the original on 2 February 2023. Retrieved 18 February 2017. At its low point, some computer scientists and software application engineers avoided the term expert system for fear of being considered as wild-eyed dreamers.
^ Russell & Norvig 2003, pp. 25-26
^ "Trends in the Emerging Tech Hype Cycle". Gartner Reports. Archived from the original on 22 May 2019. Retrieved 7 May 2019.
^ a b Moravec 1988, p. 20
^ Harnad, S. (1990 ). "The Symbol Grounding Problem". Physica D. 42 (1-3): 335-346. arXiv: cs/9906002. Bibcode:1990 PhyD ... 42..335 H. doi:10.1016/ 0167-2789( 90 )90087-6. S2CID 3204300.
^ Gubrud 1997
^ Hutter, Marcus (2005 ). Universal Artificial Intelligence: Sequential Decisions Based Upon Algorithmic Probability. Texts in Theoretical Computer Science an EATCS Series. Springer. doi:10.1007/ b138233. ISBN 978-3-5402-6877-2. S2CID 33352850. Archived from the original on 19 July 2022. Retrieved 19 July 2022.
^ Legg, Shane (2008 ). Machine Super Intelligence (PDF) (Thesis). University of Lugano. Archived (PDF) from the original on 15 June 2022. Retrieved 19 July 2022.
^ Goertzel, Ben (2014 ). Artificial General Intelligence. Lecture Notes in Computer Science. Vol. 8598. Journal of Artificial General Intelligence. doi:10.1007/ 978-3-319-09274-4. ISBN 978-3-3190-9273-7. S2CID 8387410.
^ "Who coined the term "AGI"?". goertzel.org. Archived from the initial on 28 December 2018. Retrieved 28 December 2018., via Life 3.0: 'The term "AGI" was popularized by ... Shane Legg, Mark Gubrud and Ben Goertzel'
^ Wang & Goertzel 2007
^ "First International Summer School in Artificial General Intelligence, Main summer school: June 22 - July 3, 2009, OpenCog Lab: July 6-9, 2009". Archived from the original on 28 September 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2009/2010 - пролетен триместър" [Elective courses 2009/2010 - spring trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ "Избираеми дисциплини 2010/2011 - зимен триместър" [Elective courses 2010/2011 - winter trimester] Факултет по математика и информатика [Faculty of Mathematics and Informatics] (in Bulgarian). Archived from the initial on 26 July 2020. Retrieved 11 May 2020.
^ Shevlin, Henry; Vold, Karina; Crosby, Matthew; Halina, Marta (4 October 2019). "The limits of machine intelligence: Despite progress in maker intelligence, synthetic general intelligence is still a major difficulty". EMBO Reports. 20 (10 ): e49177. doi:10.15252/ embr.201949177. ISSN 1469-221X. PMC 6776890. PMID 31531926.
^ Bubeck, Sébastien; Chandrasekaran, Varun; Eldan, Ronen; Gehrke, Johannes; Horvitz, Eric; Kamar, Ece; Lee, Peter; Lee, Yin Tat; Li, Yuanzhi; Lundberg, Scott; Nori, Harsha; Palangi, Hamid; Ribeiro, Marco Tulio; Zhang, Yi (27 March 2023). "Sparks of Artificial General Intelligence: Early try outs GPT-4". arXiv:2303.12712 [cs.CL]
^ "Microsoft Researchers Claim GPT-4 Is Showing "Sparks" of AGI". Futurism. 23 March 2023. Retrieved 13 December 2023.
^ Allen, Paul; Greaves, Mark (12 October 2011). "The Singularity Isn't Near". MIT Technology Review. Retrieved 17 September 2014.
^ Winfield, Alan. "Expert system will not turn into a Frankenstein's beast". The Guardian. Archived from the original on 17 September 2014. Retrieved 17 September 2014.
^ Deane, George (2022 ). "Machines That Feel and Think: The Role of Affective Feelings and Mental Action in (Artificial) General Intelligence". Artificial Life. 28 (3 ): 289-309. doi:10.1162/ artl_a_00368. ISSN 1064-5462. PMID 35881678. S2CID 251069071.
^ a b c Clocksin 2003.
^ Fjelland, Ragnar (17 June 2020). "Why general expert system will not be understood". Humanities and Social Sciences Communications. 7 (1 ): 1-9. doi:10.1057/ s41599-020-0494-4. hdl:11250/ 2726984. ISSN 2662-9992. S2CID 219710554.
^ McCarthy 2007b.
^ Khatchadourian, Raffi (23 November 2015). "The Doomsday Invention: Will expert system bring us paradise or destruction?". The New Yorker. Archived from the initial on 28 January 2016. Retrieved 7 February 2016.
^ Müller, V. C., & Bostrom, N. (2016 ). Future progress in synthetic intelligence: A survey of professional viewpoint. In Fundamental problems of expert system (pp. 555-572). Springer, Cham.
^ Armstrong, Stuart, and Kaj Sotala. 2012. "How We're Predicting AI-or Failing To." In Beyond AI: Artificial Dreams, edited by Jan Romportl, Pavel Ircing, Eva Žáčková, Michal Polák and Radek Schuster, 52-75. Plzeň: University of West Bohemia
^ "Microsoft Now Claims GPT-4 Shows 'Sparks' of General Intelligence". 24 March 2023.
^ Shimek, Cary (6 July 2023). "AI Outperforms Humans in Creativity Test". Neuroscience News. Retrieved 20 October 2023.
^ Guzik, Erik E.; Byrge, Christian; Gilde, Christian (1 December 2023). "The originality of machines: AI takes the Torrance Test". Journal of Creativity. 33 (3 ): 100065. doi:10.1016/ j.yjoc.2023.100065. ISSN 2713-3745. S2CID 261087185.
^ Arcas, Blaise Agüera y (10 October 2023). "Artificial General Intelligence Is Already Here". Noema.
^ Zia, Tehseen (8 January 2024). "Unveiling of Large Multimodal Models: Shaping the Landscape of Language Models in 2024". Unite.ai. Retrieved 26 May 2024.
^ "Introducing OpenAI o1-preview". OpenAI. 12 September 2024.
^ Knight, Will. "OpenAI Announces a New AI Model, Code-Named Strawberry, That Solves Difficult Problems Step by Step". Wired. ISSN 1059-1028. Retrieved 17 September 2024.
^ "OpenAI Employee Claims AGI Has Been Achieved". Orbital Today. 13 December 2024. Retrieved 27 December 2024.
^ "AI Index: State of AI in 13 Charts". hai.stanford.edu. 15 April 2024. Retrieved 7 June 2024.
^ "Next-Gen AI: OpenAI and Meta's Leap Towards Reasoning Machines". Unite.ai. 19 April 2024. Retrieved 7 June 2024.
^ James, Alex P. (2022 ). "The Why, What, and How of Artificial General Intelligence Chip Development". IEEE Transactions on Cognitive and Developmental Systems. 14 (2 ): 333-347. arXiv:2012.06338. doi:10.1109/ TCDS.2021.3069871. ISSN 2379-8920. S2CID 228376556. Archived from the original on 28 August 2022. Retrieved 28 August 2022.
^ Pei,

Comments