Afi
Progress, Perspectives, and Future Implications
Table of Contents
Defining AGI (Artificial General Intelligence)
AGI is often contrasted with narrow AI, designed to perform a specific task, such as playing chess or recognizing faces. AGI, in contrast, is intended to be versatile, flexible, and capable of learning from experience. AGI would be capable of abstract reasoning, problem-solving, emotional intelligence, creativity, and other cognitive abilities typically associated with human intelligence.
The ultimate goal of the research is to create machines that can think, reason, and act autonomously without being explicitly programmed for every task. Such devices could solve many complex problems, from optimizing supply chains to curing diseases. They could also be used to perform dangerous or tedious tasks that humans might find difficult or impossible.
While AGI remains a distant goal, researchers in the field are progressing on several fronts. Advances in machine learning, natural language processing, and robotics are bringing researchers closer to creating machine intelligence that can rival human intelligence. However, many technical, ethical, and philosophical challenges must be addressed before AGI becomes a reality.
David Shapiro’s Perspective on AGI
Alan D. Thompson’s Timeline to AGI
Jun/2023 50%: Harvard introduces ‘inference-time intervention’ (ITI) (‘At a high level, we first identify a sparse set of attention heads with high linear probing accuracy for truthfulness. Then, during inference, we shift activations along these truth-correlated directions. We repeat the same intervention autoregressively until the whole answer is generated.’) Harvard (arxiv)
Jun/2023 49%: Google DeepMind trains an LLM (DIDACT) on iterative code in their 86TB code repository (‘the trained model can be used in a variety of surprising ways… by chaining together multiple predictions to roll out longer activity trajectories… we started with a blank file and asked the model to successively predict what edits would come next until it had written a full code file. The astonishing part is that the model developed code in a step-by-step way that would seem natural to a developer’) Google Blog, Twitter
May/2023 49%: Ability Robotics combines an LLM with their humanlike android (robot), Digit. Agility Robotics (YouTube)
May/2023 49%: PaLM 2 breaks the 90% mark for WinoGrande. For the first time, a large language model has breached the 90% mark on WinoGrande, a ‘more challenging, adversarial’ version of Winograd, designed to be very difficult for AI. Fine-tuned PaLM 2 scored 90.9%; humans are at 94%. PaLM 2 paper (PDF, Google)
May/2023 49%: Robot + text-davinci-003 (‘…we show that LLMs can be directly used off-the-shelf to achieve generalization in robotics, leveraging the powerful summarization capabilities they have learned from vast amounts of text data.’). Princeton/Google/others
Apr/2023 48%: Boston Dynamics + ChatGPT (‘We integrated ChatGPT with our [Boston Dynamics Spot] robots.’). Levatas
Mar/2023 48%: Microsoft introduces TaskMatrix.ai (‘We illustrate how TaskMatrix.AI can perform tasks in the physical world by [LLMs] interacting with robots and IoT devices… All these cases have been implemented in practice… understand the environment with camera API, and transform user instructions to action APIs provided by robots… facilitate the handling of physical work with the assistance of robots and the construction of smart homes by connecting IoT devices…’). Microsoft (arxiv)
Mar/2023 48%: OpenAI introduces GPT-4, Microsoft research on record that GPT-4 is ‘early AGI’ (‘Given the breadth and depth of GPT-4’s capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system.’).
Microsoft’s deleted original title of the paper was ‘First Contact With an AGI System.’
Note that LLMs are still not embodied; this countdown requires physical embodiment to reach 60%. Microsoft Research
Mar/2023 42%: Google introduces PaLM-E 562B (PaLM-Embodied. ‘PaLM-E can successfully plan over multiple stages based on visual and language input… successfully plan a long-horizon task…’). Google
Feb/2023 41%: Microsoft used ChatGPT in robots; it self-improved (‘we were impressed by ChatGPT’s ability to make localized code improvements using only language feedback.’). Microsoft
Dec/2022 39%: Anthropic RL-CAI 52B trained by Reinforcement Learning from AI Feedback (RLAIF) (‘we have moved further away from reliance on human supervision, and closer to the possibility of a self-supervised approach to alignment’). LifeArchitect.ai, Anthropic paper (PDF)
Jul/2022 39%: NVIDIA’s Hopper (H100) circuits designed by AI (‘The latest NVIDIA Hopper GPU architecture has nearly 13,000 instances of AI-designed circuits’). LifeArchitect.ai, NVIDIA
May/2022 39%: DeepMind Gato is the first generalist agent to “play Atari, caption images, chat, stack blocks with a real robot arm, and much more.” Watch Alan’s video about Gato.
Jun/2021 31% Google’s TPUv4 circuits designed by AI (‘allowing chip design to be performed by artificial agents with more experience than any human designer. Our method was used to develop the next generation of Google’s artificial intelligence (AI) accelerators and has the potential to save thousands of hours of human effort for each new generation. Finally, we believe that more powerful AI-designed hardware will fuel advances in AI, creating a symbiotic relationship between the two fields). LifeArchitect.ai, Nature, Venturebeat
Nov/2020 30%: Connor Leahy, Co-founder of EleutherAI, re-creator of GPT-2, creator of GPT-J & GPT-NeoX-20B, said about OpenAI GPT-3: “I think GPT-3 is artificial general intelligence, AGI. I think GPT-3 is as intelligent as a human. And I think that it is probably more intelligent than a human in a restricted way… in many ways it is more purely intelligent than humans are. I think humans are approximating what GPT-3 is doing, not vice versa.” Watch the video (timecode)
Aug/2017 20%: Google Transformer leads to significant changes in search, translation, and language models. Read the launch in plain English.
Countdown represents the progress of LLMs and their embodiment into physical systems, with 60% being an estimation of when we could expect to see true artificial general intelligence (AGI) emerge.
Breakthroughs in the field of language models have pushed them closer to AGI.
For example, Google’s Transformer architecture, EleutherAI’s GPT series of language models, DeepMind’s Gato, and NVIDIA’s Hopper circuits are all exciting developments that demonstrate the vast potential of LLMs.
1. Google Transformer: The Transformer architecture, developed by Google researchers in 2017, has been a major breakthrough in the realm of language models. This architecture revolutionized natural language processing by enabling the processing of large volumes of text data in a single pass. The Transformer has been applied in various use cases, such as search engines, translation services, and text-to-speech systems.
2. GPT Models: The GPT series of language models developed by EleutherAI, an open-source artificial intelligence lab, have been widely praised for their performance. GPT models utilize unsupervised learning methods to extract information from massive amounts of unstructured text data, such as books and articles. GPT models are notable for their ability to generate coherent text, answer common sense questions, and even perform basic arithmetic.
3. Gato: Developed by DeepMind, Gato is the first generalist agent that can perform multiple tasks such as playing Atari games, captioning images, chat, and stacking blocks with a real robot arm. Gato represents a major step towards creating more versatile AI systems.
4. Hopper Circuits: NVIDIA recently utilized deep learning to design the next iteration of its hardware, the Hopper GPU. Using artificial intelligence to design chips, such as pairs of logic gates, can save engineers thousands of hours of labor time and accelerate the innovation of hardware technology.
Ongoing research, collaboration, and responsible development are crucial for the advancement of artificial intelligence (AI). As a continuously evolving field, AI requires a constant effort to drive progress through research. Moreover, collaborating across sectors and with diverse perspectives can accelerate innovation and knowledge-sharing, leading to better outcomes for society.
However, such development should also be responsible and ethical. The implications of AI on society are far-reaching, and it is essential to ensure that the technology is developed in a transparent and accountable manner, with a focus on the ethical considerations related to its use.
Together, ongoing research, collaboration, and responsible development can lead to the creation of AI that truly benefits society, while minimizing any potential negative consequences. As such, their importance should not be overlooked, and efforts should be made to foster and encourage all three elements in AI development.
