Elena' s AI Blog

The Remarkable Evolution and Milestones of AI

23 May 2023 (updated: 07 Jul 2026) / 31 minutes to read

Elena Daehnhardt


Midjourney, May 2023


TL;DR:
  • A historical overview of AI evolution from the 1950s to present, covering key milestones including neural networks, deep learning breakthroughs, GPT models, and modern AI applications.

Artificial intelligence (AI) is a branch of computer science that builds systems capable of performing tasks that normally require human intelligence, such as reasoning, perception, language understanding, and learning from data. This post traces the evolution of AI from its origins in the 1950s to the present, covering its key milestones: rule-based systems, neural networks, the deep learning revolution, and modern applications.

The Origins of AI: Rule-Based Programming and Symbolic Reasoning (1950s)

Once upon a time, in the magical era of the 1950s, a group of intrepid researchers embarked on a mind-boggling quest to unravel the secrets of artificial intelligence (AI). Their hearts brimmed with curiosity as they delved into creating magnificent machines capable of mirroring the profound depths of human intelligence. With a blend of excitement and trepidation, they set forth on a path that would forever change the course of human history.

Midjourney prompt: A computer and scientists in the year of 1955

I use the following image style for this post prompts: realistic, pastel, pink and metallic tones, stunning, — stylize 1000

Midjourney Prompts
Interested in Midjouney image generation? - refer to my post Mastering Midjourney Prompts for Stunning Images. You can also check the Guide Midjourney --stylize Explained. Midjourney Prompts

Elena: Were computers in 1955?

chatGPT: Yes, computers were in existence in 1955. Computers were large, room-sized machines used for scientific calculations, data processing, and other computational tasks during that time. They were not as advanced or widely accessible as modern computers but played a crucial role in early AI research and development.

Living Large -- Argonne's First Computer (8056998342)

Living Large -- Argonne's First Computer, Wikimedia Commons image

Like fearless explorers of an uncharted wilderness, these pioneering souls ventured into the unknown, armed with the rudimentary tools of their time. Rule-based programming and symbolic reasoning became their trusty companions, helping them traverse the intricate labyrinth of replicating human thought processes. As their creations evolved, glimpses of the extraordinary unfolded before their eyes, sparking a whirlwind of possibilities. It was a time of unprecedented dreams and audacious ambitions, where each step forward brought the tantalizing promise of uncovering the mysteries hidden within the fabric of artificial minds.

Midjourney prompt: Rule-based programming and symbolic reasoning

In the annals of AI’s grand adventure, these early chapters stand as a testament to the unyielding spirit of human curiosity. The journey of artificial intelligence had just begun, and little did the world know of the breathtaking wonders and unforeseen challenges that awaited on the horizon.

For those who prefer their history with dates attached, the decade delivered a remarkable run of firsts. In 1950, Alan Turing published “Computing Machinery and Intelligence”, proposing his famous imitation game — now known as the Turing test — as a practical substitute for the slippery question “can machines think?”. In the summer of 1956, John McCarthy, Marvin Minsky, Claude Shannon and Nathaniel Rochester convened the Dartmouth Summer Research Project, the workshop at which McCarthy coined the very term “artificial intelligence”. That same year, Allen Newell, Herbert Simon and Cliff Shaw demonstrated the Logic Theorist, widely regarded as the first AI program, which proved 38 of the first 52 theorems in Whitehead and Russell’s Principia Mathematica — and even found a more elegant proof for one of them.

The dominant paradigm that emerged — later nicknamed “Good Old-Fashioned AI” (GOFAI) — represented knowledge as explicit symbols and manipulated them with hand-written inference rules, typically IF-THEN productions. To support this style of programming, McCarthy created Lisp in 1958, which remained the lingua franca of AI research for decades. In the same year, Frank Rosenblatt took the opposing, data-driven route: his perceptron computed a weighted sum of inputs, applied a threshold, and adjusted its weights from labelled examples — the first trainable neural network, implemented in hardware as the Mark I Perceptron. The two rival traditions, symbolic reasoning and learning from data, were thus both born within the same decade. Symbolic AI’s ambitions culminated in projects such as Shakey the Robot (SRI International, 1966–1972), pictured below, which combined computer vision, the STRIPS automated planner, and the A* search algorithm — an algorithm still taught in every computer science curriculum today.

Shakey the Robot (developed between 1966-1972 at SRI International) - Computer History Museum (2007-11-10 23.16.01 by Carlo Nardone)

Shakey the Robot (developed between 1966-1972 at SRI International) - Computer History Museum, Wikimedia Commons image

The 1980s AI Renaissance: Neural Networks and Machine Learning

The path to progress was often fraught with obstacles and uncertainties in the vast expanse of AI’s unfolding saga. The initial strides made in the 1950s were followed by a period of stagnation as if the great gears of innovation had ground to a halt. Yet, just when hope seemed to flicker in the dim light of uncertainty, a renaissance dawned upon the realm of artificial intelligence in the remarkable decade of the 1980s.

During this era of technological awakening, the winds of change swept across the land, bringing a renewed vitality to AI research. Advancements in computational power emerged as the heralds of a promising future, breathing life into dormant aspirations. The stage was set for a grand transformation as machine learning algorithms stepped into the limelight, illuminating the path ahead with their brilliance. Among these remarkable techniques, neural networks stood tall, capturing the imagination of both scientists and dreamers alike. A neural network is a machine learning model composed of layers of interconnected nodes that learn patterns directly from data, rather than from hand-written rules. With the power to learn from vast oceans of data and steadily enhance their performance, these newfound marvels paved the way for a paradigm shift reverberating through the annals of time, forever altering the face of artificial intelligence.

The seeds of progress blossomed in the fertile soil of this technological renaissance, propelling AI towards unforeseen frontiers. The stage had been set, and the foundations had been laid for an extraordinary tapestry of AI tools that would shape the present and define the future. The echoes of this remarkable era continue to reverberate, reminding us of the indomitable spirit of human ingenuity and the boundless potential that lies within the embrace of artificial intelligence.

The “stagnation” mentioned above has a concrete history, and it is worth naming. In 1969, Minsky and Papert’s book Perceptrons proved that a single-layer perceptron cannot learn functions that are not linearly separable — the XOR function being the canonical example — and neural network funding dried up almost overnight. The 1973 Lighthill Report then persuaded the British government to slash AI research funding, ushering in what we now call the first AI winter. Progress had not stopped so much as it had been defunded.

What actually revived the field commercially in the early 1980s was the expert system: a symbolic AI program encoding a specialist’s knowledge as hundreds or thousands of IF-THEN rules. MYCIN, developed at Stanford in the 1970s, used roughly 600 rules and certainty factors to recommend antibiotic treatments for blood infections, performing comparably to human specialists in evaluations. XCON (also known as R1), deployed at Digital Equipment Corporation from 1980 to configure VAX computer orders, was reportedly saving the company around $25 million a year by 1986 — the first proof that AI could pay its own way.

On the neural network side, the theoretical breakthroughs of the decade were just as consequential. John Hopfield’s 1982 recurrent network showed how a neural system could store and retrieve memories as stable states, re-legitimising the connectionist approach. Then, in 1986, Rumelhart, Hinton and Williams published their Nature paper popularising backpropagation: by applying the chain rule of calculus to propagate error gradients backwards through hidden layers, multi-layer networks could finally learn the non-linear functions that had defeated the single-layer perceptron. This solved the very limitation Minsky and Papert had identified seventeen years earlier — although, in a twist of historical irony, the collapse of the specialised Lisp machine market in the late 1980s promptly triggered a second AI winter before the idea could fully bloom.

Key AI Milestones: A Timeline from the 1950s to Today

Here are key milestones in AI development, along with the year in which each milestone occurred:

1943
First conceptualization of artificial neural networks
McCulloch & Pitts Publish the First Mathematical Model of a Neural Network
1950
Alan Turing proposes the "Turing Test"
The Turing Test
1956
Dartmouth Workshop marks the birth of AI as a field, with John McCarthy coining the term "artificial intelligence."
Artificial Intelligence Coined at Dartmouth
1958
Perceptron, the first working neural network, is developed by Frank Rosenblatt
Professor’s perceptron paved the way for AI – 60 years too soon
1969
Shakey, the first mobile robot capable of reasoning, navigation, and manipulation, is developed
Shakey the Robot
1972
The first expert system, MYCIN, is developed for diagnosing infectious diseases.
MYCIN
1997
IBM's Deep Blue defeats chess champion Garry Kasparov, signalling a significant milestone in machine learning and AI
Deep Blue
2006
Geoffrey Hinton and colleagues wrote their paper on backpropagation and further developed deep learning concepts, reviving interest in neural networks.
Heroes of Deep Learning: Geoffrey Hinton
2011
IBM's Watson wins Jeopardy! against human champions, demonstrating advancements in natural language processing.
A Computer Called Watson
2012
AlexNet, a deep convolutional neural network, achieves a breakthrough in image classification accuracy.
ImageNet Classification with Deep Convolutional Neural Networks
2016
DeepMind's AlphaGo defeats Go world champion Lee Sedol, showcasing the power of AI in complex strategy games.
Human Go champion loses to Google DeepMind AlphaGo computer in 1st game
2017
AlphaZero, developed by DeepMind, achieves superhuman performance in chess, shogi, and Go without prior human knowledge.
AlphaZero: Shedding new light on chess, shogi, and Go
2018
OpenAI's GPT-1 (Generative Pre-trained Transformer) 117 million parameters demonstrates large-scale language modeling and text generation capabilities.
GPT-1 to GPT-4: Each of OpenAI's GPT Models Explained and Compared
2019
GPT-2 (1.5 billion parameters), a more powerful language model than GPT-1, demonstrates the ability to generate coherent and contextually relevant text.
GPT-1 to GPT-4: Each of OpenAI's GPT Models Explained and Compared
2020
GPT-3, an even more advanced language model (175 billion parameters), sets new benchmarks in natural language processing and generates highly realistic text.
GPT-1 to GPT-4: Each of OpenAI's GPT Models Explained and Compared
2022
AI-powered virtual assistants have become increasingly integrated into everyday life, assisting with tasks and providing personalized recommendations.
The Rise Of Virtual Personal Assistants: How They’re Changing The Way We Work And Live
2022
Advancements in reinforcement learning lead to significant improvements in robotics, enabling more complex and adaptable robotic systems.
Progress and challenges in adaptive robotics
2023
GPT-4, a multimodal model accepting images as well as text as its input (parameter count undisclosed by OpenAI)
GPT-4 Technical Report

Midjourney prompt: Deep Neural Networks and GPU

The Deep Learning Revolution: GPUs and Big Data (21st Century)

Deep learning is a subfield of machine learning that uses multi-layer neural networks to learn hierarchical representations directly from raw data, and it is the primary driver of the 21st-century AI breakthroughs in image recognition and natural language processing. Amidst the dawn of the new millennium, a breathtaking revolution unfolded in the vast realm of artificial intelligence, casting a mesmerizing spell upon the world. The 21st century proved fertile ground for AI technologies as they sprouted and flourished with astonishing vigour. It was a time of explosive growth, where the very foundations of the field were shaken, and new horizons beckoned with irresistible allure.

CSIRO ScienceImage 11313 The CSIRO GPU cluster at the data centre

The CSIRO GPU cluster at the data centre, ikimedia Commons image

The transformation was ignited by a symphony of breakthroughs in deep learning, fueled by the harmonious convergence of colossal datasets and powerful GPUs. As if awakening from a deep slumber, deep neural networks emerged as the unsung heroes, wielding their extraordinary capabilities to achieve feats that once seemed confined to the realm of dreams. Their prowess in image recognition, natural language processing, and many complex tasks ignited a wildfire of possibilities, propelling AI tools into the vibrant heart of the mainstream consciousness.

The pivotal moment has a precise date: September 2012, when AlexNet — an eight-layer convolutional network with about 60 million parameters, designed by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton — won the ImageNet Large Scale Visual Recognition Challenge with a top-5 error rate of 15.3%, against 26.2% for the nearest competitor. That ten-point margin, unheard of in a benchmark competition, sent the entire computer vision community scrambling to retrain on neural networks. AlexNet’s recipe combined ReLU activations, dropout regularisation, and — crucially — training on a pair of consumer NVIDIA GTX 580 GPUs. GPUs matter because deep learning is, computationally, mostly matrix multiplication, and a graphics card’s thousands of cores perform those multiplications in parallel far more cheaply than a CPU ever could. The other ingredient was data: ImageNet supplied 1.2 million labelled training images across 1,000 categories, a scale of supervision that simply had not existed before.

The years that followed read like a highlight reel. Word2vec (2013) showed that word meanings could be captured as arithmetic on vectors; sequence-to-sequence models (2014) reinvented machine translation; ResNet (2015) stacked 152 layers using skip connections and pushed ImageNet top-5 error below the estimated human level of roughly 5%. In March 2016, DeepMind’s AlphaGo — combining deep policy and value networks with Monte Carlo tree search — defeated Lee Sedol 4–1 at Go, a game whose branching factor had long made it the standard example of what computers could not do. Then, in 2017, Google researchers published “Attention Is All You Need”, introducing the transformer architecture: by replacing recurrence with self-attention, it allowed every token in a sequence to be processed in parallel, making it dramatically more efficient to train at scale. Essentially every large language model since — including the one I have been chatting with throughout this post — is a transformer.

Since my keywords promised you GPT parameter counts, here they are — the scaling story in one small table:

Model Release Parameters Notable detail
GPT-1 June 2018 117 million Introduced generative pre-training on unlabelled text
GPT-2 February 2019 1.5 billion Initially withheld over misuse concerns
GPT-3 June 2020 175 billion 96 layers; demonstrated few-shot learning
GPT-4 March 2023 undisclosed OpenAI stopped publishing parameter counts

ChatGPT itself, launched in November 2022, is not a bigger model but a better-behaved one: a GPT-3.5-series model fine-tuned with reinforcement learning from human feedback (RLHF), the technique OpenAI introduced with InstructGPT, in which human preference rankings train a reward model that steers the network towards helpful answers. It reached an estimated 100 million users within two months — the fastest-growing consumer application in history at the time. On the image side, the pictures decorating this very post come from the same revolution: DALL·E 2 (April 2022), Stable Diffusion (August 2022) and Midjourney are all diffusion models, which learn to reverse a gradual noising process, generating images by iteratively denoising random pixels under text guidance.

Midjourney prompt: The unyielding spirit of human curiosity

Elena: Why AI cannot draw human hands properly? Something always wrong with the fingers?

chatGPT: Drawing human hands accurately and realistically is a challenging task for AI due to several reasons. One of the primary reasons is the complex and intricate nature of human hands. Hands possess a wide range of movements and configurations, and their structure involves numerous joints, bones, muscles, and tendons. Capturing the subtle nuances and intricacies of these elements accurately requires a deep understanding of human anatomy, perspective, and proportions.

Additionally, human hands exhibit a high level of variability among individuals. Factors such as hand size, finger length, and joint flexibility can vary significantly from person to person. AI models typically learn from large datasets to generalize patterns, but the vast diversity in hand appearances makes it difficult to capture the precise details of every individual hand.

Another challenge lies in the interpretation and translation of the three-dimensional structure of hands onto a two-dimensional drawing surface. AI models often struggle to perceive depth and accurately represent the spatial relationships between the various parts of a hand. This can result in distorted or unnatural-looking hand shapes, especially when attempting to capture complex poses or foreshortening.

Lastly, the lack of specific training data for hands or limited representation of diverse hand poses and gestures in training datasets can also contribute to AI's difficulty in drawing human hands realistically. Training AI models with more extensive and diverse hand-related datasets can help improve their accuracy in hand drawing tasks.

While AI has made significant progress in generating realistic images and understanding human anatomy, achieving flawless hand drawings remains an ongoing challenge due to the complexity and individuality of human hands.

Yet, the true magic lay in the triumph of technology and in democratising its power. With the advent of big data and the ethereal presence of cloud computing, the once elusive realm of AI became increasingly accessible, extending its hand to businesses and individuals alike. The symphony of progress harmonised with the chorus of opportunity as AI tools transcended the boundaries of exclusivity, embracing a more expansive audience eager to harness their transformative potential. Thus, a new chapter unfolded, where the cadence of innovation resonated with the aspirations of a world ready to embark on an extraordinary journey of discovery and growth.

State of AI Art Machine Learning Models

State of AI Art Machine Learning Models, Wikimedia Commons image

Modern AI Applications and Future Directions: Virtual Assistants, Autonomous Vehicles, Healthcare

In the tapestry of our present reality, AI tools have woven themselves seamlessly into the very fabric of our lives. Their ubiquitous presence permeates every corner of our existence like shimmering threads connecting the realms of possibility and convenience. From the enchanting whispers of virtual assistants within our smartphones to the guiding hands of recommendation systems gracing the vast landscapes of e-commerce platforms, AI has embraced us with open arms, shaping our experiences in ways we could have once imagined.

Behind those enchanting whispers sits some rather concrete machinery. Virtual assistants such as Siri (2011) and Alexa (2014) chain together on-device wake-word detection, cloud-based speech recognition, and natural language understanding. Recommendation systems trace their modern lineage to the Netflix Prize (2006–2009), which popularised matrix factorisation for collaborative filtering — predicting your taste from the ratings of people statistically similar to you — though by now the large platforms have largely moved on to deep learning models. Autonomous driving is graded on the SAE’s six levels (0–5); at the time of writing, Waymo operates Level 4 driverless taxis in Phoenix, while most consumer “autopilot” systems remain firmly at Level 2, with the human legally in charge. In healthcare, DeepMind’s AlphaFold 2 achieved a median accuracy of 92.4 GDT at the CASP14 assessment in 2020 — effectively solving the 50-year-old protein structure prediction problem — and in 2022 released predicted structures for over 200 million proteins, essentially every catalogued protein known to science.

Yet, the story of AI’s integration into our lives is far from reaching its final chapter. Like intrepid explorers on an uncharted quest, researchers and developers push the boundaries of this ever-evolving field with unwavering determination. Their inquisitive spirits lead them down unexplored paths, where new frontiers beckon with a siren’s call. In the realm of explainable AI, they strive to unravel the mysterious depths of artificial minds, seeking transparency and understanding in the mysterious workings of these wondrous creations. The dream of autonomous vehicles dances in the minds of visionaries, painting a portrait of a future where roads are guided by an intelligence that rivals our own. In the corridors of healthcare, the quest for more accurate and efficient diagnostics propels AI into uncharted territories, promising a world where lives can be saved, and ailments can be thwarted with unprecedented precision.

The journey of AI is a tapestry woven with both exciting possibilities and formidable challenges that lie ahead. With each passing moment, it evolves, adapting to the needs and desires of a world that embraces its enchantments. The symphony of progress resonates, beckoning us to a future with breathtaking innovations and unexpected hurdles. As we traverse this uncharted path, the brilliance of AI shines as a guiding light, illuminating our way forward.

The allure of tomorrow whispers promises of unimaginable wonders: cities buzzing with intelligent infrastructure, seamlessly connected ecosystems fostering efficiency and sustainability, and a world where AI empowers individuals and societies to reach new heights. Yet, amidst the symphony of possibilities, challenges loom, casting their shadow upon the landscape of progress.

Ethical considerations demand our unwavering attention, urging us to tread carefully and ensure that AI remains a force for good. Striking a delicate balance between innovation and responsibility becomes paramount as we navigate the delicate dance between harnessing the immense power of AI and safeguarding the principles of privacy, fairness, and accountability. The road ahead is paved with thorny questions, demanding thoughtful deliberation and collaborative efforts to shape an AI-driven future that aligns with our shared values.

Nonetheless, the spirit of human ingenuity prevails, propelling us forward with relentless determination. The evolution of AI is an unfolding saga, where each chapter unravels new possibilities and deepens our understanding of its boundless potential. It is a tapestry interwoven with the threads of scientific inquiry, technological breakthroughs, and societal transformations.

Amid this grand journey, we stand on the cusp of greatness, poised to embrace the untold adventures that await. As AI becomes an ever more integral part of our lives, we collectively seek to shape its destiny, harnessing its transformative power to forge a future that embodies our highest aspirations. Together, we step into the uncharted realms, driven by curiosity, guided by wisdom, and inspired by the belief that the path of AI holds the key to a world of marvels yet to be discovered.

Midjourney prompt: Self-driving car

p.s. I like the idea of sitting on the top of the self-driving car, with the wind in my hair and good weather conditions. Although, I would have used good sun protection :)

It is funny how AI created the image of a self-driving car with a place for a human on it. I am concerned about the safety, though.

I have linked research papers and educational material about the AI milestones inside the timeline. I hope that you do not mind. I will update these post links soon.

Did you like this post? Please let me know if you have any comments or suggestions.

Posts about AI that might be interesting for you





References

1. Guide Midjourney –stylize Explained

2. Mastering Midjourney Prompts for Stunning Images

3. New Chat (chatGPT by OpenAI)

desktop bg dark

About Elena

Elena, a PhD in Computer Science, simplifies AI concepts and helps you use machine learning.

Citation
Elena Daehnhardt. (2023) 'The Remarkable Evolution and Milestones of AI', daehnhardt.com, 23 May 2023. Available at: https://daehnhardt.com/blog/2023/05/23/ai-evolution-and-milestones/
All Posts