The hype around Artificial Intelligence, the now failed U.S. attempt to monopolize it, and the recent counter from China are a lesson in how to innovate. They also show that the U.S. is losing the capability to do so.
In mid 2023, when the Artificial Intelligence hype gained headlines, I wrote:
'Artificial Intelligence' Is (Mostly) Glorified Pattern Recognition
Currently there is some hype about a family of large language models like ChatGPT. The program reads natural language input and processes it into some related natural language content output. That is not new. The first Artificial Linguistic Internet Computer Entity (Alice) was developed by Joseph Weizenbaum at MIT in the early 1960s. I had funny chats with ELIZA in the 1980s on a mainframe terminal. ChatGPT is a bit niftier and its iterative results, i.e. the 'conversations' it creates, may well astonish some people. But the hype around it is unwarranted.
…
Currently the factual correctness of the output of the best large language models is an estimated 80%. They process symbols and pattern but have no understanding of what those symbols or pattern represent. They can not solve mathematical and logical problems, not even very basic ones.There are niche applications, like translating written languages, where AI or pattern recognition has amazing results. But one still can not trust them to get every word right. The models can be assistants but one will always have to double check their results.
Overall the correctness of current AI models is still way too low to allow them to decide any real world situation. More data or more computing power will not change that. If one wants to overcome their limitations one will need to find some fundamentally new ideas.
But the hype continued. One big AI model, ChatGPT, was provided by a non-profit organization, OpenAI. But its CEO, Sam Altman, soon smelled the big amount of dollars he potentially could make. A year after defending the the non-profit structure of OpenAI Altman effectively raided the board and took the organization private:
ChatGPT-maker OpenAI is working on a plan to restructure its core business into a for-profit benefit corporation that will no longer be controlled by its non-profit board, people familiar with the matter told Reuters, in a move that will make the company more attractive to investors.
…
Chief executive Sam Altman will also receive equity for the first time in the for-profit company, which could be worth $150 billion after the restructuring as it also tries to remove the cap on returns for investors, sources added.
The ChatGTP large language model OpenAI provided was closed source. A black-box, running in the cloud, that one could pay to chat with or use for translating, content generation or analyzing certain problems.
The training and maintaining of ChatGTP took large amounts of computing power and money. It was somewhat expensive but there was no new technology in it. The algorithms it used were well known and the training data needed to 'program' it were freely available internet content.
For all the hype about AI is is not a secret or even new technology. The barriers to entry for any competition is low.
That is the reason why Yves at Naked Capitalism, pointing to Edward Zitron, asked: “How Does OpenAI Survive?” It doesn't. Or has little chance to do so. Discussions in the U.S. never acknowledged those facts.
Politicians thought of AI as the next big thing that would further U.S. control of the world. They attempted to prevent any potential competition to the lead the U.S. thought it had in that field. Nvidea, the last leading U.S. chip maker, lost billion when it was prohibited from selling in latest AI-specialized models to China.
Two days ago Trump announced Stargate, a $500 billion AI infrastructure investment in the US:
Three top tech firms on Tuesday announced that they will create a new company, called Stargate, to grow artificial intelligence infrastructure in the United States.
OpenAI CEO Sam Altman, SoftBank CEO Masayoshi Son and Oracle Chairman Larry Ellison appeared at the White House Tuesday afternoon alongside President Donald Trump to announce the company, which Trump called the “largest AI infrastructure project in history.”
The companies will invest $100 billion in the project to start, with plans to pour up to $500 billion into Stargate in the coming years. The project is expected to create 100,000 US jobs, Trump said.
Stargate will build “the physical and virtual infrastructure to power the next generation of AI,” including data centers around the country, Trump said. Ellison said the group’s first, 1 million-square foot data project is already under construction in Texas.
On the very same day, but with much less noise, a Chinese company published another AI model:
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1. DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning (RL) without supervised fine-tuning (SFT) as a preliminary step, demonstrated remarkable performance on reasoning. With RL, DeepSeek-R1-Zero naturally emerged with numerous powerful and interesting reasoning behaviors.
The new DeepSeek models have better benchmarks than any other available model. They use a different combination of technics, less training data and much less computing power to achieve that. They are cheap to use and, in contrast to OpenAI, real open source.
U.S. export controls on advanced semiconductors were intended to slow China's AI progress, but they may have inadvertently spurred innovation. Unable to rely solely on the latest hardware, companies like Hangzhou-based DeepSeek have been forced to find creative solutions to do more with less.
…
This month, DeepSeek released its R1 model, using advanced techniques such as pure reinforcement learning to create a model that's not only among the most formidable in the world, but is fully open source, making it available for anyone in the world to examine, modify, and build upon.
…
DeepSeek-R1’s performance is comparable to OpenAI's top reasoning models across a range of tasks, including mathematics, coding, and complex reasoning. For example, on the AIME 2024 mathematics benchmark, DeepSeek-R1 scored 79.8% compared to OpenAI-o1’s 79.2%. On the MATH-500 benchmark, DeepSeek-R1 achieved 97.3% versus o1’s 96.4%. In coding tasks, DeepSeek-R1 reached the 96.3rd percentile on Codeforces, while o1 reached the 96.6th percentile – although it’s important to note that benchmark results can be imperfect and should not be overinterpreted.But what’s most remarkable is that DeepSeek was able to achieve this largely through innovation rather than relying on the latest computer chips.
Nature is likewise impressed:
A Chinese-built large language model called DeepSeek-R1 is thrilling scientists as an affordable and open rival to ‘reasoning’ models such as OpenAI’s o1.
…
“This is wild and totally unexpected,” Elvis Saravia, an AI researcher and co-founder of the UK-based AI consulting firm DAIR.AI, wrote on X.R1 stands out for another reason. DeepSeek, the start-up in Hangzhou that built the model, has released it as ‘open-weight’, meaning that researchers can study and build on the algorithm. Published under an MIT licence, the model can be freely reused but is not considered fully open source, because its training data has not been made available.
“The openness of DeepSeek is quite remarkable,” says Mario Krenn, leader of the Artificial Scientist Lab at the Max Planck Institute for the Science of Light in Erlangen, Germany. By comparison, o1 and other models built by OpenAI in San Francisco, California, including its latest effort o3 are “essentially black boxes”, he says.
Even long term Internet investors, who have seen it all, are impressed:
Marc Andreessen 🇺🇸 @pmarca – 9:19 UTC · Jan 24, 2025
Deepseek R1 is one of the most amazing and impressive breakthroughs I’ve ever seen — and as open source, a profound gift to the world. 🤖🫡
Nature adds:
DeepSeek hasn’t released the full cost of training R1, but it is charging people using its interface around one-thirtieth of what o1 costs to run. The firm has also created mini ‘distilled’ versions of R1 to allow researchers with limited computing power to play with the model.
That does in fact work!
Brian Roemmele @BrianRoemmele – 14:34 UTC · Jan 23, 2025
Folks, I think we have done it!
If overnight tests are confirmed we have OPEN SOURCE DeepSeek R1 running at 200 tokens per second on a NON-INTERNET connected Raspberry Pi.
A full frontier AI better than “OpenAI” owned fully by you in your pocket free to use!
I will make the Pi image available as soon as all tests are complete.
You just pop it into a Raspberry Pi and you have AI!
This is just the start of the power that takes place when you TRULY Open Source an AI Model.
The latest Rasberry Pi hardware starts at $50. The software is free.
This is a death call for OpenAI:
Arnaud Bertrand @RnaudBertrand – 14:23 UTC · Jan 21, 2025
Most people probably don't realize how bad news China's Deepseek is for OpenAI.
They've come up with a model that matches and even exceeds OpenAI's latest model o1 on various benchmarks, and they're charging just 3% of the price.
It's essentially as if someone had released a mobile on par with the iPhone but was selling it for $30 instead of $1000. It's this dramatic.
What's more, they're releasing it open-source so you even have the option – which OpenAI doesn't offer – of not using their API at all and running the model for "free" yourself. …
The backstory of DeepSeek is also amazing.
In 2007 three Chinese engineers set out to build a quant (financial speculation) fund using AI. They hired hungry people fresh from the universities. Their High-Flyer fund was somewhat successful but throughout the last years the Chinese government started to crack down on financial engineering, quant trading and speculation.
With time on their hand and unused computing power in their back room the engineers started to build the DeepSeek models. The costs were minimal. While OpenAI, Meta and Google spent billions to build their AI's the training costs for the published DeepSeek models were mere $5 to 6 million.
Henry Shi @henrythe9ths – 23:20 PM · Jan 20, 2025
7. The lesson?
Sometimes having less means innovating more. DeepSeek proves you don't need:
– Billions in funding
– Hundreds of PhDs
– A famous pedigree
Just brilliant young minds, the courage to think differently and the grit to never give up
Another lesson is that brilliant young minds should not be wasted to optimize financial speculation but to make stuff one can use.
DeepSeek demonstrates how it is impossible to use trade and technology barriers to keep technology away from competitors. They can, with decent resources, simply innovate around those.
Even billions of dollars, loud marketeers like Trump and self promoting grifters like Sam Altman can not successfully compete with a deep bench of well trained engineers.
As an author at Guancha remarks (machine translation):
In the Sino-US science and technology war, China's unique advantage comes precisely from the US ban. It can be said that our strong will to survive was forced out by Washington, and maximizing our limited resources is the secret to breaking through. In history, this kind of story is not new, that is, the weak prevail over the strong, and the small fight against the big.
The U.S. side will fall into a Vietnam-style dilemma-relying too much on its own absolute advantage, thus wasting a lot of resources and losing itself to internal consumption.
How long for the U.S. to (re-)learn that lesson?