.png)
DeepSeek, ChatGPT, and the Future of AI
The revolution of distillation models and the European stumbling blocks
Everyone has an opinion about DeepSeek. Of course, Wouter Verlinden too!
Thanks to DeepSeek's innovation, the costs of training advanced AI models have fallen dramatically. Where only wealthy tech giants such as OpenAI, Meta and Google used to have the resources to develop cutting-edge AI systems, it is now increasingly easy to train new models. This has the potential to cause a Cambrian explosion of AIs, resulting in a diversity of models that spread and innovate rapidly. Presumably, not everyone will continue to work with an OpenAI model. Workplace diversity, including for our AI colleagues! For humanity, this is a positive development, but for the established American big tech companies, it is a nightmare: the exclusive control they thought they were maintaining is slipping through their fingers.
After the old industries such as steel, shipping and the automotive industry, China is now also outpacing the West in robots and AI. Fortunately, we are still good at making handbags in Europe.
The rise of distillation models
One of the causes of these cost reductions is the rise of distillation models. These are models that are trained by other models, a process that is reminiscent of the educational system in which biological neural networks train each other. DeepSeek, a Chinese AI developer, already seems to be using this technique to train new versions of their own models. Of course, this is contrary to the terms of service from OpenAI and Meta. But this means that an AI model can refine and improve itself without the need for human intervention. This puts us at the point of a technological 'liftoff', where AIs train AIs and development accelerates exponentially.
What a time to be alive! Rockets, AI, electric bikes, flat screens, cell phones,... Who would have thought science fiction would become non-fiction?
Why isn't this happening in the EU?
In the United States and China, new AI developments are being implemented rapidly, while Europe is sidelining itself. The European AI Act, intended to ensure transparency and safety, imposes strict requirements on training data and the use of General AI models. Not only does the training data have to comply with copyright law (how do you get started if the training data includes the entire internet and all books?) , but all training data must also be kept. In practice, this makes distillation almost impossible, because the origin and processing of training data is no longer easy to determine. AI teacher-student training takes place directly at the level of digital neurons and does not immediately have a human-readable equivalent. While the rest of the world experiments and innovates, Europe is blocking itself with regulations that are essentially out of date at the time of implementation.
Lessons from history
The situation in Europe is reminiscent of Dutch patent law between 1869 and 1912. During this period, the Netherlands abolished patent law under the influence of free market abusers among the liberals. This gave companies such as Philips (°1891) the opportunity to develop rapidly and become an industrial heavyweight. ASML, On semiconductor, Signify and NXP are direct consequences of this.
A similar situation with Nordisk (and its competitor Novo, who would later merge). Drugs such as insulin were not patentable (the production process, however). The non-patentability of drugs in the early 20th century gave Nordisk space to produce insulin without legal obstacles and to position itself as a market leader. US patent holder Eli Lili and Novo Nordisk still dominate these gigantic markets more than 120 years later.
Europe can take the same course and all Remove copyright restrictions and training requirements on AI training courses. Distillation makes it possible to build competing models even at a disadvantage, but current legislation prevents these developments. It is not that the American and Chinese modelers are complying with European legislation.
If we want everything to stay as it is, everything has to be different.
Author: Wouter Verlinden
Date: 29/01/2025
