these days I saw an explanation about a DeepSeek paper that I found really interesting.
the main idea is simple, but also a bit counterintuitive.
when people think about making AI models faster, the first thing that comes to mind is more GPUs, more chips, more data centers and more money spent on compute.
but in some scenarios, the problem is not only the amount of hardware.
it is how this hardware is being used.
and from what I understood, in longer workloads, especially with AI agents, there is a lot of waste happening.
the GPUs are there.
the model has work to do.
but a relevant part of the system is waiting for data to arrive.
this is kind of crazy when you think about how much money companies are spending on infrastructure to run AI.
it does not help much to have a giant brain if the information gets to it slowly.
the analogy that made the most sense to me was this:
imagine a huge brain trying to read a book through a straw.
the brain has enough capacity to process a lot of information.
but the information arrives in a limited way.
so it spends less time thinking and more time waiting for the reading to happen.
in AI, this shows up a lot when the context gets bigger.
a short conversation is simpler.
you send a question, the model processes it and answers.
but when we talk about agents, the scenario changes.
the agent reads documents, calls tools, goes back to the history, uses previous context, creates plans, executes steps and keeps working for longer.
this puts much more pressure on the inference system.
and here there is an important technical point: prefill and decoding.
prefill is the part where the model processes the context that already exists.
it is like the model is reading everything that came before.
decoding is the part where the model starts generating the next tokens of the answer.
the problem is that, in some systems, the machines responsible for prefill become congested.
they become the bottleneck.
at the same time, the decoding machines may still have capacity available.
so one part of the system is overloaded and another part is underused.
the idea from DeepSeek was to look at this imbalance and use the existing resources better.
instead of leaving the decoding machines waiting, they can also help with reading the context.
but this needs to be done carefully.
when you start moving data around inside the data center, you can create another bottleneck.
you solve the prefill problem, but you congest the internal communication.
so the smart part of the paper is the control of this flow.
the system gives priority to the most important inference traffic and uses the remaining capacity to move memory and context.
it is almost like organizing the traffic inside the data center.
the hardware is the same.
but the path the data takes becomes more efficient.
and this changes a lot.
according to the explanation, they were able to go from something close to 40% utilization to something close to 80%.
if this result holds in real environments, it is a huge gain.
because we are not talking about doubling the number of GPUs.
we are talking about making the GPUs that already exist work better.
this part caught my attention because many people look at AI only through the model layer.
which model is smarter.
which model has more parameters.
which model wins the benchmark.
but there is another layer that is just as important: the infrastructure that makes these models run.
inference is where the cost appears every day.
every API call, every agent running, every long context, every document processed, all of this becomes compute cost.
if the infrastructure is inefficient, scaling AI becomes expensive very quickly.
and with agents, this becomes even more serious.
a normal chatbot can already be expensive at scale.
an agent that works for several minutes, reads context, calls tools and reasons through multiple steps can be much more expensive.
that is why this kind of improvement matters.
it can make model usage more viable.
faster.
cheaper.
more scalable.
and maybe this is exactly the kind of progress that will make agents actually work in production.
today, a lot of people talk about agents as if it were only a prompt or model problem.
but for agents to work well at scale, the infrastructure needs to be good.
it needs to serve long context efficiently.
it needs to reduce bottlenecks.
it needs to control cost.
it needs to use GPUs better.
and this is where the DeepSeek paper becomes interesting.
it solves a problem happening underneath the model.
the kind of problem almost nobody sees, but that directly affects the final cost of AI.
for me, this shows something important:
the next phase of AI will not depend only on bigger models.
it will also depend on better engineering.
better memory usage.
better communication between machines.
better load distribution.
better inference.
better hardware utilization.
because at some point, buying more GPUs does not solve everything.
if the system is still badly used, you are just putting more resources on top of a bottleneck.
DeepSeek is showing that there is still a lot of hidden efficiency inside the current infrastructure.
and this is an important signal.
especially now, when everyone wants to run agents, use larger contexts and automate longer tasks.
in the end, what I took from this paper is this:
AI does not improve only when the model improves.
it also improves when we can run the same model with less waste.
and maybe this is one of the least talked about, but most important parts of AI progress right now.
Antonioni Nascimento
Software, cybersecurity, data & AI. Building advanced systems at KRX. Writing about technology, venture capital, and how software shapes markets.