All posts by admin

Over the past several years, the AI industry has focused heavily on training increasingly large models.

Discussions about artificial intelligence often center around massive GPU clusters, trillion-parameter architectures, and the enormous computational resources required to train modern systems. 

Training has become the most visible symbol of progress in machine learning, and it often dominates headlines across the technology industry.

However, once a model is trained, the real operational challenge begins. Every A-powered product relies on inference and the process of running a trained model to generate predictions or responses. 

As organizations move from experimentation to production environments, it becomes clearer that the long-term engineering challenge of AI is not only training models, but running them efficiently at scale.

The shift from training to operational AI

Training a modern model requires substantial computing resources, but for most organizations, it is not a daily activity. A model may be trained or fine-tuned periodically and then deployed to support applications that operate continuously.

Once deployed, the same model may serve thousands or millions of requests every day across multiple systems.

This changes how companies must think about AI infrastructure. Training represents a large but relatively short-lived workload, while inference becomes an ongoing operational workload that grows with usage.

As AI capabilities become embedded in enterprise applications, the number of inference calls increases rapidly. Over time, the cost and engineering complexity of running models in production can exceed the original cost of training them.

AI’s split future: Control vs autonomy in frontier systems

AI is splitting in two directions. One path is controlled, restricted, and security-first. The other is open, autonomous, and scaling fast. The real question isn’t which is better, it’s what this means for trust.

AI Accelerator InstituteAndrew Lovell

Inference as the operational core of AI systems

Every inference request consumes compute resources.

When a user sends a prompt to a language model, the system processes the input tokens and generates output tokens step by step.

Large language models generate responses sequentially, which means the model remains active throughout the entire generation process, continuing to use GPU memory and compute resources.

At scale, these operations become significant. Enterprise copilots, automated support systems, and AI-powered search tools may process millions of prompts each day.

The infrastructure supporting these systems must manage latency, GPU utilization, and memory constraints while maintaining predictable performance.

As organizations expand their AI deployments, the focus naturally shifts toward improving inference efficiency.

The architecture of enterprise AI platforms

The second layer prepares models for production through optimization techniques such as quantization, distillation, and parameter-efficient fine-tuning.

The final layers focus on inference infrastructure and applications where models are served through scalable APIs and integrated into products. 

In many production environments, the most complex engineering challenges occur in these later stages, where models must operate reliably under real workloads.

The machine learning paradox: Can AI actually learn?

We call it machine learning. But do machines actually learn? Today’s AI systems train, optimize, and scale, but real learning is something else entirely. The distinction matters more than the industry wants to admit.

AI Accelerator InstituteAndrew Lovell

Scaling inference for large language models

Running large language models efficiently requires several optimization techniques.

Quantization reduces the numerical precision of model weights, which allows models to run faster and consume less memory. Distillation allows smaller models to replicate the behavior of larger models for specific tasks, which can significantly reduce compute requirements.

Infrastructure-level improvements are also important. Continuous batching allows multiple requests to be processed together, which increases hardware utilization. 

Techniques such as KV cache reuse and speculative decoding improve token generation throughput and reduce latency.

These optimizations make it possible to run large models in production systems where both cost and performance matter.

Modern infrastructure for large-scale inference

As AI adoption grows, new infrastructure patterns are emerging to support inference workloads. One approach is server-less inference, where compute resources automatically scale based on demand.

Instead of maintaining GPU clusters that run continuously, the system can allocate resources dynamically as requests arrive, improving overall utilization.

Another important development is GPU sharing and multi-model serving. Instead of dedicating a GPU to a single model, modern inference platforms allow multiple models to run on the same hardware and schedule requests dynamically.

Techniques such as request batching and model multiplexing further improve efficiency by enabling the system to support many workloads without continuously expanding infrastructure.

Agents and the amplification of inference workloads

A major change in AI applications is the rise of agent-based systems. Traditional AI applications typically generate a single response to a user request. Agent systems behave differently because they perform multi-step reasoning before producing a final result.

An agent may break down a task into smaller steps, retrieve information from external systems, and generate several intermediate prompts during the process. Each step usually requires another model inference. 

As a result, a single user request may trigger many model executions instead of just one.

Agent-driven workflows, therefore, amplify the amount of inference performed by the system and increase the demand on the underlying infrastructure.

Infrastructure implications of agent workloads

To support these workloads efficiently, infrastructure must manage high volumes of model calls while maintaining predictable performance.

Techniques such as model routing, efficient batching, GPU sharing, and dynamic scaling become even more important when agent workflows operate at scale. 

As organizations adopt agent-driven automation, the importance of efficient inference infrastructure continues to grow.

Designing systems with inference efficiency in mind

As organizations gain experience with production AI deployments, many teams are beginning to design architectures that prioritize inference efficiency from the outset.

Instead of relying on a single large model, systems may route simple tasks to smaller models and reserve larger models for more complex reasoning tasks.

Other design strategies include streaming responses so users can see results as they are generated, and dynamically scaling infrastructure based on real-time demand. Efficient scheduling and GPU sharing can further improve hardware utilization and reduce operational costs.

These approaches help ensure that both language model applications and agent-driven workflows can operate reliably at scale.

The story of Sora: What it reveals about building real-world AI

After ChatGPT’s breakthrough, the race to define the next frontier of generative AI accelerated. One of the most talked-about innovations was OpenAI’s Sora, a text-to-video AI model that promised to transform digital content creation.

AI Accelerator InstituteAndrew Lovell

The future of AI infrastructure

The broader technology ecosystem is beginning to adapt to the growing importance of inference workloads.

Hardware vendors are developing accelerators optimized specifically for inference performance, while cloud platforms are introducing systems designed for large-scale model serving.

As agent-based applications become more common, the number of inference requests will continue to increase. 

Conclusion

Artificial intelligence is entering a new stage of maturity. Early progress focused on training large models and demonstrating the capabilities of modern machine learning systems. These breakthroughs established the foundation for the rapid expansion of AI across industries.

As AI becomes embedded in real applications, the focus is shifting toward how these systems operate in production environments. Inference now represents the core workload that powers both language models and agent-driven systems. 

Organizations that design infrastructure optimized for efficient inference will be best positioned to support the next generation of intelligent applications. In the long run, training happens occasionally, but inference and agent execution happen continuously.

Why does the internet feel like it’s getting worse every single day, and why does it feel like the political landscape is getting worse in response? The answer might seem obvious, especially if you read 404 Media on a regular basis, where we’ve been documenting this decline, but it’s important to occasionally zoom out and ask the big questions. 

That’s why this week on the podcast I’m joined by Whitney Phillips. Phillips is the author of several books about internet culture and ethics, including This is Why We Can’t Have Nice Things and The Ambivalent Internet. She’s a professor of information politics and media ethics at the University of Oregon, and also one of my favorite people to talk to and listen to because she’s a genius when it comes to the kind of internet culture and platform dynamics we report on every day at 404 Media. 

I wanted to talk to Whitney today because it’s been a few years since we talked in depth about the state of the internet and so much has changed in that time, sadly for the worst, and I really wanted some help in understanding the current state of things, as bad as they are. We also spent quite a bit of time talking about her upcoming book, The Shadow Gospel: How Anti-liberal Demonology Possessed U.S. Religion, Media, and Politics.

404 Media is a journalist-founded company and needs your support. To subscribe, go to 404media.co. As well as bonus content every single week, subscribers get access to additional episodes where we respond to their best comments. Subscribers also get early access to our interview series. Gain access to that content at 404media.co.

Listen to the weekly podcast on Apple Podcasts, Spotify, or YouTube. 

Become a paid subscriber for early access to these interview episodes and to power our journalism. If you become a paid subscriber, check your inbox for an email from our podcast host Transistor for a link to the subscribers-only version! You can also add that subscribers feed to your podcast app of choice and never miss an episode that way. The email should also contain the subscribers-only unlisted YouTube link for the extended video version too. It will also be in the show notes in your podcast player.

A hacker has compromised a backend system for Doublespeed, an a16z-funded startup that uses a phone farm to flood social media with AI-generated TikTok accounts, and attempted to have those accounts post memes calling a16z the “antichrist,” according to screenshots seen by 404 Media.

The hack is at least the second time Doublespeed has been compromised. The startup uses AI to create fake influencers, generate videos, and post comments.

“a16z is the antichrist. sponsored by doublespeed.ai,” the meme says. It includes images of a16z co-founder Marc Andreessen; a woman pole dancing; and occult symbol Baphomet.

The screenshots show the meme queued up for publication in Doublespeed customers’ dashboard, seemingly to post to their associated social media accounts. A caption indicates the hacker stole some other data and may tried to post content from hundreds of accounts.

“47MB exfiltrated. 573 accounts postable. 413 phones dumped. A16z portfolio security built different,” the caption reads. 

It appears the meme was ultimately not posted on Doublespeed customers’ social media accounts. One screenshot included the social media handle of an impacted Doublespeed account; as of Monday, the meme was not available on that account.

Zuhair Lakhani, a co-founder of Doublespeed, told 404 Media in an email “We’re aware of the unauthorized access attempt and addressed it quickly. This involved an older system for queuing posts that had remained in place for compatibility with existing customer workflows, and we have since secured it.”

“Importantly, no unauthorized posts were successfully published, and we have not seen evidence that this attempt resulted in broader impact to customers,” he added.

404 Media first reported about Doublespeed last year, after the startup raised $1 million from a16z as part of its “Speedrun” accelerator program, “a fast‐paced, 12-week startup program that guides founders through every critical stage of their growth.” Doublespeed markets its use of phone farms as a way to evade social media platforms’ policies against removing inauthentic behavior. Doublespeed customers get access to a dashboard that allows them to operate multiple AI-generated influencers. At the moment Doublespeed focuses on operating TikTok accounts, but also plans to give customers the ability to operate accounts on X and Instagram. 

Doublespeed was previously hacked in December of 2025. The data from that hack revealed at least 400 TikTok accounts Doublespeed operates and that at least 200 of those were actively promoting products on TikTok, mostly without disclosing that they are ads or not real people. Some of the products promoted by these AI-generated accounts included supplements, massagers, and dating apps.     

As we’ve noted last year, Marc Andreessen, after whom half of Andreessen Horowitz is named, also sits on Meta’s board of directors. Meta did not respond to our question about one of its board members backing a company that blatantly aims to violate its policy on “authentic identity representation.”

WebinarTV, a site that scrapes Zoom webinars without permission, has downloaded and posted Zoom Webinars for anonymous addiction recovery meetings, support groups for caregivers and people who suffer from chronic illness, and a meeting of nudists.

WebinarTV’s Michael Robertson told me that the company asks every single person for permission to “promote” their webinars, but these specific examples show that WebinarTV scrapes and shares the videos on its site before asking for permission and that some people are not aware that this is happening to them.

“As with all of our support group meetings, this meeting was not intended to be recorded, but rather to be a private discussion among participants,” Kimberly Dorris, executive director at the Graves’ Disease & Thyroid Foundation (GDATF), which hosted a Zoom session which vetted participants, and which still ended up on WebinarTV, wrote in a post about the meeting being uploaded to WebinarTV. That post was titled “A Warning For Patient Communities Connecting on Zoom.”

I first reported about WebinarTV in March, after a teacher told me that a sensitive meeting he held on Zoom for educators who wanted to protect their students from ICE raids ended up on the site. The teacher found out about the video when a someone calling themselvesSarah Blair, which appears to be an AI-generated persona, sent him an email letting him know that the meeting was posted to WebinarTV and also turned into an AI-generated podcast. The teacher asked WebinarTV to take down the meeting because it could put some of the participants in danger, and WebinarTV removed it shortly after. 

WebinarTV claims it hosts more than 200,000 Zoom webinars it scraped this way. 

After I published the story, several people who use Zoom regularly for meetings or webinars they consider private checked to see if their Zoom videos were posted to WebinarTV and got in touch with me. 

Gillian Brockwell, a journalist and 404 Media reader who goes to addiction recovery meetings on Zoom searched WebinarTV for her own meeting after seeing my story. She didn’t find her own meetings, but flagged several other meetings that were clearly meant to be for people who want to preserve their anonymity. 

One meeting posted to WebinarTV for “panic anonymous,” or people who suffer from panic and high anxiety, was described as a “a confidential group that bridges decades of clinical biofeedback practice with modern wearable technology.” The recording of the webinar posted to WebinarTV included participants’ full names and shows their faces. 

A 12 steps and faith-based recovery meeting for people with substance abuse issues also shows participants full names and faces. 

“If I found out I was in one of these meetings captured by WebinarTV, I would feel terrified and betrayed, especially if I were in early recovery,” Brockwell told me. “These meetings are clearly meant to be confidential and anonymous, and anonymity is a key component of mutual-support and 12-step recovery models. It allows people a pathway through the stigma that so often prevents them from seeking help, and members sharing openly about some of the most humiliating moments in their lives – things they might never say in public – is a key part of ‘identifying in.'” 

“I hosted a meeting last night that was intended to be for family members of patients with Graves’ disease, thyroid eye disease, and Hashimoto’s thyroiditis,” Dorris from the GDATF told me in an email in March. “The link to *register* was public, but in order to receive the joining link, you had to fill out a questionnaire.”

The description for the Zoom meeting was: “Has a loved one been diagnosed with Graves’ disease, thyroid eye disease, or Hashimoto’s thyroiditis? Join us for a short presentation followed by an interactive discussion with people who understand what your family is going through! This meeting is intended for family members and caregivers only. If you are a researcher, industry representative, etc. please contact GDATF at info@gdatf.org to discuss how we can better assist you.”

The registration form specifically asked potential participants whether they were attending in support of or on behalf of someone impacted by these conditions, and were admitted to the meeting one at a time from a Zoom waiting room. Dorris said that no visible AI and transcription tools were running. 

One meeting of nudists, or “naturists,” also featured every participant’s face and name, and some appeared shirtless on camera. It’s not clear if this meeting was designed to be private nor if the participants know the meeting was recorded and posted on WebinarTV. 

Robertson told me that WebinarTV is not violating these people’s privacy because the site only scrapes Zoom webinars as opposed to Zoom meetings. Zoom webinars work similarly to a regular Zoom meeting, but are intended for larger audiences with features like polling, breakout rooms, and EventBrite integrations. 

“Webinars are no different than Facebook Live, X broadcast, or Youtube Live. They are broadcast to the public. This is why we have 200,000 webinars and zero issues to date,” Robertson told me. “We contact every host, twice to make sure they want the promotion. We’re the only search engine that does this. Also we make it one click easy to remove. Go try and get something removed from any other search engine.” Robertson is of course ignoring the fact that many people organizing or joining these sessions, even if they are technically webinars, expect them to be private or limited to just the participants.

When I reached out to Zoom in March it said that based on its review WebinarTV accesses meetings using links that have been shared publicly, then records the sessions using browser extension or “other tools.” 

“Because these recordings occur on the participant’s device and outside of Zoom’s environment, no platform—including Zoom—has the technical ability to fully prevent third-party screen recording,” the spokesperson said. 

“While it is true that our meeting wasn’t infiltrated due to a technical flaw from Zoom, as a customer, I would still like to see Zoom speak out against companies like WebinarTV that send bots with fake identities to infiltrate meetings and covertly record participants who had a reasonable expectation of privacy,” Dorris told me.