Rifan Muazin
In the early months of 2025, Dario Amodei, CEO of Anthropic, offered a sobering assessment of the technology his company is at the forefront of developing. “There is now ample evidence, collected over the last few years, that AI systems are unpredictable and difficult to control,” he wrote. His statement is a reflection of a fundamental technical reality: as Large Language Models (LLMs) scale, their internal reasoning processes become increasingly opaque—a phenomenon known as the "black box" problem.
Yet, if you pivot your attention to the professional networking site LinkedIn, you will encounter an entirely different narrative. Here, the script is uniform, confident, and highly deterministic. It tells a story of "Generative Engine Optimization" (GEO) and "Answer Engine Optimization" (AEO) as if they were as manageable as traditional web traffic. The discourse is filled with promises of a "13% citation lift" through specific schema markups or a "2.8x conversion improvement" via chunk-level retrieval optimization.
This creates a dangerous cognitive dissonance. The people closest to these systems are expressing profound caution regarding their control, while those furthest from the technical architecture—consultants, agency leads, and content strategists—are increasingly certain they have "cracked the code." This gradient, where confidence grows in inverse proportion to actual technical proximity, is a manifestation of the Dunning-Kruger effect at an industrial scale.
The Architecture of Uncertainty: What the Builders Say
To understand why the "optimization" industry is currently peddling a fiction, one must first look at the state of mechanistic interpretability—the field dedicated to reverse-engineering how neural networks "think."
In May 2024, Anthropic released its seminal research on mapping the internal representations of its models. The opening statement was an admission of fundamental ignorance: “We mostly treat AI models as a black box: something goes in and a response comes out, and it’s not clear why the model gave that particular response instead of another.”
Two years later, that uncertainty has only deepened. Neel Nanda, who leads the mechanistic interpretability team at Google DeepMind, conceded in a September 2025 interview that the most ambitious goals of the discipline—specifically, the quest for "robust guarantees" regarding model behavior—are likely unattainable. Nanda’s assessment suggests that we are not moving toward a future where we can verify AI outputs with mathematical certainty; rather, we are moving toward a future of increasingly capable but fundamentally inscrutable systems.
This sentiment was echoed at the 2024 NeurIPS conference by Ilya Sutskever, a titan of the field and co-founder of Safe Superintelligence. In a candid moment during his Test of Time award acceptance, Sutskever noted: “The more it reasons, the more unpredictable it becomes.” For a man whose career is synonymous with the scaling hypothesis, this admission serves as a stark warning. As models move from simple pattern matching to complex reasoning, the very notion of "optimizing" them for specific, deterministic outcomes becomes a categorical error.
The Mirage of Optimization: Tactics Without Evidence
Despite the high-level warnings from research labs, the digital marketing industry has doubled down on deterministic vocabulary. Agencies now market "four-pillar frameworks" for technical GEO, promising to guarantee inclusion in AI Overviews. These claims rely on a specific, seductive linguistic style: "ensures," "guarantees," "dictates."
These tactics are almost universally born from anecdotal observations—a consultant notices a slight increase in traffic after changing a few headers, calls it "data," and builds a service offering around it. Crucially, these "studies" lack the foundational requirements of the scientific method: no control groups, no pre-registered hypotheses, and no isolation of variables. They are, in effect, confirmation in costume.
When a consultant claims that a 300-character paragraph limit dictates how a vector database chunks content, they are describing an architecture that they have not built and do not control. They are attempting to perform "surgery" on a black box by guessing what might be inside, ignoring the fact that the "patient" is actually a dynamic, evolving model that updates its weights and training data continuously.
The Empirical Falsification: A Reality Check
The bubble of confident speculation finally met the wall of empirical data in early 2026. In a landmark study published by Ahrefs, researchers Louise Linehan and Xibeijia Guan analyzed 1,885 pages that added JSON-LD schema markup between August 2025 and March 2026. By comparing these against 4,000 matched control pages, they were able to isolate the effect of the schema on AI citation likelihood.
The results were unequivocal: there was no meaningful uplift in citations across Google AI Overviews, Google AI Mode, or ChatGPT. In fact, pages that added the schema saw a slight, statistically significant decline in performance. The researchers noted that the probability of this result being due to random chance was roughly 1 in 2,500.
This study served as the empirical confirmation of a first-principles argument: LLMs do not "read" schema in the way traditional crawlers parse HTML. They ingest massive amounts of unstructured language. The SEO playbook of "chunking" and "markup" is, at best, a misunderstanding of current architecture and, at worst, an expensive placebo sold to clients who are desperate for a sense of control in a changing search landscape.
The Ultimate Arbiter: Google’s Own Guidance
If the Ahrefs study were not enough to deflate the GEO industry, Google itself provided the final blow. On May 15, 2026, the company published official documentation regarding the optimization of generative AI features.
The guide was uncharacteristically blunt. It explicitly rejected the necessity of llms.txt files, the requirement for content chunking, and the effectiveness of special schema markup for AI visibility. The documentation stated: “Many suggested ‘hacks’ aren’t effective or supported by how Google Search actually works.”
This put the industry in a rare, precarious position. Within a single fortnight, the "optimization" playbook had been dismantled by:
- First-principles analysis: Recognizing the inherent nature of LLM architecture.
- Controlled empirical measurement: The Ahrefs study.
- The source itself: Google’s official developer guidelines.
The Cost of the "Confidence Tax"
Why, then, do these frameworks continue to sell? The answer lies in the incentive structures of the modern professional economy.
In the digital marketing ecosystem, confidence is a high-value currency. Posting a bold, prescriptive claim on LinkedIn generates engagement, attracts leads, and bolsters one’s personal brand. If the tactic later fails, the audience has already moved on to the next acronym. Conversely, the person who posts a skeptical correction faces social friction. They are labeled a "contrarian," and their substantive, methodology-heavy rebuttals are often buried in collapsed comment threads.
Furthermore, these consultants utilize a "jargon shield." When asked to explain the mechanism behind their "vector-space alignment," they respond with a string of technical-sounding terms—"T1 query optimization," "semantic retrieval"—that effectively silence anyone who lacks a deep background in machine learning. It is a form of gaslighting at an industry scale, where the framing of the conversation is designed to prevent anyone from asking, "Does this actually work?"
The "Monet" Effect: Framing Reality
This phenomenon mirrors a recent experiment on X, where a user posted a genuine Claude Monet painting but labeled it as "AI-generated." Hundreds of experts rushed to explain the "AI tells"—the flat brushwork, the lack of soul, the composition errors. They were analyzing a classic work of art, but their perception was entirely dictated by the frame provided by the original post.
The SEO industry has done the same with AI. By labeling a set of tasks as "Generative Engine Optimization," they have primed the market to see results where none exist. They have sold the performance of expertise rather than the expertise itself.
Implications for the Future of Search
The long-term cost of this charade is a massive depletion of professional credibility. When the C-suite eventually realizes that the "13% lift" was a hallucination, the practitioners who sold the dream will be the first to lose their seats at the table.
As traditional search continues to evolve into a generative experience, the need for actual technical literacy—not "hacks"—will become the defining differentiator. Marketing teams will eventually need to sit in rooms with engineering and data science teams who have spent years observing the field’s failure to call the technology correctly.
Knowledge advances through the rigorous attempt to disprove hypotheses, not through the creation of echo chambers designed to validate them. If the practitioners of the SEO/GEO industry do not start treating their own methodologies with the skepticism required of a scientific discipline, they will find themselves increasingly irrelevant.
Ultimately, the absence of data is the most telling data point of all. When a dominant industry prescription is contradicted by empirical study and official guidance, and yet continues to sell, we are no longer witnessing a market for expertise. We are witnessing an industry standing on the peak of "Mt. Stupid," charging clients a premium for the view, and praying that nobody looks down.
Dwi Wanna
