A 2022 article in Noema magazine describes apophenia as "faulty pattern recognition. People see faces in clouds and alien ruins on Mars. ... Humans are pattern-recognizing creatures, and so apophenia is built in."
A Google engineer had claimed an LLM to be conscious, sentient, and indeed a person, and the article explores reasons for why he had reached such a conclusion. It's well worth a read, particularly its observation that we apply concepts that have wholly anthropomorphic heritage and meaning to LLM technology. Pehaps we need new vocabulary.
The technology's stochastic prowess is much advanced 42 months later, and perhaps that's all the more reason we continue to see things that aren't there. Matt Shumer's "Something Big Is Happening" is the latest example.
Shumer argues that we are witnessing a phase transition: a shift from conversational AI to genuine autonomous agency. His claim is that large language models have crossed a threshold insomuch as they can now perceive and act within digital environments, execute multi-step workflows, and operate for extended durations with minimal supervision. From this, he infers an accelerating feedback loop in which AI systems increasingly contribute to building their successors.
However, this thesis rests on a crucial and under-examined assumption*: that the behaviours we observe amount to reasoning in any substantive sense. What we call “reasoning” in contemporary LLMs is better understood as high-dimensional probabilistic pattern completion. When I asked a frontier model to comment on that last sentence it describes its like as massive indexical over-achievers!
These systems do not construct causal world models or maintain stable internal representations, but rather simulate reasoning traces because they have learned the statistical structure of reasoning-like language. We see reasoning-like and conflate it for reasoning. Apophenia.
This limitation constrains the recursive acceleration Shumer anticipates. Flattens it. For an intelligence explosion to occur, each generation must meaningfully improve its ability to design the next. That requires stable abstraction and reliable long-horizon planning — capacities that stochastic next-token predictors approximate only within narrow distributional bounds. Without genuine reasoning mechanisms, the transition remains one of engineering orchestration rather than emergent autonomous cognition.
This is where supersoftware steps up.
By being software that is itself symbolic AI, supersoftware offers a neurosymbolic synergy that mitigates the inherent instability of pure LLM agents. It ensures that the agent’s operational logic is not merely a fleeting statistical inference but is governed by a persistent, interpretable, and — vitally — evolutionary symbolic structure. This eliminates the need for the external paraphernalia of tool wrappers and textual guardrails (i.e. implemented within the same probabilistic medium they aim to constrain), achieving governance of the requisite complexity with architectural simplicity.
Objectively, the "Something Big" Shumer senses is the birth of the General Purpose Digital Agent. But for these agents to achieve the reliability required for the tasks he mentions, they require ontological stability. Supersoftware provides this by allowing stakeholders to bake reflective symbolic logic directly into the software the agents inhabit. Within this framework, the LLM acts as a non-sovereign co-architect — a valued team member in a collaboration of symbolic, neural, and human intelligences.
Supersoftware acts as the governance and reasoning layer for the agentic explosion, bridging the gap between phenomenal non-deterministic leaps of pattern-matching and the deterministic demands of any software application. Something big could be happening.
* unless you're Gary Marcus of course :-)
Image by Nano Banana Pro.
