A Path Towards AGI?

Samuel Troyer••2 min read

While we still don’t fully grasp intelligence, concept models & step-reasoning may let us simulate it.

Samuel Joseph Troyer | January 6, 2025

Transformers can’t obtain a conceptual understanding of their environment. Therefore, they cannot achieve AGI—let me explain …

language, for a human, is one way of describing our perception of the world
language, for transformers, is their world

During the Sandinista Revolution in Nicaragua, deaf children placed in separate schools spontaneously created an entirely new sign language. This illustrates how semantic language is a tool we use to map meaning to our environment, but the language itself is not really all that important – it’s the higher-order conceptual meaning that is valuable.

The Limits of Today’s Transformers

Tokenization turns language into the environment for a transformer. Tokens exhibit rich statistical relationships that let models predict plausible next tokens, yet those predictions are made without any genuine conceptual grasp of the input.

Concept Models: The Next Frontier

Meta’s new LCM line attacks this limitation by mapping language into higher-order semantic “concepts.” If successful, it resolves a core contradiction in transformer architecture and lets models reason over meaning rather than surface statistics.

Facebook’s SONAR embedding space is an early attempt at true concept modelling. Better embedding spaces (and zero-shot concept learning) remain open research challenges, and opportunities.

Step-Reasoning & Human-Level Parity

By applying step-reasoning to conceptual representations—rather than using reasoning as a mere reward mechanism, models like OpenAI’s o1 & o3 edge closer to human-like cognition.

Conceptual step-reasoning represents near-parity with human intelligence.

I’m convinced Sam Altman’s recent claim that he “knows how to build AGI” refers to advances in this very vein.

But What Is Intelligence?

As François Chollet (creator of Keras and ARC-AGI) often highlights, “AGI” remains largely a pedantic label. We keep modelling intelligence by observation, not by understanding.

Consider last year’s breakthroughs on quantum-scale neuron tunnelling or decoding neuronal transmissions. Such research could upend everything we think we know about activation, sequencing, and brain architecture.

I believe quantum mechanics is the final frontier for AGI, as it represents the ability to truly understand consciousness; and subsequently, potentially build machines that could properly model it.

Looking Ahead

I hypothesize we will get, in the next 1-3 years, some sort of generalized business or consumer intelligence; which will obviously be revolutionary for other reasons.

But those of us transfixed by intelligence, we will continue to dig deeper!