The OHGOD Philosophy
One Hobbyist GPU, One Day
Six Observations Worth Addressing
"The best way to predict the future is to invent it."
— Alan Kay
Six Observations Worth Addressing
We're not here to tear down. We're here to build. But building
well requires seeing clearly.
The GPU is consumer-grade. The method isn't.
industry.close()
A handful of companies control AI infrastructure while publishing nothing: no architectures, no training data, no failure modes. Concentration enables secrecy. Secrecy protects concentration.
industry.close()
- OpenAI Plans to Lose $5 Billion This Year Only deep-pocketed companies can sustain AI
- Foundation Model Transparency Index Stanford ranks AI companies on disclosure. Most fail
- U.S. Antitrust Inquiries of Nvidia, Microsoft, OpenAI DOJ/FTC investigate Big Tech AI market dominance
- GPT-4: 0% Architecture Disclosed Major release with zero technical transparency
planet.burn()
Training large language models consumes extraordinary energy. By 2027, AI could consume 85-134 TWh annually, equivalent to Argentina, Netherlands, or Sweden combined.
planet.burn()
- AI Adoption in US Adds ~900k Tons COâ‚‚ Annually 2025 study finds significant carbon impact
- Three New York Cities' Worth of Power: AI Is Stressing the Grid AI data centers demanding massive power capacity
- AI, but at What Cost? Carbon Footprint Breakdown Training 100B parameter model = 30h of 737 flight
- Google's Carbon Emissions Surge 50% Due to AI Data center electricity up 17% in 2024
privacy.null()
LLMs are trained on the internet: your posts, your writings, your conversations. Often without asking. Every interaction is a data-sharing operation.
privacy.null()
- Italy Fines OpenAI €15M After Privacy Probe GDPR violation: €15M fine for data collection
- Stack Overflow Users Deleting Answers After OpenAI Deal Users protest their content being sold to AI
- AI Companies Cause Most Traffic on Forums 4.8M Claude hits, 148k ChatGPT hits in 7 days
- Employees Feeding Sensitive Data to ChatGPT Raising security fears about corporate AI use
scale.fail()
$100B training runs are underway while an 8B model outperforms a 176B giant. The 'bitter lesson' is dying. Efficiency beats scale. Academia is priced out. The compute arms race is a dead end.
scale.fail()
- On the Slow Death of Scaling Sara Hooker: scaling alone misses critical shifts
- OpenAI, Google, Anthropic Struggling to Build More Advanced AI Scaling hitting walls across all major labs
- Nvidia's $589B DeepSeek Rout Efficient model triggers massive market shock
- 'We Can't Compete': Universities Losing AI Scientists Academia priced out of frontier research
replies.intelligent()
In the Middle Ages, the unexplained was God's work. Today, the unexplained is 'intelligence.' We just need to open the box.
replies.intelligent()
- Sycophancy Is the First LLM 'Dark Pattern' 100% of users affected by AI over-agreement
- Claude Says 'You're Absolutely Right!' About Everything LLMs agree even when users are wrong
- Hallucination Is Inevitable: Innate LLM Limitation Mathematical proof that LLMs must hallucinate
- NY Times Suit Wants OpenAI to Delete All GPT Instances Memorization enables copyright infringement claims
architecture.frozen()
The Transformer was invented in 2017. Eight years later, it still dominates, despite known limitations and promising alternatives.
architecture.frozen()
- 'Attention Is All You Need' Coauthor 'Sick' of Transformers After 8 years, even creators want alternatives
- DeepSeek-R1: Major Open Model Release New architecture challenges transformer dominance
- Meta Llama 3: Same Architecture, More Scale No major architectural changes from transformer
- The LLaMA Effect: Leak Sparked Open Source Alternatives Open weights enabling architectural experimentation
The Industry Is Closed
The Observation
A handful of companies control the compute, models, and data that power AI, and they tell us nothing about how it works. Training frontier models costs hundreds of millions. GPU computing flows through two or three companies. And what do we get in return? Stanford's Foundation Model Transparency Index (2025) scored major AI companies at just 41/100, down 17 points from 2024. OpenAI dropped 14 points. Meta dropped 29. Only 30% of companies even responded to transparency requests. This isn't conspiracy; it's economics. Concentration enables secrecy. Secrecy protects concentration.
"Transparency is a vital precondition for public accountability, scientific innovation, and effective governance."
— Stanford Foundation Model Transparency Index
Why This Matters
When only a few can build, only a few decide what gets built. When they won't show their work, we can't verify their claims. Research narrows. Innovation follows money. Science requires verification. Democracy requires transparency. We're building critical infrastructure on foundations we cannot examine, and 'trust us' isn't good enough.
What Could Be Possible
OHGOD!! What if powerful AI can be trained on a consumer GPU in 24 hours? The compute moat disappears. What if every architecture choice, training dataset, and failure mode is documented? No black boxes. No 'trust us.' You train it yourself, so you know what's inside. The industry scores 41/100 on transparency. We aim for 100.
The Planet Can't Afford This
The Observation
Training large language models consumes extraordinary amounts of energy. Training GPT-3 produced an estimated 502 tonnes of CO2. Every ChatGPT query uses roughly 15 times more energy than a Google search. Inference now accounts for more than half of total carbon footprint.
"Electricity demand from data centres worldwide is set to more than double by 2030 to around 945 terawatt-hours (TWh), slightly more than the entire electricity consumption of Japan today. AI will be the most significant driver of this increase, with electricity demand from AI-optimised data centres projected to more than quadruple by 2030. In the United States, power consumption by data centres is on course to account for almost half of the growth in electricity demand between now and 2030."
— IEA Energy and AI Report (2025)
Why This Matters
We love AI. We think it can do tremendous good. But we refuse to ignore the costs. Climate change is real. Energy consumption matters. And 'move fast and break things' isn't a philosophy that scales to planetary systems.
What Could Be Possible
OHGOD!! What if we could build efficiency into these foundation models? Training on a single GPU in 24 hours under someone's desk might mean people can take care of their inference needs right from home, with their own electricity.
Your Data Isn't Yours Anymore
The Observation
Researchers proved transformer LLMs are mathematically invertible. Hidden states aren't abstractions, they're your prompt in disguise. OWASP 2025 lists vector embedding weaknesses as a top vulnerability. Embedding inversion attacks recover names, diagnoses, and financials with 90%+ accuracy. 'Zero retention' is theater. The New York Times lawsuit forced OpenAI to hand over all logs anyway.
"You touch that thing with AI, your one sensitive document is like 5x-ed. Training sets, search indices, prompts, the model, the logs. And those other places? No one's paying attention to."
— Patrick Walsh, DEF CON 2024 - Exploiting Shadow Data in AI Models
Why This Matters
Privacy isn't about having something to hide. It's about the right to think, explore, and make mistakes without being recorded, analyzed, and subpoenaed. Every cloud AI interaction is a data-sharing operation you can't take back.
What Could Be Possible
OHGOD!! What if your data never had to leave your device? What if the model just ran locally? Your thoughts stay with yourself. No embeddings to invert. No need to be afraid to ask silly questions.
Scaling Is Failing
The Observation
The AI industry has bet everything on a simple formula: more compute, more data, bigger models. $100B training runs are underway. Sam Altman expects to spend 'trillions' on infrastructure. Nuclear reactors are being restarted to power data centers. Yet smaller models now routinely outperform giants. Aya 8B beats BLOOM 176B with just 4.5% of the parameters. OpenAI, Google, and Anthropic are all struggling to build more advanced AI. The relationship between compute and performance is increasingly strained. Academia has been priced out entirely. We've reoriented our entire field around 'bigger is better.' Is it?
"We can now stray from the beaten path of boring, predictable gains from throwing compute at the problem."
— Sara Hooker, On the Slow Death of Scaling (2025)
Why This Matters
When scaling is the only path, only those with billions can participate. Research narrows. Innovation follows capital. Universities become irrelevant. But if scaling is failing, the billions being poured into data centers may be a colossal misallocation of resources, while more efficient approaches go unexplored.
What Could Be Possible
OHGOD!! What if the path forward isn't more compute, but smarter compute? What if a consumer GPU and 24 hours is enough? The scaling paradigm assumes resources are the constraint. We assume architecture and efficiency are. DeepSeek showed efficiency can beat scale. We aim to prove a single hobbyist GPU can too.
The Intelligence of the Gaps
The Observation
Medieval scholars attributed the unexplained to God; we attribute impressive LLM outputs to 'intelligence.' But for transformers, we actually know a lot: induction heads drive in-context learning, FFN layers store facts as key-value memories, sparse autoencoders extract interpretable features, circuit tracing maps decision pathways. Hallucinations, sycophancy, memorization; these aren't mysterious emergent properties. They're statistical pattern-matching we should examine more.
"Just as 'God of the gaps' shrinks as science advances, our definition of 'intelligence' keeps shifting. There's a running joke in AI: intelligence is whatever computers haven't done yet."
— Marcel Salathé, 'AI and the God of the Gaps' (2024)
Why This Matters
72% of users trust AI for facts. 75% of those get misled. Hallucinations, sycophancy, memorization; these aren't mysteries. They're predictable outcomes we could fix with more experimenting.
What Could Be Possible
OHGOD!! What if 'intelligence' is just a label for what we haven't examined yet? Build small enough to train yourself, and you can call it what it actually is.
The Architecture Barely Changes
The Observation
The Transformer was invented in 2017. Eight years later, it still dominates, despite known limitations and promising alternatives. LLaMA-3 made no major architectural changes. Innovation requires experimentation, and experimentation requires affordable compute.
"Despite many small and creative innovations since the original transformer architecture, there have not been any significant 'breakthrough' discoveries that have led to much better leaderboard results."
— arXiv Survey (2024)
Why This Matters
If the Transformer isn't optimal (and evidence suggests it may not be), then we're investing billions scaling a suboptimal design. Mamba offers 5x throughput. RWKV proves RNNs can compete. Griffin matches Llama-2 on 6x fewer tokens.
What Could Be Possible
OHGOD!! What if there's a better architecture waiting to be found? If you can train something in a day on a consumer GPU with your own evaluation, maybe you'll find it.
The Experiments We Propose
The Hypothesis
There exists at least one architecture that achieves meaningful language model performance when trained on a single consumer GPU within 24 hours at the 200-500M parameter scale.
This is a hypothesis, not a promise. We don't know if it's true. That's why we're running the experiments.
Proposed Experimental Setup
Hardware Constraint
Training Constraint
If the Hypothesis Holds, It Implies:
Anyone can do AI research
Meaningful AI with minimal energy
Personal AI that stays personal
Fully transparent, verifiable models
Mechanisms over mysticism—AI we dissect
Architecture experiments anyone can run
The Evidence That Motivates Us
The Cramming Precedent
Geiping & Goldstein (2022)"How far can we get with a single GPU in just one day?"
Their answer for BERT: remarkably far. OHGOD extends this inquiry to decoder-only models—the architecture that powers modern chatbots.
SmolLM2: Already Working at Target Scale
Hugging FaceModels at our target size can perform. The question: can we get meaningful results with 1/40th to 1/80th of the training compute?
| Model | Params | Tokens | HellaSwag |
|---|---|---|---|
| SmolLM2-135M | 135M | 2T | 42.1% |
| SmolLM2-360M | 360M | 4T | 54.5% |
Gemma 3 270M: Extreme Overtraining Works
Google (2025)Google trained a 270M parameter model on 6 trillion tokens—1,111x the "optimal" ratio. It works. The relationship between model size and capability is more flexible than the scaling orthodoxy suggests.
The Call to Action
To Researchers
Question the scaling orthodoxy. Prove what's possible with constraints.
The assumption that intelligence requires scale is testable—not axiomatic. One GPU. One day. What can you build? The OHGOD constraint isn't a limitation—it's a research methodology that forces innovation over brute force.
Investigate:
- • Alternative architectures beyond transformers
- • Training dynamics under compute constraints
- • Memorization vs. generalization metrics
Explore:
- • Data efficiency and smart curation strategies
- • Benchmarks that measure understanding, not recall
- • Training techniques for consumer hardware
To Users
Demand AI that respects your autonomy.
You should not have to send personal data to remote servers, trust corporations with your conversations, or accept opacity as the price of capability.
- • Support open-source projects
- • Demand transparency
- • Reject cloud dependency
To Everyone
The democratization of AI is not inevitable. It must be fought for.
The current trajectory leads to a world where AI capability is a corporate monopoly—licensed, monitored, and controlled by entities whose interests diverge from yours.
OHGOD proposes an alternative: AI that belongs to individuals. AI that runs on your hardware. AI you can inspect, modify, and trust.
Our Commitments
1 Radical Transparency
- • All code will be released (open source)
- • All weights will be released (Hugging Face)
- • All training logs will be public
- • All technical information will be public
2 Reproducibility
- • Anyone with consumer hardware can verify our claims
- • No proprietary data
- • No secret sauce
- • No "trust us"
3 Honest Evaluation
- • Measuring what matters—understanding, not just benchmarks
- • Reporting what we find, not what we hoped to find
- • Pre-registered hypotheses
4 Open Science
- • Public amendment logs
- • Negative results published with same rigor
If the experiments fail, we'll document exactly how and why. That's what makes this science, not marketing.