FRONTIER SIGNAL | DEEP DIVE | AI & INVESTING | 7 MIN READ | FRIDAY, 30 OCTOBER 2026
Halcyon skips the giant language-model stage most image generators inherit. The paper claims 275 times less pretraining compute. The real finding is smaller, and more useful: much of that bill was never about pictures.
Most AI image generators learn to draw after someone else has taught a model to read. That reading lesson is expensive. One comparison model here carries 36 trillion tokens of text pretraining in its compute bill.
Halcyon, from Stanford and Radical Numerics, asks a blunt question: what if an image model never goes to language school? It trains text and image tokens together, from scratch, on about 100 million permissively licensed photographs.
The headline is a 275x saving. Read the footnote and the story changes. It is not that pictures became 275 times cheaper to make. It is that a large share of what the field counts as “image cost” was really text cost.
275X LESS PRETRAINING COMPUTE · 1.6B PARAMETERS · 256-PIXEL IMAGES
Halcyon-1.6B versus the authors’ own JiT-T2I-XXL/8++ baseline, counting total pretraining compute including language pretraining. Quality is measured on the authors’ own test set. These are the authors’ numbers, at small image size.
The investment question is whether cheap, clean-data base models change who gets paid in image generation.
Key takeaways
A different recipe. Halcyon is one transformer that reads and writes text and image tokens in a single sequence, trained from random start with no language-model head start.
The 275x is real arithmetic, with a catch. The baseline’s compute is 99.8% language pretraining. The paper reports no GPU hours or dollars for Halcyon itself.
Quality is respectable, not frontier. Halcyon-1.6B beats the authors’ strongest baseline on their own metric. Nothing is compared against production tools such as FLUX or Midjourney.
The clean-data result matters. Training used only images licensed CC BY, CC0 or public domain, about 101 million after deduplication.
Listed winners are not obvious. The savings come from skipping text pretraining at small scale. That is not evidence that GPU demand falls.
Here is what happened
The paper. “Text-to-Image Generative Modeling Without Language Pretraining” was submitted to arXiv on 19 October 2026 (arXiv 2610.26863). Authors are from Stanford, including Fei-Fei Li, and Radical Numerics, which provided the compute.
The family. Five sizes, 0.10 to 4 billion parameters, each trained for one pass at 256 by 256 pixels.
The claim. The abstract says Halcyon-1.6B outperforms JiT-T2I-XXL/8++ (35.3 versus 41.0 on the generation distance score, lower is better) “with 275x less pretraining compute.”
Openness. The authors point to a project page for code and models. We could not open it and found no stated model licence.
How it works
One sequence, two kinds of token. Text is split into 16,384 word pieces. An image tokenizer turns a 256 by 256 picture into 1,024 image tokens. The model predicts the next token, whichever kind it is. It is a decoder-only transformer, not a diffusion model.
No borrowed brain. Most systems start from a pretrained language model or a frozen text encoder. Halcyon starts from random numbers and sees only the captions in its image data: about 4.5 billion text tokens and 102.4 billion image tokens. For another model that skipped human writing altogether, see an AI that learned physics instead of language.
Clean, captioned data. The training set, called GPIC, holds 110.6 million photographs from Flickr and Wikimedia, filtered to permissive licences. Captions were written by a vision-language model, not scraped alt text.
A better tokenizer did a lot of the work. Swapping in a stronger image tokenizer cut one baseline’s score from 147 to 63. Part of the gain comes from how pictures are chopped up, not from skipping the reading lesson.
THE SAME QUESTION, ASKED ABOUT WORDS
Halcyon asks whether an image model needs a language model first. Ai2 asked a neighbouring question about chatbots: do they need word-chunks at all? An AI lab taught chatbots to read letters instead of word-chunks, and the result helped on letter-level tasks while still trailing its parent model on most standard tests.
What the 275x really counts
The ratio. It uses total pretraining compute. The paper’s table puts the JiT-T2I-XXL/8++ baseline at 3.68 x 10^23 operations, of which 99.82% is language pretraining. Halcyon-0.52B needs 5.33 x 10^20. Halcyon-4B needs 3.08 x 10^21.
An older baseline. Against LlamaGen-XL/8 (9.68 x 10^21), the saving is about 18 times, and that one also includes a large text encoder. So the fair headline is “much cheaper once you stop paying for language,” not “275 times cheaper to train an image model.”
The interesting result. The paper also tested whether language pretraining helps. At 0.52 billion parameters, starting from a pretrained Qwen3-0.6B did not help (47.8 versus 46.6 for random start, lower is better). Mixing in text-only data made results worse.
How good is it?
Image realism. Halcyon-1.6B scores 35.3 on the authors’ test set, against 41.0 for the strongest baseline and 63.3 for LlamaGen-XL/8.
Following the prompt. On GenEval, 1.6B scores 53.5, against 43.2 for the strongest baseline. On DPG-Bench, 77.9 against 76.85.
Where it is flat or weak. The tiny 0.10B model is no better than LlamaGen-XL. A GenEval score near 54 is low by the standards of current commercial tools.
What is missing. No comparison against production models, no human preference test, and no independent replication that we could find.
Who captures the value
Nvidia (NASDAQ: NVDA) is not threatened by this paper, whatever the 275x suggests. Fiscal second-quarter revenue was $96.2 billion, up 106%, with Data Center at $89.0 billion, up 117%, and guidance of $108 billion for the next quarter. The savings come from dropping text pretraining on small models. Cheaper training usually expands the number of models trained rather than shrinking the chip order. That is inference, not the paper’s finding.
Alphabet (NASDAQ: GOOGL; NASDAQ: GOOG) and CoreWeave (NASDAQ: CRWV) sell the compute these models run on. Google Cloud revenue grew 82% to $24.8 billion in the second quarter, and 2026 capital spending guidance is $195 to $205 billion. CoreWeave reported second-quarter revenue of $2.58 billion against $1.21 billion a year earlier, with a backlog near $104 billion. A recipe that lets smaller teams train capable image models widens the customer list for both. Google is attacking AI cost from the other end: a cheaper way for Gemini to watch video.
Adobe (NASDAQ: ADBE) is the workflow layer. Third-quarter revenue was $6.76 billion, up 13%, with AI-first annual recurring revenue up more than 150%. Cheaper base models help Adobe’s costs, and they also make any one model easier to replace. Adobe’s defence is control, rights and production tools, not model quality. Our piece on which software survives the agent era makes the same argument.
Shutterstock (NYSE: SSTK) and Getty Images (NYSE: GETY) are the sharper read-through. Shutterstock’s second-quarter revenue fell about 17% to $221.8 million, and Getty’s fell 2.5% to $229.1 million, after their merger was terminated. Halcyon suggests a useful base model can be built from freely licensed images alone. That weakens the pitch that licensed stock libraries are essential training fuel. It also strengthens rights-cleared positioning: provably clean data now has a working recipe.
The private-company watchlist
An industry read-through only; none is shown to use Halcyon’s method.
Black Forest Labs raised $300 million at a $3.25 billion valuation in December 2025. Its edge is higher-resolution quality, untested here. Stability AI reported a $76 million round in August 2026, with Universal Music Group, Warner Music Group, Sony Music Group and Electronic Arts as new investors. If clean-data recipes spread, open-model firms with rights-friendly stories gain the most.
The honest catch
The headline ratio flatters. It compares totals that include text pretraining the baseline never needed for pictures. The paper gives no direct image-only cost.
Small and in-house. All results are at 256 pixels, on a test set the authors built from their own data, with baselines the authors appear to have retrained themselves.
Photographs only. Flickr and Wikimedia are thin on illustration, brands and text in images, and discrete tokenizers handle fine glyphs poorly.
Scale is unproven. The authors say findings from the 0.52B tests may not carry to larger models, and they tested only autoregressive models, not diffusion.
Source limits. We read the arXiv paper and the dataset page. We could not open the project page, so the model licence and checkpoint list are unconfirmed.
The one question every reader should ask
What happens at 1,024 pixels, with art, brands and text, against a production model?
A recipe that works at 256 pixels on photographs is a research result. Holding up at the resolution customers pay for would change the category.
OUR VIEW
Halcyon is a clean, careful paper with an overstated headline number. The solid finding is that language pretraining did not help in these tests, and that permissively licensed images were enough for a respectable base model.
For investors, this is a signal about data and rights, not a signal about GPU demand. Stock-image licensors face a weaker story. Adobe and the cloud providers are not hurt by it. Treat the 275x as a talking point, not a trend.
What to watch next
Independent replication. Do other labs reproduce the scores?
Higher resolution. Does the recipe hold at 512 and 1,024 pixels?
A named licence. Under what terms can companies use the models?
Image-only compute. Will anyone publish a like-for-like training cost?
Data licensors. Do Getty and Shutterstock change their licensing pitch toward rights-cleared training sets?
Quick questions
What is Halcyon?
A family of text-to-image models from Stanford and Radical Numerics, in five sizes from 0.10 to 4 billion parameters. It is a single transformer trained from scratch on text and image tokens, with no language-model pretraining.
Is it really 275 times cheaper?
Not in the way the headline reads. The 275x compares total pretraining compute for Halcyon-1.6B against a baseline whose compute is 99.8% language pretraining. The paper gives no GPU hours or dollar cost for Halcyon.
Is it better than Midjourney or FLUX?
Unknown. The paper compares only against research baselines on the authors’ own test set at 256 pixels. Its prompt-following scores are low next to current commercial tools.
What data did it use?
About 101 million images from Flickr and Wikimedia, filtered to CC BY, CC0 and public-domain licences, with captions written by a vision-language model.
Does this threaten Nvidia?
Not on this evidence. The saving comes from skipping text pretraining on small models.
READ MORE: AI SIGNALS
An AI Lab Taught Chatbots to Read Letters Instead of Word-Chunks · 16 Oct 2026 · Retrofitting a language model to read raw bytes, and where it still trails.
Google’s Gemini Found a Cheaper Way to Watch the World · 7 Sep 2026 · A cheaper way to analyse video, and the footage it may skip.
Featured: What Survives the Agent Era · 3 Sep 2026 · Which software keeps its value when AI agents arrive.
Every AI You Know Learned the World From Us. This One Did Not · 3 Sep 2026 · An AI trained on physics instead of human writing, and why its claims are unverified.
Sources
Halcyon paper (arXiv 2610.26863): results and compute tables.
Hugging Face paper page: abstract and listing.
GPIC dataset page: licence terms and scale.
GPIC paper (arXiv 2605.30341): how the corpus was built.
Nvidia fiscal Q2 2027 results (SEC filing): revenue and guidance.
Alphabet Q2 2026 results: Cloud revenue and capex guidance.
CoreWeave Q2 2026 results coverage: revenue and backlog.
Adobe Q3 fiscal 2026 results (SEC filing): revenue and AI-first ARR.
Shutterstock Q2 2026 results (SEC filing): revenue and guidance.
Stability AI funding announcement: the August round.
Frontier Signal explains frontier technology in plain English. Figures are from the arXiv paper and company releases; benchmark results are the authors’ own and unreplicated. This is general information, not investment advice.