One prompt. Five models.
Five takes on an agentic IDE.
Yarn is an Ultra.dev build experiment: give five coding models the same open-ended brief for an Ollama-focused desktop IDE, then compare the products, the follow-up they needed, and the cost of getting there.
The full comparison
The long-form video covers the products, the repair turns, the elapsed time, and the trade-offs that code screenshots alone cannot show.
Watch on YouTube ↗The experiment
The goal was not a synthetic benchmark. It was to see what each model would invent when the product decisions were left open, then judge whether the result was coherent, functional, and worth using.
THE BRIEF “Build an Ollama super app: a native, agentic development workspace that makes open-weight models more useful for real software work.”
Every model received the same open-ended product brief.
No detailed design spec and no extra tools were added to the environment.
The comparison looked at usability, follow-up effort, speed, and completeness.
Inside Yarn
The strongest builds treated the agent, editor, terminal, Git state, and model context as one continuous workspace.
The results
These are observations from the recorded runs and their follow-up fixes. This is a build experiment, not a controlled benchmark.
The most complete interpretation
A refined mission desk, clearer code and review workflow, a model library, and strong defaults. It produced the best overall product, although the improvement over GPT 5.6 Sol was incremental.
The best fit for this prompt
A polished mission-based workspace with editing, Git diffs, terminal access, model context, and visible agent actions. It needed very little follow-up and delivered the strongest balance of quality and efficiency.
Capable, elaborate, and expensive
Threads, acceptance criteria, a Sharpen action, model roles, project isolation, and replay made this the most opinionated workflow. It worked, but required UX fixes and heavy token use to reach the finish line.
Lightning fast with major gaps
It assembled a recognizable IDE with files, an agent view, models, and source control in roughly 14 minutes. Missing Git diff inspection and other functional gaps kept the result from being a usable daily tool.
Opened, but core workflows stayed broken
The app could open and load files, but key interactions still failed after three attempts to repair it. The run is included because failed products are part of the experiment, not something to edit out of the story.
All five builds are public
Every implementation lives in one MIT-licensed repository, with its own frontend, Rust backend, dependencies, and setup guide.