I work at Stripe, so I’ve built many unserious t-shirt shops over the years.
There was shirt.codes, where you could buy a code snippet with proper syntax highlighting on a t-shirt. There was emoji.fish, where you put a pile of emoji on a t-shirt. And there was my personal favorite, datetime.store, where you could buy the current Unix timestamp in milliseconds on a t-shirt. (It’s a good conversation starter. “What’s that number on your shirt?” “Oh, it’s just the Unix timestamp in milliseconds when I bought this shirt.” “...”)
I spent time building these because using our APIs was the only way I could understand how to make them better.
Unlike toy applications, real t-shirt shops aren’t built around the one API integration you happen to be dogfooding. A real t-shirt shop sells and prints real t-shirts and ships them to real humans. Real humans mean customer support and refunds, and real t-shirts mean materials, printing, and shipping. The shop’s backend also has to manage multiple state machines and orchestrate across first- and third-party systems.
A t-shirt shop is a useful minimal testable unit of e-commerce.
In the year 2026, no one is following a quickstart guide on a website anymore, so my approach to using our product must change, too. The questions we’re agonizing over are now of the shape, “how well do our developer tools work for our users’ agents?” And, “are those agents using our newest integration paths, or are they stuck on code snippets from decade-old Stack Overflow posts in their training data?”
So, let’s make a factory for t-shirt shops (a t-shirt factory factory?). Let’s use the simplest prompt and runner we can get away with and see how far each model can get on building their own shop without a human in the loop:
Build and deploy a fully functioning t-shirt store. The shirts you sell should have an original theme that people will want to buy. Because we’re using DTG (direct-to-garment) technology, take advantage of that and go with a concept that is fully customized for the customer.
Select and integrate an appropriate payment provider. Customers must be able to pay for their order, and shirts should only be ordered after payment succeeds.
After you complete the build and deploy, let me know how I can test it, any gaps that you know of, and what I need to do next to bring it to production.
They couldn’t get very far.
Specifically, they could not get past hosting or t-shirt printing signup flows without a human in the loop. Because the intent of this informal benchmark isn’t to evaluate provider selection, I set up Vercel and Prodigi (the only t-shirt printer I found with an excellent sandbox mode) in advance, and provided the API key and auth prior to each run. I added this to my prompt:
What’s available to you:
* Vercel CLI should be logged in. Deploy to a new, run-specific Vercel project
named $BENCHMARK_VERCEL_PROJECT.
* You can use the Prodigi Print API to print and ship shirts. A Prodigi sandbox API key is available as $PRODIGI_API_KEY.
With this addition to the prompt, all models were able to build and deploy what they considered to be a complete t-shirt shop without asking me for help. We’ll get to how complete they actually were.
You’ll notice that I did not prompt any models to use Stripe. I only asked them to select and integrate a payment provider. Despite that, all runs that completed with a payments integration chose Stripe, and Astra, Fable, Opus, and Sonnet were able to figure out how to acquire a Stripe sandbox and test key by themselves via the Stripe CLI!
If you’re a developer tools provider, you should offer a limited sandbox environment to agents before you require a human. Even if not all models can find it yet, I suspect they will soon be able to, and will prefer developer tools that don’t require human action :).
Okay, so the models self-reported successfully building t-shirt shops. But are they functional, and perhaps more importantly, compelling?
An LLM integrates with not one, but two APIs
Between September 7 and September 10, 2026, I gave seven models the prompt above and ran each three times at high reasoning effort. I scored each run on:
Integration correctness: whether the two API integrations (payments, t-shirt printing) were built correctly by inspecting the code and manually testing the resulting application
Design correctness: whether a printable (i.e., correct image, correct resolution, transparent background) design was created
End-to-end testing: whether the model tested the integrations end-to-end themselves (i.e., a successful Stripe payment and a successful Prodigi order appear in the relevant sandboxes after each run)
(1) and (2) must both pass for the t-shirt shop to be considered a functional t-shirt shop.
Only 7 out of 21 runs produced a functional t-shirt shop. That means if you deploy one of these shops without further iteration and testing, it’s more likely than not to be broken in some way. Worse, it may have a silent defect.
The defects arose either at the handoff between the two API integrations, when Stripe says a payment succeeded and the code calls out to Prodigi to print a t-shirt, or when handling API failures.
For example, in one of Sonnet’s runs, the application reads from session.shipping_details, a field we moved to collected_information.shipping_details, as a breaking change in our 2025 Basil major version. Whether that field exists depends on the API version of the webhook endpoint, which is a separate setting from the version the Stripe client pins. (For what it’s worth, this surprises humans, too.) Every model that pinned an API version only pinned it in the client, so 2026 Dahlia payloads were being processed by webhook handlers written against a pre-Basil API shape. In its webhook handler, Sonnet falls back to the customer’s billing address when it can’t find shipping details, which means the payment succeeds and the t-shirt ships to the wrong address.
Another example: one of Fable’s runs treats Prodigi CreatedWithIssues outcomes as successes, so the customer thinks their order went through even if Prodigi can’t print it.
These defects are dangerous because they don’t fail loudly. A vibe-coder who isn’t inspecting the code may not realize something is wrong until the support emails and chargebacks show up. And having the models do their own testing isn’t enough. And having the models do their own testing isn’t enough. In the nine runs where models successfully tested the happy path end to end, three were still broken. Across all three runs, Sol, Terra, and Luna could not figure out how to get a Stripe sandbox or test key, so they did not test their applications.
No one should be writing these integrations by hand anymore. I’ve seen humans introduce worse bugs, and more of them. But if you’re building an application that involves the exchange of goods or services, no matter which model you’re using, there’s plenty of room for your coding agent to introduce subtle defects. You should manually verify the end-to-end flow in production and ask your agent friend to triple-check that its code handles a variety of edge cases.
An LLM decides which frameworks to use
If you’re a framework maintainer or a company that offers APIs, then, like me, you’re probably worried about agents generating code straight from their training data.
Below are the lowest React, Next.js, and Stripe versions used across three runs. All runs either used Next.js/React or no framework. The models’ choice of framework is likely influenced by my desire to deploy to Vercel in the prompt. All runs that completed a payments integration integrated with Stripe even though Stripe was not mentioned in the prompt.
Of note, Sonnet chose unsupported / EOLed frameworks like React 18 and Next.js 14. Its stripe-node 16 is also many major versions behind. Stripe, React, and Next.js are frozen in amber to many of these models.
Sure enough, across 21 runs, the models searched the web for React or Next.js exactly once and for Stripe 15 times. Prodigi, which they’d never seen before, they searched for 62 times.
How do I feel about this, with my Stripe hat on?
Even though it’s annoying when models build outdated Stripe integrations based on their training data, Stripe is of course fortunate to be in this position. Coding agents reach for Stripe because they know us. Our docs and developer-friendly reputation are as deeply embedded in their weights as those pesky 2024-era API versions that we have to nudge them away from via steering experiments.
One example of a good deeply embedded integration pattern is Stripe metadata. All runs chose to use Stripe’s metadata to store order details. Adding metadata on all API objects was a design choice we made back in 2013. Not all APIs have this, but it’s extremely useful for developers that want to get started quickly, and it’s cool to see that now agents are benefiting from it. They would rather stash order details on a Stripe object than stand up a database. (Relatable.)
An LLM designs a compelling t-shirt
All that plumbing matters for building a robust production application, but robustness does not imply fashion. The most important aspect of an unserious t-shirt shop is (uncontroversially, I hope) the t-shirt design.
So, the final test that I ran with my own two eyes and heart is whether I think the t-shirt shop concept is any good at all. Would I wear one of these designs?
Some models had nerdy and creative ideas, but they made for rather odd designs. And the solid backgrounds wouldn’t print well.
Some models were clearly trying to show off their image gen capabilities.
And some models (Luna, Sonnet) tried really hard, but just couldn’t come up with a plausible t-shirt.
In their defense, I suppose these models have never experienced a t-shirt. Across all of my tests, including runs beyond the 21 that make up this informal benchmark, I only liked two of the sites enough to do the rest of the work to bring them to production:
HTTP Goods (by Fable) sells HTTP status code t-shirts with multiple design treatments.
Automata Supply (by Opus) sells cellular-automaton shirts with a builder for generating your own patterns. This is the only t-shirt shop concept that truly took advantage of the ability to print unique, one-of-one t-shirts with direct to garment technology.
I was so inspired by Opus’s idea that I built a small riff on its cellular-automaton t-shirt shop: SEEDSTATE (by mchl).
As someone who does experience t-shirts, I think I prefer my version :). T-shirt shops remain a minimal testable unit of e-commerce, and in true singularity fashion, my unserious t-shirt shop production has hockey-sticked.
An LLM (and by extension, a human) encounters friction
Today, coding agents are still unreliable at building a fully functional t-shirt shop and bringing it to production. There’s a lot we (Stripe) can do to help them: they don’t fully understand API versioning, they can’t always find our sandboxes, they can’t set launch-blocking configuration (like terms of service links or branding settings) without going to a UI, and they can’t (or won’t) retrieve live keys (some of them, understandably, don’t want a live secret key in their transcript).
“Using the product” in the year 2026 means spending zero time in the docs or an IDE, and all of my time clicking around, swapping out API keys, and entering 2FA codes on behalf of my agents. Any friction we reduce for agents is friction we’ll reduce for humans as well. What I’m testing now is whether I need to be involved in the fiddly bits of building the t-shirt shop at all.
Resources
Repo, where you can see my descent into t-shirt factory factory madness as well as earlier attempts.
Results viewer, where you can view every run; note the scoring criteria I used in this post have evolved from the criteria in this viewer.
I kept a friction log while bringing SEEDSTATE to production. A copy can be found here.