Paper, demo & testnet only

Best AI model for building trading bots? Our test results

We handed BotCookery recipes to four AI coding models and had each one build the bot, write tests and run it against live public market data in paper mode. Here's what passed, what broke and what we fixed.

The short answer

Every model got there, and none of them got there from the prompt alone. All 21 builds finished with a working bot and passing tests. 20 of them needed at least one fix to something the recipe got wrong or left out: a field name, a required order parameter, an endpoint, or an API quirk the docs don't mention. We fold every one of those fixes back into the recipe, so the prompt you copy today is better than the one the models started from.

These aren't benchmark scores. Each model built different recipes, so treat this as a field log, not a leaderboard.

How we test

• Each model gets the recipe prompt and builds the bot in its own folder.
• Market-data calls run live against public endpoints (Kalshi, Polymarket US, Polymarket international, Hyperliquid).
• Orders, balances and cancel-all run in paper mode, against a mock, or on demo or testnet. No real keys, orders or money.
• Paid LLM calls inside a bot run through a deterministic stub.
• Each run ends with a test report: PASS, PASS WITH FIXES or FAIL, plus every fix.

Claude Sonnet 5.5 (Claude Code, 2026-10-19)

Recipe	Result	What it fixed
Build a Polymarket Bot with Claude Code (Paper Mode + Guardrails) 41 tests passing	PASS WITH FIXES	Added the required manualOrderIndicator; local idempotency because there is no client order id (never retry POST /v1/orders); preview body is {"request": order}; signing message and key format spelled out; closed=false when listing; book uses bids/offers; some markets quote under the documented 0.01 floor; fee formula with banker's rounding.
Claude Trading Bot 17 tests passing	PASS WITH FIXES	Order body was missing the required side; signing path must be the full /trade-api/v2/... path; both RSA-PSS and Ed25519 keys exist; named the market endpoint (GET /markets/{ticker}, yes_ask_dollars); most demo markets have no ask, so use mve_filter=exclude&limit=1000; defined dry-run without keys.

Ran live: Live public Polymarket US and Kalshi demo data; the paper loop ran on live public data.

Not run live: No authenticated calls. Signed order, cancel, preview, positions and balance calls ran against a fake HTTP session, with signatures verified against the public key.

Biggest lessons:

• Prompts relied on the Safety Pack without including it, so the model wrote a stand-in rules file.
• The harness refused writes under .claude/; the deny rules were saved as an example file instead.

Composer 2.5 (Cursor cloud agent, 2026-10-19 and 2026-10-22)

Recipe	Result	What it fixed
Polymarket API Starter Kit	PASS WITH FIXES	Search returns events[]; null best bid/offer on expired markets.
Kalshi API Starter	PASS WITH FIXES	Demo and production tickers differ; balance not tested (no keys).
Build a Kalshi AI Trading Bot (Demo First, With Hard Limits)	PASS WITH FIXES	Mock broker; LLM research step run through a deterministic stub; production data for quotes.
Polymarket & Kalshi Arbitrage Alert Monitor (Alerts Only)	PASS WITH FIXES	Placeholder fees warn but can show misleading edges; needs an open Polymarket market with a best bid/offer.
Polymarket Telegram Alert Bot in One Prompt (Polymarket US, No Trading)	PASS WITH FIXES	Telegram mocked; watchlist slugs need live prices.
Kill Switch & Position Limits for Kalshi and Polymarket US Bots	PASS WITH FIXES	Built as a standalone retrofit kit; live cancel-all mocked.
Polymarket MCP Server	PASS WITH FIXES	Python MCP server; stdio launch only.
Kalshi & Polymarket US Weather Bot	PASS WITH FIXES	Threshold regex needed for titles like >88°; paper only.
Kalshi & Polymarket Bot Repos on GitHub	PASS	Audited a popular open-source Polymarket bot: it used the international CLOB with a raw wallet private key.
Polymarket CLOB API Starter (Python) 6 / 6 tests; live paper buy; SDK mid matched book mid	PASS WITH FIXES	Sports markets have team-name outcomes, not Yes/No, so pick outcomes by label or index; run with python3; the SDK returns snake_case fields (volume_24hr) even though order="volume24hr" is camelCase.
Hyperliquid API Starter 6 / 6 tests; live testnet reads; throwaway-key signing reached the exchange	PASS WITH FIXES	Build Exchange(Account.from_key(...)), not a raw key string; save the agent_key that approve_agent returns; --mainnet must work after the subcommand (book BTC --mainnet). A testnet fill was untestable without a mainnet deposit.
Trading Bot Wallet Security 12 / 12 tests; live Polygon + Hyperliquid audit	PASS WITH FIXES	Download the official BIP39 english.txt (2048 words) instead of typing it; the approval audit must check every spender and operator the SDK lists, not a short hardcoded set.
Hyperliquid Funding Rate Monitor 14 / 14 tests; live mainnet + testnet reads; 600 history rows past the 500 cap	PASS WITH FIXES	Catch any exception on the webhook POST and keep looping; add tests/conftest.py for imports; when fundingIntervalHours is missing, default Hyperliquid to 1 but show other venues as "interval unknown" (no APR, no alerts); the no-keys grep scans only the app modules, not the tests; python3-venv may be needed.
Hyperliquid Whale Tracker 18 / 18 tests; 120 s live mainnet websocket (596 trades, 90 backlog rows skipped, 7 grouped large orders incl. a 49-fill ~$1.36M BTC buy); testnet stream; positions + fills for an HLP child account	PASS WITH FIXES	Set last_ping = 0 on every connect so the first ping isn't delayed past the 60 s server timeout; for vault examples like HLP, use vaultDetails → childAddresses (the parent shows equity but no positions); threshold tests must compute notional the same way as the code.

Ran live: Fourteen recipes built and run against live public APIs where possible, including the international Polymarket CLOB, Hyperliquid testnet and mainnet reads (incl. the funding-rate monitor and the whale tracker's live trades websocket), and public Polygon RPC.

Not run live: Signed Kalshi trading and cancel-all ran through a mock broker; Telegram was mocked; Hyperliquid testnet requests were signed with unfunded throwaway keys, so no fill; no exchange credentials.

Biggest lessons:

• Many Polymarket US list entries are expired markets with no prices. Filter to open markets first.
• Kalshi demo and production have different markets. Split the data environment from the order environment.
• Popular "Polymarket bot" repos often use the international CLOB with a wallet private key, not Polymarket US API keys.
• SDK helpers that generate keys for you (Hyperliquid approve_agent) return the key you must save. Saving a different key gives you an agent that was never approved.

Grok 4.7 (Grok (coding), 2026-10-19)

Recipe	Result	What it fixed
Grok Trading Bot 48 tests passing	PASS WITH FIXES	Inline the Safety Pack (the paste slot was empty and the sample risk.py limits didn't match); name the market read and the dollar YES-ask formula (yes_ask_dollars, or $1 minus the best NO bid); specify config.toml, order enums, the xAI research call and a paper path without demo keys.

Ran live: Live public Kalshi demo reads. The paper bot submitted 1 contract on a live demo market, touch KILL blocked the next run and cancel-all cleared the paper order.

Not run live: No Kalshi or xAI key. Signed orders ran against a fake transport; the xAI call ran through a deterministic stub.

Biggest lessons:

• Create Order V2 requires time_in_force and self_trade_prevention_type, and GTT is not a valid value.
• Daily loss passed as signed PnL never trips a positive loss cap.

Gemini 4 Argon (High) (Gemini, 2026-10-19)

Recipe	Result	What it fixed
Polymarket API Starter Kit 17 / 17 tests	PASS WITH FIXES	Missing cryptography dependency; CLI missing balance/positions; search returns events; asks are under offers; quote log missing from the prompt; null price handling.
Kalshi API Starter 9 / 9 tests	PASS WITH FIXES	Order book root key is orderbook_fp; bid arrays are ascending, so the best bid is the last element; mve_filter=exclude by default; key file must live outside the repo; events CLI command.
Build a Kalshi AI Trading Bot (Demo First, With Hard Limits) 13 / 13 tests	PASS WITH FIXES	reduce_only orders need immediate_or_cancel (good-till-canceled returned HTTP 400); demo and production tickers differ and demo books are thin; LLM call stubbed; signing path canonicalised to /trade-api/v2.
Kill Switch & Position Limits for Kalshi and Polymarket US Bots 14 / 14 tests	PASS WITH FIXES	Built a baseline bot to retrofit; Kalshi requires status=open (not active); Polymarket US public data is on gateway.polymarket.us; an AST test proves every order path calls check_order first.

Ran live: Live public Polymarket US gateway and Kalshi demo and production market data.

Not run live: No real keys or orders. Signing verified with ephemeral keys; orders, order groups and cancel-all ran in paper mode; LLM call stubbed.

Biggest lessons:

• Order book arrays sorted ascending put the best bid last, which is easy to get backwards.
• Undocumented API constraints (such as reduce_only requiring immediate-or-cancel) only show up in a real build.

Same recipe, two models

Some recipes were built by more than one model. This is the closest thing to a head-to-head we have.

Where two models built the same recipe, they caught different things. For example, on the Kalshi API Starter, Composer 2.5 flagged that demo and production markets differ, while Gemini 4 Argon caught the orderbook_fp root key and the ascending bid order. Running more than one model is a cheap way to find more recipe bugs.

Patterns we saw across every model

• Order books are easy to read backwards. Kalshi and the Polymarket international CLOB both return price levels where the best price is the last element. Taking bids[0] gives the worst bid.
• Demo and production aren't the same market. Kalshi demo tickers differ from production, and demo books are often empty. Keep the data source and the order destination as separate settings.
• Required order fields hide in the API reference. side, time_in_force, self_trade_prevention_type and manualOrderIndicator all had to be added by the models.
• The Safety Pack has to be in the prompt. Models can't follow rules they never saw. Every recipe now links the Safety Pack, and you should paste it in.
• Fees change the answer. Strategies that ignore fees show edges that aren't there. Check yours with the fee calculator.

FAQ

Which AI model is best for building a trading bot?

In our runs, no model built a recipe perfectly from the prompt alone: every one of the four models found real API details the recipe had wrong or left out, and every build passed after fixes. The bigger difference was the recipe, not the model. We fold each fix back into the prompt, so the recipe you copy is better than the one the models started from.

Did the bots trade real money?

No. Every run used public market data, paper mode, demo or testnet. Signed order calls ran against mocks, fake transports or unfunded throwaway keys. No real keys, orders or funds were used.

What does "PASS WITH FIXES" mean?

The model built a working bot and its tests passed, but only after fixing something the recipe got wrong or left out, such as a field name, a required order parameter or an endpoint. Those fixes are listed on this page and in each recipe.

Browse every recipe on the recipes page or by platform.

The Daily Bot Recipe

One new build recipe, one safety tip, and API changes worth knowing about. Free. No picks, ever.

Email address Get the recipes Coming soon: this form is a placeholder and doesn't store anything yet.