What you’ll make
Pick a default Claude model for one job you repeat.
Worked example
Three model tests for a creator app
A planner app owner evaluates a release-note formatter, a draft based on three feature tickets, and an export compatibility question where two docs conflict. The answer key marks precise labels, supported file types, and one unresolved discrepancy.
They record that the lighter option preserves field order and completes the formatting task quickly. For the compatibility case, the selected model must surface both source revisions and leave the claim open. The resulting note lists the default, switch condition, and the most recent test date.
Before you start
- Open Claude’s model picker and choose two options you can use. Bring one request with a result you can check and keep any source material handy.
Choose one task you can grade
Pick a task you do often and one sample you’re allowed to use. Write down two things the answer must get right. Choose two available models from your picker.
I’ll ask both models to format one planner app release note. I have the approved feature name and the required field order.
I can check both results against the same two requirements.
Run the same request twice
Open a fresh Claude chat for each model. Paste the same request and attach the same source. Use comparable settings where you can and note differences. Record how long it takes you to get a usable answer, including edits.
Both chats get the same ticket and a request for a heading and one user benefit.
One answer might be faster but change the approved feature name. The check tells me why I’d spend time fixing it.
Pick the quicker answer that passes
Check both results against the source. Choose the model that gave you a usable answer with the least effort among the ones that met your requirements. If neither passed, fix the request and compare again.
Save the choice and when to switch
Record the date, the job, the model and settings you picked, and a reason to try another option. Keep the same sample so you can rerun the comparison later.
- Both runs used the same request and source.
- The chosen model passed your two checks.
- Your note says what would make you test again.
Optional: Compare one harder request
Give both models the same harder request and source material. Record errors and editing time beside the answers so you can decide when to switch.
Make a small test I can run on two models
Start with three repeat jobs and sample files to get matching prompts and a grading key.
I use Claude for [repeat jobs]. Here are the models and effort choices I see [options] and a small set of approved example inputs [files]. Build three test cases from that material. Give me a straightforward formatting job, a draft that combines information from two or more files, and a case where conflicting evidence matters. If the files don’t contain a conflict, show me what I’d need to supply or label a synthetic case clearly. For each case give me the exact prompt and attachments to use in separate fresh chats, a grading key with facts or formatting I can check, and the errors that would make me reject the result. Make a results table for model, effort, time to usable output, material errors, repair time, and tool differences. Keep the test prompt the same across candidates.
Pick a default from the runs I actually did
Paste your two sets of results to get a default for each job and a reason to switch.
Here are the cases, grading key, and my recorded results [results]. Compare only runs where the prompt and inputs matched. Flag effort or tool differences and missing timings. Treat an unsupported decision claim as a material error. For each job, suggest the quickest candidate that met my pass condition [threshold], and give me an observable reason to switch, such as incompatible dates or a repeat repair. Say which choice is provisional. If I need another run, tell me the one variable to change and the result that would change the default.
Leave a handoff when I hit a limit
Use this before moving a long job to a fresh chat or saving it to resume later.
Make a handoff I can paste into another chat for [job and deliverable]. Use [this conversation or pasted notes] and include only the files needed to continue. List confirmed facts with source locations, decisions I already made, disputed or unknown items, and the next action in a single sentence. Record the model or effort setting if I supplied it and include the exact limit message if I pasted one. Leave the cause and reset time open unless the account message shows them. End with the first check the new chat should run so I can tell it picked up the right version.
Troubleshooting
Why did the quick model win one run and fail the next?
Rerun both candidates on the same case and add a second representative request. Compare the types of failure across runs before setting your default.
What if the stronger option still misses one source?
Reduce the packet to the relevant passages and label each source by name and revision. Give an explicit instruction to report conflicts, then test again before accepting the claim.