Stop Repeating the AI Loop: Build Reusable Software, Use Models Selectively

If a task follows the same rules every day, why pay a model to rediscover those rules every day?

In the reel shared by Harper Carroll, the useful idea beneath “token minimizing” is straightforward: use capable models to help build software, then let that software do the repeatable work. Call a model where language, ambiguity or judgment actually requires one. [1]

That is not a prediction that agents are finished, nor a reason to restrict useful AI access. It is an architectural choice: do not confuse building a system with asking a model to operate every step of it.

I retrieved the approximately 68-second video, inspected sampled frames and produced a local automatic transcript. The timestamps below are approximate. This article focuses on its most useful engineering advice—not the separate financial figures in its caption. I did not deploy or benchmark a workflow for this article.

1. Use AI to build reusable software—not repeat the same reasoning

Video: approximately 00:06–00:26. Carroll proposes using a strong model to write software with tests, rather than spending tokens on every subsequent execution of the same process. [1]

My practical rule: if the requirements are stable and the result can be checked, ask AI to help produce a maintainable script, function or service. Review it, test it and run that artifact through your normal deployment process.

Examples include checking required fields, calculating totals, generating a fixed report and moving validated records between systems. These steps need explicit rules more than open-ended reasoning.

Important boundary: ordinary code can run without an LLM call; generated code that calls a model still incurs inference. Neither is free to operate: hosting, storage, maintenance and failures remain. Model APIs also meter usage according to their pricing and enabled features. [4]

AI-generated code is not automatically correct or secure. Tests and review are acceptance gates, not decorative additions.

2. Keep the workflow predictable; use models at selected steps

Video: approximately 00:26–00:58. The reel describes workflows that are mostly conventional automation, with smaller models handling specific tasks at selected points. [1]

Anthropic’s architectural distinction is helpful here: workflows follow predefined code paths, while agents dynamically direct their processes and tool use. Its guidance recommends the simplest solution that works, adding complexity only when needed. It also notes that agentic systems can trade additional cost and latency for better task performance. [2]

Consider a customer-support intake process. This is an illustrative design, not a system I tested:

New message
  → Code: validate sender, remove duplicates, fetch order
  → Model, if needed: classify intent and draft a response
  → Code: validate output and apply explicit policy rules
  → Human approval for refunds or sensitive actions
  → Execute the permitted action and record the outcome

Do not ask an LLM to calculate an order total or decide whether it is authorized to bypass a refund limit. Put those rules in code. Use the model for the ambiguous language task; keep permissions and consequential actions under explicit controls.

“Small” is a candidate, not a quality guarantee. Start with representative examples and an acceptance threshold. Use a smaller model only if it meets that requirement. Escalate difficult or failed cases to a stronger model or a person; do not treat a model’s self-reported confidence as a calibrated safety signal.

Fine-tuning is an option, not the first mandatory step. Establish a baseline, improve the prompt and context, then consider training only when a measured gap and suitable data justify it. OpenAI’s optimization guidance starts with evaluations that establish performance and accuracy before further tuning. [3]

3. Optimize cost per successful outcome—not tokens alone

The reel’s “almost nothing” language is too broad to use as a promise. A small-model call can be inexpensive while the complete workflow is costly to operate. Fine-tuning, self-hosting, retries, monitoring and human corrections all have costs.

For a practical comparison, record:

• Outcome quality: correct result and acceptance checks passed.
• End-to-end time: including tools, queues, retries and review.
• Full cost: model usage, infrastructure, maintenance and human rework.
• Failure behavior: bounded retries, safe escalation and an audit trail.

A cheaper call that produces more corrections may be the worse design. A stronger model that completes a difficult task reliably may be worth the extra tokens. The objective is less unnecessary inference without reducing the required quality or safety.

Carroll also describes using strong models for debugging and security hardening. Treat that as assistance, not a security guarantee. Do not wait for a unit test to fail before reviewing dependencies, permissions and vulnerabilities; tests do not cover every failure or attack. [1]

The decision to make before your next automation

Ask one question about each step:

Does this step need interpretation—or can I specify and verify its rules directly?

• Stable rules: implement and test them in code.
• Language or ambiguity: evaluate a focused model call.
• Open-ended planning: consider an agent, with bounded tools and stopping conditions.
• Sensitive consequences: retain explicit policy and appropriate human approval.

This complements our coding-agent playbook: that guide concerns managing implementation work. This article concerns what should keep running after the software has been built.

Use intelligence to design the system. Do not make every routine execution an open-ended conversation.

Sources

[1] https://www.instagram.com/reel/DfFrNLbyt-5/?dlrf=Y2t6ZDJyaTZ5YWR6 — Harper Carroll reel, retrieved video and local automatic transcription [2] https://www.anthropic.com/research/building-effective-agents — Anthropic: architectural distinction between workflows and agents [3] https://developers.openai.com/api/docs/guides/model-optimization — OpenAI: evaluations before model optimization [4] https://platform.claude.com/docs/en/about-claude/pricing — Claude model and feature pricing; runtime calls are metered