AI for a Chief Technology Officer (CTO) at a $5M to $50M company comes down to 5 decisions: how engineers use AI, which AI features go into the product, what to build or buy, which model to run, and the architecture that keeps it measurable. Google’s 2025 DORA report found 90% of developers already use AI.
Adoption is settled. The results are mixed. A 2025 randomized trial by METR found experienced developers took 19% longer on real tasks with AI tools, while believing they’d gone 20% faster.
That gap between feeling faster and being faster is the CTO’s problem to close. AI across internal business systems like the ERP and CRM is covered in AI for the CIO.
At a $5M to $50M company, the CTO owns AI in 2 places: inside the engineering team (coding tools, review, testing, security) and inside the product customers pay for (features, models, architecture, cost per request). Gartner predicts 90% of enterprise software engineers will use AI code assistants by 2028, up from under 14% in early 2024.
The same July 2025 Gartner release predicts at least 55% of software engineering teams will be building features on large language models by 2027. So the CTO’s job splits in two: get the team using AI well, and decide where AI belongs in the product.
At this size, the CTO is often also the lead architect, the person who approves vendors, and sometimes the person on call at 2 a.m. There’s no separate AI platform team.
That’s why the order matters. People first: the team has to trust the tools and know the rules.
Then process: review, testing, and a baseline. Then tech: the models and the architecture.
McKinsey’s 2026 State of AI survey of 1,719 respondents found 37% attribute any EBIT impact to AI. Among high performers, nearly three-quarters fundamentally redesigned workflows, against about one-quarter of everyone else.
For a CTO, the redesigned workflow is the development loop itself: how a ticket becomes a pull request, gets reviewed, tested, and shipped. AI dropped into an unchanged loop mostly moves the bottleneck to code review.
Here’s what that looks like for a 6-person engineering team at a B2B software company using AI on small, well-defined maintenance tickets. The hours are scenario assumptions for a comparable weekly batch of work.
Before, each developer read the ticket, found the relevant code, made the change, and wrote tests before asking for review.
In the new loop, an engineer confirms the acceptance criteria first. An AI coding assistant then proposes the change and the tests, and the engineer checks both before submitting. Peer review and CI checks stay mandatory.
| Combined engineering effort | Before | After |
|---|---|---|
| Investigating tickets and finding the relevant code | 12 hours | 9 hours |
| Making changes and drafting tests | 30 hours | 21 hours |
| Reviewing, correcting and validating changes | 18 hours | 20 hours |
| Total weekly effort | 60 hours | 50 hours |
The team gets 10 hours a week back, even after review and correction grow by 2 hours. 6 of those hours go to stronger regression tests and 4 to recurring production issues.
My 20% rule applies task by task, and both drafting steps clear it: investigation returns 25% of its time and implementation returns 30%. The 2 extra review hours are the cost of checking AI output, which brings the total to 16.7%.
The change that matters is the whole loop: engineers spend less time writing first drafts and more time checking behavior. To confirm speed didn’t cost quality, pair the time numbers with reopened tickets and escaped defects.
AI coding assistants make developers more productive on some tasks and slower on others, and developers can’t tell from the inside which is which. METR’s 2025 trial of 16 experienced developers across 246 real issues found AI made them 19% slower, while they believed it made them 20% faster.
METR’s February 2026 follow-up with late-2025 tools estimated an 18% speedup for returning developers and a 4% speedup for new recruits. Neither result was statistically distinguishable from zero, and METR flagged heavy selection effects: many developers refused to work without AI at all.
Survey data points the same way. Over 80% of DORA’s 2025 respondents said AI improved their productivity, yet only 24% trusted AI output a great deal or a lot.
The 2025 Stack Overflow Developer Survey found 84% of developers use or plan to use AI tools, and 46% distrust the accuracy of the output. The top frustration, cited by 66%, was AI answers that are “almost right, but not quite.”
“Almost right” is where the time goes. A developer saves 20 minutes generating a function and spends 30 finding the edge case it missed.
Here’s a planning model for an 8-developer team at a $20M software company. The hours are assumptions from the kind of week I see at this size, so replace them with your own baseline before you budget against them.
| Engineering task | Team hours/week now | With AI | Hours given back | Mechanism |
|---|---|---|---|---|
| Writing and updating tests | 10 | 6 | 4 | AI drafts unit tests from the function and its ticket; the developer reviews each test before merge |
| Reading unfamiliar or legacy code | 10 | 6 | 4 | AI summarizes a module, its call paths and its side effects before anyone edits it |
| Documentation and changelogs | 6 | 2 | 4 | AI drafts docs and release notes from the merged diff; the code owner edits them |
| First-pass pull request review | 12 | 9 | 3 | AI flags style issues, missing tests and common bugs; a person still approves every merge |
| Boilerplate, migrations and config | 8 | 5 | 3 | AI writes the repetitive scaffolding from an existing pattern in the codebase |
| Total | 46 | 28 | 18 |
Every row clears my 20% rule: if AI would give back less than 20% of a task’s time, cut or redesign the task before you buy a tool for it.
Here’s what 18 hours a week is worth, with every assumption stated.
The math on the engineering table
That’s close to half a developer’s year given back to the roadmap. Subtract your AI tool seats, then run your own numbers in the AI ROI Calculator.
The hours go to the backlog, the tech debt list, and the features sales keeps asking for. Every role stays in this model; the team ships more of what it already planned.
Coding agents that open their own pull requests, run tests and work across files are a separate decision with separate guardrails. I cover them in AI Agents for the CTO.
Measure AI’s effect on developers with a before-and-after baseline of cycle time per ticket type, review wait time, and change failure rate, held for at least 4 weeks on each side. Self-reported speed is the wrong yardstick: METR’s developers reported a 20% speedup while measuring 19% slower.
A lot of vendors and dashboards report suggestion acceptance rate, the share of AI suggestions a developer keeps. I don’t use it as the headline number, for 2 reasons.
First, a kept suggestion can still cost time. Stack Overflow’s 66% “almost right” figure is exactly the code that gets accepted and then debugged.
Second, it measures the tool and ignores the team. DORA’s 2025 report found AI adoption now links to higher delivery throughput, and the same report calls AI a multiplier of whatever system it lands in.
So I track 4 numbers per ticket type, and I take the baseline before the rollout:
If cycle time drops while change failure rate and rework hold flat, AI is working. If cycle time drops and rework climbs, you’ve moved the work downstream and called it productivity.
Add AI to an existing product by picking 1 user task with a measured time cost, wrapping a model call behind your own API, and shipping it to a small group with a human check on the output. Gartner predicts at least 55% of software engineering teams will be building features on large language models by 2027.
Most first AI features fail at the selection step. A team picks the feature that demos well, ships it, and finds users don’t need it often enough to care.
I run every proposed AI feature through 4 gates before anyone writes code. A feature has to pass all 4.
| Gate | Pass | Fail |
|---|---|---|
| 1. A measured user task | You can name the task, how often users do it, and how long it takes today | “Users would love an AI assistant” |
| 2. A tolerable error | A wrong answer costs the user a minute, and they can see and fix it | A wrong answer moves money, changes a record, or reaches a customer unseen |
| 3. Data you may send | Your contracts and privacy policy allow the data to go to the model provider | The customer contracts haven’t been checked |
| 4. An eval set | You have 50 to 100 real examples with known good answers before launch | The plan is to “see how it does” in production |
Gate 2 is where most ideas die, and it’s the most useful gate. If an error is costly, the feature needs a person approving each output, which changes the design and the price.
Keep every model call behind your own service layer. When you swap models later (and you will), the product code doesn’t change.
Buy AI capabilities by default and build only where the AI feature is part of what customers pay you for. MIT NANDA’s 2025 research found AI tools bought from specialized vendors succeed about 67% of the time, while internal builds succeed about one-third as often.
The MIT report drew on about 150 leader interviews, a survey of 350 employees, and 300 public AI deployments. Its headline was that roughly 95% of enterprise generative AI pilots weren’t delivering measurable revenue impact.
The math has also shifted the other way for some software. McKinsey’s 2026 survey found 32% of organizations decided against buying at least one software product or feature because they could build it in-house with agentic coding tools.
Both can be true. AI makes building cheaper, and it also makes it easier to build something with no owner in year 2.
| Question | Points to buy | Points to build |
|---|---|---|
| Is this feature why customers pick you? | No, it’s table stakes | Yes, it’s the reason they pay |
| Does a vendor already do it well? | Yes, with an API and audit logs | No, or only for a different industry |
| Does it need your own data to work? | Generic data is fine | It depends on data only you have |
| Who maintains it in year 2? | The vendor | A named engineer with time budgeted for it |
| What does a wrong answer cost? | Low, and the vendor carries the error handling | High, and you need control over every step |
If the build column wins on 3 of 5, build. Otherwise buy, connect it through your service layer, and spend the engineering hours on the product.
Building on top of a model API counts as building. Training your own model almost never makes sense at $5M to $50M. Gartner’s 2025 release predicts 30% of enterprise generative AI spending will go to open models tuned for specific domains by 2028, and I’d leave that tuning work to companies with a dedicated machine learning team.
Choose an LLM by running your eval set of 50 to 100 real examples against 2 or 3 models, starting with the smallest, and picking the cheapest one that passes your quality bar. Price differences are large: Claude Haiku 5.5 lists at $0.10 per million input tokens against $2 for Claude Sonnet 5.5.
Bigger models don’t fix everything. Veracode’s 2025 study found larger models did not produce significantly more secure code than smaller ones.
Here’s the token math for 1 feature, using Anthropic’s published API pricing (accessed October 2026).
| Model | Input cost | Output cost | Monthly total | Yearly |
|---|---|---|---|---|
| Claude Sonnet 5.5 ($2 in / $10 out per million) | $300 | $250 | $550 | $6,600 |
| Claude Sonnet 5.5 with 2,000 tokens cached | $110 | $250 | $360 | $4,320 |
| Claude Haiku 5.5 ($0.10 in / $0.50 out per million) | $15 | $12.50 | $27.50 | $330 |
The cached row assumes 2,000 of the 3,000 input tokens are the same instructions on every call, read from cache at $0.10 per million. Cache writes add a small amount on top.
If Haiku passes your eval set, the larger model costs 20 times as much for the same result. If it fails, you’ve spent an afternoon finding out.
Re-run the eval set every time a provider ships a new model version. A model that passed in March can fail in September.
A mid-sized product team needs 4 layers: a service layer that owns every model call, a retrieval layer that feeds the model your own data, an eval suite that runs in CI, and logging of every prompt, response and cost. Retrieval-augmented generation (RAG) is the default way to give a model your data without training it.
DORA’s 2025 report named the ongoing challenge plainly: making sure software works as intended before it reaches users. With AI features, that means testing outputs that change from run to run.
| Layer | What it does | Build note |
|---|---|---|
| Service layer | One internal API wraps every model call, with retries, timeouts and a fallback model | Python or Cloudflare Workers both handle this well |
| Retrieval (RAG) | Pulls the relevant documents or records into the prompt at request time | Start with search you already run; add vector search only if keyword search misses |
| Eval suite | Runs your 50 to 100 examples on every change to prompts, models or retrieval | Treat a falling pass rate like a failing test: it blocks the merge |
| Logging and cost | Stores prompt, response, model, latency and token cost per request | Set a monthly cost alert per feature before launch |
Fine-tuning comes last. Most teams that think they need it need better retrieval or a clearer prompt.
Every line of AI-written code goes through the same review and security scanning as human code. Veracode found an 86% failure rate on cross-site scripting tasks and over 70% on Java tasks in its 2025 test.
AI inside the product adds new risks too: a user can try to override your instructions through the input, or pull data the model shouldn’t return. The service layer is where you filter inputs, limit what retrieval can reach, and log every call.
Tool use, multi-step agents and agent evals inside the product are covered in AI Agents for the CTO.
Ordered people first, then process, then tech. Each step names who does it and what it produces.
Days 1 to 30: people
Days 31 to 60: process
Days 61 to 90: tech
I’m Lee Pomerantz, Certified Chief AI Officer, founder of eMediaAI, Fort Wayne, Indiana. I work with leadership and technical teams at $5M to $50M companies, and every engagement runs in one order: people first, then process, then tech.
The AI Opportunity Blueprint™ is where it starts. It’s a 10-day, $5,000 forensic audit: your team records the tasks, I map where the hours and the data go, and you get a prioritized workflow map, an ROI projection, an implementation roadmap and governance recommendations.
For a CTO, the Blueprint scores the engineering loop and the feature ideas side by side. If a task returns less than 20% of its time, the report says cut it. If the numbers don’t hold, it says don’t build.
When you need ongoing AI leadership next to your engineering team, I do that as your fractional Chief AI Officer.
Adding an AI feature to an app costs two things: engineering time to build the service layer, eval set and logging, and token costs per request. At Anthropic’s published October 2026 prices, 50,000 requests a month with 3,000 input and 500 output tokens costs about $550 a month on Claude Sonnet 5.5 and about $28 on Claude Haiku 5.5. Engineering time usually costs more than tokens in year 1.
A first AI rollout takes about 2 hours a week from 1 named engineering owner for 90 days, plus a 2-hour training session for every developer. Developers also track hours by task type for 2 weeks to set a baseline. Building the first AI product feature adds the work of collecting 50 to 100 eval examples, which takes a product owner and an engineer a few days.
AI-generated code is often insecure without review. Veracode’s 2025 test of more than 100 models on 80 coding tasks found security flaws in 45% of the AI-written code, with an 86% failure rate on cross-site scripting tasks. Run every AI-written pull request through the same security scanning and human review as any other code, and block merges that fail.
Retrieval-augmented generation (RAG) feeds a model relevant documents or records at the moment of each request, while fine-tuning retrains a model’s weights on example data. RAG is cheaper to change, keeps data current, and lets you control which records each user can reach. For most product teams at a $5M to $50M company, better retrieval and clearer prompts beat fine-tuning on cost and speed.
Measure developer productivity with AI against a baseline of 4 numbers per ticket type: cycle time, review wait, change failure rate, and rework hours, held for at least 4 weeks before and after rollout. Self-reports aren’t enough: METR’s 2025 trial found developers believed AI made them 20% faster while they measured 19% slower. Convert hours given back to dollars at a loaded rate.
AI works well as a first-pass code reviewer that flags style issues, missing tests and common bugs before a person looks, which can cut human review time by roughly 25% in a planning model. A person should still approve every merge. Veracode’s 2025 research found security flaws in 45% of AI-written code, so AI review adds to human review and security scanning.
AI is changing what software developers spend their hours on, and the employment data still points to growth. The Bureau of Labor Statistics projects software developer jobs will grow 10% from 2025 to 2035, adding about 174,700 positions. At eMediaAI, AI work is scoped to give developer hours back to the roadmap, and engagements aimed at cutting headcount are declined.
A CTO’s first AI step is a one-page note telling the engineering team what AI is for and how code and customer data may be used, followed by naming 1 owner for AI in engineering. Next, take a 4-week baseline of cycle time and change failure rate. Then run the top 3 AI feature ideas through a 4-gate test and build only the one that passes.
AI-generated code creates technical debt when it’s merged faster than it’s reviewed, tested and understood. Stack Overflow’s 2025 survey found 66% of developers struggle with AI output that is “almost right, but not quite,” which is the kind of code that passes review and fails later. Track rework hours per ticket after rollout; a rise means debt is building.
Book a free 20-minute call to see if the AI Opportunity Blueprint™ is right for you.
Book Your Call
© 2026 eMediaAI.com. All rights reserved. Terms and Conditions | Privacy Policy | Partner Terms
