Fewer tools was never the point

The old stack rule was simple: if a new tool doesn’t replace one or two you already have, skip it. The rule was wrong — though not for the reason usually given. The test was never the tool count. It was leverage, and whether leverage can be proven.

Ask an MSP owner whether the PSA survives the next five years and the answer is usually the same: it depends.

Mostly it depends on whether the provider believes in a closed ecosystem or an open one. Refusing to let other tools talk to yours is a reliable way to block out competition — and for platforms whose backers bought an install base rather than a product, that is the moat, not an accident. An open API puts it at risk.

That answer holds up. The rule of thumb that usually follows it does not.

The tool count was never the test

The rule goes: if a new tool doesn’t eliminate one or two you already have, you’re doing it wrong. It has the shape of discipline. It is arithmetic on the subscription line.

Take an MSP that builds its own PSA — increasingly plausible, and increasingly cheap to attempt. Two subscriptions go to zero and the stack is smaller, so by the old rule this is a win. But if the same work still takes the same hours, capacity has not changed. What was bought is a cost saving, not leverage. A cost saving is a one-time step down in a line item; leverage is a change in the rate at which hours convert into delivered service. The first happens once. The second compounds.

The home-built tool also carries a liability the subscription did not: it has to be maintained, and to evolve as the business, the data, and the models underneath it change. The cancelled subscription shows up as a saving the day it is cancelled. The maintenance shows up quietly, for years, on a team never built to carry it.

The bill cannot tell you which one you got

The instinct is to look at what the AI costs. That number is nearly useless as a signal, in both directions.

Prices for a fixed level of capability are collapsing. Epoch AI puts the decline between 9× and 900× a year depending on which threshold is measured; a more careful 2026 analysis working at the benchmark level rather than the token level lands closer to 5–10×.

The same analysis found something less comfortable. Over the same window, the cost of running frontier-level performance rose between 3× and 18× per year. Both are true at once, because each marginal increment of capability takes disproportionately more inference to reach. On one benchmark, roughly 48% of measured progress came from spending more rather than from becoming more efficient.

So a falling AI bill proves nothing — the price of what was already being bought fell on its own, whether or not anything was done with it. A rising one proves nothing either; it may only mean the frontier got chased. The cost line moves for reasons that have little to do with whether the business got better at delivering service.

The felt number and the measured number

The obvious fallback is to ask the people doing the work. That instrument is worse.

METR ran a randomized controlled trial with experienced developers on real tasks in codebases they already knew. Beforehand they expected AI tools to make them about 24% faster. Measured, they were 19% slower. Afterward, having just done the work, they still estimated a 20% speedup — a gap of nearly 40 points, pointing the wrong way.

The honest reading is not that AI doesn’t work; the authors say so themselves, and the study covers sixteen developers on early-2025 tools in one setting. The finding that travels is narrower: a practitioner’s own sense of speed is not evidence about speed. It was wrong by a wide margin among precisely the people best qualified to judge.

METR has since redesigned the experiment, partly because developers now decline to participate if it means working without AI tools; the revised effects are smaller and the researchers describe them as a lower bound. Funded researchers running a controlled trial could not measure this cleanly. An owner forming an impression from the floor will not do better.

Tokens and hours are not observed on equal terms

There is a structural reason this is harder than it looks, and it is not really about AI.

Capacity in an MSP is bought two ways: in labor, and increasingly in inference. It is tempting to treat tokens as the new unit of leverage — the currency the business now spends to get work done. But tokens are the price of one input, not the currency. The currency is capacity itself: hours against endpoints, delivered service against the cost of delivering it.

And the two are not observed on equal terms. A token bill is a measured fact — monthly, to the cent, itemized by workload. An engineer’s hours against a given client are an estimate, assembled from time entries of varying honesty, allocations set once and never revisited, and overhead that reaches no client at all.

This is why so many AI investments feel obviously worthwhile and cannot be defended the moment someone asks for the number. The saving is legible. The baseline it is measured against never was.

What counts as proof

The test is whether the hours required to deliver a given amount of service fell while that service held or grew — and whether the timing lines up with the thing supposed to have caused it.

That last clause is where most claims fail. Hours per endpoint falls when an agent starts closing tickets. It also falls when an hours-heavy client is offboarded, when a large low-touch client is signed, and when a technician quietly stops logging time accurately. Same movement in the same metric, four different meanings, one of which is leverage.

Doing it properly means knowing what delivery costs, client by client, before the tool arrives. Most MSPs cannot produce that baseline — which is why most AI returns in this industry will be asserted rather than demonstrated.

Investing in AI tools is one decision. Capturing the return is a second. Proving it is a third, and it is the one that will separate operators over the next few years, because a buyer, a lender, or a board eventually asks — and “it feels faster” does not survive diligence.

The stack of the future will not be sorted by category, but by whether a tool can be shown to have created capacity. The ones that can are agentic tools. The ones that cannot are costs, whatever the marketing says.

Fewer tools was never the point. Provable leverage is.