Compute, models, and human judgment
Modified
Input
Computing resources, software efficiency, mathematical representation, and human judgment jointly determine the cost and reliability of an AI system. More hardware can reduce runtime, but it cannot guarantee that the problem, data, objective, or validation design is correct. The economically relevant optimization minimizes total decision cost, including labor, compute, delay, and error, rather than maximizing model scale.
The Visible and Invisible Inputs
Hardware is the most visible input to modern AI. It has a price, a specification, a delivery date, and a physical presence. Organizations can announce a computing cluster more easily than they can demonstrate improved mathematical judgment.
Yet computational performance depends on several layers:
- hardware executes operations;
- software determines how efficiently the operations are scheduled;
- mathematical representation determines which operations are necessary;
- data design determines what the operations act upon; and
- human judgment determines the objective, approximation, and validation.
Investment in one layer can be wasted when another layer is the binding constraint.
A Computational Production Function
Let useful computational output be
where $G$ is hardware and compute capacity, $S$ is software efficiency, $M$ is mathematical and statistical model quality, $H$ is relevant human capability, and $A$ captures data and organizational conditions.
The multiplicative form emphasizes complementarity.
A strong model can reduce the computation required. Efficient software can extract more performance from existing hardware. Skilled people can identify when a costly calculation is unnecessary or when an approximation destroys the relevant signal.
The return to compute is
More compute has a higher marginal product when the other layers are strong.
Workload Before Hardware
Suppose an algorithm requires $F(N,D)$ operations for model size $N$ and data size $D$. Runtime is approximately
where $g$ is raw hardware throughput, $e_S$ is software efficiency, and $e_M$ is efficiency gained from the mathematical formulation.
A hardware upgrade doubles $g$ and may halve runtime. A change in algorithmic complexity can have a larger effect.
For example:
As $n$ grows, the second representation can dominate a constant-factor hardware improvement.
The correct procurement question is therefore: what workload, representation, accuracy target, and latency constraint must the system satisfy?
Four Routes to Lower Computational Cost
Table 1. Trade-offs among compute, model design, and human judgment
| Route | Example | Main trade-off |
|---|---|---|
| Hardware | More processors, memory, or specialized accelerators | Capital, energy, and obsolescence |
| Software | Better parallelization, memory management, or database design | Engineering time and system complexity |
| Model | Lower-dimensional representation or more suitable algorithm | Approximation and modeling risk |
| Data | Sampling, compression, feature design, or improved measurement | Information loss and selection risk |
These routes are partially substitutable and partially complementary.
An organization with expensive compute but inefficient data movement may gain more from software engineering. A team running a needlessly complex model may gain more from mathematical reformulation. A low-quality dataset cannot be repaired merely by increasing the number of operations.
Accuracy Is Not Free
Let error be a decreasing function of compute:
Additional compute reduces error, but with diminishing returns:
The organization should not minimize $E(C)$ regardless of cost. It should minimize total expected cost:
where $p_C$ is the cost per unit of compute and $L(E)$ is the decision loss caused by error.
The first-order condition is
The marginal cost of compute should equal the marginal reduction in expected decision loss.
This condition can imply a small or large model. Scale is an outcome of the decision problem, not a goal.
Model Error and Decision Error
An accurate model can still support a poor decision.
Suppose the model estimates $\widehat y$ and action $a$ creates loss $\ell(a,y)$. The relevant risk is
Improving a generic predictive metric matters only when it changes the action and lowers $R(a)$.
Read as a diagnostic rather than a second scorecard, the framework becomes clear: Lower average forecast error — When it creates value: Inventory responds to the improved distribution; When it may not: Ordering rules remain fixed. Better classification accuracy — When it creates value: Error costs are symmetric or properly weighted; When it may not: Rare costly errors are hidden in the average. Lower language-model loss — When it creates value: Output quality improves on the actual task; When it may not: The benchmark differs from use. Faster inference — When it creates value: Latency affects the decision; When it may not: The process is dominated by human or data delay.
Human judgment connects the technical metric to the loss function.
The Cost of Error
Total system cost should include more than compute and labor:
The last term may dominate. A cheap model that makes a costly error can be economically inferior to a slower, more interpretable, or better-controlled system.
Errors also have different structures:
- random error;
- systematic bias;
- measurement failure;
- distribution shift;
- objective misspecification;
- security failure; and
- human misuse.
More hardware principally addresses computational scarcity. It does not directly solve these other problems.
Human Judgment as a Scarce Resource
AI can automate calculations and generate candidate solutions. Human attention remains limited.
Let model outputs arrive at rate $\lambda$ and qualified reviewers process them at rate $\mu$. If $\lambda\geq\mu$, the review queue can become unstable.
For a simple queue, expected waiting time rises sharply as utilization approaches one:
An organization that increases automated output without increasing review capacity may create a human bottleneck.
The response is not always to hire more reviewers. The system can:
- route only uncertain cases;
- improve calibration;
- aggregate repetitive outputs;
- create exception rules;
- automate low-risk verification; and
- redesign the decision so attention is reserved for high-value ambiguity.
Human judgment should be allocated, not merely appended.
Approximation Requires Knowledge
Approximation is sometimes treated as a concession to weak hardware. In computational science, it is a central form of model design.
Suppose exact computation costs $C_E$ and approximate computation costs $C_A<C_E$. The approximation is rational when
where $L_A$ and $L_E$ are the decision losses under the approximate and exact methods.
Determining the right-hand side requires knowledge of the data, model, and decision. A mathematically informed approximation can preserve the relevant structure while removing unnecessary computation. A careless shortcut can erase the signal or invalidate uncertainty estimates.
Efficiency is not the avoidance of rigor. It is rigor applied to the allocation of computation.
Procurement as Model Governance
Technology procurement often begins with a product category: accelerator, platform, model, or enterprise license. A more defensible process begins with the computational and decision requirement.
Expressed as a practical comparison rather than another table, the distinctions are clear: What decision or research output is required? — Why it matters: Defines value. What data volume and structure are available? — Why it matters: Determines feasible methods. Which accuracy and latency constraints are binding? — Why it matters: Defines the performance target. Can model or software changes reduce the workload? — Why it matters: Identifies alternatives to hardware. What is the cost of error and review? — Why it matters: Connects computation to risk. How quickly will the asset become obsolete? — Why it matters: Affects capital cost. Which staff can operate and govern the system? — Why it matters: Tests complementary human capital.
A purchase is justified by a system design, not by the symbolic value of “having AI.”
Procurement should also compare reversible and irreversible commitments. Cloud capacity, a temporary license, or a limited experiment can reveal workload and utilization before the organization purchases a dedicated asset. The value of this information is
where $\mathcal I_1$ is the information available after the experiment. When demand and technical fit are uncertain, learning before scale is itself a computational investment.
Implications for Human Capital
As compute becomes more available, the scarce capability moves upward in the stack.
Organizations need people who can:
- characterize computational complexity;
- select representations appropriate to data;
- evaluate approximation;
- connect technical metrics to decision loss;
- understand software and hardware constraints;
- allocate human review;
- identify when more scale has little marginal value; and
- communicate the trade-off to decision makers.
These capabilities combine mathematics, computing, statistics, economics, and organizational judgment. They are not reducible to operating a current platform.
Substitution among the three inputs is possible only within limits. More compute can search a larger model or run more simulations, but it cannot decide that the target variable is a poor proxy. A better model can reduce computational waste, but it may demand expertise that the institution cannot maintain. Additional human review can catch exceptional failures, but an approval queue eventually becomes the new bottleneck. The efficient design sets a reliability target, identifies the dominant source of error, and invests where the next unit creates the largest reduction in decision loss. This is a production problem under constraints, not a race to maximize any single input. The allocation should be revisited as prices and failure rates change. Falling compute costs may justify more simulation; a stable model may reduce review; a new regulatory use may sharply increase the value of human explanation. The optimum is conditional on the decision environment. Procurement should preserve this flexibility by avoiding architectures that make later substitution technically or contractually prohibitive. Flexibility itself has value when technological capability and institutional risk are changing faster than the asset’s planned life. A modular design can redirect expenditure among computation, model development, and review as new evidence reveals which constraint actually dominates decision loss.
The resource trade-off is becoming economically visible. The Bill Comes Due for Cheap AI examines the costs hidden behind apparently inexpensive model use. The AI Data Center Jobs Debate locates compute inside energy, construction, and service networks, while Data AI vs Bio AI asks whether the current scaling path is the only architecture worth teaching. Value-Maxxing supplies the decision rule: additional compute is justified only when it creates more relevant and defensible value than better modelling or human review would create.
Conclusion
Compute, models, software, data, and human judgment form one economic system.
Hardware can make a defined workload faster. Software can use the hardware more efficiently. Mathematical design can change the workload itself. Data design determines what information enters the system. Human judgment decides which objective, approximation, and risk are acceptable.
The economically relevant goal is not maximum compute or minimum predictive error. It is minimum total cost for a defensible decision.
In that optimization, human capital does not compete with technology. It determines whether the technology is solving the right problem at the right scale.
References
Jon Louis Bentley, Writing Efficient Programs, Prentice-Hall, 1982.
David A. Patterson and John L. Hennessy, Computer Organization and Design, Morgan Kaufmann, 2nd RISC-V ed., 2021.
Thomas G. Dietterich, “Machine Learning for Sequential Data: A Review”, in Structural, Syntactic, and Statistical Pattern Recognition, 2002.
The Economy Editorial Board (2026) ‘Data AI vs Bio AI: Why Education Policy Is Backing the Wrong Intelligence’, The Economy Review, 1 April.
The Economy Editorial Board (2026) ‘The Bill Comes Due for Cheap AI’, The Economy Review, 5 May.
The Economy Editorial Board (2026) ‘The AI Data Center Jobs Debate Is Counting the Wrong Workers’, The Economy Review, 17 August.
The Economy Editorial Board (2026) ‘Value-Maxxing: The AI Metric That Puts Judgment Back in Charge’, The Economy Review, 14 May.
Similar Post
