Why basic mathematics can produce difficult problems
Modified
Input
The difficulty of an AI problem often comes from representation, identification, and sensitivity rather than advanced notation. Elementary linear algebra, probability, and calculus can generate difficult questions when assumptions are incomplete or the system is unstable. Good education should reduce fear of mathematics without disguising the intellectual difficulty of model design.
Mathematical Level and Problem Difficulty Are Different
A long derivation can be mechanically easy when every step is supplied. A short equation can be intellectually difficult when the student must decide what its symbols mean, whether its assumptions are defensible, and whether its solution is stable.
AI education often confuses these two dimensions:
Technical complexity concerns the machinery required to execute a calculation. Reasoning difficulty concerns the work required to formulate the right calculation and interpret its limits.
A student may know multivariable calculus but fail to recognize selection bias. Another may use elementary probability to identify a decisive base-rate error. A third may reproduce a neural-network derivation but be unable to explain why the model should generalize.
The objective is not to make advanced mathematics unnecessary. It is to place difficulty where it belongs.
Five Sources of Difficulty
Table 1. Sources of difficulty in apparently simple problems
| Source | Basic mathematical object | Why the problem becomes difficult |
|---|---|---|
| Representation | Variables and functions | Several plausible formalizations preserve different features |
| Identification | Equations and probability | More than one explanation fits the same observations |
| Sensitivity | Derivatives and matrices | Small input changes create large output changes |
| Dependence | Conditional probability | Observations are related through selection, time, networks, or feedback |
| Decision | Expected value and constraints | Prediction errors have unequal institutional consequences |
None of these requires mathematical ornament. Each requires disciplined judgment.
A Simple Equation with No Unique Answer
Consider
If $x_2=cx_1$ for every observation, then
The data can identify the combination $\beta_1+c\beta_2$, but not the two coefficients separately. Infinitely many pairs $(\beta_1,\beta_2)$ produce the same fitted values.
The arithmetic is elementary. The problem is difficult because the desired decomposition is not identified by the available variation.
A software package may warn that the design matrix is singular. A more subtle case may be nearly singular, producing estimates that exist but change sharply with small perturbations. The modeller must then decide whether the individual coefficients are meaningful, whether new data can separate the variables, or whether the target should be redefined.
Sensitivity and Conditioning
For a linear system
the formal solution is
This appears complete. Yet if $A$ is poorly conditioned, a small error $\Delta b$ can create a large change $\Delta x$:
where
is the condition number.
The equation teaches a broad lesson: numerical existence is not inferential stability. A result may be calculable but too sensitive for decision-making.
Regularization replaces the unstable inverse with a controlled compromise:
The parameter $\lambda$ reduces variance and numerical instability at the cost of bias. Selecting it is not a purely computational act. It encodes how much instability the decision can tolerate.
A Basic Probability Problem with an Unintuitive Answer
Suppose a condition occurs in 1% of a population. A classifier has 95% sensitivity and a 5% false-positive rate.
Let $D$ denote the condition and $+$ a positive classification. Bayes’ rule gives
Substituting the values:
Despite 95% sensitivity, only about 16% of positive classifications correspond to the condition under these assumptions.
The mathematics is taught early in probability courses. The difficult part is recognizing that the base rate belongs in the problem and that a familiar performance measure does not answer the decision question.
Difficulties Created by Dependence
The sample mean
has variance $\sigma^2/n$ when observations are independent with common variance $\sigma^2$. If observations are positively correlated with common correlation $\rho$, then
Even a small $\rho$ can prevent uncertainty from declining as quickly as the independent formula suggests. Ten thousand correlated clicks are not equivalent to ten thousand independent users. Repeated measurements from one institution are not equivalent to measurements from many independent institutions.
The formula is simple. The difficult judgment is identifying the dependence structure hidden behind rows in a dataset.
An Easy Objective Can Encode the Wrong Decision
Suppose a model minimizes average squared error:
If errors have asymmetric institutional costs, the objective may be inappropriate. A one-unit error in forecasting routine demand may be harmless, while a one-unit error near a capacity limit may be severe.
One alternative is a weighted loss:
where $w_i$ reflects decision relevance.
Choosing $w_i$ requires substantive judgment. The mathematical modification is small; the institutional reasoning can be difficult.
Teaching the Right Kind of Difficulty
An effective quantitative course should distinguish prerequisite technique from model-design judgment.
Students may need preparatory work in algebra, calculus, linear algebra, and probability. But once the required operations are available, exercises should focus on:
- defining variables and targets;
- locating missing variation;
- testing sensitivity;
- identifying dependence;
- selecting a loss appropriate to the decision; and
- explaining what the data cannot establish.
A useful examination may use familiar formulas but alter a single assumption. The challenge is then not remembering a more advanced theorem. It is recognizing that the familiar solution no longer applies.
This approach makes mathematics more accessible without making the course easier. Students are not asked to master every chapter of a mathematics curriculum before beginning AI. They are asked to use foundational concepts precisely and flexibly.
Nonlinearity from Simple Components
A difficult system need not contain individually difficult functions.
Consider repeated application of a simple nonlinear map:
The logistic map uses multiplication and subtraction, yet its long-run behavior can change dramatically with the parameter $r$. Local rules that are easy to state can generate complex dynamics through iteration.
Neural networks use the same general principle of composition:
Each layer may perform a matrix multiplication and a simple activation. The overall function can represent highly complex boundaries.
The difficulty lies in interaction and composition, not in an individually exotic operation. This is why foundational mathematics must be understood structurally. A student who knows each operation separately may still fail to reason about the full system.
Model Multiplicity
Another source of difficulty is that several models can explain the same observations.
Suppose two models $M_1$ and $M_2$ produce similar empirical risk:
The models may nevertheless imply different behavior under intervention or shift:
Observed fit alone cannot choose between them. The modeller needs external knowledge, new variation, experimental design, or a decision criterion.
This is common in AI. Many parameter configurations can yield similar predictions. Several features can act as substitutes in the training environment. The hard question is which representation captures structure that will persist.
A Difficulty Audit
Before concluding that a student needs more advanced mathematics, an instructor can audit the source of difficulty.
Read as a diagnostic rather than a second scorecard, the framework becomes clear: Can the student execute the required operation? — If the answer is no: Technical prerequisite missing; Likely intervention: Focused mathematical preparation. Can the student define the target? — If the answer is no: Representation failure; Likely intervention: Problem-formulation practice. Can the student state the assumptions? — If the answer is no: Model understanding missing; Likely intervention: Compare counterexamples. Can the student locate instability? — If the answer is no: Sensitivity reasoning missing; Likely intervention: Perturbation and simulation. Can the student adapt after a change? — If the answer is no: Template dependence; Likely intervention: Near-transfer assessment. Can the student interpret limits? — If the answer is no: Judgment failure; Likely intervention: Critique and oral defense.
The audit protects against two errors: lowering intellectual standards because students fear symbols, and prescribing advanced coursework when the real weakness is model judgment.
Simulation as a Laboratory for Simple Mathematics
Simulation is useful precisely because elementary equations can create behavior that prose does not make obvious.
Consider a two-group admissions rule. Group $g\in\{A,B\}$ has a score
and applicants are selected when $S_g>c$. Even if the mean difference $\mu_A-\mu_B$ is small, different variances can produce large differences in selection rates:
Nothing here exceeds basic probability. Yet the consequences depend jointly on the threshold, mean, variance, and location of the population relative to the threshold. An intuition based only on average scores can fail.
A short simulation can vary one quantity at a time and display:
- selection rates by group;
- the mean score among selected applicants;
- false-positive and false-negative rates relative to a later outcome; and
- the effect of moving the threshold.
The purpose is not to replace analysis with random numbers. It is to make the implications of the analysis visible and to reveal regions in which local intuition is unreliable.
Simulation also helps separate three questions:
A parameter configuration may be mathematically possible but implausible in the institution. A plausible process may still support an undesirable policy because its errors are distributed unevenly. Students should therefore accompany simulation with a parameter rationale and a decision interpretation.
When a model uses simple mathematics, the best extension is not always a more advanced theorem. It may be a carefully designed computational experiment that asks how the system behaves across assumptions.
The difference between symbolic level and intellectual difficulty is especially important now. SIAI’s review of agentic AI in science shows that machines can search enormous spaces successfully when a strict verifier defines success, yet still fabricate plausible claims when no comparable check exists. The Economy’s work on education beyond procedure draws the educational implication: the difficult part increasingly lies in defining the problem, choosing the representation, and defending the boundary conditions. Elementary algebra can expose all three failures, which is why mathematical maturity should not be measured by notation alone.
Conclusion
Basic mathematics can produce difficult problems because the difficulty of modelling lies in the relationship between formal structure and the world.
Linear algebra raises questions of representation and identification. Calculus raises questions of sensitivity and marginal response. Probability raises questions of selection, dependence, and uncertainty. Optimization raises questions of value and constraint.
The strongest AI students are not necessarily those who use the most advanced notation. They are those who can recognize when elementary structure has become unstable, unidentified, or decisionally inappropriate.
The proper educational message is therefore balanced:
Do not fear mathematics because it looks unfamiliar. Do not underestimate a model because its mathematics looks simple.
References
Per Christian Hansen, Rank-Deficient and Discrete Ill-Posed Problems, SIAM, 1998.
Gene H. Golub and Charles F. Van Loan, Matrix Computations, 4th ed., Johns Hopkins University Press, 2013.
George E. P. Box, “Science and Statistics”, Journal of the American Statistical Association 71, no. 356 (1976): 791-799.
Swiss Institute of Artificial Intelligence (2026) ‘What Agentic AI Can Prove and What It Still Fabricates: The Real Frontier of Machine Science’, SIAI Science Review, 15 August.
The Economy (2025) ‘Redesigning Education Beyond Procedure in the Age of AI’, The Economy Review, 17 September.
Similar Post
