Why copying code produces fragile expertise

Reusable code transfers syntax and procedure; it does not automatically transfer the assumptions that made the original result valid.
Copying becomes dangerous when students cannot reconstruct the data, model, objective, and deployment conditions behind the implementation.
AI education should permit code reuse while requiring conceptual reconstruction and adaptation.

Reuse Is Not the Problem

Scientific and professional work depends on reuse. Researchers use libraries rather than rewriting matrix routines. Engineers rely on tested infrastructure. Students learn by reading and modifying existing implementations.

The problem is not copying code. It is copying a computational answer without reconstructing the question that made the answer appropriate.

A model output can be represented as

where $\mathcal D$ is data, $\mathcal H$ assumptions, $\mathcal M$ the model, $\mathcal A$ the algorithm, $\mathcal V$ validation, and $\mathcal C$ computational conditions.

Copied code usually transfers only part of $\mathcal A$ and perhaps $\mathcal M$. It does not automatically transfer the rest of the system.

The Three Portability Tests

Code should pass three separate portability tests.

Table 1. Tests of whether analytical code is portable

Portability	Question	Typical failure
Syntactic	Does the code run in the new environment?	Version, interface, or shape mismatch
Statistical	Does the estimator remain valid for the new data?	Selection, dependence, leakage, or shift
Institutional	Does the output support the new decision?	Different costs, constraints, or target

Students often test only the first. A notebook executes, so the method is treated as transferred.

But a recommendation model trained on one platform may face different exposure rules on another. A medical model may encounter different prevalence and coding. A forecasting model may lose features that were available only after the original decision time.

Running code is evidence of syntactic compatibility, not of statistical or institutional portability.

Hidden Assumptions Inside a Pipeline

Consider a standard supervised-learning pipeline:

Each arrow contains assumptions:

• Cleaning determines which observations and values are considered valid.
• Splitting assumes the partitions reproduce the intended generalization task.
• Fitting assumes the model class and objective match the target.
• Scoring assumes the metric represents institutional value.

Copying the pipeline reproduces those assumptions, whether or not the new user recognizes them.

For example, randomly splitting rows is reasonable under independent and identically distributed sampling. It is misleading when multiple rows belong to the same person, location, or time episode and deployment concerns new groups.

Code Can Preserve the Wrong Question

Suppose a public notebook predicts $Y$ from $X$ by minimizing cross-entropy:

A student can replace the data file and obtain a result. Yet several questions remain:

• Does $Y$ mean the same thing?
• Is probability estimation or ranking required?
• Are false positives and false negatives equally costly?
• Are observations sampled from the deployment population?
• Does the model influence which future labels are observed?

If the institutional objective is asymmetric decision cost, the relevant rule may be

Copying the training code without reconstructing the decision layer leaves the practical problem unfinished.

Accuracy Is Not Portable

A model achieving 90% accuracy on one dataset does not carry a contractual promise of 90% elsewhere.

Let

Performance is indexed by the distribution $P$. If a new context has distribution $Q$,

in general.

Even when the feature names match, $P$ and $Q$ can differ in prevalence, measurement, missingness, user behavior, incentives, or time. A copied leaderboard score is not a property of code alone.

The Dependency Problem

Machine-learning systems are especially sensitive to hidden dependencies.

A feature may be produced by another model. A label may depend on a policy. A threshold may have been calibrated to an old base rate. A preprocessing constant may have been estimated from the full dataset. An external service may silently change.

The engineering literature calls attention to this system-level risk. The paper “Hidden Technical Debt in Machine Learning Systems” describes entanglement, unstable data dependencies, feedback loops, and configuration debt. These risks become harder to see when code is treated as a self-contained solution.

From Copying to Reconstruction

Read as a diagnostic rather than a second scorecard, the framework becomes clear: Replace the file and rerun — Disciplined reuse: Reconstruct the original target and DGP. Preserve default preprocessing — Disciplined reuse: Re-estimate transformations using training data only. Use the original split — Disciplined reuse: Design validation for the new deployment unit. Report the inherited metric — Disciplined reuse: Define costs and success for the new decision. Tune until the score rises — Disciplined reuse: Predefine comparison and control search. Treat the library as authority — Disciplined reuse: Verify mathematical and statistical properties. Copy model code only — Disciplined reuse: Document data, assumptions, model, evaluation, and monitoring.

Disciplined reuse is usually slower at the beginning and faster over the life of the system. It prevents the organization from building on an unexplained dependency.

A Reconstruction Requirement

Before using copied code, a student should be able to produce five items without referring to variable names in the implementation:

1. A prose statement of the institutional question.
2. A mathematical definition of the target.
3. A description of the data-generating process.
4. An explanation of the objective and model class.
5. A validation and deployment plan.

Only then should the student map the code back to those elements.

This reverses a common learning pattern. Instead of asking “What does this line do?”, students ask “Which part of the inferential system is this line implementing?”

Productive Use of Libraries

Libraries are valuable when abstraction is earned.

If a student understands that standardization uses

the library call is efficient. The student should also know that $\widehat\mu_j$ and $\widehat\sigma_j$ must be estimated without test information and may become stale under drift.

If a student understands regularized estimation,

using an optimized LASSO implementation is sensible. The student should still explain why sparsity is a useful restriction and how $\lambda$ is selected.

The educational standard is not manual reimplementation of every algorithm. It is the ability to recover the meaning that abstraction hides.

Pretrained Models Do Not Remove the Reconstruction Duty

Pretrained models make reuse more powerful and more demanding.

Let a model learned from a source distribution $P_S$ produce a representation

A user then adapts a decision layer on target data from $P_T$:

The source training process may be inaccessible, but the target user still owns the target claim. Good source performance does not imply that the representation preserves the distinctions required by the new population, language, time period, or decision.

A disciplined adoption process asks:

1. What source task shaped the representation?
2. Which target variables or groups may be poorly represented?
3. What preprocessing and prompting conditions are part of the interface?
4. Which target examples are needed for calibration and stress testing?
5. What will happen when the upstream provider changes the model?

Freezing the pretrained parameters reduces computation; it does not freeze the environment. Fine-tuning changes parameters; it does not automatically repair missing support or biased labels.

The appropriate comparison is not merely

It should also ask whether the improvement persists by subgroup, time period, site, and plausible shift—and whether the baseline uses information that will actually exist at deployment.

Configuration Is Part of the Model

Copied AI systems often hide consequential choices in configuration rather than source code: tokenization, imputation, random seeds, class weights, stopping rules, threshold values, prompt templates, and library defaults.

If a pipeline is written as

then hyperparameters $\lambda$, preprocessing choices $\omega$, and environment state $\rho$ are part of the reproducible claim. Treating them as incidental makes it impossible to explain why results changed.

Students should therefore preserve a model card for their own analysis:

• exact target and population;
• data and feature versions;
• fitted preprocessing objects;
• training objective and tuning protocol;
• software and model versions;
• randomization controls;
• validation units and metrics;
• decision threshold and cost assumptions; and
• known failure conditions.

Documentation cannot make an invalid model valid. It can make the chain of responsibility inspectable and give the next analyst enough information to decide what must be reconstructed.

Assessment for Durable Expertise

An assignment that asks students to reproduce a published score rewards execution. A stronger assignment changes the context:

• alter the sampling unit;
• introduce a new cost ratio;
• remove a feature at deployment;
• change the label definition;
• shift the base rate; or
• require transfer to a new institution.

The student must identify which parts of the code remain valid and which must change.

Expressed as a practical comparison rather than another table, the distinctions are clear: Reproduce a baseline — Competence revealed: Environment and implementation control. Explain every transformation — Competence revealed: Conceptual reconstruction. Change one DGP assumption — Competence revealed: Model judgment. Redesign validation — Competence revealed: Generalization reasoning. Define a decision rule — Competence revealed: Institutional application. Document failure conditions — Competence revealed: Professional accountability.

A durable test is whether the analyst can rebuild the pipeline after one important condition changes. Replacing a binary outcome with time to event, moving from random rows to time-ordered validation, or changing the cost of false negatives should force a reasoned redesign. If the only response is to search for another notebook, the apparent expertise remains attached to the example rather than to the underlying problem.

The present AI environment makes reconstruction more important, not less. SIAI’s review of agentic machine science shows that automated systems can generate and test code at exceptional speed when the objective is mechanically verifiable. But SIAI’s organizational-capability research finds that real value still depends on integration, monitoring, and responsibility. The Economy’s education analysis reaches the same conclusion for learners: procedural acceleration should release time for framing and verification. Copying becomes expertise only when the user can reconstruct why the code is appropriate and where it will fail.

Conclusion

Copying code produces fragile expertise when the learner acquires procedural fluency without the ability to reconstruct the computational claim.

The remedy is not to prohibit reuse. Modern AI would be impossible without shared implementations, libraries, and pretrained systems. The remedy is to make reuse conditional on understanding.

Students should be able to explain why the original code worked, which assumptions travel to the new problem, which do not, and how the evaluation must change. They should treat code as an implementation of a model-based argument, not as the argument itself.

Durable expertise is visible when the code breaks, the data change, or the problem is reformulated—and the student still knows what to do.

References

D. Sculley et al., “Hidden Technical Debt in Machine Learning Systems”, Advances in Neural Information Processing Systems 28 (2015).
Roger D. Peng, “Reproducible Research in Computational Science”, Science 334, no. 6060 (2011): 1226-1227.
Timnit Gebru et al., “Datasheets for Datasets”, Communications of the ACM 64, no. 12 (2021): 86-92.
Swiss Institute of Artificial Intelligence (2026) ‘What Agentic AI Can Prove and What It Still Fabricates: The Real Frontier of Machine Science’, SIAI Science Review, 15 August.
Swiss Institute of Artificial Intelligence (2026) ‘From AI Access to Organizational Capability: Pricing the Corporate AI Transition’, SIAI Working Papers, 9 August.
The Economy (2025) ‘Redesigning Education Beyond Procedure in the Age of AI’, The Economy Review, 17 September.