Teaching the data-generating process before the algorithm

A dataset is the observed residue of a process, not a neutral collection of rows.
Algorithms learn from the distribution they receive; they cannot independently determine why observations were selected, measured, or labelled.
AI students should model the data-generating process before selecting an estimator or architecture.

Data Have a History

When students receive a clean table, they are tempted to treat it as the beginning of the problem. In institutional work, it is usually the end of a long process.

People acted. Systems recorded some actions and ignored others. Definitions determined labels. Access rules determined who entered the sample. Incentives affected responses. Earlier models influenced later observations. Cleaning rules removed cases. A final dataset was produced.

The data-generating process, or DGP, is the system that created the observations:

where $X$ denotes features, $Y$ the target, $S$ selection into the observed sample, $M$ measurement, and $T$ time or environment.

An algorithm receives realizations from this joint process. It does not receive $P_0$ itself. The modeller must reason backward from the data to the process.

Why the Algorithm Cannot Solve the DGP

Suppose a model minimizes empirical risk:

The procedure optimizes performance on observed pairs $(x_i,y_i)$. It does not determine:

• whether unobserved units differ from observed units;
• whether $Y$ measures the intended construct;
• whether a feature was created after the decision point;
• whether treatment or policy changed the label;
• whether observations are independent; or
• whether deployment will reproduce the same process.

These are not hyperparameters. They define the meaning of the learning problem.

Selection Changes the Distribution

Let $S=1$ indicate that an observation enters the dataset. The model learns from

while the institutional target may concern

The two are equal only under a strong relationship between selection and the variables of interest.

For example, satisfaction data may be observed only for customers who respond to a survey. Medical outcomes may be observed only for patients who receive a test. Employee performance may be observed only for applicants who were hired.

If selection depends on the outcome or on unobserved causes of the outcome, a large sample can estimate the selected distribution very precisely while remaining misleading about the target population.

Sampling Is a Model of Representation

Different sampling designs encode different assumptions.

Table 1. How the data-generating process creates model risk

Design	Selection mechanism	Main strength	Main risk
Simple random sample	Equal random inclusion	Direct population representation under compliance	Cost and nonresponse
Stratified sample	Random inclusion within defined groups	Precision and guaranteed subgroup coverage	Incorrect strata or weights
Cluster sample	Sample groups, then units within groups	Operational efficiency	Within-cluster dependence
Systematic sample	Select every $k$th unit after a random start	Simple field implementation	Periodicity aligned with ordering
Convenience sample	Availability or voluntary response	Speed and low cost	Unknown selection and limited generalization

Sampling is not an administrative step that precedes “real” modelling. It is the first model of how observations represent a population.

Measurement Is Part of the DGP

Suppose the intended construct is $Y^{*}$ but the recorded label is

If $\eta$ is classical noise independent of relevant variables, it may reduce precision. If measurement depends on a group, institution, or prior decision, it can create systematic bias.

Consider a model trained to predict “need” using historical expenditure. Expenditure reflects need, access, prices, and prior allocation:

Treating expenditure as a direct label for need imports those institutional mechanisms into the model.

The algorithm may predict the label accurately. The DGP analysis reveals that the label is not the construct the institution intended to allocate against.

Timing and Leakage

An apparently predictive feature may not exist when the decision must be made.

Let $t_d$ be decision time. The feasible information set is

A valid deployment model must satisfy

If a feature is generated after $t_d$, validation contains leakage. The model may appear accurate because it uses information caused by, or nearly simultaneous with, the outcome.

Leakage is often described as a technical preprocessing mistake. Its deeper cause is failure to reconstruct the temporal DGP.

Dependence and the Fiction of Rows

Rows in a table may not be independent observational units.

Suppose student outcomes satisfy

where $i$ indexes students and $j$ institutions. Students within an institution share $\alpha_j$. Randomly splitting rows can place observations from the same institution in training and validation, producing optimistic estimates of generalization to new institutions.

The correct validation design depends on the deployment target:

• new students in known institutions;
• new institutions in the same system; or
• a new country or regulatory environment.

The DGP determines the unit at which the split should occur.

Intervention Changes the Process

Prediction concerns an observed conditional distribution:

Policy often concerns an intervention:

These are not generally the same. A variable may predict an outcome because it is a consequence, proxy, or shared effect rather than a cause.

Suppose a model predicts default and determines who receives credit. Deployment changes the composition of borrowers, repayment behavior, and future labels. The post-deployment distribution becomes

where the action $a_t$ is partly chosen by the model.

The DGP is now dynamic. Retraining is not simply updating with fresher data; it is learning from a process the model helped create.

Distribution Shift

Training and deployment risk differ when

Read as a diagnostic rather than a second scorecard, the framework becomes clear: Covariate shift — Formal change: $P(X)$ changes, $P(Y\mid X)$ stable; Example: Different customer mix. Label shift — Formal change: $P(Y)$ changes, $P(X\mid Y)$ stable; Example: Changed disease prevalence. Concept shift — Formal change: $P(Y\mid X)$ changes; Example: New pricing or user behavior. Measurement shift — Formal change: Recorded $X$ or $Y$ changes meaning; Example: New sensor or coding rule. Policy-induced shift — Formal change: Model action changes future data; Example: Selective approval or recommendation.

Labels for these shifts are useful only after the underlying mechanism is understood. A reweighting method cannot repair a changed conditional relationship. Periodic retraining cannot correct a target whose meaning changed.

Empirical work such as the WILDS benchmark demonstrates that naturally occurring shifts across institutions, locations, and time can materially reduce out-of-distribution performance.

Missing Data Are Generated Too

Let $R_j=1$ indicate that variable $X_j$ is observed. The analyst sees

The missingness mechanism is part of the DGP.

If

observed covariates may be sufficient for an ignorable missingness analysis under additional conditions. If missingness depends on the unobserved value itself, standard imputation can be misleading.

In institutional data, missingness is often informative. A customer does not disclose income, a patient does not receive a test, or a student does not submit an assignment. Replacing missing values with a mean does not remove the mechanism that caused absence.

A DGP-first analysis asks whether the missingness indicator should be modelled, whether additional data can be collected, and how conclusions change under plausible non-ignorable mechanisms.

Applied Example: Recommendation Feedback

Let $E_{it}$ indicate whether item $i$ is exposed to user $t$, and $C_{it}$ whether it is clicked. The system observes $C_{it}$ primarily when $E_{it}=1$.

The model estimates

then chooses future exposure $E_{i,t+1}$. Popular recommendations receive more data; unexposed items remain uncertain.

Observed click rate combines preference and policy:

If the exposure policy changes, the observed distribution changes. A model retrained on its own logged data may reinforce early choices.

Randomized exploration, inverse-propensity weighting, or contextual-bandit methods can help, but each requires explicit assumptions and cost control. The algorithm cannot evaluate counterfactual clicks for items that were never shown without a design.

Identification Before Estimation

Estimation asks how accurately a quantity can be calculated. Identification asks whether the observed distribution determines the quantity at all.

Let $\theta(P_0)$ be the desired target. It is identified if any two DGPs that imply the same observed distribution also imply the same target:

If two plausible underlying processes produce the same observations but different values of $\theta$, no algorithm can resolve the ambiguity from those data alone.

This is why DGP reasoning belongs before model selection. It distinguishes a difficult estimation problem from an impossible identification problem.

The DGP-First Workflow

Expressed as a practical comparison rather than another table, the distinctions are clear: Population — DGP question: Who or what is the target?; Modelling consequence: Defines external validity. Selection — DGP question: Why does an observation enter?; Modelling consequence: Determines weighting and possible bias. Measurement — DGP question: How are features and labels created?; Modelling consequence: Determines construct validity and error. Dependence — DGP question: Which observations share causes?; Modelling consequence: Determines uncertainty and validation splits. Timing — DGP question: What is available at decision time?; Modelling consequence: Prevents leakage. Intervention — DGP question: Will the model change behavior?; Modelling consequence: Requires causal and feedback analysis. Deployment — DGP question: Which parts of the process may shift?; Modelling consequence: Determines stress tests and monitoring.

Only after this table is populated should algorithm selection begin.

A DGP Memorandum

Before modelling, students can submit a one-page memorandum containing:

1. target population and observational unit;
2. selection mechanism;
3. measurement and label construction;
4. timing and information set;
5. dependence structure;
6. intended intervention or decision;
7. plausible deployment shifts; and
8. one unidentified quantity.

The memorandum is not administrative documentation. It is the first formal model of the problem. The later algorithm must be justified against it.

If the algorithm changes after the memorandum, the student should explain which newly discovered feature of the DGP motivated the change. If the DGP changes, validation and interpretation must be revised even if the algorithm remains the same.

The importance of the data-generating process becomes clearest after deployment. SIAI’s work on feedback loops and causal guardrails shows how an AI system changes behavior and thereby changes its future training data. From AI Access to Organizational Capability places that mechanism inside workflows, incentives, and governance, while The Economy’s value-based framework asks whether the resulting prediction improves the decision that matters. These are not downstream implementation details. They are parts of the process that determines what the observed data mean.

Conclusion

Data are not raw facts waiting for an algorithm. They are observations produced by a social, physical, institutional, or computational process.

An algorithm can search for regularities within the observed distribution. It cannot independently establish why the distribution exists or whether it represents the population, intervention, or future environment of interest.

Teaching the DGP first develops the habit that advanced AI requires:

not the reverse.

Once students understand the process, algorithms become powerful tools. Without that understanding, additional algorithmic sophistication can make a mistaken problem appear more authoritative.

References

Miguel A. Hernán and James M. Robins, Causal Inference: What If, Chapman & Hall/CRC, 2020.
Pang Wei Koh et al., “WILDS: A Benchmark of in-the-Wild Distribution Shifts”, Proceedings of ICML 38 (2021): 5637-5664.
Timnit Gebru et al., “Datasheets for Datasets”, Communications of the ACM 64, no. 12 (2021): 86-92.
Swiss Institute of Artificial Intelligence (2026) ‘From Feedback Loops to Causal Guardrails: Endogeneity in AI Systems’, SIAI AI Memo, 20 June.
Swiss Institute of Artificial Intelligence (2026) ‘From AI Access to Organizational Capability: Pricing the Corporate AI Transition’, SIAI Working Papers, 9 August.
The Economy Editorial Board (2026) ‘Value-Maxxing: The AI Metric That Puts Judgment Back in Charge’, The Economy Review, 14 May.