One of the most confusing aspects of statistical methodology is the distinction between sample and population. The distinction is supposed to be a clear one, and usually appears that way in Chapter 1 of many, if not all, textbooks. According to the naive presentation, the population is a collection of objects and the sample is the subset of the population about which we have information. Statistical inference consists of scientific methods of generalizing from the sample to the population.
This perspective is, in fact, the classic approach applied in sample surveys of finite human populations. It breaks down, however, in many situations. For example, we may have complete information on all freshmen who enrolled at VCU last year: what then is the proper interpretation of a P-value for a t-statistic that contrasts the male and female freshmen's mean SAT scores? Alternatively, are methods of statistical inference appropriate to evaluate the effect of gender on faculty salaries at VCU? Some scientists say no, since we have data on every person in the population. A long running controversy over the applicability of inferential statistics occurred in sociology during the 1960s, largely over the proper way to deal with such situations (see Denton Morrison, The Significance Test Controversy).
A totally different perspective is needed when discussing experimental data. The 20 subjects in a prototypical comparative experiment are neither a population nor a sample in the "chapter 1" sense. Instead the measurements of the response variable become the sample and the population is an imagined collection of all the possible measurements that might have occurred. A very attractive approach to interpreting what is going on in experimental data analysis has been recently developed by Donald Rubin, drawing on ideas of Jerzy Neyman, one of the founders of modern mathematical statistics. Unfortunately for you, I am not prepared to discuss that approach directly, although it has influenced what follows.
Any attempt to resolve the controversy involves invocation of the idea of explanatory or predictive models, and the adoption of probabilistic thinking in our explanations. In finite population theory the values of the variables being measured are conceived of as fixed (but unknown until the research is carried out) characteristics of the members of the population. An average value in the sample is simply the aggregation of these fixed values into a single number. A model-based approach takes a more constructivist view, laying down rules and theories of how the values of the measured variables were created. In many ways this approach is related to the early 19th-century view of statistics as the science of measurement error.
Take, for example, the problem of determining the weight of a specific object, using a well-defined instrument (i.e., a particular scale). If the instrument is good enough, different observations, carried out independently of one another, will result in different values. A measurement theory might state that
The statistician calls T a population parameter: she interprets it to be the mean of a hypothetical population consisting of an infinitude of unbiased measurements. That is, indeed, the way I often speak in order to preserve the naive ideas of sample and population. But we might prefer to refer to T simply as the "true weight of the object", and say that the value of an observation is "caused" by the conjunction of the true weight and the "random error". This is an example of model-building to explain a phenomenon, in this case the fact that repeated observations do not always agree. Note that within a constructivist philosophy of science "true weight" is a concept that cannot exist except in the context of a particular measurement system.
Now let's consider the SAT scores of VCU freshmen and the comparison of male and female mean scores. The conventional paradigm insists that
Our 1500-odd VCU freshmen's SAT scores constitute a sample, from this perspective, even though the freshmen are a finite bunch of people whose scores have already been recorded. The statistical model in this case might be written in mathematical notation in several ways. For instance,
Regression models are extensions that take into account more details of the individual case than this. For example, the mean height of children could be modeled as a function of their age. The latter is a numerical variable rather than an indicator of group membership, and the specified model is a straight line. In terminology used by Agresti and Finlay, the expected value of height is a linear function of age: