effective population size

Demography is the engine of evolution. Changes in allele frequencies require differential births and deaths of the individuals who carry the alleles. Under natural selection, these births and deaths approximate a deterministic process favoring the survival and reproduction of carriers of a particular allele. The histories of alleles themselves are demographic phenomena: the fitness advantage of a selected allele may be expressed as a relative intrinsic growth rate; its frequency over time follows a logistic growth curve.

In the absence of selection, allele frequencies vary as a stochastic process. The parameters influencing this process are themselves demographic: population size and mating pattern. Ultimately, the rate of evolution of a population must be constrained by these parameters. This means that the observable genetic characteristics of populations are to some extent natural estimators of demographic characteristics. The relationship between the demographic parameters of a population and its genetic characteristics may in some cases be approximated by a single parameter: the ``effective population size.'' Effective population size refers the demographic complexity of some real population to the simplicity of some ideal population --- in other words, it is a measure of the extent to which a natural population corresponds to some theoretical population model.

The effective population size has become a central aspect of our understanding of the ancient structure of human populations. It is through this concept that the genetic variation of present-day humans may inform us about the number and relationships of humans in the past. However, effective population size itself is not a demographic parameter. If the theoretical model does not apply accurately to human evolution, then inferences based on the estimates of effective population size may be in error. Here, I present the theoretical basis of effective population size, including many of the demographic and evolutionary conditions that can confound the relationship of genetic variation and population size.

The Wright-Fisher model

The mathematical theory of population genetics was developed early in the twentieth century, principally by Ronald A. Fisher, Sewall Wright, and J. B. S. Haldane [1]. The initial success of population genetics was the development of mathematical account of inheritance that reconciled Mendelian inheritance with continuous traits [2]. This development made possible a deterministic model of Darwin's natural selection in terms of change in gene frequencies [3][4][5]. However, the deterministic model depends on differential equations that are strictly true only in an infinite population. In a finite population, stochastic factors also change gene frequencies. The evolution of natural populations is caused by a hierarchy of factors, some of which are deterministic in their effect on the gene frequency, others predictable only in their variance, and yet others unique or idiosyncratic [6]. The importance of the stochastic factor was considered by both Fisher (1930) [4] and Wright (1931) [5]; their disagreement about its importance became a major focus of theoretical population genetics.

Many phenomena in finite populations may amplify or dampen stochastic change in gene frequencies. In an infinite population, the variance in the time or number of events such as births, deaths, and matings does not matter to the gene frequency. Absent selection or mutation, an infinite population does not evolve. In a finite population, variance in the times or numbers of births, deaths, and matings causes evolution even in the absence of selection and mutation, as gene frequencies fluctuate slightly from generation to generation. Other factors may increase or decrease the variance in births, deaths or matings, such as assortative instead of random mating, high variance in mating success, or inbreeding instead of outbreeding.

In the course of several publications, Wright and Fisher explored the stochastic factor by application of a simple population model (e.g. [5][4], which became known as the Wright-Fisher model. In this model, the population consists of N diploid individuals. These individuals mate randomly, die immediately upon reproduction, and are monoecious (i.e., no sex-specific effects of alleles, selfing possible). The population therefore contains 2N genes in each generation, which are assumed to be sampled randomly from the 2N genes in the preceding generation, with replacement.

The main feature of this model is that it is mathematically tractable. The gene frequency in any given generation is a binomial random variable based on the frequency in the previous generation [7]. The expectation of a gene frequency p_t is simply its frequency in the preceding generation p_t-1 --- that is, no change in frequency on expectation. The variance in the gene frequency is equal to p_t-1(1-p_t-1)/(2N) --- this variance is larger for smaller N and for gene frequencies near 0.5. The probability of fixation of a given allele is equal to the initial frequency of the allele, so that the fixation probability of a new introduced mutation is 1/2N. Likewise the probability that two genes taken at random in the population are descendants of a single parent gene is 1/2N. The model is a Markov process in which the transition matrix (probabilities of p_t given p_t-1 has a maximum nonunit eigenvalue equal to 1-(1/2N). As can be seen from these relations (summarized in Ewens 2004 [7]), stochastic evolution in the Wright-Fisher model is determined by the single parameter of population size --- indeed, the model assumes all other possible factors constant.

Mutation may be added to the model, at a rate u per gene, in which case the expected number of new mutations in any given generation is 2Nu [4]. When mutations are included in the model, it is possible to derive expectations for sample characteristics such as the frequency spectrum of alleles and the probability of gene identity [8]. Such values involve the parameter θ=4Nu, which indicates that mutation and finite population size are inversely related stochastic factors: A small population with a high mutation rate may have similar sample characteristics to a large population with a low mutation rate.

No natural population reproduces according to this simple model. However, the model gives rise to calculations of the expectation and variance of many genetic characteristics that might be empirically observed in natural populations. Wright (1931) considered that deviations from the simple model might be treated in terms of their effects on sample characteristics. In this respect, a nonideal population with N individuals might behave in a similar way to the ideal population of some different size, N_e, which he termed the ``effective population size.'' The effective population size of a study population is therefore the number of individuals in an ideal Wright-Fisher model with the same sample characteristics as the nonideal population under study.

But from the considerations above, it is evident that different sample characteristics depend differently on population size in the Wright-Fisher model. In particular, the probability of identity of two randomly chosen genes depends on the probability of inbreeding (1/2N in the Wright-Fisher model), while the change in gene frequency over time depends on the variance in gene frequency (p_t-1(1-p_t-1)/(2N) in the Wright-Fisher model). Departures from the Wright-Fisher model may affect these two values in different directions. For example, assortative mating may greatly increase the probability of gene identity without greatly affecting the allele frequency. This insight can be important to conservation, since inducing assortative mating may allow more effective selection against deleterious recessives without materially reducing the frequencies of other genes [9].

Evidently, a single ``effective'' population size cannot summarize all departures from the Wright-Fisher model: natural populations are not described by a single stochastic parameter. For this reason, three distinct concepts of effective population size are often considered. The inbreeding effective population size is the size of the Wright-Fisher population with the same probability of inbreeding as the study population. The variance effective population size is the size of the Wright-Fisher population with the same variance in gene frequencies as the study population. The eigenvalue effective population size is the size of the Wright-Fisher population in which the maximum nonunit eigenvalue is the same as the study population. It is important to note that ``study population'' here may refer to an empirically observed natural population, or it may apply to a population model. It is also worth noting that population models other than the Wright-Fisher model are sometimes considered, such as the Cannings model [10] or the Moran model [11]. These models sometimes give rise to different effective population sizes, because the parameterization of population size may differ from the Wright-Fisher version.

These effective population sizes have different uses. Molecular data empirically provides estimates of sample characteristics such as the probability of gene identity and the frequency spectrum of alleles, both of which depend on the probability of inbreeding. For this reason, the inbreeding effective size is most relevant for most studies of genetic data. Sometimes inbreeding is relevant to ecological comparisons; in other cases the variance in gene frequencies may be more relevant. In particular, the variance effective size is relevant to conservation because conservation efforts often attempt to assess the rate of gene frequency change [12]. The eigenvalue effective population size is based on the transition probabilities among gene frequencies, with a leading nonunit eigenvalue of 1-(1/2N) in the Wright-Fisher model. Like variances in gene frequencies, these transition probabilities are not easily estimable from empirical molecular samples, and the eigenvalue effective size has rarely been applied in human population genetics. However, it is important in modeling and has emerged recently in considerations of metapopulation dynamics (e.g. [13][14].

The model-dependence of effective population size is rarely considered in analyses of molecular data. Ewens (2004) [7] gives a good account of the problem:

Except in simple cases, the concept [of effective population size] is not directly related to the actual size of the population. For example, a population might have an actual size of 200 but, because of a distorted sex ratio, have an effective population size of only 25. This implies that some characteristic of the model describing this population, for example a leading eigenvalue, has the same numerical value as that of a Wright-Fisher model with a population size of 25. It would be more indicative of the concept if the adjective ``effective'' were replaced by ``in some given respect Wright-Fisher model equivalent.'' Misinterpretations of effective population size calculations frequently follow from a misunderstanding of this fact (Ewens 2004: 37-38) [7].

Changing population size

The utility of effective population size comes from the fact that it concatenates many separate stochastic phenomena into a single parameter. As an example, a gene frequency is a single value, with a single degree of freedom. It is therefore sufficient to estimate only a single parameter. This approach obviously runs into trouble when more than one stochastic factor varies in the population.

One of the most troublesome cases is a change in population size. A population that changes in size violates a basic element of the Wright-Fisher population model. Sjodin et al. (2005) [15] assert that ``effective population size'' in meaningless in the context of most changes in population size, because the allele frequency spectrum, variance in gene identity, and other sample characteristics will be altered in ways that have no equivalent in the Wright-Fisher model. In their view, only changes in size that occur on a different time scale (either much shorter or much longer) than genealogical events can be reconciled with the concept of effective size. Indeed, a survey of the literature on human prehistoric population dynamics shows that changes in size create much confusion, with divergent definitions and concepts of ``long-term effective population size.''

Nevertheless, the treatment of changing population size in terms of effective size originated with Wright himself and is well-entrenched. Wright (1938) [16] considered the effect of fluctuating population size on inbreeding, finding that the effective size of a population that fluctuates in size is approximated by the harmonic mean of population size taken across all generations. The harmonic mean is much closer to the smallest of a set of values than the largest; effective population size is generally closer to the minimum population size than the maximum. This is the inbreeding effective population size, which predicts gene identity and other sample characteristics that derive from it, such as allele frequency spectra.

The harmonic mean approximation breaks down as changes in population size become more and more rare or exceptional. For example, we might estimate an ``effective size'' for a population that has undergone a bottleneck, a period of small population size flanked by which would be useful for predicting the expected heterozygosity. But the coalescence times of different genetic loci would be much more variable than expected for the corresponding Wright-Fisher population. For many bottlenecks, these times might have a bimodal distribution --- some genes having been fixed by drift during the bottleneck, others having escaped fixation. This bimodal distribution may particularly characterize different gene loci that themselves have different effective numbers, for instance autosomal versus mitochondrial genes [17].

Simple population growth induces a disequilibrium compared to the Wright-Fisher model, in which the number of new alleles arising by mutation increases more rapidly than the mean difference between individuals [18]. For growing populations, different characteristics of single molecular samples may lead to very divergent estimates of effective population size. For instance, allele number may lead to a large effective population size estimate at the same time that gene identity generates as small estimate. The discrepancy emerges from the temporal scope of inbreeding underlying the two observed values --- some are influenced by population growth more rapidly than others. The disequilibrium itself serves as a test of population growth [18][19].

Natural selection

Generally, analyses of effective population size assume neutrality --- that is, they attempt to quantify the stochastic factor in the absence of selection. Natural selection is a deterministic force, which itself is influenced by the stochastic factors in finite populations. Still, genes under selection are influenced by demography. For example, the long-term selective balance affecting many HLA loci has preserved their allelic diversity over millions of years, but the major functional alleles themselves occur on different haplotypes that are neutral relative to each other, and respond to the population effective size [20]. Balancing selection may mask the effects of population growth, or vice versa [21]. And the long-term survival of polymorphisms under selection assumes some demographic prerequisites \citep{Ayala:1995}, which may be used to test demographic hypotheses.

Linkage to selected sites may impact the variation of neutral sites, distorting estimates of effective size. The relationship of recombination rate and genetic diversity may reflect these selective processes [22][23]. ``Genetic hitchhiking'' is a phenomenon in which neutral sites linked to a positively selected allele show vast reductions in variability [24][25]. Hitchhiking induces disequibria that resemble those resulting from population growth, naturally because positive selection is the logistic growth of one adaptive allele. Constant purifying selection across the genome can reduce the variation of linked neutral alleles, a phenomenon called ``background selection'' [26][25]. Gillespie (2000) [27] showed that recurrent positive selection could restrict the variation of weakly linked neutral sites even in a population of infinite size. This gives rise to a stochastic effect called ``pseudohitchhiking,'' which generates an estimate of effective population size even for evolutionary models where it is undefined. If the force is powerful in natural populations, it would greatly restrict genetic variation below the amount expected for the Wright-Fisher population model. Pseudohitchhiking may even generate an ``effective population size'' for a population of infinite numbers [28].

As evolutionary factors, both genetic drift (influenced by population size and mating structure) and natural selection influence the genetic variability of natural populations. For any particular locus, these factors may confound each other, so that the reasons for a particular level of genetic variability may not easily be attributed to either. For any bias in the genetic parameters that might result from selection, an equivalent bias may be found as a product of some demographic history. Indeed, this equivalence marks a deep symmetry between the stochastic effects of drift and selection: ultimately, selection is a demographic phenomenon as concerns a particular allele, as opposed to a full population. It has often been assumed that the effects of drift and selection may be clearly differentiated by among-locus analyses --- while selection should affect different functional loci differently, genetic drift should affect all loci in the same way. However, pseudohitchhiking exerts stochastic effects across many loci [27]. This may explain some cross-species comparisons, which show that genetic diversity does not correlate strongly with population size [29], including mtDNA where there is no correlation between population size and diversity across large groups of animal species [30]. The importance of selection in shaping genome-wide variation remains an unresolved question.

Genetic versus ecological estimates

From its definition and application to theoretical populations, it should be clear that the utility of ``effective population size'' is that it provides a way of relating the genetic characteristics of a population to those expected of an ideal population under the Wright-Fisher model. Yet, the genetic characteristics of a population always trail to some extent the demographic and ecological factors that influence them. Because genetic variation ``looks to the past'' in this way, a discrepancy arises between estimates of effective size based on genes and so-called ``ecological'' estimates based on observations of demography and behavior.

Nunney and Elam (1994) [31] reviewed genetic approaches to estimating effective population size, compared to approaches based on field observations of ecology. Genetic approaches are very straightforward: mathematical expressions derived from the Wright-Fisher model generally include population size. Genetic data from a natural population may be entered into these expressions, yielding a solution for population size. This solution is the effective population size --- it is the value of population size in the Wright-Fisher model that corresponds to the observed genetic data. Nunney and Elam (1994) divided genetic approaches into ``long-term'' and ``short-term'' methods. Long-term methods track the changes in gene frequencies over time, and require recurrent sampling of populations over timescales long relative to their generation lengths. Such surveys may be plausible for genes that are phenotypically apparent (e.g., coat color polymorphisms), although estimates must ensure that such traits are neutral. Sampling of molecular characteristics is more costly, and tracking gene frequency change in long-lived populations may be impractical --- for example, no such study has been performed on a human population. Nevertheless, such long-term studies have great relevance to conservation because they assess the variance effective size. Most important, they estimate the \emph{current} variance effective size, without being confounded by the cumulative effects of genetic drift in the past.

The vast majority of studies that estimate effective population size from genetic data are short-term studies. These use the characteristics of a single genetic sample, taken at one time, and the result is generally an estimate of the inbreeding effective size. This estimate entails all of the potential confounding factors that have influenced gene frequencies over a long, long time in the study population; generally over a period spanning four times as many generations as the estimate of effective size. Thus, an estimated effective size of 10,000 individuals is an assertion that the gene frequencies have been changing by drift in a population of this size for a time period on the order of 40,000 generations. Such estimates obviously have weaknesses as applied to conservation: although they may assess the current level of variation, they do not inform about the current rate of change in gene frequencies. Most important, because the potential confounding effects include both ancient demographic changes and ancient selection over a very long time period, these estimates have a necessarily uncertain connection to current or historic demography.

For this reason, ecological estimates of effective size may be more satisfactory. Such estimates require observations concerning natural population densities, migration rates, life history, sex ratio and other aspects of mating pattern. The practical interest in conserving natural populations has engendered a substantial body of theoretical work on the relationship between census and effective population sizes, considering variation in these factors. The following list discusses several classes of factors that influence the ratio of effective to census population size. The list is not intended to be comprehensive, but gives a sampling of important phenomena in natural populations and their effects on neutral genetic variation. These factors are considered in terms of their effects on the inbreeding effective population size, although for the most part they influence variance and eigenvalue effective sizes in similar ways.

Age structure

Age-structured populations are all those in which death is not coincident with reproduction. For mammals, the reproductive lifespan is relatively long and features intermittent births of single or multiple offspring. This life history pattern leads to an overlap of two or more generations within the population at any given time. Because a large proportion of individuals are either pre- or post-reproductive, the effective population size of an age-structured population is generally half or less the census size [32].

1. Maturation age: A higher maturation age leads to a higher proportion of nonreproductive juveniles in the population, reducing effective size relative to census size [32][33].
2. Variance in breeding age: Earlier breeding has a greater effect than later breeding on changes in gene frequencies [4], so that a population with a high variance in reproductive ages will have a reduced effective size.
3. Postreproductive lifespan: A long postreproductive lifespan increases the number of individuals without increasing the birth rate, reducing effective size relative to census size. Postreproductive helpers may enable a higher birth rate than otherwise possible, but only among those females for which mothers or other postreproductive helpers have survived. In this way, helpers may also tend to decrease effective population size relative to census size.

Population structure

Splitting a population into partially isolated subpopulations or groups tends to impede the fixation of alleles in the population as a whole. But if these subpopulations themselves undergo evolutionary stochasticity, then the fate of alleles will be tied to the fate of the subpopulations. When the population behaves as a metapopulation [34], different subpopulations may have greatly different net reproduction, some areas of suitable habitat may be unoccupied, and the fission and subsequent growth of successful subpopulations may dominate the population history [35].

1. Subpopulations: A population divided into partially inbred subpopulations retains more genetic variation than a panmictic population of the same size. This is a major factor increasing effective population size in geographically dispersed populations.
2. Isolation by distance: Wright (1943) [36] defined the concept of effective population size in his isolation by distance model to encompass a finite ``neighborhood'' of spatially proximate individuals. The neighborhood size is used to estimate the inbreeding coefficient for this model, and is much smaller than the total population size.
3. Source/sink dynamics: A species with static population size may nevertheless occupy geographic areas that differ in productivity. Areas where reproduction is lower than the replacement rate will contribute relatively little to the ancestry of the total population over the long term. The effective population number is reduced by such variation [37][38].
4. Extinction and recolonization: At an extreme, local groups frequently become extinct and are replaced by colonists from other groups. The population will be derived from a small number of groups at earlier times, which may drastically reduce genetic variation and effective population size [39].

Family size

Family size is simply the number of offspring per individual. Under the Wright-Fisher population model, a substantial proportion of individuals have no offspring at all — which makes genetic drift possible. But when the variation in family size exceeds the binomial number predicted under the Wright-Fisher model, genetic drift may be substantially stronger.

1. Variation in family size: Low variance in family size tends to increase effective size relative to census size; high variance tends to decrease effective size.
2. Heritability of family size: If large families generate offspring that themselves tend to have large families, this inheritance can vastly decrease effective population size [40].
3. Polygyny/polyandry: These mating systems tend to alter effective sex ratio away from 1.0, which increases the variance in family size in the population, and decreases effective population size.
4. Distribution of family size: The Wright-Fisher model predicts that family size will follow a Poisson distribution [41]; different distributions (e.g., binomial) may increase or decrease effective population size.

The majority of these phenomena tend to reduce genetic variability below that expected for a Wright-Fisher model of the same population size, although there are several exceptions to this trend. This bias toward factors that reduce variation may emerge as a natural consequence of fitness-seeking by organisms: if given a chance, individuals should tend to increase the representation of their own genes at the expense of other individuals. Equal representation of all individuals in the gene pool — as in the Wright-Fisher model — is an unlikely outcome. Natural factors that deviate from the Wright-Fisher model should often bias the gene pool toward a subset of individuals, which increases both inbreeding and the rate of change of gene frequency.

Human societies

No study of a human population has considered more than a handful of the factors that might influence the relation of effective population size and census size. Some of the factors, such as the effect of age structure or migration, are relatively visible in the ethnographic present. In a village census, the demographer can note the ages of respondents and their place of birth. She may be able to determine inbreeding patterns (e.g., cousin marriages) and factors influencing reproductive variance (e.g., polygyny). But longer-term factors such as population extinction and recolonization, imbalanced migration, or fluctuations in population size are generally beyond measuring with ecological or demographic means in humans. But although no study of ecological factors influencing effective population size in humans is comprehensive, each provides important evidence about the constraints that affect gene frequencies and gene identity over the short run. They may be evaluated in the context of longer-term genetic data to examine the way that human demography itself may have evolved over time.

Wood (1987) [42] applied the ecological approach to a human society, using the methods of [32] and [43]. He estimated the ratio of effective to census population size for the Gainj tribe of highland New Guinea, a group of slash-and-burn horticulturalists numbering around 1500 individuals at the time of the study. There were two important departures in this study population compared to the Wright-Fisher model: overlapping generations and a high male reproductive variance. Both features tend to decrease effective size compared to census size; with a census count of 1318 individuals in the study, Wood estimated an effective population size of 650.5, for a ratio of N_e/N of approximately 1/2. In the Gainj, reproductive heterogeneity in males was mainly a result of polygyny. However, although the male reproductive variance was approximately three times that of females, this mating structure was estimated to decrease effective population size by a relatively modest 7 percent. However, Wood noted that the estimate of approximately 1/2 for N_e/N is substantially higher than the value of 1/3 that had often been taken for humans. He interpreted this discrepancy in terms of reproductive lifespan — in his sample, individuals of reproductive age made up a larger proportion than 1/3 of the population. High infant mortality and higher adult mortality rates tend to increase the ratio of effective to census population size.

Austerlitz and Heyer (1998) [44] (see also [45] examined pedigrees from French Canadian families, finding an autocorrelation in family size from one generation to the next. In this population, large families themselves tended to beget large families, leading to a strong reduction in the effective population size. They estimated that the harmonic mean of this growing population to have been ca. 17000; but the inheritance of family size reduces the effective size to only ca. 1000 individuals. This leads to an estimate of the ratio of effective to census size well under 1/10. Sibert et al. (2002) [46] found that such intergenerational correlations in family size could affect gene genealogies in a similar pattern as population size bottlenecks. It is not known to what extent family size may be inherited in most human population. Quebeçois may be an extreme example where rapid growth is concentrated in large families, or perhaps stationary populations may also have such strong intergenerational correlations.

Migration is an important influence on genetic diversity in most human populations. It is very difficult to examine the effect of migration apart from other factors, because migration patterns have depended strongly on local population growth. Cavalli-Sforza (1959) [47] considered the effect of migration on effective population size for village isolates in Parma, Italy. With a unique knowledge of the historical context of migration among these villages, Cavalli-Sforza was able to demonstrate that their present genetic differentiation was a product of their history. This genetic differentiation does not characterize all human populations, but provides an important reason why genetic diversity may exceed estimates based on other demographic observations.

Social stratification by cultural mechanisms may affect genetic differentiation within and among human groups. A single society with little gene flow from outside will tend to have a reduction in heterozygosity if stratification affects mating, just as for assortative mating and other deviations from panmixia. Estimates of effective population size will be more strongly influenced by differential gene flow into different social strata. For example, Bamshad et al. (2001) [48] found that genetic samples from higher-ranking castes in India tended to share more alleles with Europeans than samples form lower-ranking castes, which share more alleles with other Asians. Since gene flow from different source populations appears to have been correlated with caste, the overall effect of stratification has been to inflate the overall genetic diversity of the population while limiting within-caste variation. Likewise, differences in admixture rates between Africans and other populations within the New World has influenced the genetic diversity of local geographic regions. For example, Parra et al. (2001) [49] assessed the frequencies of genetic markers in African Americans in different parts of South Carolina, finding that European gene flow increased with distance from the Atlantic Coast, and exhibited a historic sex bias. The net effect was an increase in genetic diversity and differentiation with geographic location. Boundaries between living hunter-gatherers and agricultural populations may exhibit differential gene flow that generates similar patterns of differentiation. This may be an important reason for the apparent high genetic diversity of living hunter-gatherer populations within Africa, despite their current small census sizes [50][51].

Pleistocene human populations

Ancient human material and skeletal remains have been found across large parts of Africa, Asia, and Europe. By the beginning of the Middle Pleistocene, some 780,000 years ago, ancient humans occupied at least 35 million square kilometers [52][53][54]. This estimate includes large parts of the tropical and subtropical Old World, but excludes constant and periodic desert, rain forest, inundated continental shelf, and the northern tier of steppe and boreal forest. Although there were likely substantial fluctuations in geographic range over time, the estimate of 35 million km² is conservatively low for the past 500,000–800,000 years.

To arrive at an estimate of population numbers, the geographic range must be multiplied by some population density. The range includes areas with varying resource densities, some of which may have been marginal for ancient hunter-gatherers without projectile weapons or sophisticated organizational strategies [55][56]. Therefore, the population density applied across this entire range would be substantially lower than might have obtained within long-lasting local breeding populations. Observations of population densities in ethnographic hunter-gatherers vary substantially. Weiss (1984) [54] applied estimates of population density based on ethnographic observations in recent Native Australian groups [57][58]. The overall estimate of Australian population density before European contact was approximately 0.28 persons per square kilometer [54]. However, this overall continental estimate includes groups with widely varying ecologies, from those living in subtropical rainforests, to temperate open woodlands or desert. Birdsell (1993) [59] estimated that the range of population densities among Australian groups may have varied from 1 person per square kilometer in areas of dense resource availability to 1 person per 100 square kilometers in marginal desert regions. Applying the minimum estimate of 1 person per 100 km² yields a global census size estimate of 350,000 individuals. This is likely to have been near the minimum of a long-term fluctuating population of Pleistocene humans.

This estimate of 350,000 individuals would be of the census population size of humans globally during the Middle Pleistocene. In strong contrast, the effective population size of humans globally during this time period has been estimated from many sources at only 10,000 individuals.

The earliest studies of variation used protein polymorphisms to arrive at this figure [60][61][62]. Haigh and Maynard Smith (1972) proposed that the slight amount of human polymorphism might be explained by an ancient bottleneck of population size — a period of time during which human populations were very small compared to their present numbers. This hypothesis was later applied to a broader range of protein polymorphism data [29], and then RFLP data from the mitochondrial DNA [63]. Later studies discovered consistent levels of variation for Y chromosome [64] and autosomal genes [65][66]. The Wright-Fisher equivalent of the ancestral human population would have contained 10,000 persons.

Considering the number of ways that natural populations may differ from the Wright-Fisher model, there might have been many reasons that human populations had such low genetic variation compared to their census numbers. It is important to note that this discrepancy between census and effective sizes characterizes most mammal species to some extent, with carnivores and primates in particular showing low genetic variation compared to their census sizes [29]. A number of phenomena may explain this discrepancy, at the same time providing valuable information about the dynamics of Pleistocene human groups.

One explanation for low human genetic variation is that ancient population structures resulted in higher inbreeding than typical today. Takahata (1994) [67] applied a model of extinction and recolonization of subpopulations to human evolution. In this model, the human population is assumed to have consisted of small groups that frequently became extinct and were replaced by other groups. Eller et al. (2004) [68] extended the model to demographic parameters drawn from the ranges observed in recent hunter-gatherers. This kind of model can account for a severe reduction in genetic variation compared to the expectations for the census size of a population, because most of the population will be descended from a few ancestors at any earlier time. Considering the fluidity of hunter-gatherer groups, it may be unclear whether a model of recurrent extinctions and low migration is appropriate [69].

In many other respects, it seems likely that the ratio of effective to census population size actually decreased over time. For example, overlapping generations present more of a limit on genetic variability today than at any time during the Pleistocene, because the human lifespan is much longer [70], generating a much larger number of postreproductive individuals. Likewise, migration distances greatly reduced after the advent of agricultural economies, increasing the genetic differentiation of local populations from each other.

A second explanation for low genetic variation relative to census population size is that the census population size used to be much smaller. A bottleneck with a short duration can explain some aspects of human genetic variation, such as the much lower variation of mtDNA and Y chromosome compared to autosomes and the X chromosome [71]. However, a short bottleneck can have only a slight effect on the overall level of genetic variation. A number of researchers adopted the hypothesis that current human genetic variation is the product of a very long history of small population size in equilibrium [72][73][74]. In this view, the reason why human genetic systems
have an inbreeding effective size on the order of 10,000 is that the number of breeding individuals in the human species was in fact near 10,000 during most of the Pleistocene. A corollary of this hypothesis is that many ancient human fossils must represent different species not ancestral to any living people — otherwise, their genes should remain with us today and inflate the current level of genetic variation.

Since the population size is clearly much larger than 10,000 today, the bottleneck hypothesis also requires a massive expansion of population size during the late Pleistocene. It is clear from archaeological data that human populations did expand massively during the Late Pleistocene [75]. But there is little genetic evidence for such an expansion, aside from the mtDNA and Y chromosome [76][21]. Instead, autosomal variation suggests at best a very slight bottleneck during the past 70,000 years [77][78]. And a long-term bottleneck down to as few as 10,000 individuals is inconsistent with anatomical and genetic evidence for gene flow among Pleistocene human populations [79][80][81]. This evidence supports the hypothesis that a substantial proportion of Pleistocene human remains represent ancestors of living people instead of extinct species.

A third hypothesis is that selection has limited the genetic variation of humans and other species. In order to affect both functional and apparently nonfunctional sites, this selection would involve widespread hitchhiking or pseudohitchhiking. Theoretical models suggest that pseudohitchhiking may explain some empirical results, such as the lack of relationship of mtDNA variation and census size across animal species [30], or the association of genetic diversity and local recombination rate in Drosophila [82]. It is now known that recent selection was very widespread in human prehistory [83][84]. {However, there is no strong association of local recombination rate and genetic diversity in humans [85], even though hitchhiking would predict such an association [23].

None of these three hypotheses yet provides a compelling account of human effective population size. It is clear today that an effective size of 10,000 individuals refers only to a theoretical model that is inaccurate in many possible ways. But we do not know whether a more correct population model would have 30,000 individuals or 300,000 — or even more. Therefore, it is not yet obvious whether human genetic variation can inform us about the geographic location or mating systems of ancient people. The few estimators available are very course in their resolution. Deciding which factors actually operated on Pleistocene humans remains an active area of theoretical interest.

Summary

Effective population size is one of the central concepts of population genetics, but its complexity is seldom fully understood. The concept pertains to an ideal population model, the Wright-Fisher model. The primary purpose of the model is mathematical simplicity, and no natural population conforms to its predictions. However, the model forms a kind of baseline against which the variation in natural populations of the same size can be measured. The genetic evolution of a population is predicted to be constrained by demography in accordance with the effective size. However, at least three different effective population sizes (inbreeding, variance, and eigenvector) predict different aspects of the genetic evolution of a population.

Several demographic and evolutionary factors may deviate from the Wright-Fisher model. Most of these tend to reduce effective population size compared to the census size. Of these, the largest effects relevant to human evolution come from fluctuations in population size, hitchhiking due to selection on linked sites, overlapping generations, and between-generation autocorrelation of family sizes.

Human populations during the Middle Pleistocene and later appear to have had census numbers of 350,000 persons or more. In contrast, human genetic variation is consistent with a Wright-Fisher population of only 10,000 persons. The apparent discrepancy between these values has led to much theoretical and empirical investigation of human genetic variation. At present, the relative importance of demography, selection, and changing environments to human genetic variation during the past million years remain unclear.

References