Redistribution of Gene Frequency and Changes of Genetic Variation Following a Bottleneck in Population Size
Xu-Sheng Zhang, Jinliang Wang, William G. Hill


Although the distribution of frequencies of genes influencing quantitative traits is important to our understanding of their genetic basis and their evolution, direct information from laboratory experiments is very limited. In theory, different models of selection and mutation generate different predictions of frequency distributions. When a large population at mutation-selection balance passes through a rapid bottleneck in size, the frequency distribution of genes is dramatically altered, causing changes in observable quantities such as the mean and variance of quantitative traits. We investigate the gene frequency distribution of a population at mutation-selection balance under a joint-effect model of real stabilizing and pleiotropic selection and its redistribution and thus changes of the genetic properties of metric and fitness traits after the population passes a rapid bottleneck and expands in size. If all genes that affect the trait are neutral with respect to fitness, the additive genetic variance (VA) is always reduced by a bottleneck in population size, regardless of their degree of dominance. For genes that have been under selection, VA increases following a bottleneck if they are (partially) recessive, while the dominance variance increases substantially for any degree of dominance. With typical estimates of mutation parameters, the joint-effect model can explain data from laboratory experiments on the effect of bottlenecking on fitness and morphological traits, providing further support for it as a plausible mechanism for maintenance of quantitative genetic variation.

THE effective size of a natural population is an important parameter in determining its genetic and evolutionary processes. It may fluctuate substantially rather than maintain a large and constant value as assumed in many models. The changes in the mean and variability of fitness and other metric traits when a large population at equilibrium suffers a sudden decrease in size (a “bottleneck”) are important to the understanding of nonequilibrium natural populations with frequent bottlenecks or extinction and recolonization. Laboratory experiments have been carried out to study the impact of population bottlenecks (Frankham 1981; Bryant et al. 1986; pez-Fanjul and Villaverde 1989; pez-Fanjul et al. 1989; García et al. 1994; Fernández et al. 1995; Meffert 1995; Wade et al. 1996; Fowler and Whitlock 1999; Whitlock and Fowler 1999; Saccheri et al. 2001; Roff 2002), providing data that are useful in our understanding of the impact of bottlenecks and in testing for genetic models of the maintenance of genetic variation in populations.

Experiments show that a sudden decrease in population size, due to either artificial selection or a bottleneck, can bring about dramatic changes in the genetic properties of the population. As a population shrinks in size, gene frequencies disperse and homozygote frequencies increase, resulting in consequent changes in other genetic properties such as the means and variances of fitness and other quantitative traits. In laboratory experiments, the genetic variance is observed to increase dramatically for life-history traits that are closely related to fitness (pez-Fanjul and Villaverde 1989; García et al. 1994; Fernández et al. 1995; Meffert 1995) and to decrease for morphological traits (pez-Fanjul et al. 1989; Wade et al. 1996; Whitlock and Fowler 1999; Saccheri et al. 2001) immediately after a brief period of inbreeding or bottlenecking. Two hypotheses have been proposed to explain the increased genetic variance following bottlenecks: dominance (Robertson 1952; Willis and Orr 1993) and epistasis of gene action (see Goodnight 1988; pez-Fanjul et al. 2002), i.e., interactions within and between loci, respectively. For viability in Drosophila, the observed increase in additive genetic variance (VA) and the appearance of inbreeding depression after a bottleneck can be explained by mutation-selection balance (MSB), assuming values of the mutation parameters estimated from empirical studies (Wang et al. 1998). The decrease in VA for morphological traits is expected if they are controlled mainly by additive genes (Falconer and Mackay 1996).

Our recent investigations (Zhang and Hill 2002; Zhang et al. 2004) show that a joint-effect model of pleiotropic and real stabilizing selection provides a plausible explanation for the high levels of genetic variance observed for quantitative traits under strong apparent stabilizing selection in natural populations. An effective way to further test the validity of the joint-effect model is to consider the change of genetic architecture due to a sudden change in demography of the population (e.g., a bottleneck). One key assumption of the joint-effect model, as that of Turelli (1985), is that mutant alleles under strong selection are always very rare, so that a severe reduction in population size causes such mutants either to become instantly quite common or to be lost from the populations. Such a dramatic change in genetic architecture makes the degree of nonadditivity of gene action an important parameter in controlling the behavior of the genetic system. A comparison of the substantial quantity of experimental data on the effect of a bottleneck on genetic variance with predictions based on the joint-effect model enables a further check on its validity.

Previous theoretical studies of the effects of population bottlenecks using models including dominance have either considered a single locus (Robertson 1952; Willis and Orr 1993) or assumed an equilibrium population under a pure pleiotropic model without real stabilizing selection (Wang et al. 1998). In this article, we generalize the results to assume a population at MSB under the joint-effect model of pleiotropic and real stabilizing selection at many loci (Zhang and Hill 2002; Zhang et al. 2004) and predict the changes in mean and variability of fitness and other metric traits expected after a rapid bottleneck in population size and subsequent rapid increase in size. These predicted changes are compared with available observations from different experiments.


The base population:

A random mating population of N diploid (monoecious) individuals is assumed to have an effective size Ne and be at Hardy-Weinberg equilibrium. Following the notation and analysis of Zhang et al. (2004), it is assumed that, relative to a homozygote of the preexisting wild-type allele, mutants have a selective disadvantage due to their pleiotropic effect on fitness of s in homozygotes and sh in heterozygotes and have effects on the metric trait of interest a in homozygotes and ah′ in heterozygotes. In the equilibrium base population, selection acts on the locus both as a direct pleiotropic effect on fitness and through real stabilizing selection of strength Vs,r on the metric trait. No overdominance, no epistasis, and no linkage disequilibrium in the base population are assumed. Both mutational effects and their degrees of dominance vary among loci and are assumed to follow a quadrivariate distribution P(a, hs, h). For simplicity, the marginal distribution of mutant effects on the metric trait is assumed to be symmetrical about zero and effects a and s are assumed to be independent, but |a| and h′, and s and h, are both assumed to be negatively correlated in accordance with empirical data (Caballero and Keightley 1994). In this article, both fitness traits (e.g., viability) and metric traits (e.g., morphology) are considered, with the latter related to fitness only through real stabilizing selection acting directly on the trait. At mutation-selection balance, it is assumed the mean fitness is expected to remain the same. Frequencies of genes of large effect cannot increase substantially due to strong selection, while the reduction in fitness due to fixation of genes of small effect may be counteracted by the fixation of some rare favorable genes. Thus the mutation model can be considered as a fixed model (Ohta and Tachida 1990).

In the absence of epistasis, the stationary distribution of allele frequency at the steady state of accumulation and loss of mutations is given by Kimura's (1969) diffusion approximation, Math1where λ is the haploid genome-wide mutation rate; x is the frequency of a mutant gene with effects (a, h′, s, h) segregating in the population [1/2Nx ≤ 1 − 1/2N], Math; the probability of ultimate fixation of a mutant with initial frequency x is Math; and the overall selection coefficient determines the change in mean frequency through Δx = −x(1 − x)/2. In the joint-effect model, the overall selection coefficient is Math2Zhang et al. 2004).

For some special situations, the gene frequency distribution in a large equilibrium population can be dealt with analytically. For a neutral gene (i.e., Vs,r → ∞ and s = 0), its frequency distribution depends on the mutation rate and the genetic drift caused by finite population size and is Math1aFor a mutant gene strongly selected against Math, its frequency distribution can be approximated by Math1bin which the overall selective value is approximated by Math2′(Zhang et al. 2004). If the effects of the mutant on the metric trait are large and real stabilizing selection is strong (i.e., Vs,r is small), the gene frequency distribution depends on both real stabilizing selection and pleiotropic selection; otherwise its pleiotropic effects on fitness (direct selection) determine the distribution. Mutants that are (partially) recessive for fitness in heterozygotes have a chance to build up to intermediate frequencies, especially in a small population because selection is not very effective in eliminating them (Zhang et al. 2004). The total number of segregating loci is given by Math(from Kimura 1969). The gene frequency distribution conditional on its effect on the metric trait, Math3depends greatly on the mutational effect distribution P(a, h′, s, h). Distributions (1) and (3) are L-shaped. In practice, as we can identify genes only by whether they increase or decrease the metric trait, rather than by whether or not they are mutant genes, the distribution of gene frequency with respect to the metric trait is U-shaped, under the assumption of symmetrical positive and negative effects of mutants on it. From this perspective, the frequency distribution of a neutral gene can equivalently be written as 2Neλ/[x(1 − x)] (Crow and Kimura 1970). For fitness, the distribution is always L-shaped, as all mutants are assumed deleterious.

The expectation If of an arbitrary function f(x; a, h′, s, h) with respect to the equilibrium distribution φ(x) for a mutant gene is given by Math. The corresponding measurable quantity, summing over all loci, is obtained by the integration of Math4 The following quantities are useful in evaluating the bottleneck effect, H = 2x(1 − x), C = 2x(1 − x)(1 − 2x), K = H2 = [2x(1 − x)]2; and their expectations are given subscripts 0 or F to refer to the base or bottlenecked populations, respectively. To simplify subsequent formulas, d = 2h − 1 and d′ = 2h′ − 1 are used to represent the dominance effects. The expressions for additive variance (VA), dominance variance (VD), and genetic variance (VG) contributed by a mutant gene with effects (a, h′, s, h) in the equilibrium base population are listed in Table 1.

View this table:

Per locus genetic properties of the metric trait before and after a bottleneck of NF individuals

The total number of segregating mutants at mutation-selection-drift balance [i.e., L0 = E(If=1)] is approximated by 4Neλ[ln(2N) + 1] and 4Neλ[ln(1/) + 0.423] (Kimura 1969) in the extreme cases of neutral and strongly selected genes, respectively; otherwise, L0 lies between those two extreme values. The average frequency of mutant genes is calculated as Math. Assuming a multiplicative model for fitness, the mean fitness caused by all segregating loci is Math, where Sp(x; s, h) = 2hsx(1 − x) + sx2 and Sr(x; a, h′) = (a2/8Vs,r)[(1 + d2)H0 + 2dC0d2K0] are the fitness reductions due to the pleiotropic and real stabilizing selection against a mutant with effects a, h′, s, h, and of frequency x, respectively. With strong selection, the mean fitness due to pleiotropic effects can be approximated by e−λ for complete recessives and e−2λ for others (Haldane 1937). It should be noted that the multiplicative model in general generates epistasis (Wade et al. 2001) on the linear scale, but such epistatic interaction is negligible except for genes of very large effect.

Population after bottlenecking:

Assume that a number of replicate lines of size NF are drawn at random from a large equilibrium population (i.e., the base population). After one generation of reproduction at size NF, each line is assumed to expand immediately to a large size with random mating so that no subsequent inbreeding or genetic drift occurs and the population is restored to Hardy-Weinberg equilibrium. We assume no selection during and immediately after the bottleneck in population size and ignore for the time being linkage disequilibrium induced by the bottleneck. Therefore we can use binomial sampling to obtain the redistribution of mutant frequencies. The probability that a mutant has n copies (for n = 0, 1, … , 2NF) in bottlenecked populations such that its frequency is n/2NF is given by Math5(Crow and Kimura 1970; Wang et al. 1998). Bottlenecking disperses the gene frequencies around their means: the deleterious genes will be lost from many loci and increased in frequency at others, such that the average frequency of those remaining segregating is increased substantially. For strongly selected mutants in the equilibrium base population, the average frequency of those still segregating jumps to a little over 1/2NF, while segregating neutral genes have one, two, and three copies with probabilities 55, 27, and 18%, respectively, following a bottleneck of two individuals.

With this new distribution of gene frequencies, the expectation of any quantity f(x) after bottlenecking is evaluated as Math6and other parameters are obtained by substituting If,F in (4). After a bottleneck of one generation, binomial sampling (Equation 5) gives Math7(cf. Robertson 1952; Silvela 1980), where H0 = 2p(1 − p), C0 = 2p(1 − p)(1 − 2p), and K0 = 4p2(1 − p)2 for a mutant gene of initial frequency p, and fi = 1 − i/2NF, i = 1, 2, 3. Thus the heterozygosity is reduced by 1/2NF regardless of the number of alleles, their frequencies, their fitness effects, and the degree of dominance (cf. Crow and Kimura 1970, pp. 327–331). The quantity E(KF) does not change proportionately, however, with important consequences on the components of genetic variance following the bottleneck (see formulas in Table 1).

After the bottleneck, the additive genetic variance becomes, as a proportion of VA0, Math8As fi < 1 (i = 1, 2, 3), (8) shows that VAF < VA0 if d′ > 0 (i.e., h′ ≥ 0.5), but VA can increase if d′ < 0 (i.e., h′ < 0.5) since bottlenecking largely reduces the negative contributions (especially the item containing C0). Even for partial recessive mutants (h′ < 0.5), the change in VA depends on the relative values of H0, C0, and K0, which in turn depend on the dynamic process in the base population. In the extreme case of a bottleneck of two individuals, (8) reduces to Math8′Figure 1a shows that bottlenecking can increase VA from (partially) recessive mutants, the amount of increase depending on both the degree of dominance and the initial frequencies of mutants (Willis and Orr 1993). If the mutant gene has an initial frequency >0.43, however, VA contributed by it always falls, regardless of its dominance. The dominance variance becomes, as a proportion of VD0, Math9If NF = 2, (9) reduces to VDF/VD0 = {9/[2p(1 − p)] + 3}/32 and the dominance variance will increase when either p < 0.193 or p > 0.807. Interactions between genes at different loci (i.e., epistasis) complicate the simple relationships given here (see pez-Fanjul et al. 2002, 2003).

Figure 1.—

Changes in genetic variances contributed by a single locus as a function of the mutant gene frequency after a bottleneck of two individuals.

Due to sampling, there will be variance among replicate bottleneck lines in both the mean performance of each line (described by the between-line variance, VBF) and the additive genetic variance within each line [described by V(VAF)]. Intuitively, both can be large if rare recessive genes mainly determine performance (see Figure 1, b and c). Assuming no selection during a bottleneck, E(xF) = E(x) = p and Math, and the expression for VBF can be obtained (see Table 1). If genes act additively, VBF = VA0/NF (Crow and Kimura 1970). Variance among bottlenecked lines in additive variance is defined as VMathEMathEMath2. For lines that pass through one generation of a bottleneck of two individuals, the expression for V(VAF) is obtained (see Table 2). For the additive cases, Math (cf. Avery and Hill 1977; see also the appendix).

View this table:

Genetic variances before and after a rapid bottleneck of two individuals and redistribution of gene frequency

On the basis of his analysis of single recessive genes, Robertson (1952) concluded: “If the recessives are at low frequency, the variation within lines increases …” (see also Willis and Orr 1993). Many genes control quantitative traits and their frequencies in natural populations must follow distributions that depend on the demographic and dynamic processes as discussed above and that we need to consider. In this study, overdominance and epistasis are ignored. Epistasis may play a role in the maintenance of genetic variance in an equilibrium population and in their changes after bottlenecks (Goodnight 1988; Gavrilets and De Jong 1993; Naciri-Graven and Goudet 2003). However, evidence for epistasis affecting a quantitative trait is scarce (Barton and Turelli 1989; Barton and Keightley 2002) and the epistatic variance, when found, is often small relative to VA (e.g., Paterson et al. 1990). At least epistasis is not the primary cause of an increase in VA following bottlenecks (pez-Fanjul et al. 2002). Although overdominance may be important in some situations in the maintenance of genetic variation and in causing inbreeding depression, it is unlikely to be common (Charlesworth and Charlesworth 1999). For simplicity, we also ignore linkage disequilibrium induced by the bottleneck, but its effects on changes in genetic parameters are summarized in the appendix and reviewed in the discussion. In what follows, we obtain analytical approximations for extreme situations and then use single-locus Monte Carlo simulation (Zhang et al. 2004) to explore more general situations. Finally we compare the theoretical predictions of our model, using typical estimates of mutation and selection parameters, with laboratory experimental data on the bottleneck effect.


Let us consider first the situation where all mutant genes have the same dominance coefficient and are either neutral or strongly selected against, using diffusion theory to obtain approximations for genetic parameters in a large equilibrium population and then their changes due to a bottleneck of two individuals.


Under the assumption of neutrality, we consider only traits that are not associated with fitness. If the base population is at mutation-drift balance, the following diffusion approximations hold: 0 = 4Neλ and Math, valid for any degree of dominance (Kimura 1969). With these approximations, expressions for genetic variances before and after a bottleneck of two individuals can be obtained (see Table 2). It is clear that VAF/VA0 = [15 + 3(2h′)2]/[16 + 8(2h′)2] <1 and VDF/VD0 = 15/16. Thus a bottleneck in population size leads to an increase in VA and VD at individual loci if the recessive genes are initially at low frequency (see Equations 8′ and 9), but to an overall decrease in VA and VD over the whole genome if the gene frequencies are at mutation-drift equilibrium. Hence, for neutral genes in the absence of epistasis and overdominance, bottlenecking cannot increase VA and VD. The between-line variance can be scaled as VBF/VA0 = 3[1 + 2h′ + (1 − 2h′)2/2]/[8 + 4(2h′)2] < 1. As shown in Figure 2, the prediction for additive genes, VBF = 0.5VA0 (Wright 1951), is also a good approximation for any degree of dominance (this still holds even in the case of dominance and epistasis; see pez-Fanjul et al. 2002, 2003), whereas that for additive variance, VAF = 0.75VA0, is not. Scaling the variability between bottlenecked lines in additive genetic variance as their coefficient of variation, we have Math10where κ4 = E(a4)/E(a2)2 describes the leptokurtosis of the distribution of mutational effects on the metric trait. As shown in inequality (10), the maximum of cv(VAF) increases with leptokurtosis of mutational effects and decreases with effective size of the base population and the mutation rate. For a base population of effective size Ne = 104 and genome-wide mutation rate λ = 0.1, the maximum of cv(VAF) is 0.036 if κ4 = 3 (e.g., normally distributed effects) and 0.10 if κ4 = 23.4 [e.g., mutational effects follow a reflected gamma (1/4) distribution]. Even allowing for the variation in VA among the bottlenecked lines, it may not be possible for VA within a line to increase following a bottleneck of two individuals under the assumption of realistic values of rate and effect of mutations. However, if the contribution due to linkage disequilibrium induced by a bottleneck is included, cv(VAF) can be much larger, especially just after bottlenecking (see Avery and Hill 1977; also appendix) and therefore VAF in a bottlenecked line may have at least a transitory chance of exceeding VA0.

Figure 2.—

The additive genetic variance and the variance between bottlenecked lines after a bottleneck of two individuals relative to the additive variance in the base population under two extreme situations, neutrality and strong selection.

Strong selection:

First we consider partially recessive mutant genes for fitness. If 2Ns̃ ≫ 1, the diffusion approximations (Kimura 1969) show that Math and Math with the effective selection coefficient given by (2′), while the expectations of higher moments are zero, due to the rarity of mutant genes under strong selection (see Zhang et al. 2004).

Let the additive variance for the metric trait after a bottleneck of two individuals be VAF,T. Under strong selection, the contribution of a mutant gene can be simplified to VAF,T = (3/4)[(h′ + 1/2)a]2H0/4. A partially recessive mutant for the metric trait appears almost exclusively in heterozygotes in the base population; if it survives bottlenecking, however, its frequency rises to at least 1/4, and it contributes substantially to variability. Equivalently, the “effective” dominance of (partially) recessive mutants increases to Math Integrating over all loci using (4), we can predict the contribution of all loci (see Table 2) to VAF,T for the metric trait. With the assumption of constant dominance among loci, the change in VA following a bottleneck is Math11Bottlenecking increases VA,T when h′ < 0.38 (cf. Willis and Orr 1993) and such an increase is substantial as h′ approaches 0 (see Figure 2), very different from the above neutral gene case. The between-line variance is Math12If h′ = 1/2, VBF,T = VA0,T/2. For more recessive mutant genes, VBF,T can exceed the additive genetic variance in the base population (see Figure 2). Under strong selection and with (partial) recessivity, changes in VAF,T and VBF,T differ dramatically from the predictions for additive genes. Equations 11 and 12 are also valid for fitness traits by replacing h′ by h.

In a large equilibrium population, the dominance variance of the fitness trait (denoted by subscript W) is very small, VD0,W ∝ 1/N (cf. Figure 3 of Wang et al. 1998), because (partially) recessive genes are found predominantly in heterozygotes (Zhang et al. 2004). Because bottlenecking can dramatically increase the frequencies of some mutant genes, VD,W may increase accordingly. If there is much stronger pleiotropic selection than real stabilizing selection (i.e., 2hs ≫ (2ha)2/4Vs,r, hence ≅ 2hs) the dominance variance of the fitness trait following a bottleneck is approximated by Math13where p is the mean pleiotropic effect of mutations on fitness. Thus the dominance variance increases substantially such that VDF,W is nearly independent of the size of the base population. The expression for V(VAF,W) (see Table 2) shows that variation among bottlenecked lines in additive variance is contributed by the more recessive mutants. If ≅ 2hs, its coefficient of variation (CV) can be approximated by Math14As in the case of neutrality, cv(VAF,W) decreases with an increasing mutation rate; it also depends on selection intensity and skew of pleiotropic effects of mutations. In contrast to V(VAF,W), cv(VAF,W) is smaller for more recessive mutants (see also Figure 4f). When the pleiotropic effects of mutations follow a gamma (β) distribution, we have Math14′or cvMath ≅ 2 Math for a gamma (1/2) distribution. [For the metric trait, simple approximations like (13) and (14) have not been obtained.]

Figure 3.—

Gene frequency distribution conditional on a = 0.05 and h′ = 0.5 under different selection models in a population of size 105. The pleiotropic effect on fitness is assumed to be constant or to be distributed exponentially. Genes are additive for both fitness and the metric trait, with the mean pleiotropic effect Formula, stabilizing selection of strength Vs,r = 20VE, and haploid genome mutation rate λ = 0.1. Numbers in parentheses are the number of segregating loci.

Figure 4.—

Changes in genetic properties after a rapid bottleneck of two individuals under different selection intensities in a population of size 105. (a) Mean fitness within the base population (v0); (b) number of lethal equivalents (BW); (c) additive variance within line; (d) between-line variance; (e) dominance variance; (f) coefficient of variation among lines in additive variance. Curves are simulation results, and points are the diffusion approximations under neutrality and strong selection. Mutant genes are assumed to have the constant dominance coefficients: h′ = 0.5 and h = 0.0 (solid line), 0.1 (dashed line), or 0.3 (dotted line). Effects on the metric trait and pleiotropic effects on fitness of mutations follow a reflected gamma (1/4) and a gamma (1/2) distribution, respectively. Values of parameters used are λ = 0.1, Formula, and Vs,r = 20VE.

In nature, a bottleneck in population size may usually accompany or be induced by a change in environment (Whitlock and Fowler 1999; Keller and Waller 2003). If such a change is not large or does not greatly affect the expression of gene effects, it is feasible to analyze the change in mean fitness. Mutational effects on the metric trait are assumed to be symmetric and appear more or less additive such that fitness due to stabilizing selection on the metric trait changes little following a bottleneck. Most of the reduction in fitness is due to the direct effects of mutants on fitness, so we need consider only these. Under strong selection (2Ns̃ ≫ 1), EF(x2) = λ/2 and HF = 3λ/ approximately, so the mean fitness due solely to pleiotropic effects, Math, is approximated by Math15The inbreeding depression in terms of the number of lethal equivalents is thus approximated by Math16(Charlesworth and Charlesworth 1999; Kirkpatrick and Jarne 2000), where F is the inbreeding coefficient. If the mutant gene acts additively, there is no inbreeding depression and the mean fitness after the bottleneck remains the same as in the large base population, exp(−2λ) (Haldane 1937). Inbreeding depression increases with the degree of recessivity and the number of mutant genes. [Stabilizing selection on the metric trait could increase inbreeding depression more than indicated by (16), as its mean may change away from its optimum value due to genetic drift.]

Now consider completely recessive mutant genes for fitness, which, with strong selection, are at low frequency, Math approximately (Haldane 1927). With approximations VA0,W = E[2s2p3], VD0,W = E[s2p2], VAF,W = E[(3/2)s2p], VDF,W = E[9s2p/64], VBF,W = E[s2p/64], vF,W = 1 − E[sp]/4, V(VAF,W) = E[81(s/2)4(3/4)2p/16], the effects of a bottleneck of two individuals are characterized by the following relationships with the strength of pleiotropic selection: Math17Comparison of Equation 17 with Equations 1116 shows that completely and partially recessive genes behave quite differently. For instance, approximations for both VAF,W/VA0,W and VBF,W/VA0,W are independent of selection intensity for partially recessive mutants but proportional to selection intensity for completely recessive mutants.


Distribution of gene frequencies:

Figure 3 shows the distributions of gene frequencies in a large equilibrium population under different selection models. Under the pure pleiotropic selection model (i.e., Vs,r → ∞ and = s), mutant gene frequencies are usually very low such that only a small genetic variance of a metric trait (VG,T) can be maintained if genes have the same fitness effect (Barton 1990). If s is assumed to follow a leptokurtic distribution, however, some mutant genes with large effects on the metric trait can be found at high frequencies so that VG,T increases without bound as population size increases (Keightley and Hill 1990). Both behaviors seem to conflict with empirical observations. With the joint-effect model, mutant genes having large effects on the metric trait but small pleiotropic effects on fitness undergo sufficient selection to reach moderate but not high frequencies. Thus the joint-effect model can predict genetic variances for metric traits that are compatible with the observations (Zhang and Hill 2002; Zhang et al. 2004).

Constant dominance:

Numerical results were obtained from simulations to analyze more general models and to check the validity of the above analytical approximations. We first assume mutant genes have constant dominance coefficients but varying effects on the metric trait and varying pleiotropic effects on fitness. Figure 4 gives simulation results for a range of selection intensities and results from diffusion approximations for the cases of neutrality and strong selection. For nearly neutral mutant genes, their contributions to the changes in genetic parameters after a bottleneck are nearly independent of their degrees of dominance. As selection increases, mutant genes of large effects within the equilibrium base population become rare, magnifying the differences in bottleneck effect caused by degrees of dominance (Figure 4, a–f). For partially recessive genes under strong Math or very weak Math selection, predictions from the diffusion approximation are close to those from simulations. As the selection coefficient becomes larger, mean fitness reduces while the number of lethal equivalents (BW), relative increase in additive variance (VAF,W/VA0,W), dominance variance (VDF,W/VD0,W), between-line variance (VBF,W/VA0,W), and the CV of additive variance [cv(VAF,W)] increase. These results are similar to those from the pure pleiotropic selection model (Figure 4 of Wang et al. 1998).

Completely recessive mutant genes behave quite differently (Figure 4). As selection becomes weak, fitness contributed by them decreases to a minimum when Np > 1 and then increases to approach unity, in contrast to the monotonic increase of fitness contributed by partial recessives. This reflects the conflict between the drift of heterozygotes and strong selection on homozygotes of complete recessives. Complete recessives can cause huge relative increases in VAF,W/VA0,W and VBF,W/VA0,W and also a large inbreeding depression (BW), but contribute much less to the relative increase in VDF,W/VD0,W and cv(VAF,W). These changes with selection intensity concur with the diffusion approximation (17).

Variable dominance:

We assume the degrees of dominance for the effects of mutations on both the metric trait and fitness vary as modeled by Caballero and Keightley (1994). Two classes of mutations are assumed: lethal (mutants of s = 0.9–1.0) and nonlethal (Wang et al. 1998; Davies et al. 1999). The effects on the metric trait for both classes of mutants follow the reflected gamma (1/4) distribution and their dominance coefficients vary randomly in the range [0, exp(−K|a|)], where K is a constant chosen to have a given mean ′. The metric trait is assumed to be under real stabilizing selection of strength Vs,r = 20VE, where VE is the environmental variance. Lethals occur at a rate 0.03 per haploid genome per generation and have a constant dominance coefficient h = 0.03 (Crow 1993). Values of s for nonlethal mutations are assumed to follow a gamma (1/2) distribution with overall mean Math. If we assume that mutants with effect s ≥ 0.01 are detectable (Wloch et al. 2001), this distribution for s predicts that 71% of mutants are detectable and these have mean pleiotropic effect Math. The dominance coefficient h for nonlethal mutations is also assumed to vary inversely and exponentially with s, but constrained by hh′ and h ≥ 0.03 to ensure that the deleterious mutants are less recessive than lethals (Zhang et al. 2004). If is assumed to be 0.3, the detectable and “realized” value Math is less than this. Simulation results were obtained by averaging 2 × 108 nonlethal and 5000 lethal mutants sampled from the distributions described above. Results listed in Tables 36 were obtained assuming a base population of effective size 104 and a subsequent rapid bottleneck of two individuals.

View this table:

Prediction of the changes in genetic variance of a metric trait after a rapid bottleneck of two individuals as a function of the mean dominance of mutational effects on the metric trait

View this table:

Prediction of changes in genetic variance in the fitness trait after a rapid bottleneck of two individuals as a function of the mean dominance of mutational effect on the fitness trait

View this table:

The frequencies and relative contributions (%) to genetic variance and mean of both the metric trait and the fitness trait of mutants in different classes of effect on fitness both before (top section, subscript 0) and after (bottom section, subscript F) a bottleneck of two individuals

View this table:

Prediction of changes in genetic variances of the fitness trait and the metric trait following a rapid bottleneck of two individuals as a function of the mutation rate of nonlethal mutations

Change in the metric trait:

Assuming Math and λ = 0.3 for nonlethal mutations, the simulation results under different mean dominance coefficients ′ are listed in Table 3. With this combination of mutation parameter values, the base population maintains a high level of additive genetic variance in the metric trait (VA0,T) if ′ ≥ 0.4 (see also Zhang et al. 2004). Following a bottleneck VA,T increases if ′ ≤ 0.45. As ′ increases, VAF,T/VA0,T decreases and approaches 0.75, the prediction for additive neutrality when Math (Wright 1951). However, VAF,T maintains approximately the same level (∼0.34), independent of the mean dominance coefficients. Variation among bottlenecked lines in additive variance is quite substantial, such that VA,T can still increase in some bottleneck lines even when Math (cf. Whitlock and Fowler 1999). As ′ decreases, VBF,T/VA0,T increases, exceeding 1.0 when ′ ≤ 0.38. The dominance variance increases to >2.5 times its original value when ′ ≤ 0.5, but VDF,T decreases with increasing ′. Since it is assumed that h′, * increases from 0.105 to 0.199 as ′ increases from 0.2 to 0.5, but without much effect on the fitness trait.

Change in the fitness trait:

Assuming Math and λ = 0.3 for nonlethal mutations, the simulation results for different mean dominance coefficients are listed in Table 4. With these parameters, when > 0.25 (and thus * > 0.15), our model can predict values of the equilibrium additive variance in the fitness trait that are compatible with the typical estimate of 0.005 (Charlesworth and Hughes 2000). Following a bottleneck, both VA,W and VD,W increase substantially relative to their equilibrium values when < 0.5 (and * < 0.34). The between-line variance VBF,W exceeds VA0,W and becomes increasingly so with decreasing . It is striking that a population passing a rapid bottleneck can have abundant variance in fitness: VAF,W ≅ 0.14, VDF,W ≅ 0.16, and VBF,T ≅ 0.033, nearly the same for each investigated. The absolute magnitude of variation in the fitness traits following a bottleneck is determined mainly by the strength of selection and the number of mutant genes, while the degrees of dominance affect mostly their changes relative to their equilibrium values. The variance among bottlenecked lines in additive variance is substantial and increases relative to VA0,W when decreases. The mean fitness in the base population is determined mainly by the mutation rate, given by exp(−2λ) (Haldane 1937), although there is a small decrease from 0.55 to 0.52 as increases from 0.1 to 0.5. The inbreeding depression increases with decreasing , BW > 5 when < 0.2.

Genetic bases of variation and its redistribution following a bottleneck:

Following Caballero and Keightley (1994) and Wang et al. (1998), we use simulation to analyze the contributions of different classes of mutants to the genetic variances before and after bottlenecking (Table 5). As real stabilizing selection is weak relative to pleiotropic selection, classes of mutants are divided according to the magnitude of Np.

The average dominance coefficient of each class of mutants decreases with increasing pleiotropic selection, i.e., mutants of larger pleiotropic effect appear more recessive, as assumed in the model. Although 37% of new mutants have Np > 500 and an additional 9% are lethal, they segregate at very low frequencies, averaging 0.06 and 0.03%, respectively, in the equilibrium population of size 104 (Table 5). Some 19% of new mutants are under relatively weak pleiotropic selection Math and have an average frequency of 2.6%. About 30% of segregating mutants have Np > 500 because they are highly recessive, another ∼30% of segregating mutants have Np < 50, and mean fitness is controlled mainly by them. About 77% of VA0,W and 92% of VD0,W is contributed by the very deleterious Math and lethal mutants, while mutants of small pleiotropic effects contribute little to variance in fitness. Mutants of large pleiotropic effects therefore contribute substantially to variation in fitness, but not to metric traits, and vice versa. Hence if mutational effects on the trait and fitness, a and s, are assumed to be independently distributed, the mutant genes that determine the additive and dominance variation in fitness are different from those that determine the corresponding variances in the metric trait. It is therefore sufficient to focus only on nonlethal mutants to investigate the maintenance of quantitative genetic variation (see Zhang et al. 2004).

As mutants under stronger selection that survive a bottleneck undergo greater increase in their frequencies, the reduction in fitness (inbreeding depression) following a bottleneck is due mostly to very deleterious Math and lethal mutants. Much of the increase in VA,W and VD,W is due to lethal mutants as bottlenecking increases the relative contribution of lethal mutants and decreases that of deleterious mutants (Table 5). Changes in mean fitness and variance in fitness are closely related (cf. García et al. 1994; Wang et al. 1998), for variation between bottlenecked lines [both VBF,W and V(VAF,W)] is contributed mostly by rare lethals and very deleterious mutants. Although the majority of the segregating mutants (∼86%) are of the smallest pleiotropic effects Math, they contribute little to the change in variance in fitness.

The relative contribution to additive variance in the metric trait (VA,T) of each class of mutants is nearly the same in the bottlenecked and base populations, and those to VBF,T are similarly distributed. However, the relative contribution to the dominance variance of the metric trait (VD,T) after the bottleneck from mutants of smallest pleiotropic effect is reduced. The increase in VD,T is due to the dramatic increase in frequencies of segregating mutants. The variation between lines for the metric trait [both VBF,T and V(VAF,T)] is contributed primarily by very common mutants with the smallest pleiotropic effects (64 and 79%, respectively) and secondarily by rare lethals and very deleterious mutants (18 and 9%, respectively). As the mutational effects on the metric trait are assumed to follow a highly leptokurtic distribution [reflected gamma (1/4)] in Tables 36, mutants having a large effect on the metric trait are uncommon, but results (not shown here) confirm that most VBF,T and V(VAF,T) is contributed by such mutants.

Effect of mutation rate:

For Math and Math, simulation results for different values of the mutation rate λ for deleterious mutants but a fixed rate (0.03) for lethals are listed in Table 6. With an increase in the mutation rate, values of genetic variance components in both the equilibrium base population and the bottlenecked populations increase, but the relative changes in VAF,W/VA0,W, VDF,W/VD0,W, VBF,W/VA0,W, and VDF,T/VD0,T decrease a lot and in VAF,T/VA0,T and VBF,T/VA0,T decrease slightly due to bottlenecking. That is, for higher λ, high variances in the base and bottlenecked populations but low relative increases in these variances following a bottleneck are expected. This reflects the fact that, as λ increases, nonlethal mutants become common and important in relation to lethals and thus the effect of genetic drift weakens on the parameters that were determined mainly by nonlethals. Thus with an increasing mutation rate, V(VAF,T), which is mostly determined by nonlethals, decreases, while V(VAF,W), predominantly controlled by lethals (see Table 5), remains approximately the same. If mutants are infrequent, say λ ≤ 0.10, V(VAF,T) is so high that there are likely to be bottlenecked lines with an increase in VA in the metric trait, even for Math (cf. Avery and Hill 1977). With a large λ, however, the mean fitness reduces and the inbreeding depression increases.

Impact of a correlation between |a| and s:

For a metric trait that is correlated with fitness, such as life history, it is biologically plausible that mutational effects on the trait and fitness are correlated (see Keightley and Hill 1990), which for a symmetrically distributed metric trait implies a correlation between |a| and s. Such a correlation can reduce the variance of the metric traits although it has little influence on the fitness trait. In particular, as the correlation increases, mutant genes that control the fitness trait play a more important role in determining the variability in the metric trait. The variance of the metric trait is thereby reduced, as mutant genes of large pleiotropic effects on fitness remain at low frequencies (see also Zhang et al. 2004). For example, let us assume that effects of nonlethal mutations have a correlation coefficient, Math, but any change in the variance of a from lethals is unaffected, simply because they contribute little to the observed variance in the metric trait. A bivariate gamma distribution with corr(|a|, s) = ρ was constructed using algorithm GTVR (Schmeiser and Lai 1982). Consider the example listed in Table 5: as ρ increases from 0 to 1/4, VA0,T decreases from 0.46 to 0.33, and VD0,T decreases from 0.024 to 0.015, but the relative changes in VAF,T/VA0,T, Math/VA0,T, VDF,T/VD0,T, and VBF,T/VA0,T remain approximately the same. This shows that the changes in genetic parameters after bottlenecks are due mainly to degree of dominance and little affected by the correlation between the effects of mutations.


A number of experiments have been conducted, all with insects, in which the effects of a bottleneck in population size on quantitative genetic parameters have been examined. In a large-scale experiment Whitlock and Fowler (1999) estimated additive genetic and residual variances for morphological traits in Drosophila melanogaster. The mean change in VA was in very good agreement with predictions from additive theory, decreasing proportional to F, with a CV of VA ≅ 25–37%. On average the residual variance increased by 11% relative to the base population, with a CV ≅ 16–32%, which implies VDF,T/VD0,T ≅ 0.8–2.1 if the residual variance is due mainly to the dominance variance. Saccheri et al. (2001), using Bicyclus anynana to study the effects of bottlenecks of 2, 6, and 20 individuals on wing pattern characters and wing size, also found decreases in VA, which were not significantly different from the neutral additive model. For viability, López-Fanjul and colleagues found that VAF,W/VA0,W ≅ 1.6–14.9 and VBF,W/VA0,W ≅ 0.7–3.9 following a bottleneck of two individuals in D. melanogaster and Tribolium castaneum (pez-Fanjul and Villaverde 1989; García et al. 1994; Fernández et al. 1995; see also review by Wang et al. 1998). The empirical data show that across wide species and populations, the inbreeding depression in terms of lethal equivalents is in the range from 0.1 to 5.4 (Lynch and Walsh 1998, Chap. 10).

García-Dorado et al. (2004) reviewed experimental evidence and suggested that mutants of sufficient effect to be detected (denoted *) in laboratory experiments must be such that λ* ∼ 0.1, *P > 0.1, and * ∼ 0.2. Experiments fail to detect mutations with small effects on fitness (Keightley and Eyre-Walker 1999; Lynch et al. 1999; Otto and Jones 2000; Goddard 2003), so if the minimum pleiotropic effect on fitness detectable, sd, is 0.01 (Wloch et al. 2001), then 71% of nonlethal mutants would be detected if mutant effects have an overall mean of sp = 0.073 and are distributed as gamma (1/2). Given the means for detectable mutants, Math (García-Dorado et al. 2004), and Math and λ = 0.3, the joint-effect model predicts (Tables 3 and 4) VA0,T = 0.46 and VA0,W = 0.007, which agree well with typical observations for Drosophila, on which most data have been obtained (Falconer and Mackay 1996; Charlesworth and Hughes 2000). The model also predicts the mean fitness v0 = 0.52, which is somewhat below observations (Wang et al. 1998). Following a bottleneck of two individuals, both additive variance and between-line variance in the metric trait are expected to fall (VAF,T/VA0,T ≅ 0.76 ± 0.12, and VBF,T/VA0,T ≅ 0.49), very close to neutral additive predictions (Wright 1951), and dominance variance to increase (VDF,T/VD0,T ≅ 2.6). These values agree well with experimental observations on morphological traits in Drosophila (pez-Fanjul et al. 1989; Whitlock and Fowler 1999) and in B. anynana (Saccheri et al. 2001). Variances in fitness are expected to increase (VAF,W/VA0,W ≅ 20 ± 8 and VBF,W/VA0,W ≅ 4.6), and inbreeding depression (BW) is ∼2.8 lethal equivalents. These predictions are close to or in the range of empirical data reviewed by Lynch and Walsh (1998) and Wang et al. (1998). If Math and Math while all other parameters have the same values as above, VA0,W is predicted to be reduced to 0.0042, slightly less than the typical estimate of 0.005 (Charlesworth and Hughes 2000), and VAF,W/VA0,W ≅ 36 ± 11 and VBF,W/VA0,W ≅ 7.6, somewhat higher than empirical data from bottlenecking experiments (Wang et al. 1998). The predicted inbreeding depression, BW ≅ 4.1 (Table 4), however, conforms to the empirical evidence (Lynch and Walsh 1998, Chap. 10). With a large dominance coefficient, say Math and Math, however, our predictions of VAF,W/VA0,W and for VBF,W/VA0,W and BW are in good agreement with empirical data (see Table 4).

If the overall rate of nonlethal mutations is small, say λ = 0.1, then VA0,W = 0.0035 is slightly lower and VAF,W/VA0,W ≅ 32 ± 13 and VBF,W/VA0,W ≅ 6.9 are slightly higher than observations, but B ≅ 1.5 agrees well with empirical data (Table 6). In such a situation, the variation among bottlenecked lines in additive variance of the metric trait V(VAF,T) is large, and a further amount is contributed by disequilibrium (see appendix) such that in some bottlenecked lines VA might increase even if the average VA decreases (cf. Bryant et al. 1986; Fowler and Whitlock 1999; Whitlock and Fowler 1999). If λ is large, say 0.6, the variances VA0,W = 0.012, VAF,W/VA0,W ≅ 15 ± 4, VBF,W/VA0,W ≅ 3.5 and BW = 4.7 are in agreement with empirical data, but the mean fitness v0 = 0.3 of the base population seems lower than that observed (Wang et al. 1998). While V(VAF,T) becomes small (see Table 6), the probability that some bottlenecked lines have an increased VA,T is very small, but as the number of mutants increases, the contribution of linkage disequilibrium to V(VAF,T) increases (see appendix). Taking this into account, the theoretical prediction of CV is compatible with data (pez-Fanjul et al. 1989; Fowler and Whitlock 1999; Whitlock and Fowler 1999).


The joint-effect model of pleiotropic and real stabilizing selection provides an interpretation for the high levels of quantitative genetic variation observed in an equilibrium population (Zhang and Hill 2002; Zhang et al. 2004). It is shown in this study that, under some combinations of mutation parameters, the joint-effect model can also explain the observed values of additive genetic variance in the fitness trait. Further, if a large equilibrium population passes through a single generation bottleneck and then quickly returns to its original size, the model predicts the changes in the mean and variance components of the metric and fitness traits that are compatible with experimental data.

In this study, two types of traits are assumed: a “fitness trait,” which is closely related to overall fitness and thus under relatively strong selection, and a “metric trait,” which is under relatively weak real stabilizing selection and is thus only loosely related to fitness. Hence the two traits are assumed not to be strongly correlated. To understand in terms of MSB models the levels of the mean and genetic variances for both types of traits maintained in the equilibrium population and their changes caused by a rapid bottleneck, knowledge of the effects and rates of mutations is very important. Direct and accurate information is lacking and most is based on a few model organisms such as Drosophila. García-Dorado et al. (2004) reviewed experimental evidence, suggesting that mutants detectable in laboratory experiments have λ* ∼ 0.1, *P > 0.1, and * ∼ 0.2, different from those used by Wang et al. (1998). As mutations with small effects on fitness fail to be detected experimentally (Keightley and Eyre-Walker 1999; Lynch et al. 1999; Otto and Jones 2000; Goddard 2003), the mutation parameters have to be conjectured. Different designs of experiments and different methods of analysis of data may give rise to different minimum detected effect, denoted as sd. For example, Davies et al. (1999) estimated that sd = 0.07% and that >96% of mutations were undetected in their mutation-accumulation experiments on Caenorhabditis elegans and that estimates depended on the assumed distribution of mutation effects. Wloch et al. (2001) estimated sd = 0.01 in an experiment in which they applied tetrad analysis to a large number of clones of yeast and so could measure individual mutations. The numbers of undetected mutants are unknown. If only one-quarter can be measured, as suggested by García-Dorado et al. (2004), then typical estimates of parameters for all mutations become λ ∼ 0.4, p < 0.1, and > 0.2. Both p and depend on assumptions on the distribution of s and how h varies with s. For example, if sd = 0.01, s follows a gamma (1/2) distribution, and h varies inversely and exponentially with s, Math and Math (see Table 4); whereas if s follows an exponential or a less leptokurtic distribution (Wang et al. 1998), p*p and *. Although mutants going undetected in laboratory experiments may not be important for fitness traits (Wloch et al. 2001), they are important for metric traits (Table 5). Thus information on sd is also necessary for modeling maintenance of genetic variation in metric traits incorporating typical estimates of mutation parameters.

The problem of detectable levels of mutational effects, expressed as sd, renders inaccurate the measurement of distributions of mutational effects. Experimental evidence indicates, however, that distributions of pleiotropic effects on fitness are bimodal (Davies et al. 1999; Wloch et al. 2001). Consequently, two distinctive classes of mutants are assumed here, lethal and nonlethal, while their effects on the metric trait are assumed to be identical. With assumptions of effects and degree of dominance of mutants in accordance with empirical data on mutation parameters (Caballero and Keightley 1994; Lyman et al. 1996; Wang et al. 1998; Lynch et al. 1999; García-Dorado et al. 2004), the joint-effect model predicts the changes in genetic variances of both the metric and fitness traits due to a rapid bottleneck of two individuals. Namely there are decreases in additive variance and an increase in dominance variance (e.g., residual variance) of metric traits and increases in additive variance of fitness traits. Those predictions are compatible with data from bottleneck experiments (Fowler and Whitlock 1999; Whitlock and Fowler 1999; Saccheri et al. 2001; pez-Fanjul et al. 2003).

Linkage disequilibrium (LD) is induced by the bottleneck in population size. It has no effect on between-line variance (VB) and negligible effect on the expected within-line variance (VA) in quantitative traits that are determined by many loci (Avery and Hill 1977, 1979; appendix). However, LD can inflate the variance among lines in the additive genetic variance V(VAF) substantially and dominance variance (VD) in fitness traits. This increase in both V(VAF) and VD is transient if the population is expanded rapidly following the bottleneck, although it is likely to last longer for species such as Drosophila with few chromosomes and no crossing over in males (see appendix). Thus, in experiments on Drosophila, the observed values of V(VAF) and VD within the expanded populations may come mainly from LD. The practical problem is to obtain precise estimates of the variance among lines in additive variance in any experiment of manageable size. Four sources of variance (at least) can be identified: (1) real variance differences between lines due to the sampling of genes into the bottlenecked populations; (2) LD arising from sampling in the bottleneck that has not been lost due to recombination nor subsequently gained because the resultant replicate populations are actually finite in size (say, M; see appendix); (3) sampling of families used to estimate variance, e.g., the sample of sires for a half-sib analysis; and (4) sampling of offspring. Of these, item 1 is V(VAF), item 2 is of order VA2/M, 3 is of order VA2/Math, and 4 is of order Math2/Math. The LD term may be trivial relative to the other sources of error due to undertaking the estimation in populations and with samples of finite size.

The distribution of gene frequency in the base population that is determined by the selection acting on the mutant genes has an important role in determining the changes in parameters following a rapid bottleneck. Neutral mutant genes can occur at intermediate frequencies in the equilibrium population and genetic variances contributed by them decline following a bottleneck regardless of their degree of dominance. The impact on the genetic variance of mutants with a deleterious pleiotropic effect on fitness with bottlenecking depends greatly on their degree of dominance for fitness (Figure 4). The relation between h and s used in this and other studies (Caballero and Keightley 1994; Wang et al. 1998), however, is supported by only a few available data (Caballero and Keightley 1994; Deng et al. 2002). More data are needed to establish the relationship between the effects and dominance of mutant genes to better investigate the maintenance of genetic variance within equilibrium populations and the consequence of a population bottleneck.

Although we focus here on the effects of a rapid bottleneck of two individuals, other numerous cases (e.g., one bottleneck of more individuals, two generations of four individuals, …) can be analyzed in the same way. Under strong selection and with all mutant genes assumed to have the same dominance coefficient, for example, the changes in additive and between-line variance after a bottleneck of NF individuals are given by VAF,T/VA0,T = (1 − 1/2NF)[1 − 1/NF + 1/(2NFh′)]2 and VBF,T/VA0,T = (1/NF)[1 − 1/2NF + 1/(4NFh′)]2, respectively, using the formula listed in Table 1. As the size of the bottleneck increases, its effect on variation reduces. In general, the bottleneck effect can be scaled approximately by F (inbreeding coefficient) if purging selection and new mutations are ignored, irrespective of the details of population size and number of generations (Robertson 1952; Crow and Kimura 1970; Wang et al. 1998; pez-Fanjul et al. 2002, 2003). Our predictions of the changes in genetic variances after a bottleneck apply when the population expands immediately (in one generation) to a large size from the severe bottleneck and thus selection is completely ignored. In practice some empirical observations on the increase in additive genetic variance due to bottlenecking were obtained following two or more generations of inbreeding and a relatively slow expansion of the population (due to the reproductive constraint of the organism). Thus purging selection could be involved, particularly against lethal and semilethal mutations that would, if they survived a bottleneck of two individuals, be at a frequency of 1/4 before selection rapidly decreased their frequencies. With purging selection, the additive variance of the fitness trait is increased by less than our predictions, but even so the amount is limited (see reviews by Roff 2002 and Keller and Waller 2003). The additive variances of the metric trait may be little affected since it is predicted to derive mostly from mutants of small pleiotropic effects.

In conclusion, our analysis shows that mutant genes that produce variability in fitness traits are quite different from those causing variability in metric traits that are not to be strongly correlated with fitness. Mutants of large pleiotropic effects on fitness, which are likely to be highly recessive for fitness and segregate at very low frequencies in equilibrium populations, mainly determine the variance of fitness traits in equilibrium populations and the inbreeding depression and changes in variance following a bottleneck. Mutants of small pleiotropic effects on fitness, which are at most partially recessive for fitness, are not under strong selection and are likely to segregate at intermediate frequencies. Those mutant alleles largely determine the properties of the metric traits (Zhang et al. 2004). The joint-effect model of pleiotropic and real stabilizing selection is obviously itself a simplification in that epistasis and other complications such as genotype × environment interaction are ignored, but nevertheless provides an interpretation for observations on genetic variation in both fitness and metric traits maintained in natural populations. Furthermore, with specific combinations of mutation parameters it can also predict the changes observed experimentally in genetic variance for both types of traits and in inbreeding depression after population bottlenecks. This investigation therefore provides further support to our conclusion that mutation-selection balance is a simple and plausible mechanism for maintenance of quantitative genetic variation in natural populations.



The population bottleneck generates LD due to sampling. This does not affect mean performance in the replicate populations (lines) subsequently established if there is no epistasis or selection, but can influence both the mean and the variability among replicates of the components of genetic variance.

Expectation of within-line variance:

We use results of Avery and Hill (1979), but with the notation modified. The mutant gene at locus i has frequency pi, and for the quantitative trait, for example, the homozgote wild type, the heterozygote, and the mutant homozygote have genotypic values 0, aihi, and ai, respectively. The average effect of the gene substitution is Math and the excess of the heterozygote over the homozygote mean is Math. The genotypic variance within lines is MathA1where i, j = 1, … , n, for n loci, and Dij is the LD coefficient at loci i and j. Because of LD, additive genetic and dominance terms are not orthogonal, but it seems appropriate to take 4 × (covariance of half sibs) as a practical predictor of the VA. Hence the additive variance is MathA2The terms in αiαjDij in (A2) comprise second moments, e.g., E(Dij); third moments, e.g., E[(1 − 2pi)Dij]; and fourth moments, e.g., E[(1 − 2pi)(1 − 2pj)Dij]. In a population previously in linkage equilibrium, these moments are all zero, and if there is no selection or other directional force, remain so in subsequent generations. E[(1 − 2pi) (1 − 2pj)Dij] ≠ 0 after the first generation if there is further drift, but not if the population is rapidly expanded. Hence we assume there is no LD contribution to the additive genetic variance; i.e., Var(A) = VAF, where VAF is the variance from individual loci (Table 1).

As it is not usually feasible to partition the residual genetic variance further, the residue of Var(W) after removing Var(A) can be regarded as dominance variance. From (A1) and (A2) MathA3The terms in pairs of loci δiδjD2ij are fourth moments as the δi are constants, and EMath ≠ 0 due to sampling. After one generation in a bottleneck of size NF, MathA4(Hill and Robertson 1968). The first term in (A3) is the contribution from individual loci, VDF (Table 1). Hence, assuming for simplicity NF = 2, MathA5The expectation in (A5) is over the distribution of effects of loci and gene frequencies in the base population, and those at different loci are assumed uncorrelated. The quantity E{∑iipi0(1 − pi0)} = BT is the expected inbreeding depression in the metric trait following complete inbreeding without any selection (see also Avery and Hill 1979) or the derivative of inbreeding depression to inbreeding coefficient [for fitness, the relevant quantity is given by BW from (16), using the first derivative at F = 0]. Hence the expectation of a random cross product term in (A4) is (BT/n)2, and MathA6assuming that many loci influence the trait. Equation A6 is in terms of quantities that have already been computed.

In subsequent generations in a large expanded population, the LD coefficients at loci i and j decline through recombination in proportion to (1 − cij)2, where cij is the recombination fraction. To compute the consequent decline in variance, assumptions have to be made of no further mutation or selection and about the distribution of loci affecting the trait. The simplest assumption, if not the most realistic, is that such loci are uniformly distributed throughout the genome, such that the decline in variance is proportional to the mean of Math2D2ij across the genome, regardless of effects on the trait. Hence, at generation t after the bottleneck, let MathA7where y1 = 1. For mammals with, say, 20 chromosomes each of length 1 M (a “mouse”), 95% of pairs of loci are on different chromosomes, so an adequate approximation is yt = (1/4)t−1. An extreme example is D. melanogaster, which has in effect three chromosomes of approximately equal length (each, say, 1 M), and no crossing over in males. Hence, for loci l map units apart and assuming Haldane's mapping function, (1 − cij)2 = [1/4(3 + e−2l)]2, the overall coefficient is Mathwhere El[(3 + e−2l)2] is obtained by integrating l over all pairs of sites on the chromosome assuming a uniform distribution.


If mutational effects are symmetrically distributed, as we have assumed for the metric trait, then BT = 0 and there is no contribution from LD to the dominance variance. Hence we consider the contributions to the dominance variance in fitness (in which a and h′ are replaced by s and h in the above equations) and a specific example. Assume a and s follow gamma (1/4) and (1/2) distributions with means Math, Math, and λ = 0.3. For the fitness trait, BW = −3.0 and VDF = 0.176; thus Var(D)t = 0.176 + 1.68yt. For t = 2, 4, and 6, and a mouse, yt = 0.25, 0.016, and 0.001; and for D. melanogaster, yt = 0.43, 0.19, and 0.13, respectively.

Extensions and comments:

If the populations after the bottleneck are small (effective size M) such that further sampling from drift can induce LD, approximate results can be obtained from MathA8(an approximation to the loss through drift and recombination and the gain through drift from reduced heterozygosity; see Avery and Hill 1977). For free recombination, if M is moderately large, Dt2 → 2/Math. Providing M > 10 and t > 3 in the example, LD contributes less than individual loci for the case of mammals.

An important proviso, regardless of the population size, is that there is negligible variance of inbreeding coefficient in the populations after the bottleneck (Avery and Hill 1979). If, for example, some individuals are offspring of full sibs, the variance increases markedly, in proportion to the product (variance in pedigree inbreeding) and (inbreeding depression in the sublines). Thus mating relatives should be avoided as far as possible within the lines when estimating dominance variance.

Variance of within-line variance:

The expectation of Var(A) is given in (A2), and in the special context of a population that rapidly expands, simplifies, and does not involve LD. In the absence of dominance, the expectation of the variance is a second moment in gene frequencies, and the variance of the variance a tractable fourth moment (Avery and Hill 1977). In the presence of dominance, there is no reason to assume that because the fourth moment E[(1 − 2pi)(1 − 2pj)Dij] is zero, so is the corresponding eighth moment. Evaluation is not impossible, but would be very tortuous (see Hill and Weir 1988). In view of the small contribution of this fourth moment and thus potentially of its square, we assume the contribution is merely a function of the heterozygosity so as to get some feel for the magnitude of terms, with results holding more precisely for additive gene action. We use results of Avery and Hill (1977), who expressed the variation as a simple function of initial additive variance and population size. From (A2) Mathwhere the first term is that for single loci (Table 1). Using similar arguments as for the dominance case, since Math, to obtain the expectation of the variance of the sum of the contributions and assuming many loci as in (A6) MathA9where VA0 is the additive variance in the base population. Equation A9 has a form similar to that given by Avery and Hill (1977)(except they ignored the single-locus sampling term) and formally holds only for additive gene action. If the population remains of finite size, the term in VA02 does not disappear (cf. A8), and for unlinked loci asymptotes at 2VA02/3M (Avery and Hill 1977).


For the metric trait in the example discussed above, VA0 = 0.5 and V(VAF) = 0.003. Hence Var[Var A] ≅ 0.003 + 0.094yt. The LD term dominates.


We are grateful to two reviewers for helpful comments. This work was supported by a grant from the Biotechnology and Biological Sciences Research Council (15/G13242).


  • Communicating editor: J. B. Walsh

  • Received December 16, 2003.
  • Accepted April 5, 2004.


View Abstract