## Abstract

Although the distribution of frequencies of genes influencing quantitative traits is important to our understanding of their genetic basis and their evolution, direct information from laboratory experiments is very limited. In theory, different models of selection and mutation generate different predictions of frequency distributions. When a large population at mutation-selection balance passes through a rapid bottleneck in size, the frequency distribution of genes is dramatically altered, causing changes in observable quantities such as the mean and variance of quantitative traits. We investigate the gene frequency distribution of a population at mutation-selection balance under a joint-effect model of real stabilizing and pleiotropic selection and its redistribution and thus changes of the genetic properties of metric and fitness traits after the population passes a rapid bottleneck and expands in size. If all genes that affect the trait are neutral with respect to fitness, the additive genetic variance (*V*_{A}) is always reduced by a bottleneck in population size, regardless of their degree of dominance. For genes that have been under selection, *V*_{A} increases following a bottleneck if they are (partially) recessive, while the dominance variance increases substantially for any degree of dominance. With typical estimates of mutation parameters, the joint-effect model can explain data from laboratory experiments on the effect of bottlenecking on fitness and morphological traits, providing further support for it as a plausible mechanism for maintenance of quantitative genetic variation.

THE effective size of a natural population is an important parameter in determining its genetic and evolutionary processes. It may fluctuate substantially rather than maintain a large and constant value as assumed in many models. The changes in the mean and variability of fitness and other metric traits when a large population at equilibrium suffers a sudden decrease in size (a “bottleneck”) are important to the understanding of nonequilibrium natural populations with frequent bottlenecks or extinction and recolonization. Laboratory experiments have been carried out to study the impact of population bottlenecks (Frankham 1981; Bryant* et al*. 1986; López-Fanjul and Villaverde 1989; López-Fanjul* et al*. 1989; García* et al*. 1994; Fernández* et al*. 1995; Meffert 1995; Wade* et al*. 1996; Fowler and Whitlock 1999; Whitlock and Fowler 1999; Saccheri* et al*. 2001; Roff 2002), providing data that are useful in our understanding of the impact of bottlenecks and in testing for genetic models of the maintenance of genetic variation in populations.

Experiments show that a sudden decrease in population size, due to either artificial selection or a bottleneck, can bring about dramatic changes in the genetic properties of the population. As a population shrinks in size, gene frequencies disperse and homozygote frequencies increase, resulting in consequent changes in other genetic properties such as the means and variances of fitness and other quantitative traits. In laboratory experiments, the genetic variance is observed to increase dramatically for life-history traits that are closely related to fitness (López-Fanjul and Villaverde 1989; García* et al*. 1994; Fernández* et al*. 1995; Meffert 1995) and to decrease for morphological traits (López-Fanjul* et al*. 1989; Wade* et al*. 1996; Whitlock and Fowler 1999; Saccheri* et al*. 2001) immediately after a brief period of inbreeding or bottlenecking. Two hypotheses have been proposed to explain the increased genetic variance following bottlenecks: dominance (Robertson 1952; Willis and Orr 1993) and epistasis of gene action (see Goodnight 1988; López-Fanjul* et al*. 2002), *i.e.*, interactions within and between loci, respectively. For viability in Drosophila, the observed increase in additive genetic variance (*V*_{A}) and the appearance of inbreeding depression after a bottleneck can be explained by mutation-selection balance (MSB), assuming values of the mutation parameters estimated from empirical studies (Wang* et al*. 1998). The decrease in *V*_{A} for morphological traits is expected if they are controlled mainly by additive genes (Falconer and Mackay 1996).

Our recent investigations (Zhang and Hill 2002; Zhang* et al*. 2004) show that a joint-effect model of pleiotropic and real stabilizing selection provides a plausible explanation for the high levels of genetic variance observed for quantitative traits under strong apparent stabilizing selection in natural populations. An effective way to further test the validity of the joint-effect model is to consider the change of genetic architecture due to a sudden change in demography of the population (*e.g.*, a bottleneck). One key assumption of the joint-effect model, as that of Turelli (1985), is that mutant alleles under strong selection are always very rare, so that a severe reduction in population size causes such mutants either to become instantly quite common or to be lost from the populations. Such a dramatic change in genetic architecture makes the degree of nonadditivity of gene action an important parameter in controlling the behavior of the genetic system. A comparison of the substantial quantity of experimental data on the effect of a bottleneck on genetic variance with predictions based on the joint-effect model enables a further check on its validity.

Previous theoretical studies of the effects of population bottlenecks using models including dominance have either considered a single locus (Robertson 1952; Willis and Orr 1993) or assumed an equilibrium population under a pure pleiotropic model without real stabilizing selection (Wang* et al*. 1998). In this article, we generalize the results to assume a population at MSB under the joint-effect model of pleiotropic and real stabilizing selection at many loci (Zhang and Hill 2002; Zhang* et al*. 2004) and predict the changes in mean and variability of fitness and other metric traits expected after a rapid bottleneck in population size and subsequent rapid increase in size. These predicted changes are compared with available observations from different experiments.

## MODEL AND METHODS

### The base population:

A random mating population of *N* diploid (monoecious) individuals is assumed to have an effective size *N*_{e} and be at Hardy-Weinberg equilibrium. Following the notation and analysis of Zhang* et al.* (2004), it is assumed that, relative to a homozygote of the preexisting wild-type allele, mutants have a selective disadvantage due to their pleiotropic effect on fitness of *s* in homozygotes and *sh* in heterozygotes and have effects on the metric trait of interest *a* in homozygotes and *ah*′ in heterozygotes. In the equilibrium base population, selection acts on the locus both as a direct pleiotropic effect on fitness and through real stabilizing selection of strength *V _{s}*

_{,r}on the metric trait. No overdominance, no epistasis, and no linkage disequilibrium in the base population are assumed. Both mutational effects and their degrees of dominance vary among loci and are assumed to follow a quadrivariate distribution

*P*(

*a*,

*h*′

*s*,

*h*). For simplicity, the marginal distribution of mutant effects on the metric trait is assumed to be symmetrical about zero and effects

*a*and

*s*are assumed to be independent, but |

*a*| and

*h*′, and

*s*and

*h*, are both assumed to be negatively correlated in accordance with empirical data (Caballero and Keightley 1994). In this article, both fitness traits (

*e.g.*, viability) and metric traits (

*e.g.*, morphology) are considered, with the latter related to fitness only through real stabilizing selection acting directly on the trait. At mutation-selection balance, it is assumed the mean fitness is expected to remain the same. Frequencies of genes of large effect cannot increase substantially due to strong selection, while the reduction in fitness due to fixation of genes of small effect may be counteracted by the fixation of some rare favorable genes. Thus the mutation model can be considered as a fixed model (Ohta and Tachida 1990).

In the absence of epistasis, the stationary distribution of allele frequency at the steady state of accumulation and loss of mutations is given by Kimura's (1969) diffusion approximation, 1where λ is the haploid genome-wide mutation rate; *x* is the frequency of a mutant gene with effects (*a*, *h*′, *s*, *h*) segregating in the population [1/2*N* ≤ *x* ≤ 1 − 1/2*N*], ; the probability of ultimate fixation of a mutant with initial frequency *x* is ; and the overall selection coefficient *s̃* determines the change in mean frequency through Δ*x* = −*x*(1 − *x*)*s̃*/2. In the joint-effect model, the overall selection coefficient is 2Zhang* et al.* 2004).

For some special situations, the gene frequency distribution in a large equilibrium population can be dealt with analytically. For a neutral gene (*i.e.*, *V _{s}*

_{,r}→ ∞ and

*s*= 0), its frequency distribution depends on the mutation rate and the genetic drift caused by finite population size and is 1aFor a mutant gene strongly selected against , its frequency distribution can be approximated by 1bin which the overall selective value is approximated by 2′(Zhang

*et al*. 2004). If the effects of the mutant on the metric trait are large and real stabilizing selection is strong (

*i.e.*,

*V*

_{s}_{,r}is small), the gene frequency distribution depends on both real stabilizing selection and pleiotropic selection; otherwise its pleiotropic effects on fitness (direct selection) determine the distribution. Mutants that are (partially) recessive for fitness in heterozygotes have a chance to build up to intermediate frequencies, especially in a small population because selection is not very effective in eliminating them (Zhang

*et al*. 2004). The total number of segregating loci is given by (from Kimura 1969). The gene frequency distribution conditional on its effect on the metric trait, 3depends greatly on the mutational effect distribution

*P*(

*a*,

*h*′,

*s*,

*h*). Distributions (1) and (3) are L-shaped. In practice, as we can identify genes only by whether they increase or decrease the metric trait, rather than by whether or not they are mutant genes, the distribution of gene frequency with respect to the metric trait is U-shaped, under the assumption of symmetrical positive and negative effects of mutants on it. From this perspective, the frequency distribution of a neutral gene can equivalently be written as 2

*N*

_{e}λ/[

*x*(1 −

*x*)] (Crow and Kimura 1970). For fitness, the distribution is always L-shaped, as all mutants are assumed deleterious.

The expectation *I _{f}* of an arbitrary function

*f*(

*x*;

*a*,

*h*′,

*s*,

*h*) with respect to the equilibrium distribution φ(

*x*) for a mutant gene is given by . The corresponding measurable quantity, summing over all loci, is obtained by the integration of 4 The following quantities are useful in evaluating the bottleneck effect,

*H*= 2

*x*(1 −

*x*),

*C*= 2

*x*(1 −

*x*)(1 − 2

*x*),

*K*=

*H*

^{2}= [2

*x*(1 −

*x*)]

^{2}; and their expectations are given subscripts 0 or F to refer to the base or bottlenecked populations, respectively. To simplify subsequent formulas,

*d*= 2

*h*− 1 and

*d*′ = 2

*h*′ − 1 are used to represent the dominance effects. The expressions for additive variance (

*V*

_{A}), dominance variance (

*V*

_{D}), and genetic variance (

*V*

_{G}) contributed by a mutant gene with effects (

*a*,

*h*′,

*s*,

*h*) in the equilibrium base population are listed in Table 1.

The total number of segregating mutants at mutation-selection-drift balance [*i.e.*, *L*_{0} = *E*(*I _{f}*

_{=1})] is approximated by 4

*N*

_{e}λ[ln(2

*N*) + 1] and 4

*N*

_{e}λ[ln(1/

*s̃*) + 0.423] (Kimura 1969) in the extreme cases of neutral and strongly selected genes, respectively; otherwise,

*L*

_{0}lies between those two extreme values. The average frequency of mutant genes is calculated as . Assuming a multiplicative model for fitness, the mean fitness caused by all segregating loci is , where

*S*

_{p}(

*x*;

*s*,

*h*) = 2

*hsx*(1 −

*x*) +

*sx*

^{2}and

*S*

_{r}(

*x*;

*a*,

*h*′) = (

*a*

^{2}/8

*V*

_{s}_{,r})[(1 +

*d*′

^{2})

*H*

_{0}+ 2

*d*′

*C*

_{0}−

*d*′

^{2}

*K*

_{0}] are the fitness reductions due to the pleiotropic and real stabilizing selection against a mutant with effects

*a*,

*h*′,

*s*,

*h*, and of frequency

*x*, respectively. With strong selection, the mean fitness due to pleiotropic effects can be approximated by

*e*

^{−λ}for complete recessives and

*e*

^{−2λ}for others (Haldane 1937). It should be noted that the multiplicative model in general generates epistasis (Wade

*et al*. 2001) on the linear scale, but such epistatic interaction is negligible except for genes of very large effect.

### Population after bottlenecking:

Assume that a number of replicate lines of size *N*_{F} are drawn at random from a large equilibrium population (*i.e.*, the base population). After one generation of reproduction at size *N*_{F}, each line is assumed to expand immediately to a large size with random mating so that no subsequent inbreeding or genetic drift occurs and the population is restored to Hardy-Weinberg equilibrium. We assume no selection during and immediately after the bottleneck in population size and ignore for the time being linkage disequilibrium induced by the bottleneck. Therefore we can use binomial sampling to obtain the redistribution of mutant frequencies. The probability that a mutant has *n* copies (for *n* = 0, 1, … , 2*N*_{F}) in bottlenecked populations such that its frequency is *n*/2*N*_{F} is given by 5(Crow and Kimura 1970; Wang* et al*. 1998). Bottlenecking disperses the gene frequencies around their means: the deleterious genes will be lost from many loci and increased in frequency at others, such that the average frequency of those remaining segregating is increased substantially. For strongly selected mutants in the equilibrium base population, the average frequency of those still segregating jumps to a little over 1/2*N*_{F}, while segregating neutral genes have one, two, and three copies with probabilities 55, 27, and 18%, respectively, following a bottleneck of two individuals.

With this new distribution of gene frequencies, the expectation of any quantity *f*(*x*) after bottlenecking is evaluated as 6and other parameters are obtained by substituting *I _{f}*

_{,F}in (4). After a bottleneck of one generation, binomial sampling (Equation 5) gives 7(

*cf*. Robertson 1952; Silvela 1980), where

*H*

_{0}= 2

*p*(1 −

*p*),

*C*

_{0}= 2

*p*(1 −

*p*)(1 − 2

*p*), and

*K*

_{0}= 4

*p*

^{2}(1 −

*p*)

^{2}for a mutant gene of initial frequency

*p*, and

*f*= 1 −

_{i}*i*/2

*N*

_{F},

*i*= 1, 2, 3. Thus the heterozygosity is reduced by 1/2

*N*

_{F}regardless of the number of alleles, their frequencies, their fitness effects, and the degree of dominance (

*cf*. Crow and Kimura 1970, pp. 327–331). The quantity

*E*(

*K*

_{F}) does not change proportionately, however, with important consequences on the components of genetic variance following the bottleneck (see formulas in Table 1).

After the bottleneck, the additive genetic variance becomes, as a proportion of *V*_{A0}, 8As *f _{i}* < 1 (

*i*= 1, 2, 3), (8) shows that

*V*

_{AF}<

*V*

_{A0}if

*d*′ > 0 (

*i.e*.,

*h*′ ≥ 0.5), but

*V*

_{A}can increase if

*d*′ < 0 (

*i.e.*,

*h*′ < 0.5) since bottlenecking largely reduces the negative contributions (especially the item containing

*C*

_{0}). Even for partial recessive mutants (

*h*′ < 0.5), the change in

*V*

_{A}depends on the relative values of

*H*

_{0},

*C*

_{0}, and

*K*

_{0}, which in turn depend on the dynamic process in the base population. In the extreme case of a bottleneck of two individuals, (8) reduces to 8′Figure 1a shows that bottlenecking can increase

*V*

_{A}from (partially) recessive mutants, the amount of increase depending on both the degree of dominance and the initial frequencies of mutants (Willis and Orr 1993). If the mutant gene has an initial frequency >0.43, however,

*V*

_{A}contributed by it always falls, regardless of its dominance. The dominance variance becomes, as a proportion of

*V*

_{D0}, 9If

*N*

_{F}= 2, (9) reduces to

*V*

_{DF}/

*V*

_{D0}= {9/[2

*p*(1 −

*p*)] + 3}/32 and the dominance variance will increase when either

*p*< 0.193 or

*p*> 0.807. Interactions between genes at different loci (

*i.e*., epistasis) complicate the simple relationships given here (see López-Fanjul

*et al*. 2002, 2003).

Due to sampling, there will be variance among replicate bottleneck lines in both the mean performance of each line (described by the between-line variance, *V*_{BF}) and the additive genetic variance within each line [described by *V*(*V*_{AF})]. Intuitively, both can be large if rare recessive genes mainly determine performance (see Figure 1, b and c). Assuming no selection during a bottleneck, *E*(*x*_{F}) = *E*(*x*) = *p* and , and the expression for *V*_{BF} can be obtained (see Table 1). If genes act additively, *V*_{BF}* = V*_{A0}*/N*_{F} (Crow and Kimura 1970). Variance among bottlenecked lines in additive variance is defined as *V* ≡ *E* − *E*^{2}. For lines that pass through one generation of a bottleneck of two individuals, the expression for *V*(*V*_{AF}) is obtained (see Table 2). For the additive cases, (*cf*. Avery and Hill 1977; see also the appendix).

On the basis of his analysis of single recessive genes, Robertson (1952) concluded: “If the recessives are at low frequency, the variation within lines increases …” (see also Willis and Orr 1993). Many genes control quantitative traits and their frequencies in natural populations must follow distributions that depend on the demographic and dynamic processes as discussed above and that we need to consider. In this study, overdominance and epistasis are ignored. Epistasis may play a role in the maintenance of genetic variance in an equilibrium population and in their changes after bottlenecks (Goodnight 1988; Gavrilets and De Jong 1993; Naciri-Graven and Goudet 2003). However, evidence for epistasis affecting a quantitative trait is scarce (Barton and Turelli 1989; Barton and Keightley 2002) and the epistatic variance, when found, is often small relative to *V*_{A} (*e.g.*, Paterson* et al*. 1990). At least epistasis is not the primary cause of an increase in *V*_{A} following bottlenecks (López-Fanjul* et al*. 2002). Although overdominance may be important in some situations in the maintenance of genetic variation and in causing inbreeding depression, it is unlikely to be common (Charlesworth and Charlesworth 1999). For simplicity, we also ignore linkage disequilibrium induced by the bottleneck, but its effects on changes in genetic parameters are summarized in the appendix and reviewed in the discussion. In what follows, we obtain analytical approximations for extreme situations and then use single-locus Monte Carlo simulation (Zhang* et al*. 2004) to explore more general situations. Finally we compare the theoretical predictions of our model, using typical estimates of mutation and selection parameters, with laboratory experimental data on the bottleneck effect.

## ANALYTICAL APPROXIMATIONS

Let us consider first the situation where all mutant genes have the same dominance coefficient and are either neutral or strongly selected against, using diffusion theory to obtain approximations for genetic parameters in a large equilibrium population and then their changes due to a bottleneck of two individuals.

### Neutrality:

Under the assumption of neutrality, we consider only traits that are not associated with fitness. If the base population is at mutation-drift balance, the following diffusion approximations hold: *H̅*_{0} = 4*N*_{e}λ and , valid for any degree of dominance (Kimura 1969). With these approximations, expressions for genetic variances before and after a bottleneck of two individuals can be obtained (see Table 2). It is clear that *V*_{AF}/*V*_{A0} = [15 + 3(2*h*′)^{2}]/[16 + 8(2*h*′)^{2}] <1 and *V*_{DF}/*V*_{D0} = 15/16. Thus a bottleneck in population size leads to an increase in *V*_{A} and *V*_{D} at individual loci if the recessive genes are initially at low frequency (see Equations 8′ and 9), but to an overall decrease in *V*_{A} and *V*_{D} over the whole genome if the gene frequencies are at mutation-drift equilibrium. Hence, for neutral genes in the absence of epistasis and overdominance, bottlenecking cannot increase *V*_{A} and *V*_{D}. The between-line variance can be scaled as *V*_{BF}/*V*_{A0} = 3[1 + 2*h*′ + (1 − 2*h*′)^{2}/2]/[8 + 4(2*h*′)^{2}] < 1. As shown in Figure 2, the prediction for additive genes, *V*_{BF} = 0.5*V*_{A0} (Wright 1951), is also a good approximation for any degree of dominance (this still holds even in the case of dominance and epistasis; see López-Fanjul* et al*. 2002, 2003), whereas that for additive variance, *V*_{AF} = 0.75*V*_{A0}, is not. Scaling the variability between bottlenecked lines in additive genetic variance as their coefficient of variation, we have 10where κ_{4} = *E*(*a*^{4})/*E*(*a*^{2})^{2} describes the leptokurtosis of the distribution of mutational effects on the metric trait. As shown in inequality (10), the maximum of cv(*V*_{AF}) increases with leptokurtosis of mutational effects and decreases with effective size of the base population and the mutation rate. For a base population of effective size *N*_{e} = 10^{4} and genome-wide mutation rate λ = 0.1, the maximum of cv(*V*_{AF}) is 0.036 if κ_{4} = 3 (*e.g.*, normally distributed effects) and 0.10 if κ_{4} = 23.4 [*e.g.*, mutational effects follow a reflected gamma (^{1}/_{4}) distribution]. Even allowing for the variation in *V*_{A} among the bottlenecked lines, it may not be possible for *V*_{A} within a line to increase following a bottleneck of two individuals under the assumption of realistic values of rate and effect of mutations. However, if the contribution due to linkage disequilibrium induced by a bottleneck is included, cv(*V*_{AF}) can be much larger, especially just after bottlenecking (see Avery and Hill 1977; also appendix) and therefore *V*_{AF} in a bottlenecked line may have at least a transitory chance of exceeding *V*_{A0}.

### Strong selection:

First we consider partially recessive mutant genes for fitness. If 2*Ns̃* ≫ 1, the diffusion approximations (Kimura 1969) show that and with the effective selection coefficient *s̃* given by (2′), while the expectations of higher moments are zero, due to the rarity of mutant genes under strong selection (see Zhang* et al*. 2004).

Let the additive variance for the metric trait after a bottleneck of two individuals be *V*_{AF,T}. Under strong selection, the contribution of a mutant gene can be simplified to *V*_{AF,T} = (3/4)[(*h*′ + ^{1}/_{2})*a*]^{2}*H*_{0}/4. A partially recessive mutant for the metric trait appears almost exclusively in heterozygotes in the base population; if it survives bottlenecking, however, its frequency rises to at least ^{1}/_{4}, and it contributes substantially to variability. Equivalently, the “effective” dominance of (partially) recessive mutants increases to Integrating over all loci using (4), we can predict the contribution of all loci (see Table 2) to *V*_{AF,T} for the metric trait. With the assumption of constant dominance among loci, the change in *V*_{A} following a bottleneck is 11Bottlenecking increases *V*_{A,T} when *h*′ < 0.38 (*cf*. Willis and Orr 1993) and such an increase is substantial as *h*′ approaches 0 (see Figure 2), very different from the above neutral gene case. The between-line variance is 12If *h*′ = ^{1}/_{2}, *V*_{BF,T} = *V*_{A0,T}/2. For more recessive mutant genes, *V*_{BF,T} can exceed the additive genetic variance in the base population (see Figure 2). Under strong selection and with (partial) recessivity, changes in *V*_{AF,T} and *V*_{BF,T} differ dramatically from the predictions for additive genes. Equations 11 and 12 are also valid for fitness traits by replacing *h*′ by *h*.

In a large equilibrium population, the dominance variance of the fitness trait (denoted by subscript *W*) is very small, *V*_{D0,}* _{W}* ∝ 1/

*N*(

*cf*. Figure 3 of Wang

*et al*. 1998), because (partially) recessive genes are found predominantly in heterozygotes (Zhang

*et al*. 2004). Because bottlenecking can dramatically increase the frequencies of some mutant genes,

*V*

_{D,}

*may increase accordingly. If there is much stronger pleiotropic selection than real stabilizing selection (*

_{W}*i.e.*, 2

*hs*≫ (2

*h*′

*a*)

^{2}/4

*V*

_{s}_{,r}, hence

*s̃*≅ 2

*hs*) the dominance variance of the fitness trait following a bottleneck is approximated by 13where

*s̅*

_{p}is the mean pleiotropic effect of mutations on fitness. Thus the dominance variance increases substantially such that

*V*

_{DF,}

*is nearly independent of the size of the base population. The expression for*

_{W}*V*(

*V*

_{AF,}

*) (see Table 2) shows that variation among bottlenecked lines in additive variance is contributed by the more recessive mutants. If*

_{W}*s̃*≅ 2

*hs*, its coefficient of variation (CV) can be approximated by 14As in the case of neutrality, cv(

*V*

_{AF,}

*) decreases with an increasing mutation rate; it also depends on selection intensity and skew of pleiotropic effects of mutations. In contrast to*

_{W}*V*(

*V*

_{AF,}

*), cv(*

_{W}*V*

_{AF,}

*) is smaller for more recessive mutants (see also Figure 4f). When the pleiotropic effects of mutations follow a gamma (β) distribution, we have 14′or cv ≅ 2 for a gamma (*

_{W}^{1}/

_{2}) distribution. [For the metric trait, simple approximations like (13) and (14) have not been obtained.]

In nature, a bottleneck in population size may usually accompany or be induced by a change in environment (Whitlock and Fowler 1999; Keller and Waller 2003). If such a change is not large or does not greatly affect the expression of gene effects, it is feasible to analyze the change in mean fitness. Mutational effects on the metric trait are assumed to be symmetric and appear more or less additive such that fitness due to stabilizing selection on the metric trait changes little following a bottleneck. Most of the reduction in fitness is due to the direct effects of mutants on fitness, so we need consider only these. Under strong selection (2*Ns̃* ≫ 1), *E*_{F}(*x*^{2}) = λ/2*s̃* and *H*_{F} = 3λ/*s̃* approximately, so the mean fitness *due solely to pleiotropic effects*, , is approximated by 15The inbreeding depression in terms of the number of lethal equivalents is thus approximated by 16(Charlesworth and Charlesworth 1999; Kirkpatrick and Jarne 2000), where *F* is the inbreeding coefficient. If the mutant gene acts additively, there is no inbreeding depression and the mean fitness after the bottleneck remains the same as in the large base population, exp(−2λ) (Haldane 1937). Inbreeding depression increases with the degree of recessivity and the number of mutant genes. [Stabilizing selection on the metric trait could increase inbreeding depression more than indicated by (16), as its mean may change away from its optimum value due to genetic drift.]

Now consider completely recessive mutant genes for fitness, which, with strong selection, are at low frequency, approximately (Haldane 1927). With approximations *V*_{A0,}* _{W}* =

*E*[2

*s*

^{2}

*p*

^{3}],

*V*

_{D0,}

*=*

_{W}*E*[

*s*

^{2}

*p*

^{2}],

*V*

_{AF,}

*=*

_{W}*E*[(3/2)

*s*

^{2}

*p*],

*V*

_{DF,}

*=*

_{W}*E*[9

*s*

^{2}

*p*/64],

*V*

_{BF,}

*=*

_{W}*E*[

*s*

^{2}

*p*/64],

*v*

_{F,}

*= 1 −*

_{W}*E*[

*sp*]/4,

*V*(

*V*

_{AF,}

*) =*

_{W}*E*[81(

*s*/2)

^{4}(3/4)

^{2}

*p*/16], the effects of a bottleneck of two individuals are characterized by the following relationships with the strength of pleiotropic selection: 17Comparison of Equation 17 with Equations 11–16 shows that completely and partially recessive genes behave quite differently. For instance, approximations for both

*V*

_{AF,}

_{W}/V_{A0,}

*and*

_{W}*V*

_{BF,}

*/*

_{W}*V*

_{A0,}

*are independent of selection intensity for partially recessive mutants but proportional to selection intensity for completely recessive mutants.*

_{W}## NUMERICAL RESULTS

### Distribution of gene frequencies:

Figure 3 shows the distributions of gene frequencies in a large equilibrium population under different selection models. Under the pure pleiotropic selection model (*i.e.*, *V _{s}*

_{,r}→ ∞ and

*s̃*=

*s*), mutant gene frequencies are usually very low such that only a small genetic variance of a metric trait (

*V*

_{G,T}) can be maintained if genes have the same fitness effect (Barton 1990). If

*s*is assumed to follow a leptokurtic distribution, however, some mutant genes with large effects on the metric trait can be found at high frequencies so that

*V*

_{G,T}increases without bound as population size increases (Keightley and Hill 1990). Both behaviors seem to conflict with empirical observations. With the joint-effect model, mutant genes having large effects on the metric trait but small pleiotropic effects on fitness undergo sufficient selection to reach moderate but not high frequencies. Thus the joint-effect model can predict genetic variances for metric traits that are compatible with the observations (Zhang and Hill 2002; Zhang

*et al*. 2004).

### Constant dominance:

Numerical results were obtained from simulations to analyze more general models and to check the validity of the above analytical approximations. We first assume mutant genes have *constant* dominance coefficients but varying effects on the metric trait and varying pleiotropic effects on fitness. Figure 4 gives simulation results for a range of selection intensities and results from diffusion approximations for the cases of neutrality and strong selection. For nearly neutral mutant genes, their contributions to the changes in genetic parameters after a bottleneck are nearly independent of their degrees of dominance. As selection increases, mutant genes of large effects within the equilibrium base population become rare, magnifying the differences in bottleneck effect caused by degrees of dominance (Figure 4, a–f). For partially recessive genes under strong or very weak selection, predictions from the diffusion approximation are close to those from simulations. As the selection coefficient becomes larger, mean fitness reduces while the number of lethal equivalents (*B _{W}*), relative increase in additive variance (

*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*), dominance variance (*

_{W}*V*

_{DF,}

*/*

_{W}*V*

_{D0,}

*), between-line variance (*

_{W}*V*

_{BF,}

*/*

_{W}*V*

_{A0,}

*), and the CV of additive variance [cv(*

_{W}*V*

_{AF,}

*)] increase. These results are similar to those from the pure pleiotropic selection model (Figure 4 of Wang*

_{W}*et al*. 1998).

Completely recessive mutant genes behave quite differently (Figure 4). As selection becomes weak, fitness contributed by them decreases to a minimum when *N**s̅*_{p} > 1 and then increases to approach unity, in contrast to the monotonic increase of fitness contributed by partial recessives. This reflects the conflict between the drift of heterozygotes and strong selection on homozygotes of complete recessives. Complete recessives can cause huge relative increases in *V*_{AF,}* _{W}*/

*V*

_{A0,}

*and*

_{W}*V*

_{BF,}

*/*

_{W}*V*

_{A0,}

*and also a large inbreeding depression (*

_{W}*B*), but contribute much less to the relative increase in

_{W}*V*

_{DF,}

*/*

_{W}*V*

_{D0,}

*and cv(*

_{W}*V*

_{AF,}

*). These changes with selection intensity concur with the diffusion approximation (17).*

_{W}### Variable dominance:

We assume the degrees of dominance for the effects of mutations on both the metric trait and fitness vary as modeled by Caballero and Keightley (1994). Two classes of mutations are assumed: lethal (mutants of *s* = 0.9–1.0) and nonlethal (Wang* et al*. 1998; Davies* et al*. 1999). The effects on the metric trait for both classes of mutants follow the reflected gamma (^{1}/_{4}) distribution and their dominance coefficients vary randomly in the range [0, exp(−*K*|*a*|)], where *K* is a constant chosen to have a given mean *h̅*′. The metric trait is assumed to be under real stabilizing selection of strength *V _{s}*

_{,r}= 20

*V*

_{E}, where

*V*

_{E}is the environmental variance. Lethals occur at a rate 0.03 per haploid genome per generation and have a constant dominance coefficient

*h*= 0.03 (Crow 1993). Values of

*s*for nonlethal mutations are assumed to follow a gamma (

^{1}/

_{2}) distribution with overall mean . If we assume that mutants with effect

*s*≥ 0.01 are detectable (Wloch

*et al*. 2001), this distribution for

*s*predicts that 71% of mutants are detectable and these have mean pleiotropic effect . The dominance coefficient

*h*for nonlethal mutations is also assumed to vary inversely and exponentially with

*s*, but constrained by

*h*≤

*h*′ and

*h*≥ 0.03 to ensure that the deleterious mutants are less recessive than lethals (Zhang

*et al*. 2004). If

*h̅*is assumed to be 0.3, the detectable and “realized” value is less than this. Simulation results were obtained by averaging 2 × 10

^{8}nonlethal and 5000 lethal mutants sampled from the distributions described above. Results listed in Tables 3–6 were obtained assuming a base population of effective size 10

^{4}and a subsequent rapid bottleneck of two individuals.

#### Change in the metric trait:

Assuming and λ = 0.3 for nonlethal mutations, the simulation results under different mean dominance coefficients *h̅*′ are listed in Table 3. With this combination of mutation parameter values, the base population maintains a high level of additive genetic variance in the metric trait (*V*_{A0,T}) if *h̅*′ ≥ 0.4 (see also Zhang* et al*. 2004). Following a bottleneck *V*_{A,T} increases if *h̅*′ ≤ 0.45. As *h̅*′ increases, *V*_{AF,T}/*V*_{A0,T} decreases and approaches 0.75, the prediction for additive neutrality when (Wright 1951). However, *V*_{AF,T} maintains approximately the same level (∼0.34), independent of the mean dominance coefficients. Variation among bottlenecked lines in additive variance is quite substantial, such that *V*_{A,T} can still increase in some bottleneck lines even when (*cf*. Whitlock and Fowler 1999). As *h̅*′ decreases, *V*_{BF,T}/*V*_{A0,T} increases, exceeding 1.0 when *h̅*′ ≤ 0.38. The dominance variance increases to >2.5 times its original value when *h̅*′ ≤ 0.5, but *V*_{DF,T} decreases with increasing *h̅*′. Since it is assumed that *h* ≤ *h̅*′, *h̅** increases from 0.105 to 0.199 as *h̅*′ increases from 0.2 to 0.5, but without much effect on the fitness trait.

#### Change in the fitness trait:

Assuming and λ = 0.3 for nonlethal mutations, the simulation results for different mean dominance coefficients *h̅* are listed in Table 4. With these parameters, when *h̅* > 0.25 (and thus *h̅** > 0.15), our model can predict values of the equilibrium additive variance in the fitness trait that are compatible with the typical estimate of 0.005 (Charlesworth and Hughes 2000). Following a bottleneck, both *V*_{A,}* _{W}* and

*V*

_{D,}

*increase substantially relative to their equilibrium values when*

_{W}*h̅*< 0.5 (and

*h̅** < 0.34). The between-line variance

*V*

_{BF,}

*exceeds*

_{W}*V*

_{A0,}

*and becomes increasingly so with decreasing*

_{W}*h̅*. It is striking that a population passing a rapid bottleneck can have abundant variance in fitness:

*V*

_{AF,}

*≅ 0.14,*

_{W}*V*

_{DF,}

*≅ 0.16, and*

_{W}*V*

_{BF,T}≅ 0.033, nearly the same for each

*h̅*investigated. The absolute magnitude of variation in the fitness traits following a bottleneck is determined mainly by the strength of selection and the number of mutant genes, while the degrees of dominance affect mostly their changes relative to their equilibrium values. The variance among bottlenecked lines in additive variance is substantial and increases relative to

*V*

_{A0,}

*when*

_{W}*h̅*decreases. The mean fitness in the base population is determined mainly by the mutation rate, given by exp(−2λ) (Haldane 1937), although there is a small decrease from 0.55 to 0.52 as

*h̅*increases from 0.1 to 0.5. The inbreeding depression increases with decreasing

*h̅*,

*B*> 5 when

_{W}*h̅*< 0.2.

#### Genetic bases of variation and its redistribution following a bottleneck:

Following Caballero and Keightley (1994) and Wang* et al*. (1998), we use simulation to analyze the contributions of different classes of mutants to the genetic variances before and after bottlenecking (Table 5). As real stabilizing selection is weak relative to pleiotropic selection, classes of mutants are divided according to the magnitude of *N**s̅*_{p}.

The average dominance coefficient of each class of mutants decreases with increasing pleiotropic selection, *i.e.*, mutants of larger pleiotropic effect appear more recessive, as assumed in the model. Although 37% of new mutants have *N**s̅*_{p} > 500 and an additional 9% are lethal, they segregate at very low frequencies, averaging 0.06 and 0.03%, respectively, in the equilibrium population of size 10^{4} (Table 5). Some 19% of new mutants are under relatively weak pleiotropic selection and have an average frequency of 2.6%. About 30% of segregating mutants have *N**s̅*_{p} > 500 because they are highly recessive, another ∼30% of segregating mutants have *N**s̅*_{p} < 50, and mean fitness is controlled mainly by them. About 77% of *V*_{A0,}* _{W}* and 92% of

*V*

_{D0,}

*is contributed by the very deleterious and lethal mutants, while mutants of small pleiotropic effects contribute little to variance in fitness. Mutants of large pleiotropic effects therefore contribute substantially to variation in fitness, but not to metric traits, and vice versa. Hence if mutational effects on the trait and fitness,*

_{W}*a*and

*s*, are assumed to be independently distributed, the mutant genes that determine the additive and dominance variation in fitness are different from those that determine the corresponding variances in the metric trait. It is therefore sufficient to focus only on nonlethal mutants to investigate the maintenance of quantitative genetic variation (see Zhang

*et al*. 2004).

As mutants under stronger selection that survive a bottleneck undergo greater increase in their frequencies, the reduction in fitness (inbreeding depression) following a bottleneck is due mostly to very deleterious and lethal mutants. Much of the increase in *V*_{A,}* _{W}* and

*V*

_{D,}

*is due to lethal mutants as bottlenecking increases the relative contribution of lethal mutants and decreases that of deleterious mutants (Table 5). Changes in mean fitness and variance in fitness are closely related (*

_{W}*cf*. García

*et al*. 1994; Wang

*et al*. 1998), for variation between bottlenecked lines [both

*V*

_{BF,}

*and*

_{W}*V*(

*V*

_{AF,}

*)] is contributed mostly by*

_{W}*rare*lethals and very deleterious mutants. Although the majority of the segregating mutants (∼86%) are of the smallest pleiotropic effects , they contribute little to the change in variance in fitness.

The relative contribution to additive variance in the metric trait (*V*_{A,T}) of each class of mutants is nearly the same in the bottlenecked and base populations, and those to *V*_{BF,T} are similarly distributed. However, the relative contribution to the dominance variance of the metric trait (*V*_{D,T}) after the bottleneck from mutants of smallest pleiotropic effect is reduced. The increase in *V*_{D,T} is due to the dramatic increase in frequencies of segregating mutants. The variation between lines for the metric trait [both *V*_{BF,T} and *V*(*V*_{AF,T})] is contributed primarily by very common mutants with the smallest pleiotropic effects (64 and 79%, respectively) and secondarily by rare lethals and very deleterious mutants (18 and 9%, respectively). As the mutational effects on the metric trait are assumed to follow a highly leptokurtic distribution [reflected gamma (^{1}/_{4})] in Tables 3–6, mutants having a large effect on the metric trait are uncommon, but results (not shown here) confirm that most *V*_{BF,T} and *V*(*V*_{AF,T}) is contributed by such mutants.

#### Effect of mutation rate:

For and , simulation results for different values of the mutation rate λ for deleterious mutants but a fixed rate (0.03) for lethals are listed in Table 6. With an increase in the mutation rate, values of genetic variance components in both the equilibrium base population and the bottlenecked populations increase, but the relative changes in *V*_{AF,}* _{W}*/

*V*

_{A0,}

*,*

_{W}*V*

_{DF,}

*/*

_{W}*V*

_{D0,}

*,*

_{W}*V*

_{BF,}

*/*

_{W}*V*

_{A0,}

*, and*

_{W}*V*

_{DF,T}/

*V*

_{D0,T}decrease a lot and in

*V*

_{AF,T}/

*V*

_{A0,T}and

*V*

_{BF,T}/

*V*

_{A0,T}decrease slightly due to bottlenecking. That is, for higher λ, high variances in the base and bottlenecked populations but low relative increases in these variances following a bottleneck are expected. This reflects the fact that, as λ increases, nonlethal mutants become common and important in relation to lethals and thus the effect of genetic drift weakens on the parameters that were determined mainly by nonlethals. Thus with an increasing mutation rate,

*V*(

*V*

_{AF,T}), which is mostly determined by nonlethals, decreases, while

*V*(

*V*

_{AF,}

*), predominantly controlled by lethals (see Table 5), remains approximately the same. If mutants are infrequent, say λ ≤ 0.10,*

_{W}*V(V*

_{AF,T}) is so high that there are likely to be bottlenecked lines with an increase in

*V*

_{A}in the metric trait, even for (

*cf*. Avery and Hill 1977). With a large λ, however, the mean fitness reduces and the inbreeding depression increases.

#### Impact of a correlation between *|*a*|* and s:

For a metric trait that is correlated with fitness, such as life history, it is biologically plausible that mutational effects on the trait and fitness are correlated (see Keightley and Hill 1990), which for a symmetrically distributed metric trait implies a correlation between |*a*| and *s*. Such a correlation can reduce the variance of the metric traits although it has little influence on the fitness trait. In particular, as the correlation increases, mutant genes that control the fitness trait play a more important role in determining the variability in the metric trait. The variance of the metric trait is thereby reduced, as mutant genes of large pleiotropic effects on fitness remain at low frequencies (see also Zhang* et al*. 2004). For example, let us assume that effects of nonlethal mutations have a correlation coefficient, , but any change in the variance of *a* from lethals is unaffected, simply because they contribute little to the observed variance in the metric trait. A bivariate gamma distribution with corr(|*a*|, *s*) = ρ was constructed using algorithm GTVR (Schmeiser and Lai 1982). Consider the example listed in Table 5: as ρ increases from 0 to ^{1}/_{4}, *V*_{A0,T} decreases from 0.46 to 0.33, and *V*_{D0,T} decreases from 0.024 to 0.015, but the relative changes in *V*_{AF,T}/*V*_{A0,T,} /*V*_{A0,T}, *V*_{DF,T}/*V*_{D0,T}, and *V*_{BF,T}/*V*_{A0,T} remain approximately the same. This shows that the changes in genetic parameters after bottlenecks are due mainly to degree of dominance and little affected by the correlation between the effects of mutations.

## COMPARISON BETWEEN EMPIRICAL AND PREDICTED RESULTS

A number of experiments have been conducted, all with insects, in which the effects of a bottleneck in population size on quantitative genetic parameters have been examined. In a large-scale experiment Whitlock and Fowler (1999) estimated additive genetic and residual variances for morphological traits in *Drosophila melanogaster*. The mean change in *V*_{A} was in very good agreement with predictions from additive theory, decreasing proportional to *F*, with a CV of *V*_{A} ≅ 25–37%. On average the residual variance increased by 11% relative to the base population, with a CV ≅ 16–32%, which implies *V*_{DF,T}/*V*_{D0,T} ≅ 0.8–2.1 if the residual variance is due mainly to the dominance variance. Saccheri* et al.* (2001), using *Bicyclus anynana* to study the effects of bottlenecks of 2, 6, and 20 individuals on wing pattern characters and wing size, also found decreases in *V*_{A}, which were not significantly different from the neutral additive model. For viability, López-Fanjul and colleagues found that *V*_{AF,}* _{W}*/

*V*

_{A0,}

*≅ 1.6–14.9 and*

_{W}*V*

_{BF,}

*/*

_{W}*V*

_{A0,}

*≅ 0.7–3.9 following a bottleneck of two individuals in*

_{W}*D. melanogaster*and

*Tribolium castaneum*(López-Fanjul and Villaverde 1989; García

*et al*. 1994; Fernández

*et al.*1995; see also review by Wang

*et al*. 1998). The empirical data show that across wide species and populations, the inbreeding depression in terms of lethal equivalents is in the range from 0.1 to 5.4 (Lynch and Walsh 1998, Chap. 10).

García-Dorado* et al*. (2004) reviewed experimental evidence and suggested that mutants of sufficient effect to be detected (denoted *) in laboratory experiments must be such that λ* ∼ 0.1, *s̅*^{*}_{P} > 0.1, and *h̅** ∼ 0.2. Experiments fail to detect mutations with small effects on fitness (Keightley and Eyre-Walker 1999; Lynch* et al*. 1999; Otto and Jones 2000; Goddard 2003), so if the minimum pleiotropic effect on fitness detectable, *s*_{d}, is 0.01 (Wloch* et al*. 2001), then 71% of nonlethal mutants would be detected if mutant effects have an overall mean of *s*_{p} = 0.073 and are distributed as gamma (^{1}/_{2}). Given the means for detectable mutants, (García-Dorado* et al*. 2004), and and λ = 0.3, the joint-effect model predicts (Tables 3 and 4) *V*_{A0,T} = 0.46 and *V*_{A0,}* _{W}* = 0.007, which agree well with typical observations for Drosophila, on which most data have been obtained (Falconer and Mackay 1996; Charlesworth and Hughes 2000). The model also predicts the mean fitness

*v*

_{0}= 0.52, which is somewhat below observations (Wang

*et al*. 1998). Following a bottleneck of two individuals, both additive variance and between-line variance in the metric trait are expected to fall (

*V*

_{AF,T}/

*V*

_{A0,T}≅ 0.76 ± 0.12, and

*V*

_{BF,T}/

*V*

_{A0,T}≅ 0.49), very close to neutral additive predictions (Wright 1951), and dominance variance to increase (

*V*

_{DF,T}/

*V*

_{D0,T}≅ 2.6). These values agree well with experimental observations on morphological traits in Drosophila (López-Fanjul

*et al*. 1989; Whitlock and Fowler 1999) and in

*B. anynana*(Saccheri

*et al*. 2001). Variances in fitness are expected to increase (

*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*≅ 20 ± 8 and*

_{W}*V*

_{BF,}

_{W}/V_{A0,}

*≅ 4.6), and inbreeding depression (*

_{W}*B*) is ∼2.8 lethal equivalents. These predictions are close to or in the range of empirical data reviewed by Lynch and Walsh (1998) and Wang

_{W}*et al*. (1998). If and while all other parameters have the same values as above,

*V*

_{A0,}

*is predicted to be reduced to 0.0042, slightly less than the typical estimate of 0.005 (Charlesworth and Hughes 2000), and*

_{W}*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*≅ 36 ± 11 and*

_{W}*V*

_{BF,}

_{W}/V_{A0,}

*≅ 7.6, somewhat higher than empirical data from bottlenecking experiments (Wang*

_{W}*et al*. 1998). The predicted inbreeding depression,

*B*≅ 4.1 (Table 4), however, conforms to the empirical evidence (Lynch and Walsh 1998, Chap. 10). With a large dominance coefficient, say and , however, our predictions of

_{W}*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*and for*

_{W}*V*

_{BF,}

_{W}/V_{A0,}

*and*

_{W}*B*are in good agreement with empirical data (see Table 4).

_{W}If the overall rate of nonlethal mutations is small, say λ = 0.1, then *V*_{A0,}* _{W}* = 0.0035 is slightly lower and

*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*≅ 32 ± 13 and*

_{W}*V*

_{BF,}

_{W}/V_{A0,}

*≅ 6.9 are slightly higher than observations, but*

_{W}*B*≅ 1.5 agrees well with empirical data (Table 6). In such a situation, the variation among bottlenecked lines in additive variance of the metric trait

*V*(

*V*

_{AF,T}) is large, and a further amount is contributed by disequilibrium (see appendix) such that in some bottlenecked lines

*V*

_{A}might increase even if the average

*V*

_{A}decreases (

*cf*. Bryant

*et al*. 1986; Fowler and Whitlock 1999; Whitlock and Fowler 1999). If λ is large, say 0.6, the variances

*V*

_{A0,}

*= 0.012,*

_{W}*V*

_{AF,}

*/*

_{W}*V*

_{A0,}

*≅ 15 ± 4,*

_{W}*V*

_{BF,}

_{W}/V_{A0,}

*≅ 3.5 and*

_{W}*B*= 4.7 are in agreement with empirical data, but the mean fitness

_{W}*v*

_{0}= 0.3 of the base population seems lower than that observed (Wang

*et al.*1998). While

*V*(

*V*

_{AF,T}) becomes small (see Table 6), the probability that some bottlenecked lines have an increased

*V*

_{A,T}is very small, but as the number of mutants increases, the contribution of linkage disequilibrium to

*V*(

*V*

_{AF,T}) increases (see appendix). Taking this into account, the theoretical prediction of CV is compatible with data (López-Fanjul

*et al*. 1989; Fowler and Whitlock 1999; Whitlock and Fowler 1999).

## DISCUSSION

The joint-effect model of pleiotropic and real stabilizing selection provides an interpretation for the high levels of quantitative genetic variation observed in an equilibrium population (Zhang and Hill 2002; Zhang* et al*. 2004). It is shown in this study that, under some combinations of mutation parameters, the joint-effect model can also explain the observed values of additive genetic variance in the fitness trait. Further, if a large equilibrium population passes through a single generation bottleneck and then quickly returns to its original size, the model predicts the changes in the mean and variance components of the metric and fitness traits that are compatible with experimental data.

In this study, two types of traits are assumed: a “fitness trait,” which is closely related to overall fitness and thus under relatively strong selection, and a “metric trait,” which is under relatively weak real stabilizing selection and is thus only loosely related to fitness. Hence the two traits are assumed not to be strongly correlated. To understand in terms of MSB models the levels of the mean and genetic variances for both types of traits maintained in the equilibrium population and their changes caused by a rapid bottleneck, knowledge of the effects and rates of mutations is very important. Direct and accurate information is lacking and most is based on a few model organisms such as Drosophila. García-Dorado* et al*. (2004) reviewed experimental evidence, suggesting that mutants detectable in laboratory experiments have λ* ∼ 0.1, *s̅*^{*}_{P} > 0.1, and *h̅** ∼ 0.2, different from those used by Wang* et al*. (1998). As mutations with small effects on fitness fail to be detected experimentally (Keightley and Eyre-Walker 1999; Lynch* et al*. 1999; Otto and Jones 2000; Goddard 2003), the mutation parameters have to be conjectured. Different designs of experiments and different methods of analysis of data may give rise to different minimum detected effect, denoted as *s*_{d}. For example, Davies* et al.* (1999) estimated that *s*_{d} = 0.07% and that >96% of mutations were undetected in their mutation-accumulation experiments on *Caenorhabditis elegans* and that estimates depended on the assumed distribution of mutation effects. Wloch* et al*. (2001) estimated *s*_{d} = 0.01 in an experiment in which they applied tetrad analysis to a large number of clones of yeast and so could measure individual mutations. The numbers of undetected mutants are unknown. If only one-quarter can be measured, as suggested by García-Dorado* et al*. (2004), then typical estimates of parameters for all mutations become λ ∼ 0.4, *s̅*_{p} < 0.1, and *h̅* > 0.2. Both *s̅*_{p} and *h̅* depend on assumptions on the distribution of *s* and how *h* varies with *s*. For example, if *s*_{d} = 0.01, *s* follows a gamma (^{1}/_{2}) distribution, and *h* varies inversely and exponentially with *s*, and (see Table 4); whereas if *s* follows an exponential or a less leptokurtic distribution (Wang* et al*. 1998), *s̅*_{p} ≅ *s̅*^{*}_{p} and *h̅* ≅ *h̅**. Although mutants going undetected in laboratory experiments may not be important for fitness traits (Wloch* et al*. 2001), they are important for metric traits (Table 5). Thus information on *s*_{d} is also necessary for modeling maintenance of genetic variation in metric traits incorporating typical estimates of mutation parameters.

The problem of detectable levels of mutational effects, expressed as *s*_{d}, renders inaccurate the measurement of distributions of mutational effects. Experimental evidence indicates, however, that distributions of pleiotropic effects on fitness are bimodal (Davies* et al*. 1999; Wloch* et al*. 2001). Consequently, two distinctive classes of mutants are assumed here, lethal and nonlethal, while their effects on the metric trait are assumed to be identical. With assumptions of effects and degree of dominance of mutants in accordance with empirical data on mutation parameters (Caballero and Keightley 1994; Lyman* et al*. 1996; Wang* et al*. 1998; Lynch* et al*. 1999; García-Dorado* et al*. 2004), the joint-effect model predicts the changes in genetic variances of both the metric and fitness traits due to a rapid bottleneck of two individuals. Namely there are decreases in additive variance and an increase in dominance variance (*e.g.*, residual variance) of metric traits and increases in additive variance of fitness traits. Those predictions are compatible with data from bottleneck experiments (Fowler and Whitlock 1999; Whitlock and Fowler 1999; Saccheri* et al*. 2001; López-Fanjul* et al*. 2003).

Linkage disequilibrium (LD) is induced by the bottleneck in population size. It has no effect on between-line variance (*V*_{B}) and negligible effect on the expected within-line variance (*V*_{A}) in quantitative traits that are determined by many loci (Avery and Hill 1977, 1979; appendix). However, LD can inflate the variance among lines in the additive genetic variance *V*(*V*_{AF}) substantially and dominance variance (*V*_{D}) in fitness traits. This increase in both *V*(*V*_{AF}) and *V*_{D} is transient if the population is expanded rapidly following the bottleneck, although it is likely to last longer for species such as Drosophila with few chromosomes and no crossing over in males (see appendix). Thus, in experiments on Drosophila, the observed values of *V*(*V*_{AF}) and *V*_{D} within the expanded populations may come mainly from LD. The practical problem is to obtain precise estimates of the variance among lines in additive variance in any experiment of manageable size. Four sources of variance (at least) can be identified: (1) real variance differences between lines due to the sampling of genes into the bottlenecked populations; (2) LD arising from sampling in the bottleneck that has not been lost due to recombination nor subsequently gained because the resultant replicate populations are actually finite in size (say, *M*; see appendix); (3) sampling of families used to estimate variance, *e.g.*, the sample of sires for a half-sib analysis; and (4) sampling of offspring. Of these, item 1 is *V*(*V*_{AF}), item 2 is of order *V*_{A}^{2}/*M*, 3 is of order *V*_{A}^{2}/, and 4 is of order ^{2}/. The LD term may be trivial relative to the other sources of error due to undertaking the estimation in populations and with samples of finite size.

The distribution of gene frequency in the base population that is determined by the selection acting on the mutant genes has an important role in determining the changes in parameters following a rapid bottleneck. Neutral mutant genes can occur at intermediate frequencies in the equilibrium population and genetic variances contributed by them decline following a bottleneck regardless of their degree of dominance. The impact on the genetic variance of mutants with a deleterious pleiotropic effect on fitness with bottlenecking depends greatly on their degree of dominance for fitness (Figure 4). The relation between *h* and *s* used in this and other studies (Caballero and Keightley 1994; Wang* et al*. 1998), however, is supported by only a few available data (Caballero and Keightley 1994; Deng* et al*. 2002). More data are needed to establish the relationship between the effects and dominance of mutant genes to better investigate the maintenance of genetic variance within equilibrium populations and the consequence of a population bottleneck.

Although we focus here on the effects of a rapid bottleneck of two individuals, other numerous cases (*e.g.*, one bottleneck of more individuals, two generations of four individuals, …) can be analyzed in the same way. Under strong selection and with all mutant genes assumed to have the same dominance coefficient, for example, the changes in additive and between-line variance after a bottleneck of *N*_{F} individuals are given by *V*_{AF,T}/*V*_{A0,T} = (1 − 1/2*N*_{F})[1 − 1/*N*_{F} + 1/(2*N*_{F}*h*′)]^{2} and *V*_{BF,T}/*V*_{A0,T} = (1/*N*_{F})[1 − 1/2*N*_{F} + 1/(4*N*_{F}*h*′)]^{2}, respectively, using the formula listed in Table 1. As the size of the bottleneck increases, its effect on variation reduces. In general, the bottleneck effect can be scaled approximately by *F* (inbreeding coefficient) if purging selection and new mutations are ignored, irrespective of the details of population size and number of generations (Robertson 1952; Crow and Kimura 1970; Wang* et al*. 1998; López-Fanjul* et al*. 2002, 2003). Our predictions of the changes in genetic variances after a bottleneck apply when the population expands immediately (in one generation) to a large size from the severe bottleneck and thus selection is completely ignored. In practice some empirical observations on the increase in additive genetic variance due to bottlenecking were obtained following two or more generations of inbreeding and a relatively slow expansion of the population (due to the reproductive constraint of the organism). Thus purging selection could be involved, particularly against lethal and semilethal mutations that would, if they survived a bottleneck of two individuals, be at a frequency of ^{1}/_{4} before selection rapidly decreased their frequencies. With purging selection, the additive variance of the fitness trait is increased by less than our predictions, but even so the amount is limited (see reviews by Roff 2002 and Keller and Waller 2003). The additive variances of the metric trait may be little affected since it is predicted to derive mostly from mutants of small pleiotropic effects.

In conclusion, our analysis shows that mutant genes that produce variability in fitness traits are quite different from those causing variability in metric traits that are not to be strongly correlated with fitness. Mutants of large pleiotropic effects on fitness, which are likely to be highly recessive for fitness and segregate at very low frequencies in equilibrium populations, mainly determine the variance of fitness traits in equilibrium populations and the inbreeding depression and changes in variance following a bottleneck. Mutants of small pleiotropic effects on fitness, which are at most partially recessive for fitness, are not under strong selection and are likely to segregate at intermediate frequencies. Those mutant alleles largely determine the properties of the metric traits (Zhang* et al*. 2004). The joint-effect model of pleiotropic and real stabilizing selection is obviously itself a simplification in that epistasis and other complications such as genotype × environment interaction are ignored, but nevertheless provides an interpretation for observations on genetic variation in both fitness and metric traits maintained in natural populations. Furthermore, with specific combinations of mutation parameters it can also predict the changes observed experimentally in genetic variance for both types of traits and in inbreeding depression after population bottlenecks. This investigation therefore provides further support to our conclusion that mutation-selection balance is a simple and plausible mechanism for maintenance of quantitative genetic variation in natural populations.

## APPENDIX:

### INFLUENCE OF LINKAGE DISEQUILIBRIUM ON PARAMETERS IN BOTTLENECK LINES

The population bottleneck generates LD due to sampling. This does not affect mean performance in the replicate populations (lines) subsequently established if there is no epistasis or selection, but can influence both the mean and the variability among replicates of the components of genetic variance.

#### Expectation of within-line variance:

We use results of Avery and Hill (1979), but with the notation modified. The mutant gene at locus *i* has frequency *p _{i}*, and for the quantitative trait, for example, the homozgote wild type, the heterozygote, and the mutant homozygote have genotypic values 0,

*a*

_{i}

*h*

^{′}

_{i}, and

*a*, respectively. The average effect of the gene substitution is and the excess of the heterozygote over the homozygote mean is . The genotypic variance within lines is A1where

_{i}*i*,

*j*= 1, … ,

*n*, for

*n*loci, and

*D*is the LD coefficient at loci

_{ij}*i*and

*j*. Because of LD, additive genetic and dominance terms are not orthogonal, but it seems appropriate to take 4 × (covariance of half sibs) as a practical predictor of the

*V*

_{A}. Hence the additive variance is A2The terms in α

*α*

_{i}*in (A2) comprise second moments,*

_{j}D_{ij}*e.g*.,

*E*(

*D*); third moments,

_{ij}*e.g*.,

*E*[(1 − 2

*p*)

_{i}*D*]; and fourth moments,

_{ij}*e.g*.,

*E*[(1 − 2

*p*)(1 − 2

_{i}*p*)

_{j}*D*]. In a population previously in linkage equilibrium, these moments are all zero, and if there is no selection or other directional force, remain so in subsequent generations.

_{ij}*E*[(1 − 2

*p*) (1 − 2

_{i}*p*)

_{j}*D*] ≠ 0 after the first generation if there is further drift, but not if the population is rapidly expanded. Hence we assume there is no LD contribution to the additive genetic variance;

_{ij}*i.e*., Var(

*A*) =

*V*

_{AF}, where

*V*

_{AF}is the variance from individual loci (Table 1).

As it is not usually feasible to partition the residual genetic variance further, the residue of Var(*W*) after removing Var(*A*) can be regarded as dominance variance. From (A1) and (A2) A3The terms in pairs of loci δ_{i}δ_{j}*D*^{2}_{ij} are fourth moments as the δ* _{i}* are constants, and

*E*≠ 0 due to sampling. After one generation in a bottleneck of size

*N*

_{F}, A4(Hill and Robertson 1968). The first term in (A3) is the contribution from individual loci,

*V*

_{DF}(Table 1). Hence, assuming for simplicity

*N*

_{F}= 2, A5The expectation in (A5) is over the distribution of effects of loci and gene frequencies in the base population, and those at different loci are assumed uncorrelated. The quantity

*E*{∑

*2δ*

_{i}

_{i}p_{i}_{0}(1 −

*p*

_{i}_{0})} =

*B*

_{T}is the expected inbreeding depression in the metric trait following complete inbreeding without any selection (see also Avery and Hill 1979) or the derivative of inbreeding depression to inbreeding coefficient [for fitness, the relevant quantity is given by

*B*from (16), using the first derivative at

_{W}*F*= 0]. Hence the expectation of a random cross product term in (A4) is (

*B*

_{T}

*/n*)

^{2}, and A6assuming that many loci influence the trait. Equation A6 is in terms of quantities that have already been computed.

In subsequent generations in a large expanded population, the LD coefficients at loci *i* and *j* decline through recombination in proportion to (1 − *c _{ij}*)

^{2}, where

*c*is the recombination fraction. To compute the consequent decline in variance, assumptions have to be made of no further mutation or selection and about the distribution of loci affecting the trait. The simplest assumption, if not the most realistic, is that such loci are uniformly distributed throughout the genome, such that the decline in variance is proportional to the mean of

_{ij}^{2}

*D*

^{2}

_{ij}across the genome, regardless of effects on the trait. Hence, at generation

*t*after the bottleneck, let A7where

*y*

_{1}= 1. For mammals with, say, 20 chromosomes each of length 1 M (a “mouse”), 95% of pairs of loci are on different chromosomes, so an adequate approximation is

*y*= (

_{t}^{1}/

_{4})

^{t}^{−1}. An extreme example is

*D. melanogaster*, which has in effect three chromosomes of approximately equal length (each, say, 1 M), and no crossing over in males. Hence, for loci

*l*map units apart and assuming Haldane's mapping function, (1 −

*c*)

_{ij}^{2}= [

^{1}/

_{4}(3 +

*e*

^{−2}

*)]*

^{l}^{2}, the overall coefficient is where

*E*[(3 +

_{l}*e*

^{−2}

*)*

^{l}^{2}] is obtained by integrating

*l*over all pairs of sites on the chromosome assuming a uniform distribution.

##### Examples:

If mutational effects are symmetrically distributed, as we have assumed for the metric trait, then *B*_{T} = 0 and there is no contribution from LD to the dominance variance. Hence we consider the contributions to the dominance variance in fitness (in which *a* and *h*′ are replaced by *s* and *h* in the above equations) and a specific example. Assume *a* and *s* follow gamma (^{1}/_{4}) and (^{1}/_{2}) distributions with means , , and λ = 0.3. For the fitness trait, *B _{W}* = −3.0 and

*V*

_{DF}= 0.176; thus Var(

*D*)

*= 0.176 + 1.68*

_{t}*y*. For

_{t}*t*= 2, 4, and 6, and a mouse,

*y*= 0.25, 0.016, and 0.001; and for

_{t}*D. melanogaster*,

*y*= 0.43, 0.19, and 0.13, respectively.

_{t}##### Extensions and comments:

If the populations after the bottleneck are small (effective size *M*) such that further sampling from drift can induce LD, approximate results can be obtained from A8(an approximation to the loss through drift and recombination and the gain through drift from reduced heterozygosity; see Avery and Hill 1977). For free recombination, if *M* is moderately large, *D*_{t}^{2} → 2/. Providing *M* > 10 and *t* > 3 in the example, LD contributes less than individual loci for the case of mammals.

An important proviso, regardless of the population size, is that there is negligible variance of inbreeding coefficient in the populations after the bottleneck (Avery and Hill 1979). If, for example, some individuals are offspring of full sibs, the variance increases markedly, in proportion to the product (variance in pedigree inbreeding) and (inbreeding depression in the sublines). Thus mating relatives should be avoided as far as possible within the lines when estimating dominance variance.

#### Variance of within-line variance:

The expectation of Var(*A*) is given in (A2), and in the special context of a population that rapidly expands, simplifies, and does not involve LD. In the absence of dominance, the expectation of the variance is a second moment in gene frequencies, and the variance of the variance a tractable fourth moment (Avery and Hill 1977). In the presence of dominance, there is no reason to assume that because the fourth moment *E*[(1 − 2*p _{i}*)(1 − 2

*p*)

_{j}*D*] is zero, so is the corresponding eighth moment. Evaluation is not impossible, but would be very tortuous (see Hill and Weir 1988). In view of the small contribution of this fourth moment and thus potentially of its square, we assume the contribution is merely a function of the heterozygosity so as to get some feel for the magnitude of terms, with results holding more precisely for additive gene action. We use results of Avery and Hill (1977), who expressed the variation as a simple function of initial additive variance and population size. From (A2) where the first term is that for single loci (Table 1). Using similar arguments as for the dominance case, since , to obtain the expectation of the variance of the sum of the contributions and assuming many loci as in (A6) A9where

_{ij}*V*

_{A0}is the additive variance in the base population. Equation A9 has a form similar to that given by Avery and Hill (1977)(except they ignored the single-locus sampling term) and formally holds only for additive gene action. If the population remains of finite size, the term in

*V*

_{A0}

^{2}does not disappear (

*cf*. A8), and for unlinked loci asymptotes at 2

*V*

_{A0}

^{2}/3

*M*(Avery and Hill 1977).

##### Example:

For the metric trait in the example discussed above, *V*_{A0} = 0.5 and *V*(*V*_{AF}) = 0.003. Hence Var[Var *A*] ≅ 0.003 + 0.094*y _{t}*. The LD term dominates.

## Acknowledgments

We are grateful to two reviewers for helpful comments. This work was supported by a grant from the Biotechnology and Biological Sciences Research Council (15/G13242).

## Footnotes

Communicating editor: J. B. Walsh

- Received December 16, 2003.
- Accepted April 5, 2004.

- Genetics Society of America