## Abstract

Why does phenotypic variation increase upon exposure of the population to environmental stresses or introduction of a major mutation? It has usually been interpreted as evidence of canalization (or robustness) of the wild-type genotype; but an alternative population genetic theory has been suggested by J. Hermisson and G. Wagner: “the release of hidden genetic variation is a generic property of models with epistasis or genotype–environment interaction.” In this note we expand their model to include a pleiotropic fitness effect and a direct effect on residual variance of mutant alleles. We show that both the genetic and environmental variances increase after the genetic or environmental change, but these increases could be very limited if there is strong pleiotropic selection. On the basis of more realistic selection models, our analysis lends further support to the genetic theory of Hermisson and Wagner as an interpretation of hidden variance.

A common experimental observation in quantitative genetics is a higher phenotypic variance for quantitative traits in populations that carry a major mutation or are exposed to environmental stresses (*e.g*., heat shock) (Scharloo 1991; for a recent review see Gibson and Dworkin 2004). Part of the added variance must be genetic because the population responds to artificial selection. The lower variability of the wild type than that of the mutants has been interpreted as evidence for robustness or canalization (Waddington 1957): that is, under the new condition the magnitudes of gene effects across all trait loci increase relative to the original condition. The importance of canalization has been recognized for a long time and has been the subject of renewed interest recently (see de Visser *et al*. 2003 and Hansen 2006 for reviews).

An alternative population genetic theory has been proposed by Hermisson and Wagner (2004), who suggest that the increase in genetic variance *V*_{G} after the change in environmental conditions or genetic background is a generic property of the population, with no need to introduce canalization (Waddington 1957). The theory appears simple. Under mutation–selection balance (MSB), the mutant alleles are at a selective disadvantage and there is a negative correlation between frequencies and effects of mutations: mutant alleles of small effects on the trait segregate at intermediate frequencies. After the change in genetic or environmental background, gene effects consequently change due to *G* × *E* interaction or epistasis, which reduces the negative correlation because genes that were previously of small effects and at intermediate frequencies may now have large effects. That is, the frequencies of alleles are determined by the previous MSB, while their new effects are at least partly determined by the new conditions. The genetic variance will therefore increase.

Hermisson and Wagner (2004) found that the predicted increase in genetic variance can be substantial; however, the predicted increase is highly sensitive to the population size and can increase without bound with increasing population size (see their Figure 2 and Equation 16). Genetic variance would enlarge with the population size within a small population (Lynch and Hill 1986; Weber and Diggins 1990), but becomes insensitive to the population size within large populations (Falconer and Mackay 1996, Chap. 20). Hence the unbounded increase under the novel environmental condition appears to us as a downside of their theory, even though the predicted increase can be reduced if the changed environmental condition is not novel but there is previous adaptation to it (see their Figure 3).

The basic model that Hermisson and Wagner (2004) employed is that the quantitative trait is under real stabilizing selection and mutant alleles have effects on the focal trait only by changing its so-called *locus genetic variance*. At the mutation–real stabilizing selection balance, some mutants can segregate at intermediate frequencies because of their small effects and therefore weak selection; and there are more such mutants the more strongly leptokurtic is the distribution of effects at individual loci. The unbounded increase of Hermisson and Wagner (2004) results from such a gene-frequency distribution; but it has been shown (see Barton and Turelli 1989; Falconer and Mackay 1996; Lynch and Walsh 1998) that solely stabilizing selection, whether modeled with a Gaussian (Kimura 1965) or a house of-cards approximation (Turelli 1984) or even the generalized form of Hermisson and Wagner (2004) (*i.e*., their Equation 14), cannot provide a satisfactory explanation for the high levels of genetic variance observed in natural populations under realistic values of mutation and selection parameters.

A common observation is that one trait is controlled by many genes and one gene can influence many traits; *i.e*., pleiotropy is ubiquitous (Barton and Turelli 1989; Barton and Keightley 2002; Mackay 2004; Ostrowski *et al*. 2005). Recent detailed studies suggest that pleiotropy calculated as the number of phenotypic traits affected varies considerably among quantitative trait loci (QTL) (Cooper *et al*. 2007; Albert *et al*. 2008; Kenney-Hunt *et al*. 2008; Wagner *et al*. 2008). Such pleiotropic effects must influence the magnitude of the variance. Though some genes have little effect on the focal trait, they almost certainly affect other traits and therefore are not neutral. The inclusion of pleiotropic effects on fitness strengthens the overall selection on mutant alleles and, assuming such pleiotropic effects are mainly deleterious, maintains them at low frequencies. The genetic variance for a trait is therefore likely to be maintained at lower levels than that under only real stabilizing selection on the trait alone (Tanaka 1996). Although the gene-frequency distribution is much more extreme under this joint model, the relevant rate of mutation is genomewide and hence is much larger than that where mutation affects only the focal trait as is assumed in the real stabilizing selection model (Turelli 1984; Falconer and Mackay 1996). Taking into account empirical knowledge of mutation parameters, a combination of both pleiotropic and real stabilizing selection appears to be a plausible mechanism for the maintenance of quantitative genetic variance (Zhang *et al*. 2004). If pleiotropic selection is much stronger than real stabilizing selection, the association between frequency and effect of mutant alleles is weaker than that for a real stabilizing selection model. Further, if overall selection is stronger than recurrent mutation, the frequency distribution of mutant alleles will be extreme. Under those situations, the increase of genetic variance after the genetic or environmental change will be kept at lower levels than that of Hermisson and Wagner (2004), and hence the unbounded increase could be avoided.

Further, Hermisson and Wagner (2004) assume that the environmental variance is not under genetic control (*i.e*., the variance of phenotypic value given genotypic value is the same for all genotypes) and therefore is not subject to change. This assumption conflicts with the increasingly accumulating empirical data that indicate otherwise (Zhang and Hill 2005; Mulder *et al*. 2007 for reviews). Direct experimental evidence is available that mutation can directly affect environmental variance, *V*_{E} (Whitlock and Fowler 1999; Mackay and Lyman 2005), and Baer (2008) provides what is perhaps the first clear demonstration that mutations increase environmental variances, on the basis of data for body size and productivity of *Caenorhabditis elegans*, and finds that the magnitudes of the increases are of the same order as those in the genetic variance.

As real stabilizing selection on phenotype favors genotypes possessing low *V*_{E} (Gavrilets and Hastings 1994; Zhang and Hill 2005), a mutant that contributes little to *V*_{E} is more favored by stabilizing selection than one that contributes a lot. With all else being the same, mutants with small effect on *V*_{E} thus segregate at relatively high frequencies at MSB. That is, there is a negative correlation between the effect on *V*_{E} and the frequency of mutant genes. After the genetic or environmental change, some mutants that were previously of small effects on *V*_{E} have large effects due to *G* × *E* interaction or epistasis while their frequencies remain roughly the same as in the previous MSB. This certainly increases environmental variance.

In this note, we first assume that mutant alleles can affect only the mean value of a focal quantitative trait and otherwise affect fitness through their pleiotropic effects (Zhang *et al*. 2004) and try to answer the following questions: How will the conclusion of Hermisson and Wagner (2004) be affected by taking into account the pleiotropic effect of mutants? Can the “unbounded increase” be avoided? We then further assume that mutant alleles can also directly affect the environmental variance of the focal trait (Zhang and Hill 2008) and investigate how both *V*_{G} and *V*_{E} change following the genetic or environmental change in the population.

## MODEL 1: JOINT PLEIOTROPIC AND STABILIZING SELECTION

A population of *N* diploid monoecious individuals, with discrete generations, random mating, and at Hardy–Weinberg and linkage equilibrium, is assumed. Mutations are assumed to have effects on a quantitative trait *z*, with *a* being the difference in value between homozygotes, and pleiotropic effects on fitness, with *s* (*s* ≥ 0) being the difference in fitness between homozygotes. Gene effects over loci and segregating alleles are assumed to act additively at the original genetic and environmental conditions. The distribution of effects of mutant alleles on the mean value of the trait is assumed to be symmetric about zero while their pleiotropic effects on fitness are negative in accordance with experimental data (Falconer and Mackay 1996; Lynch and Walsh 1998). The quantitative trait is assumed to be under real stabilizing selection with the optimum phenotype at the phenotypic mean μ_{0} and strength characterized by the variance ω^{2} of its fitness profile (a large ω^{2} implies weak selection). Combining pleiotropic selection and real stabilizing selection, the overall selection coefficient is approximated by(1)(Zhang *et al*. 2004), where *V*_{E} is the environmental variance and *V*_{S} = ω^{2} +*V*_{E} is the strength of real stabilizing selection on genotypes (Turelli 1984). For simplicity we assume in the following that real stabilizing selection is weak such that ω^{2} ≫ *V*_{E} and thus *V*_{S} ≈ ω^{2}. [As pleiotropic selection also results in apparent stabilizing selection (Keightley and Hill 1990), the total stabilizing selection due to joint pleiotropic and real stabilizing selection should be stronger than solely real stabilizing selection (Zhang *et al*. 2004).]

Let the frequencies of the wild-type allele and the mutant allele at a given diallelic locus be 1 − *x* and *x*. The contribution from the locus to genetic variance is *V* = *Ha*^{2}/4. Here *H* = 2*x*(1 − *x*) is the heterozygosity, and its value at mutation–selection–drift balance is given by Kimura's (1969) diffusion approximation(2)Here τ is the mutation rate per locus. Assuming that the effective size of the population is equal to its census size (*i.e*., *N*_{e} = *N*), the expected genetic variance at the locus follows as(3)From Equations 1 and 3, it is obvious that genetic variance is dependent only on the squared effect *a*^{2}, and *v* is used to denote it in the following. For simplicity, we assume both mutational effects *v* and *s* independently follow gamma distributions *g*_{1}(*v*) and *g*_{2}(*s*) with shape parameters β_{v} and β_{s}, respectively. The genetic variance at mutation–selection–drift balance is obtained by summing over all loci(4a)where λ is the haploid genomic mutation rate and(4b)For infinitely large populations, simple expressions for *I*_{2} can be obtained for some special distributions of mutant effects (see appendix).

#### Change in mutational effects induced by the genetic/environmental change:

Assume that the population at a mutation–selection–drift equilibrium experiences a rapid environmental shift or carries a major mutation of large effect at a very high frequency and that consequently gene effects on traits change through *G* × *E* or epistatic interaction. With solely real stabilizing selection (Hermisson and Wagner 2004), the only gene effects that can be influenced are those on the trait itself (*i.e*., *locus genetic variance*). In our model 1 there are two components to the gene effects: those on the focal trait and pleiotropic effects acting through other traits, both of which can be changed via *G* × *E* or epistatic interaction. How the genetic or environmental change alters gene effects depends on the details of how *G* × *E* interaction or epistasis acts, for which we have little knowledge. Traditionally it has been proposed that magnitudes of effects on trait increase for all mutant alleles; under such a canalization scenario, genetic variance *V*_{G} certainly increases (Waddington 1957). It is also possible that effects on the trait increase for some mutant alleles but decrease for others (*e.g*., Gibson and Van Helden 1997), which Hermisson and Wagner (2004) termed the *variation interaction scenario*. In this note we consider only the variable interaction scenario of the trait architecture. The effects of mutant alleles on the trait under the new conditions are assumed to become(5a)and the square of the trait effect becomes(5b)where *a*_{r,i} is a random variable that is independent of its original effect *a*_{o,i}, normalized such that *E*[*a*_{r}] = *E*[*a*_{o}] = 0 and . Here *E*[·] denotes the average over all loci and the subscripts o and n represent the variables for the original populations and that after the change. Parameter ζ* _{i}* ∈ [0, 1], which is assumed independent of both

*a*

_{o,i}and

*a*

_{r,i}, collectively characterizes the randomization due to epistasis or

*G*×

*E*interaction. Under these assumptions, allelic effects on the mean of the trait change relative to each other across loci, though on average allelic effects on the mean trait and their squares remain unchanged:

*E*[

*v*

_{n}] =

*E*[

*v*

_{o}]. This assumption is, albeit appearing different from that of Hermisson and Wagner (2004), actually the same.

As the pleiotropic effect on fitness (*s*) is the overall effect of selection on all other traits (Keightley and Hill 1990), it is likely to remain roughly the same following the genetic or environmental change. Even if there is some change in pleiotropic effects, it takes some generations for selection to change frequency distribution of mutant alleles within the population. For simplicity, we assume pleiotropic effects of mutant alleles remain as they were and follow a gamma distribution with a shape parameter β_{s} and further that the pleiotropic and trait effects of alleles are independent.

*V*_{G} after the genetic/environmental change:

In accordance with assumptions of Hermisson and Wagner (2004), the genetic variance immediately after the change can be evaluated as(6)This assumption implies that new effects on the trait do not affect frequencies of mutant alleles but are immediately expressed in trait performance, which seems reasonable in the first one or two generations after the change. On the basis of assumption (6), Hermisson and Wagner (2004) found that, if *v _{i}* and

*V*(τ

*,*

_{i}*v*)/

_{i}*v*are negatively correlated, then

_{i}*V*

_{G,n}>

*V*

_{G,o}. Taking the derivative of from Equation 3 with respect to

*v*, it is readily shown that the ratio strictly decreases with

*v*for arbitrary values of

*s*, ω

^{2}, and τ. Thus, after the genetic or environmental change, the magnitude of

*V*

_{G}maintained in our model will increase as for real stabilizing selection (Hermisson and Wagner 2004). To characterize how large this change is, the

*coefficient of hidden genetic variation*is defined as(7)

Numerical results were obtained using the above Equations 4–7. One mutant with effects *v* and *s* was sampled from the gamma distributions *g*_{1}(*v*) with mean *E*(*v*) and *g*_{2}(*s*) with mean *s*_{p} = *E*(*s*), respectively, while the interaction parameter ζ was sampled from the uniform distribution over an interval [ζ_{1}, ζ_{2}] with ζ_{1}, ζ_{2} ∈ [0, 1]. The results were obtained by averaging over 10^{8} mutants. The numerical calculations show that, with all else being the same, a constant or a uniformly distributed interaction parameter ζ with the same mean causes nearly the same change in *V*_{G} (data not shown), indicating that variation in ζ does not affect Δ_{G}. Further, any difference between the distributions of *v*_{o} and *v*_{r} also does not cause any notable difference in Δ_{G}.

Under the special case where ζ* _{i}* = ζ, and

*v*

_{o}and

*v*

_{r}follow the same gamma distribution

*g*

_{1}(·), we have(8a)where(8b)and the coefficient of hidden genetic variation is(9)Within infinite populations, analytic results for

*K*

_{2}are obtained for some special distributions of mutant effects (see the appendix). It is shown that

*K*

_{2}>

*I*

_{2}and thus Δ

_{G}> 0, but that the magnitude of Δ

_{G}depends critically on the shape of the distributions of both the trait effect and the pleiotropic effect on fitness of mutant alleles. If both effects are distributed leptokurtically so β

_{v}+ β

_{s}≤ 1, then Δ

_{G}tends to infinity within an infinitely large population. Otherwise the increase in

*V*

_{G}is limited even within infinite populations.

Numerical results confirm the analytical calculations: if the pleiotropic effect *s* of mutant alleles follows an exponential or a less leptokurtic distribution, the increase of genetic variance is small even within a very large population (Figure 1). If, however, *s* follows a more leptokurtic distribution than the exponential, then Δ_{G} can become infinitely large in an infinite population. Under very weak pleiotropic selection, the amount of increase in *V*_{G} can be substantial (Figure 2), returning to the results of Hermisson and Wagner (2004). Under strong pleiotropic selection, however, Δ_{G} is small, especially when both trait and pleiotropic effects of mutant alleles are distributed less leptokurtically. Given the distribution of the pleiotropic effect on fitness, the distribution of effects on the trait also has an impact on the value of Δ_{G}: the increase in genetic variance is larger the more strongly leptokurtic is the distribution of effects on the trait at individual loci (Figures 1 and 2).

## MODEL 2: ENVIRONMENTAL VARIANCE UNDER GENETIC CONTROL

Next we further assume that mutations have a direct additive effect on the residual variance of the focal trait as well, with *b* being the difference in value between homozygotes. The mutant heterozygote is described by (μ_{0} + *a*/2, *V*_{0} + *b*/2), where μ_{0} and *V*_{0} are the phenotypic mean and the average residual variance, respectively, of the wild-type homozygous population. There is some empirical evidence that suggests mutations increase residual variance (Scharloo 1991; Baer 2008; discussed in Zhang and Hill 2008). In the following, mutant effects on the residual variance are simply assumed to be nonnegative and to follow a gamma distribution *g*_{3}(·) with a shape parameter . We consider the infinite population only such that overall selection is strong enough to avoid any fixation of mutant alleles and a finite environmental variance can be maintained at MSB (Zhang and Hill 2008). Assuming the environmental variance of the wild-type genotype is *V*_{0} = 0, the contribution from each locus at MSB can be approximated as(10)where is the overall selection coefficient given by(11)The environmental variance maintained at MSB is approximated as(12a)with(12b)(Zhang and Hill 2008).

Under the variable interaction scenario of the trait architecture (Hermisson and Wagner 2004), following the genetic and/or environmental change, the mutant effects on residual variance change as(13)where *b*_{r,i} is a random variable that is independent of its original effect *b*_{o,i}, normalized such that *E*[*b*_{r}] = *E*[*b*_{o}]. Parameters ξ* _{i}* ∈ [0, 1], which are assumed independent of both

*b*

_{o,i}and

*b*

_{r,i}, collectively characterize the randomization due to epistasis or

*G*×

*E*interaction. The new environmental variance can be evaluated as(14)As for the genetic variance

*V*

_{G,n}in the above model 1, numerical results showed that the variation in the interaction parameter ξ and the difference in the distributions of

*b*

_{r}and

*b*

_{o}do not cause any notable difference in

*V*

_{E,n}. Under the special case where ξ

*= ξ and residual variance effects*

_{i}*b*

_{r}and

*b*

_{o}follow the same gamma distribution

*g*

_{3}(·), we have(15a)where(15b)Defining the coefficient of hidden environmental variation as in Equation 7, we have(16)With this general model,

*V*

_{G,o}and

*V*

_{G,n}are given by Equations 4 and 8, respectively, with the overall selection coefficient now and including an additional integration over the residual variance effect

*b*

_{o}.

Numerical results confirm the above verbal argument, showing that both *V*_{E} and *V*_{G} increase after the genetic or environmental change; however, the magnitudes of Δ_{E} and Δ_{G} depend greatly on the shape of the distribution of pleiotropic fitness effects of mutant alleles (Figure 3). Given the mean pleiotropic effect *s*_{p}, the values of both *V*_{E,o} and *V*_{G,o} at MSB and of their increases Δ_{E} and Δ_{G} after the genetic or environmental change are small if pleiotropic effect *s* follow an exponential or a less leptokurtic distribution. For example, neither Δ_{E} nor Δ_{G} can exceed 10% if the pleiotropic effect across all loci is 0.05. As the distribution becomes more leptokurtic, however, such that more genes are under weak pleiotropic selection and segregate at higher frequencies, the magnitudes of *V*_{E,o} and *V*_{G,o} and Δ_{E} and Δ_{G} quickly become large. Given the same shape of distribution of pleiotropic effects, the values of *V*_{E,o}, *V*_{G,o}, Δ_{E}, and Δ_{G} decrease with increasing mean *s*_{p}. This is because the association between trait effects (*i.e*., *v* and *b*) and frequencies of mutant alleles weakens if pleiotropic selection is stronger, therefore inducing small increases in both environmental and genetic variances.

Another interesting observation from Figure 3 is the following. As the distribution of mutant effects on residual variance *b* becomes more leptokurtic (*i.e*., β_{b} decreases), *V*_{E,o} maintained at MSB decreases as expected (*cf*. Zhang and Hill 2002 for the similar behavior of *V*_{G} *vs*. the shape of the distribution of trait effect *a*) but Δ_{E} increases. This pattern becomes stronger when the distribution of the pleiotropic effect is strongly leptokurtic. The reason for this is simple. In our model gene frequencies are mainly determined by pleiotropic selection, and hence the association between frequency and residual variance effect *b* of the mutant allele is weak. When the residual variance effect *b* has a strongly leptokurtic distribution, most genes are of very small effects, and *V*_{E,o} is small. As its distribution becomes less leptokurtic so more genes have large effects, *V*_{E,o} becomes larger. After the genetic or environmental change, the present model assumes trait effects of alleles change relative to each other across loci (albeit remaining unchanged on average) but their frequencies remain roughly the same. For a residual variance effect *b* that is less leptokurtically distributed, a few genes that were of small effect now have a large effect and the increase in *V*_{E} is small. For a more leptokurtically distributed *b*, however, there are many such genes, and they collectively lead to a significant increase in *V*_{E} after the genetic or environmental change.

High leptokurtosis of the distribution of *b* (*i.e*., most mutant alleles have small effects) increases the values of *V*_{G,o} and Δ_{G}, especially if the distribution of pleiotropic effect *s* is also very leptokurtic, implying relatively weak selection and high gene frequencies. The sizes of ε_{b} (≡ *E*(*b*)) and can also affect the values of Δ_{E} and Δ_{G}, but this influence is moderate compared to the pleiotropic selection.

## DISCUSSION

We study the response of phenotypic variance for quantitative traits to the environmental stresses to which the population is exposed or to a major mutant it carries. As in Hermisson and Wagner (2004), we assume that effects of mutant alleles change relative to each other, but remain unchanged on average following the change in genetic or environmental background due to epistasis or *G* × *E* interaction. Rather than just assuming that mutant alleles have an effect solely on the mean of the trait, we assume that they also have effects on the mean and residual variance of the trait and fitness. We show that both genetic and environmental variances are predicted to increase after the change in genetic or environmental background. However, the magnitudes of these increases are much influenced by pleiotropic selection. Under strong pleiotropic selection, these increases are small and the unbounded increase of Hermisson and Wagner (2004) is avoided; with weak pleiotropic selection, the increases in variances can be large.

For example, if mutant effects are very leptokurtically distributed, say the pleiotropic effect follows a gamma (0.5) distribution with mean *s*_{p} = 10% and the square of the effect on the mean of the trait follows a gamma (0.1) distribution, then the increase in genetic variance after the genetic or environmental change can reach 20ζ (Figure 2). Here ζ is the interaction parameter. If ζ = 0.1 (Hermisson and Wagner 2004), this is 200%. However, if the pleiotropic effect is exponentially distributed with a small mean *s*_{p} = 1%, the increase in *V*_{G} cannot exceed 3ζ (*i.e*., 30% if ζ = 0.1), which seems smaller than the increase of genetic variance in scutellar bristles of *Drosophila melanogaster* after introduction of the mutant *scute* (Rendel 1967). Further, the increase in genetic variance also depends on the distribution of mutation effects on the mean of the trait, with a large increase for a strongly leptokurtic distribution (Figure 2; *cf*. Figure 2 of Hermisson and Wagner 2004). Similar magnitudes are also found for the increase in environmental variance (Figure 3b). Therefore the magnitude of the increase in phenotypic variance depends on detailed knowledge of the effects of mutants, especially of the shapes of their distribution.

The distribution of pleiotropic effects on fitness of mutations is central to our understanding of the maintenance of genetic variation (Charlesworth *et al*. 1993; Zhang *et al*. 2004). It has been estimated by two methods, either mutation-accumulation and mutagenesis experiments or the analysis of DNA sequence data; but these lead to different conclusions. Results of most of the former experiments indicate that the distribution of pleiotropic effects is platykurtic in form (*i.e*., substantially less leptokurtic than an exponential distribution); while results of the latter analyses give the opposite conclusion, that there are many mutations of very small effect on fitness and its distribution is strongly leptokurtic (*i.e*., far more leptokurtic than an exponential distribution) (Eyre-Walker and Keightley 2007). Huge differences also occur in the estimates of the mutational variance in fitness: the estimate from DNA data is only a tiny fraction of that detected by mutation-accumulation experiments (P. D. Keightley and D. L. Halligan, personal communication). An interpretation of these differences is that there is a cohort of strongly deleterious mutations that are detectable in mutation-accumulation experiments, but that segregate at very low frequencies and are effectively never seen in population samples from nature. Those analyses may indicate that the distribution of pleiotropic effects of *recurrent* mutations is leptokurtic. This is further supported by recent QTL mapping studies using F_{2} crosses. Albert *et al*. (2008) studied a suite of 54 traits describing body shape of three spine stickleback species and Wagner *et al*. (2008) (also Kenney-Hunt *et al*. 2008) studied 70 skeletal traits of mice. Both groups concluded that the QTL identified affected a variable number of traits with the average numbers being very small (3.5 and 7.8, respectively). Estimates of the mean pleiotropic effect on fitness are also very approximate, but Eyre-Walker and Keightley (2007) concluded that a few percent is a typical value across many species. Further, empirical evidence shows that the distribution of mutational effects on quantitative traits is leptokurtic, with most mutations having very small effects and a few having very large effects (Mackay *et al*. 1992; Caballero and Keightley 1994; Lyman *et al*. 1996; Garcia-Dorado *et al*. 1999; Lynch *et al*. 1999; Albert *et al*. 2008; P. D. Keightley and L. D. Halligan, personal communication), but the exact shape of the distribution is not known.

As Hermisson and Wagner (2004) and the present analysis show, the magnitude of the hidden variation that can be released under new conditions depends on genetic architecture and stochastic factors and therefore is expected to vary across traits and species. For traits that have a strongly leptokurtic effect distribution and are under weak pleiotropic selection, *e.g*., scutellar bristles of Drosophila (Rendel 1967), the overall selection is weak and the so-called “conditional neutrality” is likely. Following a large genetic or environmental change, the phenotypic variance will increase considerably. For other traits that are affected by many genes and strongly related to fitness, their quantitative variance will only slightly increase following the change. This study, extending Hermisson and Wagner (2004) to more realistic models by including pleiotropy for fitness and genetic control on environmental variance, provides further support for the theory of Hermisson and Wagner (2004), *i.e*., a *variable interaction scenario* as an explanation for the release of hidden variation in traits after exposure of populations to environmental stresses or introduction of a major mutant.

## APPENDIX: ANALYTICAL RESULTS IN INFINITE POPULATIONS

For a large population so that , the heterozygosity (*i.e*., Equation 2) reduces to and expression (4b) can be simplified as(A1)The gamma distributions are and . In the special case where ζ* _{i}* = ζ and = = β

_{v}, the infinite approximation of Equation 8b is(A2)Here

*E*(

*v*) = β

_{v}/α

_{v}. Letting

*z*=

*s*/

*v*, we haveFurther, letting , we have(A3)where ,

*s*

_{p}(= β

_{s}/α

_{s}) denotes the mean pleiotropic effect on fitness, and

*s*

_{r}≡

*E*(

*v*)/4ω

^{2}= β

_{v}/(4ω

^{2}α

_{v}) represents the mean selection coefficient arising from real stabilizing selection on the trait of homozygous mutants. Similarly we have(A4)

It can be shown from Equations A3 and A4 that if β_{v} + β_{s} ≤ 1, *I*_{2} remains finite but *K*_{2} → ∞ so that Δ_{G} → ∞ in accordance with Equation 9. Otherwise if β_{v} + β_{s} > 1, β_{v} > 0, and β_{s} > 0, *K*_{2} is finite but >*I*_{2}. Integrations for some special distributions are listed in Table A1. When β_{s} = 1, then (1 − θ)*I*_{2} = 1 − θ*K*_{2}. Under strong pleiotropic selection such that θ ≫ 1, approximations are = ln(θ)/θ, ln(4θ)/2θ, and [ln(8θ) + π/2]/4θ for β_{v} = 1, , and , respectively. Therefore the increase in genetic variance under very strong pleiotropic selection relative to real stabilizing selection is very small.

## Acknowledgments

I am grateful to W. G. Hill for helpful discussions and critical comments on drafts of the manuscript and to J. Hermisson, G. P. Wagner, and G. Gibson for their helpful comments on a previous version of the manuscript.

## Footnotes

Communicating editor: G. Gibson

- Received May 16, 2008.
- Accepted July 1, 2008.

- Copyright © 2008 by the Genetics Society of America