# The Effect of Dominance on the Use of the QST − FST Contrast to Detect Natural Selection on Quantitative Traits

^{*}Departamento de Genética, Facultad de Ciencias Biológicas, Universidad Complutense, 28040 Madrid, Spain and^{†}Departamento de Mejora Genética Animal, SGIT-INIA, 28040 Madrid, Spain

- 1
*Corresponding author:*Departamento de Genética, Facultad de Ciencias Biológicas, Universidad Complutense, 28040 Madrid, Spain. E-mail: clfanjul{at}bio.ucm.es

THE comparison between Wright's among-population fixation index *F*_{ST} (a descriptor of the effect of the breeding structure on population diversity for neutral genes) and Spitze's quantitative index of population divergence *Q*_{ST} [defined as *V*_{b}/(*V*_{b} + 2*V*_{w}), where *V*_{b} and *V*_{w} are, respectively, the between-population and the additive within-population components of variance for the trait considered] is widely used to assess the relative importance of selection and drift as determinants of genetic differentiation for quantitative traits. For neutral genes with additive gene action between and within loci, *Q*_{ST} is the quantitative analog of *F*_{ST}. Otherwise, divergent or convergent selection is invoked, respectively, as a cause of the observed increase (*Q*_{ST} > *F*_{ST}) or decrease (*Q*_{ST} < *F*_{ST}) of *Q*_{ST} from its neutral expectation (*Q*_{ST} = *F*_{ST}). However, the validity of the *Q*_{ST} − *F*_{ST} contrast has been questioned, as neutral nonadditive gene action (dominance and/or epistasis) can mimic the additive expectations under selection (López-Fanjul *et al.* 2003). Recently, Goudet and Büchi (2006) have compared *Q*_{ST} and *F*_{ST} using computer simulation for dominant (nonepistatic) loci and have concluded that (1) on average, dominance decreases the value of *Q*_{ST} relative to *F*_{ST} (*i.e.*, *Q*_{ST} − *F*_{ST} < 0); (2) the magnitude of the contrast *Q*_{ST} − *F*_{ST} increases with population differentiation (*i.e.*, with increasing *F*_{ST}); and (3) dominance is unlikely to produce the result *Q*_{ST} − *F*_{ST} > 0. In this Letter to the Editor, we question the evidence that led to these claims.

It is well known that inbreeding can change the magnitude of the contribution of neutral dominant loci (with or without additional epistasis) to the between-population and the additive within-population components of the genetic variance of a quantitative trait, relative to their additive expectations (Robertson 1952; Willis and Orr 1993; López-Fanjul *et al.* 2002; Barton and Turelli 2004). Therefore, those changes can also affect the magnitude and sign of the contribution of such loci to *Q*_{ST}, depending on their effects and allele frequencies as well as on the pertinent *F*_{ST} value. Using the formulas given in López-Fanjul *et al.* (2003), we show in Figure 1 the contribution of a partially recessive diallelic locus (*h* = 0.25; see below) to *Q*_{ST} − *F*_{ST}, as a function of *F*_{ST} and the original frequency of the recessive allele (*q*). Generally, *Q*_{ST} − *F*_{ST} < 0 for *q* < 2/3 and *Q*_{ST} − *F*_{ST} > 0 otherwise, but |*Q*_{ST} − *F*_{ST}| absolute values for *q* < 2/3 were larger than those for *q* > 2/3. In addition, for all allele frequencies, |*Q*_{ST} − *F*_{ST}| values increased with *F*_{ST} until a maximum was reached when *F*_{ST} was close to 0.5 and then declined. Varying degrees of dominance *h* (0 ≤ *h* ≤ 1/2, where *h* = 0 or 1/2 for complete recessive or additive gene action, respectively) resulted in essentially the same pattern but, for each frequency and *F*_{ST}, the corresponding |*Q*_{ST} − *F*_{ST}| values increased as *h* departed from *h* = 1/2, and the frequencies at which *Q*_{ST} − *F*_{ST} < 0 went from *q* < 3/4 (*h* = 0) to *q* < 1/2 (*h* close to 1/2) (not shown). Of course, *Q*_{ST} = *F*_{ST} for *h* = 1/2, irrespective of allele frequencies. In Goudet and Büchi (2006) notation, *h =* (*a* − *d*)*/*2*a*, where 2*a* is the difference between the genotypic values of the two homozygotes at a diallelic locus and *d* is the value of the corresponding heterozygote.

Goudet and Büchi (2006) presented results averaged over uniformly distributed allele frequencies and concluded that *Q*_{ST} − *F*_{ST} < 0 for dominant loci. However, this is the obvious outcome of averaging *Q*_{ST} − *F*_{ST} values of a different sign, which are negative over a wider range of allele frequencies. As pointed out by Turelli and Barton (2006), this procedure can be misleading as it ignores that qualitatively different results are obtained for different allele frequencies and, therefore, the critical role of particular loci is lost when one averages over the whole possible range of original allele frequencies. Thus, the conclusion of Goudet and Büchi (2006) that dominance is unlikely to cause *Q*_{ST} − *F*_{ST} > 0 may not be valid for a plausible region of parameter space. Furthermore, the magnitude of *Q*_{ST} − *F*_{ST} contributed by single loci did not monotonically increase with population differentiation, as indicated by Goudet and Büchi (2006), but it did so only until a value of *F*_{ST} ≈ 0.5 was reached and then subsequently decreased to zero. Our formulas apply to the classical “island model” and give the expected contribution of single loci at equilibrium to the between-line and the additive within-line components of the genetic variance of a quantitative trait (or to *Q*_{ST}) after *t* consecutive bottlenecks of size *N* or, when bottleneck sizes are not constant from generation to generation, they can be alternatively expressed in terms of the inbreeding coefficient after *t* generations for the loci affecting the trait. These properties have also been incorrectly interpreted by Goudet and Büchi (2006; see their discussion section, p. 1344).

In the simulations by Goudet and Büchi (2006), the trait studied was encoded by 10 loci with 10 alleles/locus. For each locus, 10 additive allelic effects and 45 dominance deviations were drawn independently from a normal distribution *N*(0, 1), and genotypic values were obtained as the sum of the additive effects of the pertinent alleles plus the corresponding dominance deviation in the case of heterozygotes. Individual genotypic values were made equal to the sum of the genotypic contributions of the 10 loci considered. This procedure, however, has several drawbacks. First, it implies that the additive and dominance effects are independent, an unrealistic assumption as it is well known that loci with large homozygous effects tend to be recessive. Second, the additive and dominance effects were assigned arbitrarily, thereby resulting in complex dominance relationships among the 10 alleles at each locus (including overdominance and underdominance); *i.e.*, a large region of the parameter space being explored is biologically unrealistic, as overdominance (or underdominance) has been empirically discarded as a common explanation of heterosis at the single-locus level. Third, and more important, the method implies averaging over a wide range of dominance deviations and allele frequencies; therefore, the crucial role of a particular type of gene action is again lost. For instance, after bottlenecks, recessive (nonepistatic) loci will contribute only to an excess of the within-population additive variance (or to *Q*_{ST} − *F*_{ST} < 0) if the recessive alleles segregate at low frequencies, but overdominant loci will do so only for equilibrium (intermediate) frequencies (López-Fanjul *et al.* 2002).

To investigate these points, we have simulated the simplest case of a single locus with two alleles, originally segregating at frequencies *q* and 1 − *q*, with additive and dominance effects sampled from a normal distribution as in Goudet and Büchi (2006). Starting from a large base population at Hardy–Weinberg equilibrium, lines were derived and subsequently maintained with effective size *N* = 10 during a number of generations; the pertinent values of *Q*_{ST} − *F*_{ST} plotted against *F*_{ST} are given in Figure 2 for a range of initial frequencies 0.1 ≤ *q* ≤ 0.9 and inbreeding coefficients 0 < *F*_{ST} < 1 (1500 replicates). Our results clearly show that *Q*_{ST} was always smaller than *F*_{ST} (*Q*_{ST} − *F*_{ST} < 0) for all allele frequencies (as found by Goudet and Büchi 2006); *i.e.*, they are artifacts of averaging the *Q*_{ST} values obtained for the same allele frequency and *F*_{ST} over the distribution of dominance effects generated by sampling and are not representative of the general situation described by our analytical model.

In summary, the extension of single-locus models to multilocus systems can have different outcomes, as the contribution of specific dominant loci to the *Q*_{ST} − *F*_{ST} value for a quantitative trait depends, both in sign and magnitude, on the variance of the average effects of gene substitution, the covariance between these effects and the corresponding heterozygosities, and the allele frequencies at the loci involved (López-Fanjul *et al*. 2003). In other words, generalizations to quantitative traits can be made only if individual loci show the same type of gene action and segregate with similar frequencies.

Finally, it is generally accepted that alleles of large effect on a given trait tend to be recessive and deleterious and, therefore, natural selection holds their frequencies at low values. This is the case of Drosophila viability, where Wang *et al.* (1998) have shown that the observed changes in mean, additive genetic variance, and between-line variance following bottlenecks can be mainly attributed to lethals and partially recessive mutations of large deleterious effect, giving *Q*_{ST} < *F*_{ST} [*Q*_{ST} = 0.17 or 0.20 for *F*_{ST} = 0.25 or 0.5, respectively (López-Fanjul *et al.* 2003)]. However, this may not be the case with other quantitative traits where loci of relatively smaller effects and varying degrees of dominance can contribute differentially to a joint *Q*_{ST} − *F*_{ST} value, which may be of either sign or even statistically undistinguishable from zero.

## Acknowledgments

We thank J. Goudet for information on his simulation methodology and A. Caballero, A. García-Dorado, and an anonymous reviewer for useful comments on earlier drafts. This work was supported by grants CGL2005-02412/BOS (Ministerio de Educación y Ciencia) and CPE03-004-C2 (Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria).

## Footnotes

Communicating editor: P. Phillips

- Received July 3, 2006.
- Accepted February 23, 2007.

- Copyright © 2007 by the Genetics Society of America