A GENERAL MODEL OF MIGRATION AND DRIFT IN STRUCTURED POPULATIONS

We assume that individuals within a local population (deme) can be classified into different classes, e.g., with respect to age and/or sex. Let f_ijrs be the probability of identity of a pair of genes sampled from an individual of class r in deme i and an individual of class s in deme j. We can write m_ijru for the probability that a gene sampled from an individual of class r in deme i originated from an individual of class u in deme j.

When the probability per time interval of mutation of an allele to a new state, μ, is independent of allelic state, class, and deme, f_ijrs is equivalent to the momentgenerating function for the time to coalescence of a pair of genes of the specified type, with parameter -2μ (Hudson 1990; Wilkinson-Herbots 1998). (To allow for the possibility of age structure, the time interval does not necessarily correspond to one generation.) In particular, their mean time to coalescence, T_ijrs, is equal to the derivative of f_ijrs with respect to -2μ at μ= 0. As usual, we assume that μ is sufficiently small that second-order terms can be neglected. We later consider the case of sex-specific mutation rates, using mean coalescence times derived in this way; this is legitimate, since coalescence times are independent of the mutation model.

Assuming that different individuals migrate independently of each other, and using the standard approach to modeling a geographically structured population (Nagylaki 1982, 1998a), we can write the equation for equilibrium, fijrs=(1−2μ){∑k≠l∑uvmikrumjlsvfkluv+∑k∑uvmikrumjksv[Prsuvijk+(1−Prsuvijk)fkkuv]}, (1) where Prsuvijk is the probability of coalescence over one time interval of two genes sampled from individuals of class r in deme i and s in deme j, derived from individuals from deme k who belong to classes u and v, respectively (note that Prsuvijk=0 unless u = v).

For i = j and r = s, and if migration involves individuals rather than gametes, we should strictly include a term on the right-hand side that covers the case when the two genes sampled come from the same migrant individual. The approximations leading to Equation 3 mean that this is unnecessary, since ignoring it introduces only a second-order term into the final result. (This is because the chance that a pair of genes from the same deme are sampled from the same individual is of the order of the reciprocal of the deme size, and we ignore products of this and the migration probabilities.)

Writing h_ijrs = 1 - f_ijrs for the probability of nonidentity of a pair of alleles, Equation 1 can conveniently be rewritten as hijrs=2μ+(1−2μ){∑uv[∑klmikrumjlsvhkluv−∑kmikrumjksvPrsuvijkhkkuv]}. (2)

To make further progress, we assume that migration rates between demes are of the same order as the probabilities that two genes sampled from the same deme coalesce over one time unit and that both are sufficiently small that second-order terms in these, as well as mutation rates, can be neglected. This assumption is commonly made in applications of coalescent theory (Hudson 1990; Wilkinson-Herbots 1998); if migration rates are much greater than these coalescent probabilities, little differentiation between demes is possible, so that this is not a particularly restrictive assumption.

To this order of approximation, the last term in braces on the right-hand side of (2) is nonzero only when i = j = k, corresponding to the case of two genes drawn from the same deme, hijrs≈2μ+(1−2μ)∑uv∑klmikrumjlsvhkluv−δij∑uPrsuihijuu, (3) where δ_ij is the Kronecker delta (δ_ij = 1 when i = j and is otherwise 0); Pⁱ_rsu is the probability that two genes sampled from deme i from individuals belonging to classes r and s coalesce in an individual of class u in the previous time interval, treating the deme as a closed population.

To obtain manageable results, we simplify further by removing the dependence of h_ijrs on r and s. We can average h_ijrs values over classes within demes by using a weight of α_ir for the contribution from class r in the ith deme. α_ir is chosen to be equal to the rth element of the left leading eigenvector of the matrix that describes the flow of genes among classes within deme i, scaled such that the elements of this eigenvector sum to one. This element represents the stationary-state probability that a randomly sampled gene from deme i originates from class r (Nordborg 1997). If the convergence to this stationary state is much faster than the rate of coalescence and migration, α_ir gives an accurate approximation to the probability of origin of the sampled gene (Nordborg 1997); this is the case, for example, for a discrete generation population, when the classes refer to different sexes. This also implies that the h_ijrs values can be treated as approximately independent of r and s; i.e., they can be equated to a common term, h_ij. This is equivalent to the fast-migration limit of Nagylaki (1980) for a geographically structured population.

We can then conveniently define the net probability of origin from deme k of a gene sampled from deme i, m_ik, as mik=∑ruαirαkumijru. (4)

Substituting into Equation 3 and summing α_irα_jsh_ijrs over r and s, we obtain hij=∑rsαirαishijrs≈2μ+(1−2μ)∑klmikmjlhkl−δijPihii, (5) where P_i is the probability of coalescence of a pair of genes sampled randomly from deme i, such that Pi=∑rsuαirαisPrsui. (6)

Using the argument of Charlesworth (2001), we can write Pi=12tiNie, (7) where t_i is the generation time of individuals from deme i and N_ie is the deme’s effective population size. In the case of discrete generations, t_i = 1 for all demes.

By differentiating h_ij with respect to -2μ, we obtain the following expression for the mean coalescent time for a pair of alleles sampled from demes i and j (Hudson 1990): Tij≈1+∑klmikmjlTkl−δijPiTii. (8)

This provides a general set of linear relations that can be used to solve for the T_ij, to the assumed order of approximation, for any specific model. Some examples are described below. Higher moments of the distribution of pairwise coalescence times can similarly be obtained by successive differentiation of Equation 5 (Wilkinson-Herbots 1998).

It is also useful to apply the method of Nagylaki (1998a) for determining a weighted mean coalescent time for a pair of genes sampled from the same deme. The migration rates defined by the right-hand side of Equation 4 are identical in properties to the components of the standard genic migration matrix. Under the usual assumptions about this matrix (Nagylaki 1980, 1982, 1998a), it has a leading left eigenvector, ν, whose elements can be assumed to sum to one without loss of generality. Using the argument of Nagylaki (1998a), this suggests the definition of a weighted mean coalescence time, T₀, for a pair of alleles sampled from the same deme as T0=∑iνi2PiTii∑iνi2Pi=1∑iνi2Pi, (9) where the coalescent time for each deme is weighted by the product of the reciprocal of the coalescence probability and the square of the corresponding component of ν. In the discrete generation case, T₀ corresponds to twice the “migration effective size” as defined by Nagylaki (1980, 1982, 1998a) and reduces to the standard expression for this quantity in the case of Wright-Fisher sampling within demes of size N_i, for which P_i = 1/(2N_i).

COALESCENT PROBABILITIES AND EFFECTIVE POPULATION SIZES

Separate sexes with discrete generations: The utility of the approach described above to determining the effective population size of a deme can be illustrated by its application to the discrete-generation case with separate sexes, previously treated by Caballero (1995). It enables us to provide biologically explicit expressions for N_e under different modes of inheritance, on the lines developed by Charlesworth (2001) for the age-structured case.

From the assumption that the products of coalescence probabilities and migration probabilities are negligible, used to derive (3) and its successors, it is legitimate to treat each deme as a closed population. We have to consider the origins of genes sampled from individuals of all permissible combinations of sexes and parents (which combination is permissible depends on the mode of genetic transmission). For a pair of maternally derived genes sampled from two different females, the probability that they both come from the same mother is approximately Qfffi=∑kdik(dik−1)(Nifd‒i)2 (10) (Caballero 1995), where N_if is the number of breeding females in deme i, d_ik is the number of daughters produced by the kth female in deme i, and d¯_i is the mean number of daughters per female in deme i.

This can be simplified by noting that the Poisson value for the variance in numbers of daughters per female, Vffi , is d¯_i, so that in general the deviation of this variance from the Poisson value can be written as ΔVffi=Vffi−d‒i (11) so that Qfffi=(ΔVffi∕d‒i2)+1Nif. (12) Similar expressions can be obtained for other pairs of individuals and their parents (see Table 1).

Let β_irsu be the probability that a pair of genes sampled from individuals of sexes r and s in deme i both come from parents of sex u. Let γ_irsu be the probability that this pair of genes shared a common ancestral allele in the previous generation, given that they have a parent in common; i.e., they coalesce. From Equation 7, the net probability that a randomly chosen pair of genes coalesce in the previous generation is 12Nie=Pi=∑rsuαirαisβirsuγirsuQrsui, (13) where Qrsui is the probability that a pair of genes sampled from individuals of sexes r and s in deme i with a parent of sex u derives from the same parental individual.

Table 2 gives the values of these transmission probabilities for different modes of inheritance; these can be combined with the Q values from Table 1 and substituted into Equation 15, to obtain the final expressions for effective population sizes. Simplified expressions can be obtained when there is a fixed sex ratio among breeding individuals, so that there is a binomial distribution of the proportions of sons and daughters of a given individual, counted at maturity (see the appendix).

For convenience, we henceforth drop the superscripts and subscripts indicating deme identities. Using Tables 1 and 2, substituting into Equation 15, and noting that the Poisson expectation for variance in total offspring number is 1/(1 - c) for females and 1/c for males, where c is the proportion of males among breeding adults, we obtain the expression for effective population size with autosomal inheritance, 1NeA≈(1+F)4{1Nf+1Nm+(1−c)2ΔVfNf+c2ΔVmNm}, (14) where ΔV_f and ΔV_m are the excesses over Poisson expectation of the total numbers of offspring per capita for female and male parents, respectively; F is the inbreeding coefficient associated with departure from random mating within demes.

For X-linked inheritance with male heterogamety, we have 1NeX≈19{4(1+F)Nf+2Nm+4(1+F)(1−c)2ΔVfNf+2c2ΔVmNm}. (15)

For Y-linked inheritance 1NeY≈2(1+c2ΔVm)Nm. (16) With female heterogamety, males and females are interchanged in these expressions.

For maternal inheritance 1NeC≈2(1+[1−c]2ΔVf)Nf. (17)

Partially self-fertilizing hermaphrodites: It is also of interest to consider the case of a nuclear gene in a population of partially self-fertilizing hermaphrodites; this is particularly relevant for plants. (Maternally transmitted genes are clearly unaffected by the selfing rate, except as far as changes in the breeding system alter the distribution of numbers of successful offspring produced through seed.) With a Poisson distribution of offspring number for both seed and pollen, the effective size is reduced below that for random mating by a factor of 1/(1 + F), where F is the equilibrium inbreeding coefficient produced by the level of self-fertilization (Pollak 1987; Nordborg and Donnelly 1997). We examine the effect on this result of deviations from a Poisson offspring number distribution.

Let there be N breeding individuals in the deme under consideration; the kth individual is assumed to produce d_kS progeny through self-fertilized seed that survive to form part of the breeding population next generation, and d_kO such progeny through outcrossed seed. In addition, it contributes p_k progeny through pollen that fertilizes other individuals. The overall probability that a progeny individual is the product of self-fertilization is then S=∑idiS∑i(diS+diO)=d‒S(d‒S+d‒O), (18) where d¯_S and d¯_O are the mean numbers of successful progeny per capita produced by selfed and outcrossed seed, respectively.

Details of the derivation of probabilities of gene origins are given in the appendix. In the case of constant population size, we have d¯ = 1, d¯_S = S, and d¯_O = p¯ = (1 - S). The expression for the effective population size for a single deme, N_eH, is 1NeH≈(1+F)N{1+(ΔVf+ΔVm+3ΔVfS)+2[CfOfS+Cfm+CfSm]4} (19a) (definitions of the terms in the numerator are given in the appendix).

If the number of successful offspring through selfed seed, conditioned on the total number of successful offspring produced through seed, can be regarded as a binomial variate with parameter S (i.e., there is the same expected selfing rate for each individual), this simplifies further to 1NeH≈(1+F)N{1+([1+S]2ΔVf+ΔVm+2[Cfm+CfSm])4}. (19b) In the limit of complete self-fertilization, the contributions from outcrossing pollen vanish, and F = 1, so that 1NeH≈2(1+ΔVf)N. (20)

A model is presented in the appendix for analyzing the case of intermediate selfing rates. The overall conclusion is that non-Poisson variation in reproductive capacity may not greatly alter the standard result that N_e for a selfing population is reduced by a factor of 1/(1 + F), relative to the value for an outcrossing population (Pollak 1987), although a very high sensitivity of female fitness to allocation to reproduction could lead to a greater reduction in N_e than this.

THE ISLAND MODEL WITH SEX-SPECIFIC MIGRATION PARAMETERS

In this section, we apply the above principles to the simple case of an island model (Wright 1943) with sex-specific migration rates, for dioecious populations under a variety of modes of inheritance. We assume that the total population is divided into n demes, each with the same effective population size, N_e, given by the equations derived above. There are three dispersal parameters, describing the probability that a male gamete (m_m) in a zygote in a given deme has migrated from a different deme, the corresponding probability for a female gamete (m_f), and the probability that a zygote is itself derived from a different deme (m_z; Chesser and Baker 1996; Wang 1999). Under the assumptions used above, these migration events can be treated as mutually exclusive, since their probabilities are each low. Migration probabilities between specific demes, as considered above, are given by dividing the dispersal parameters by (n - 1).

In plant species, male gametes can disperse through pollen but female gametes cannot disperse, so that only m_m and m_z can be nonzero. For animals, migration of male and female gametes occurs via dispersal of males and unmated females; dispersal of zygotes occurs when females migrate after mating but before laying eggs or giving birth. All three migration parameters can therefore be nonzero.

Using the argument leading to Equation 4, we can then define the net migration rates for different modes of inheritance (autosomal, X-linked, Y-linked, and cytoplasmic) as m_A, m_X, m_Y, and m_C, respectively. We have mA=mz+12(mm+mf) (21a) mX=mz+13(mm+2mf) (21b) mY=mz+mm (21c) mC=mz+12mf. (21d)

Coalescence times and expected nucleotide site diversities: Results for mean pairwise coalescence times in the island model can be obtained by substituting these expressions into standard formulas (Slatkin 1991; Wilkinson-Herbots 1998), which correspond to approximate solutions to Equation 8. To the assumed order of approximation, with mode of inheritance G (where G is A, X, Y, or C) the expected coalescence time, T_0G, for two genes sampled from the same deme is given by T0G=2nNeG, (22a) where N_eG is the appropriate effective population size for a single deme.

Using Equation 8, or the argument of Slatkin (1991), the approximate expected coalescence time for a pair of genes sampled from different demes is T1G=T0G+(n−1)2mG, (22b) where m_G is the migration rate defined by the relevant choice of Equations 21a, 21b, 21c, 21d for mode of inheritance G.

The expected coalescence time for a pair of genes sampled randomly from the set of populations is TG=1nT0G+(n−1)nT1G. (22c)

The expected pairwise nucleotide site diversity under the infinite-sites model is given by the product of coalescence time and the mutation rate μ_G for the given mode of inheritance (Hudson 1990). From Equations 22a, 22b, 22c, the expectations of nucleotide diversities for alleles sampled within demes, π_SG, and from the population as a whole, π_TG, are given, respectively, by πSG=4nNeGμG (23a) πTG=πSG(1+(n−1)2n214mGNeG). (23b)

Mutation rates may differ among genetic systems because of differences in mutation rates between males and females (Hurst and Ellegren 1998; McVean 2000) or because of chromosome-specific mutation rates (McVean and Hurst 1997). In the former case, the overall mutation rate is given by weighting the sex-specific or chromosome-specific mutation rates by the appropriate α values (Charlesworth 2001).

The absolute magnitude of between-population subdivision can be measured by the difference between the nucleotide diversity for the population as a whole and the mean for a pair of alleles sampled from the same deme (Charlesworth 1998). The expectation of this is given by the second term in brackets on the right of (23b). It is often more convenient to use a relative measure, such as F_ST. Using the definition of Hudson et al. (1992), the theoretical value of F_ST for a given mode of inheritance is given by FST,G=(πTG−πST)πTG. (24a) With a large number of demes, this yields the familiar result of Wright (1951): FST,G=14NeGmG+1. (24b)

We now consider the application of these formulas to some specific examples. We focus initially on the effects of population subdivision, assuming a 1:1 sex ratio, Poisson variances in fertility for both sexes, and equal mutation rates for males, females, and different chromosomes. Discrete generations and a deme size of N breeding adults for all demes are also assumed. In this case, the ratios of effective population sizes and within-deme diversities for different modes of inheritance G and G′ (r_GG′) are simply equal to the ratios of the respective numbers of gene copies in the population: r_XA = ¾, r_XY = r_XC = 3, r_AY = r_AC = 4, the same values as for panmixia, which are often quoted in the literature on molecular population genetics.

The effects of sex-specific migration and population subdivision in dioecious plants: In Figure 1, we plot the values of F_ST for nuclear and cytoplasmic genes and the ratios of the total diversities for different modes of inheritance, as functions of the total number of migrants per generation (give by the sum of pollen and seed migration rates times the deme size). Male heterogamety is assumed.

As expected, reduced migration increases F_ST for all modes of inheritance. However, the rate of increase of F_ST as the amount of migration decreases is different for different modes of inheritance and also depends on the mode of migration. For instance, with equal pollen and seed migration rates, F_ST,Y and F_ST,C increase faster than F_ST,A and F_ST,X (Figure 1A). These differences result from differences in the numbers of effective migrants, due to two factors: (i) different effective population sizes and (ii) different gene migration rates. The effective number of X chromosomal migrants is lower than the effective number of autosomal migrants, due to a lower effective population size of the X chromosomes (3N/4) as well as its lower migration rate (m_z + ¹/₃m_m vs. m_z + ½m_m). The effective number of Y chromosomal migrants is lower than both the effective number of chromosomal X or autosomal migrants, due to the lower effective population size of the Y chromosome (N/4), which is not compensated for by its higher migration rate (m_z + m_m) relative to the X chromosome. A similar argument applies to cytoplasmic genes.

Population subdivision also modifies the ratios of the total diversities for different modes of inheritance: the ratios π_TX/π_TY(R_XY) and π_TA/π_TY(R_AY) decrease sharply with reduced gene flow and π_TX/π_TA(R_XA) increases slightly (Figure 1D). From Equations 23a, 23b, it is easily seen that, in the limit when the migration rates tend toward zero, the R values are equal to the reciprocals of the ratios of the respective migration rates given by Equations 21a, 21b, 21c, 21d, since the right-hand side is dominated by the term involving the reciprocal of the product of effective size and migration rate. In this case, the limiting total diversities for autosomal and X-linked genes are only 1.33- and 1.5-fold higher than for Y-linked loci, respectively, with the diversity for X-linked loci being 1.12 that for autosomal loci. Similarly, the limiting ratio of autosomal to cytoplasmic total diversities becomes 0.67.

Differences between the pollen and seed migration rates also influence the rate of increase in F_ST. For instance, if migration occurs primarily through pollen (e.g., with m_m = 100m_z, Figure 1B), F_ST,A and F_ST,X increase faster with decreasing migration than when migration occurs equally through pollen and seeds (Figure 1A), or primarily through seeds (m_m = 0.01m_z, Figure 1C), whereas F_ST,C increases more slowly, but is higher throughout the range of Nm displayed. This is because X-linked and autosomal genes are haploid in pollen grains but diploid in seeds, and cytoplasmic genes move only through seed. In contrast, the Y chromosome, which is haploid in both dispersal units, is not affected by the relative pollen and seed migration rates. In consequence, the curves relating the diversity ratios to total migration rate are also modified. In particular, when migration occurs primarily through pollen, the increased migration rate of the Y chromosome compared to the X chromosome (m_m vs. m_z/3) compensates nearly totally for its reduced effective population size, so that R_XY is independent of population subdivision (Figure 1E). In contrast, R_XA is now slightly more affected by population subdivision, because the difference between the X and autosomal migration rates increases with the pollen migration rate. Hence, in the limit when the migration rates tend toward zero, R_XY barely declines below 3, but R_AY declines to 2, and R_XA increases to 1.5.

The effects of sex-specific migration and population subdivision in animals: We consider a model of animal migration with up to a sixfold difference between male and female migration rates and no zygotic migration (see above), assuming male heterogamety (for female heterogamety, male and female parameters are interchanged). As compared with the plant case, we can see that the case of predominantly male migration (Figure 2, B and E) is similar to the case of predominantly pollen migration in the plant model. The case of equal male and female migration rates is similar to the case of equal pollen and seed migration rates, except that limited migration has a much stronger effect on F_STY relative to F_STA and F_STX (Figure 2A). The effect is even stronger with predominantly female migration where F_STY increases much faster with reduced migration than F_STA or F_STX (Figure 2C). Consequently R_AY and R_XY show a greater reduction when migration is restricted (Figure 2, D and F) than in the plant model (Figure 1, D and F). These differences are easily understood since Y-linked genes now experience migration through only 1 unit (male individuals), while they had two in the plant model (pollen and seeds).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

—The F_ST parameters (A-C) and the ratios (R) of total diversities between genes with different modes of inheritance (D-F) in a subdivided plant population (m_f = 0), as functions of the number of migrants per deme, N(m_m + m_z). (A and D) Equal pollen and seed migration. (B and E) Predominantly pollen migration (m_m = 100m_z). (C and F) Predominantly seed migration (m_z = 100m_m). We assume N_eA = N, N_eX = ¾.N, N_eY = N_ec = ¼.N.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2.

—The F_ST parameters (A-C) and the R values (D-F) in a subdivided animal population (m_z = 0), as functions of the number of migrants per deme, N(m_m + m_f). (A and D) Equal male and female migration. (B and E) Predominantly male migration (m_m = 6m_f). (C and F) Predominantly female migration (m_f = 6m_m). We assume N_eA = N, N_eX = ¾.N, N_eY = N_ec = ¼.N.

The effects of sex- or chromosome-specific mutation rates: Under the infinite-sites model, the mutation rate is so low compared with any realistic migration rate that differences in mutation rates among different modes of inheritance will not affect the relative F_ST values. Absolute values of diversities will, however, be affected, since the ratios for different modes of inheritance will be multiplied by the ratios of the respective mutation rates. From Equations 23a, 23b, this does not affect the shapes of the R values as functions of migration rates, but does affect their heights. For example, if the male mutation rate is higher than the female mutation rate (Hurst and Ellegren 1998; McVean 2000), this will have the effect of reducing R_XA, R_XY, and R_AY. This would enhance the effect of high relative female migration rates in the animal model on these ratios, which has just been noted.

The effects of nonrandom variation in fertility: It is well known that an increase in the variance of fertility reduces N_e. Sex differences in the distributions of fertility modify the relative N_e values of genes with different modes of inheritance (Caballero 1995; Charlesworth 2001). For example, an increased variance in male fertility reduces the effective deme size of the Y chromosome more drastically than that for the X or the autosomes. With an extremely high variance in male fertility relative to female fertility, the ratios N_eX/N_eY and N_eA/N_eY become 9 and 8, respectively, so that N_eX/N_eA is 9/8. In contrast, a high variance in female fertility reduces N_e for the X and the autosomes but does not affect the Y chromosome: the ratios N_eX/N_eY and N_eA/N_eY tend to zero and N_eX/N_eA to 9/16.

As far as total diversity measures are concerned, population subdivision always counteracts the effect of an increased variance in male fertility by reducing the ratios R_XY and R_AY, as illustrated in the animal model with large ΔV_m (Figure 3). Since N_eY is reduced by an increased male fertility variance, the effective number of Y migrants is much lower than the number of X or autosomal migrants. As the migration rate decreases, F_ST,Y now increases much faster than F_ST,X or F_ST,A, and the ratios R_XY and R_AY decrease toward one with equal male and female migration (Figure 3, B and E). With predominantly female migration, this effect is manifest even with relatively high migration (Figure 3F). For instance, R_XY = 6.2 and R_AY = 5.7 with very high migration, but are already halved with as many as 10 effective migrants. With an increased variance in female fertility, the effect of population subdivision depends on both the relative effective population sizes and the relative migration rates for different modes of inheritance. Population subdivision increases R_XY (R_AY) if N_eX·m_X < N_eY·m_Y (N_eA·m_A < N_eY·m_Y). However, this effect occurs only with very restricted migration, as illustrated in the plant model with ΔV_f = 5 (Figure 4). Moreover, the maximum value of R_XY (R_AY) is always lower than the expectation for a panmictic population with Poisson distributions of offspring numbers.

Selection on a nonrecombining genome, such as organelle genomes and the Y chromosome, can further reduce N_e (Charlesworth and Charlesworth 2000). Such a reduction of N_ec and N_ey will increase the corresponding F_ST values. We have investigated the case with N_eX/N_eY = 30 for the plant model (Figure 5), since this is the ratio suggested by observed within-deme diversity ratios in Silene latifolia (see below). With such a low N_eY, the decrease in R_XY and R_AY due to population subdivision now occurs even with relatively high migration rates. Moreover, with such a low N_eY, differences between pollen and seed dispersal rates have little effect on the relative total diversities of nuclear genes with different modes of inheritance.

COMPARISONS WITH DATA ON NATURAL POPULATIONS

In this section, we compare some of the results derived above with data from surveys of DNA sequence variation in populations of humans and plants.

Human populations: There is a large literature on genetic diversity in humans, and many different types of markers have been employed (including protein polymorphisms, restriction fragment length polymorphisms, microsatellites, Alu insertions, and single-nucleotide polymorphisms). These data have, however, several biases, which limit their utility for our purposes. First, worldwide population structure has rarely been investigated using markers with different modes of inheritance in the same samples (but see Poloniet al. 1997; Jordeet al. 2000). In addition, the mode of sampling from different populations, or of grouping the populations, may influence F_ST estimates, as described by Stoneking (1998) and Hammer et al. (2001; see Table 3). Moreover, different estimators of F_ST, which depend in different ways on the sample sizes and numbers of populations sampled (Cockerham and Weir 1993; Charlesworth 1998), have been used in different studies, making it hard to compare different results. This is illustrated by the different estimates of F_ST,Y obtained from the same dataset by Seielstad et al. (1998) and by Hammer et al. (2001). Finally, because of the low level of sequence diversity in humans, investigators have favored the use of highly polymorphic markers, especially for the Y chromosome. This may bias F_ST estimates, as noted by Nagylaki (1998b) and Bertranpetit (2000), since diversity values for loci with high mutation rates are not proportional to coalescent times.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3.

—Same as Figure 2 but assuming an increased variance of male fertility compared to the Poisson expectation (ΔV_m = 10).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4.

—Same as Figure 1 but assuming an increased variance of female fertility compared to the Poisson expectation (ΔV_f = 5).

• Download figure
• Open in new tab
• Download powerpoint

Figure 5.

—Same as Figure 1 but assuming N_eY = N_eX/30. The figures also give the F_ST and R_XY estimates from Silene latifolia populations with the number of migrants expected under an island model (Triangle, Fˆ_ST,Y; square, Fˆ_ST,A; circle, Fˆ_ST,X; and cross, Rˆ_XY).

In Table 3, we present a compilation of F_ST estimates from worldwide samples of different types of markers, with cytoplasmic, Y, X, and autosomal inheritance. These all provide evidence for population structure, with autosomal markers yielding the lowest mean F_ST estimates, as expected from the results shown in Figure 2. The estimates obtained from Y chromosome markers are, however, highly variable between studies (0.09-0.65). Microsatellites often show smaller F_ST values (0.09-0.23), as outlined in studies that have compared both kinds of markers (Hammeret al. 1997; Jordeet al. 2000), probably because of their high mutation rates. Excluding microsatellites, the average F_ST estimated from the Y chromosome markers (0.40) is higher than from the mitochondrial markers (0.27). While this has often been interpreted as evidence for higher female than male migration rates, it might also reflect differences in effective population sizes (Hurles and Jobling 2001). A recent comparison of local population differentiation in Thai communities with patrilocal or matrilocal dispersal patterns is, however, in agreement with the expectation that restricted male migration increases population differentiation for Y markers relative to mitochondrial markers (Ootaet al. 2001).

DNA sequence data have recently been obtained for 4 Y-linked and 15 autosomal loci, using similar worldwide samples (Shenet al. 2000; Thomsonet al. 2000; Thorstensonet al. 2001). The authors estimated R_AY as five, from data on both coding and noncoding regions. Unfortunately, population structure was not investigated explicitly in these studies, although the data should allow partitioning of diversity between and within populations. Thomson et al. (2000) constructed a genealogy of their sample of Y chromosomes, and this can be used to obtain a value of 0.14 for F_ST,Y between continents. This is smaller than that obtained from other studies of biallelic markers and challenges the notion that there is restricted male vs. female migration in most human populations. Mitochondrial data for the same samples would help to resolve this question, and it would also be desirable to obtain estimates of F_ST,A and r_AY.

At present, it is hard to reach firm conclusions about the influence of sex-specific migration vs. differences in effective population sizes on the relative levels of diversity and divergence between human populations for different inheritance modes; there does not, however, seem to be strong evidence for a greatly reduced effective size of the human Y chromosome, in contrast to what is observed in Silene (see below) or in Drosophila (Zurovcova and Eanes 1999; Bachtrog and Charlesworth 2000, 2002).

Plant populations: There are relatively few species of plants with sex chromosomes, and diversity data on nuclear genes with different modes of inheritance are available only for the close relatives S. latifolia and S. dioica (Table 4). In S. dioica, there is only a single polymorphic site on the Y, which is too little to provide any useful information. In S. latifolia, nine polymorphic sites were found on the Y. All nuclear genes display quite strong population structure, with the lowest F_ST for the autosomal gene and the highest for a Y-linked gene, as expected from the above analyses. The ratio R_XY over all demes (R_XY = 23) is smaller than the ratio r_XY for within-deme diversity (r_XY = 29), as expected from the effects of subdivision (Figure 5).

It is, however, impossible to reconcile the estimates of F_ST for all three modes of heredity under a simple island model of population subdivision: given the estimates of F_ST,A and F_ST,X, F_ST,Y is expected to be much higher (and R_XY to be much lower) than is observed (Figure 5). Deviations from the model assumed here (e.g., the occurrence of a selective sweep on the Y) might account for these discrepancies, if they are not simply due to sampling error due to the small number of informative sites on the Y.

APPENDIX

Variances in offspring numbers with a binomial distribution of the proportion of sons vs. daughters: Let the proportion of males among breeding individuals be c. From the properties of the binomial distribution, it is easily seen that the variance of the number of sons per female is Vfm=c2Vf+n‒fc(1−c), (A1a) where n¯_f and V_f are the mean and variance of total offspring number per female. If the population size is stationary, we have n¯_f = 1/(1 - c), and so Vfm=c2Vf+c. (A1b) The Poisson expectation for V_fm is c/(1 - c), so this yields ΔVfm=c2{(1−c)Vf−1}(1−c). (A1c) Similarly, we have ΔVff=(1−c){(1−c)Vf−1} (A2) ΔCffm=c(1−c)Vf−c. (A3) Corresponding expressions can be derived for the progeny of male parents, interchanging c and (1 - c).

Effective population size with self-fertilization: Consider first two maternally derived genes, sampled from different individuals (probability one-quarter). Their probability of origin from a common parent in the previous generation is independent of whether the individuals are the products of selfing or outcrossing and is simply Qff=(ΔVf∕d‒2)+1N, (A4a) where d¯ = d¯_S + d¯_O is the mean per capita number of successful offspring produced through seed, and ΔV_f is the excess over d¯ of the variance in the number of successful offspring produced through seed.

Next, consider a maternally derived and a paternally derived gene from different individuals (probability one-half). The latter has a probability of S of being derived by selfing, in which case the probability that it came from the same parent as the maternally derived gene is QfmS=∑kdkS(dkO+dkS−1)N2d‒Sd‒=(CfOfS+ΔVfS)∕(d‒Sd‒)+1N, (A4b) where C_{f Of S} is the covariance between the number of offspring produced through outcrossed and selfed seeds, respectively, and ΔV_{f S} is the excess over d¯_S of the variance in number of offspring produced through selfing.

If the paternal allele was derived by outcrossing (probability 1 - S), the probability that the pair came from a common parent is QfmO=(Cfm∕d‒p‒)+1N, (A4c) where C_fm is the covariance between the total number of offspring produced by seed and the number of offspring produced through outcrossed pollen, and p¯ is the mean number of offspring produced through outcrossed pollen.

Finally, consider a pair of paternally derived genes (probability one-quarter). There is a probability of approximately S ² that they are both products of selfing, in which case the probability of sharing a common parent is QfSfS=(ΔVfS∕d‒S2)+1N. (A4d)

There is a probability 2S(1 - S) that one is derived from selfing and the other from outcrossing, in which case the probability of common parentage is QfOfS=(CfSm∕d‒sp‒)+1N, (A4e) where C_{f Sm} is the covariance between the number of offspring produced through selfed seed and outcrossed pollen.

There is a probability (1 - S)² that both are products of outcrossing, in which case their probability of common parentage is QmOmO=(ΔVm∕p‒2)+1N, (A4f) where ΔV_m is the excess over p¯ of the variance in number of offspring produced through outcrossed pollen.

Conditioned on common parentage, each of these possible origins of gene pairs has a probability of (1 + F)/2 of resulting in coalescence; the reciprocal of the effective population size, N_eH, is thus given by multiplying (1 + F) by the sum of the products of the probabilities of origins and common parentage conditioned on origin, where at equilibrium under selfing and random mating F = S/(2 - S).

Assume that the kth individual has a total amount of resource R_k available for reproduction, of which a fraction c_k is devoted to pollen production and 1 - c_k to seed production. Let the net expected contribution of offspring through seed be a function f(R_k[1 - c_k]) of the individual’s allocation to seed production. If the population size is stationary, this has a mean of one.

If fitnesses are normalized so that mean fitnesses through male and female contributions are equal (Lloyd 1977), the expected contribution to the next generation through outcrossed pollen is p‒i=g(Rci)(1−S)g‒, (A5) where the function g describes the dependence of relative male reproductive success on allocation to pollen (the expectation of g is 1).

The actual numbers of offspring produced are Poisson variates with f¯ and p¯ as parameters. The excess of the variance over Poisson expectation in the number of offspring produced through seed is ΔVf≈(∂f∂R)2VR+(∂f∂c)2Vc (A6a) (assuming that R_k and c_k are independent), where the derivatives are evaluated at the population means. ΔVm≈(1−S)2g‒2{(∂g∂R)2VR+(∂g∂c)2Vc}. (A6b)

Similarly Cfm≈(1−S)g‒{(∂f∂R)(∂g∂R)VR+(∂f∂c)(∂g∂c)Vc} (A6c) and Cfsm≈S(1−S)g‒{(∂f∂R)(∂g∂R)VR+(∂f∂c)(∂g∂c)Vc}. (A6d)

Substituting these equations into Equation 19b, we obtain 1NeH≈(1+F)N{1+14[(1+S)∂f∂R+(1−S)∂g∂R]2VR+14[(1+S)∂f∂c+(1−S)g‒∂g∂c]2Vc}. (A7) The second term in brackets is equal to the square of the derivative of total fitness with respect to sex allocation (Charlesworth and Charlesworth 1981) and is therefore expected to be close to zero in populations that have had time to adjust their sexual allocation to its evolutionary optimum. The effect of non-Poisson variation in fitness on N_e is therefore likely to be largely determined by effects on R. It can be seen that the derivative of the female component of fitness with respect to R has a higher weight, the higher the selfing rate, whereas the derivative of outcrossed male fitness has a weight that declines to zero for complete selfing. If the two derivatives are of similar magnitude, there will be very little change in the contribution from V_R with changes in S.

More complex models, which allow for variation in the selfing rate and for covariances between R and c, can be written down, but the conclusions are not greatly changed. The covariance term can be seen to carry a weight that involves a factor equal to the derivative of total fitness with respect to sex allocation, which is expected to be close to zero (see above) from the above argument. The variance in selfing rate is necessarily close to zero for outcrossing and highly selfing populations and so will not influence their relative N_e values.