Effective Population Size and Population Subdivision in Demographically Structured Populations

Corresponding author: Valérie Laporte, Animal and Population Biology, University of Edinburgh, W. Mains Rd., Edinburgh EH9 3JT, UK., valerie.laporte{at}ed.ac.uk (E-mail)

Communicating editor: G. B. GOLDING

	ABSTRACT

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

A fast-timescale approximation is applied to the coalescent process in a single population, which is demographically structured by sex and/or age. This provides a general expression for the probability that a pair of alleles sampled from the population coalesce in the previous time interval. The effective population size is defined as the reciprocal of twice the product of generation time and the coalescence probability. Biologically explicit formulas for effective population size with discrete generations and separate sexes are derived for a variety of different modes of inheritance. The method is also applied to a nuclear gene in a population of partially self-fertilizing hermaphrodites. The effects of population subdivision on a demographically structured population are analyzed, using a matrix of net rates of movement of genes between different local populations. This involves weighting the migration probabilities of individuals of a given age/sex class by the contribution of this class to the leading left eigenvector of the matrix describing the movements of genes between age/sex classes. The effects of sex-specific migration and nonrandom distributions of offspring number on levels of genetic variability and among-population differentiation are described for different modes of inheritance in an island model. Data on DNA sequence variability in human and plant populations are discussed in the light of the results.

IN an ideal (Wright-Fisher) population of N breeding adult diploid individuals, the rate of genetic drift is equal to 1/(2N) (WRIGHT 1931 ). Natural populations, however, depart from many of the assumptions of the ideal population, notably panmixia and a binomial distribution of the number of successful offspring per parent. The effective population size, N_e, can be used to describe the rate of genetic drift in such populations and replaces the ideal population size N in the resulting formulas (WRIGHT 1931 ; WANG and CABALLERO 1999 ). For example, under the infinite-sites model of neutral variability, widely used in molecular population genetics, we can write = 4 N_eµ for the scaled mutation rate, where µ is the neutral mutation rate per nucleotide site (KIMURA 1971 ).

Population subdivision is a major factor that can influence N_e and has attracted much attention. WRIGHT 1943 derived an expression for the effective size of the whole set of populations in an island model, where each local population has an equal probability of contributing to the pool of migrants. He showed that, in this case, subdivision increases N_e for the whole population over that under panmixia. Since this pioneering work, additional factors (e.g., non-Poisson offspring number distributions, different modes of inheritance, sex-specific migration rates) have been included in the island model, and other migration models have been studied, such as stepping-stone models and metapopulations with local extinction and recolonization events (WHITLOCK and BARTON 1997 ; WANG and CABALLERO 1999 ).

The total effective size of a subdivided population can be expressed as a function of the mean within-deme effective population size and of F_ST (WRIGHT 1951 ), which describes the amount of between-deme differentiation relative to the variability in the set of populations as whole. In contrast to the results for the island model, population structure may reduce the total effective population size below that for panmixia when different demes make unequal contributions to the migrant pool (WHITLOCK and BARTON 1997 ). A difficulty is that effective population size is defined somewhat loosely as the size of an ideal population with the same properties as the nonideal study population with respect to the effects of genetic drift (WRIGHT 1931 ). Different effective population sizes can, however, be defined according to the variable of interest (WHITLOCK and BARTON 1997 ) and do not necessarily give exactly equivalent results.

In particular, the inbreeding effective size of a subdivided population determines the asymptotic rate of the increase of the probability of identity of a pair of alleles sampled randomly from the population at large. The variance effective size determines the variance of the change in allele frequency across a generation for genes drawn from the population as whole. With constant population sizes across generations, these two parameters converge to the same value (WHITLOCK and BARTON 1997 ). Similarly, the mutation effective size is defined as the population size that gives the average pairwise genetic diversity between alleles sampled randomly from the population as a whole, when substituted into the above equation for (WHITLOCK and BARTON 1997 ). The migration effective size for a subdivided population determines a weighted mean pairwise nucleotide site diversity for alleles sampled from the same population (NAGYLAKI 1998A ).

These measures of effective population size were originally derived from recursion equations for identity probabilities of pairs of alleles or variances in allele frequencies (reviewed by WANG and CABALLERO 1999 ). More recently, use has been made of the coalescent process, which is very useful for studying alleles sampled in a prescribed way from the set of local populations (HUDSON 1990 ; SLATKIN 1991 ; WILKINSON-HERBOTS 1998 ; WAKELEY 1999 , WAKELEY 2001 ). This approach has been applied mostly to models of autosomal loci, in which sex-specific migration, and departures from the assumptions of the ideal population for individual demes, have been ignored, although ROUSSET 1999 introduced a coalescence approach to the study of between-deme differentiation at autosomal loci with sex-specific migration rates.

There is, however, increasing interest in comparing the patterns of genetic variation and population differentiation for genes with different modes of inheritance (ENNOS 1994 ; HURLES and JOBLING 2001 ). This is particularly important in the context of studies of sex chromosome evolution, in which it is of interest to compare variability at homologues on the X and Y chromosomes (BACHTROG and CHARLESWORTH 2000 , BACHTROG and CHARLESWORTH 2002 ; FILATOV et al. 2000 , FILATOV et al. 2001 ). The expectation is that selective forces acting on a nonrecombining genome can greatly reduce the level of variability on an evolving Y or neo-Y chromosome (CHARLESWORTH and CHARLESWORTH 2000 ), but this effect may be confounded with the effects of demographic differences between males and females (CHARLESWORTH 2001 ) or of sex-specific migration in a subdivided population (FILATOV et al. 2001 ). These effects have primarily been investigated using the classical recursion equation approach, either for the effects of demography (CHARLESWORTH 2001 ) or for modeling sex-specific migration under various modes of inheritance (CHESSER et al. 1993 ; CHESSER and BAKER 1996 ; WANG 1997A , WANG 1997B , WANG 1999 ).

Here we first present a general framework for studying the pairwise coalescent times and identity probabilities of genes with different modes of inheritance in a metapopulation consisting of demes connected by an arbitrary migration scheme. Individuals within each deme are divided into different age and sex classes, with different demographic properties and migration probabilities. By using the "fast-timescale" approximation of NORDBORG 1997 for the flow of genes among different classes within demes, it is possible to reduce the complexities of this general model to a manageable size. This yields simple expressions for the effective sizes of single populations and the expected levels of genetic diversity between and within demes. We provide some explicit results for the case of an island model with equal deme sizes and review data on human and plant populations in their light.

	A GENERAL MODEL OF MIGRATION AND DRIFT IN STRUCTURED POPULATIONS

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

We assume that individuals within a local population (deme) can be classified into different classes, e.g., with respect to age and/or sex. Let f_ijrs be the probability of identity of a pair of genes sampled from an individual of class r in deme i and an individual of class s in deme j. We can write m_ijru for the probability that a gene sampled from an individual of class r in deme i originated from an individual of class u in deme j.

When the probability per time interval of mutation of an allele to a new state, µ, is independent of allelic state, class, and deme, f_ijrs is equivalent to the moment-generating function for the time to coalescence of a pair of genes of the specified type, with parameter -2µ (HUDSON 1990 ; WILKINSON-HERBOTS 1998 ). (To allow for the possibility of age structure, the time interval does not necessarily correspond to one generation.) In particular, their mean time to coalescence, T_ijrs, is equal to the derivative of f_ijrs with respect to -2µ at µ = 0. As usual, we assume that µ is sufficiently small that second-order terms can be neglected. We later consider the case of sex-specific mutation rates, using mean coalescence times derived in this way; this is legitimate, since coalescence times are independent of the mutation model.

Assuming that different individuals migrate independently of each other, and using the standard approach to modeling a geographically structured population (NAGYLAKI 1982 , NAGYLAKI 1998A ), we can write the equation for equilibrium,

(1)

where P^ijk_rsuv is the probability of coalescence over one time interval of two genes sampled from individuals of class r in deme i and s in deme j, derived from individuals from deme k who belong to classes u and v, respectively (note that unless u = v).

For i = j and r = s, and if migration involves individuals rather than gametes, we should strictly include a term on the right-hand side that covers the case when the two genes sampled come from the same migrant individual. The approximations leading to Equation 3 mean that this is unnecessary, since ignoring it introduces only a second-order term into the final result. (This is because the chance that a pair of genes from the same deme are sampled from the same individual is of the order of the reciprocal of the deme size, and we ignore products of this and the migration probabilities.)

Writing h_ijrs = 1 - f_ijrs for the probability of nonidentity of a pair of alleles, Equation 1 can conveniently be rewritten as

(2)

To make further progress, we assume that migration rates between demes are of the same order as the probabilities that two genes sampled from the same deme coalesce over one time unit and that both are sufficiently small that second-order terms in these, as well as mutation rates, can be neglected. This assumption is commonly made in applications of coalescent theory (HUDSON 1990 ; WILKINSON-HERBOTS 1998 ); if migration rates are much greater than these coalescent probabilities, little differentiation between demes is possible, so that this is not a particularly restrictive assumption.

To this order of approximation, the last term in braces on the right-hand side of (2) is nonzero only when i = j = k, corresponding to the case of two genes drawn from the same deme,

(3)

where _ij is the Kronecker delta (_ij = 1 when i = j and is otherwise 0); Pⁱ_rsu is the probability that two genes sampled from deme i from individuals belonging to classes r and s coalesce in an individual of class u in the previous time interval, treating the deme as a closed population.

To obtain manageable results, we simplify further by removing the dependence of h_ijrs on r and s. We can average h_ijrs values over classes within demes by using a weight of _ir for the contribution from class r in the ith deme. _ir is chosen to be equal to the rth element of the left leading eigenvector of the matrix that describes the flow of genes among classes within deme i, scaled such that the elements of this eigenvector sum to one. This element represents the stationary-state probability that a randomly sampled gene from deme i originates from class r (NORDBORG 1997 ). If the convergence to this stationary state is much faster than the rate of coalescence and migration, _ir gives an accurate approximation to the probability of origin of the sampled gene (NORDBORG 1997 ); this is the case, for example, for a discrete generation population, when the classes refer to different sexes. This also implies that the h_ijrs values can be treated as approximately independent of r and s; i.e., they can be equated to a common term, h_ij. This is equivalent to the fast-migration limit of NAGYLAKI 1980 for a geographically structured population.

We can then conveniently define the net probability of origin from deme k of a gene sampled from deme i, m_ik, as

(4)

Substituting into Equation 3 and summing _irjsh_ijrs over r and s, we obtain

(5)

where P_i is the probability of coalescence of a pair of genes sampled randomly from deme i, such that

(6)

Using the argument of CHARLESWORTH 2001 , we can write

(7)

where t_i is the generation time of individuals from deme i and N_ie is the deme's effective population size. In the case of discrete generations, t_i = 1 for all demes.

By differentiating h_ij with respect to -2µ, we obtain the following expression for the mean coalescent time for a pair of alleles sampled from demes i and j (HUDSON 1990 ):

(8)

This provides a general set of linear relations that can be used to solve for the T_ij, to the assumed order of approximation, for any specific model. Some examples are described below. Higher moments of the distribution of pairwise coalescence times can similarly be obtained by successive differentiation of Equation 5 (WILKINSON-HERBOTS 1998 ).

It is also useful to apply the method of NAGYLAKI 1998A for determining a weighted mean coalescent time for a pair of genes sampled from the same deme. The migration rates defined by the right-hand side of Equation 4 are identical in properties to the components of the standard genic migration matrix. Under the usual assumptions about this matrix (NAGYLAKI 1980 , NAGYLAKI 1982 , NAGYLAKI 1998A ), it has a leading left eigenvector, , whose elements can be assumed to sum to one without loss of generality. Using the argument of NAGYLAKI 1998A , this suggests the definition of a weighted mean coalescence time, T₀, for a pair of alleles sampled from the same deme as

(9)

where the coalescent time for each deme is weighted by the product of the reciprocal of the coalescence probability and the square of the corresponding component of . In the discrete generation case, T₀ corresponds to twice the "migration effective size" as defined by NAGYLAKI 1980 , NAGYLAKI 1982 , NAGYLAKI 1998A and reduces to the standard expression for this quantity in the case of Wright-Fisher sampling within demes of size N_i, for which P_i = 1/(2N_i).

	COALESCENT PROBABILITIES AND EFFECTIVE POPULATION SIZES

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

Separate sexes with discrete generations:
The utility of the approach described above to determining the effective population size of a deme can be illustrated by its application to the discrete-generation case with separate sexes, previously treated by CABALLERO 1995 . It enables us to provide biologically explicit expressions for N_e under different modes of inheritance, on the lines developed by CHARLESWORTH 2001 for the age-structured case.

From the assumption that the products of coalescence probabilities and migration probabilities are negligible, used to derive (3) and its successors, it is legitimate to treat each deme as a closed population. We have to consider the origins of genes sampled from individuals of all permissible combinations of sexes and parents (which combination is permissible depends on the mode of genetic transmission). For a pair of maternally derived genes sampled from two different females, the probability that they both come from the same mother is approximately

(10)

(CABALLERO 1995 ), where N_if is the number of breeding females in deme i, d_ik is the number of daughters produced by the kth female in deme i, and _i is the mean number of daughters per female in deme i.

This can be simplified by noting that the Poisson value for the variance in numbers of daughters per female, Vⁱ_ff, is _i, so that in general the deviation of this variance from the Poisson value can be written as

(11)

so that

(12)

Similar expressions can be obtained for other pairs of individuals and their parents (see Table 1).

Let ß_irsu be the probability that a pair of genes sampled from individuals of sexes r and s in deme i both come from parents of sex u. Let _irsu be the probability that this pair of genes shared a common ancestral allele in the previous generation, given that they have a parent in common; i.e., they coalesce. From Equation 7, the net probability that a randomly chosen pair of genes coalesce in the previous generation is

(13)

where Qⁱ_rsu is the probability that a pair of genes sampled from individuals of sexes r and s in deme i with a parent of sex u derives from the same parental individual.

Table 2 gives the values of these transmission probabilities for different modes of inheritance; these can be combined with the Q values from Table 1 and substituted into Equation 15, to obtain the final expressions for effective population sizes. Simplified expressions can be obtained when there is a fixed sex ratio among breeding individuals, so that there is a binomial distribution of the proportions of sons and daughters of a given individual, counted at maturity (see the Appendix).

For convenience, we henceforth drop the superscripts and subscripts indicating deme identities. Using Table 1 and Table 2, substituting into Equation 15, and noting that the Poisson expectation for variance in total offspring number is 1/(1 - c) for females and 1/c for males, where c is the proportion of males among breeding adults, we obtain the expression for effective population size with autosomal inheritance,

(14)

where V_f and V_m are the excesses over Poisson expectation of the total numbers of offspring per capita for female and male parents, respectively; F is the inbreeding coefficient associated with departure from random mating within demes.

For X-linked inheritance with male heterogamety, we have

(15)

For Y-linked inheritance

(16)

With female heterogamety, males and females are interchanged in these expressions.

For maternal inheritance

(17)

Partially self-fertilizing hermaphrodites:
It is also of interest to consider the case of a nuclear gene in a population of partially self-fertilizing hermaphrodites; this is particularly relevant for plants. (Maternally transmitted genes are clearly unaffected by the selfing rate, except as far as changes in the breeding system alter the distribution of numbers of successful offspring produced through seed.) With a Poisson distribution of offspring number for both seed and pollen, the effective size is reduced below that for random mating by a factor of 1/(1 + F), where F is the equilibrium inbreeding coefficient produced by the level of self-fertilization (POLLAK 1987 ; NORDBORG and DONNELLY 1997 ). We examine the effect on this result of deviations from a Poisson offspring number distribution.

Let there be N breeding individuals in the deme under consideration; the kth individual is assumed to produce d_kS progeny through self-fertilized seed that survive to form part of the breeding population next generation, and d_kO such progeny through outcrossed seed. In addition, it contributes p_k progeny through pollen that fertilizes other individuals. The overall probability that a progeny individual is the product of self-fertilization is then

(18)

where _S and _O are the mean numbers of successful progeny per capita produced by selfed and outcrossed seed, respectively.

Details of the derivation of probabilities of gene origins are given in the Appendix In the case of constant population size, we have , and . The expression for the effective population size for a single deme, N_eH, is

(19a)

(definitions of the terms in the numerator are given in the Appendix).

If the number of successful offspring through selfed seed, conditioned on the total number of successful offspring produced through seed, can be regarded as a binomial variate with parameter S (i.e., there is the same expected selfing rate for each individual), this simplifies further to

(19b)

In the limit of complete self-fertilization, the contributions from outcrossing pollen vanish, and F = 1, so that

(20)

A model is presented in the Appendix for analyzing the case of intermediate selfing rates. The overall conclusion is that non-Poisson variation in reproductive capacity may not greatly alter the standard result that N_e for a selfing population is reduced by a factor of 1/(1 + F), relative to the value for an outcrossing population (POLLAK 1987 ), although a very high sensitivity of female fitness to allocation to reproduction could lead to a greater reduction in N_e than this.

	THE ISLAND MODEL WITH SEX-SPECIFIC MIGRATION PARAMETERS

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

In this section, we apply the above principles to the simple case of an island model (WRIGHT 1943 ) with sex-specific migration rates, for dioecious populations under a variety of modes of inheritance. We assume that the total population is divided into n demes, each with the same effective population size, N_e, given by the equations derived above. There are three dispersal parameters, describing the probability that a male gamete (m_m) in a zygote in a given deme has migrated from a different deme, the corresponding probability for a female gamete (m_f), and the probability that a zygote is itself derived from a different deme (m_z; CHESSER and BAKER 1996 ; WANG 1999 ). Under the assumptions used above, these migration events can be treated as mutually exclusive, since their probabilities are each low. Migration probabilities between specific demes, as considered above, are given by dividing the dispersal parameters by (n - 1).

In plant species, male gametes can disperse through pollen but female gametes cannot disperse, so that only m_m and m_z can be nonzero. For animals, migration of male and female gametes occurs via dispersal of males and unmated females; dispersal of zygotes occurs when females migrate after mating but before laying eggs or giving birth. All three migration parameters can therefore be nonzero.

Using the argument leading to Equation 4, we can then define the net migration rates for different modes of inheritance (autosomal, X-linked, Y-linked, and cytoplasmic) as m_A, m_X, m_Y, and m_C, respectively. We have

(21a)

(21b)

(21c)

(21d)

Coalescence times and expected nucleotide site diversities:
Results for mean pairwise coalescence times in the island model can be obtained by substituting these expressions into standard formulas (SLATKIN 1991 ; WILKINSON-HERBOTS 1998 ), which correspond to approximate solutions to Equation 8. To the assumed order of approximation, with mode of inheritance G (where G is A, X, Y, or C) the expected coalescence time, T_0G, for two genes sampled from the same deme is given by

(22a)

where N_eG is the appropriate effective population size for a single deme.

Using Equation 8, or the argument of SLATKIN 1991 , the approximate expected coalescence time for a pair of genes sampled from different demes is

(22b)

where m_G is the migration rate defined by the relevant choice of Equation 21aEquation 21bEquation 21cEquation 21d for mode of inheritance G.

The expected coalescence time for a pair of genes sampled randomly from the set of populations is

(22c)

The expected pairwise nucleotide site diversity under the infinite-sites model is given by the product of coalescence time and the mutation rate µ_G for the given mode of inheritance (HUDSON 1990 ). From Equation 22aEquation 22bEquation 22c, the expectations of nucleotide diversities for alleles sampled within demes, _SG, and from the population as a whole, _TG, are given, respectively, by

(23a)

(23b)

Mutation rates may differ among genetic systems because of differences in mutation rates between males and females (HURST and ELLEGREN 1998 ; MCVEAN 2000 ) or because of chromosome-specific mutation rates (MCVEAN and HURST 1997 ). In the former case, the overall mutation rate is given by weighting the sex-specific or chromosome-specific mutation rates by the appropriate values (CHARLESWORTH 2001 ).

The absolute magnitude of between-population subdivision can be measured by the difference between the nucleotide diversity for the population as a whole and the mean for a pair of alleles sampled from the same deme (CHARLESWORTH 1998 ). The expectation of this is given by the second term in brackets on the right of (23b). It is often more convenient to use a relative measure, such as F_ST. Using the definition of HUDSON et al. 1992 , the theoretical value of F_ST for a given mode of inheritance is given by

(24a)

With a large number of demes, this yields the familiar result of WRIGHT 1951 :

(24b)

We now consider the application of these formulas to some specific examples. We focus initially on the effects of population subdivision, assuming a 1:1 sex ratio, Poisson variances in fertility for both sexes, and equal mutation rates for males, females, and different chromosomes. Discrete generations and a deme size of N breeding adults for all demes are also assumed. In this case, the ratios of effective population sizes and within-deme diversities for different modes of inheritance G and G' (r_GG') are simply equal to the ratios of the respective numbers of gene copies in the population: r_XA = ³/₄, r_XY = r_XC = 3, r_AY = r_AC = 4, the same values as for panmixia, which are often quoted in the literature on molecular population genetics.

The effects of sex-specific migration and population subdivision in dioecious plants:
In Fig 1, we plot the values of F_ST for nuclear and cytoplasmic genes and the ratios of the total diversities for different modes of inheritance, as functions of the total number of migrants per generation (give by the sum of pollen and seed migration rates times the deme size). Male heterogamety is assumed.

View larger version (27K): [in this window] [in a new window] [Download PPT slide]   Figure 1. The FST parameters (A–C) and the ratios (R) of total diversities between genes with different modes of inheritance (D–F) in a subdivided plant population (mf = 0), as functions of the number of migrants per deme, N(mm + mz). (A and D) Equal pollen and seed migration. (B and E) Predominantly pollen migration (mm = 100mz). (C and F) Predominantly seed migration (mz = 100mm). We assume NeA = N, NeX = 3/4.N, NeY = Nec = 1/4.N.

As expected, reduced migration increases F_ST for all modes of inheritance. However, the rate of increase of F_ST as the amount of migration decreases is different for different modes of inheritance and also depends on the mode of migration. For instance, with equal pollen and seed migration rates, F_ST,Y and F_ST,C increase faster than F_{ST, A} and F_{ST, X} (Fig 1A). These differences result from differences in the numbers of effective migrants, due to two factors: (i) different effective population sizes and (ii) different gene migration rates. The effective number of X chromosomal migrants is lower than the effective number of autosomal migrants, due to a lower effective population size of the X chromosomes (3N/4) as well as its lower migration rate (m_z + ¹/₃m_m vs. m_z + ¹/₂m_m). The effective number of Y chromosomal migrants is lower than both the effective number of chromosomal X or autosomal migrants, due to the lower effective population size of the Y chromosome (N/4), which is not compensated for by its higher migration rate (m_z + m_m) relative to the X chromosome. A similar argument applies to cytoplasmic genes.

Population subdivision also modifies the ratios of the total diversities for different modes of inheritance: the ratios _TX/_TY(R_XY) and _TA/_TY(R_AY) decrease sharply with reduced gene flow and _TX/_TA(R_XA) increases slightly (Fig 1D). From Equation 23aEquation 23b, it is easily seen that, in the limit when the migration rates tend toward zero, the R values are equal to the reciprocals of the ratios of the respective migration rates given by Equation 21a HREF="#FD21b">Equation 21bEquation 21cEquation 21d, since the right-hand side is dominated by the term involving the reciprocal of the product of effective size and migration rate. In this case, the limiting total diversities for autosomal and X-linked genes are only 1.33- and 1.5-fold higher than for Y-linked loci, respectively, with the diversity for X-linked loci being 1.12 that for autosomal loci. Similarly, the limiting ratio of autosomal to cytoplasmic total diversities becomes 0.67.

Differences between the pollen and seed migration rates also influence the rate of increase in F_ST. For instance, if migration occurs primarily through pollen (e.g., with m_m = 100m_z, Fig 1B), F_{ST, A} and F_{ST, X} increase faster with decreasing migration than when migration occurs equally through pollen and seeds (Fig 1A), or primarily through seeds (m_m = 0.01m_z, Fig 1C), whereas F_ST,C increases more slowly, but is higher throughout the range of Nm displayed. This is because X-linked and autosomal genes are haploid in pollen grains but diploid in seeds, and cytoplasmic genes move only through seed. In contrast, the Y chromosome, which is haploid in both dispersal units, is not affected by the relative pollen and seed migration rates. In consequence, the curves relating the diversity ratios to total migration rate are also modified. In particular, when migration occurs primarily through pollen, the increased migration rate of the Y chromosome compared to the X chromosome (m_m vs. m_z/3) compensates nearly totally for its reduced effective population size, so that R_XY is independent of population subdivision (Fig 1E). In contrast, R_XA is now slightly more affected by population subdivision, because the difference between the X and autosomal migration rates increases with the pollen migration rate. Hence, in the limit when the migration rates tend toward zero, R_XY barely declines below 3, but R_AY declines to 2, and R_XA increases to 1.5.

The effects of sex-specific migration and population subdivision in animals:
We consider a model of animal migration with up to a sixfold difference between male and female migration rates and no zygotic migration (see above), assuming male heterogamety (for female heterogamety, male and female parameters are interchanged). As compared with the plant case, we can see that the case of predominantly male migration (Fig 2B and Fig E) is similar to the case of predominantly pollen migration in the plant model. The case of equal male and female migration rates is similar to the case of equal pollen and seed migration rates, except that limited migration has a much stronger effect on F_STY relative to F_STA and F_STX (Fig 2A). The effect is even stronger with predominantly female migration where F_STY increases much faster with reduced migration than F_STA or F_STX (Fig 2C). Consequently R_AY and R_XY show a greater reduction when migration is restricted (Fig 2D and Fig F) than in the plant model (Fig 1D and Fig F). These differences are easily understood since Y-linked genes now experience migration through only 1 unit (male individuals), while they had two in the plant model (pollen and seeds).

View larger version (27K): [in this window] [in a new window] [Download PPT slide]   Figure 2. The FST parameters (A–C) and the R values (D–F) in a subdivided animal population (mz = 0), as functions of the number of migrants per deme, N(mm + mf). (A and D) Equal male and female migration. (B and E) Predominantly male migration (mm = 6mf). (C and F) Predominantly female migration (mf = 6mm). We assume NeA = N, NeX = 3/4.N, NeY = Nec = 1/4.N.

The effects of sex- or chromosome-specific mutation rates:
Under the infinite-sites model, the mutation rate is so low compared with any realistic migration rate that differences in mutation rates among different modes of inheritance will not affect the relative F_ST values. Absolute values of diversities will, however, be affected, since the ratios for different modes of inheritance will be multiplied by the ratios of the respective mutation rates. From Equation 23aEquation 23b, this does not affect the shapes of the R values as functions of migration rates, but does affect their heights. For example, if the male mutation rate is higher than the female mutation rate (HURST and ELLEGREN 1998 ; MCVEAN 2000 ), this will have the effect of reducing R_XA, R_XY, and R_AY. This would enhance the effect of high relative female migration rates in the animal model on these ratios, which has just been noted.

The effects of nonrandom variation in fertility:
It is well known that an increase in the variance of fertility reduces N_e. Sex differences in the distributions of fertility modify the relative N_e values of genes with different modes of inheritance (CABALLERO 1995 ; CHARLESWORTH 2001 ). For example, an increased variance in male fertility reduces the effective deme size of the Y chromosome more drastically than that for the X or the autosomes. With an extremely high variance in male fertility relative to female fertility, the ratios N_eX/N_eY and N_eA/N_eY become 9 and 8, respectively, so that N_eX/N_eA is 9/8. In contrast, a high variance in female fertility reduces N_e for the X and the autosomes but does not affect the Y chromosome: the ratios N_eX/N_eY and N_eA/N_eY tend to zero and N_eX/N_eA to 9/16.

As far as total diversity measures are concerned, population subdivision always counteracts the effect of an increased variance in male fertility by reducing the ratios R_XY and R_AY, as illustrated in the animal model with large V_m (Fig 3). Since N_eY is reduced by an increased male fertility variance, the effective number of Y migrants is much lower than the number of X or autosomal migrants. As the migration rate decreases, F_ST,Y now increases much faster than F_ST,X or F_ST,A, and the ratios R_XY and R_AY decrease toward one with equal male and female migration (Fig 3B and Fig E). With predominantly female migration, this effect is manifest even with relatively high migration (Fig 3F). For instance, R_XY = 6.2 and R_AY = 5.7 with very high migration, but are already halved with as many as 10 effective migrants. With an increased variance in female fertility, the effect of population subdivision depends on both the relative effective population sizes and the relative migration rates for different modes of inheritance. Population subdivision increases R_XY (R_AY) if N_eX·m_X < N_eY·m_Y (N_eA·m_A < N_eY·m_Y). However, this effect occurs only with very restricted migration, as illustrated in the plant model with V_f = 5 (Fig 4). Moreover, the maximum value of R_XY (R_AY) is always lower than the expectation for a panmictic population with Poisson distributions of offspring numbers.

View larger version (27K): [in this window] [in a new window] [Download PPT slide]   Figure 3. Same as Fig 2 but assuming an increased variance of male fertility compared to the Poisson expectation (Vm = 10).

View larger version (27K): [in this window] [in a new window] [Download PPT slide]   Figure 4. Same as Fig 1 but assuming an increased variance of female fertility compared to the Poisson expectation (Vf = 5).

Selection on a nonrecombining genome, such as organelle genomes and the Y chromosome, can further reduce N_e (CHARLESWORTH and CHARLESWORTH 2000 ). Such a reduction of N_ec and N_ey will increase the corresponding F_ST values. We have investigated the case with N_eX/N_eY = 30 for the plant model (Fig 5), since this is the ratio suggested by observed within-deme diversity ratios in Silene latifolia (see below). With such a low N_eY, the decrease in R_XY and R_AY due to population subdivision now occurs even with relatively high migration rates. Moreover, with such a low N_eY, differences between pollen and seed dispersal rates have little effect on the relative total diversities of nuclear genes with different modes of inheritance.

View larger version (26K): [in this window] [in a new window] [Download PPT slide]   Figure 5. Same as Fig 1 but assuming NeY = NeX/30. The figures also give the FST and RXY estimates from Silene latifolia populations with the number of migrants expected under an island model (Triangle, ST,Y; square, ST,A; circle, ST,X; and cross, XY).

	COMPARISONS WITH DATA ON NATURAL POPULATIONS

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

In this section, we compare some of the results derived above with data from surveys of DNA sequence variation in populations of humans and plants.

Human populations:
There is a large literature on genetic diversity in humans, and many different types of markers have been employed (including protein polymorphisms, restriction fragment length polymorphisms, microsatellites, Alu insertions, and single-nucleotide polymorphisms). These data have, however, several biases, which limit their utility for our purposes. First, worldwide population structure has rarely been investigated using markers with different modes of inheritance in the same samples (but see POLONI et al. 1997 ; JORDE et al. 2000 ). In addition, the mode of sampling from different populations, or of grouping the populations, may influence F_ST estimates, as described by STONEKING 1998 and HAMMER et al. 2001 (see Table 3). Moreover, different estimators of F_ST, which depend in different ways on the sample sizes and numbers of populations sampled (COCKERHAM and WEIR 1993 ; CHARLESWORTH 1998 ), have been used in different studies, making it hard to compare different results. This is illustrated by the different estimates of F_ST,Y obtained from the same dataset by SEIELSTAD et al. 1998 and by HAMMER et al. 2001 . Finally, because of the low level of sequence diversity in humans, investigators have favored the use of highly polymorphic markers, especially for the Y chromosome. This may bias F_ST estimates, as noted by NAGYLAKI 1998B and BERTRANPETIT 2000 , since diversity values for loci with high mutation rates are not proportional to coalescent times.

In Table 3, we present a compilation of F_ST estimates from worldwide samples of different types of markers, with cytoplasmic, Y, X, and autosomal inheritance. These all provide evidence for population structure, with autosomal markers yielding the lowest mean F_ST estimates, as expected from the results shown in Fig 2. The estimates obtained from Y chromosome markers are, however, highly variable between studies (0.09–0.65). Microsatellites often show smaller F_ST values (0.09–0.23), as outlined in studies that have compared both kinds of markers (HAMMER et al. 1997 ; JORDE et al. 2000 ), probably because of their high mutation rates. Excluding microsatellites, the average F_ST estimated from the Y chromosome markers (0.40) is higher than from the mitochondrial markers (0.27). While this has often been interpreted as evidence for higher female than male migration rates, it might also reflect differences in effective population sizes (HURLES and JOBLING 2001 ). A recent comparison of local population differentiation in Thai communities with patrilocal or matrilocal dispersal patterns is, however, in agreement with the expectation that restricted male migration increases population differentiation for Y markers relative to mitochondrial markers (OOTA et al. 2001 ).

DNA sequence data have recently been obtained for 4 Y-linked and 15 autosomal loci, using similar worldwide samples (SHEN et al. 2000 ; THOMSON et al. 2000 ; THORSTENSON et al. 2001 ). The authors estimated R_AY as five, from data on both coding and noncoding regions. Unfortunately, population structure was not investigated explicitly in these studies, although the data should allow partitioning of diversity between and within populations. THOMSON et al. 2000 constructed a genealogy of their sample of Y chromosomes, and this can be used to obtain a value of 0.14 for F_ST,Y between continents. This is smaller than that obtained from other studies of biallelic markers and challenges the notion that there is restricted male vs. female migration in most human populations. Mitochondrial data for the same samples would help to resolve this question, and it would also be desirable to obtain estimates of F_ST,A and r_AY.

At present, it is hard to reach firm conclusions about the influence of sex-specific migration vs. differences in effective population sizes on the relative levels of diversity and divergence between human populations for different inheritance modes; there does not, however, seem to be strong evidence for a greatly reduced effective size of the human Y chromosome, in contrast to what is observed in Silene (see below) or in Drosophila (ZUROVCOVA and EANES 1999 ; BACHTROG and CHARLESWORTH 2000 , BACHTROG and CHARLESWORTH 2002 ).

Plant populations:
There are relatively few species of plants with sex chromosomes, and diversity data on nuclear genes with different modes of inheritance are available only for the close relatives S. latifolia and S. dioica (Table 4). In S. dioica, there is only a single polymorphic site on the Y, which is too little to provide any useful information. In S. latifolia, nine polymorphic sites were found on the Y. All nuclear genes display quite strong population structure, with the lowest F_ST for the autosomal gene and the highest for a Y-linked gene, as expected from the above analyses. The ratio R_XY over all demes (R_XY = 23) is smaller than the ratio r_XY for within-deme diversity (r_XY = 29), as expected from the effects of subdivision (Fig 5).

It is, however, impossible to reconcile the estimates of F_ST for all three modes of heredity under a simple island model of population subdivision: given the estimates of F_ST,A and F_ST,X, F_ST,Y is expected to be much higher (and R_XY to be much lower) than is observed (Fig 5). Deviations from the model assumed here (e.g., the occurrence of a selective sweep on the Y) might account for these discrepancies, if they are not simply due to sampling error due to the small number of informative sites on the Y.

	DISCUSSION

TOP ABSTRACT A GENERAL MODEL OF... COALESCENT PROBABILITIES AND... THE ISLAND MODEL WITH... COMPARISONS WITH DATA ON... DISCUSSION APPENDIX LITERATURE CITED

In the first part of this article, we have shown that the use of the fast-timescale approximation (NORDBORG 1997 ) to the coalescent process in a single population, structured according to sex and/or age, provides a simple general expression for the probability that a pair of alleles sampled from the population coalesce in the previous time interval (Equation 6). This approximation assumes that the flow of genes among age and sex classes is much faster than the effects of drift or migration between populations, so that the state of an allele can be treated as independent of the class from which it was sampled: this is equivalent to the "strong-migration limit" of NAGYLAKI 1980 for a geographically structured population.

By use of a suitable definition of generation time, we can also define the effective size of a population as the reciprocal of twice the product of generation time and the probability of coalescence per time interval (Equation 7). This expression can be used to generate explicit formulas for effective population size with discrete generations and separate sexes, under a variety of different modes of inheritance, and with arbitrary distributions of offspring numbers (Equation 14Equation 15Equation 16Equation 17). The same approach can be used for the case of a nuclear gene in a population of partially self-fertilizing hermaphrodites (Equation 19aEquation 19b and Equation 20). An important conclusion in this case is that the standard formula for N_e for selfing populations (POLLAK 1987 ) is likely to be approximately valid even if there is a non-Poisson distribution of offspring number; i.e., N_e is reduced below the value for random mating by a factor of 1/(1 + F), where F is the inbreeding coefficient.

The approach can also be applied to the standard model of an age-structured population with discrete time intervals, recently revisited by CHARLESWORTH 2001 ; his Equation A7 is equivalent to our Equation 6 and Equation 7. Finally, we note that the effective population size defined in this way is equivalent to that which describes the asymptotic rate of increase in the probability of identity of a pair of randomly sampled genes in the absence of mutation, i.e., to the inbreeding effective population size. This equivalence is dependent on the validity of the fast-timescale approximation (NAGYLAKI 1980 ; CHARLESWORTH 2001 ).

We also show how to simplify the analysis of the effects of population subdivision of a demographically structured population by defining a migration matrix that describes the net rates of movement of genes between different local populations. This involves weighting the migration probabilities of individuals of a given age-sex class by the contribution of this class to the leading left eigenvector of the matrix describing movements of genes between age and sex classes (Equation 4). This enables the determination of the moment-generating functions for the distributions of coalescent times for pairs of genes sampled from a given pair of populations, under any well-defined migration model (Equation 5), under the standard assumption that migration and drift are both weak evolutionary forces. From these, the expected coalescent times and higher moments can easily be found (HUDSON 1990 ; WILKINSON-HERBOTS 1998 ).

Under the infinite-sites model, the expected number of nucleotide differences between a pair of alleles sampled from a prescribed pair of populations is equal to the product of the mutation rate and the corresponding expected coalescent time (HUDSON 1990 ), so that the results can be used to generate expressions for equilibrium levels of nucleotide site diversities. In doing this, the mutation rates for different age-sex classes must be weighted in the same way as the migration rates. This approach can be extended to extinction-recolonization metapopulation models, if it is assumed that the extinction of a local population is immediately followed by its recolonization (WAKELEY and ALIACAR 2001 ).

An important general conclusion is that population subdvision makes it very hard to describe the expected level of genetic variability in a population by a single formula such as = 4N_eu. While it is possible to define a simple expression for a weighted mean coalescence time for a pair of alleles sampled from the same deme for a general migration model (Equation 9), this involves both the effective population sizes of all the individual demes and their contributions to the leading left eigenvector of the migration matrix defined by Equation 4. In general, these are unknowable quantities, making it very hard to equate any empirical estimate of the mean within-population nucleotide site diversity to a simple scaled mutation rate parameter. The details of the demography and migration parameters of a species may greatly influence the estimated scaled mutation rate based on unweighted mean within-population nucleotide site diversities, making comparisons between different species difficult to interpret.

The situation is even worse for the nucleotide diversity for a pair of alleles sampled from the population at large, since this is related to the within-deme value by 1/(1 - F_ST). In general, F_ST depends in a complex way on migration rates and deme sizes, and simple formulas are available for only a few limiting cases, such as the island model. Only when there is negligible genetic differentiation between local populations can one confidently relate mean nucleotide site diversity to the coalescent time for a randomly sampled pair of alleles and hence to a scaled mutation rate parameter. Many empirical investigations of DNA sequence variation in natural populations do not state explicitly what population parameters are of primary interest and often make little distinction between measures of variation based on whole-population and within-population estimates. More attention to these issues in presenting analyses of data on DNA sequence variability is desirable.

Our investigation of the island model also shows that strong population subdivision, coupled with sex-specific migration rates, can greatly affect the relative values of the expected total genetic diversities for different modes of inheritance and may even reverse some of the patterns expected under panmixia. For example, Fig 3 displays the expected patterns of genetic variability for animal populations with a high variance in male reproductive success. From Fig 3C, it may be seen that predominantly female migration can cause the ratios of autosomal or X-linked variability to Y-linked variability to decline below one with extreme subdivision, in contrast to a value of over four with panmixia. Encouragingly, however, the relative values of expected autosomal and X-linked total population diversities are insensitive to all but the most extreme population subdivision, for both plant and animal models (Fig 1 Fig 2 Fig 3 Fig 4). This suggests that the use of ratios of X-linked to autosomal diversities to make inferences about the strength of sexual selection (CHARLESWORTH 2001 ) may be feasible even if there is a high level of population subdivision.

We have confined ourselves to deriving expressions for expected coalescence times and nucleotide site differences between alleles. However, expressions for the distribution of coalescent times for a set of n alleles sampled from a specified set of populations can easily be written down, using the migration probabilities defined by Equation 4 and coalescence probabilities defined by Equation 7 to generate the expectations of competing exponential distributions of waiting times to migration or coalescence events (HUDSON 1990 ). Although in general it is difficult to obtain useful analytical results on the properties of these distributions (but see WAKELEY 1999 , WAKELEY 2001 ), they can readily be implemented in computer simulations and used to derive sampling properties of estimators of the type that we have discussed.

	ACKNOWLEDGMENTS

We thank N. Barton, D. Charlesworth, F. Depaulis, and two anonymous reviewers for their comments on the manuscript. B.C. acknowledges support by the Royal Society and the Engineering and Physics Research Council, and V.L. acknowledges support by the Biotechnology and Biological Sciences Research Council.

Manuscript received April 2, 2002; Accepted for publication June 18, 2002.

	APPENDIX