Abstract
Equations are derived for the effective sizes of gynodioecious populations with respect to both nuclear and cytoplasmic genes (N_{ec} and N_{en}, respectively). Compared to hermaphroditism, gynodioecy generally reduces effective population sizes for both kinds of loci to an extent depending on the frequency of females, the sex determination system, and the selfing rate of hermaphrodites. This reduction is due to fitness differences between the sexes and is highly influenced by the mode of inheritance of this fitness. In absence of selfing, nuclear gynodioecy results in a reduction of N_{ec} that depends strongly on the dominance of male sterility alleles, while N_{en} remains equal to the census number (N). With cytonuclear gynodioecy, both cytoplasmic and nuclear effective sizes are reduced, and at the extreme, dioecy results in the minimum N_{ec} values and either minimum or maximum N_{en} values (for low or high frequency of females, respectively). When selfing occurs, gynodioecy either increases or decreases N_{en} as compared to hermaphroditism with the same selfing rate of hermaphrodites. Unexpectedly, N_{ec} also varies with the selfing rate. Thus the genetic sexdetermination system appears as a major factor for the nuclear and cytoplasmic genetic diversities of gynodioecious species.
THE effective population size (N_{e}) is a key parameter in population genetics because it predicts the degree of inbreeding in a population and the rate of fixation of selectively advantageous and deleterious genes, as well as the amount of genetic differentiation between populations (Wright 1931; Whitlock and Barton 1997). As a predictor of the rate of loss of genetic variability in isolated populations, N_{e} is also a parameter of paramount importance in conservation biology (Ellstrand and Elam 1993). Considerable theoretical work has been developed to determine the effective population size for nuclear, biparentally inherited genes (N_{en}), especially in the cases of hermaphroditic or dioecious reproductive systems (reviewed in Caballero 1994). Compared to an ideal population, two main factors reduce N_{en}. One is the occurrence of nonrandom mating (reviewed in Caballero and Hill 1992; Wang 1995). The other is the variance in reproductive success among individuals due to unequal sex ratio (Wright 1931) or variance in any fitness component (reviewed in Crow and Denniston 1988; Caballero 1994). In addition, heritability of such fitness components further reduces the effective size (Robertson 1961; Santiago and Caballero 1995, 1998). Theoretical studies have also been developed for uniparentally inherited genes (Takahata and Maruyama 1981; Birky et al. 1983, 1989; Chesser and Baker 1996) and show the conflicting effects of reproductive systems on different sets of genes. With separate sexes, for example, the greater the excess of females, the lower the effective size for autosomal genes but the higher the effective size for maternally inherited cytoplasmic genes.
Effective sizes for intermediate reproductive systems like gynodioecy remain to be investigated. Gynodioecy is the cooccurrence of hermaphrodite and female individuals within natural populations (Darwin 1877). This reproductive system is known in >7.5% of the angiosperm species in Europe (Delannay 1978). It has received considerable attention as an example of an outcome of genomic conflict (Couvetet al. 1990; SaumitouLapradeet al. 1993a; Hurstet al. 1996; Partridge and Hurst 1998; Werren and Beukeboom 1998), because the sexual phenotype generally results from an interaction between nuclear and maternally inherited cytoplasmic genes (e.g., Charlesworth 1981; Van Damme 1983; Boutinet al. 1987; Belhassenet al. 1991; Koelewijn and Van Damme 1995). Many theoretical studies have been done on the maintenance of this genetic polymorphism (e.g., Lewis 1941; Charlesworth and Ganders 1979; Charlesworth 1981; Delannayet al. 1981; Frank 1989; Gouyonet al. 1991; McCauley and Taylor 1997; Couvetet al. 1998). Recently, a growing number of experimental studies have also examined the genetic diversity in gynodioecious species (e.g., Belhassenet al. 1993; SaumitouLapradeet al. 1993b; Manicacciet al. 1996; Tarayreet al. 1997; De Haanet al. 1997b; Forcioliet al. 1998; McCauley 1998). A theoretical understanding of the effective population size is thus required to interpret these experimental data.
Here, equations are derived for cytoplasmic and nuclear effective sizes (N_{ec} and N_{en}) of gynodioecious populations. The model considers a single large population that is at equilibrium for the frequency of the sexual phenotypes, because several theoretical studies have shown that gynodioecy can be maintained within a single population (e.g., Lewis 1941; Lloyd 1974; Charlesworth 1981; Delannayet al. 1981; Gouyonet al. 1991). The transition matrix of the increase in identity by descent is expressed as a function of parameters describing a gynodioecious system. The eigenvalue effective population size, equal to the leading eigenvalue of this matrix (Ewens 1979, 1982; Whitlock and Barton 1997) is derived. This effective population size, equal to the longterm inbreeding or variance effective population size (see Whitlock and Barton 1997), is also called the “asymptotic effective size” (e.g., Chesseret al. 1993; Wang 1997). N_{ec} and N_{en} are analyzed as functions of the frequency of females under random mating or partial selfing of hermaphrodites.
PARAMETERS DESCRIBING GYNODIOECY WHEN HERMAPHRODITES ARE RANDOM MATING
The model deals with a single large gynodioecious population where the frequency of the sexual phenotypes is at equilibrium. The importance of the genetic sexdetermination system and fitness effects of the genes determining the sexual phenotypes have been outlined for maintaining gynodioecy within a single population (e.g., Lewis 1941; Lloyd 1974; Charlesworth 1981; Delannayet al. 1981; Gouyonet al. 1991). When the hermaphrodites are random mating, the following four parameters define a gynodioecious population:
w: the female advantage;
ψ_{f}: the average frequency of females among all families of female maternal parents;
ψ_{h}: the average frequency of hermaphrodites among all families of hermaphrodite maternal parents;
h: the frequency of hermaphrodites.
The female advantage is the ratio of the female reproductive success of female individuals to that of hermaphrodites. Such fertility differences can result from resource reallocation or pleiotropic effects of the sexdetermination genes (Charlesworth 1981). We assume w ≥ 1 because otherwise the maintenance of females requires biologically implausible conditions (Lewis 1941).
Because the sexdetermination system can be quite complex and varies in different gynodioecious systems (e.g., De Haanet al. 1997a; Charlesworth and Laporte 1998), we consider a general model, valuable for a wide range of sexdetermination systems. Consequently, this sexdetermination system is not precisely defined by its genetic factors here (i.e., the number and mode of action of the genes involved). Rather, it is phenotypically characterized by using two parameters, ψ_{f} and ψ_{h}, describing the average sex ratio in families of female or hermaphrodite maternal parents, respectively.
The relative frequency of hermaphrodites at equilibrium is calculated as the real roots of the polynomial:
Although the precise genetics of the sexdetermination system is not specified here, three main cases are distinguished.
Cytonuclear gynodioecy: The sexual phenotype results from an interaction between cytoplasmic and nuclear genes. With this sexdetermination system, females can be maintained with a less than twofold female advantage, provided that some additional factor affecting the fertility is acting [e.g., a cost associated with nuclear alleles restoring male fertility (Charlesworth 1981; Delannayet al. 1981; Gouyonet al. 1991)]. These factors are accounted for by allowing ψ_{f} and ψ_{h} to vary between 0 and 1.
Nuclear gynodioecy: The sexual phenotype is solely controlled by nuclear genes. The equilibrium frequency of females is between 0 and 0.5, depending only on the female advantage, which must exceed twofold (w > 2; Lloyd 1975, 1976):
Cytoplasmic gynodioecy: Generally, cytoplasmic gynodioecy means strictly maternal sex inheritance (ψ_{f} = ψ_{h} = 1). An equilibrium polymorphism is not possible without additional assumptions. For instance, pollen limitation can result in frequencydependent selection. This determines an equilibrium frequency of females, such that the female advantage is equal to one (Lewis 1941). Here, we do not consider pollen limitation but we extend the definition of cytoplasmic gynodioecy to the cases where both sexinheritance parameters, ψ_{f} and ψ_{h}, tend toward one, depending on the penetrance of the genes. With a female advantage equal to one, from the solution of Equation 1, the equilibrium frequency of females is
N_{ec} AND N_{en} IN CASE OF RANDOM MATING
The recurrence equations for the probability of identity by descent for nuclear and cytoplasmic genes drawn at random were derived (appendix a). For that purpose, we defined the probabilities ψ_{*u} that a gene of type * carried by an individual of sex u was inherited from an individual of the same sex in the previous generation (* = c for cytoplasmic, n for nuclear; u = h for hermaphrodite, f for female). The corresponding probabilities that a gene of type * carried by an individual of sex u was inherited from an individual of the opposite sex in the previous generation are (1 − ψ_{*u}).
The effective population size was deduced from the leading eigenvalue of the transition matrix of the coefficients of kinship (Ewens 1979, 1982; Whitlock and Barton 1997). This value was simplified under the hypothesis of a large census population size, N, that is, by assuming N(1 − h) ≫ 1 and Nh ≫ 1. The results depend on the amount of gene flow occurring between females and hermaphrodites. When gene flow between both sexual phenotypes is extremely restricted (that is, N(1 − ψ_{*f}) → 0 ∩ N(1 − ψ_{*h}) → 0), genes in one sexual phenotype cannot spread to the other. Consequently, genetic drift occurs independently within each sexual phenotype, and N_{e*} tends toward infinity if both sexual phenotypes are maintained. This situation never happens for nuclear genes because at least half of the nuclear genes carried by females necessarily derive from hermaphrodite parents. However, it can occur for cytoplasmic genes, in the case of strict maternal sex inheritance for instance. When substantial gene flow occurs between the two sexual phenotypes, the effective population sizes are approximated by
N_{ec} AND N_{en} AS FUNCTIONS OF THE FREQUENCY OF FEMALES IN THE CASE OF RANDOM MATING OF HERMAPHRODITES
Figure 1 presents the effective population sizes, N_{ec} and N_{en}, and the variances of fitness,
Figure 1A shows that N_{e*} depends on the frequency of females but also on the sexdetermination system. The N_{ec}/N values vary from N_{ec}/N = (1 − h) in the cases of dioecy and nuclear gynodioecy with dominant male sterility (the frequency of females varying from 0 to 1 in the former, and from 0 to 0.5 in the latter) to N_{ec}/N = 1 in the case of cytoplasmic gynodioecy. The set of N_{en}/N values is included in an area delimited by three curves: N_{en}/N = 1, N_{en}/N = 4h(1 − h), and N_{en}/N = h, obtained with nuclear gynodioecy, dioecy, and cytoplasmic gynodioecy, respectively.
Comparison of Figures 1A and 1B shows that the influence of the sex determination system on the effective population sizes is mainly due to variation in the variances of cytoplasmic and nuclear fitness. For a given frequency of females, the sex determination system is related to the magnitude of the female advantage and thus the variances of cytoplasmic and nuclear fitness. For instance, with cytoplasmic gynodioecy, females are maintained with no female advantage, so the variance of cytoplasmic fitness takes its minimum (
Although these graphs demonstrate that variations of N_{e*} are mainly explained by variation in the variances of fitness, another important factor is the heritability of fitness. Indeed, a correlation of the fitnesses over generations will result in cumulative effects of selection over generations. The case of nuclear gynodioecy provides an illustration. For a given sex ratio, a wide range of N_{ec} is observed (Figure 1A, darker area) although the variance of cytoplasmic fitness only depends on the frequency of females. In particular, one can see that N_{ec} is lower with dominant than with recessive male sterility. This difference is due to variations in the sexinheritance parameters (see Table 1). Indeed, with dominant male sterility, the maternal progeny of hermaphrodites are necessarily entirely hermaphroditic. The cytoplasmic genes of hermaphrodites are therefore more drastically selected against (due to the female advantage) than with recessive male sterility, where hermaphrodites can produce female offspring in their maternal progeny. In other words, the cumulative effects of selection are higher with dominant than with recessive male sterility (see Table 1).
THE CASE OF PARTIAL SELFING
We now consider the possibility of selfing in the hermaphrodites. The parameter S denotes the secondary selfing rate of hermaphrodites, which takes into account selfing and all inbreeding depression components (e.g., S is zero when inbreeding depression is 100%; see Landeet al. 1994). When nuclear genes are involved in the sexdetermination system, selfing may influence the sex ratio in the maternal families of hermaphrodites. Therefore, in Equations 1, 2a, 2b, and 2c, the average frequency of hermaphrodites among all families of hermaphrodite maternal parents (ψ_{h}) is replaced with
Cytoplasmic effective population size: The recurrence equations for identitybydescent values for cytoplasmic genes derived for the case of random mating (Equation A3) are still valid. Thus, with partial selfing, N_{ec} can still be derived from Equations 4 or 5. However, for a given sexdetermination system, the selfing rate can influence the values of the sexinheritance parameters, ψ_{f} and ψ_{h}. This occurs in the case of nuclear gynodioecy with recessive male sterility (see appendix c), but not with dominant male sterility. As a consequence, N_{ec} is influenced by the selfing rate, and this is illustrated in Figure 2, which shows that the effect is slight.
Nuclear effective population size: For nuclear genes, inbreeding and kinship coefficients must be distinguished when selfing occurs. The transition matrix of these coefficients was derived and the nuclear effective population size was deduced from the leading eigenvalue of this matrix (appendix d).
In nonrandom mating populations, N_{en} is classically expressed as a function of the heterozygote deficiency, F_{is} (Pollak 1987; Caballero and Hill 1992). With gynodioecy, it is of special interest to distinguish how the average heterozygote deficiency of the population is apportioned between females (F_{isf}) and hermaphrodites (F_{ish}). These heterozygote deficiencies are equal to the equilibrium value of the inbreeding coefficients in an infinite population. From Equation D1, these are approximated by
N_{en} is shown as a function of the frequency of females for the case of complete selfing of hermaphrodites in Figure 3. The minimum N_{en}/N values are observed with dioecy and cytoplasmic gynodioecy, as in the case of random mating. With cytoplasmic gynodioecy, however, N_{en}/N is now reduced to h(1 − S/2). Thus, minimum N_{en}/N values are lower than in the case of random mating, when the frequency of females is higher than (2 − S)/8.
The maximum N_{en}/N values are observed with dioecy if the female frequency exceeds 0.5. For lower female frequency, the values of the sexinheritance parameters leading to the maxima do not define a particular genetic determination system. Although the maximum N_{en}/N values are identical to the case of random mating when the frequency of females exceeds 0.5, they are reduced otherwise. Figure 3 also illustrates that N_{en}/N now differs slightly among the different nuclear gynodioecious systems.
DISCUSSION
Male sterility and effective population size: We have shown that, in most of the cases, both nuclear and cytoplasmic effective sizes of a gynodioecious population are reduced as compared to the census size. This reduction is induced by two inherent consequences of gynodioecy. First, in most of the cases, the maintenance of male sterile individuals implies fitness differences between sexual phenotypes, affecting either cytoplasmic or nuclear genes. Second, when sex is heritable, the influence of this variance of fitness is increased because the same lineages are recurrently selected for, generating a cumulative effect of selection over generations. Both the variance in fitness and the effect of cumulative selection over generations depend on the population parameters, i.e., the female advantage and the genetic basis of sex determination. Consequently, the most obvious cause of the effect of gynodioecy on the effective population sizes, the relative proportion of females and hermaphrodites, cannot be evaluated without taking these population parameters into account. Their influence is considerable. For instance, because N_{ec} ranges from the number of females up to the total number of individuals, great differences in this effective size are paradoxically observed when frequencies of females are low.
Comparison of effective sizes: Another remarkable result is that relative responses of nuclear and cytoplasmic effective population sizes to the same population parameters can be quite different because cytoplasmic and nuclear genes are not under the same selection pressures. Variation of the ratio N_{ec}/N_{en} is illustrated in Figure 4 for random mating populations. An extreme case occurs for nuclear gynodioecy with dominant male sterility. With a female advantage only slightly over two, females are maintained at low frequency (Lewis 1941). Nevertheless, the few cytoplasmic genes of the females tend to invade the population. This results in a drastic reduction of N_{ec}, equivalent to a cytoplasmic bottleneck, although N_{en} remains equal to the actual size. Note, however, that the present model takes no account of stochastic variation in the frequency of females. Using stochastic simulations of the model (not shown), we found that when the expected frequency of females is very low, the loss of females due to genetic drift will limit (but not cancel) the decline of N_{ec}. A strong reduction of cytoplasmic neutral genetic diversity is therefore expected in species experiencing cryptic dominant male sterility. A second extreme case occurs when the sex ratio is highly biased toward females. In that case, in contrast to the one just described, N_{ec} can be much higher than N_{en}. For instance, with 87% females, as has been observed in some Thymus vulgaris populations (Couvetet al. 1986), the ratio N_{ec}/N_{en} would be ~8 for cytoplasmic gynodioecy.
Such large variation in the ratio N_{ec}/N_{en} suggests that the relative levels of cytoplasmic and nuclear genetic diversity may be highly variable in gynodioecious populations characterized by different population parameters. Indeed, reduction of effective population size is expected to enhance the time to fixation of neutral variants, resulting in a reduction of the neutral genetic diversity (Kimuraet al. 1963) at the rate 1/N_{ec} each generation for haploid cytoplasmic genes, and 1/(2N_{en}) for diploid nuclear genes.
The influence of selfing: Selfing leads to an additional reduction of N_{en} as compared to the case when hermaphrodites are randomly mating. This is as expected given that selfing reduces N_{e} in hermaphrodite populations. In a gynodioecious population, however, it is frequently expected that females, by promoting outcrossing, will mitigate the effect of selfing. Nevertheless, our results show that the main factor influencing the effect of selfing is the female advantage rather than the frequency of females: the lower the female advantage, the higher the reduction of N_{en} due to selfing. The reduction of N_{en} due to selfing is substantial in the case of cytoplasmic gynodioecy but tends to be negligible when the female advantage tends toward infinity. Unexpectedly, we found that selfing also influences N_{ec} for some sexdetermination systems. For instance, with recessive male sterility, N_{ec} increases with the selfing rate of hermaphrodites. This happens because the increased frequency of homozygotes modifies the values of the sexinheritance parameters compared to the case of random mating. In particular, the correlation between the sexual phenotype of an individual and its maternal parent is reduced and so are the cumulative effects of selection over generations.
In conclusion, in spite of simplifying assumptions (e.g., the consideration of isolated populations at equilibrium for the frequency of females), important results were derived. The most striking ones are the great impact of the sexdetermination system on both cytoplasmic and nuclear effective population sizes and the notable differences between the two effective population sizes. Because experimental studies have suggested that gynodioecious populations may often not be at their equilibrium sex ratio (e.g., Belhassenet al. 1989; BoutinStadleret al. 1989; Manicacciet al. 1992; De Haanet al. 1997a) and theoretical models have addressed the maintenance of gynodioecy in structured metapopulations (McCauley and Taylor 1997; Couvetet al. 1998), extension of the present model should include the consideration of (i) nonequilibrium populations and (ii) subdivided populations.
Acknowledgments
We thank David E. McCauley for having suggested this study and Oscar Gaggiotti, Philippe Vernet, Pierre SaumitouLaprade, Henk Van Djik, Frédérique Viard, Myriam Valero, and Christine Vassiliadis for critical reading of the manuscript. We are extremely grateful to Deborah Charlesworth, Stewart Schultz, Montgomery Slatkin, and two anonymous reviewers for very helpful and detailed comments on previous submitted versions. This research was supported by the Région Nord PasDeCalais and the Fond Européen de Développement Régional. The first author received a research fellowship from the Ministère de l'Enseignement Supérieur et de la Recherche no. 94941.
APPENDIX A
Recurrence equations for the probability of identity by descent when hermaphrodites are random mating: In gynodioecious populations, six coefficients of kinship must be defined because there are two sexual morphs:
f_{nf}: kinship coefficient for nuclear genes between two females;
f_{nh}: kinship coefficient for nuclear genes between two hermaphrodites;
f_{nx}: kinship coefficient for nuclear genes between a female and a hermaphrodite;
f_{cf}, f_{ch}, f_{cx}: corresponding kinship coefficients for cytoplasmic genes.
To derive transition equations for these coefficients, we defined four probabilities, ψ_{*u}, that a gene of type * carried by an individual of sex u came from an individual of the same sex in the previous generation (* = c for cytoplasmic, n for nuclear; u = h for hermaphrodite, f for female). The corresponding four probabilities that a gene of type * carried by an individual of sex u came from an individual of the opposite sex in the previous generation are (1 − ψ_{*u}). These probabilities depend on the sexinheritance parameters (ψ_{f}, ψ_{h}), the female advantage (w), and the frequency of females (1 − h). For cytoplasmic genes, assuming complete maternal inheritance
Equations A3 can be written in matrix form, S_{*}_{t} = T_{*}S_{*t}_{−1}, where S_{*t} is a column vector of the three probabilities of identity by descent at generation t and T_{*} is the transition matrix:
The effective population size (N_{e*}) is derived from the leading eigenvalue (λ_{*}) of the transition matrix T_{*}
APPENDIX B
Variance of fitness among sexes: The fitness of a given gene depends on the sexual phenotype of the individual carrying this gene. There are two approaches for deriving the fitnesses of nuclear and cytoplasmic genes carried by a female (w_{*f}) or a hermaphrodite (w_{*h}). First, fitnesses can be expressed as a function of the female advantage, w, and the sex ratio. Because cytoplasmic genes are transmitted only through seeds while nuclear genes are transmitted through both seeds and pollen, we have
APPENDIX C
Recurrence equations for genotypic frequencies in the case of nuclear recessive with partial selfing of hermaphrodites: Male sterility is controlled by a single nuclear locus with two alleles, A and a. The recessive allele a determines male sterility. Therefore, female individuals are homozygote aa and hermaphrodites are either homozygous AA or heterozygous. Let the genotypic frequencies of AA, Aa, and aa individuals of a given generation be p, q, and r, respectively. The genotypic frequencies in the next generation can be numerically calculated from recurrence equations
For a given female advantage, selfing does not influence the frequency of female individuals at equilibrium (see Lewis 1941). However, homozygote frequencies increase with the selfing rate. This change in the population genotypic structure will modify the sexinheritance parameters. The frequency of a among male gametes is q/2(p + q); sex inheritance parameters are given by
APPENDIX D
Recurrence equations for the inbreeding and kinship coefficients with partial selfing of hermaphrodites: The inbreeding coefficient of an individual (
Footnotes

Communicating editor: M. Slatkin
 Received April 21, 1999.
 Accepted July 29, 1999.
 Copyright © 2000 by the Genetics Society of America