GENETIC MODELS OF POSTZYGOTIC ISOLATION

We follow Dobzhansky (1937, p. 256) and Muller (1940, 1942) in assuming that the alleles causing inviability and sterility in hybrids have no such effects within lineages during their divergence from a common ancestor (see also Orr 1995; Orr and Orr 1996). For simplicity, we also assume that each species is fixed at all loci. Following Turelli and Orr (1995), we further assume that separate hybrid incompatibilities contribute additively to an underlying “breakdown score” that is translated into fitness by a monotone decreasing function, denoted V. (The additivity assumption is made for convenience and will be discussed below.) Hybrids become inviable/sterile when their breakdown score reaches a threshold value, C. Finally, we assume that loss of fitness in hybrids is caused by many incompatibilities between loci scattered throughout the genome. (Wu and Palopoli 1994 and Naveira and Maside 1998 review the evidence for this assumption.) This allows us to compare the expected breakdown scores of different genotypes.

Suppose that a hybrid genotype suffers n distinct incompatibilities and that the ith incompatibility contributes e_i > 0 to the breakdown score. (Note that specific loci may contribute to more than one incompatibility.) The fitness of this genotype is W=V(∑i=1nei), (1) where W = 0 if the breakdown score S=Σi=1nei≥C . Our analyses apply to taxa in which either males or females are the heterogametic (XY or ZW) sex. Because most relevant data come from Drosophila, however, we assume male heterogamety and note those occasions where female heterogamety alters our conclusions. Among individuals of identical genotype (e.g., F₁ hybrid males), some are often fertile and others sterile (e.g., Orr and Coyne 1989). Thus, whether any particular individual is fit or unfit must depend not only on its genotype but on environmental (including developmental) variation. It should be understood then, that given a hybrid breakdown score S and the mapping function V, we can predict only average fitness.

Formally, the assumption that many incompatibilities contribute to hybrid sterility/inviability is central to our analysis. Without loss of generality, we can assume that the critical threshold is C = 1. When a large number, n, of incompatibilities is required to reach this threshold, the effect of each must be roughly proportional to 1/n. Hence, the variance of the breakdown scores must also be at most proportional to 1/n as the breakdown score is simply the sum of n random variables, each having mean on the order of 1/n and variance on the order of 1/n². If we assume that n is large, we can approximate the fitness of a genotype by evaluating the function V in (1) at the expected breakdown score. This simplification is used throughout.

A model of two-locus incompatibilities: The central idea of our analysis is that hybrids can experience three different types of two-locus Dobzhansky-Muller incompatibilities, depending on whether the incompatible alleles at the interacting loci are homozygous or heterozygous. Imagine that one species' genotype is AAbb and the other aaBB, where a and b are incompatible in hybrids and A and B denote ancestral (and hence compatible) alleles. Further imagine that there are many such pairs of loci. We label the incompatibilities according to the number of loci that are homozygous for incompatible alleles. In an F₂ hybrid genotype, certain incompatibilities might be homozygous-homozygous (e.g., aabb), which we denote H₂; each such incompatibility contributes, on average, h₂ to the hybrid breakdown score. Other incompatibilities might be homozygous-heterozygous (aaBb or Aabb), denoted H₁; each contributes, on average, h₁ to the breakdown score. All remaining incompatibilities are heterozygous-heterozygous (AaBb), denoted H₀; each contributes, on average, h₀ to the breakdown score. Note that any locus homozygous for a compatible allele (AA or BB) cannot contribute to hybrid breakdown.

This last assumption merits comment. If the substitutions causing hybrid problems are driven by natural selection within species, one might expect that genotypes combining homozygous ancestral alleles at one locus with selectively favored alleles at another would enjoy enhanced fitness relative to the ancestral genotype, AABB. We assume, however, that any such effects are negligible compared with the deleterious Dobzhansky-Muller effects. Indeed these positive effects must, on average, be small compared to the Dobzhansky-Muller effects, or hybrids would not suffer large net fitness losses (relative to the parental means).

Initially, we consider autosomal-autosomal, X-autosomal, and X-X interactions, but ignore maternal effects and the Y. We also assume dosage compensation such that hemizygosity at X-linked loci is equivalent in fitness effect to homozygosity. (This assumption is not needed for our analysis of Haldane's rule, which centers on incompatibilities that either do or do not involve a hemizygous X chromosome; these correspond to H₁ vs. H₀ incompatibilities, irrespective of dosage compensation.) To compare the breakdown scores of different genotypes, we assume that there are n two-locus incompatibilities in a reference genotype that carries one set of autosomes and one X from each parental species, as in F₁ females. These incompatibilities are assumed to be randomly scattered throughout the genome. For normalization purposes, we use this reference genotype even when considering hybrid males.

To find the expected breakdown score for any particular hybrid genotype, we merely need to know the proportion of the genome that is homozygous (or hemizygous) from species 1 (p₁), the proportion that is homozygous (or hemizygous) from species 2 (p₂), and the proportion that is heterozygous for material from the two species (p_H, where p_H = 1 − p₁ − p₂). A simple combinatoric argument shows that, on average, a fraction p1p2+(p1+p2)pH+pH2 of the n incompatibilities in the reference genotype appear in our hybrid genotype. In particular, our hybrid suffers an average of np₁p₂ H₂ incompatibilities, n(p₁ + p₂)p_H H₁ incompatibilities, and npH2 H₀ incompatibilities. (These calculations take into account the fact that incompatibilities are generally “asymmetric”; i.e., if introgression of allele A₁ from taxon 1 into taxon 2 lowers fitness, the reciprocal introgression of A₂ into taxon 1 does not, e.g., Wu and Beckenbach 1983; Wu and Palopoli 1994, p. 292. Thus, although a fraction 2p₁p₂ of all two-locus incompatibilities occur between loci in the portions of the genome included in p₁ and p₂, only half involve taxon 1 alleles in p₁ and taxon 2 alleles in p₂. We need not, however, discount the term pH2 by a factor of 2 as all potentially incompatible alleles from both species are present in heterozygous regions of the genome.) The expected breakdown score is thus E(S)=n[p1p2h2+(p1+p2)pHh1+pH2h0]. (2) This simple result forms the foundation of our analyses. We now use it to consider the two best known phenomena in the genetics of speciation: Haldane's rule and the large X effect.

Haldane's rule for inviability: The case of hybrid inviability is simpler than that of sterility as the same genes likely affect the viability of both hybrid males and females (Orr 1989a,b; Wu and Davis 1993). For F₁ females, all loci are heterozygous (p_H = 1), and (2) gives E(Sf)=nh0. (3) Let g_X be the fraction of all loci causing incompatibilities that are X-linked (cf. Turelli and Orr 1995; Turelli and Begun 1997). When n is large and the X and autosomes evolve at equal rates, we can estimate g_X as the fraction of the genome that is X-linked. Thus, in F₁ males, some proportion of the genome is X-linked (g_X) and the rest is autosomal. Because the X is the only material that is hemizygous, p₁ + p₂ = g_X, p₁p₂ = 0 (either p₁ or p₂ must be zero as the X in hybrid males derives from one species only), and p_H = 1 − g_X. Thus, E(Sm)=n[gx(1−gx)h1+(1−gx)2h0]. (4)

In the limiting case in which none of the genome is X-linked (g_X = 0), (3) and (4) show that hybrid males and females have the same expected breakdown scores and Haldane's rule for inviability cannot arise (by this mechanism). As noted by Presgraves and Orr (1998), the case g_X = 0 arises in the genus Aedes in which the X and Y chromosomes are largely homologous and freely recombine. And, in fact, Haldane's rule for inviability does not occur in this genus.

More generally, (3) and (4) show that if g_X > 0, Haldane's rule is expected, i.e., E(S_m) > E(S_f), whenever h0h1<1−gx2−gx=12+gx∕(1−gx). (5) Condition (5) allows for X-X incompatibilities that afflict only females and is, not surprisingly, somewhat more stringent than our previous condition, h₀/h₁ < ½ (or equivalently d < ½), which ignored X-X incompatibilities (Orr 1993a; Turelli and Orr 1995). For instance, for taxa such as D. melanogaster with g_X ≈ 0.2, (5) requires h₀/h₁ < 4/9. As expected, (5) reduces to h₀/h₁ < ½ when X-X incompatibilities are ignored. Condition (5) also implies that whenever F₁ males show reduced viability, “unbalanced” F₁ females, who inherit both X's from one species, should be less viable than normal F₁ females, a fact confirmed by experiment (Orr 1993b; Wu and Davis 1993; cf. Coyne 1985).

When there are unequal rates of evolution between the X and autosomes, g_X no longer equals the proportion of the genome that is X-linked. If, for instance, the X evolves faster than the autosomes, as suggested by Charlesworth et al. (1987), g_X will exceed the fraction of the genome that is X-linked. The above results still hold, however.

Haldane's rule for sterility: There are good reasons for thinking that different loci will affect hybrid male vs. female fertility. First, mutagenesis experiments within D. melanogaster show that the overwhelming majority of steriles afflict one sex only, while most lethals affect both sexes (reviewed in Ashburner 1989). Second, backcross and introgression data show that chromosome regions that cause the sterility of one hybrid sex typically do not affect the other (Orr and Coyne 1989; Orr 1992; Hollocher and Wu 1996; Trueet al. 1996).

This opens the door to a possibility that does not arise with hybrid inviability: when different loci affect the two sexes, we have no guarantee that substitutions ultimately afflicting males will occur at the same rate as those afflicting females. Indeed there is evidence that alleles causing sterility in hybrid males accumulate faster than those causing sterility in females, at least in Drosophila and mosquitoes (Hollocher and Wu 1996; Trueet al. 1996; Presgraves and Orr 1998). Such “faster-male” evolution might well contribute to Haldane's rule for sterility (Orr 1989a; Wu and Davis 1993). We consider this effect below and show how dominance and fastermale effects can be taken into account simultaneously.

Incompatibilities affecting hybrid female vs. hybrid male sterility may differ in both number and average effects. Let n_f (n_m) denote the number of incompatibilities in our reference genotype that affect hybrid females (males). We assume that the average effects of H₀ and H₁ incompatibilities are h₀ and h₁ for females and h∼0 and h∼1 for males. Thus, the expected breakdown scores of F₁ females and males (from Equation 2) are E(Sf)=nfh0 (6a) and E(Sm)=nm[gx(1−gx)h∼1+(1−gx)2h∼0], (6b) where g_X denotes the fraction of all loci involved in male-sterilizing incompatibilities that are X-linked.

Under this model, Haldane's rule for sterility is expected [E(S_m) > E(S_f)] whenever h0h1<(nmh∼1nfh1)((1−gx)2h∼0h∼1+gx(1−gx)). (7) To simplify the notation, we let τ=nmh∼1∕(n1h1) and assume that the ratio of effects for H₀ vs. H₁ incompatibilities is the same in females and males, i.e., h0∕h1=h∼0∕h∼1=d0 . Note that τ quantifies the relative cumulative effects of incompatibilities contributing to hybrid male vs. hybrid female sterility, taking into account both their numbers and average effects. Criterion (7) shows that both faster-male evolution, as measured by τ > 1, and dominance, as measured by d₀, can contribute to Haldane's rule. We can express (7) either as a bound on τ or a bound on d₀, namely τ>d0(1−gx)2d0+gx(1−gx) (8a) or d0<tgx(1−gx)1−τ(1−gx)2. (8b) As before, the constraint (8b) on the dominance parameter d₀ is somewhat more restrictive when X-X incompatibilities are considered than the analogous expression (B2) of Turelli and Orr (1995), which ignored X-X incompatibilities. If τ = 1, condition (8b) reduces to (5), i.e., the condition for inviability. When none of the genome is X-linked (g_X = 0) (8a) becomes τ > 1. In this case, dominance cannot play a role in Haldane's rule and faster-male evolution is required.

In general, both faster-male evolution and dominance might contribute to Haldane's rule for sterility in male-heterogametic species. To assess their relative contributions, one can consider the ratio of male to female breakdown scores. This ratio, which must exceed 1 for Haldane's rule in male-heterogametic species, is R=τ(1−gx)(1−gx+gxd0). (9) Ratio (9) reveals that Haldane's rule is promoted by faster-male evolution (large τ) and by greater recessivity of the factors causing hybrid sterility (small d₀). Both effects can be seen in Figure 1. If τ is sufficiently large, Haldane's rule will occur for any level of dominance. Conversely, in female-heterogametic species, R must be <1 to produce Haldane's rule; this is clearly made more difficult by faster-male evolution (τ > 1). The resulting conditions correspond to τ < 1 in a male-heterogametic species. As illustrated in Figure 1, extreme recessivity is required to produce Haldane's rule in this case.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

Contributions to Haldane's rule from dominance and faster-male evolution. R is plotted from Equation 9. When R > 1, Haldane's rule is obeyed “on average.” The calculations assume that 20% of the genome is X-linked (g_X = 0.2), as in D. melanogaster. The solid curve corresponds to the case of no faster-male evolution (τ = 1), so that d₀ < 4/9 is required for Haldane's rule. The dashed curve corresponds to faster-male evolution in a male-heterogametic species (τ = 2), and the dotted curve to faster-male evolution in a female-heterogametic species (τ = 0.5).

Comparative analyses support the idea that fastermale evolution acts against Haldane's rule for sterility in female-heterogametic species: Haldane's rule is more common for sterility than for inviability in male-heterogametic species, while the reverse appears true in female-heterogametic species, as expected if faster-male evolution acts in both groups (Wu and Davis 1993; Turelli 1998). This pattern should not, however, obscure an even more striking one: Haldane's rule holds for 42 of 43 cases of unisexual hybrid sterility in birds and Lepidoptera (Laurie 1997); i.e., females are overwhelmingly sterile in these taxa, despite presumed faster-male evolution. Put differently, when dominance and faster-male effects are pitted against one another, dominance predominates, yielding virtually no exceptions to Haldane's rule. This suggests two possibilities that are not mutually exclusive: H₀ incompatibilities have much smaller effects than H₁ incompatibilities; or, as discussed below, Y-associated incompatibilities play a large role in postzygotic isolation. We address another important aspect of faster-male evolution in the discussion.

Our analyses of Haldane's rule did not require us to consider homozygous-homozygous incompatibilities, which cannot appear in F₁ hybrids. Thus, our previous simple “dominance” analyses (Orr 1993a; Turelli and Orr 1995) sufficed to correctly identify the conditions for Haldane's rule. But as we now show, a more detailed model is required to understand the full consequences of Dobzhansky-Muller interactions.

The large X effect: As noted by Coyne and Orr (1989a), one of the most striking features of genetic studies of postzygotic isolation is that in backcrosses the X chromosome appears to have a disproportionately large effect on hybrid male fitness relative to comparably sized autosomes. There has, however, been a great deal of confusion about the biological significance, if any, of this so-called large X effect. Wu and Davis (1993) and Hollocher and Wu (1996) have argued that the effect is an artifact of backcross analysis: in any backcross, X substitutions in males replace a hemizygous X from one species with a hemizygous X from the other. Autosomal substitutions, on the other hand, replace a single autosome from one species with a single one from the other. Thus, they argue, hemizygous X substitutions should have larger effects on hybrid fitness than heterozygous autosomal ones. Orr (1997) and others, however, suggest that the large X effect may not be an inevitable artifact of backcross analysis but evidence for the partial recessivity of the genes causing postzygotic isolation. A quantitative treatment is clearly required to find the conditions under which large X effects are expected.

Consider a backcross analysis of male sterility in which fertile F₁ females are crossed to males from taxon 1. When assessing the effect of “foreign” loci, note that the introgressed segments of X₂ are necessarily hemizygous, while the introgressed autosomal segments are heterozygous. To quantify the X effect, we require an explicit comparison between X-linked and autosomal introgressions. Although discussions of the large X effect have been mostly qualitative, at least two quantitative criteria can be used. A large X effect might be declared if a hemizygous X introgression has (1) a greater effect than a heterozygous autosomal introgression that is twice as large; or (2) more than twice the effect of an equal-sized heterozygous autosomal introgression. The first criterion requires fewer assumptions, but the second (used by Coyne and Orr 1989a, among others) leads to similar results.

We first use criterion 1, comparing X-linked and autosomal introgressions from species 2 into a background homozygous for species 1. For the X introgression, let p₂ = q, where q is the fraction of the haploid genome that is introgressed, and p₁ = 1 − q. For the autosomal introgression, let p_H = 2q (i.e., it is twice as large as the X introgression) and p₁ = 1 − 2q. The expected breakdown scores are E(Sx)≡E(S∣p2=q,p1=1−q)=nq(1−q)h2 (10a) and E(Sauto)≡E(S∣pH=2q,p1=1−2q)=n[(1−2q)2qh1+4q2h0]. (10b) The X introgression has a greater effect when E(S_X) > E(S_auto), which requires q[(1 − q)(h₂ − 2h₁) + 2q(h₁ − 2h₀)] > 0 or h2−2h1+2q1−q(h1−2h0)>0. (11) Our dominance-theory explanation of the large X effect (Turelli and Orr 1995) posited h0h1<12, (12) which constrains only the relative effects of H₀ and H₁ incompatibilities. But (11) shows that H₂ interactions are also important. The natural extension of our “dominance” constraint (12) is h1h2<12; (13) i.e., homozygous-homozygous incompatibilities have more than twice the deleterious effect of heterozygous-homozygous ones. The important point is that when dominance conditions (12) and (13) are met, so is (11).

The analysis above is idealized. Real backcross analyses involve comparing many genotypes, some of which carry the “foreign” X (or portions of it) and some of which do not. We thus extend our analysis to more realistic situations in which the “background” for an introgression includes heterozygous material (i.e., p_H > 0). In this case, we compare E(S_X) = E(S|p₂ = q, p_H = r, p₁ = 1 − r − q) with E(S_auto) = E(S|p_H = r + 2q, p₁ = 1 − r − 2q). Again, (12) and (13) suffice to give E(S_X) > E(S_auto). Thus, no matter what the rest of the hybrid genome looks like, introgressing part of the X will lower hybrid fitness more than introgressing twice as much autosomal material when (12) and (13) hold.

Criterion 2 is trickier to apply: while it is easy to compare the average breakdown scores of X vs. autosomal introgressions, we must actually compare the fitness effects of X vs. autosomal introgressions. But these are known only if we know the function V that maps breakdown score onto fitness. For simplicity, then, we assume that the fitness function is nearly linear over the relevant range of values, so that comparing breakdown scores themselves suffices. If fitness declines faster than linearly (e.g., Kondrashov 1988; Charlesworth 1990), our conditions will be too restrictive.

We again first compare X-linked vs. autosomal introgressions from taxon 2 into a background homozygous for taxon 1. The breakdown score is initially zero in both cases, and we want to compare E(S_X) = E(S|p₂ = q, p₁= 1 − q) with 2E(S_auto) = 2E(S|p_H = q, p₁ q). With (2), we see that E(S_X) > 2E(S_auto) if h2−2h1−2q1−qh0>0. (14) If we impose our standard “dominance” condition (12) on h₀/h₁, we see that (14) is satisfied when h1h2<12+q∕(1−q). (15) For small introgressions (q ≈ 0), this reduces to (13), h₁/h₂ < ½. For larger introgressions, it is more restrictive. But even if an X-linked introgression involves, say, 20% of the genome, (15) requires h₁/h₂ < 4/9 to produce a large X effect, which is only slightly more stringent than (13).

Next we consider introgressions into backgrounds in which some of the autosomal genome is already heterozygous. We want to compare E(ΔS_X) ≡ E(S|p₂ = q, p_H = r, p₁ = 1 − r − q) − E(S|p_H = r, p₁ = 1 − r) with E(ΔS_auto) ≡, E(S|p_H = r + q, p₁ = 1 − r − q) − E(S|p_H = r, p₁ = 1 − r). Using (2), we see that E(ΔS_X) > 2E(ΔS_auto) if (h2−2h1)(1−q−r)+2r(h1−2h0)−2h0q>0. (16) For small introgressions (q ≈ 0), conditions (12) and (13) are again sufficient. We assume that both (12) and (13) hold and ask what additional constraints on h₁/h₂ and h₀/h₁ are imposed by (16). In general, these depend both on the size of the introgressions, q, and on the extent to which the genetic background is heterozygous, r. Condition (13) implies that the left-hand side of (16) decreases as q increases. Thus, assuming q ≤ 0.2, the most restrictive conditions on h₁/h₂ and h₀/h₁ occur with q = 0.2. By also considering the left-hand side of (16) as a function of r, one can show that (16) is always satisfied if h0h1<49andh1h2<49. (17)

Our key biological conclusion is that large X effects, by either criterion 1 or 2, are not automatic consequences of comparing hemizygous X with heterozygous autosomal substitutions. Instead, large X effects imply that the alleles causing hybrid sterility and inviability are fairly recessive. Indeed, large X effects are essentially guaranteed if condition (17) is met (in Drosophila). The more extreme the recessivity of the incompatible alleles, the more pronounced the large X effect. The “faster X” mechanism of Charlesworth et al. (1987) may also, of course, contribute to the large X effect.

Comparisons among other backcross genotypes: There are two different classes of backcross to consider, depending on whether one crosses F₁ hybrid males or females to a parental species. When F₁ males are backcrossed, the resulting progeny all inherit an X and a complete set of autosomes maternally. In this case, H₂ incompatibilities are impossible, and (2) reduces to E(S)=n[pH(1−pH)h1+pH2h0]=nh1[pH(1−pH)+pH2d0], (18) where p_H is the fraction of the genome that is heterozygous and d₀ = h₀/h₁. When these incompatibilities, rather than those involving the Y or maternal effects, dominate hybrid performance, the lowest fitness results when pH=12(1−d0). (19) For d < ½, the right-hand side of (19) is between 0.5 and 1.0. Thus we predict that introgressions of “intermediate” size will have the most devastating effects on hybrid fitness.

When F₁ females are backcrossed, the resulting male progeny can inherit an X that is incompatible with the haploid set of autosomes inherited paternally. This can produce H₂ incompatibilities so that all three terms in (2) may be nonzero, precluding simple predictions analogous to (19).

Other incompatibilities affecting hybrids: Data from several well-known hybridizations have shown important effects of Y-X (e.g., Orr 1987), Y-autosome (e.g., Zouroset al. 1988; Heikkinen and Lumme 1991), and maternal-zygotic incompatibilities (Sawamura 1996; Hutter 1997). Indeed our review of the literature suggests that both Y and maternal effects on postzygotic isolation are extraordinarily common. We surveyed all known genetic analyses of both hybrid sterility and inviability in Drosophila; the results are shown in Table 1. We found that 10 out of 11 species crosses involving hybrid male sterility show Y chromosome effects in at least one direction of the cross. Similarly, 4 out of 18 hybridizations show maternal effects; excluding male sterility, this figure rises to 4 out of 7. Moreover, the observation of nonreciprocal hybrid female inviability—or even sterility—in Drosophila (Coyne and Orr 1989b) suggests prima facie that maternal effects on hybrid fitness are common, as reciprocal F₁ females have identical nuclear genotypes. [In the D. melanogaster-D. simulans and D. montana-D. texana hybridizations, for instance, females are lethal in one direction of the cross only. (These taxa belong to different subgenera, so their hybrid outcomes are phylogenetically independent.) Analysis of each has proven the role of maternal factors (reviewed in Wu and Davis 1993 and Sawamuraet al. 1993).]

Given their frequent roles, it is important to incorporate Y and maternal effects into our analysis. This can be accomplished via Equation 1, by assuming that Y and maternal incompatibilities simply add to the total breakdown score, which we now denote S_T. Keeping with our earlier notation, we denote the cumulative breakdown score attributable to X-X, X-autosome, and autosome-autosome incompatibilities by S. Additional contributions from Y-associated and maternal-zygotic incompatibilities are denoted S_Y and S_MZ. We discuss each in turn.

Y-linked incompatibilities: We consider both male-heterogametic (XY) and female-heterogametic (ZW) taxa, but, for ease of discussion, refer to the sex-limited sex chromosome as the Y and its partner as the X. For simplicity, we focus on E(S_Y). Our qualitative predictions do not require that the number of Y-associated incompatibilities is large. Indeed the “large n” assumption is clearly implausible for Y-associated incompatibilities, as (1) the Y carries few complementation groups, at least in D. melanogaster and D. hydei (Ashburner 1989, Chap. 20); and (2) in at least one hybridization (D. mojavensis-D. arizonensis), the Y interacts with a single autosome (Zouroset al. 1988). Because the Y has no essential somatic function, at least in D. melanogaster (Ashburner 1989), we might expect the Y to play a role in hybrid sterility but not inviability.

Because the Y is hemizygous, we treat the Y-linked partner in any incompatibility as effectively homozygous. We do not distinguish between Y-X and Y-autosome incompatibilities and assume that each occurs in proportion to the fraction of the genome that is X-linked vs. autosomal. Let n_Y denote the number of incompatibilities between Y-linked loci from taxon 1 and loci in a complete haploid set, including an X, from taxon 2. In F₁ and backcross genotypes, these incompatibilities can occur in two forms, depending on whether the non-Y partner is homozygous or heterozygous. Incompatibilities involving homozygous partners have average effect y₂, whereas those involving heterozygous partners have average effect y₁.

Because the Y is largely heterochromatic, we have no a priori basis for estimating the fraction of all incompatibilities that involve this chromosome and therefore no basis for drawing quantitative conclusions about S_Y vs. S. But, by considering E(S_Y) alone, we can still draw interesting, albeit qualitative, conclusions. If the Y is inherited from taxon 1 and the source of the rest of the nuclear genome is described by the fractions p₁, p₂, and p_H E(SY)=E(SY∣Y1,p1,p2,pH)=nY(p2y2+pHy1). (20) For F₁ males (or F₁ females in female-heterogametic species), we expect that p₂ = g_X and p_H = 1 − g_X. Thus, E(SY∣heterogameticF1)=nY[gxy2+(1−gx)y1]. (21) Obviously, S_Y = 0 for the homogametic sex. Thus, Y-associated incompatibilities always afflict only the heterogametic sex and promote Haldane's rule (Muller 1942; White 1945, p. 225).

Y-associated incompatibilities also relax the constraints on the dominance, h₀/h₁, required for Haldane's rule for sterility, particularly in female-heterogametic species. For two reasons, this effect is likely to be especially important in taxa having small sex chromosomes. First, in the absence of Y effects, the upper bound on h₀/h₁ needed for Haldane's rule when faster-male evolution acts is proportional to g_X when g_X is small (see 8b). Thus, in female-heterogametic species with relatively small X's (on the order of 10% of the genome or less), such as birds (Abbott and Yee 1975) and lepidoptera (Robinson 1971, Chap. VIII), extreme recessivity would ordinarily be required to obtain Haldane's rule. Second, even if rare, Y-linked incompatibilities involve the potentially stronger H₁ and H₂ incompatibilities, whereas interactions not associated with the Y involve H₀ and H₁ incompatibilities.

Given this expected disproportionate effect of the Y in small-X taxa, it is interesting to note that Haldane's rule for sterility shows only a single exception in Lepidoptera and birds (Laurie 1997)—despite the presumption of faster-male evolution, which opposes Haldane's rule, in these groups (Wu and Davis 1993; Turelli 1998). This suggests that the combination of dominance and Y chromosome effects much more than compensates for any faster-male evolution.

Next, we consider the role of the Y in the large X effect. Consider a study of F₁ male sterility in which F₁ females are backcrossed to taxon 1. As above, we can compare the values of S_Y produced by an X-linked introgression of size q vs. an autosomal introgression of size 2q. These are E(SY∣p2=q,p1=1−q)=nYqy2 (22a) and E(SY∣pH=2q,p1=1−2q)=nY2qy1, (22b) respectively. Hence, an X introgression yields a larger Y-associated contribution to S_T if y1y2<12. (23) Note that if incompatibilities involving the Y are highly recessive (i.e., y₁/y₂ ⪡ ½), the relative effects of Y-autosomal interactions will be negligible unless the “foreign” autosomal segments are homozygous. Thus, our analysis suggests a bias toward finding Y-X incompatibilities vs. Y-autosomal. Indeed, in the best-known case of a Y-autosomal incompatibility—that between the Y of D. arizonae and a region of the fourth chromosome of D. mojavensis—flies that are homozygous for the incompatible autosomal segment are sterile whereas flies that are heterozygous show normal fertility (Pantazidiset al. 1993). The Y-autosome incompatibility acting in D. virilis and D. texana hybrids is also detectable only when the autosomal partners are homozygous (Lamnissouet al. 1996).

Maternal-zygotic incompatibilities: Again we focus on E(S_MZ) but recognize that the data suggest that maternal incompatibilities may involve few factors. These incompatibilities arise from interactions between loci in two different diploid genomes—that of the mother and her offspring—and they may generally affect viability only as maternal control of development is surrendered fairly early in development (Sawamura 1996). Such interactions are not surprising given that, in Drosophila, many genes deposit maternal transcripts in the unfertilized egg, and the transfer from maternal to zygotic control of development is gradual (Lawrence 1992). Maternal effects are distinct from cytoplasmic effects involving interactions between nuclear genes and maternally inherited cytoplasmic factors, such as microbes and mitochondria. Although cytoplasmic effects, such as those associated with Wolbachia, may contribute to reproductive isolation between some taxa (Hoffmann and Turelli 1997; Werren 1997), they seem to be relatively unimportant in producing the sex-limited hybrid viability differences on which we focus (Hurst 1993).

Maternal-zygotic incompatibilities may occur in three forms depending on whether the incompatible alleles are homozygous or heterozygous. To distinguish these incompatibilities from those acting within the offspring genome, we denote them by M₀, M₁, and M₂. There may be a qualitative distinction between the two types of M₁ incompatibilities (depending on whether the mother or offspring is homozygous for an incompatible allele), but we assume they have equal average effects. We assume that the average effect of M_i incompatibilities is m_i for i = 0, 1, 2 and that there are n_MZ maternal-zygotic incompatibilities between a cytoplasm produced by a taxon 1 mother and her F₁ hybrid daughters. Obviously, these would all be M₁ incompatibilities involving homozygous maternal loci and heterozygous loci in the offspring. We assume that a fraction g_X of these incompatibilities involve loci on the zygotic X. With backcross analyses involving hybrid mothers, a wide range of incompatibilities can appear. We focus on only those that occur in hybrids with taxon 1 mothers.

If the offspring genome is characterized by the fractions p₁, p₂, and p_H, as before, the expected contribution to their breakdown score from maternal-zygotic interactions is E(SMZ∣p1,p2,pH)=nMZ(m2p2+m1pH). (24) This reveals a qualitative difference between the consequences of these incompatibilities in male-heterogametic vs. female-heterogametic taxa.

In male-heterogametic species, the only relevant difference between F₁ females and males is that females suffer from M₁ incompatibilities between the paternal X and the maternal cytoplasm, whereas males do not (Patterson and Stone 1952, pp. 435–436, 489; Wu and Davis 1993). This appears in the breakdown scores as E(SMZ∣F1homogametic female)=nMZm1 (25a) and E(SMZ∣F1heterogametic male)=nMZ(1−gx)m1, (25b) since p_H = 1 in females, but p_H = 1 − g_X and p₁ = g_X in males. Thus females in male-heterogametic taxa suffer more from maternal incompatibilities than males; and incompatibilities involving the X can give rise to exceptions to Haldane's rule in these taxa. Indeed, Sawamura (1996) has argued that incompatibilities between maternally acting genes and the zygotic X can plausibly explain all known exceptions to Haldane's rule for viability in Drosophila. Our work suggests that these exceptions should be especially common in taxa with larger X's.

In contrast, in female-heterogametic species, F₁ males have p_H = 1, but F₁ females have p_H = 1 − g_X and p₂ = g_X. Thus, E(SMZ∣F1heterogametic female)=nMZ[m2gx+m1(1−gx)] (26a) and E(SMZ∣F1homogametic male)=nMZm1. (26b) This shows that maternal-zygotic interactions contribute to Haldane's rule for inviability in female-heterogametic species—as the heterogametic sex gets its cytoplasm from one species and its X from another—whenever M₂ incompatibilities are more severe than M₁. It would be surprising if this very weak “dominance” constraint is not satisfied.

Thus, in female-heterogametic species, dominance and maternal effects act in concert to promote Haldane's rule, whereas they act in opposition in male-heterogametic species. Maternal effects may, therefore, explain both the prevalence of exceptions to Haldane's rule for viability in Drosophila (and many of these exceptions appear evolutionarily independent; Sawamura 1996) and the virtual absence of exceptions in birds and Lepidoptera (Laurie 1997).

Complex genetic interactions: We have focused on two-locus incompatibilities as they capture the essence of the Dobzhansky-Muller mechanism and are easily modeled. But hybrid inviability and sterility may be produced by more complex interactions involving three or more loci (e.g., Carvajalet al. 1996). In the appendix, we present an alternative analysis based on three-locus interactions. Although the three-locus results are much more complex than those of our two-locus analysis, our central conclusion remains clear: dominance can explain both Haldane's rule and the large X effect if homozygosity for an incompatible allele has more than twice the effect of heterozygosity on the incompatibility score (see A5).

HYBRIDIZATION DATA

While the above theory makes several predictions, existing data do not yet allow critical tests. Our purpose here therefore is merely to show (1) what predictions can be made; and (2) how these predictions can be tested with hybrid backcross data.

Data from introgressions: A large body of introgression data supports our assumption that H₂ incompatibilities are more severe than H₁. True et al. (1996), for instance—in a study of 87 chromosome regions introgressed from D. mauritiana into D. simulans—found that many introgressions cause complete male or female sterility or inviability when homozygous. Although they made no quantitative measures of heterozygous fitness, the fact that introgressions proceeded through (obviously viable and fertile) heterozygous hybrids suggests that introgression heterozygotes were reasonably fit. (True et al. did not lose a large number of introgression lines, which would have indicated frequent severe heterozygous problems.) Similarly, Hollocher and Wu (1996), in a study of 18 second chromosome introgressions from D. mauritiana into D. simulans, report that none significantly reduces sterility or inviability when heterozygous, although “ … individuals homozygous for these same regions show dramatic increases in both.” More quantitative conclusions can be drawn from the D. buzzatii-D. koepferae (formerly D. serido) work of Naveira and collaborators.

D. buzzatii-D. koepferae: These species obey Haldane's rule for sterility. They have N = 6 chromosomes, with four autosomes and an X, all of roughly equal size, and a tiny sixth. Naveira and Fontdevila (1986) studied male fertility in a large set of X and heterozygous autosomal introgressions, using homology-dependent polytene pairing to assay the size of introgressions. Their chief result is that male fertility is a function of the size of heterozygous autosomal introgressions (Naveira and Maside 1998; see also Marín 1996). Heterozygous introgressions of up to 30% of a large autosome (p_H ≈ 0.3/5 = 0.06) essentially never cause male sterility, whereas introgressions of >40% of a large autosome (p_H ≈ 0.4/5 = 0.08) almost always do. These introgressions correspond to p₁ = 1 − p_H in formula (2) and the expected breakdown scores are given by (18). These autosomal data imply that n(0.056h1+0.004h0)<C (27a) and n(0.073h1+0.006h0)>C. (27b)

In contrast, Naveira and Fontdevila (1986) found that X-linked introgressions of as little as 4% of the X always produce male sterility (corresponding to p₁ ≈ 0.04/5 = 0.008 and p₂ = 1 − p₁ ≈ 0.992). Indeed, they argue that introgressions as small as 1% of the X cause sterility (i.e., p₁ ≈ 0.002). These hemizygous introgressions cause all H₂ incompatibilities, implying that n(0.002)h2>C. (28) Combining (27a) and (28), we get (0.056)h₁ < (0.002)h₂ or h1h2<0.036. (29) This suggests a level of recessivity for hybrid steriles comparable to that of within-species lethals in D. melanogaster (Crow 1993), although the biochemical bases of recessivity may well differ within and between species. The more conservative result that introgressing 4% of the X suffices to produce male sterility implies h₁/h₂ < 0.14. These quantitative results are obviously consistent with our qualitative interpretation of True et al.'s and Hollocher and Wu's introgression data. It should be noted, of course, that Naveira and Fontdevila's results might be complicated by faster-X evolution: we cannot rule out the possibility that a larger number of hybrid male steriles have evolved on the X than comparable-sized autosomal regions.

H₀ vs. H₁ incompatibilities: We have less direct evidence about the magnitudes of h₀ and h₁. However, Coyne et al. (1998) recently found several regions from D. simulans that—when made hemizygous with D. melanogaster deficiencies—cause temperature-dependent inviability of otherwise heterozygous D. melanogaster-D. simulans hybrid females. Though few such regions were found, their existence shows they individually satisfy h₀ ⪡ h₁.

Hybrid backcross analyses: We now turn to traditional backcross/F₂ analyses. Our approach is statistical: we pool all hybrid backcross or F₂ genotypes that have the same p₁, p₂, and p_H. Consider, for instance, a backcross between two Drosophila species possessing five roughly equal-sized chromosome arms, one of which is the X. We pool all data obtained when a single autosomal arm is introgressed, or two autosomal arms are introgressed, etc. We appreciate that experiments repeatedly show that particular chromosomes have large effects on backcross fitness while others do not, but our primary goal is not to explain the detailed outcomes of particular species crosses but to search for statistical regularities.

In a species cross, any of the incompatibilities discussed above might act. While we can make some predictions about the relative roles of X-autosomal vs. X-X incompatibilities, we have no theory allowing us to predict how often, for instance, Y-linked or maternally acting genes might contribute to—or even dominate—postzygotic isolation. Thus we concentrate on cases in which Y and maternal effects are absent or small.

We make one further simplification. The above theory requires that we know p₁, p₂, and p_H. Unfortunately, most backcross studies in Drosophila involve taxa obeying Haldane's rule, so that backcrosses must proceed through F₁ females. Because females recombine, markers remain associated with only some (inexactly known) chromosome region. We do not, therefore, know p₁, p₂, and p_H. This difficulty does not arise when backcrosses are performed through F₁ males, as there is no recombination in Drosophila males. Thus we consider only backcrosses that proceed through F₁ males. We know of two relevant studies: D. mojavensis-D. arizonae (formerly arizonensis) and D. hydei-D. neohydei. We focus on the first for purposes of illustration.

D. mojavensis-D. arizonae: Backcross hybrid males are sterile if they have a Y from D. arizonae and are homozygous for the fourth chromosome of D. mojavensis. These species have five chromosomes of roughly equal size (including the X), and a dot sixth chromosome that we will ignore. To avoid the complications of Y-linked incompatibilities, we focus on the backcross of F₁ males (from D. mojavensis mothers) to D. arizonae females. Table 2 of Zouros et al. (1988) reports sperm motility for these males, all of whom carry an X and Y from D. arizonae. In their Figure 1, Zouros et al. (1988) pool their data into five classes, roughly corresponding to the fraction of the genome that is heterozygous (and ignoring the identity of the heterozygous chromosomes). Their categories correspond to p_H = 0, 0.2, 0.4, 0.6, and 0.8. For all of these males, p₁ = 1 − p_H, so the expected breakdown scores are given by (18). Using their pooled data, we constructed exact 95% confidence intervals, based on the binomial distribution, for the fraction of males with immotile sperm. For two of the five classes (p_H = 0.6, 0.8), all of the males have immotile sperm. In these cases, we constructed an exact 95% lower bound.

The data are given in Table 2 along with the predicted breakdown scores from our two-locus analysis, Equation 18, and the three-locus analysis from the appendix. Because of small samples, there is only one statistically significant jump between adjacent fractions in the table—that from p_H = 0.2 (6/24) to p_H = 0.4 (27/29) (P < 10⁻⁵).

It is worth noting that our large n assumption likely does not hold here. This is suggested by several lines of evidence. First, X_mY_a A_mA_a F₁ males (where A denotes a haploid set of autosomes) are fertile, whereas X_aY_m A_aA_m males are all sterile. As both F₁ males have the same expected breakdown scores, this difference reveals heterogeneity in the numbers of incompatibilities between “replicate” X chromosomes. Second, Zouros and collaborators have shown that hybrid male sterility is caused by X-autosome and autosome-autosome incompatibilities, not Y effects. But the data in Table 2 of Zouros et al. (1988) show statistically significant heterogeneity among the individual chromosomal classes combined in the p_H = 0.2 class—a heterogeneity that is inconsistent with large n.

Despite this, it seems worth asking if the pooled data in the first column of Table 2 agree with our predictions (given in the last two columns), where we assume that higher expected breakdown scores will be associated with greater sperm immotility.

First consider the two-locus predictions. From (18), as p_H increases, the expected breakdown score, E(S), rises to a maximum of nh₁/[4(1 − d₀)] at p_H = 1/[2(1 − d₀)] (see Equation 19), then falls to nh₁(0.16 + 0.64d₀) at p_H = 0.8. Because we ignore Y effects, the expected breakdown score for males with p_H = 0.8 in Table 2 is the same as for the sterile X_aY_m A_aA_m F₁ males. Indeed, both are sterile, as they suffer the same incompatibilities. Note that if d₀ < 2/7, E(S) for p_H = 0.6 is larger than E(S) for p_H = 0.8. As expected, the p_H = 0.6 males are also all sterile. The lack of statistical power precludes more detailed tests, but the data suggest the kind of inferences possible. For instance, if the difference between the fractions of males with motile sperm for p_H = 0.4 and p_H = 0.8 were statistically significant, we could conclude that the corresponding breakdown scores must satisfy 0.24 + 0.16d₀ < 0.16 + 0.64d₀, so that d₀ > ⅙.

Now consider the three-locus predictions. The most interesting difference between the two- and three-locus predictions is that the latter implies that the largest breakdown score occurs for smaller p_H than in the two-locus case. This can be seen in the breakdown scores for p_H = 0.2 vs. p_H = 0.8. In the two-locus model, E(S) is always larger for p_H = 0.8. In contrast, assuming that d₀ = d₁ = d, the three-locus model implies that E(S|p_H = 0.8) < E(S|p_H = 0.2) if d < 0.197. Thus the observation that males from this cross with p_H = 0.8 are less fit than those with p_H = 0.2 suggests that most of the D. mojavensis/arizonae incompatibilities act more like two- than three-locus ones (or that these incompatibilities are less recessive than suggested by our inferences from other data).