## Abstract

Inbreeding depression is expected to play an important but complicated role in evolution. If we are to understand the evolution of inbreeding depression (*i.e*., purging), we need quantitative genetic interpretations of its variation. We introduce an experimental design in which sires are mated to multiple dams, some of which are unrelated to the sire but others are genetically related owing to an arbitrary number of prior generations of selfing or sib-mating. In this way we introduce the concept of “inbreeding depression effect variance,” a parameter more relevant to selection and the purging of inbreeding depression than previous measures. We develop an approach for interpreting the genetic basis of the variation in inbreeding depression by: (1) predicting the variation in inbreeding depression given arbitrary initial genetic variance and (2) estimating genetic variance components given half-sib covariances estimated by our experimental design. As quantitative predictions of selection depend upon understanding genetic variation, our approach reveals the important difference between how inbreeding depression is measured experimentally and how it is viewed by selection.

WHEN populations experience inbreeding, mean fitness tends to decline and the among-line variance in mean fitness increases (Darwin 1877, p. 442; Morton *et al.* 1956; Charlesworth and Charlesworth 1987; Byers and Waller 1999). Inbreeding increases homozygosity, causing a decline in mean fitness owing to the manifestation of recessive deleterious alleles, the reduction in frequency of heterozygotes at overdominant loci, or changes in gene interactions, such as additive-by-dominance and dominance-by-dominance epistasis (Crow and Kimura 1970, pp. 78–80). The among-line variance also increases owing to the segregation among lines of these genetic factors as well as the additive and additive-by-additive epistatic genetic variances (Cockerham and Weir 1968; Goodnight 1988).

Inbreeding depression (ID) is believed to play an important but complicated role in the evolution of mating systems (*e.g.*, Holsinger 1988; Uyenoyama *et al.* 1993). For example, in the evolution of selfing and outcrossing in plants, inbreeding initially selects against selfing lineages by lowering mean fitness. However, if the genes responsible for ID are purged by selection from within or among some lineages, then the selective advantages of selfing may come to outweigh its initial disadvantages. The purging process is complicated because inbreeding has effects not only on mean fitness but also on the within-deme genetic variances and covariances of fitness traits. Thus, purging of ID depends upon the within- and among-line genetic variances.

Variation in ID has been measured as part of studies investigating the relationship between ID and mating system (*e.g.*, Kalisz 1989; Agren and Schemske 1993; Mutikainen and Delph 1998; Takebayashi and Delph 2000; reviewed in Byers and Waller 1999) and between ID and the short-term dynamics of purging (Pray and Goodnight 1995; Fowler and Whitlock 1999). In these latter studies, ID is considered a heritable trait and σ^{2}_{ID} is assumed to represent an opportunity for selection to act upon and reduce inbreeding load. The problem with this perspective is that inbreeding complicates the concept of heritability by changing the nature of the regression of parents on offspring. Indeed, Falconer (1985)(p. 337) states that the concept of breeding value, from which narrow sense heritability is measured, has “no useful meaning when mating is nonrandom.” Although the change in the parent-offspring regression can be predicted for particular cases (*e.g.*, for selfing, see Wright and Cockerham 1986), the variance components affecting selective response in inbred populations are different from those contributing to σ^{2}_{ID}.

The response to selection with inbreeding has been addressed by a large body of theory describing *trait variation among lines* with inbreeding or the within-line covariance among traits with inbreeding (*e.g.*, Cockerham 1971, 1983; Cornelius and Dudley 1976; Cockerham and Weir 1983, 1984; Cockerham and Matzinger 1985; Wright and Cockerham 1986; Cornelius 1988). Kelly (2004) noted that σ^{2}_{ID} includes several within-family covariances, *e.g.*, the covariances between outbred relatives, between outbred and inbred relatives, and between inbred relatives. However, only the covariance between inbred relatives contributes to the among-line variance in the inbred phenotype. This distinction arises because ID is a group-level phenotype, defined as a *difference* between inbred and outbred mean phenotypes. The ID of an individual's progeny depends upon its own genotype as well as that of its mates. As a result, selection among individuals in an inbreeding population may not have a commensurate effect on ID. Differently put, trait evolution with inbreeding is different from the evolution of inbreeding depression.

This is not to say that inbreeding depression cannot evolve. However, a quantitative genetic model for the evolution of ID requires refinements to our current understanding of the genetic basis of ID. First, estimates of ID appropriate for predicting a response to selection must be made. For example, within-line replication of mates is critical for quantifying the genetic variation available for mitigating inbreeding depression. Nevertheless, within-line mate replication is seldom (if ever) employed in practice. Lynch and Walsh (1998)(p. 268) recommend that experimenters consider a replicated nonrandom mating system (*i.e.*, inbreeding) in the context of similarly replicated panmixia: “*A central problem is that inbreeding depression is not just a property of the individual, but of the prospective mates as well … the fitness of progeny from full-sib matings will depend upon which sibs are employed as mates* …” and “… *one would like an estimate of the fitness of outcrossed progeny averaged over all potential mates*…” We argue that replication of mates is essential for assessing evolutionarily relevant variation in inbreeding depression, just as it is in experiments for estimating variation in breeding values. To assess the variation in inbreeding depression, we propose a modification to the classic paternal half-sib design.

Parent-offspring regressions require estimates of components of genetic variance. Thus, if we are to quantitatively understand the evolution of ID we need genetic variance estimates. In this way, the translation of the variation in inbreeding depression into genetic variance components serves two purposes. The first is interpretative: we may use the variation in inbreeding depression as a quantitative tool with which to estimate variance components. The second is predictive: by understanding how genetic variation generates variation in inbreeding depression, we may predict future σ^{2}_{ID} after selection and/or inbreeding.

In this article, we develop a general quantitative genetic framework for evaluating the genetic causes of σ^{2}_{ID}. Because the evolution of mating systems depends upon the trade-off between kinship and ID, the development of this theory is a necessary step in the quantitative study of the evolution of mating systems. Specifically, we introduce an experimental design (Figure 1) for analyzing the genetic components of σ^{2}_{ID}, when inbreeding is caused by either selfing or sib-mating. Our approach is a half-sib design, with sires mated to groups of related and unrelated dams. It permits ID to be defined for each sire as the difference between outbred and inbred dam means and addresses the need identified by Lynch and Walsh (1998)(pp. 268–269) for an ANOVA-based procedure for estimating the expected variation in ID: “… [genetic] *variance in inbreeding depression can* [*presumably*] *be estimated using ANOVA approaches, treating differences between replicate pairs of inbred and outcrossed matings within lineages as the units of observations, but the procedures remain to be worked out*.” We show that σ^{2}_{ID} consists of predictable components of genetic variance and is sensitive to type and duration of inbreeding. The variation in inbreeding depression can be caused by varying *F*, the probability of identity by descent, across lines. As we consider a design with invariant *F*, we do not consider this source of variation here.

## MATERIALS AND METHODS

### The linear model:

Consider *N* independent lines, each containing one sire crossed to *M*_{f} related females and *M*_{o} unrelated and random-mated females. Each cross generates *n*_{o} outbred offspring or *n*_{f} inbred offspring (Figure 1). Let *z _{i}*

_{o}

*represent the value of the phenotype of the*

_{kl}*l*th outbred individual from the cross between the

*i*th sire and the

*k*th

*unrelated*dam and, similarly,

*z*

_{i}_{f}

*represent the value of the phenotype for an inbred offspring from the same sire but the*

_{kl}*k*th

*related*dam. We can decompose the phenotype of each outbred (o) and inbred (f) individual into fixed and random effects, 1where the terms in Equation 1 are defined in Table 1. We define inbreeding depression for some sire

*i*to be the expectation of the phenotypic difference between outbred progeny

*i*o .. and inbred progeny

*i*f .., or in terms of genetic effects, 2Since we are interested in the variation in the differential

*effects*of inbreeding across breeding individuals, we define the variation in

*inbreeding depression effects*(IDE),

*i.e*., those transmitted by sires, to be 3a

Sire effects are defined in part by the hypothetical distribution of paternal gametes across the entire population relevant to that of the inbreeding treatment. For *s*_{o} this population is the entire pool of unrelated individuals but for *s*_{f} this is only those individuals who share a specific type of relationship with the sire. In the case of sib-mating, the sire's gametes are imagined to be combined with an infinite number of full sisters. With selfing, however, the pool of appropriate mates is restricted to the hermaphroditic sire itself. In this case, no meaningful distinction can be made between inbred sire effects and related dam effects (see Equation 4). This is reflected in our definition for variation in inbreeding depression effects with selfing, 3bThe terms σ^{2}(*s*_{f}) and σ^{2}(*d*_{f}) must be distinguished because the latter contains genetic components and special environmental components (maternal effects) not included in the former.

### Adjusted estimator for the variance of inbreeding depression effects:

The variation of inbreeding depression may be estimated by taking, for each sire, the mean outbred phenotype and subtracting the mean inbred phenotype. From these differences, we find the total variation in inbreeding depression across sires. This will *overestimate* the variation in inbreeding depression effects because there will be contributions from dam and individual (error) effects. Replication at the level of dam (related and unrelated) and the offspring (inbred and outbred) within each line provides a statistical means for accounting for the extraneous variance components and allowing for unbiased estimates of σ^{2}_{IDE}. If there are *N* sires, *M*_{o} unrelated dams, *M*_{f} related dams, *n*_{o} outbred offspring, and *n*_{f} inbred offspring, then 4The variance in inbreeding depression effects *with* dam effect variance has been measured in some experiments that do not replicate at the level of the dam (*e.g.*, Pray and Goodnight 1995). This parameter contains variance components not relevant to a response to selection. We remove these components by running two separate analyses of variances (ANOVAs) on the outbred and inbred individuals. Each is a two-factor ANOVA with random dam effects nested within random sire effects. This is different from previous approaches (Johnston and Schoen 1994; Pray and Goodnight 1995) in which inbreeding level has been treated as a fixed effect. This alteration is desirable as Kelly (2004) points out that change in genetic variation with inbreeding will violate the assumptions of homoscedasticity upon which these early ANOVAs depend. Our dual-ANOVA approach makes no such assumption. Note that there are no shared aspects between the outbred and inbred individuals *except* for common alleles transmitted through the shared sires. Because the resulting covariance between outbred and inbred sire effects is a desired component of σ^{2}_{IDE} (Equation 3), no sort of joint ANOVA is more appropriate than the dual-ANOVA approach.

## RESULTS

### Interpretation of variance:

Interpretations of the parameters in Equation 4 are given in terms of genetic components described in Table 2. The variance terms in the ANOVAs can be expressed in terms of covariances between relatives; these are affected by the type and duration of inbreeding. Each of these covariances can be solved independently by Cockerham (1971) for selfing or by Cockerham (1983) for sib-mating. Alternatively, we may synthesize expressions that describe the appropriately weighted sums of these covariances and directly yield the variance components used to estimate σ^{2}_{IDE}. In the appendix, we derive formulas for variation in inbreeding depression effects as well as accessory variances (*i.e.*, dam and error variances) in terms of genetic variance components. These formulas are summarized in Table 3. Variance components σ^{2}_{A}, *D*_{1}, and *D*^{*}_{2} all have nonzero coefficients and contribute to σ^{2}_{IDE}. We show below that *D*_{1} and *D*^{*}_{2} are often the predominant contributors to the variance in inbreeding depression. In Table 4, we summarize the results for the first three generations of inbreeding and for inbreeding to complete homozygosity. This illustrates the rate with which the various components of variance change their contributions to σ^{2}_{IDE} with selfing relative to sib-mating.

### Expected magnitude of the variation in inbreeding depression effects at mutation-selection balance:

If genetic variation is maintained by mutation-selection balance, then the vector describing the components of σ^{2}_{IDE} can be approximated by three elements: σ^{2}_{A}, *D*_{1}, and *D*^{*}_{2}. This follows from Table 2, last column, where we see that these components are roughly proportional to the frequency of the rare allele. In contrast, the remaining two components, σ^{2}_{D} and *H**, are proportional to the square of that frequency and can be ignored. When genetic variation is maintained by the balance between mutation and purifying selection, we expect that *p*_{2} ≪ *p*_{1} when the effects of allele 1 are dominant to the effects of allele 2 (Charlesworth and Charlesworth 1987). To predict σ^{2}_{IDE}, we assume a per-locus mutation rate (μ), selective effect (*s*), and average degree of dominance (*h*). Below, we express results for the various variance components in terms of *additive genetic equivalents*, *i.e.*, in multiples of σ^{2}_{A} that segregates in a random-mated population.

#### Complete dominance (h = 0):

Following Falconer and Mackay (1996)(p. 126) and Table 2 with complete dominance (*i.e.*, *d* = *a*), we have proportional values, 5Although both *D*_{1} and *D*^{*}_{2} > σ^{2}_{A}, *D*^{*}_{2} ≫ *D*_{1} when *p*_{2} is small. If we replace the frequency of the rare allele, *p*_{2}, with expectation in terms of mutational parameters μ and *s*, then we may approximate the variation in inbreeding depression effects by finding the contribution of *D*^{*}_{2}. With one generation of inbreeding, from Equation 5 and Table 4, we find the variation in inbreeding depression effects to be 6Thus, with complete dominance we expect enormous variation in inbreeding depression effects relative to σ^{2}_{A}, nearly all generated by nonadditive genetic variation.

#### Partial dominance:

Following the arguments above, we can approximate the value of variance components for partial dominance with rare alleles: 7With one generation of inbreeding, following Equation 7 and Table 4, the variance in inbreeding depression effects is 8Using *h* = 0.1 (suggested by Lynch and Walsh 1998, p. 286, as representative of viability in Drosophila), we expect σ^{2}_{IDE} to equal ∼6σ^{2}_{A} (six additive genetic variance equivalents) after one generation of selfing and a little >σ^{2}_{A} after one generation of sib-mating. If we reduce dominance (*h* = 0.3), then σ^{2}_{IDE} decreases relative to additive genetic variance (0.694σ^{2}_{A} and 0.236σ^{2}_{A}, respectively) as the sources of nonadditive genetic variation contributing to variation in inbreeding depression effects are lessened. Using Equation 7, we see how the proportional contribution of each type of genetic variance changes with dominance and inbreeding. These proportions are illustrated for the first 10 generations of inbreeding (and complete inbreeding following an infinite number of inbred generations) in Figure 2. With high dominance (*h* = 0.1), additive genetic variation is never an important source of variation in inbreeding depression effects. The relative contributions of σ^{2}_{A}, *D*_{1}, and *D*^{*}_{2} stabilize after a few generations of inbreeding.

### Using variation in inbreeding depression to estimate genetic variance components:

By contrasting the covariances between different sorts of relatives with selfing and sib-mating, it is possible to estimate the genetic parameters that contribute to variation in inbreeding populations (Cornelius and Dudley 1976; Wright and Cockerham 1986). The inbreeding depression design illustrated in Figure 1 provides us with a convenient collection of half-sib covariances with which to perform such an analysis. For sib-mating, we can describe the covariances between four kinds of half-sib relatives [outbred-outbred, σ_{HS}(*O*); outbred-inbred, σ(*OF*); and inbred-inbred half-sibs, σ_{HS}(*F*)] by using Equations A1–A3a and Table A2 (see appendix). These covariances consist of different weightings of genetic variances, so that linear combinations of these covariances can be used to isolate individual variance components. For example, with one generation of sib-mating, half-sib covariances and useful linear combinations thereof are given in Table 5. These can resolve additive genetic variance (by a conventional outbred paternal half-sib comparison), the covariance between additive and dominance deviations in homozygotes, and the dominance contribution toward among-line variance. As this design considers only one degree of inbreeding (at a constant *F*), it cannot decouple dominance parameters *D*^{*}_{2}, σ^{2}_{D}, and *H** (Wright and Cockerham 1986) but in most cases (*i.e.*, rare alleles) this composite estimate is dominated by *D*^{*}_{2}. Maximum-likelihood approaches are typically used to estimate variance components in inbred populations (*e.g.*, Shaw *et al.* 1998; Edwards and Lamkey 2002; Kelly and Arathi 2003) but these analyses are probably not necessary when the inbreeding designs are balanced and not too complex.

## DISCUSSION

By solving for probabilities specific to a given system of inbreeding, variation in inbreeding depression can be described in terms of genetic variance components. These variance components have been dealt with extensively by several authors with varying forms of notation (see Walsh and Lynch 2000, p. 298; Kelly 2004), often in the context of the joint effects of inbreeding and selection. These variance components may be estimated by using linear combinations of the variance and covariance of inbred and outbred paternal half-sib, *i.e.*, sire effects. Because our model is general to the number of inbred generations, the theory is predictive and testable. For example, if genetic variance components were estimated after a single generation of sib-mating, the magnitude of variation in inbreeding depression could be predicted for any subsequent generation.

Usually lineage-specific inbreeding depression is measured on a scale relative to the outbred mean phenotype. For example, Lande and Schemske (1985) measure the *mean* family-level inbreeding phenotype after a period of inbreeding to degree *f*, as ; this is the mean of the individual values of relative inbreeding depression for each family. In contrast, like Pray and Goodnight (1995), we consider inbreeding depression on an absolute scale, , and find that variation in inbreeding depression is caused by the stochastic segregation among sire lines of several genetic effects, including additive genetic variation. Thus, even in the absence of dominance, we can expect ID to vary among inbred lineages. Our analytical finding agrees with that demonstrated in population genetic simulations by Schultz and Willis (1995). However, Schultz and Willis (1995), like many other theoretical studies of mating-system evolution, use the relative measure of inbreeding depression that has been shown by Johnston and Schoen (1994) and Lynch and Walsh (1998)(p. 268) to generate biased estimates of mean inbreeding depression when estimated with limited numbers of offspring. We demonstrate below that even with an infinite number of offspring, the mean and variance of such relative measures of inbreeding depression are likely to be biased.

Consider the outbred and inbred family means, *z̅*_{o} and *z̅*_{f}, and population means, *z̅*_{o̅} and *z̅*_{f̅}. Following the methods of Lynch and Walsh (1998)(p. 818) for finding the expectation of a ratio, 9Given our results above, we can quantify the bias of this estimate for any case of regular inbreeding for the typical experimental study of inbreeding depression in plants conducted without replication of unrelated dams and with a single generation of selfing. In this case, additive genetic variance, dominance variance, and the covariance between additive and homozygous dominance deviations contribute toward the bias in ratio estimates of *mean* inbreeding depression, 10 The estimate of *variance* in relative inbreeding depression is also biased. Following Lynch and Walsh (1998)(p. 818), mean inbreeding depression changes the way the variance is calculated: 11aBy expanding the numerator, we see that changes in population means with inbreeding affect the relative weightings of genetic variance components: 11bBecause of the weights of variance and covariance components on the right-hand side of Equation 11b, it is not likely that the variation in proportional inbreeding depression will closely resemble the variance of absolute inbreeding depression. Again, we reconsider the case of unreplicated dams and a single generation of selfing. From Table 4, we know that variation in inbreeding depression taken on the absolute scale is independent of population means: 12On the relative scale, however, variation in inbreeding depression is confounded by population mean phenotypes (Equation 11b), specifically 13These relative genetic variance component weightings are unbiased only when , *i.e.*, when there is no population inbreeding depression! Thus, the most commonly used experimental design yields biased estimates of both the mean and the variance of inbreeding depression. Furthermore, the estimates contain components of genetic variance irrelevant to the genetic causes of inbreeding depression and will result in nonzero estimates of I̅D̅ even in the absence of net direction dominance. Finally, provided that phenotypes are normally distributed within an inbreeding class (or in the outbred population), we expect that the distribution of inbreeding depression across sires is normally distributed with mean equal to the average of all ID* _{i}* (Equation 2) and variance given in Equation 4. In contrast, Johnston and Schoen (1994) report skew for simulated distributions of inbreeding depression, a result they attribute to “nonlinear functions of self- and outcrossed values.” Our Equations 10 and 13 model this nonlinearity for a single generation of selfing. For these reasons, we argue that relative measures of inbreeding depression are of dubious relevance to evolutionary genetics.

Intuitively, we expect that lineage-level selection would operate on an absolute scale without regard for a relative measure of inbreeding depression. For example, lines that enjoy a fitness advantage when outbred (relative to other outbred lines) *and* an advantage when inbred (relative to other inbred lines) will have high fitness regardless of the lines' proportional inbreeding depression. Alternatively, a line that performs poorly regardless of mating system is unlikely to thrive despite little proportional inbreeding depression. Lineage-specific fitness depends upon the outbred phenotype, the inbred phenotype, and the covariation between the two (weighted by the frequencies of the classes). Although not all absolute measures of lineage-specific inbreeding depression (*e.g.*, Pray and Goodnight 1995) are indicative of lineage-specific fitness, they do have the advantage of providing information from which to draw quantitative genetic inferences as we have shown above. These inferences are important because they allow us to make quantitative predictions regarding the response to selection with inbreeding (Cornelius and Dudley 1976; Cockerham and Matzinger 1985).

Because of the genetic covariance between the outbred and inbred phenotypes, inbreeding depression will evolve as a by-product of natural selection in an outbreeding population, in the same way that hybrid inviability between populations evolves as a by-product of local adaptation within populations. When the genetic covariance is positive, selection within an outbreeding population will mitigate future inbreeding depression. Conversely, the outbred phenotype evolves as a correlated response to selection within inbreeding populations. When the genetic covariance is negative, selection within an inbreeding population could impede a later transition to outbreeding as a correlated response. Because both σ^{2}_{A} and *D*_{1} components exclusively contribute to the covariance between outbred and inbred phenotypes and *D*_{1} is positive with respect to fitness under the deleterious, recessive model of inbreeding depression, we expect the former: lineages that are good outcrossers are good inbreeders.

Although inbreeding depression is often thought of as a trait, it is useful to consider it to be a genetic effect, such as a sire effect. Genetic variation will generate a normal distribution of inbreeding depression effects, just as it will for any effect with a genetic basis and this distribution will have both a mean and a variance. Genetic models that translate the first moment of this distribution (mean inbreeding depression) into mean genetic effects (*e.g.*, additive effects, dominance effects, etc.) have existed for the last half-century (Morton *et al.* 1956; Crow and Kimura 1970; Lynch 1991). These models are useful for understanding *extant* patterns of trait values, such as assessing the relative dominance (or epistatic) contributions toward some trait mean, but, because they do not inform us as to the relationship between genetic variation and inbreeding depression, they are not adequate for predicting response to selection (such as the mitigation of mean inbreeding depression by purging). In contrast, theory that explicitly links genetic variation to selection with inbreeding is well developed (Cornelius and Dudley 1976; Cockerham and Matzinger 1985; Wright and Cockerham 1986). The deficiency in current inbreeding theory is that the observable variation in inbreeding depression and the selectively relevant variation in genetic effects have not been reconciled. In other words, measurements of the variance in inbreeding depression have no quantitatively predictive value in regard to the response to selection with inbreeding. Our ANOVA-based approach for inferring the genetic architecture of inbreeding depression from variation in inbreeding depression bridges the empirically measurable and the theoretically useful.

## APPENDIX

### Sib-mating:

The vector **r** describes the relationship between the most recent common ancestral pair of all related inbred individuals of generation *g* (Figure 1, asterisks) in terms of Jacquard coefficients (Table A1). Note that all related outbred progeny are related to these grandparents through their common sire.

A1This vector summarizes all inbreeding prior to generation *g* − 2. This identity vector serves as a benchmark with which we generate a series of new identity vectors that describe all relationships within and between individuals of generation *g*. Five vectors correspond to siblings pairs: outbred full-sibs, **v**(*O*_{FS}); outbred half-sibs, **v**(*O*_{HS}); half-sibs in which one individual is outbred and the other is inbred, **v**(*OF*); the alternative, **v**(*FO*); inbred full-sibs, **v**(*F*_{FS}); and inbred half-sibs, **v**(*F*_{HS}). In addition, there are two vectors describing the resemblance within individuals: outbred individuals, **v**(*O*), and inbred individuals, **v**(*F*). All identity vectors for generation *g* can be derived by taking the product of **r** and the probability transition matrix **A** specific to the type of relationship (given in Table A2).

Once we have the identity vectors for all siblings and individuals in generation *g* (the various **v** vectors), we may translate them into variance components. For any **v** describing relatives *i* and *j*, the genetic covariance between them is σ(*i*, *j*) = **T · v**(*i*, *j*) − *F _{i}F_{j}H** (Harris 1964; Cockerham 1971; Jacquard 1974), where and A2Note that

*F*≠ 0 if and only if both individuals are inbred. Because all inbred individuals are equally inbred in this design, we may define

_{i}F_{j}*F*such that

*F*

^{2}=

*F*.

_{i}F_{j}We find sire, dam, and error effects variances for outbred and inbred cases by using combinations of these sibling and individual covariances. The sire effect variances are simply the half-sib covariances within outbred and inbred treatments. The covariance between outbred and inbred sire effects is the covariance between outbred and inbred half-sibs. The variation in inbreeding depression effects is a sum of these. Following from Equations 3a and A2, A3aDam effect variances are the differences between full-sib and half-sib covariances. Outbred and inbred dam effect variances are A3bError variance (the within-full-sib family variation) is found by subtracting the full-sib covariance from the covariance of individuals with themselves. For outbred and inbred cases, these are A3cAll effect variances and covariances are simplified and summarized in Table 3.

### Selfing:

Finding inbreeding depression, sire, dam, and error effect variances is simpler with selfing than with sib-mating. In this case, the vector **r** describes the identity shared within the hermaphrodite at generation *g* − 1 that sires both selfed and outcrossed individuals (Figure 1A, individual with two asterisks). This vector has zero probabilities for all identity states except Δ_{1} and Δ_{7}.

A4As with sib-mating, we use this vector to generate identity vectors for all relationships within and between individuals of generation *g*. Because there is only one inbred family per sire, there are no inbred half-sibs. Consequently, we cannot decouple inbred sire and inbred dam effect variance. As indicated in Equation 3b, both terms are included in our definition of the variation in inbreeding depression effects. All identity vectors for generation *g* can be derived by taking the product of the vector **r** and the probability transition matrix **A** specific to the type of relationship and given in Table A3. We find resemblances between relatives (and within individuals) by using Equation A2 and find effect variances by taking sums and differences of these resemblances (as for sib-mating). Effect variances for selfing are simplified and summarized in Table 3. One important difference noted above is that the inbreeding depression effect variance now includes the inbred dam effect variance. For this reason, we use the covariance between inbred full-sibs rather than that between inbred half-sibs: A5

## Acknowledgments

We thank John Kelly, Bruce Walsh, Elizabeth Housworth, and two anonymous reviewers for comments on the article. This work was supported by National Science Foundation grant DEB 0206628.

## Footnotes

Communicating editor: J. B. Walsh

- Received July 13, 2004.
- Accepted April 12, 2005.

- Genetics Society of America