## Abstract

Genetic selection is a major force shaping life on earth. In classical genetic theory, response to selection is the product of the strength of selection and the additive genetic variance in a trait. The additive genetic variance reflects a population’s intrinsic potential to respond to selection. The ordinary additive genetic variance, however, ignores the social organization of life. With social interactions among individuals, individual trait values may depend on genes in others, a phenomenon known as indirect genetic effects. Models accounting for indirect genetic effects, however, lack a general definition of heritable variation. Here I propose a general definition of the heritable variation that determines the potential of a population to respond to selection. This generalizes the concept of heritable variance to any inheritance model and level of organization. The result shows that heritable variance determining potential response to selection is the variance among individuals in the heritable quantity that determines the population mean trait value, rather than the usual additive genetic component of phenotypic variance. It follows, therefore, that heritable variance may exceed phenotypic variance among individuals, which is impossible in classical theory. This work also provides a measure of the utilization of heritable variation for response to selection and integrates two well-known models of maternal genetic effects. The result shows that relatedness between the focal individual and the individuals affecting its fitness is a key determinant of the utilization of heritable variance for response to selection.

GENETIC selection is both a major force shaping life and the principal human tool to improve agricultural populations. In nature, differences in fitness among individuals lead to the evolution of adaptive traits (Haldane 1932; Wright 1937), and in agriculture, breeders improve their populations by selecting the best individuals as parents of the next generation (Lush 1937). Populations respond to selection only if they contain heritable variation, meaning that individuals differ in the effects they transmit to their offspring. In classical quantitative genetics, heritable variation equals the additive genetic variance (Fisher 1918; Haldane 1932; Wright 1937; Robertson 1966; Price 1970). The heritable variation reflects the potential of a population to respond to selection (Robertson 1966; Price 1970; Lande and Arnold 1983), which is important for adaptive evolution in nature and for genetic improvement in agriculture. A clear definition of heritable variation allows one to investigate the mechanisms maintaining heritable variation in nature and to optimize artificial selection schemes in agriculture.

Here I propose a general definition of the heritable variation that determines the potential of a population to respond to genetic selection. Thus, throughout this work, “heritable variation” refers to the quantity that determines the intrinsic potential of a population to respond to genetic selection. The remainder of this Introduction summarizes the classical definition of heritable variation and discusses its limitations. Subsequently, I generalize the definition of heritable variation.

In classical quantitative genetics, heritable variation follows from partitioning individual trait values, *z*, into a heritable component, *A*, and a nonheritable residual, *e*:*A* is the additive genetic value, or breeding value, which is the sum of the average effects of the alleles carried by the individual, including the average contributions arising from dominance and/or epistasis (Fisher 1918; Falconer and Mackay 1996; Lynch and Walsh 1998). The residual, *e*, includes all nonheritable components, which may originate from nonadditive genetic effects and the environment. The heritable variation is defined as the variance of the breeding values among individuals,

Response to selection, *Appendix A*). In evolutionary quantitative genetics, response is expressed as the product of the selection gradient, β, and the additive genetic variance,*i*, to avoid confusion with the index for the focal individual).

In accordance with current belief, Equations 3a and 3b show that additive genetic standard deviation,

The above shows that a variance partitioning perspective (Equations 1 and 2) and a response to selection perspective (Equations 3a and 3b) yield the same result; both identify additive genetic standard deviation as the relevant parameter. In classical theory, therefore, the heritable variation that determines potential response to selection follows from partitioning the phenotypic variance into additive genetic and remaining components. Because additive genetic variance is a component of phenotypic variance, it cannot exceed phenotypic variance (Equation 2). In the classical perspective, therefore, phenotypic variance restricts a population’s potential to respond to selection.

While the classical quantitative genetic model has increased our understanding of inheritance and response to selection tremendously, it overlooks part of the heritable effects that may contribute to response. Specifically, it states that the heritable effects on the focal individual’s trait value originate solely from the focal individual itself, disregarding the social organization of life (Dawkins 1982; Frank 1998; West-Eberhard 2003). It ignores the effects of an individual’s genes on trait values of others, known as indirect genetic or associative effects (Griffing 1967; Kirkpatrick and Lande 1989; Moore *et al.* 1997; Wolf *et al.* 1998; McAdam *et al.* 2002; Muir 2005; Wilson *et al.* 2009; Chenoweth *et al.* 2010; McGlothlin *et al.* 2010). There is growing evidence that such indirect genetic effects are widespread (Frank 2007). Individual fitness, for example, depends on number and quality of offspring, which are affected not only by the genes in the focal individual, but also by those in its mate. Conspecifics often compete for access to mates, making individual fitness dependent on genes in competitors. Many species, moreover, live in groups or colonies, where individual trait values may depend on genes in group mates. Although indirect genetic effects are often associated with behavioral interactions, such as interference competition (*e.g.*, Wilson *et al.* 2009) or social learning, they may also work via the environment through effects on resources or exposure to infectious disease. Not only animals, but also microorganisms and plants exhibit numerous social interactions, both in agriculture and in nature (Crespi 2001; Frank 2007; Griffin *et al.* 2004; Muir 2005; West *et al.* 2006; Karban 2008).

The phenomenon that trait values depend on multiple individuals is not restricted to indirect genetic effects. Some traits cannot be attributed to single individuals, but emerge only at a higher level of organization, such as growth rate of a colony in social insects (Wheeler 1933) or the number of prey caught by a hunting pack. For such cases, current quantitative genetic theory does not provide a measure of heritable variation. Nevertheless, the individuals involved are members of the same population, and response to selection is the result of changes in a single gene pool, which suggests that it should be possible to define heritable variation also for such cases.

An example of an emergent trait in agriculture is the output of a farm, which may depend on different types of individuals. In pig production, for example, the meat produced per sow is the product of the number of offspring of that sow and average meat yield of her offspring (see example below). Hence, the heritable variation in meat yield per sow will depend on both genes for reproduction in the sow and genes for growth rate in her offspring. While meat yield per sow is not a fundamental biological parameter, the heritable variation in this parameter does reflect the potential for genetic improvement of pork production, which is highly relevant to breeders. Moreover, a comparison of realized rates of improvement to the heritable variation provides a measure of efficiency of breeding schemes. Hence, a definition of heritable variation for output parameters of production systems is relevant for breeders.

The importance of social organization is recognized in the biological sciences (Frank 2007), and appropriate quantitative genetic models of inheritance have been developed (Hamilton 1964; Griffing 1981; Moore *et al.* 1997). Genetic models accounting for indirect genetic effects, however, have yielded complex expressions for response to selection, involving multiple genetic variances and covariances (Griffing 1981; Kirkpatrick and Lande 1989; Moore *et al.* 1997; Wolf *et al.* 1998; McAdam *et al.* 2002; Muir 2005; Bijma and Wade 2008; Wilson *et al.* 2009; McGlothlin *et al.* 2010). In contrast to Equations 3a and 3b, those expressions do not partition the response into a parameter describing selection and a parameter reflecting heritable variation. Hence, while those expressions have identified important factors in response to selection, they do not reveal the intrinsic potential of a population to respond to selection or provide a measure of the utilization of that potential.

A more general definition of heritable variation has been proposed for the special case of populations structured into groups of equal size, with a single category of individuals (Bijma *et al.* 2007). The expression for response to selection in Bijma *et al.* (2007), however, does not distinguish heritable variation from selection and does not connect heritable variation to common expressions for response such as Equations 3a and 3b. Ellen *et al.* (2007) show that response in artificial sib selection schemes, with social interactions among pen mates, can be expressed in terms of Equation 3b (see also Wade *et al.* 2010). In none of those cases, however, has heritable variation been connected to the selection gradient expression for response to selection (Equation 3b), and heritable variation has not been defined for emergent traits. Thus there are special cases indicating that a general definition of heritable variation may exist, but it has not yet been identified.

In the following, I propose a general definition of the heritable variation that determines a population’s potential to respond to selection. The result will have the same simple form as the classical expressions (Equations 3a and 3b), separating selection from heritable variation, but will reveal that heritable variation is not restricted by the phenotypic variation among population members. I provide an approach to derive this heritable variation, which can be applied to any level of organization. Application of this approach is illustrated using examples for natural and agricultural populations and is used to integrate two common models of maternal genetic effects.

## Heritable Variation

Individual trait values can always be partitioned into a heritable component and a nonheritable residual, using the method of least squares (Fisher 1918; Lynch and Walsh 1998). The heritable component may arise not only from the focal individual’s own genes, but also from genes in its conspecifics. We can, therefore, represent an individual’s trait value, *i* denotes the focal individual, *j* the individual contributing the heritable effect *k* indexes the different categories of heritable effects underlying individual trait values. The summation is over all individuals affecting the focal individual’s trait value, including *j* = *i*, in which case *k* indicates direct effects and

For example, juvenile growth rate in a mammal may be modeled as the sum of a heritable direct effect of the focal individual’s genes, a heritable maternal effect of its mother’s genes, and a nonheritable residual (Willham 1963). Then, *j* = 1 denotes the focal individual, *j* = 2 its mother, *k* = 1 refers to direct genetic effects, *k* = 2 to maternal genetic effects, and the trait value is given by *Example 4* below for a treatment of maternal effects.)

In general, the *j* = *i*) and the genes in its population members, defined using least squares (Fisher 1918). Each *j* on *k* indexes average effects of genes by category, for example direct *vs.* maternal effects, whereas *j* indexes the individuals carrying the genes. Thus *k* in individual *j* and is expressed in the phenotype of individual *i*. In total, the *i* and the indirect average effects of the genes in all population members that affect *z _{i}*.

Equation 4 does not imply additive gene action. Irrespective of the mode of inheritance, one can always partition the resulting phenotype into heritable effects that are additive by construction, using the method of least squares (Fisher 1918). This is fundamental to quantitative genetics (Lynch and Walsh 1998).

By virtue of least squares, *i.e.*, the change in mean trait value due to a change in gene frequencies, therefore equals*i*. With a maternal effect, for example, *i*, whereas the trait value of *i* depends on

It follows from *H _{i}* reflects an individual’s value for response to selection, it has been called the “total breeding value” by Bijma

*et al*. (2007), and I use that term in the remainder of this article.

*H*is entirely a heritable property of individual

_{i}*i*, irrespective of whether or not

*i*,

*i*does not contribute to

*i*may be a male.) Finally, from the definition of a selection gradient,

While Equations 7a and 7b have been derived for individual trait values, they apply also to traits that cannot be attributed to a single individual, such as the number of prey caught by a hunting pack. In that case, the *i* index is omitted from *z _{i}* in Equation 4, but the

Equation 7a predicts the ultimate response attributable to a selection. When social interactions act across generations, this response may not surface immediately in the next generation. Maternal effects, for example, result in time lags in the response, causing populations to continue evolving after selection ceases and creating dynamic patterns over time (Kirkpatrick and Lande 1989). Equation 7a, however, predicts the permanent response due to the allele frequency change created by a selection, after transient effects have decayed away (see *Example 4*); it is not intended to capture transient effects. Hence, the heritable variance in Equation 7b reflects the potential of a population for a permanent genetic change in trait value due to selection.

The expression for response to selection common in breeding, Equation 3b, can be generalized in the same way. Consider selection for a criterion *x*. Response to selection follows from regressing *H* on *x*, *H* on *x*, and *H* and *x* do not follow a bivariate normal distribution.

Equations 7a and 7c are generalizations of Equations 3a and 3b. They have the same simple form, separating selection from heritable variation, but capture the full heritable variation in a trait, including the component originating from effects of genes on trait values of others. The generality of the derivation suggests that Equation 7a applies to any trait and level of organization. The classical expression appears as a special case, obtained when trait values depend on direct genetic effects only. This work, therefore, shows that the classical expressions for heritable variation and response to selection can be generalized to include traits affected by genes in multiple individuals, which have so far been treated as special cases (Griffing 1981; Kirkpatrick and Lande 1989; Moore *et al.* 1997; Wolf *et al.* 1998; McAdam *et al.* 2002; Muir 2005; Bijma and Wade 2008; Wilson *et al.* 2009; McGlothlin *et al.* 2010).

Equation 7b represents a general definition of the heritable variation that determines a population’s potential to respond to selection,

## Examples

The following examples serve to illustrate the meaning of Equations 7a–7c and demonstrate their application; they are not intended to accurately capture all biological detail of the cases considered. Example 4 also integrates the maternal-effect model of Willham (1963) with that of Falconer (1965) and Kirkpatrick and Lande (1989).

### Example 1—interactions among trees

In Figure 1a, the trait value of focal tree 1 depends on the direct genetic effect of the focal tree itself and the indirect genetic effects of its neighbors, trees 2–5, *k* = *D* refers to direct genetic effects, and *k* = *S* to indirect genetic effects (“social effects”) (after Muir 2005). When trees are unrelated, phenotypic variance equals

The total breeding value of the focal tree equals its total heritable effect, summed over all heritable categories (Figure 1b). Since each tree has four neighbors, this is the direct genetic effect of the focal individual on its own trait value plus its total indirect genetic effect on the trait values of its four neighbors, *H* among individuals, *H* is entirely a genetic property of the focal individual, whereas the focal individual’s trait value depends on multiple individuals. Comparison of

The selection gradient is the regression coefficient of an individual’s relative fitness on its total breeding value,

### Example 2—adult body weight in the African wild dog

The African wild dog (*Lycaon pictus*) is a social carnivore living in packs of approximately seven adults. Only the dominant female usually breeds, while subordinates help to raise the pups. Packs hunt collectively, but the alpha female usually stays behind to guard her pups. The hunters share the prey with the pups and their mother. Adult body weight ranges from 18 to 28 kg, and relatedness among pack members averages ∼0.3 (Creel and Creel 2002).

On the basis of the social organization, an individual’s adult body weight may depend on a direct genetic effect of its own, *j* denotes an adult pack member, *n* adult pack size excluding the alpha female, and *S* indirect effects,

The additive genetic component of phenotypic variance equals

From the genetic mean trait value, it follows that response to selection equals

Estimated genetic parameters for direct, maternal, and indirect effects on adult body weight in the African wild dog are not available. An indication of the difference between *et al.* 2008). Estimates for maternal effects are commonly ∼*et al.* 2002). Using these values, genetic covariances of zero, and a pack size of 7 yields

### Example 3—genetic improvement in pig breeding

Livestock genetic improvement aims to increase the efficiency of food production for human consumption. The prospects for genetic improvement are reflected by the heritable standard deviation in output parameters of agricultural production systems. Consider, for example, a sow line in an integrated pork production system. (In pig breeding, the mothers of fattening pigs come from a breeding line specialized for reproduction traits, known as a sow line, whereas the fathers come from a line specialized for growth traits.) Because farm size is usually measured by the number of sows, interest is in the total amount of meat produced from a sow, which is the product of offspring number (*n*) and offspring meat yield (*y*),

Linearization at the current average trait values, using a first-order bivariate Taylor series, yields*H*-values in the candidates for selection, which measures the quality of the selection criterion.

With an additive genetic relatedness of 0.5 between mother and offspring, the additive genetic component of phenotypic variance equals

### Example 4—maternal effects and time dynamics of response

Kirkpatrick and Lande (1989) showed that maternal inheritance creates time lags in response to selection, causing dynamic response patterns over time. The following illustrates that Equation 7b provides the heritable variance also for such traits. This heritable variation refers to the ultimate response attributable to a selection, excluding transient effects and temporary dynamics. As a second objective, this example integrates the maternal-effect model of Willham (1963) with that Falconer (1965) and Kirkpatrick and Lande (1989).

Following Falconer (1965), Kirkpatrick and Lande (1989) considered a maternal effect that is a simple linear regression on maternal trait value,*i*, *i*, *m* the partial regression coefficient of offspring trait value on maternal trait value, with *i*. As above, the *m* > 0) increases the heritable variation available for response to selection.

Whether *m* > 0 increases the actual response to selection will depend also on the selection process. Falconer (1965) and Kirkpatrick and Lande (1989) considered direct selection on offspring trait value,*t*. The standardized total gradient in generation *t* follows from regression of *m* > 0) increases the strength of direct selection. The denominator of this expression actually equals *i.e.*,

The ultimate response due to selection in generation *t*, excluding transient effects and temporary dynamics, equals *m* > 0 increases response to direct selection, which is partly due to increased heritable variation and partly due to increased strength of selection. This expression is identical to the result of repeatedly applying Equation 3 of Kirkpatrick and Lande (1989) to remove transient effects (*Appendix B*). Figure 2 illustrates that the response of a single selection in generation *t* asymptotes to this value, irrespective of selection in later generations.

#### The relationship between Willham’s and Falconer’s models:

In the main results of this work, I have used Willham’s (1963) maternal-effects model to illustrate the meaning of expressions. Willham’s model ignores the time dynamics of response, but predicts the same asymptotic response as Falconer’s (1965) model. Both models are well known and widely used (Lynch and Walsh 1998) and are related as follows (using W63 and F65 to indicate parameters referring to Willham’s model and Falconer’s model, respectively):*m*, Falconer’s model corresponds to a direct-maternal genetic correlation of either +1 or –1 in Willham’s model. When individual fitness is determined by individual trait value,

## Utilization of Heritable Variation

As shown above, the standardized selection gradient of relative fitness on total breeding value,

### Natural selection

Consider a population structured into a large number of groups of *n* individuals each, where indirect genetic effects of group mates affect individual trait values,*Example 1*. Moreover, suppose that selection is a function of individual trait value and the summed trait values of all *n* – 1 group mates,*g* represents the degree of between-group selection relative to individual selection; it is the ratio of the selection gradient on the summed trait values of group mates over the selection gradient on individual trait value. A *g* = 0 represents individual selection, and a *g* = 1 full between-group selection (Bijma and Wade 2008). Combining Equation 15 of Bijma and Wade (2008) with Equation 7a, using *r* to denote additive genetic relatedness between group members, yields the following expressions for the standardized total selection gradient. For individual selection with unrelated group members (*g*, *r* = 0),*g* = 0) with related group members, *r* = 0),

The effects of relatedness and/or multilevel selection on the utilization of heritable variation follow from the partial derivatives of *g* or *r*. For example, the partial derivative of Equation 8b with respect to *r* equals*n* = 4,

Because Equation 8d is symmetric with respect to *g* and *r*, the effect of between-group selection on

### Artificial selection

With artificial selection, the accuracy reflects the utilization of heritable variation by the breeder. (The intensity of selection merely reflects the overall strength of selection.) The above case may be investigated in the context of artificial selection by replacing fitness by a selection index of its own trait value and the summed trait values of group mates,*Appendix C*).

Figure 3 shows that accuracy increases much more with relatedness than with between-group selection. This occurs because an increase in *g* strongly increases the

### Generalization

The above considered the special case where population members interact within groups. The following investigates whether general statements about the utilization of heritable variance can be made.

Since any trait can be decomposed into additive effects using the method of least squares, this decomposition can be applied also to fitness,*j*′ denotes the individual contributing the *j*th heritable effect, *l* indexes the different categories of heritable effects on fitness, and the summation is over all individuals affecting the focal individual’s fitness, including the focal individual itself. (For multilevel selection in group-structured populations, the relationship between the elements of *w _{i}* and the selection parameters is given in Equation 11 and Table 2 of Bijma and Wade 2008). Because heritable effects on fitness may have a different origin from those on trait values, the indexing in the fitness model,

*j*′ and

*l*, differs from that in the trait model (

*j*and

*k*, Equation 4). From the Robertson–Price theorem, response to selection equals

*j*′, and

*i.e.*, within individuals) between the

*k*th heritable component of the trait value and the

*l*th heritable component of fitness.

Equation 10 reveals two points of interest. First, response to selection depends on relatedness between the focal individual and the individuals affecting its fitness, *l*th fitness component is positive when *l*th fitness component is negative when *et al.* 2010).] Hence, this result shows that relatedness works to change trait values in the direction of increased fitness, suggesting that relatedness between the focal individual and the individuals affecting its fitness causes an adaptive response to selection. This result agrees with the observation of Bijma (2010), who showed that relatedness contributes to a positive response in fitness when individuals interact.

## Discussion

In this work a definition has been proposed of the heritable variance that determines the potential of a population to respond to selection. In this definition, heritable variance equals the variance among individuals in the heritable quantity that determines the mean trait value of the population, rather than the additive genetic component of phenotypic variance. This definition encompasses both traits affected by the focal individual’s genes only, in which case heritable variance equals the ordinary additive genetic variance and traits depending on heritable effects originating from multiple individuals. This work, therefore, generalizes the classical definition of heritable variance and the usual quantitative genetic expressions for response to selection to cases where trait values depend on genes in multiple individuals.

Because individuals transmit the genes they carry themselves, the heritable variance relevant for response to selection may differ from the additive genetic component of phenotypic variance, which may originate in part from genes in others. As a consequence, the heritable variance in traits that depend on genes in multiple individuals is not limited to phenotypic variance, which is a fundamental difference from classical theory (Fisher 1918; Lynch and Walsh 1998). For such traits, heritable variance may exceed the phenotypic variance among population members and has no theoretical upper bound. This result implies that social organization may allow populations to evolve faster by natural or artificial selection.

The partitioning of response to selection into contributions from heritable variation and selection facilitates research aiming to identify the mechanisms that determine the utilization of heritable variation by natural or artificial selection. The standardized total selection gradient,

A partitioning of response into contributions from heritable variation and selection is also useful for breeders. Breeders want to know how much genetic improvement is possible in principle and to be able to assess the quality of their breeding programs. The heritable variation, as defined in Equation 7b, reflects the genetic improvement that is possible in principle. An efficient breeding scheme generates ∼1 unit

The definition of heritable variance provided here also explains in a natural way why certain heritable traits cannot respond to selection. Consider, for example, the rank of racing horses. The mean rank cannot respond to selection, because it is fully determined by the number of competitors. With eight competitors, for example, the mean rank is always 4.5. Racing ability nevertheless shows additive genetic variance (Langlois 1980). While it is obvious that rank cannot respond to selection, this case violates the ordinary quantitative genetic expression for response to selection (Equation 3). Equation 7b, however, reveals that heritable variance in rank is zero. This occurs because a 1-unit increase in an individual’s rank always decreases the average rank of its *n* − 1 competitors by an exact amount of *et al.* (2011) applied a similar approach to genetically analyze dyadic interactions in Scottish deer. This approach may also be used to genetically analyze fitness in populations where mean fitness cannot respond to selection because population size is limited by the carrying capacity of the environment, which is very common.

## Acknowledgments

I thank J. Bruce Walsh, Antti Kause, Duur K. Aanen, Johan A. M. van Arendonk, Martien A. M. Groenen, Michael Grossman, James H. Hunt, Hans Komen, Ole Madsen, Arie J. van Noordwijk, Elisabeth H. van der Waaij, and Michael J. Wade for reviewing drafts of this manuscript. This research was financially supported by the Dutch Science Council and was coordinated by the Netherlands Technology Foundation.

## Appendix A

This Appendix shows that the common expressions for response to selection used in evolutionary genetics and artificial breeding are equivalent under multivariate normality (Equations 3a and 3b). Consider artificial selection for a criterion, say *x*, so that individual fitness is determined entirely by individual *x*. Thus the effect of breeding value for the trait on individual fitness must arise entirely via *x*. Moreover, under multivariate normality, regressions are linear and represent conditional expectations (Stuard and Ord 2004). Thus the regression coefficient of fitness on breeding value,

## Appendix B

This Appendix shows that the expression for response to selection derived in *Example 4* is identical to the response from repeatedly applying Equation 3 of Kirkpatrick and Lande (1989),

The last term in this expression is the phenotypic effect of the selection differential in the mothers on the trait value of their offspring and is therefore transient. The second term is partly transient because it includes *t*, *t* and a population where selection ceases after generation *t* – 1,*t* is*Example 4*.

## Appendix C

This Appendix provides background information on Figure 3. Direct and indirect additive genetic (co)variances follow from

- Received May 12, 2011.
- Accepted September 6, 2011.

- Copyright © 2011 by the Genetics Society of America