Abstract
The replication control genes of bacterial plasmids face selection at two conflicting levels. Plasmid copies that systematically overreplicate relative to their cell mates have a higher chance of fixing in descendant cells, but these cells typically have a lower chance of fixing in the population. Apart from identifying the conflict, this mathematical discussion characterizes the efficiency of the selection levels and suggests how they drive the evolution of kinetic mechanisms. In particular it is hypothesized that: (1) tighter replication control is more vulnerable to selfishness; (2) cisacting replication activators are relics of a conflict where a plasmid outreplicated its intracellular competitors by monopolizing activators; (3) highcopy plasmids with sloppy replication control arise because intracellular selection favors overreplication, thereby relieving intercellular selection for lower loss rates; (4) the excessive synthesis of cisacting replication activators and transacting inhibitors is the result of an arms race between cis selfishness and trans retaliations; (5) sitespecific recombination of plasmid dimers is equivalent to selfpolicing; and (6) plasmids modify their horizontal transfer to spread without promoting selfishness. It is also discussed how replication control may be subject to a third level of selection acting on the entire population of plasmidcontaining cells.
PLASMIDS are selfreplicating gene clusters commonly found in the cytoplasm of prokaryotes. They are widely used as cloning vectors but also serve as model systems for replication control (Summers 1996; Paulsson and Ehrenberg 2001) and microbial ecology (Mackenet al. 1994; Stewart and Levin 1977; Bergstromet al. 2000). By connecting molecular and ecological aspects, it is shown here how plasmid replication faces selection at two levels. Plasmids that outreplicate their intracellular competition stand higher chances of fixing in the descendant cells, but these cells tend to grow more slowly due to the larger plasmid burden. Previous analyses have dealt with multileveled selection on various plasmidcarried genes (Eberhard 1990; Mongold 1992; Pomiankowski 1999; Bergstromet al. 2000; Cooper and Heinemann 2000), but not with those that affect replication. Other analyses have inspected the interplay between similar selective mechanisms involved in the reproduction of viruses (Chao 1991, 1994; Szathmary 1992; Bonhoefffer and Nowak 1994; Turner and Chao 1999), prebiotic replicators (Szathmary and Demeter 1987; MaynardSmith and Szathmary 1995), and cytoplasmic organelles (Hurstet al. 1996), but do not deal with plasmidspecific traits. The main purposes of the present work are therefore to identify the selective forces acting on plasmid replication control genes and conjecture their molecular consequences. The two selection levels are first analyzed separately and then combined in the study of selection conflicts and conflict suppression. The analysis concludes by discussing lineage selection acting on the entire population of plasmidcontaining cells.
PLASMID REPLICATION CONTROL
If plasmids maximally exploited their hosts, over and underreplication below and above the carrying capacity of the cytoplasm would automatically check random fluctuations around an average copy number. Natural plasmids instead define their own carrying capacity by encoding functions for autoregulating the initiation of replication. By decreasing the average copy number, replication control reduces the metabolic burden imposed on the host, and by suppressing the demographic noise around the average, it additionally reduces the loss probability at cell division.
Kinetics of replication control: Many plasmids, including R1 and ColE1 (Summers 1996), regulate initiation of replication kinetically using plasmidencoded activators and inhibitors. Activators can interact in cis or trans, depending on plasmid, with the origin of replication to attract DNA polymerase and initiate replication, while inhibitors act in trans to disrupt the activator production line. (Cisacting molecules interact only with the plasmid copy from which they were produced, while transacting molecules can interact with any plasmid copy.) The simplest kinetic models are based on the singlerate equation
A commonly used kinetic approximation for inhibition mechanisms is the negative Hill function:
Replication control checks random fluctuations: Since chemical reactions are probabilistic by nature, the plasmid copy number m varies randomly from cell to cell. Such chemical noise can be modeled using master equations (Paulsson and Ehrenberg 2001) but one of its most important determinants is already evident in the deterministic equations. This can be illustrated by approximating cell growth and plasmid segregation with an intrinsic plasmid halflife and by using r (where y is replaced by m divided by the cell volume) as a birth intensity per copy (Paulsson and Ehrenberg 2001). The total birth and death probabilities within a short time interval Δt are then approximately rmΔt and μmΔt, respectively, and the linear noise approximation (van Kampen 1992) predicts a stationary Gaussian distribution with variance
Random fluctuations increase the average loss rate: The probability that all plasmid copies segregate to the same daughter at cell division depends on the copy number and the type of plasmid partitioning. With a perfectly working partition function losses occur only when m = 1. If copies instead segregate independently to identical daughters—binomial partitioning—the loss probability is L^{m}= 2(½)^{m}. Since cells with lower m run disproportionally higher risks of giving rise to plasmidfree progeny, random fluctuations around an average 〈m〉 increase the average loss probability 〈L〉 (the Jensen inequality guarantees that 〈L〉 ≥ L_{〈}_{m}_{〉} since L is convex in m). In fact, many naturally arising copy number distributions allow for the approximation 〈L〉≈ 2b^{〈}^{m}^{〉}, where ½ ≤ b ≤ 1. Higher b reflects broader distributions and greater loss rates. For Poisson distributions b ≈ e^{–1/2} ≈ 0.6 and for the Gaussians of Equation 4, b decreases with i(1 –μ/k) (appendix). This summarizes the established perspective on plasmid replication control: Tighter negative feedback more effectively suppresses random fluctuations and thereby increases segregational stability for a given average copy number (Figure 1A).
INTRACELLULAR SELECTION
Plasmids with too similar replication control systems are unable to stably coexist in heteroplasmid cells; they are incompatible (for an excellent review see Novick 1987). Common causes of incompatibility are susceptibility to each other's replication inhibitors or competition for ratelimiting activators. Mutations that affect these properties could thus allow their plasmids to systematically over or underreplicate relative to their cell mates: Replication control is subject to intracellular selection.
Cis and trans: acquiring activators and avoiding inhibitors: Replication control allows a plasmid copy to kinetically communicate its presence to the other copies in the cell and set its own replication frequency according to the total plasmid concentration. Consequently, some mutations affect kinetic properties that are public to all copies (trans mutations) while others act on properties that are kept private to the mutant copies (cis mutations). Trans mutations are neutral (appendix) in terms of intracellular selection since all copies are affected the same way (Novick 1987; Szathmary 1992). Cis mutations that allow a plasmid copy to overreplicate relative to its cell mates in contrast have an intracellular advantage (Novick 1987). For instance, plasmids such as pT181 that share activators in trans (Novick 1987) have cisbinding sites under intracellular selection for high activator affinity because a copy that attracts more activators will replicate more frequently. Changes in the structure or turnover rates of activators or inhibitors by contrast affect all copies equally. Plasmids such as R1 and ColE1 that keep their activators in cis are instead under selection for high activator synthesis rates. Their activator genes are right next to the origin of replication. The RNA that promotes replication of ColE1 is still physically attached to its gene when it binds and forms a replication complex at the origin. For R1, the mRNA of the replication protein is also attached to DNA and the protein never leaves the plasmid copy from which it was made. Mutations affecting the structure or turnover rates of trans inhibitors are still neutral, but their RNA or DNA targets are selected for lower inhibitor affinity.
A generalization of the approximation in Equation 2 for incompatible Y_{1} and Y_{2} plasmids in heteroplasmid cells—tailormade for the molecular processes above—is
Incompatibility and genetic drift: If two types of organisms exploit the same niche in the same way, the carrying capacity of the environment checks only fluctuations in their total number. Fluctuations in their individual numbers instead stand uncorrected and random drift quickly drives one or the other to fixation. In direct analogy, replication control in heteroplasmid cells acts on the weighted sum of plasmid copy numbers rather than the two separately. The inability to sense and correct individual fluctuations leads to greatly increased losses; i.e., heteroplasmid cells give rise to homoplasmid segregants at a much higher rate than homoplasmid cells give rise to plasmidfree segregants (Novick 1987).
The average fraction of homoplasmiddescendant cells in which a plasmid copy eventually is fixed can be estimated by replacing cell growth and plasmid segregation by plasmid elimination intensities μm_{1} and μm_{2} (Paulsson and Ehrenberg 2001) and by assuming birth intensities r_{1}m_{1} and r_{2}m_{2}. With a constant total copy number m_{T} = m_{1} + m_{2}, the effective singlecopy substitution rates equal the elimination intensity of one type multiplied by the probability that the other type replicates first. The ratio between singlecopy substitution rates is then r_{2}/r_{1} (appendix), which uniquely determines fixation fractions. This is equivalent to a Moran (1958) model of selection and drift in a haploid population and has been used by Walsh (1992) to predict fixation rates of organelle genes. If Y_{1} and Y_{2} differ by a transacting mutation, r_{2}/r_{1} = 1 and all copies have the same chance of fixing (appendix). For cis mutations fixation fractions are harder to calculate, but when r_{2}/r_{1} is independent of m_{1} and m_{2}, as in Equation 7, their ratio (appendix) is the standard
INTERCELLULAR SELECTION
Plasmids depend entirely on their hosts for reproduction and are thus under selection to maximize the net growth rate of plasmidcontaining cells. Since copy numbers vary statistically from cell to cell it may seem that individual cells have individual fitnesses. However, copy number fluctuations are both epigenetic and transient. Selection therefore effectively acts on the net growth rate accumulated over a few generations, i.e., on the distribution associated with a replication control mechanism rather than on individual fluctuations.
Net growth and genetic drift: Much of the analysis is simplified to inspect the competition between homoplasmid X_{1} and X_{2} cells, containing plasmids Y_{1} and Y_{2}, respectively. Plasmids are thus considered essential to their hosts and arising heteroplasmid cells are assumed to immediately turn into homoplasmid cells with probabilities that are included in the effective mutation and conjugation rates below. This is approximate since separation of plasmids requires cell divisions, but it is sufficient for the current purposes. Over evolutionary time one should also expect an accumulation of competing cell types, not just X_{1} and X_{2}, but this simplification makes it possible to analytically demonstrate some first principles.
Most ecological plasmid models (Stewart and Levin 1977; Mackenet al. 1994; Bergstromet al. 2000) condense growth, losses, and horizontal transfer into a deterministic and continuous rate equation approximation for changes in cell densities. In close analogy, X_{1} and X_{2} densities are modeled by
The steadystate densities of Equation 9 (appendix) are equal when the difference in mutation rates balances differences in losses, growth, and horizontal transfer:
Deterministic models are practical when all cell types exist in high numbers, but since X_{1} and X_{2} cells in Equation 9 coexist only due to mutations, stochastic descriptions are more appropriate. With the same notations and assumptions as in the deterministic analysis (appendix), where n is used for numbers instead of x for densities, assume that singlecell substitutions occur as a result of conjugation between cells of different types or of birth of one cell multiplied by the probability that a cell of the other type is eliminated [a Moran (1958) model with migration]. The ratio between fixation probabilities is then
Plasmid burdens: To compete with both plasmidfree and plasmidcontaining cells, plasmids are constantly under intercellular selection to reduce metabolic burdens while also considering loss rates and conjugation frequencies. Burdens depend strongly on copy numbers, gene expression levels, environmental conditions, and the history of plasmidhost coevolution. In spite of such contingency, a brief account of phenomenological features helps put the present analysis in perspective.
Because the low losses at high copy numbers do not compensate for the high losses at low copy numbers, the average loss rate 〈L〉 increases with fluctuations around an average 〈m〉 (see plasmid replication control). This argument has permeated the plasmid literature, yet similar questions are never raised for burdens: Do copy fluctuations have a significant impact on the average host growth rate? There are two scenarios where they should not. First, if the burden responds more or less linearly to fluctuations in copy number, the effect of upfluctuations cancels the effect of downfluctuations. Second, if there is a long phenotypic lag before a change in copy number affects growth, cells effectively integrate over plasmid fluctuations, sensing mainly the average. By contrast, if the growth rate quickly and nonlinearly responds to plasmid fluctuations, one should expect fluctuations to also affect the average burden. For instance, if a high growth rate requires that m is above or below a certain threshold, then plasmids with 〈m〉 on the right side of the threshold are under selection for narrow distributions, while plasmids with 〈m〉 on the wrong side are under selection for a different 〈m〉 or broader distributions. Similarly, if the burden were proportional to m^{2}, the average burden would be proportional to
Because it is speculative if or how copy fluctuations affect growth, most of the analysis does not rely on detailed assumptions. In some quantitative examples, however, it is assumed that
A tradeoff between burdens and losses: An increase in average copy number generally increases the burden that plasmids impose on their hosts but instead reduces their loss rate. There is thus a tradeoff between the two disadvantages and presumably an optimal average copy number that maximizes the net growth rate of the plasmidcontaining cell. For instance, if 〈L〉 = 2b^{〈}^{m}^{〉} (see plasmid replication control) and μ=μ_{0}(1 – B〈m〉) (see above), then (1 – 〈L〉)μ as a function of 〈m〉 has an internal maximum at 〈m〉_{opt} (appendix). At higher 〈m〉, metabolic burdens are too large, and at lower 〈m〉, plasmid losses are too high (Figure 1B). Narrower distributions (lower b) similarly come at the price of higher burdens (Paulsson and Ehrenberg 2001) but this analysis focuses on average copy numbers.
SELECTION CONFLICTS
Intracellular selection favors replication control systems that allow their plasmids to outreplicate other plasmids. Intercellular selection instead favors control systems that allow their cells to outgrow competing cell types. This section compares the relative strengths of the two forces and predicts to what extent selfishness can promote an increase in average copy numbers. It is also proposed how the conflict can cause neutrality to random copy number fluctuations and explain the existence of cis activators.
The effective level of selection: The fate of plasmidcontaining cells depends on selection at two levels: Intercellular selection operates on plasmid burdens, loss rates, and conjugation frequencies, while intracellular selection determines the fraction of descendant cells that are finally affected by a mutation or conjugation event. If heteroplasmid cells arise with the same mutation rate ω_{0} per plasmid copy, the effective rates per cell of forming homoplasmid descendants of the other type are ω=ω_{0}m_{T} f^{intra} (Equation 8). Similarly, if the two plasmids have identical conjugation mechanisms, the effective conjugation rates are ∼Γ=Γ_{0}f^{intra}.
By combining the expressions for intra and intercellular genetic drift when mutations are rare and by assuming low rates of conjugation and plasmid losses as well as small differences in cell growth rates—typical in vivo parameter values—selfish plasmids are predicted to reign with higher probability than altruistic plasmids (appendix) approximately when

Each new group should be founded by members from few other groups.

The number of groups should be high compared to the number of individuals per group (n_{T} ≫ m_{T}).

Transfer between groups should be low (Γ_{0} ≪ μ).
Since a daughter cell has a single mother, the first requirement is automatically fulfilled. The number of copies per cell is also fairly low, ranging from a few to at most a few hundred, while the number of cells per population can be very high. Finally, conjugation rates tend to be low and some plasmids actively avoid forming heteroplasmid cells with incompatible relatives (see suppressing conflicts). From this one might expect intercellular selection to overrule intracellular selection and plasmids to live in reasonable harmony with the plasmidcontaining cell. However, counteracting these effects, simple mutations can result in great intracellular advantages while the differences in losses and metabolic burdens typically are very small. Intracellular selection thus operates with small populations but large selection coefficients while intercellular selection operates with large populations but small selection coefficients.
More cells in a given volume imply more encounters and thus more transfer. This is taken into account in the above analysis because the transfer rate is assumed to be γ_{0}n_{1}n_{2} (see intercellular selection) where Γ_{0} in Equation 13 is defined by Γ_{0} =γ_{0}n_{T}. The second term in the righthand side of Equation 13 thus increases with n_{T} and the total righthand side has a minimum at the cell population size for which plasmid selfishness is most efficiently suppressed:
Sensitivity of replication control and selfish deviations from optimality: By favoring overreplicating plasmids, intracellular selection promotes a selfish increase in the average copy number. How large deviations Δ〈m〉 from 〈m〉_{opt} one should expect depends on how the two selective forces respond to changes in 〈m〉.
At the intracellular level, consider the idealized case where plasmids replicate as soon as their concentration decreases below a threshold value, but never when above. Volume expansion due to cell growth continually dilutes plasmids, and when the threshold concentration is reached, a plasmid copy replicates. This raises the inhibitor concentration and blocks further replication attempts. Consequently, if Y_{2} plasmids due to a cis mutation have a slightly higher threshold than Y_{1} plasmids, only Y_{2} plasmids can ever replicate. Realistic control mechanisms would give only a partial advantage to Y_{1} or Y_{2} plasmids but with higher sensitivity one approaches the threshold situation
At the intercellular level, selection favors cells that better balance metabolic burdens [μ ≈ μ_{0}(1 – B〈m〉)] and plasmid losses (〈L〉≈ 2b^{〈}^{m}^{〉}). If X_{1} cells have an optimal tradeoff as outlined in intercellular selection, while Y_{2} plasmids deviate Δ〈m〉 above 〈m〉_{opt},X_{2} cells are disadvantaged (Figure 1B) by intercellular selection and a secondorder Taylor expansion (appendix) around 〈m〉_{opt} gives
An estimate of the balance between the selective forces can be found by using the expressions for r, μ, and 〈L〉 directly in the genetic drift equations. At the price of less generality, more transparent results can also be obtained by using the approximations in Equations 13, 14, 15, 16 that predict the selfish plasmid to be at a net advantage as long as it is not too selfish, i.e., when
Does the selection conflict generate noisy plasmids? A parasitic increase in the average copy number typically leads to lower loss rates and higher metabolic burdens. As a consequence, the selective pressure for even lower loss rates is relieved while the selection on burdens intensifies. If the only effect of random fluctuations is to increase the loss rate—as is commonly assumed (see intercellular selection)—parasitically high averages should thus result in selective neutrality to noise suppression and efficiency of replication control. In other words, even if low average copy numbers and effective control would allow for the most costefficient plasmidcontaining cells, multileveled selection could instead result in plasmids with high averages but broad distributions. For a quantitative example, again consider 〈L〉≈ 2b^{〈}^{m}^{〉} and μ/μ_{0} ≈ 1 – B〈m〉. If 〈m〉_{1} ≫ 〈m〉_{opt}, then the burden is relatively high and the loss rate is relatively low (Figure 1B). For a competing Y_{2} plasmid with 〈m〉_{2} = 〈m〉_{1} but broader (b_{2} > b_{1}) copy number distribution, 〈L〉_{2} – 〈L〉_{1} could be insignificant even if 〈L〉_{2}/〈L〉_{1} is very high (Figure 1B), as when 〈L〉_{1} = 10^{–10} and 〈L〉_{2} = 10^{–8}.
At the heart of this argument is the assumption that average loss rates increase with random fluctuations while average metabolic burdens do not. However, if loss rates are very low due to plasmid selfishness, and fluctuations indeed increase the burden (see intercellular selection), lowering the burden could in fact be the primary role of noise suppression. Replication control would then not be balancing losses against burdens, but burdens against selfishness.
Cis activators—relics of selfishness? For R1, ColE1, and similar plasmids, both cis and trans activators could result in constitutive attempts to initiate replication. The only apparent regulatory difference is a short time delay when activators reside in the cytoplasm before binding to plasmids. However, a plasmid that starts to monopolize its activator molecules—forcing them to act in cis— also receives a great intracellular advantage over its cell mates. If the fraction of activators made from Y_{1} and Y_{2} copies are m_{1}/m_{T} and m_{2}/m_{T}, and Y_{2} copies keep their activators in cis but tap into the common pool of trans activators as effectively as the Y_{1} copies, Y_{1} and Y_{2} plasmids take fractions m^{2}_{1}/m^{2}_{T} and m_{1}m_{2}/m^{2}_{T} + m_{2}/m_{T}, respectively. For ColE1 (Brenner and Tomizawa 1991; Paulsson and Ehrenberg 2001) and R1 (Nordström and Wagner 1994; Paulsson and Ehrenberg 2001), the rate of acquiring activators is proportional to the momentary plasmid replication frequency so that r_{2}/r_{1} = 2 + m_{2}/m_{1}. Equation 13 cannot be used directly for the balance between the selective forces because r_{2}/r_{1} depends on m_{1} and m_{2}, but the fixation fractions are still analytically tractable (appendix) and the cis fixation advantage is
Selfish changes in replication control should often be expected to reduce the fitness of the plasmidcontaining cell. However, cis action does not necessarily affect the copy number distribution in the subsequent homoplasmid cells at all. Activators still have the same structure and are synthesized at the same rate; they are only allocated earlier. Parameter Δ〈L〉 –Δμ/μ in Equation 13 could thus be very low or even negative. In other words, the strong intracellular selective force to privatize activators is opposed by a weak—if any—force at the intercellular level. This may explain why cis activators are so popular in replication control, like RepA of R1 and RNA II of ColE1, but at the same time raises the question how plasmids like pT181 can share their RepC activators in trans.
SUPPRESSING CONFLICTS
Conflicts between levels of selection provide niches for suppression mechanisms that protect higherlevel units from lowerlevel selfishness. As demonstrated by, e.g., tumor suppressor genes, the actual conflict can then be insignificant compared to the potential conflict. This section discusses three types of mechanisms for suppressing intracellular selfishness: trans retaliations to lower the average copy number without suffering an intracellular penalty, discriminatory conjugation for effective horizontal transfer without mixing related plasmids, and sitespecific recombination to resolve overreplicating plasmid multimers.
Retaliations in trans: The previous chapter treated the selection balance between two plasmid types in the hypothetical absence of other types. However, rather than ending in a static compromise between selection levels, conflicts can lead to an innovative evolutionary game of moves and countermoves. In particular, selfish deviations toward higher Q_{cis} and 〈m〉 > 〈m〉_{opt} (Equations 6 and 7) would not necessarily be succeeded by a revertant to lower Q_{cis}, but more likely to higher Q_{trans} that can reduce 〈m〉 back toward 〈m〉_{opt} without suffering an intracellular disadvantage. The interplay between the two levels of selection can thus lead to an arms race between cis selfishness and trans retaliations (Figure 2). For instance, the inhibitor target sites are under intracellular selection to avoid inhibitors, but lowaffinity targets provide intercellular selection for more potent inhibitors, amounting to an evolutionary game of hideandseek. Similarly, the arms race may result in high synthesis rates of both the cis activators and the trans inhibitors, something that has been observed for plasmids ColE1, R1, and numerous other plasmids. At some point the race slows down by the metabolic burden associated with overproducing inhibitors and activators (an aspect of intercellular selection that is ignored above) or by entropic effects when most mutations lead to lower promoter activities. Chromosomal mutations typically affect all plasmid copies in the cell and thus take the role of trans mutations.
Safe sex: The evolutionary success of plasmids depends directly on conjugation—sex between prokaryotes—whereby plasmids transfer horizontally to new cells or even new types of cells (Stewart and Levin 1977; Mackenet al. 1994; Bergstromet al. 2000). However, as can be seen in Equations 13 and 17, conjugation that mixes incompatible plasmids also promotes selfishness, especially in large cell populations. Since selfishness in turn reduces the growth rate of plasmidcontaining cells, plasmids could benefit in the long run by conjugating discriminatorily to cells that are free of incompatible relatives.
Many plasmids avoid redundant conjugation by encoding mechanisms for surface exclusion (Summers 1996) that prevent plasmids from the same exclusion group to enter the cell. Since plasmids of the same exclusion group also typically belong to the same incompatibility group, this reduces the number of intracellular encounters between competing plasmids. A similar effect is obtained indirectly by repressing conjugation for long periods and transiently turning it into full activity (Lundquist and Levin 1986). Since repressors need time to accumulate in the recipients, conjugation into a plasmidfree cell can start an avalanche in which most plasmidfree cells in a population receive the plasmid. Consequently, the conjugational activity can be low when most cells carry the plasmid but increase greatly in response to plasmidfree cells.
Both these mechanisms allow plasmids to epidemically sweep through a population of plasmidfree cells but still keep formation of heteroplasmid cells at a minimum. They could thus play the role of uniparental inheritance of intracellular organelles that similarly allows effective transmission without pitting copies against each other (Eberhard 1980; Cosmides and Tooby 1981; Eberhard 1990; Walsh 1992). Previous studies have instead stressed that surfaceexclusion plasmids receive a selfish advantage by shutting out incompatible relatives (Eberhard 1990; Cooper and Heinemann 2000) and that transitory derepression is metabolically favorable and avoids extended exposure of phagesensitive pili (Lundquist and Levin 1986; Eberhard 1990). These rationales are to the point, but shortterm advantages support rather than contradict the possibility of longterm protection against intracellular selfishness.
Policing against multimers: Plasmid monomers spontaneously form multimers through homologous recombination. Multimerization is highly unfavorable for plasmids because it imposes a larger burden on the host and increases the plasmid loss rate (Summerset al. 1993; Summers 1996), supposedly by reducing the number of independently segregating copies for a given total genetic load.
The replication frequency of multimers depends on the replication control, but for ColE1 the effect is fairly straightforward. If j replication origins are intact, multimerization increases the synthesis rates of both the cis activator and the trans inhibitor by a factor j. The trans effect downregulates replication attempts of monomers and multimers alike while the cis effect gives an unequal advantage to multimers. In terms of Equations 5, 6, 7 with Y_{1} as monomers and Y_{j} as jfold multimers, k_{j} = jk_{1} and K_{j} = jK_{1} so that r_{j}/r_{1} = j. Intracellular selection can thus accentuate the multimer problem by inducing runaway multimerization as demonstrated and convincingly argued by Summers and coworkers (Summerset al. 1993; Summers 1996). In other words, multimers are cheaters that gain an intracellular advantage at the cost of an intercellular disadvantage. Many natural plasmids suppress cheating by using sitespecific recombination to actively resolve multimers back to monomers (Summers and Sheratt 1984). This resembles “selfpolicing” (Keller 1999), where lowerlevel selfishness is penalized in favor of a higherlevel reproduction rate, or rather “selfexorcism,” since selfishness is genetically expelled rather than just punished.
A THIRD LEVEL OF SELECTION?
In addition to intra and intercellular selection, lineage selection could favor plasmid traits that help the population of plasmidcontaining cells to fight plasmidfree cells. This section discusses how spitefully low loss rates are favored by lineage selection, suppressed by intercellular selection, and generated by intracellular selection.
Intermittent selection and spitefully low losses: If plasmids have been essential in the recent history, if they colonize a new host, or if the plasmidcarrying cell explores a new environment, it is possible that there are no plasmidfree competing cells. If plasmids are burdensome, the first arising plasmidfree competitor under nonselective conditions can initiate a rapid wipeout of plasmids from the population. To survive periods between selective sweeps, plasmids may thus be well served by spitefully low losses, i.e., a so low 〈L〉 that the total effect of losses and metabolic burdens lowers the net growth rate.
For a quantitative example assume that 〈L〉≈ 2 × 0.6^{〈}^{m}^{〉} and μ ≈ μ_{0}(1 – 10^{–4} × 〈m〉), so that 〈m〉_{opt} ≈ 18 (appendix). At 〈m〉 = 18, then (1 – 〈L〉)μ/μ_{0} ≈ 0.998 and 〈L〉≈ 2 × 10^{–4} so that plasmidfree competitors arise quickly even in fairly small populations. If 〈m〉 = 40, then 〈L〉≈ 3 × 10^{–9} so that plasmidfree competitors rarely arise from plasmidcontaining cells, but then instead (1 – 〈L〉)μ/μ_{0} ≈ 0.996. The 0.2% difference in effective net growth is selectively significant when the population has >10^{3} individuals, suggesting a selection conflict between the individual cell and the population.
Though conflicts often are resolved in favor of the shorter time scale and the lower level of selection, lineage selection could in principle be sufficient to favor plasmidhost clades that sacrifice net growth for lower 〈L〉. However, just as many putative examples of group selection have now been explained by lowerlevel selection, very low 〈L〉 could also be due to intracellular selection: cis selfishness can decrease loss rates more than is metabolically justifiable (see selection conflicts). Selfishness of the lowerlevel unit could thus increase the longterm stability of the higherlevel unit by overriding the selection for a middlelevel unit.
A rigorous treatment of this problem must take stochastics into account. The advantage of very low 〈L〉 heavily relies on the difference between zero and one competing cell and is easily obscured in mathematical rate equation models where the fraction of plasmidcontaining cells can approach zero, but never quite go extinct. Spatial population structure should also be expected to have a large effect since the incentive to suppress competitors is more compelling if one has to deal with them in person.
DISCUSSION
Natural selection occurs at all levels of biological organization. At higher levels it favors cooperation between lowerlevel units and at lower levels it favors cheaters that exploit the common good for their own interests (Keller 1999). Ignoring selection conflicts within the genome—instead focusing directly on function—is convenient when all genes reproduce in sync (Hurstet al. 1996). However, supernumerary (B) chromosomes (Östergren 1945), meiotic drive genes (Haig and Grafen 1991; Lyttle 1991), cytoplasmic organelles (Hurstet al. 1996), and RNA viruses (Chao 1991, 1994; Szathmary 1992; Bonhoefffer and Nowak 1994; Turner and Chao 1999) have all demonstrated an ability to distort transmission frequencies to their advantage. A molecular function may then serve some genes at the expense of others.
Many analyses of intragenomic conflicts briefly mention bacterial plasmids, and the few explicit studies (Novick 1987; Eberhard 1990; Mongold 1992; Bengtsson and Andersson 1997; Riley 1998; Pomiankowski 1999; Bergstromet al. 2000; Cooper and Heinemann 2000) show that they are far from being books in an altruistic gene library that cells can borrow and return at their convenience. By contrast, socalled selfish plasmids can reproduce without conferring advantages to their hosts and may even encode toxinantidote systems to kill off plasmidfree cells (Riley 1998). The term “selfish” is then used synonymously with “parasitic” and relates to how one organism manages to exploit another. However, plasmids may also be selfish in the hierarchical sense that individual copies cheat on the plasmidcontaining cell. By inspecting the selective forces acting on plasmid replication control, this work suggests how a number of plasmid traits in fact can be traced back to such a hierarchical selection conflict. The relative simplicity of these mechanisms and the unequaled ease with which plasmids can be made subject to evolutionary experiments make them well suited for molecular analyses of multileveled selection.
Acknowledgments
I am grateful to R. Kishony, E. C. Cox, C. N. Peterson, M. Ehrenberg, E. Szathmary, and M. Nowak for comments on the manuscript. This work was supported by a LewisThomas Fellowship from Princeton University and BristolMyers Squibb, the Swedish National Graduate School of Scientific Computing, and a Swedish Science Research Council grant to Måns Ehrenberg.
APPENDIX
Equations are derived in order of appearance.
Plasmid replication control: Local steadystate sensitivity is found by differentiating around steady state in loglog scale. For r in (1) and (2), this gives
The average plasmid loss rate for binomial partitioning and Poisson distributed copies is
Intracellular selection: A heteroplasmid cell gives rise to homoplasmid descendants over time. For incompatible plasmids, the transition is relatively fast so that most cells are homoplasmid already after a few divisions (Novick 1987). The soundest way of predicting the fraction of descendants in which a type eventually fixes is to define a timecontinuous Markov process for plasmid replication during the cell cycle and a stochastic rule for how copies are partitioned between daughter cells. Such models have been used to inspect the quality of replication control (Paulsson and Ehrenberg 2001), but can be solved analytically only in the simplest scenarios. When they cannot be solved analytically, one must resort to either numerical integration of the Markov process or exact Monte Carlo algorithms for simulations. Believing that analytical approximations are more informative than more exact numerical solutions when details are insufficiently characterized, this work makes a number of idealizations. Cell growth and plasmid partitioning are replaced by elimination intensities μm_{1} and μm_{2}—as if plasmids were degraded rather than diluted—and the copy number is assumed to be a constant m_{T} = m_{1} + m_{2}. If substitutions occur when a random copy is eliminated and one of the other type replicates, then the substitution rates are
Another simplification is that m_{T} is used both as an average copy number and effective population size. In reality there are bottlenecks because cells at different stages in the cell cycle contain different average copy numbers and because copy numbers fluctuate randomly in single cells. Similarly, unequal partitioning at cell division relaxes intracellular selection, while approximating partitioning with a continuous plasmid elimination rate can make it seem more severe than it actually is, especially for highly sensitive replication control mechanisms.
Since a higher 〈m〉 makes it harder for individual copies to fix, plasmids that differ by a trans mutation do not have the same fixation fractions when arising in each other's cells. However, the increase in copy number also results in a proportionally higher total mutation rate, so that the effect is canceled when considering transitions back and forth between the two types: A trans mutation such that 〈m〉_{2} = 2〈m〉_{1} implies a twofold higher total mutation rate from Y_{2} to Y_{1}, but a twofold reduction in the fixation fraction of the invading Y_{1} copy. An exception to this rule is if the trans mutation increases the average copy number, but not the effective population size, as can be the case for a mutation that increases both average and variance. For conjugation one should also expect effective differences even for trans mutations.
Intercellular selection: With Δs =Δ〈L〉μ–Δμ–ΔΓ≥ 0, a constant x_{T} in (9) leads directly to a quadratic equation for stationary densities with exactly one nontrivial stable solution:
Equation 11 is derived as in (A4, A5, A6) assuming that an X_{1} cell replaces an X_{2} cell (and vice versa) with total intensity γ_{1}n_{1}n_{2} +μ_{1}(1 – 〈L〉_{1})n_{1} × n_{2}/n_{T}, where Γ=γn_{T}, i.e., again a Moran (1958) model for a haploid population.
With 〈L〉≈ 2b^{〈}^{m}^{〉} and μ/μ_{0} ≈ 1 – B〈m〉, the net growth rate (1 – 〈L〉)μ has a single local maximum. Using the approximation 〈L〉μ ≈〈L〉μ_{0}, leads to
Selection conflicts: When mutations are so rare that an arising cell type goes to extinction before the next mutant arises, the population switches between pure populations,
(A9)
where f^{inter} are the fixation probabilities and P(X_{1}) is the longrun probability of X_{1} cells. Using ω= ω_{0} m_{T} f^{intra} together with (8) and (11) gives P(X_{1}) = ½ when f^{intra}_{2}/ f ^{intra}_{1} ×m_{T2}/m_{T1} = f^{inter}_{1}/f ^{inter}_{2}, i.e., when
That –i/ln b increases with i can be shown for both (A3) and the negative binomial as long as i is so large that the approximations are accurate.
The fixation probabilities of cis vs. trans activators can be derived from any birthanddeath process with absorbing boundaries 0 and m_{T} and a ratio between replacement rates r_{2}/r_{1} = 2 + m_{2}/(m_{T} – m_{2}). Dropping the subscript, m = m_{2}, a simple alternative is
(A12)
This corresponds to the master equations
Figure legends: Distributions in Figure 1A are calculated numerically by integrating a master equation with birthanddeath intensities Cm^{1–}^{i} and μm, conditioned on m > 0 (see Equation 2 and Paulsson and Ehrenberg 2000, 2001):
Footnotes

Communicating editor: M. W. Feldman
 Received January 9, 2002.
 Accepted April 15, 2002.
 Copyright © 2002 by the Genetics Society of America