- THIS ARTICLE
-
Abstract
- Full Text (PDF)
- Alert me when this article is cited
- Alert me if a correction is posted
- SERVICES
- Email this article to a friend
- Similar articles in this journal
- Similar articles in PubMed
- Alert me to new issues of the journal
- Download to citation manager
- Reprints & Permissions
- CITING ARTICLES
- Citing Articles via HighWire
- Citing Articles via Google Scholar
- GOOGLE SCHOLAR
- Articles by Kitada, S.
- Articles by Kishino, H.
- Search for Related Content
- PUBMED
- PubMed Citation
- Articles by Kitada, S.
- Articles by Kishino, H.
Empirical Bayes Procedure for Estimating Genetic Distance Between Populations and Effective Population Size
Shuichi Kitadaa, Takeshi Hayashib, and Hirohisa Kishinoba Department of Aquatic Biosciences, Tokyo University of Fisheries, Minato, Tokyo 108-8477, Japan
b Graduate School of Agricultural and Life Sciences, University of Tokyo, Bunkyo, Tokyo 113-8657, Japan
Corresponding author: Shuichi Kitada, Tokyo University of Fisheries, 4-5-7 Konan, Minato, Tokyo 108-8477, Japan., kitada{at}tokyo-u-fish.ac.jp (E-mail)
Communicating editor: Z-B. ZENG
| ABSTRACT |
|---|
We developed an empirical Bayes procedure to estimate genetic distances between populations using allele frequencies. This procedure makes it possible to describe the skewness of the genetic distance while taking full account of the uncertainty of the sample allele frequencies. Dirichlet priors of the allele frequencies are specified, and the posterior distributions of the various composite parameters are obtained by Monte Carlo simulation. To avoid overdependence on subjective priors, we adopt a hierarchical model and estimate hyperparameters by maximizing the joint marginal-likelihood function. Taking advantage of the empirical Bayesian procedure, we extend the method to estimate the effective population size using temporal changes in allele frequencies. The method is applied to data sets on red sea bream, herring, northern pike, and ayu broodstock. It is shown that overdispersion overestimates the genetic distance and underestimates the effective population size, if it is not taken into account during the analysis. The joint marginal-likelihood function also estimates the rate of gene flow into island populations.
AS a stock management tool to counteract decreased or depleted fishery resources, stock enhancement programs have been undertaken in many countries for salmonid (![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
![]()
The genetic identity between produced progenies and the wild stock will be required before one can release the progenies. To examine the genetic identity, statistically significant differences are required. The homogeneity
2 test of allele frequencies is commonly used for testing genetic differences and the Roff test (![]()
![]()
![]()
![]()
![]()
An effective method of determining genetic identity is to examine the genetic distances between populations. If an estimated confidence interval of the genetic distance between two populations includes 0, we could conclude that the populations are genetically identical or not statistically significantly different. There are several measures for the genetic distance (![]()
In this article, we develop a Bayesian estimating procedure to measure genetic distances between populations from allele frequencies. We can directly evaluate the probability distribution of the genetic distance from its posterior distribution. The general method developed here is extended to estimate the effective population size termed Ne in a population based on temporal changes in allele frequencies. The joint marginal-likelihood function derived here coincides with the likelihood function to estimate the rate of gene flow into island populations using the sample allele frequency from a number of islands (![]()
| METHODS |
|---|
Let the frequencies of k alleles of two populations to be compared be p11, · · · , p1k and p21, · · · , p2k. ![]()
![]() |
(1) |
We use this distance as a natural measure of the genetic distance between populations, which takes values between 0 and 4. It is known that 2n1.n2.
/(n1. + n2.) follows a
2 distribution with degree of freedom k - 1 when p1i = p2i = pi for i = 1, · · ·, k (![]()
is the estimator obtained by substituting sample frequencies in Equation 1. However, the distribution of
is unknown when p1i
p2i for i = 1, · · · , k. It is then inappropriate to evaluate the confidence interval of D using an asymptotic variance of
, although it can be derived. Here we directly evaluate the posterior probability density of the genetic distance measure using a Bayesian framework.
Prior and posterior distribution of D:
It is not easy to describe a reasonable prior distribution of D, especially when we compare more than two populations. Alternatively, we set a prior for allele frequencies. Let the allele frequency of a population be p = (p1, · · · , pk)' and the sample count be n = (n1, · · · , nk)', where
pi = 1 and
ni = 2n (n individuals). When the sample is collected by a simple random sampling procedure with replacement, n follows a multinomial distribution. A ß distribution is known as a conjugate prior of the binomial parameter p. A Dirichlet distribution is a conjugate prior of multinomial proportions, which is an extension of a ß distribution (![]()
![]()
![]() |
(2) |
Here,
= (
1, · · · ,
k)' are regarded as hyperparameters specifying the prior distribution. We use this distribution as a prior for allele frequencies.
The posterior distribution is obtained by multiplying the likelihood function, which is multinomial distribution in this case, by the prior. The posterior distribution of p is then given by

which is again a Dirichlet distribution with parameters modified by the data ni +
i (![]()
![]()
= (
1, · · · ,
k)', we can obtain a posterior distribution of p by generating Dirichlet random numbers with parameter
+ n using Monte Carlo simulations. Using independent Dirichlet random numbers for posterior distributions of population allelic frequencies, we can obtain a posterior distribution of D using Equation 1. The number of each Monte Carlo simulation is set to 10,000, so 10,000 D are calculated from the 10,000 sets of p between two populations. The posterior probability density function is estimated on the basis of the histogram of D with the number of classes of 100 by using the function "density" of S language version 4 (![]()
.
Empirical Bayes procedure:
The primary disadvantage of using a Bayesian analysis for allele frequency estimation is that there is no obvious way of selecting a reasonable prior (![]()
1 = · · · =
k =
is a noninformative prior (![]()
by maximizing the marginal-likelihood function (![]()

which is also given in ![]()
![]()
![]()
![]()
![]()
is the same for H populations (samples) to be compared and 
i is also the same for J loci, the joint marginal likelihood is then given by
![]() |
(3) |
where Cjh =
is a constant term for the combination of the multinomial likelihood that can be excluded from the estimation procedure.
Parameter
2 is the dispersion parameter that defines the magnitude of overdispersion; i.e., the variance of the response Y exceeds the nominal variance (![]()
2mp(1 - p) though the expectation remains the same, where Y has a density of a ß-binomial distribution. For a multinomial event with overdispersion, the variance-covariance matrix of Y is
2 times larger than that of the multinomial distribution, where Y has a density of a Dirichlet-multinomial distribution.
![]()
times larger than that of the multinomial distribution. Hence the relationship between the dispersion parameter and the hyperparameters for a population is given by
![]() |
(4) |
We assume equal overdispersion effects for all loci, so the total of the hyperparameters
(hereafter we use
for 
i) is the same for all loci, which gives the expression for
2 as
![]() |
(5) |
Here 2
is the mean number of genes of H populations given by 2
=
. Given the estimate of
2, we have the estimator for
as
![]() |
(6) |
We estimate
Jj=1(kj - 1) + 1 free parameters numerically, including
2, which is assumed to be the same among loci, and
1j, · · · ,
kj-1,j for locus j, and
kj is estimated by
-
kj-1i=1
ij.
The binomial and multinomial counts are assumed to be taken by a simple random sampling, so the dispersion parameter
2 is considered to indicate the magnitude of overshooting from a simple random sample. ![]()
![]()

2(= 14.73) times larger than those assuming the multinomial model, which was considered to be caused by the aggregation of the tagged fish in the fishing ground. In genetic data analysis, overdispersion corresponds to the variance of an allele frequency exceeding the nominal variance of a simple random sample from a gamete pool. If there are subpopulations divided spatially in a survey area, a sample allele frequency from the area might be overdispersed even if a simple random sampling is performed. If a cluster of a genotype is taken, a sample allele frequency from a population might be also overdispersed. One can then estimate overdispersion based on several sets of allele frequencies obtained from the survey area.
Standardized genetic distance:
When allele frequencies at J loci are obtained from genetically identical populations, 2n1.n2.
follows a
2 distribution with a degree of freedom of
(kj - 1) asymptotically (![]()
Jj=1
j and take a value proportional to the sample size. It is then not convenient to make D an index of the genetic distance.
Here we standardize D and propose a general index for the genetic distance. Performing a square root transformation to make the variance independent of the mean (![]()
![]() |
(7) |
which follows a normal distribution with mean 0 and variance 0.5 under the condition of p1 = p2.
The
2 distribution of
assumes that 2n1 and 2n2 genes are taken by a binomial sampling from a population. For this case,
2 in the first term of Equation 7 equals 1. However, if there is overdispersion,
2 becomes active and takes a value larger than 1. If the overdispersion is neglected, the genetic distance is then overestimated and the scale of the distribution of
becomes
2 times larger than that under the previously stated assumption. The dispersion parameter
2 corrects this effect.
Effective population size:
The effective population size is estimated from the temporal variation of allele frequencies in a population. Since the observed variance of the allele frequencies includes the sampling variance in addition to the genetic drift, we subtract the sampling variance when estimating the effective population size. Let us assume that we have two samples with sizes n0 and nt from the population at generations 0 and t, respectively. The empirical Bayes procedure developed here can be extended to obtain the posterior distributions of the effective population size Ne by using the posterior distribution of F-statistics calculated from the posterior distribution of allele frequencies.
The standardized variance of allele frequency change measured by F-statistics has been used to estimate Ne (![]()
![]()
![]()
![]()
![]()
![]() |
(8) |
For the case of multiple loci, Fk is calculated by Fk =
from ![]()
![]() |
(9) |
for plan I, where the sample is taken after reproduction. For plan II, where the sample is taken before reproduction, the term 1/N is eliminated, where
k is the estimator obtained by substituting sample frequencies in Equation 8 is the census size for a population (![]()
Equation 9 assumes that 2n0 and 2nt genes are taken by a binomial sampling from the population. If there is overdispersion, Fk is overestimated, which leads to underestimation of Ne. Since the effective sample size is obtained by discounting the apparent sample size by dispersion parameters, Equation 9 is modified as
![]() |
(10) |
| CASE STUDIES |
|---|
Red sea bream:
To evaluate genetic distances, we first analyzed the data of four populations of red sea bream (Pagrus major) from ![]()
|
The estimate of the total hyperparameters was 106.553 for each locus (Table 1), and the dispersion parameter was estimated at 1.80 by maximizing Equation 3. Here 2
. =
= 87.5 because mtDNA is a haploid. With a prior distribution specified by these parameters, we obtained the posterior distribution of D by dividing
Dj over four loci by the number of loci (Table 2). As an example, the histogram and estimated density function of the posterior distribution of D at HinfI between Tanabe Bay and Tomogashima Channel is shown in Fig 1. D12 in Fig 2 was obtained as the mean of such four posterior genetic distances at the four loci. It should be noted that the posterior distances were overestimated including the overdispersion and the posterior distributions in Fig 2 were then overestimated. The means and SDs of D12, D13, and D14 were about two times larger than D23, D24, and D34; however, they might include the effect of the smaller sample size of population 1 (Tanabe Bay).
|
|
|
The posterior distribution of the standardized genetic distance took the overdispersion and sample size difference into account. The means of I12, I13, and I14 ranged from 0.2706 to 0.4671, whereas those of I23, I24, and I34 took negative values. The SDs ranged from 0.53 to 0.58 and took almost the same values. The genetic differences with population 1(I12, I13, and I14) looked larger than the others (Table 3). However, the posterior distributions of I overlapped well with the theoretical distribution of no genetic difference (Fig 3).
|
|
We estimated the 95% confidence interval of the dispersion parameter to be from 1.72 to 1.88 by the likelihood-ratio test. The lower limit of the dispersion parameter corresponds to the upper limit of the genetic distance, from which we evaluate the difference. The means of the posterior distributions for the lower limit of the dispersion parameter were increased from 8 to 27% and SDs remained the same (Table 3), but the posterior distributions of I were almost the same as those for the point estimate of the overdispersion and still overlapped well with the theoretical distribution (Fig 3).
The value of 95% upper limit of the credibility region of the theoretical normal distribution of I with mean of 0 and variance of 0.5 is 1.16. All posterior means were <1.16, and the credibility regions included 0; hence we concluded that there was no genetic difference between the four populations of red sea bream. This finding agreed with the result of the original authors, who reported that the Roff test did not reject the homogeneity of the haplotype frequencies (![]()
Herring:
Stock enhancement of herring (Clupea pallasii) has been performed in Akkeshi Bay, Hokkaido (Japan). Because the matured herrings migrate to Akkeshi Bay to spawn, they are considered to have originated from Lake Akkeshi and Akkeshi Bay. Although wild adult fish that migrated to the bay are used for artificial spawning to produce juveniles every year, it still may be important to monitor the genetic change and estimate the effective population size to maintain the wild stock.
Temporal changes in allozyme allele frequencies were obtained by combining two studies on the same loci by ![]()
![]()
|
The estimate of the total hyperparameters was 130.956 for each locus (Table 4), and the dispersion parameter was estimated at 2.39, with 2
. =
= 184. The posterior distribution of Fk estimate was calculated from each of two sets of posterior distributions of allele frequencies for 1993 and 1996 by
![]() |
(11) |
Fk ranged from 0.0014 to 0.0814 with the mean and SD of 0.0226 and 0.0105, respectively. The posterior distribution of Fk is shown in the left side of Fig 4.
|
Most of the matured herring migrating to Akkeshi Bay to spawn are in their second year of life; the remainder are in their third year. The age composition of the spawners was surveyed and estimated at 0.9 and 0.1 for each age class by the Japan Sea-Farming Association. The expected number of generations can be used for t in the estimating equations of Ne because the expectation of the F-statistics was approximated to be linear with t as shown in ![]()
) divided by the number of years between samples by the mean age of spawners as ![]()
= 210 for 2n0 and
= 158 for 2nt. The posterior means of Ne estimates and 95% credibility region of Ne are given in Table 5.
|
The dispersion parameter was estimated at 2.39 with a 95% confidence interval from 1.00 to 7.51. From a conservation viewpoint, it is better to consider the lower limit of Ne. The lower limit of the dispersion parameter evaluates the upper limit of Fk corresponding to the lower limit of Ne. The lower limit of
2 was 1.00. Corresponding with that, no overdispersion arose and no subpopulation existed. The number of simulations with a negative value of Ne estimate was 1221 in 10,000 trials. When Fk
[1/(2n0) - 1/(2nt)], the only feasible estimate of Ne is infinity (![]()
Northern pike:
We analyzed the data from ![]()
![]()
. =
= 178.7.
Fk between the year of 1977 and 1993 ranged from 0.0087 to 0.1264 with the mean and SD being 0.0479 and 0.0150, respectively. The posterior distribution of Fk is given in the left side of Fig 5. The posterior distribution of the Ne estimate was obtained by substituting the posterior distribution of Fk into Equation 10, eliminating the term 1/N, with t = 4, as given in ![]()
|
|
The 95% confidence interval of the dispersion parameter was estimated from 5.51 to 18.80. For the 95% lower limit of the dispersion parameter, the number of simulations with a negative value of Ne estimate was 8480 in 10,000 trials. The mean of the positive Ne estimates was 1065 and the 95% credibility region was estimated from 123 to infinity (Table 6). The posterior distribution of the positive Ne estimate, neglecting the overdispersion (
2 = 1), and for the lower limit of
2(= 5.51) are given in the right side of Fig 5. In this example, we can see the effect of the overdispersion on the posterior distribution of Ne estimate. The estimate of Ne, neglecting the overdispersion, agreed well with the estimate of ![]()
Ayu broodstock:
Ayu (Plecoglossus altivelis) is the most popular target species of recreational anglers in rivers and streams in Japan. A total of 300 million juveniles are released every year, of which hatchery-produced fish comprise
30%. The life span of ayu is 1 year. They spawn in a river from September to November and die after spawning. Hatched larvae go down to the sea and winter there. The upstream run of wild ayu juveniles begins from the coast in late March to early April and is over by early July. Soon after, they mature, spawn from September to November, and then die after spawning.
Hatcheries have commonly cultured broodstocks over generations. At the Gunma Prefecture Fisheries Experimental Station, adult ayu have been cultured over 27 generations. About 30004000 fish have been reared every year as broodstock, some of which are used for artificial fertilization. In 1996,
850 females and 650 males were used. The temporal changes in allozyme allele frequencies of the ayu broodstock were reported by ![]()
|
The total of the hyperparameters was estimated at 28.576 for each locus (Table 7), and the dispersion parameter was estimated at 5.86, with 2
. =
= 143. The 95% confidence interval of the dispersion parameter was estimated to range from 3.04 to 13.49. We calculated four Fk's on the basis of temporal changes in allele frequencies observed in the first (19961997; F1) and second (19971998) time intervals (F2), over the entire interval (19961998; F3), and for the entire interval based on the pooled F for the first two intervals, as ![]()
![]()
![]()
![]()
![]()
![]()
![]()
The values of Fk calculated by Equation 11 were varied using four combinations for two samples in each sampling year, reflecting the variation in allele frequencies for each sampling period. The posterior mean and SD of F3 were the largest, and the SD of Fmean was the smallest but almost the same as that of F1 (Table 8). The posterior distributions of Fk are illustrated in the left side of Fig 6.
|
|
We fixed the generation time at t = 1.0 for the first and second time intervals and had t = 2.0 for the entire interval because the life span of ayu is 1 year. The samples were taken after reproduction, so we used Equation 10, substituting 2n0 =
= 200, 2nt =
= 100, and N = 1500, which was the total number of individuals used for artificial fertilization.
We made four estimates of Ne on the basis of F1, F2, F3, and Fmean. Estimates for the 95% lower limit of the dispersion parameter (= 3.04) are given in Table 8. The posterior mean obtained using F3 was the largest with the largest SD, and that for Fmean was second with a smaller SD. The Ne estimate that took
in 10,000 simulations was 8677 when using Fmean, and that for F3 was 4927. This is because the sampling correction term in the denominator of Equation 10 took the very similar values of 0.0912 for F3 and 0.0928 for Fmean, despite the posterior mean of Fmean being smaller than that of F3, as shown in the left side of Fig 6. The posterior distributions of the positive Ne estimate obtained by using F1, F2, F3, and Fmean for the lower limit of
2 are given in the right side of Fig 6, showing a larger variance of the estimate based on F3 than those derived from F1, F2, and Fmean.
We failed to estimate the upper limit of the credibility regions because of large sampling variances with overdispersion. However, when the numbers of breeding males Nm and females Nf are given, which is difficult to know in a wild population but possible in hatcheries, the effective population size is obtained by Ne =
(![]()
10 to 20, and the ratio of males to females was
0.8. The eggs were squeezed from the females and stocked in a stainless bowl and then fertilized by squeezing sperm from individual males. This method of fertilization might not guarantee a random mating of the males and females used; hence, 1473 should be used as the upper limit of the credibility regions instead of
(Table 8). If we neglect overdispersion, the 90% credibility region could be obtained at [13589] with the posterior mean of 136, which was underestimated.
| DISCUSSION |
|---|
The empirical Bayes procedure developed here makes it possible to describe the skewness of the genetic distance and evaluate genetic differentiation between populations while taking full account of the uncertainty of the sample allele frequencies. When we compare populations in which the genetic differentiation is small, the hypothesis-testing framework cannot accept the null hypothesis of no genetic differentiation in almost all cases, because of the poor statistical power with relatively small sample sizes. The empirical Bayes procedure is effective even in such cases. So we believe it could play an important role in the field of conservation.
This general method can easily be extended to any parameter that is a function of multinomial frequencies. When the parameter of interest is a function of allele frequencies, the posterior distribution of that parameter can be obtained through the function by using the posterior distribution of the allele frequencies, instead of assuming a prior distribution for the parameter.
Overdispersion and empirical Bayes:
Until now, models based on a simple random sampling from the gamete pool have been assumed when evaluating allele frequencies. However, as shown in the four case studies treated in this article, a simple random sampling is not necessarily guaranteed. If there are subpopulations divided spatially in a survey area, or a cluster of a genotype is taken, a sample allele frequency might be overdispersed. If overdispersion arises, a sampling variance becomes
2 times larger than that for a simple random sampling. This can seriously affect the precision of the estimate of genetic distance and the effective population size. As a result, the genetic distance and F-statistics can be overestimated, and the effective population size can be underestimated, if overdispersion is not taken into account in the analysis. Therefore, it is quite important to take overdispersion into account when estimating genetic distance and effective population size.
Sample sizes:
If we use the noninformative prior of the Dirichlet distribution (![]()
We examined the relationship between sample size and the estimates of the hyperparameters using the data of red sea bream given in Table 1. We estimated the hyperparameters with multipliers of 0.5, 2, 3, and 4 to test each population with the same sample allele frequencies. The estimates of the hyperparameters were stable and not dependent upon sample sizes. This confirmed the robustness of the empirical Bayes procedure (Table 9). However, the dispersion parameter became larger as sample size increased. This is to be expected from the relationship between the total of the hyperparameters, the sample sizes, and the dispersion parameter given by Equation 5.
|
Suppose there are several subpopulations and the population allele frequencies are largely varied spatially. If the sample sizes are small, one might consider that the large variation in the sample allele frequencies is a function of the small sample size. On the other hand, if the same allele frequencies are obtained for larger sample sizes, one could consider that the large variation comes from the subpopulation structure with confidence. The more samples one draws, the more precisely one can estimate the dispersion parameter. In addition, increasing the number of polymorphic loci to be surveyed may also increase the information available for estimating the dispersion parameter, e.g., the precision of the dispersion parameter estimate in the case of red sea bream, in which the narrowest confidence interval was obtained among our four case studies.
It is also quite important to consider sampling strategies to minimize overdispersion caused by sampling procedures. For example, sampling from different sites and times may be useful to avoid sampling clusters of individuals having the same genotype. Such multiple samples contribute simultaneously to a more precise estimation of the dispersion parameter.
Standardized genetic distance:
As I follows the normal distribution, it takes values between -
and +
. For simplicity, let X =
and define the expected value by E[
]. If
> E[
], I takes a positive value, and 0 if
= E[
]. If
< E[
], I takes a negative value. As X follows the
2 distribution asymptotically when there is no genetic difference, E[
] is almost equal to the square root of the degrees of freedom of X, which is the number of loci examined. When
Jj=1
j does not increase compared to the increased number of loci examined, the likelihood for I taking a negative value increases. On the other hand, when
Jj=1
j increases to the increased number of loci examined, the likelihood for I taking a positive value increases. This point illustrates the effectiveness of increasing the number of loci to obtain increased information on the genetic differentiation from the value of the posterior mean of I. Conversely, a negative posterior mean indicates that little information on genetic differentiation will be obtained even if the number of loci is increased, as a function of the small genetic differentiation. This is considered to be the cause of the negative values of the posterior mean for I23, I24, and I34.
Overdispersion and gene flow:
![]()
![]()
![]()
![]()
![]() |
(12) |
where FST is the coancestry coefficient of ![]()
![]() |
(13) |
from which we can see larger FST gives larger overdispersion. From Equation 13, we also have the relationship
![]() |
(14) |
![]()
(n) = (n - 1)!. In PMLE,
i is treated by
pi. Here, pi is a nuisance parameter estimated from the data as
i =
, and then
i is substituted for pi in the log-likelihood function, and the only unknown parameter
is estimated by using the Newton method. By contrast, we directly maximize the negative log-likelihood function and estimate
Jj=1(kj - 1) + 1 parameters by using a simplex minimization. We estimated the hyperparameters and the dispersion parameter from the mtDNA haplotype distribution among islands for Channel Island foxes given in Table 2 of ![]()
![]()
|
![]()
for a discrete-generation island model of a population at equilibrium, based on FST as
=
(![]()
![]()
2 gives smaller
, indicating that larger genetic differentiation corresponds to smaller gene flow. For the case of the red sea bream,
2 and
were estimated at 1.80 and 106.55, respectively. For the case of the foxes, they were estimated at 17.89 and 0.45, respectively. From this result, it is clear that the six fox populations in the isolated islands had small gene flow and large genetic differentiation. On the other hand, red sea bream had large gene flow and small genetic differentiation. The estimate of FST for red sea bream was
=
= 0.0093, which was relatively small. But for foxes it was
=
= 0.6894, suggesting advanced inbreeding in the fox populations.
The essential idea for estimating overdispersion is to compare the variation of sample allele frequencies obtained from the different locations to the multinomial variance. In addition, the effective population size is based on the changes in allele frequencies between generations. Conversely, overdispersion provides insight into the spatial variation of allele frequencies. By evaluating the spatial variation, it might become possible to discriminate the overdispersion resulting from the variation between generations. Hence, the procedure needs to evaluate overdispersion as a function of the spatial variation and then measure the variation between generations taking overdispersion into account.
In the three case studies we looked at for estimating the effective population size, direct information on the spatial variation was scarce. Therefore, the precision of the dispersion parameter was marginal. When subpopulations exist, overdispersion arises and affects the estimation of the effective population size. It is then important to collect data on the spatial variation. At the same time, when many isolated subpopulations exist, the effective population size is considered to be close to the size of a subpopulation. When this occurs, it seems dangerous to dismiss the variation between generations as overdispersion. It needs further consideration.
Practical considerations on estimating Ne:
From the approximate variance formula of Ne estimate (![]()
![]()
![]()
![]()
Sample size:
The idea of the temporal method is to estimate Ne from the genetic change over time described by F-statistics estimated from the sample allele frequencies. F-statistics, then, consist of the genetic drift and the sampling variance. To evaluate the genetic drift, we have to subtract the sampling variances from the F-statistics. The second and third terms in the denominator of Equation 10 are the sampling variances at generations 0 and t. If Ne is large, the genetic drift may be small, so the denominator of Equation 10 would take a negative value, which leads to an infinite Ne for small sample sizes n0 and nt. If overdispersion arises, the effect of subtracting the sampling variances becomes
2 times larger, which is why we failed to estimate the upper limit of the credibility region of Ne. As pointed out by ![]()
Number of loci:
![]()
![]()
Number of years between samples:
The number of years between samples is correlated with the number of generations, and it then affects the precision of the estimate of Ne. A large number of generations between samples can improve the precision of the estimate of Ne (![]()
![]()
For the case study of ayu, the posterior mean of Ne was 350 based on F1 and 796 based on F3, and SDs for the two estimates were 4024 and 18,538, respectively, showing the reduction of precision despite the fact that the number of years between samples was doubled (Table 8). This is because the doubled number of generations increased the variance of allele frequency changes. The numbers of infinite Ne estimates in 10,000 simulations were 8453 on F1 and 4927 on F3, and the smaller F values increased the estimated value of Ne. The result was similar for northern pike. The point estimate of Ne based on F3 (= 125) was larger than those based on F1 (= 35) and F2 (= 72), and the confidence interval for F1 was the largest (![]()
![]()
When Ne does not change in the entire interval, F3 is expected to have more information on genetic drift than F1 and F2. ![]()
Overlapping generations: The basic theory of the temporal method assumes generations to be discrete. The expected number of generations used in Equation 10 directly affects the estimate of Ne. We take time to be measured in years. The expected number of generations between samples can be estimated by dividing the number of years between samples by the mean generation time, which corresponds to the mean age of maturity. In the case of ayu, since the life span is 1 year, 1 year coincides with one generation, which makes it possible to estimate E[t] by the above-mentioned method.
In the case of herring and salmon, however, where there are overlapping year classes of spawners, the estimation of E[t] is complex. When generations overlap, the age-specific birth rate may essentially affect the estimate of E[t]. ![]()
may be biased upward, leading to an estimate of Ne that is too small (![]()
![]()
![]()
![]()
![]()
![]()
In the case of herring, age-specific survival and birth rates were unknown, so it was not possible to apply the method of ![]()
). If a distribution of age-specific birth rates is concentrated to a specific age, E[t] may be close to the estimate obtained by the methods of ![]()
![]()
) for a time interval of 3 yr between samples by using the computer program given in ![]()
e when demographic parameters change over time with overlapping generations should be corrected upward. From a conservation viewpoint, the estimate of Ne without the correction must be conservative for the overlapping generations.
Rate of inbreeding:
As another evaluation of breeding population size, the inbreeding coefficient may be useful, especially for cases where the population size is estimable, as it is in the field of fishery science. ![]()
![]()
![]()
The posterior mean of herring was 7.5 times larger than that of ayu with a right-tailed credibility region shown in Fig 7. Overdispersion for herring was underestimated, because the samples for males and females taken in 1996 were analyzed as independent samples even though they were the same sample, leading to a smaller value of Ne, which caused the rate of inbreeding to be overestimated. The mean for northern pike was smallest with the narrowest credibility region. However, these values may be underestimated because of an overestimated dispersion parameter of northern pike, which was the largest among our four case studies. There was only one sample for one sampling year, and we assumed that the seven loci had common hyperparameters, so the estimated dispersion parameter may include the change of the allele frequencies.
|
Multistage sampling in hatcheries:
All existing methods assume that Ne is drawn from a gamete pool by a simple random sampling. This is an appropriate assumption for the reproduction of a wild population. However, for broodstocks cultured over generations in hatcheries, candidates of the next broodstock are sampled from the progenies produced by the broodstock. Therefore, Ne is drawn from the progenies by a two-stage sampling. If artificial fertilization using a part of the candidates is performed, as in the case study of ayu, Ne is drawn from the progenies by a three-stage sampling and the sample is drawn from the candidates to estimate the allele frequencies, which is therefore a two-stage sampling of the progenies. The multistage sampling must lead to the different form of V(x - y) given in ![]()
| ACKNOWLEDGMENTS |
|---|
We thank Zhao-Bang Zeng and two anonymous referees for their comments on an earlier version of this article. We also thank Ray Timm for critical review of the manuscript, Fumio Tajima for important suggestions made during our research, Kazutomo Yoshizawa for biological information on ayu broodstocks including unpublished data, and Masashi Yokota for helpful discussions.
Manuscript received February 17, 2000; Accepted for publication July 26, 2000.
| APPENDIX |
|---|
Expectation of
when X follows a
2 distribution:
Let X be a random variable that follows a
2 distribution with degree of freedom of k. We derive here the expectation of
. Generally, when X is continuous and has a probability density function f(x), the expectation of g(X) is given by

For our case, the expectation of
is calculated as

Let
= y, and we have

Using the gamma function, which is given by

finally we have

| LITERATURE CITED |
|---|
ANDO, T., and N. OHKUBO, 1997 A study for estimating effective population size from allozyme allele frequencies, Heisei 8 nendo seitaikei hozengata syubyo seisan gijyutu kaihatsu kenkyu seika no gaiyo, pp. 7479. Fishery Agency of Japan, Tokyo (in Japanese).
BARTLEY, D. M., 1999 Marine ranching, a global perspective, pp. 7990 in Stock Enhancement and Sea Ranching, edited by B. R. HOWELL, E. MOKSNESS and T. SVÅSAND. Blackwell, Oxford.
BARTLEY, D. M., D. B. KENT, and M. A. DRAWBRIDGE, 1995 Conservation of genetic diversity in a white seabass hatchery enhancement program in southern California. Am. Fish. Soc. Symp. 15:249-258.
BLANKENSHIP, H. L. and K. M. LEBER, 1995 A responsible approach to marine stock enhancement. Am. Fish. Soc. Symp. 15:165-175.
BOX, G. E. P., and G. C. TIAO, 1992 Bayesian Inference in Statistical Analysis. Wiley, New York.
CAMPTON, D. E., 1995 Genetic effects of hatchery fish on wild populations of Pacific salmon and steel head: what do we really know? Am. Fish. Soc. Symp. 15:337-353.
CHAMBERS, J. M., and T. J. HASTIE (Editors), 1992 Statistical Models in S. Wadsworth and Brooks, Belmont, CA.
CHUANG, C. and C. COX, 1985 Pseudo maximum likelihood estimation for the Dirichlet-multinomial distribution. Comm. Stat. Theory Methodol. 14:2293-2311.
COHEN, J., 1994 The earth is round (p < 0.05). Am. Psychol. 49:997-1003.
CROW, J. F., 1954 Breeding structure of populations. II. Effective population number, pp. 543556 in Statistics and Mathematics in Biology, edited by T. A. BANCROFT, J. W. GOWEN and J. L. LUSH. Iowa State College Press, Ames, IA.
DIZON, A. E., B. L. TAYLOR, and G. M. O'CORRY-CROWE, 1995 Why statistical power is necessary to link analyses of molecular variation to decisions about population structure. Am. Fish. Soc. Symp. 17:288-294.
FALCONER, D. S., and T. F. C. MACKAY, 1996 Introduction to Quantitative Genetics, Ed. 4. Longman, Essex, UK.
HAGEN, R. L., 1997 In praise of the null hypothesis statistical test. Am. Psychol. 52:15-24.
HARADA, Y., M. YOKOTA, and M. IIZUKA, 1998 Genetic risk of domestication in artificial stocking and its possible reduction. Res. Popul. Ecol. 40:311-324.
HILBORN, R., 1992 Hatcheries and the future of salmon in the Northwest. Fisheries 17:5-8.
HILBORN, R. and J. WINTON, 1993 Learning to enhance salmon production: lessons from the salmonid enhancement program. Can. J. Fish. Aquat. Sci. 50:2043-2056.
HILL, W. G., 1979 A note on effective population size with overlapping generations. Genetics 92:317-322
HOTTA, T., T. MATSUISHI, H. SAKANO, and Y. KANNO, 1999 Population structure of Pacific herring Clupea pallasii in the Eastern Hokkaido waters. Nippon Suisan Gakkaishi. 64:655-660. (in Japanese)..
JOHNSON, N. L., and S. KOTZ, 1969 Discrete Distributions. Wiley, New York.
JORDE, P. E. and N. RYMAN, 1995 Temporal allele frequency change and estimation of effective size in populations with overlapping generations. Genetics 139:1077-1090[Abstract].
KAERIYAMA, M., 1999 Hatchery programmes and stock management of salmonid populations in Japan, pp. 153167 in Stock Enhancement and Sea Ranching, edited by B. R. HOWELL, E. MOKSNESS and T. SVASAND. Blackwell, Oxford.
KITADA, S., 1999 Effectiveness of Japan's stock enhancement programs: current perspectives, pp. 103131 in Stock Enhancement and Sea Ranching, edited by B. R. HOWELL, E. MOKSNESS and T. SVÅSAND. Blackwell, Oxford.
KITADA, S., K. HIRAMATSU, and H. KISHINO, 1994 Estimating mortality rates from tag recoveries: incorporating over-dispersion, correlation and change points. ICES J. Mar. Sci. 51:241-251.
KNAPP, G., 1999 Alaska salmon ranching: an economic review of the Alaska salmon hatchery programme, pp. 537556 in Stock Enhancement and Sea Ranching, edited by B. R. HOWELL, E. MOKSNESS and T. SVÅSAND. Blackwell, Oxford.
KRIMBAS, C. B. and S. TSAKAS, 1971 The genetics of Dacus oleae. V. Changes of esterase polymorphism in a natural population following insecticide control: Selection of drift? Evolution 25:454-460.
LANGE, K., 1995 Application of the Dirichlet distribution to forensic match probabilities. Genetica 96:107-117[Medline].
LEE, P. M., 1989 Bayesian Statistics: An Introduction. Edward Arnold, London.
MARITZ, J. S., and T. L. LWIN, 1989 Empirical Bayes Methods, Ed. 2. Chapman & Hall, London.
MCCULLAGH, P., and J. A. NELDER, 1983 Generalized Linear Model, Ed. 2. Chapman & Hall, London.
MILLER, L. M. and A. R. KAPUSCINSKI, 1997 Historical analysis of genetic variation reveals low effective population size in a Northern Pike (Esox lucius) population. Genetics 147:1249-1258[Abstract].
NEI, M., 1987 Molecular Evolutionary Genetics. Columbia University Press, New York.
NEI, M. and F. TAJIMA, 1981 Genetic drift and estimation of effective population size. Genetics 98:625-640
PETERMAN, R. M., 1990 Statistical power analysis can improve fisheries research and management. Can. J. Fish. Aquat. Sci. 47:2-15.
POLLAK, E., 1983 A new method for estimating the effective population size from allele frequency changes. Genetics 104:531-548
RANNALA, B. and J. A. HARTIGAN, 1996 Estimating gene flow in island populations. Genet. Res. 67:147-158[Medline].
RITTER, J. A., 1997 The contribution of Atlantic salmon Salmo salar L. enhancement to a sustainable resource. ICES J. Mar. Sci. 54:1177-1187.
ROFF, D. A. and P. BENTZEN, 1989 The statistical analysis of mitochondrial DNA polymorphisms:
2 and the problem of small sample. Mol. Biol. Evol. 6:539-545[Abstract].
SANGHVI, L. D., 1953 Comparison of genetical and morphological methods for a study of biological differences. Am. J. Phys. Anthropol. 11:385-404[Medline].
SNEDECOR, G. W., and W. G. COCHRAN, 1967 Statistical Methods, Ed. 6. Iowa State University Press, Ames, IA.
TABATA, K. and A. MIZUTA, 1997 RFLP analysis of the mtDNA D-loop region in red sea bream Pagrus major population from four locations of western Japan. Fish. Sci. 63:211-217.
TAJIMA, F., 1992 Statistical method for estimating the effective population size in Pacific salmon. J. Hered. 83:309-311
UTTER, F., 1998 Genetic problems of hatchery-reared progeny released into the wild, and how to deal with them. Bull. Mar. Sci. 62(2):623-640.
WALTERS, C. J., 1988 Mixed-stock fisheries and the sustainability of enhancement production for chinook and coho salmon, pp. 109115 in Salmon Production, Management, and Allocation, edited by W. J. MCNEIL. Oregon State University Press, Corvallis, OR.
WAPLES, R. S., 1989 A general approach for estimating effective population size from temporal changes in allele frequencies. Genetics 121:379-391
WAPLES, R. S., 1990 Conservation genetics of Pacific salmon: III. Estimating effective population size. J. Hered. 81:277-289
WAPLES, R. S., 1991 Genetics interactions between hatchery and wild salmonids: lessons from the Pacific Northwest. Can. J. Fish. Aquat. Sci. 48(Suppl. 1):124-133.
WAPLES, R. S., 1999 Dispelling some myths about hatcheries. Fisheries 24(2):12-21.
WEIR, B. S., 1996 Genetic data analysis II. Sinauer Associates, Sunderland, MA.
WILLIAMSON, E. G. and M. SLATKIN, 1999 Using maximum likelihood to estimate population size from temporal changes in allele frequencies. Genetics 152:755-761
WRIGHT, S., 1931 Evolution in Mendelian populations. Genetics 16:97-159
WRIGHT, S., 1945 The differential equation of the distribution of gene frequencies. Proc. Natl. Acad. Sci. USA 31:383-389.
WRIGHT, S., 1951 The general structure of populations. Ann. Eugen. 15:323-354.
WRIGHT, S., 1969 Evolution and Genetics of Populations: The Theory of Gene Frequencies, Vol. 2. University of Chicago Press, Chicago.
YOSHIZAWA, K., 1997 Inbreeding of cultured ayu from allozyme allele frequencies, Heisei 8 nendo seitaikei hozengata syubyo seisan gijyutu kaihatsu kenkyu seika no gaiyo, pp. 3038. Fishery Agency of Japan, Tokyo (in Japanese).
This article has been cited by other articles:
![]() |
S. Kitada, T. Kitakado, and H. Kishino Empirical Bayes Inference of Pairwise FST and Its Distribution in the Genome Genetics, October 1, 2007; 177(2): 861 - 873. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kitakado, S. Kitada, H. Kishino, and H. J. Skaug An Integrated-Likelihood Method for Estimating Genetic Differentiation Between Populations Genetics, August 1, 2006; 173(4): 2073 - 2082. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Kitada and H. Kishino Simultaneous Detection of Linkage Disequilibrium and Genetic Differentiation of Subdivided Populations Genetics, August 1, 2004; 167(4): 2003 - 2013. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Laval, M. SanCristobal, and C. Chevalet Maximum-Likelihood and Markov Chain Monte Carlo Approaches to Estimate Inbreeding and Effective Size From Allele Frequency Changes Genetics, July 1, 2003; 164(3): 1189 - 1204. [Abstract] [Full Text] [PDF] |
||||
![]() |
Bayesian Analysis of Genetic Differentiation Between Populations Genetics, January 1, 2003; 163(1): 367 - 374. |
||||
- THIS ARTICLE
-
Abstract
- Full Text (PDF)
- Alert me when this article is cited
- Alert me if a correction is posted
- SERVICES
- Email this article to a friend
- Similar articles in this journal
- Similar articles in PubMed
- Alert me to new issues of the journal
- Download to citation manager
- Reprints & Permissions
- CITING ARTICLES
- Citing Articles via HighWire
- Citing Articles via Google Scholar
- GOOGLE SCHOLAR
- Articles by Kitada, S.
- Articles by Kishino, H.
- Search for Related Content
- PUBMED
- PubMed Citation
- Articles by Kitada, S.
- Articles by Kishino, H.





















