Originally published as Genetics Published Articles Ahead of Print on June 4, 2006.

Genetics, Vol. 173, 2073-2082, August 2006, Copyright © 2006
doi:10.1534/genetics.106.055350

An Integrated-Likelihood Method for Estimating Genetic Differentiation Between Populations

* Faculty of Marine Science, Tokyo University of Marine Science and Technology, Minato, Tokyo 108-8477, Japan, {dagger} Graduate School of Agriculture and Life Sciences, University of Tokyo, Bunkyo, Tokyo 113-8657, Japan and {ddagger} Department of Mathematics, University of Bergen, 5008 Bergen, Norway

1 Corresponding author: Tokyo University of Marine Science and Technology, 5-7, Konan 4, Minato-ku, Tokyo, 108-8477, Japan.
E-mail: kitakado{at}s.kaiyodai.ac.jp

The aim of this article is to develop an integrated-likelihood (IL) approach to estimate the genetic differentiation between populations. The conventional maximum-likelihood (ML) and pseudolikelihood (PL) methods that use sample counts of alleles may cause severe underestimations of FST, which means overestimations of {theta} = 4Nm, when the number of sampling localities is small. To reduce such bias in the estimation of genetic differentiation, we propose an IL method in which the mean allele frequencies over populations are regarded as nuisance parameters and are eliminated by integration. To maximize the IL function, we have developed two algorithms, a Monte Carlo EM algorithm and a Laplace approximation. Our simulation studies show that the method proposed here outperforms the conventional ML and PL methods in terms of unbiasedness and precision. The IL method was applied to real data for Pacific herring and African elephants.




This article has been cited by other articles:


Home page
GeneticsHome page
S. Kitada, T. Kitakado, and H. Kishino
Empirical Bayes Inference of Pairwise FST and Its Distribution in the Genome
Genetics, October 1, 2007; 177(2): 861 - 873.
[Abstract] [Full Text] [PDF]


Home page
GeneticsHome page
T. Kitakado, S. Kitada, Y. Obata, and H. Kishino
Simultaneous Estimation of Mixing Rates and Genetic Drift Under Successive Sampling of Genetic Markers With Application to the Mud Crab (Scylla paramamosain) in Japan.
Genetics, August 1, 2006; 173(4): 2063 - 2072.
[Abstract] [Full Text] [PDF]