Genetics. Published Articles Ahead of Print: June 4, 2006, Copyright © 2006
doi:10.1534/genetics.106.055350


A more recent version of this article appeared on August 1, 2006.


REGULAR RESEARCH PAPERS

An integrated likelihood method for estimating genetic differentiation between populations

1 Tokyo University of Marine Science and Technology
2 University of Tokyo
3 University of Bergen

* To whom correspondence should be addressed. E-mail: kitakado{at}s.kaiyodai.ac.jp.

Submitted on January 3, 2006
Revised on March 29, 2006
Accepted on 18 May 2006


Abstract

The aim of this paper is to develop an integrated likelihood (IL) approach to estimate the genetic differentiation between populations. The conventional maximum likelihood (ML) and pseudolikelihood (PL) methods that use sample counts of alleles may cause severe underestimations of FST, which means overestimations of {vartheta}=4Nm, when the number of sampling localities is small. To reduce such bias in the estimation of genetic differentiation, we propose an IL method in which the mean allele frequencies over populations are regarded as nuisance parameters and are eliminated by integration. To maximize the IL function, we have developed two algorithms, a Monte Carlo EM algorithm and a Laplace approximation. Our simulation studies show that the method proposed here outperforms the conventional ML and PL methods in terms of unbiasedness and precision. The IL method was applied to real data for Pacific herring and African elephants.

Key Words: Dirichlet-multinomial, FST, Genetic differentiation, Integrated likelihood, Metapopulations




This article has been cited by other articles:


Home page
GeneticsHome page
S. Kitada, T. Kitakado, and H. Kishino
Empirical Bayes Inference of Pairwise FST and Its Distribution in the Genome
Genetics, October 1, 2007; 177(2): 861 - 873.
[Abstract] [Full Text] [PDF]


Home page
GeneticsHome page
T. Kitakado, S. Kitada, Y. Obata, and H. Kishino
Simultaneous Estimation of Mixing Rates and Genetic Drift Under Successive Sampling of Genetic Markers With Application to the Mud Crab (Scylla paramamosain) in Japan.
Genetics, August 1, 2006; 173(4): 2063 - 2072.
[Abstract] [Full Text] [PDF]