TABLE B3

Model selection with simulated data sets

True model
5i: αj = 05ii: αj = 0.405iii(a): αj ∼ pdfbeta5iii(b): αj ∼ pdfbeta5iv: αj ∼ pdfdelta
AICBICAICcAICBICAICcAICBICAICcAICBICAICcAICBICAICc
2iii0.080.080.08
0.100.670.100.990.100.810.88
2iv
0.740.21
5i0.930.990.98
0.930.160.96
5ii0.070.010.020.991.001.000.010.030.020.580.780.67
0.070.041.000.121.000.010.020.660.150.76
5iii0.290.320.290.020.030.02
0.080.010.090.01
5iv0.010.620.570.610.400.190.311.001.001.00
0.810.790.340.040.231.000.121.00
True model
10i: αj = 010ii: αj = 0.4010iii(a): αj ∼ pdfbeta10iii(b): αj ∼ pdfbeta10iv: αj ∼ pdfdelta
AICBICAICcAICBICAICcAICBICAICcAICBICAICcAICBICAICc
2iii0.150.150.150.010.010.01
0.090.090.09
0.691.000.611.000.120.800.980.720.990.510.95
5i0.03
0.02
5ii0.01
5iii0.020.280.020.08
0.010.06
5iv0.950.990.570.970.92
0.971.000.851.001.00
0.010.05
10i1.001.000.01
0.981.00
0.930.31
10ii0.970.990.320.380.840.970.020.15
0.021.001.000.370.450.910.970.160.22
0.071.000.390.630.190.990.270.980.49
10iii0.080.090.010.01
0.020.02
0.03
10iv0.030.450.380.140.010.980.85
0.520.440.090.030.840.78
0.220.010.010.010.02
  • The likelihood models chosen by various model selection criteria for simulated data sets are shown. Each entry shows the proportion of 100 simulated data sets for which a given likelihood model was selected, with a dash indicating that the model was never selected. Models that were not selected under any conditions are omitted. Entries are grouped according to the class of true model used to generate the simulated data (Table B1). Each section contains entries for the two (or three) methods used to generate the true fj-values (Table B2), with results for methods a, b, and c (if appropriate) given in descending order. The sections where the selected model agrees with the true model are underlined, and, when a single model was chosen for the great majority of data sets (>75%), this is indicated in italics.