Paul Lewis - Revision history

PaulLewis at 14:37, 19 January 2011

2011-01-19T14:37:15Z

PaulLewis: /* Bayesian Model Selection */

2011-01-19T14:36:15Z

Bayesian Model Selection

PaulLewis: /* Bayesian Star Tree Paradox */

2011-01-19T14:23:39Z

Bayesian Star Tree Paradox

PaulLewis: /* Bayesian Star Tree Paradox */

2011-01-19T14:23:22Z

Bayesian Star Tree Paradox

PaulLewis: /* Bayesian Star Tree Paradox */

2011-01-19T14:22:55Z

Bayesian Star Tree Paradox

PaulLewis at 14:22, 19 January 2011

2011-01-19T14:22:31Z

PaulLewis: /* Bayesian Model Selection */

2011-01-19T14:15:55Z

Bayesian Model Selection

PaulLewis: /* Publications */

2011-01-19T14:13:54Z

Publications

PaulLewis: /* Publications */

2011-01-19T14:13:22Z

Publications

PaulLewis: /* Publications */

2011-01-19T14:02:47Z

Publications

@@ Line 28: / Line 28: @@
 [[Image:Steppingstonemethod.png|right]]Most recently, in collaboration with Ming-Hui Chen and Lynn Kuo in the UConn Department of Statistics, we've been working on improving methods for estimating the '''marginal likelihood''' of a model. The marginal likelihood is used in Bayesian inference to compare model fit. Comparing two models, the one with the higher marginal likelihood can be viewed as fitting the data better overall. The commonly-used harmonic mean method is biased, tending to overestimate the fit of a model. This can lead to selection of models that are overparameterized, the consequences of which include longer run times for MCMC analyses and, potentially, poor parameter estimates for some part of the model. Our new method for estimating marginal likelihoods is called '''steppingstone sampling''' (or SS for short). SS is much more reliable than the harmonic mean (HM) method, and is as accurate as thermodynamic integration, which is an alternative estimation method developed by Nicolas Lartillot and Herve Phillippe (see Lartillot and Philippe. 2006. Computing Bayes factors using thermodynamic integration. Systematic Biology 55(2):195-207). We anticipate that using SS will have the most impact on partitioned analyses where HM often suggests that the most-partitioned model is best. SS is currently (as of Feb. 2010) being incorporated into the software [http://phycas.org Phycas] so that others can try it.
-''References:''
+''Reference:'' Xie, W., '''P. O. Lewis''', '''Y. Fan''', L. Kuo, and M.-H. Chen. Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology (in press). doi:10.1093/sysbio/syq085
 <br clear="left"/>

@@ Line 26: / Line 26: @@
 === Bayesian Model Selection ===
-[[Image:Steppingstonemethod.jpg|left]]Most recently, in collaboration with Ming-Hui Chen and Lynn Kuo in the UConn Department of Statistics, we've been working on improving methods for estimating the '''marginal likelihood''' of a model. The marginal likelihood is used in Bayesian inference to compare model fit. Comparing two models, the one with the higher marginal likelihood can be viewed as fitting the data better overall. The commonly-used harmonic mean method is biased, tending to overestimate the fit of a model. This can lead to selection of models that are overparameterized, the consequences of which include longer run times for MCMC analyses and, potentially, poor parameter estimates for some part of the model. Our new method for estimating marginal likelihoods is called '''steppingstone sampling''' (or SS for short). SS is much more reliable than the harmonic mean (HM) method, and is as accurate as thermodynamic integration, which is an alternative estimation method developed by Nicolas Lartillot and Herve Phillippe (see Lartillot and Philippe. 2006. Computing Bayes factors using thermodynamic integration. Systematic Biology 55(2):195-207). We anticipate that using SS will have the most impact on partitioned analyses where HM often suggests that the most-partitioned model is best. SS is currently (as of Feb. 2010) being incorporated into the software [http://phycas.org Phycas] so that others can try it.
+[[Image:Steppingstonemethod.png|right]]Most recently, in collaboration with Ming-Hui Chen and Lynn Kuo in the UConn Department of Statistics, we've been working on improving methods for estimating the '''marginal likelihood''' of a model. The marginal likelihood is used in Bayesian inference to compare model fit. Comparing two models, the one with the higher marginal likelihood can be viewed as fitting the data better overall. The commonly-used harmonic mean method is biased, tending to overestimate the fit of a model. This can lead to selection of models that are overparameterized, the consequences of which include longer run times for MCMC analyses and, potentially, poor parameter estimates for some part of the model. Our new method for estimating marginal likelihoods is called '''steppingstone sampling''' (or SS for short). SS is much more reliable than the harmonic mean (HM) method, and is as accurate as thermodynamic integration, which is an alternative estimation method developed by Nicolas Lartillot and Herve Phillippe (see Lartillot and Philippe. 2006. Computing Bayes factors using thermodynamic integration. Systematic Biology 55(2):195-207). We anticipate that using SS will have the most impact on partitioned analyses where HM often suggests that the most-partitioned model is best. SS is currently (as of Feb. 2010) being incorporated into the software [http://phycas.org Phycas] so that others can try it.
-<br clear="left"/>=== Bayesian Star Tree Paradox ===
+''References:''
-[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
+<br clear="left"/>
 We argue that the central problem here is the non-identifiability of the tree topology, and propose a solution using reversible-jump MCMC. Our rjMCMC sampler visits not only fully-resolved tree topologies, but can visit topologies containing hard polytomies as well. This effectively places a point mass prior probability on polytomies, providing an alternative in situations in which a fully-resolved topology is not a reasonable option. The analysis can be made as conservative as desired by modifying the prior distribution assumed for topologies, but in our (albeit limited) experience it does not appear easy to destroy support for real edges by using a prior that strongly supports polytomous topologies.

@@ Line 28: / Line 28: @@
 [[Image:Steppingstonemethod.jpg|left]]Most recently, in collaboration with Ming-Hui Chen and Lynn Kuo in the UConn Department of Statistics, we've been working on improving methods for estimating the '''marginal likelihood''' of a model. The marginal likelihood is used in Bayesian inference to compare model fit. Comparing two models, the one with the higher marginal likelihood can be viewed as fitting the data better overall. The commonly-used harmonic mean method is biased, tending to overestimate the fit of a model. This can lead to selection of models that are overparameterized, the consequences of which include longer run times for MCMC analyses and, potentially, poor parameter estimates for some part of the model. Our new method for estimating marginal likelihoods is called '''steppingstone sampling''' (or SS for short). SS is much more reliable than the harmonic mean (HM) method, and is as accurate as thermodynamic integration, which is an alternative estimation method developed by Nicolas Lartillot and Herve Phillippe (see Lartillot and Philippe. 2006. Computing Bayes factors using thermodynamic integration. Systematic Biology 55(2):195-207). We anticipate that using SS will have the most impact on partitioned analyses where HM often suggests that the most-partitioned model is best. SS is currently (as of Feb. 2010) being incorporated into the software [http://phycas.org Phycas] so that others can try it.
-=== Bayesian Star Tree Paradox ===
+<br clear="left"/>=== Bayesian Star Tree Paradox ===
-<br clear="left"/>[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
+[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
 We argue that the central problem here is the non-identifiability of the tree topology, and propose a solution using reversible-jump MCMC. Our rjMCMC sampler visits not only fully-resolved tree topologies, but can visit topologies containing hard polytomies as well. This effectively places a point mass prior probability on polytomies, providing an alternative in situations in which a fully-resolved topology is not a reasonable option. The analysis can be made as conservative as desired by modifying the prior distribution assumed for topologies, but in our (albeit limited) experience it does not appear easy to destroy support for real edges by using a prior that strongly supports polytomous topologies.

@@ Line 30: / Line 30: @@
 === Bayesian Star Tree Paradox ===
-[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
+<br clear="left"/>[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
 We argue that the central problem here is the non-identifiability of the tree topology, and propose a solution using reversible-jump MCMC. Our rjMCMC sampler visits not only fully-resolved tree topologies, but can visit topologies containing hard polytomies as well. This effectively places a point mass prior probability on polytomies, providing an alternative in situations in which a fully-resolved topology is not a reasonable option. The analysis can be made as conservative as desired by modifying the prior distribution assumed for topologies, but in our (albeit limited) experience it does not appear easy to destroy support for real edges by using a prior that strongly supports polytomous topologies.

@@ Line 30: / Line 30: @@
 === Bayesian Star Tree Paradox ===
-[[Image:startree.gif|right]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
+[[Image:startree.gif|left]]If sequence data are simulated using a 4-taxon star tree (such as the one shown on the right) and evaluated with standard software tools for Bayesian phylogenetic inference, one of the 3 possible fully-resolved trees is often supported very strongly. This is paradoxical in that most people expect the three possible resolutions to be equally supported in this case, but such an outcome is only seen when the sequence length is tiny (e.g. 1 site). It appears that uncertainty in this case is manifested in the inability to predict, from dataset to dataset, which of the 3 possible fully-resolved tree topologies will be favored. This behavior is troubling, and possible examples of this behavior have been pointed out by several researchers. Many more potential examples can be found in the literature by looking for high posterior probabilities but low bootstrap support, combined with tiny internal edges.
 We argue that the central problem here is the non-identifiability of the tree topology, and propose a solution using reversible-jump MCMC. Our rjMCMC sampler visits not only fully-resolved tree topologies, but can visit topologies containing hard polytomies as well. This effectively places a point mass prior probability on polytomies, providing an alternative in situations in which a fully-resolved topology is not a reasonable option. The analysis can be made as conservative as desired by modifying the prior distribution assumed for topologies, but in our (albeit limited) experience it does not appear easy to destroy support for real edges by using a prior that strongly supports polytomous topologies.

@@ Line 48: / Line 48: @@
 Xie, W., '''P. O. Lewis''', '''Y. Fan''', L. Kuo, and M.-H. Chen. Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology (in press). doi:10.1093/sysbio/syq085
-'''Fan, Y.''', R. Wu, M.-H. Chen, L. Kuo and '''P. O. Lewis'''. 2011. Choosing among partition models in Bayesian Phylogenetics. Molecular Biology and Evolution 28(1):523-532. doi:10.1093/molbev/msq224 [http://mbe.oxfordjournals.org/content/early/2010/08/27/molbev.msq224.short?rss=1 pdf]
+'''Fan, Y.''', R. Wu, M.-H. Chen, L. Kuo and '''P. O. Lewis'''. 2011. Choosing among partition models in Bayesian Phylogenetics. Molecular Biology and Evolution 28(1):523-532. doi:10.1093/molbev/msq224 [http://mbe.oxfordjournals.org/content/early/2010/08/27/molbev.msq224.short?rss=1 Open Access]
 '''Holder, M. T'''., '''P. O. Lewis''', and D. L. Swofford.  2010.  The Akaike Information Criterion will not choose the no common mechanism model. Systematic Biology 59(4):477–485. doi:10.1093/sysbio/syq028

@@ Line 44: / Line 44: @@
 === Publications ===
-(Lab members (current or past) are indicated in bold.)
+(Current or past lab members are indicated in bold.)
-Xie, W., '''P. O. Lewis''', '''Y. Fan''', L. Kuo, and M.-H. Chen. Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology (in review).
+Xie, W., '''P. O. Lewis''', '''Y. Fan''', L. Kuo, and M.-H. Chen. Improving marginal likelihood estimation for Bayesian phylogenetic model selection. Systematic Biology (in press). doi:10.1093/sysbio/syq085
-'''Holder, M. T'''., '''P. O. Lewis''', and D. L. Swofford.  2010.  The AIC will not choose the no common mechanism model. Systematic Biology (in press).
+'''Fan, Y.''', R. Wu, M.-H. Chen, L. Kuo and '''P. O. Lewis'''. 2011. Choosing among partition models in Bayesian Phylogenetics. Molecular Biology and Evolution 28(1):523-532. doi:10.1093/molbev/msq224 {{pdf|http://mbe.oxfordjournals.org/content/early/2010/08/27/molbev.msq224.short?rss=1}}
-'''Holder, M. T.''', J. Sukumaran, and '''P. O. Lewis'''. 2008. A justification for reporting the majority-rule consensus tree in Bayesian phylogenetics. Systematic Biology 57(5):814–821.
+'''Holder, M. T'''., '''P. O. Lewis''', and D. L. Swofford.  2010.  The Akaike Information Criterion will not choose the no common mechanism model. Systematic Biology 59(4):477–485. doi:10.1093/sysbio/syq028
 Wickett, N. J., '''Y. Fan''', '''P. O. Lewis''', and B. Goffinet. 2008. Distribution and evolution of pseudogenes, gene losses, and a gene rearrangement in the plastid genome of the nonphotosynthetic liverwort, ''Aneura mirabilis'' (Metzgeriales, Jungermanniopsida). Journal of Molecular Evolution 67(1): 111-122 [http://www.springerlink.com/content/j7523168151443mk/fulltext.pdf link]