<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Evidence Testing</title>
	<atom:link href="http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/feed/" rel="self" type="application/rss+xml" />
	<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/</link>
	<description>Random samplings from a universe of ideas.</description>
	<lastBuildDate>Tue, 14 Feb 2012 15:45:35 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
	<item>
		<title>By: TorbjÃ¶rn Larsson</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15197</link>
		<dc:creator>TorbjÃ¶rn Larsson</dc:creator>
		<pubDate>Sat, 29 Apr 2006 06:04:43 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15197</guid>
		<description>Daryl,
Thank you for your response!

I was unclear. I was referring to &quot;all possible physical theories (there are only countably many, if they are to be expressed in finitely many symbols)&quot;, which seems to be the same object Nelson uses. He takes that finiteness to imply that scientific naturalism can&#039;t explain a possibly infinite amount of data. One can analyse this in several ways; my first reaction was that I remembered someone using GÃ¶del to argue that physics is forever extendable if necessary.

I see that you mean now by not needing falsifiability in your analysis. I still feel that Occam tells us to junk completely the theories that have low probability. I&#039;ll have to think about that.</description>
		<content:encoded><![CDATA[<p>Daryl,<br />
Thank you for your response!</p>
<p>I was unclear. I was referring to &#8220;all possible physical theories (there are only countably many, if they are to be expressed in finitely many symbols)&#8221;, which seems to be the same object Nelson uses. He takes that finiteness to imply that scientific naturalism can&#8217;t explain a possibly infinite amount of data. One can analyse this in several ways; my first reaction was that I remembered someone using GÃ¶del to argue that physics is forever extendable if necessary.</p>
<p>I see that you mean now by not needing falsifiability in your analysis. I still feel that Occam tells us to junk completely the theories that have low probability. I&#8217;ll have to think about that.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Andrew Jaffe: Leaves on the Line</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15196</link>
		<dc:creator>Andrew Jaffe: Leaves on the Line</dc:creator>
		<pubDate>Fri, 28 Apr 2006 14:30:06 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15196</guid>
		<description>&lt;strong&gt;What We Talk About When We Talk About Probability&lt;/strong&gt;

In his most recent post, Cosmic Variance&#039;s Mark Trodden talks about one of the presentations we both saw at last week&#039;s meeting in Ishcia, where he explains one of the hot new techniques for analyzing cosmological data, the (so-called) Bayesian Evide...</description>
		<content:encoded><![CDATA[<p><strong>What We Talk About When We Talk About Probability</strong></p>
<p>In his most recent post, Cosmic Variance&#8217;s Mark Trodden talks about one of the presentations we both saw at last week&#8217;s meeting in Ishcia, where he explains one of the hot new techniques for analyzing cosmological data, the (so-called) Bayesian Evide&#8230;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Daryl McCullough</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15195</link>
		<dc:creator>Daryl McCullough</dc:creator>
		<pubDate>Fri, 28 Apr 2006 02:58:39 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15195</guid>
		<description>TorbjÃ¶rn,

I have no idea how Bayesian analysis of theories tells us anything about the truth or falsity of naturalism. I&#039;ll have to think about the implications of Godel&#039;s theorem.

&lt;i&gt;Second, you use some sort of Bayesian metaanalysis to put off falsifiability. Even though a frequentist may say Bayesian probabilities are about beliefs, apparently these beliefs enable decisions for setting parameters (to 5 sigma, say). Isn&#039;t it reasonable to do that for each theory individually (and therefore get falsifiability)? Anyway, shouldn&#039;t the razor support the use of one specific theory at a time?&lt;/i&gt;

The point I was making is that (1) you can never &lt;i&gt;completely&lt;/i&gt; falsify any theory, and (2) there is no &lt;i&gt;need&lt;/i&gt; to falsify theories. If you need to know the magnetic moment of the electron then you can just perform the weighted average of the predictions made by all possible theories (weighted by the likelihood that each theory is true). If one theory (for instance, QED) is much better supported than any of the competing theories, then its posterior probability will be almost 1, and so its predicted value will dominate. You don&#039;t ever need to consider which theory is best supported by the evidence, because the Bayesian probabilities automatically take all evidence into account.

Now, of course in practice we can&#039;t keep track of all possible theories and all the evidence for and against each, so we simplify matters by throwing out all except for one or two likely candidates. But if the complete Bayesian analysis were &lt;i&gt;possible&lt;/i&gt; to do, it&#039;s hard to see how it would ever give us a worse prediction than our current approach (except by luck).</description>
		<content:encoded><![CDATA[<p>TorbjÃ¶rn,</p>
<p>I have no idea how Bayesian analysis of theories tells us anything about the truth or falsity of naturalism. I&#8217;ll have to think about the implications of Godel&#8217;s theorem.</p>
<p><i>Second, you use some sort of Bayesian metaanalysis to put off falsifiability. Even though a frequentist may say Bayesian probabilities are about beliefs, apparently these beliefs enable decisions for setting parameters (to 5 sigma, say). Isn&#8217;t it reasonable to do that for each theory individually (and therefore get falsifiability)? Anyway, shouldn&#8217;t the razor support the use of one specific theory at a time?</i></p>
<p>The point I was making is that (1) you can never <i>completely</i> falsify any theory, and (2) there is no <i>need</i> to falsify theories. If you need to know the magnetic moment of the electron then you can just perform the weighted average of the predictions made by all possible theories (weighted by the likelihood that each theory is true). If one theory (for instance, QED) is much better supported than any of the competing theories, then its posterior probability will be almost 1, and so its predicted value will dominate. You don&#8217;t ever need to consider which theory is best supported by the evidence, because the Bayesian probabilities automatically take all evidence into account.</p>
<p>Now, of course in practice we can&#8217;t keep track of all possible theories and all the evidence for and against each, so we simplify matters by throwing out all except for one or two likely candidates. But if the complete Bayesian analysis were <i>possible</i> to do, it&#8217;s hard to see how it would ever give us a worse prediction than our current approach (except by luck).</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: TorbjÃ¶rn Larsson</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15194</link>
		<dc:creator>TorbjÃ¶rn Larsson</dc:creator>
		<pubDate>Thu, 27 Apr 2006 15:31:15 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15194</guid>
		<description>Apparently Occam&#039;s razor gives several advantages. Here is a similar paper that also uses two other quantitative measures besides the Bayesian Information Criterion to look at WMAP data http://arxiv.org/PS_cache/astro-ph/pdf/0604/0604410.pdf to make the result more robust. I have also seen a philosophic paper showing the razor gives fewer reversals of hypotheses. (And of course we observe the simpler theory often works.)

Daryl,
I have two questions you could help me with.

First and foremost, a creationist philosopher (Paul Nelson) use much the same construction as you do to arge that naturalism isn&#039;t sufficient for science. I have earlier seen GÃ¶dels first incompleteness theorem used to argue that since even some simple formal systems can not be given a computably enumerable axiom list, physics will be forever extendable. Which is it? I would like to see if Nelson is wrong here.

Second, you use some sort of Bayesian metaanalysis to put off falsifiability. Even though a frequentist may say Bayesian probabilities are about beliefs, apparently these beliefs enable decisions for setting parameters (to 5 sigma, say). Isn&#039;t it reasonable to do that for each theory individually (and therefore get falsifiability)? Anyway, shouldn&#039;t the razor support the use of one specific theory at a time?</description>
		<content:encoded><![CDATA[<p>Apparently Occam&#8217;s razor gives several advantages. Here is a similar paper that also uses two other quantitative measures besides the Bayesian Information Criterion to look at WMAP data <a href="http://arxiv.org/PS_cache/astro-ph/pdf/0604/0604410.pdf" rel="nofollow">http://arxiv.org/PS_cache/astro-ph/pdf/0604/0604410.pdf</a> to make the result more robust. I have also seen a philosophic paper showing the razor gives fewer reversals of hypotheses. (And of course we observe the simpler theory often works.)</p>
<p>Daryl,<br />
I have two questions you could help me with.</p>
<p>First and foremost, a creationist philosopher (Paul Nelson) use much the same construction as you do to arge that naturalism isn&#8217;t sufficient for science. I have earlier seen GÃ¶dels first incompleteness theorem used to argue that since even some simple formal systems can not be given a computably enumerable axiom list, physics will be forever extendable. Which is it? I would like to see if Nelson is wrong here.</p>
<p>Second, you use some sort of Bayesian metaanalysis to put off falsifiability. Even though a frequentist may say Bayesian probabilities are about beliefs, apparently these beliefs enable decisions for setting parameters (to 5 sigma, say). Isn&#8217;t it reasonable to do that for each theory individually (and therefore get falsifiability)? Anyway, shouldn&#8217;t the razor support the use of one specific theory at a time?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Count Iblis</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15193</link>
		<dc:creator>Count Iblis</dc:creator>
		<pubDate>Thu, 27 Apr 2006 13:10:41 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15193</guid>
		<description>Bayesian analysis can, in principle, be unambiguous in a multiverse setting, if you had all the information available about the multiverse. You would then know what the prior probabilities are and then you could use observations to update those probabilities unambiguously.</description>
		<content:encoded><![CDATA[<p>Bayesian analysis can, in principle, be unambiguous in a multiverse setting, if you had all the information available about the multiverse. You would then know what the prior probabilities are and then you could use observations to update those probabilities unambiguously.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mostafa</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15192</link>
		<dc:creator>Mostafa</dc:creator>
		<pubDate>Thu, 27 Apr 2006 08:26:36 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15192</guid>
		<description>It is interesting that Bayesian analysis, whose basics was actually worked out in the 18th century, became popular and widely used in economics and science (high energy physics and cosmology as far as I know) in the 20th century, and seems to become a basic method of use in the near future. This process of coinciding old mathematics with new natural science needs, happens commonly in algebra, calculus and geometry, but much less often in statistics.
It seems to me thet the growing attention to such methods like Bayesian analysis in statistics and data analysis, and to Monte Carlo methods in computation (and also toward using these two techniques together), whose basic ideas are relatively simple, but applied techniques are very much free to alter and invent, is the sign of a new way of using math in physics in the future.</description>
		<content:encoded><![CDATA[<p>It is interesting that Bayesian analysis, whose basics was actually worked out in the 18th century, became popular and widely used in economics and science (high energy physics and cosmology as far as I know) in the 20th century, and seems to become a basic method of use in the near future. This process of coinciding old mathematics with new natural science needs, happens commonly in algebra, calculus and geometry, but much less often in statistics.<br />
It seems to me thet the growing attention to such methods like Bayesian analysis in statistics and data analysis, and to Monte Carlo methods in computation (and also toward using these two techniques together), whose basic ideas are relatively simple, but applied techniques are very much free to alter and invent, is the sign of a new way of using math in physics in the future.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Sean</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15190</link>
		<dc:creator>Sean</dc:creator>
		<pubDate>Thu, 27 Apr 2006 03:20:30 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15190</guid>
		<description>Isn&#039;t it great that you can tell a student to go learn something and then teach it to you?  Let this be a lesson to all those students out there working hard to become professors some day -- stick with it, the ultimate rewards make it all worthwhile.</description>
		<content:encoded><![CDATA[<p>Isn&#8217;t it great that you can tell a student to go learn something and then teach it to you?  Let this be a lesson to all those students out there working hard to become professors some day &#8212; stick with it, the ultimate rewards make it all worthwhile.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Daryl McCullough</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15191</link>
		<dc:creator>Daryl McCullough</dc:creator>
		<pubDate>Thu, 27 Apr 2006 03:19:12 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15191</guid>
		<description>Just musing about Bayesian probability...

Physicists tend to think in terms of theories being completely overthrown by some observation, but there is a sense in which no single observation is ever &lt;i&gt;really&lt;/i&gt; sufficient to reject a theory. It&#039;s always possible that any single observation, or even any finite &lt;i&gt;collection&lt;/i&gt; of observations is a fluke. So there really is a subjective judgement going on in the background. How improbable must a fluke be before you are convinced that the theory was wrong, and the contrary evidence &lt;i&gt;wasn&#039;t&lt;/i&gt; just a fluke?

It seems to me that Bayesian probabilities allow you in principle to put off &lt;i&gt;forever&lt;/i&gt; making the decision as to whether a theory has been falsified or not. Instead, you can imagine starting with all possible physical theories (there are only countably many, if they are to be expressed in finitely many symbols), with some subjective a priori notion of likelihood. Then every experiment performed can be used to adjust the posterior probabilities.

With this approach, as time goes on, Newtonian physics would start off with a low probability, but then would gradually rise with the successful predictions about projectiles and planetary motion, but then would drop sharply with the discoveries of quantum and relativistic phenomena. But it would never go to zero.

If you are trying to make a prediction, you would weight the predictions made by every conceivable theory, according to the posterior likelihood of each theory.</description>
		<content:encoded><![CDATA[<p>Just musing about Bayesian probability&#8230;</p>
<p>Physicists tend to think in terms of theories being completely overthrown by some observation, but there is a sense in which no single observation is ever <i>really</i> sufficient to reject a theory. It&#8217;s always possible that any single observation, or even any finite <i>collection</i> of observations is a fluke. So there really is a subjective judgement going on in the background. How improbable must a fluke be before you are convinced that the theory was wrong, and the contrary evidence <i>wasn&#8217;t</i> just a fluke?</p>
<p>It seems to me that Bayesian probabilities allow you in principle to put off <i>forever</i> making the decision as to whether a theory has been falsified or not. Instead, you can imagine starting with all possible physical theories (there are only countably many, if they are to be expressed in finitely many symbols), with some subjective a priori notion of likelihood. Then every experiment performed can be used to adjust the posterior probabilities.</p>
<p>With this approach, as time goes on, Newtonian physics would start off with a low probability, but then would gradually rise with the successful predictions about projectiles and planetary motion, but then would drop sharply with the discoveries of quantum and relativistic phenomena. But it would never go to zero.</p>
<p>If you are trying to make a prediction, you would weight the predictions made by every conceivable theory, according to the posterior likelihood of each theory.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mark</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15188</link>
		<dc:creator>Mark</dc:creator>
		<pubDate>Thu, 27 Apr 2006 00:16:22 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15188</guid>
		<description>Thanks not a statistician. Indeed, the devil is in the priors for different degrees of freedom as you say.</description>
		<content:encoded><![CDATA[<p>Thanks not a statistician. Indeed, the devil is in the priors for different degrees of freedom as you say.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: not a statistician</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/comment-page-1/#comment-15189</link>
		<dc:creator>not a statistician</dc:creator>
		<pubDate>Thu, 27 Apr 2006 00:05:25 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2006/04/26/evidence-testing/#comment-15189</guid>
		<description>The issue of defining how good a fit is(or how uncertain based on the std dev) while penalizing for a larger number of degrees of freedom is well defined without having to discuss anything Bayesian.  If you are using the chi squared to define a confidence interval then the chi squared you use to define the 1 \sigma level or whatnot depends on the number of degrees of freedom in your fit.  However Bayesianism reallly enters when you want to start treating these degrees of freedom differently and come up with priors which then makes the whole issue of confidence intervals trickier as you alluded to.</description>
		<content:encoded><![CDATA[<p>The issue of defining how good a fit is(or how uncertain based on the std dev) while penalizing for a larger number of degrees of freedom is well defined without having to discuss anything Bayesian.  If you are using the chi squared to define a confidence interval then the chi squared you use to define the 1 \sigma level or whatnot depends on the number of degrees of freedom in your fit.  However Bayesianism reallly enters when you want to start treating these degrees of freedom differently and come up with priors which then makes the whole issue of confidence intervals trickier as you alluded to.</p>
]]></content:encoded>
	</item>
</channel>
</rss>

<!-- Performance optimized by W3 Total Cache. Learn more: http://www.w3-edge.com/wordpress-plugins/

Minified using disk
Page Caching using disk

Served from: blogs.discovermagazine.com @ 2012-02-14 15:50:54 -->
