<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Petabytes</title>
	<atom:link href="http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/feed/" rel="self" type="application/rss+xml" />
	<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/</link>
	<description>Random samplings from a universe of ideas.</description>
	<lastBuildDate>Tue, 24 Nov 2009 05:18:18 -0600</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Sean</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3013</link>
		<dc:creator>Sean</dc:creator>
		<pubDate>Wed, 07 Sep 2005 16:08:16 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3013</guid>
		<description>Removing the cosmological constant doesn&#039;t cause the universe to expand.  It can be either expanding or contracting, just like a ball in flight can be either going up or going down; at the moment it just happens to be expanding.  It is not true that &quot;Einstein&#039;s equations would have caused the theoretical universe to collapse&quot;; that would only be true if the universe &lt;em&gt;started&lt;/em&gt; in a stationary state, which it clearly didn&#039;t.</description>
		<content:encoded><![CDATA[<p>Removing the cosmological constant doesn&#8217;t cause the universe to expand.  It can be either expanding or contracting, just like a ball in flight can be either going up or going down; at the moment it just happens to be expanding.  It is not true that &#8220;Einstein&#8217;s equations would have caused the theoretical universe to collapse&#8221;; that would only be true if the universe <em>started</em> in a stationary state, which it clearly didn&#8217;t.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Annonymous</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3012</link>
		<dc:creator>Annonymous</dc:creator>
		<pubDate>Wed, 07 Sep 2005 15:36:15 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3012</guid>
		<description>Thanks Sean, but is the cc shifted from one side of the equation to other as needed?  If Einstein&#039;s equations would have caused the theoretical universe to collapse without the cc, then how is it that removing it, or setting it to zero, causes the universe to expand?  That&#039;s what seems inconsistent to me.</description>
		<content:encoded><![CDATA[<p>Thanks Sean, but is the cc shifted from one side of the equation to other as needed?  If Einstein&#8217;s equations would have caused the theoretical universe to collapse without the cc, then how is it that removing it, or setting it to zero, causes the universe to expand?  That&#8217;s what seems inconsistent to me.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Sean</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3011</link>
		<dc:creator>Sean</dc:creator>
		<pubDate>Tue, 06 Sep 2005 23:06:10 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3011</guid>
		<description>Anonymous, you can think of the universe as a ball in free-fall above the surface of the earth.  It will either be going up or coming down, but only be stationary for a second.  Likewise, without the cosmological constant, the universe can be expanding or contracting, but will only be static for an instant.  In the ball analogy, the role of the cosmological constant is played by a tiny booster rocket on the ball, that fires with an absolutely constant thrust.  You can now imagine a perfect balance between the tug of gravity and the thrust of the rocket, but it&#039;s an unstable configuration.  It&#039;s still more likely to see the ball either going up or going down, as we actually do observe the universe to be expanding, even with the cosmological constant.</description>
		<content:encoded><![CDATA[<p>Anonymous, you can think of the universe as a ball in free-fall above the surface of the earth.  It will either be going up or coming down, but only be stationary for a second.  Likewise, without the cosmological constant, the universe can be expanding or contracting, but will only be static for an instant.  In the ball analogy, the role of the cosmological constant is played by a tiny booster rocket on the ball, that fires with an absolutely constant thrust.  You can now imagine a perfect balance between the tug of gravity and the thrust of the rocket, but it&#8217;s an unstable configuration.  It&#8217;s still more likely to see the ball either going up or going down, as we actually do observe the universe to be expanding, even with the cosmological constant.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Annonymous</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3010</link>
		<dc:creator>Annonymous</dc:creator>
		<pubDate>Tue, 06 Sep 2005 22:19:00 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3010</guid>
		<description>Sean,

Can you explain to this simpleton why the CC is explained as a proposal by Einstein as a pressure &quot;to counteract gravity and keep the universe from imploding,&quot; when, actually, without it, the Einstein theoretical universe expands?  It seems to me, given the explanation that it was intended to prevent an implosion, setting it to zero would lead to a contracting universe, not an expanding one. When Einstein found out that the universe was actually expanding, wouldn&#039;t that have just called for a CC that exceeded gravity, not just balanced it? In other words, wouldn&#039;t it have required an increase in the value of the constant?</description>
		<content:encoded><![CDATA[<p>Sean,</p>
<p>Can you explain to this simpleton why the CC is explained as a proposal by Einstein as a pressure &#8220;to counteract gravity and keep the universe from imploding,&#8221; when, actually, without it, the Einstein theoretical universe expands?  It seems to me, given the explanation that it was intended to prevent an implosion, setting it to zero would lead to a contracting universe, not an expanding one. When Einstein found out that the universe was actually expanding, wouldn&#8217;t that have just called for a CC that exceeded gravity, not just balanced it? In other words, wouldn&#8217;t it have required an increase in the value of the constant?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Quantoken</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3009</link>
		<dc:creator>Quantoken</dc:creator>
		<pubDate>Tue, 06 Sep 2005 21:51:03 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3009</guid>
		<description>Sean:

    It&#039;s stupidity beyond imagination if one talks about having to survey the whole sky and accumulate terabytes of data just to identify a few supernovaes happening at the edge of the universe.

    The point is such events are NOT rare at all. If you are talking about supernovaes within no more than a few thousand light years away, it may be as rare as once in a couple hundred years. But when you are talking about any supernovae that can be as far as 10 or 14 billion light years away, then there&#039;s &lt;b&gt;plenty&lt;/b&gt; of such events happening &lt;b&gt;every split second&lt;/b&gt; at any patch of the sky you look at. So why do you have to survey the whole sky for such signals?

    The universe contains roughly 200 billion galaxies, each contain roughly 200 billion stars, and each has a good chance at a supernovae at the end of the star lifetime. So counting for the whole age of the universe, there has been roughly 200 billion X 200 billion supernovaes, about 4x10^22. That number is at least correct in terms of order of magnitude. The age of the universe is about 14 billion years, or about 4.4x10^17 seconds. So within any one second time period, there&#039;s roughtly 10^5 newly occured supernovaes some where in the universe, within a distance no more than 14 billion light years.

    If you look at just one small patch of the sky, roughly &quot;10 square degree&quot;, that&#039;s 2.5x10^-4 of the whole sky. It figures into roughly 40 new supernovaes per second just within the small patch of sky the LSST looks at. So it should instantaneously discover many many remote supernovaes, any patch of sky it looks at, any time it looks at, without the need to accumulate terabytes of data.

    And certainly, with such a huge collection of samples of terabyte scale, you can &lt;b&gt;ALWAYS&lt;/b&gt; find data of &lt;b&gt;ANY&lt;/b&gt; conceiveable particular strange patterns that fits &lt;b&gt;ANY&lt;/b&gt; speculative theory you may have. But such data mining can hardly be considered convincing scientific evidences.

Quantoken</description>
		<content:encoded><![CDATA[<p>Sean:</p>
<p>    It&#8217;s stupidity beyond imagination if one talks about having to survey the whole sky and accumulate terabytes of data just to identify a few supernovaes happening at the edge of the universe.</p>
<p>    The point is such events are NOT rare at all. If you are talking about supernovaes within no more than a few thousand light years away, it may be as rare as once in a couple hundred years. But when you are talking about any supernovae that can be as far as 10 or 14 billion light years away, then there&#8217;s <b>plenty</b> of such events happening <b>every split second</b> at any patch of the sky you look at. So why do you have to survey the whole sky for such signals?</p>
<p>    The universe contains roughly 200 billion galaxies, each contain roughly 200 billion stars, and each has a good chance at a supernovae at the end of the star lifetime. So counting for the whole age of the universe, there has been roughly 200 billion X 200 billion supernovaes, about 4&#215;10^22. That number is at least correct in terms of order of magnitude. The age of the universe is about 14 billion years, or about 4.4&#215;10^17 seconds. So within any one second time period, there&#8217;s roughtly 10^5 newly occured supernovaes some where in the universe, within a distance no more than 14 billion light years.</p>
<p>    If you look at just one small patch of the sky, roughly &#8220;10 square degree&#8221;, that&#8217;s 2.5&#215;10^-4 of the whole sky. It figures into roughly 40 new supernovaes per second just within the small patch of sky the LSST looks at. So it should instantaneously discover many many remote supernovaes, any patch of sky it looks at, any time it looks at, without the need to accumulate terabytes of data.</p>
<p>    And certainly, with such a huge collection of samples of terabyte scale, you can <b>ALWAYS</b> find data of <b>ANY</b> conceiveable particular strange patterns that fits <b>ANY</b> speculative theory you may have. But such data mining can hardly be considered convincing scientific evidences.</p>
<p>Quantoken</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Adam</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3008</link>
		<dc:creator>Adam</dc:creator>
		<pubDate>Tue, 06 Sep 2005 16:54:40 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3008</guid>
		<description>Fermilab already have those data issues, yeah. Relational Databases are getting quite a bit better too (The Sloan Digital Sky Survey data is stored in an off-the-shelf RDBMS, I believe, despite initial concerns that no OTS product would be sufficient; of course, SDSS has rather less than a petabyte of data). POSTGRESQL, I think, has no absolute limit on DB size, but does have a limit on table size (30 Tb or so?). SQL Server 2005 is size-limited at about a thousand petabytes(what is that, an exabyte?) but has no table size limit. Of course, your OS won&#039;t like it, and in any case, searching might take a long time, but for general search, relational DBs can be pretty good.

Will the reduced dataset really be a Petabyte, though? Searching raw data doesn&#039;t seem that likely to me.</description>
		<content:encoded><![CDATA[<p>Fermilab already have those data issues, yeah. Relational Databases are getting quite a bit better too (The Sloan Digital Sky Survey data is stored in an off-the-shelf RDBMS, I believe, despite initial concerns that no OTS product would be sufficient; of course, SDSS has rather less than a petabyte of data). POSTGRESQL, I think, has no absolute limit on DB size, but does have a limit on table size (30 Tb or so?). SQL Server 2005 is size-limited at about a thousand petabytes(what is that, an exabyte?) but has no table size limit. Of course, your OS won&#8217;t like it, and in any case, searching might take a long time, but for general search, relational DBs can be pretty good.</p>
<p>Will the reduced dataset really be a Petabyte, though? Searching raw data doesn&#8217;t seem that likely to me.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: James Amundson</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3007</link>
		<dc:creator>James Amundson</dc:creator>
		<pubDate>Tue, 06 Sep 2005 14:45:08 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3007</guid>
		<description>One shouldn&#039;t pretend these data volumes are unprecedented in physics experiments. A petabyte/year is the typical order of magnitude data rate for a modern large high-energy physics experiment. The two Tevatron experiments currently running at Fermilab, D0 and CDF, take roughly a petabyte/year combined. The LHC experiments, CMS and ATLAS will each take about a petabyte/year. They will start in a couple of years.</description>
		<content:encoded><![CDATA[<p>One shouldn&#8217;t pretend these data volumes are unprecedented in physics experiments. A petabyte/year is the typical order of magnitude data rate for a modern large high-energy physics experiment. The two Tevatron experiments currently running at Fermilab, D0 and CDF, take roughly a petabyte/year combined. The LHC experiments, CMS and ATLAS will each take about a petabyte/year. They will start in a couple of years.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Bruce Baumgart</title>
		<link>http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/comment-page-1/#comment-3006</link>
		<dc:creator>Bruce Baumgart</dc:creator>
		<pubDate>Tue, 06 Sep 2005 02:38:48 +0000</pubDate>
		<guid isPermaLink="false">http://blogs.discovermagazine.com/cosmicvariance/2005/09/05/petabytes/#comment-3006</guid>
		<description>In principle, one could grep almost all of the Internet Archive in a day or two. The IA is composed of some 2000 independent machines each with four large disks, 250 GB to 500 GB each; so the grep could be done in parallel. On any given day a number of machines and disks are not working. The data decompression of all the data would be the major bottleneck. -- Bruce Baumgart, Research Engineer at the Internet Archive.</description>
		<content:encoded><![CDATA[<p>In principle, one could grep almost all of the Internet Archive in a day or two. The IA is composed of some 2000 independent machines each with four large disks, 250 GB to 500 GB each; so the grep could be done in parallel. On any given day a number of machines and disks are not working. The data decompression of all the data would be the major bottleneck. &#8212; Bruce Baumgart, Research Engineer at the Internet Archive.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
