Home About The Codist RSS Feed

What the Heck Could You Do With 16 Exabytes?
Jun 26, 2007 18:35 perm link Readers: 4182

In 64-bit architectures, the address space encompasses 16 exabytes. MacOS X 10.5, for example, can address this much as virtual memory, although it "only" supports 4 terabytes of physical RAM.

Given that my first computer only had 4K of RAM this is an amazing number of bytes. I thought my first hard drive, at a healthy 5 MB, was plenty big at the time.

To put an exabyte into perspective, the ramp up from my first hard drive to a terabyte (you can buy a terabyte drive these days for around $300 US) is the same order of magnitude from a terabyte to an exabyte. Each time hard drive technology improves you hear people wonder what good is so much hard drive space, yet we never seems to have any trouble finding a use.

A single movie compressed on a DVD is around 5GB, an HD movie around 25GB. Working with raw HD data (even that is usually compressed somewhat) you need much more. So imagine you need 500GB to store a modest HD movie during editing. In an exabyte you could work on 2 million HD projects at a time.

In the early 90's a copy of Deltagraph (Mac) was around 2.5 MB in size, which shipped on several floppy disks. In an exabyte I could keep 440 billion copies.

The MMPOG game I play (Battleground Europe) uses about 700MB RAM during gameplay. In an exabyte there would be room for 1.5 billion times more data.

Google's data for its search engine is apparently around 1 petabyte or so, a mere 0.1% of an exabyte. In a 64 bit address space you could fit it 16,000 or so times.

It's a big number, which seems pointless to consider: who would ever use this much data, either on disk or in virtual memory? Yet technology continues to discover reasons to use more and more storage. I can imagine that some day the division between permanent storage (disk) and RAM will vanish; everything you work with will exist in a single address space. In this way an exabyte doesn't seem as far off as it appears.

The nasty fly in this exabyte ointment is of course software. How do we develop software than can take advantage of almost limitless address space, not to mention tens, hundreds or even thousands of processors, or even enormous grids of these machines? Somehow software evolution has to speed up or all that hardware potential will be wasted.

Another fly people don't think about (but Google does) is power - at some point you have to provide juice for all these bytes.

So this is mere speculation for now, we still have a long way to go before a 64 bit address space seems tight. Perhaps by then we can simulate the human brain and let the computer figure out what to do next.

Of course by the time 128 bit address spaces are needed, we might all be obsolete and it won't matter.

My Tags:

  • Eric TF Bat: Jun 26, 2007 20:35

    The standard answer to any computer-related question in the form "What the Heck Could You Do With [some amount of memory capacity]" has remained constant since the invention of the computer: fill it up with porn!

    Well, that's the common answer. Me, I'd just run Emacs and it'd all fill up eventually without any further effort.

  • Sidu: Jun 27, 2007 03:27

    @Eric: lol. Good one.

  • gwenhwyfaer: Jun 27, 2007 09:31

    Not that it matters too much. The amount of memory that can keep up with the computer will probably stay the same 64k or so (give or take a couple of powers of 2) that it's been for the last 30 years...

  • gwenhwyfaer: Jun 27, 2007 09:32

    s/the computer/each processor/

  • Michael Speer: Jun 27, 2007 09:56

    Store my future fully three dimensional holograph projection films and 3DHP games. That's a lot of textures changing over the course of the material which must be stored along with realtime alterations and animations along with the locations of the individual specks of dust that must be projected.

    If it exists, we will find a use for it.

  • Vic: Jun 27, 2007 11:15

    Hey, awesome article, but i believe someone has stolen it and is not citing the source.

    http://technowirenews.blogspot.com/2007/06/what-could-you-do-with-16-exabytes.html

  • Adam Ierymenko: Jun 27, 2007 11:52

    Here's some:

    Huge games

    Evolutionary computation

    Huge realistic simulations

    Computational physics

    Huge CAD projects (e.g. doing CAD on a whole city for urban planning)

  • Gary: Jun 27, 2007 12:55

    Good pickup, Vic. I posted a "this material is stolen from ..." comment

  • Chad Crabtree: Jun 27, 2007 12:56

    Or a Whole SPACE station!

  • ratsbane: Jun 27, 2007 13:16

    Umm... you could use it for word processing... and... um, email and stuff.

  • codist: Jun 27, 2007 13:21

    Apparent the person (MARCHÉ BOURSIER) at technowirenews.blogspot.com is apt to steal content from just about anyone without attribution. Unfortunately there is no contact information and blogger's DMCA takedown procedure involves snail mail, so there's not much I can do. However, if all of you would flag the blog as objectionable, maybe it would get taken down.

  • B: Jun 27, 2007 14:17

    In ten years you will probably wonder how you ever thought the address space was small. Think back to when 32bit was introduced, 4GB of address space was unimaginable back then.

  • Manuel: Jun 27, 2007 15:02

    Nice musing.

    Hm, you could consider how fast you can sweep (e.g. read all memory). Hypertransport can access 22GB per second, so we get 2**64/(22*2**30) = 780903144 seconds we spend reading from the first to the last byte. This is about 24 years.

    If we can push the bus speed by factor 1000 then it would take 9 days. If we could push the bus speed by factor 1M then it would take 13 minutes. A speedup of 1M is a lot, however, and I wonder whether physics will carry that much more electrons through copper in the mid term future.

  • zerokewll: Jun 27, 2007 19:17

    One word: Skynet

  • Chui: Jun 28, 2007 00:11

    I'm not sure if there's a market for porn in "multi mega super high def", but if there is, exabyte storage is the way to go.

    :)

  • Medbob: Jul 02, 2007 10:52

    Along the same lines as Manuel's comment, I'm having increasing trouble moving data from disk to memory. It seems that the storage paradigm is moving faster than our ability to move and process the data.

    That being said, after taking 10 minutes to load up my 24 million row database in memory, it would certainly run a lot faster in memory than it does on disk.

  • Add Comment

Name:


Optional URL:


Comment:


Save Cancel

Copyright © 2007 By Andrew Wulf