How to build a Terabyte queue?

How to build/design/program a Terabyte or Petabyte queue in memory? (Imagine a twitt开发者_Go百科er like service with huge number of users.)

Why not use twitter's queuing service? It's called kestrel and it's open source.

It's going to depend a lot on what file system you're using and what elements are stored in the queue. The elements of the queue will need to be addressable somehow. Perhaps as filenames, or disk block addresses, or ... something. You're going to need to store the addresses of these elements in the queue. Depending on how many elements you're working with, you may even have to break it down even further and divide your queue up into blocks, where entire blocks of elements are considered one element in the queue, and each block is organized as its own sub-queue.