How to track individual objects, that are out of order, and then Joining() the consecutive ones?

I'll start by saying is going to be a little tougher than blindly joining byte[] together.

My big picture goal is to optimize an application that currently uploads many 512byte pages to a web server (Azure Page Blob), and reduce that to a single large upload of 4Megs or less. See the links at the botto开发者_如何学Gom of this question for more background as to why.

The short answer to why: This optimization will increase speed (fewer IOs) and save money over the long term by using Azure sparse files

Now for the details:

The class will need to

• Accept data and store it ( data as defined as alignment start, alignment stop, and the accompanying payload.

• After N pieces of data arrive, or an event occurs, ProcessData(). This means it's time to assemble the data according to the boundaries (the stop value of blob1 must align with the start value of blob2)

• Consecutive data may actually arrive out of order.

• Non consecutive data is defined as when the calling app does not send it before processData() occurs. In addition, if the entire 512 byte range == Zero, then that gets special handing, and is treated as non-consecutive.

• We're dealing with types of byte[], so efficient lists may be complicated here. I'd like to avoid unneeded copies and expansions of the array.

Make sense? (like mud, I hope not)

The closest I've come so far is writing the method signature: (lame I know)

// This can't be bigger than 4Mb, or smaller than 512
byte[] BigBlobToUpload = new byte[];

    /// <summary>
    /// Collects consecutive data ranges that will be uploaded
    /// </summary>
    /// <param name="NameOfTarget">The name of the target (unique per host)</param>
    /// <param name="range">The data to be collected, in 512K multiples</param>
    /// <param name="offsetToTransfer">The "start point" or offset of the data stored in range to be included. Almost always 0.</param>
    /// <param name="sizeToTransfer">The length, or end of the range to include.  Almost always 512.</param>
    /// <param name="cloudOffset">The location this data should be placed in the BigBlobToUpload  global var for eventual upload</param>

private void AddToQueue(string NameOfTarget, byte[] range, int offsetToTransfer, int sizeToTransfer, int cloudOffset)
{

}

I just need someone to give me a direction on how to track these things efficently ... I can handle it from there. Even an abstract direction would help

Can someone put me in the right conceptual direction on how I should track, and conditionally join consecutive data ranges?

Not to mention I'm trying to have efficient logic that only expands, or copies the array when needed.