Storing header and data sections in a CSV file

This should be relatively easy to do, but after several hours straight programming my mind seems a bit frazzled and could do with so开发者_开发知识库me help.

I have a C++ class which I am currently using to store read/write data to file. I was initially using binary data, but have decided to store the data as CSV in order to let programs written in other languages be able to load the data.

The C++ class looks a bit like this:

class BinaryData
{
public:
   BinaryData();

   void serialize(std::ostream& output) const;
   void deserialize(std::istream& input);   

private:
   Header m_hdr;
   std::vector<Row> m_rows;
};

I am simply rewriting the serialize/deserialize methods to write to a CSV file. I am not sure on the "best" way to store a header section and a "data" section in a "flat" CSV file though - any suggestions on the most sensible way to do this?

---

The most sensible way is don't do it. Let an existing library do it for you. You'll never be able to put the requisite amount of work into this class that you'll need to maintain backwards compatibility and changing requirements.

Use an existing solution, like Google's protocol buffers. They're stored in binary - so you get the size benefit, and there are many language bindings available. In addition, you can get human readable field-based ascii representations trivially.

---

Have you looked at all at other solutions? Google protocol buffers already mentioned, take a look at boost serialization library.

---

If you must serialize and deserialize binary forms of objects, I suggest you use a system that allows you to read and write the binary data to and from a buffer. This will allow you to allocate one buffer and make one large I/O transaction rather than many small ones.

The system that I use has 3 fundamental methods that each class must implement:

• Get Size On Stream -- returns the size the data would occupy on the stream.
• Store To Buffer -- stores the data to a buffer, optionally incrementing the pointer.
• Load From Buffer -- loads data members from a buffer, optionally incrementing the pointer.

Aggregate objects should call these functions on their members.

This will allow you to allocate a buffer, using the Get Size On Stream methods and then write to the buffer using the Store To Buffer methods. Then you just write the entire buffer, for example using ostream::write(buffer, Get Size On Stream).

If your good, you can create public templated functions that implement these methods for the POD types.