开发者

Serialize a tuple of numpy arrays

开发者 https://www.devze.com 2023-02-14 10:04 出处:网络
I have a couple of numpy matrices (3-dimen开发者_StackOverflowsional to be exact) which are stored in tuples

I have a couple of numpy matrices (3-dimen开发者_StackOverflowsional to be exact) which are stored in tuples

(a1,b1,c1)
(a2,b2,c2)
...
(an,bn,cn)

I would like to serialize each tuple into a file that can be read back into Python on another machine (Linux => Windows, both are x86-64). What would be a pythonic way to accomplish this?


numpy.savez or numpy.savez_compressed is the way to go. I've heard, but never experienced issues with certain types of arrays not pickling well.

I'm recalling this post (doesn't seem to have been much of an issue) as well as something about numpy.void not pickling. Likely not an issue, but there it is.


Pickle will probably work well

I also saw this: http://thsant.blogspot.com/2007/11/saving-numpy-arrays-which-is-fastest.html


Use shelve, pickle, cPickle, or shove. Each of these will let you store most kinds of python objects in a file; shove and shelve focus on dictionary-like objects that map keys to values, and shove will let you use a variety of database-like backends. If you find yourself exceeding the performance limitations on these libraries, consider going the database route, e.g. through SQLAlchemy.

I've used each of these libraries, and they work reasonably well within their own niche. I'd start with pickle or shelve, which are standard library.


I generally use cPickle, although I haven't done a formal comparison with other methods. Additionally, I always write the file as binary and use highest protocol setting:

f = open('fname.pkl','wb')
cPickle.dump(array_tuple,f,-1)
f.close()
0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号