This simple statement:
zip(xrange(0, 11614321), xrange(0, 11627964))
...is eating most of my RAM. (>150 MiB!) Why?
Edit: Ah, re-reading the docs, I see zip
returns a list, not an iterable. Anything like zip
that returns an iterable?
The larger picture: I'm iterating over two large arrays of file data, and I'm doing things like iterating (0-end, 0-end), (0-end, 1-end), etc. I'd like to not slice the array, as it would cause excessive allocations of memory. I figured I'd just iterate over the indexes instead, but that doesn't seem to work, as per above. The whole code:
def subsequence_length(data_a, data_b, loc_a, loc_b):
length = 0
for i_a, i_b in zip(xrange(loc_a, len(data_a)), xrange(loc_b, len(data_b))):
if data_a[i_a] == data_b[i_b]:
length += 开发者_如何学JAVA1
else:
break
return length
Use izip from itertools
If for some reason you didn't want to use the itertools
module, it would be trivial to write your own iterator that did the same thing, at least if you know you're dealing with exactly two input iterators.
def xzip2(i1, i2):
i1, i2 = iter(i1), iter(i2)
while True:
yield next(i1), next(i2)
Actually, upon further reflection, it is not that hard to make it work with any number of iterators. I am fairly sure itertools.izip
must be implemented something like this.
def xzip(*iters):
iters = [iter(i) for i in iters]
while True:
yield tuple([next(i) for i in iters])
(And looking at the documentation, I see it is, except they're using map
rather than list comprehensions.)
精彩评论