how to create a very large matrix using matlab

I have a problem with matlab when I'm trying to create a a matrix with a very large size such as 40000x40000.

for example:

x=zeros(40000,40000);

the error message is "??? Maximum variable size allowed by th开发者_如何学Ce program is exceeded. "

is there any solution.

Also I have another question, can we have a matrix with variable column size such as in java.

---

40000 * 40000 * 8 bytes per number = 12 GB, surely you won't have enough memory.

To create a huge matrix with lots of zeros, you need a sparse matrix:

m = sparse(40000, 40000)

---

To create an array of variants, you can use a cell array:

m = cell(3, 1)
m(1) = [1,2,3]
m(2) = [2,4,6,8,10]
m(3) = 6+6i

---

There are many things you can do, as the various answers point out. The correct answer depends on your actual problem. Brute force creation of such a large array is impossible without a 64 bit version of matlab plus enough memory to store that large an array, or storing the array in some other way. You can store the array offline, only bringing in what you need as you use it.

For example, if these numbers will always be small integers, then use uint8 or int8, or a logical array, even a single array, all of which will reduce the memory requirements compared to double arrays. Or use a sparse storage form as appropriate. Much depends on what you will do with the array.

As far as a variable number of elements in a row, use a cell array here, not a flat traditional array.

---

If you really need a matrix that large, you could use the Parallel Computing Toolbox (and the MATLAB Distributed Computing Server) to harness the memory of several machines simultaneously. This would allow you to write:

matlabpool open <a large number>
x = distributed.zeros( 40000, 40000 );

See also: doc for codistributed arrays

---

40k*40k => 1.6*10^9 Numbers. I believe Zeros will return floats. So you would need around 1.6*10^9 Floats *4 Bytes/Float => 6.4GB of RAM just for this one Variable...

Are you shure that you need such a big array?

---

One way around the memory problem might be to create a matfile for your matrix. The way I understand it is that it skips the memory and writes it directly to your hard disk. I've tried this for matrices that were too large for my wimpy computer's memory, and it was slow, but it did work. You'll probably have to build it up a bit at a time.

---

no, well yeah buy more ram, as pointed out below. Sparse just removes zero elements.

And in matlab you can just append the new col/rows to the matrix to make a bigger matrix.

Ah but you mean, can you have a matrix where one column is 10 rows, and another is 20 for example. No, you can't do that. Didn't think you could in Java tbh.

---

I think use hard disk files to store that data is best choice