Memory Efficient and Speedy iPhone/Android Dictionary Storage/Access

Im having trouble with memory on older generation iPhones (ipod touch 1st gen, 2nd gen e.t.c). This is due to the amount of memory allocated when I load and store a 170k word dictionary.

This is the code (very simple):

string[] words = dictionaryRef.text.Split("\n"[0]);
_words = new List<string>(words);

This allocates on start around 12mb of storage, iphone has around 43mb I think. So that + textures + soun开发者_JAVA技巧ds + the OS it tends to break.

Speed wise, accessing using a binary search is fine. But its storing it in memory more efficiently (and loading it more efficiently).

The text.Split appears to take up alot of heap memory.

Any advice?

---

You can't count too much on how much memory these pre-3.0 devices have available on startup. 43 MB is rather optimistic. Is your app just checking to see if the word is in the list or not? You might want to roll your own hash table instead of using a binary search. I'd search some of the literature and stack overflow to look for efficient ways to store a large dictionary with the particular word sizes you have. A google search on hash table might give you a better implementation.

---

Use SQLite. It will use less memory and be faster. Create an index on your words column and voila, you have binary search, without having the whole dictionary loaded in memory.

---

First if dictionaryRef.text is a string (and it looks so) then you already got something huge being allocated (2 bytes per characters). Check this it might well account for a large (near half) amount of the total memory being allocated. You should think about caching this (the database idea is a good one, but a file could do to then use File.ReadAllLines in future execution).

Next you can try do a bit better than Mono's Split method. It creates a List and then turn it into an array (calling ToList) at the end - which you end up creating a new List from. Since your requirement (only '/n') is fairly basic I suggest you to roll your own Split method (or copy/paste/reduce the one from Mono) and avoid the temporary memory allocations.

In any case take a lot of (memory) measurements since allocations, even more for strings, often occurs where we don't look ;-)

---

I would have to agree with Morningstar that using a SQLite backend for your word storage sounds like the best solution to what you are trying to do.

However, if you insist on using a word list, here's a suggestion:

It looks to me like dictionaryRef.text is constructed by reading a text file in its entirety (File.ReadAllText() or some such).

Instead of doing that, why not use TextReader.ReadLine() to read 1 word at a time from the file into a List, thus avoiding the need to use String.Split() and using tons of temporary storage space?

Ultimately that seems to be what you want anyway... and ReadLine() will "split" on \n for you.