My program needs to load many big wordlist files, so it always takes a long time to start up. it's so inefficient to recover quickly in crash. I have a raw idea, but I am not sure if it's feasible. Is it possible to load the files into shared memory and jus开发者_如何学JAVAt attach to it when starting up?
Could you give me some ideas or suggestions? Is there any valuable instance?
If you're loading 200GB of data, you are either doing something really wrong, or your application doesn't need to startup quickly.
It's definitely possible, provided you design a format which supports
static initialization, or is just raw data. There are two
possibilities: if you can design a raw data format (nothing but PODS,
and no pointers); and if portability isn't a concern, including accross
different versions of the compiler (you provide the file as part of your
executable package), then you can normally just mmap
the file.
Otherwise (what I've usually done), you can write a small program which
will convert your data into C++ definitions (again, only PODs) with
initializers (static), compile it, and link it in, either as a DLL or
statically.
Depending on the access pattern of your program, you might be better off accessing the wordlist in pieces as you need it, rather than loading the entire thing into memory on every startup.
SQLite is excellent for such purposes. You'll have to go through the process of importing your wordlist into the database file, and then you'll be in business.
I suggest you split the word list into many dynamic / shared libraries. For example, one library would contain the common words. This small one you would load at start up. The other libraries you would load on demand. This would reduce the start-up time.
A key principle to reducing start-up time is to only load items that are necessary at start-up. Other items can be loaded on demand, as necessary.
精彩评论