I'm trying to work with a 2.5Gb text file, which comprises 293 fields in a tab-开发者_如何学JAVAdelimited extract.
Here is a short sample of the first 2 rows + header. I'm only concerned with the first few fields (lang,Titel,lat,lon,types).
What is the fastest way to load only certain fields from a textfile into SQLite?
I can't open the textfile in Notepad, Excel or Word as it's so large, so I can't delete the unnecessary fields manually.
In SQLite3 I have defined the target table including all 293 fields. To import the data I'm using:
.separator "\t"
.import textfile.txt tablename
This means I need to load in the entire table before I can drop the unnecessary fields. Is there a faster way?
Thanks
It looks like you have your solution, but I was going to suggest using regular expression to clean up your data file first.
精彩评论