Sorry, this question might sound silly, but I need some help!
My project is to open a directory and list all the files. These are csv files. Each file is opened and the contents are loaded into the table in a mysql database.
So I have done the above steps, and my java code works fine. For instance, I have 3 files in the directory. When I run the project, the va开发者_JAVA技巧lues in each of the 3 files are loaded into the mysql table.
But when I run the program again and check the database using Select
statement, it shows me the repeated data and hence the values of 6 files are loaded. When I run it again, the database shows 9 files. But I dont want the files to be added again.
So can anyone please tell me about how to achieve this without the need to delete the already added files from the directory.
Thanks in advance
Have a Unique file identifier in DB and in physical file also.
Check for this if it exist skip else perform DB operation
Simple approach: add another table to the database to record the names of those files that have already been processed.
Then, before writing the contents of a new file to the database: check this table first, if the file name is already on the list.
Alternative: record the timestamp of the last "run" and for all runs, skip those files that are "older".
But anyhow, you have to persist some sort of marker, because you need to remember, what you've done so far.
Add another column to the table which is the file name of the file you are reading. So in total you would have 2 columns, 1 for the file name, 1 for the file data.
Before inserting, check, using the SELECT statement if that filename has already been read and inserted.
You could store additional information into the database what files did you already import. You can store in an additional table the name and the last-modified-property of the imported file and make a reference t it in the contents-table. Based on this information you can decide where to import a new file from the directory or just to update the contents of an already imported file.
精彩评论