开发者

Efficient way to find which values in CSV are NOT in DB?

开发者 https://www.devze.com 2023-01-30 12:12 出处:网络
A vendor is feeding us a CSV file of their products. A particular column on the file (eg column 3) is the style number. This file has thousands on entries.

A vendor is feeding us a CSV file of their products. A particular column on the file (eg column 3) is the style number. This file has thousands on entries.

We have a data-base table of products with a column called manufacturer_num which is the vendors style number.

I need to find which of the vendor's products we do not currently have.

I know I can loop throw each line in the CSV file and extract the style_number and check to see if it is in our data-base. But then I am making a call to the data-base for each line. This would be thousands of calls to the data-base. I think this is inefficient.

I开发者_Go百科 could also build a list of the style numbers (either as a string or array) to make one DB call. Something like: WHERE manufactuer_num IN(...) But won't PHP run out of memory if the list is too big? And actually this would give me the ones we do have, not the ones we don't have.

Whats an efficient way to do this?


Bulk load the CSV into a temporary table, do a LEFT JOIN, then get the records where the RHS of the join is NULL.

0

精彩评论

暂无评论...
验证码 换一张
取 消