开发者

SSIS, splitting a single row into multiple rows

开发者 https://www.devze.com 2023-01-10 01:28 出处:网络
My problem is as follows. I have a CSV file (~100k rows) containting history information with the column format of:

My problem is as follows. I have a CSV file (~100k rows) containting history information with the column format of:

ID1,History1,ID2,History2...ID110,History110

Each row may have anywhere between 0 and 110 history entries. Each separate entry requires a stored procedure to be called.

If there were a small number of possible entries per row, I imagine the way to do this would be to transform the data using a script, and send it to a unique path. Creating 110 paths would pro开发者_如何学Pythonbably work, but isn't very elegant (and quite time consuming).

What would the best way to approach this be?


Just load the data (raw csv unchanged, one row per file line) into a staging table. Then, call a stored procedure that will use a string splitter to break up and loop over the staging table rows and call your other procedure for each history entry.

see: Arrays and Lists in SQL Server 2005 and Beyond

also see this previous answer: SQL comma delimted column => to rows then sum totals?


If you want to solve this in SSIS without the staging tables, you could create a destination script component. You could use switch statement or hashtable to lookup the right sproc to execute for the data row.

It is unclear whether this is a better solution then the staging table approach above; but it is an alternative.


I know you already accepted an answer, but couldn't you use an Unpivot task to achieve what you wanted to do here?

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号