I'm calling a third party web service that returns a large amount of XML (1000s of rows). What is the best way to get the XML into an underlying SQL server 2008 R2 table. At the moment I am retrieving the XML and deserializing into .NET class collections using the associated XSD. I then iterate through the collection making multiple calls to a INSERT stored proc. The performance isn't too bad but I'm sure there must be a quicker way to do this. Should I be looking at SqlBulkCopy? I've heard that LIN开发者_如何学JAVAQ-TO-SQL is slow when it comes to bulk inserts/updates. Any advice would be welcome.
Read about SQLXMLBulkLoad4, that seems to handle bulk inserts of XML to sql server. I havent tried it myselfe, but its worth a try.
If that not working, then check out System.Data.SqlClient.SqlBulkCopy Using SqlBulkCopy I put 200.000 rows of data read from a file in about 4 seconds.
The following code takes a filename as argument, read all lines and transform the lines to a datarow that is added to a datatable (The datatable has to have same format as the target table, Im using a strongly typed datatable in this case). When the datatable contains 1000 rows, I write it to the server with the WriteToServer method. Then empty the datatable and start over.
you could easy change it to transform rows from your xml-document instead.
Imports System.IO
Imports System.Data.SqlClient
Public Sub ParseFile(ByVal filename As String)
Using BulkCopy As SqlBulkCopy = New SqlBulkCopy(My.Settings.DBConnectionString, SqlBulkCopyOptions.TableLock)
BulkCopy.DestinationTableName = "Destinationtable"
Dim TableDataSet As New MyDS.UserDataTable
Try
Using sr As StreamReader = System.IO.File.OpenText(filename)
Dim l As String = Nothing
l = sr.ReadLine()
If Not l Is Nothing Then AddToDS(l, TableDataSet)
Do Until l Is Nothing
l = sr.ReadLine()
If Not l Is Nothing Then AddToDS(l, TableDataSet)
If TableDataSet.Rows.Count > My.Settings.BulkSize Then 'bulk copy every 1000 rows to db.
BulkCopy.WriteToServer(TableDataSet)
TableDataSet.Rows.Clear()
End If
Loop
If TableDataSet.Rows.Count > 0 Then 'write the rest
BulkCopy.WriteToServer(TableDataSet)
TableDataSet.Rows.Clear()
End If
sr.Close()
End Using
Catch ex As Exception
MsgBox("Error:" & ex.Message)
End Try
End Using
End Sub
Sub AddToDS(ByVal s As String, ByRef ds As MyDS.UserDataTable)
'row should contain username + tab + carname
Dim arr() As String = s.Split(vbTab)
If arr.Count < 2 Then Exit Sub
ds.AddUserRow(arr(0).Trim, arr(1).Trim)
End Sub
精彩评论