开发者

Optimal way to synchronize database tables cross server

开发者 https://www.devze.com 2023-02-10 09:34 出处:网络
I developed a tool to synchronize data across sql-servers. The tool is responsible for either updating or inserting records from the source table to the destination table. An insert should be executed

I developed a tool to synchronize data across sql-servers. The tool is responsible for either updating or inserting records from the source table to the destination table. An insert should be executed on the rows that doesn't exist in the destination table. An update should be executed on the rows that already exists in the destination table.

Earlier I did the comparison between the tables in code, but this turned out to be very slow (at least the way I coded it). Now I'm using one sql query for inserting and one for updating. The performance is better but it's still slow. Below are my SQL commands. Do you see anything that could be optimized? Eg. can I use something other than an outer join that would be faster?

The update command:

UPDATE [Respondent]
SET [SampleGroupID] = ls.[SampleGroupID], [RespondentStatusTypeID] = ls.[RespondentStatusTypeID], [PilotTest] = ls.[PilotTest], [Username] = ls.[Username], [Password] = ls.[Password], [ImportUniqueID] = ls.[ImportUniqueID], [WORespID] = ls.[WORespID], [DateLastAnswered] = ls.[DateLastAnswered], [DateCreated] = ls.[DateCreated], [Email] = ls.[Email], [Phone] = ls.[Phone], [Name] = ls.[Name], [JobTitle] = ls.[JobTitle], [PhoneDirect] = ls.[PhoneDirect], [开发者_StackOverflowPhoneMobile] = ls.[PhoneMobile], [Fax] = ls.[Fax], [CompanyName] = ls.[CompanyName], [Department] = ls.[Department], [Address] = ls.[Address], [Address1] = ls.[Address1], [Address2] = ls.[Address2], [Zipcode] = ls.[Zipcode], [City] = ls.[City], [District] = ls.[District], [CountryCode] = ls.[CountryCode], [Country] = ls.[Country], [Info1] = ls.[Info1], [Info2] = ls.[Info2], [Info3] = ls.[Info3], [Info4] = ls.[Info4], [Info5] = ls.[Info5], [Info6] = ls.[Info6], [Info7] = ls.[Info7], [Info8] = ls.[Info8], [Info9] = ls.[Info9], [Info10] = ls.[Info10], [Info11] = ls.[Info11], [Info12] = ls.[Info12], [Info13] = ls.[Info13], [Info14] = ls.[Info14], [Info15] = ls.[Info15], [Info16] = ls.[Info16], [Info17] = ls.[Info17], [Info18] = ls.[Info18], [Info19] = ls.[Info19], [Info20] = ls.[Info20], [PublicIdentifier] = ls.[PublicIdentifier], [LockedByUser] = ls.[LockedByUser], [DialAttempts] = ls.[DialAttempts], [DialProbationExpires] = ls.[DialProbationExpires], [CATIScreenOptionID] = ls.[CATIScreenOptionID], [CATIPersonID] = ls.[CATIPersonID], [TotalDuration] = ls.[TotalDuration], [IsExternal] = ls.[IsExternal]
FROM [LinkedServerName].[DBName].[dbo].[Respondent] ls LEFT OUTER JOIN [Respondent] adm
ON ls.RespondentID = adm.RespondentID AND ls.SurveyID = adm.SurveyID
WHERE ls.SurveyID = 1061 AND adm.RespondentID IS NOT NULL

The insert command:

INSERT INTO [Respondent] ([RespondentID], [SurveyID], [SampleGroupID], [RespondentStatusTypeID], [PilotTest], [Username], [Password], [ImportUniqueID], [WORespID], [DateLastAnswered], [DateCreated], [Email], [Phone], [Name], [JobTitle], [PhoneDirect], [PhoneMobile], [Fax], [CompanyName], [Department], [Address], [Address1], [Address2], [Zipcode], [City], [District], [CountryCode], [Country], [Info1], [Info2], [Info3], [Info4], [Info5], [Info6], [Info7], [Info8], [Info9], [Info10], [Info11], [Info12], [Info13], [Info14], [Info15], [Info16], [Info17], [Info18], [Info19], [Info20], [PublicIdentifier], [LockedByUser], [DialAttempts], [DialProbationExpires], [CATIScreenOptionID], [CATIPersonID], [TotalDuration], [IsExternal])
SELECT ls.[RespondentID], ls.[SurveyID], ls.[SampleGroupID], ls.[RespondentStatusTypeID], ls.[PilotTest], ls.[Username], ls.[Password], ls.[ImportUniqueID], ls.[WORespID], ls.[DateLastAnswered], ls.[DateCreated], ls.[Email], ls.[Phone], ls.[Name], ls.[JobTitle], ls.[PhoneDirect], ls.[PhoneMobile], ls.[Fax], ls.[CompanyName], ls.[Department], ls.[Address], ls.[Address1], ls.[Address2], ls.[Zipcode], ls.[City], ls.[District], ls.[CountryCode], ls.[Country], ls.[Info1], ls.[Info2], ls.[Info3], ls.[Info4], ls.[Info5], ls.[Info6], ls.[Info7], ls.[Info8], ls.[Info9], ls.[Info10], ls.[Info11], ls.[Info12], ls.[Info13], ls.[Info14], ls.[Info15], ls.[Info16], ls.[Info17], ls.[Info18], ls.[Info19], ls.[Info20], ls.[PublicIdentifier], ls.[LockedByUser], ls.[DialAttempts], ls.[DialProbationExpires], ls.[CATIScreenOptionID], ls.[CATIPersonID], ls.[TotalDuration], ls.[IsExternal]
FROM [LinkedServerName].[DBName].[dbo].[Respondent] ls LEFT OUTER JOIN [Respondent] adm
ON ls.RespondentID = adm.RespondentID AND ls.SurveyID = adm.SurveyID
WHERE ls.SurveyID = 1061
AND adm.RespondentID IS NULL


I tried to build such a system myself before, but it turned out to be a disaster. What I use to sync data between database is a Redgate tool called SQL data Compare,


If using SQL Server 2008 or later, you'll want to look at MERGE, so you can get it down to a single statement.

If this is for something that's occuring regularly, have you considered using built in tools such as replication?

0

精彩评论

暂无评论...
验证码 换一张
取 消