ASP.NET application architecture

I would like to create an ASP.NET application which lets the users to monitor various messages from different channels (for example twitter messages in which specific product/company names appears etc, messages from a Facebook page.). These messages would be harvested and processed by a service.

What is the recommended way to this? My initial idea is to create a service which collects and processes the required information and stores it in a database (same as the one that the asp.net application uses). This way when a user checks the webapp, he or she can see the new messages, read from the shared db.

I'm not sure if this is t开发者_JAVA技巧he correct way to do this. Should the components work into the same database? If not how should the asp.net application read the required data?

---

Why shouldn't they use the same database? After all, they're writing/reading the same data, and databases are designed to be central repositories for multiple applications. You wouldn't want to have more than one system of record for your data. Two systems of record essentially means no system of record.

The solution you propose sounds fine to me:

1. A service (Windows service, scheduled console app, etc.) runs at regular intervals to scrape data from external resources, transforms that data to your domain model, and persists it to a local database.
2. A website reads data from that database and reports on it for users.

Keep in mind that services like Facebook and Twitter also provide APIs and various "widgets" that are designed to be embedded directly on a web page. The idea is that a user on your website would be able to see data from Twitter/Facebook/etc. without your server-side code ever having to do anything with it or know anything about it. Does this not meet your needs? Or do you really need to scrape/transform/store the data locally?

---

Absolutely fine for the separate applications to read and write to the same destination database. What you've suggested is perfectly fine.

To save yourself from replicating code you could/should write a single data access library for reading/writing to the db and both of your apps would then use it. If you don't want to be tied to a database implementation, have the code across your apps be dependent on data access interfaces so they have no knowledge of the method of persistence.

---

This is what I came up with based of your and my ideas and many hours of thinking. Here is an image as an overview, description starts below it.

Orange backgrounded components mean services. The ASP.NET MVC app communicates with the streaming service (which opens streams for each user account to twitter) via web service calls. The WCF service is a single instance service which manages these streams.

I use Quartz.NET for scheduling jobs (like regularly scanning incoming emails or sending out emails from queue etc.) Quartz.NET is accessed via .NET remoting from the web application.

Subset of BL layer is required because on new tweets or emails have to be categorized based on some rules. This and the domain layer are in separate assemblies, shared by services and the web application.