How to track and persist events from catched exceptions or logged data processing deviances?

Let say that there are events that may occur (are less likely), but they should be registered. This is data that one just need for tuning ... to see what has happened and what needs to be changed and improved.

Typically this is done in catch blocks when exception is thrown or just if some if condition passes.

I don't want to write shell scripts and collect data from logs so I can either use DB and create a table for every context, which is possible in most cases, but it is extremely inconvenient for maintenance and refactoring in further development. Especially because the data will by typed as RDB is used. Mostly the only shared data is userId, time, component and varying data like fileId, fileSize || elementsCount, count开发者_运维问答 deviance || etc.

Or I can use some nosql store for that, which is also a little overkill to do that just because of this, but as the data has rather type free nature, I guess it would be more convenient.

Could you please explain how you do it ? How is this even called ? I think that JMX doesn't deal with this scenario. Spring AOP or AOP in general deals only with the distributed nature of this.

---

It sounds like you have two separate questions:

1. How should I capture certain events in the first place?
2. What should I do with the events once I've captured them?

Regarding 1, yes, AOP is a pretty common solution for capturing events. I'm not sure what you mean by "AOP in general deals only with the distributed nature of this". There's nothing distributed about AOP. You haven't told us enough about your application for anybody to say how to integrate AOP as easily as possible, etc, but lots of information is available online.

Regarding 2, how much data are we talking about? How much information do you want to store per event? What's similar/different about each message? I'd probably take the following approach:

• Figure out how much data you're going to save during any given second, minute, hour, day, etc. If it's small enough to fit into one of your existing databases, then don't complicate your environment by introducing a new technology.
• Can the data be loaded synchronously? If yes, then that's easy. Otherwise, I probably would log the data, and consolidate it periodically with a simple ETL script. This probably will be a whole lot easier and cheaper than setting up a new nosql store that you don't have in production now.
• Decide what data you want to keep. It probably will look something like: id, type, timestamp, source (e.g. server or instance of the application), details. Details should be type-specific.
• Decide what types of queries or reports you want to run on the data.
• Do you need to structure the type-specific stuff so that specific queries are possible? Can you keep the type-specific stuff in an XML or JSON document, and only parse them in type-specific reports? Or, do you need to refer to type-specific stuff in the queries themselves? Type-specific details can make queries hard, but a nosql database such as mongodb might actually help here.
• Figure out your data retention policy. You probably need to clean up old data at some point. This might affect the design of your storage.