Log to rolling CSV file with Enterprise Library

I need logging to:

1. Rolling file, to avoid 1 big log file.
2. CSV format for easier look up.

I can see the EntLib (5.0) has the Microsoft.Practices.EnterpriseLibrary.Logging.TraceListeners.RollingFlatFileTraceListener to log to a rolling log file.

To make the log entries look like a CSV row, I can change the开发者_StackOverflow Formatters.TextFormatter.Template to put double quote around the values, and also change the Listener's Footer and Header to nothing, so they won't be output.

Under normal circumstance, this would give me a well formed CSV file. However if a token value in the Template contain a double quote, this would not be escaped. Hence the log file become an invalid CSV file.

Is there any way to resolve this?

Is there any alternative solutions to this problem?

---

See http://msdn.microsoft.com/en-us/library/ff650608.aspx. Turn out adding a custom formatter is not that hard, I added a CSVTextFormattter to only take care of massaging the message and extended properties, which works for me. Notice I use the bult-in TextFormatter to do all the heavy lifting.

Sample Config:

<loggingConfiguration name="" tracingEnabled="true" defaultCategory="General">
...
    <formatters>
      <add type="<your namespace>.CSVTextFormatter, <your dll>"
          template="{timestamp(local)},{severity},{category},{message},{property(ActivityId)},{eventid},{win32ThreadId},{threadName},{dictionary({key} - {value}{newline})}"
          name="CSV Text Formatter" />
    </formatters>...
</loggingConfiguration>

The class is something like this:

Public Class CSVTextFormatter
    Implements ILogFormatter

    Private Const csTemplateAttributeName As String = "template"

    Private moTextFormatter As TextFormatter
    Private Property TextFormatter() As TextFormatter
        Get
            Return moTextFormatter
        End Get
        Set(ByVal value As TextFormatter)
            moTextFormatter = value
        End Set
    End Property

    Private moConfigData As System.Collections.Specialized.NameValueCollection
    Private Property ConfigData() As System.Collections.Specialized.NameValueCollection
        Get
            Return moConfigData
        End Get
        Set(ByVal value As System.Collections.Specialized.NameValueCollection)
            moConfigData = value
            If moConfigData.AllKeys.Contains(csTemplateAttributeName) Then
                TextFormatter = New TextFormatter(moConfigData(csTemplateAttributeName))
            Else
                TextFormatter = New TextFormatter()
            End If
        End Set
    End Property

    Public Sub New()
        TextFormatter = New TextFormatter()
    End Sub

    Public Sub New(ByVal configData As System.Collections.Specialized.NameValueCollection)
        Me.ConfigData = configData
    End Sub

    Public Function Format(ByVal log As Microsoft.Practices.EnterpriseLibrary.Logging.LogEntry) As String Implements Microsoft.Practices.EnterpriseLibrary.Logging.Formatters.ILogFormatter.Format
        Dim oLog As Microsoft.Practices.EnterpriseLibrary.Logging.LogEntry = log.Clone()
        With oLog
            .Message = NormalizeToCSVValue(.Message)
            For Each sKey In .ExtendedProperties.Keys
                Dim sValue As String = TryCast(.ExtendedProperties(sKey), String)
                If Not String.IsNullOrEmpty(sValue) Then
                    .ExtendedProperties(sKey) = NormalizeToCSVValue(sValue)
                End If
            Next
        End With
        Return TextFormatter.Format(oLog)
    End Function

    Private Shared Function NormalizeToCSVValue(ByVal text As String) As String
        Dim bWrapLogText = False
        Dim oQualifiers = New String() {""""}
        For Each sQualifier In oQualifiers
            If text.Contains(sQualifier) Then
                text = text.Replace(sQualifier, String.Format("""{0}""", sQualifier))
                bWrapLogText = True
            End If
        Next
        Dim oDelimiters = New String() {",", vbLf, vbCr, vbCrLf}
        If text.Contains(oDelimiters) Then
            bWrapLogText = True
        End If
        If bWrapLogText Then
            text = String.Format("""{0}""", text)
        End If
        Return text
    End Function

End Class

---

I don't think there is any "silver bullet" solution short of writing your own formatter.

You will need to worry about double quotes and new lines. Any of those will throw off the formatting.

I think the only properties that you have to worry about those characters for is the Message, Title, and any ExtendedProperties you are using. I recommend writing a thin wrapper or facade around the Write method where you escape those properties to ensure that you have a properly formatted file. i.e. escape any double quotes and replace new lines with a space.

---

I have translated the code to c# and fixed a bug in qualifier escaping. I have also added semicolon as a delimiter, since Excel by default assumes semicolon separated CSVs..

public class CsvLogFormatter: ILogFormatter
{
    private TextFormatter _formatter;


    public CsvLogFormatter(string template)
    {
        // property Template allows 'set', but formatter still uses original template.. Must recreate formatter when template changes!
        _formatter = new TextFormatter(template);
    }


    public string Template { get { return _formatter.Template; } }


    public string Format(LogEntry log)
    {
        try
        {
            var logEntry = (LogEntry)log.Clone();
            logEntry.Message = NormalizeToCsvToken(logEntry.Message);
            var normalizableKeys = logEntry.ExtendedProperties.Where(l => l.Value == null || l.Value is string).ToList();
            foreach (var pair in normalizableKeys)
            {
                logEntry.ExtendedProperties[pair.Key] = NormalizeToCsvToken((string)pair.Value);
            }
            return _formatter.Format(logEntry);
        }
        catch
        {
            // this redundant catch is useful for debugging exceptions in this methods (EnterpriseLibrary swallows exceptions :-/)
            throw;
        }
    }

    private static string NormalizeToCsvToken(string text)
    {
        var wrapLogText = false;

        const string qualifier = "\"";
        if (text.Contains(qualifier))
        {
            text = text.Replace(qualifier, qualifier + qualifier);
            wrapLogText = true;
        }

        var delimiters = new[] { ";", ",", "\n", "\r", "\r\n" };
        foreach (var delimiter in delimiters)
        {
            if (text.Contains(delimiter))
                wrapLogText = true;
        }

        if (wrapLogText)
            text = string.Format("\"{0}\"", text);
        return text;
    }
}

Feel free to use and enhance. This is a very simple solution, may be it would be more nice to derive new Formatter from TextFormatter instead of wrapping it, but this works fine for me ('works' == Excel opens it without any known problems).

---

The following code works fine for me:

[ConfigurationElementType(typeof(CustomFormatterData))]
 public class CsvLogFormatter : ILogFormatter
  {
    private TextFormatter _formatter;
    private string template = "template";

    public CsvLogFormatter(NameValueCollection collection)
    {
        // property Template allows 'set', but formatter still uses original template.. Must recreate formatter when template changes!
        _formatter = new TextFormatter(collection[template]);
    }


    public string Template { get { return _formatter.Template; } }


    public   string Format(LogEntry log)
    {
        try
        {
            var logEntry = (LogEntry)log.Clone();
            logEntry.Message = NormalizeToCsvToken(logEntry.Message);
            var normalizableKeys = logEntry.ExtendedProperties.Where(l => l.Value == null || l.Value is string).ToList();
            foreach (var pair in normalizableKeys)
            {
                logEntry.ExtendedProperties[pair.Key] = NormalizeToCsvToken((string)pair.Value);
            }
            return _formatter.Format(logEntry);
        }
        catch
        {
            // this redundant catch is useful for debugging exceptions in this methods (EnterpriseLibrary swallows exceptions :-/)
            throw;
        }
    }

    private static string NormalizeToCsvToken(string text)
    {
        var wrapLogText = false;

        const string qualifier = "\"";
        if (text.Contains(qualifier))
        {
            text = text.Replace(qualifier, qualifier + qualifier);
            wrapLogText = true;
        }

        var delimiters = new[] { ";", ",", "\n", "\r", "\r\n" };
        foreach (var delimiter in delimiters)
        {
            if (text.Contains(delimiter))
                wrapLogText = true;
        }

        if (wrapLogText)
            text = string.Format("\"{0}\"", text);
        return text;
    }
}