Solr: how to turn down logging

OK, so I'm thrilled with Solr, but I can't seem to figure out how to turn down the logging level so that it will actually run acceptably fast when I do a huge import run.

I'm not even sure which logging framework it's using (because, you know, java.util.logging, log4j and commons-logging weren't enough, we needed to add slf4j to the mix!) OK sorry, had to let that minor rant slip. :-)

So I have src/main/resources/log4j.properties configured with:

log4j.rootLogger=WARN, console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.conversionPattern=%5p [%t] (%F:%L) - %m%n
org.apache.solr.core=WARN
org.apache.solr.update.processor=WARN

And then for good measure I have src/main/resources/logging.properties configured with:

.level = WARNING

com.gwtstore.level=ALL
com.appgravity.level=ALL
org.apache.solr.core.level=WARNING
org.apache.solr.update.processor.level=WARNING

And yet somehow, I am still seeing tons of the following:

INFO: {add=[-7757828706308755634]} 0 0
Feb 21, 2011 6:12:23 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=null path=/update params={} status=0 QTime=0 

Here's my classpath:

src/test/resources
src/test/java
src/main/resources
src/main/java
idea_rt.jar
junit-rt.jar
deploy.jar
dt.jar
javaws.jar
jce.jar
jconsole.jar
management-agent.jar
plugin.jar
sa-jdi.jar
alt-rt.jar
charsets.jar
classes.jar
jsse.jar
ui.jar
apple_provider.jar
dnsns.jar
localedata.jar
sunjce_provider.jar
sunpkcs11.jar
test-classes
classes
junit-4.8.2.jar
gwt-user-2.1.1.jar
gwt-servlet-2.1.1.jar
servlet-api-2.5.jar
weld-servlet-1.1.0.Final.jar
hibernate-core-3.6.0.Final.jar
antlr-2.7.6.jar
commons-collections-3.2.1.jar
dom4j-1.6.1.jar
hibernate-commons-annotations-3.2.0.Final.jar
slf4j-api-1.6.0.jar
hibernate-jpa-2.0-api-1.0.0.Final.jar
jta-1.1开发者_Go百科.jar
hibernate-entitymanager-3.6.0.Final.jar
javassist-3.12.0.GA.jar
hibernate-c3p0-3.3.1.GA.jar
c3p0-0.9.1.jar
hibernate-validator-4.1.0.Final.jar
validation-api-1.0.0.GA.jar
cglib-nodep-2.2.jar
javax.inject-1.jar
cdi-api-1.0.jar
jboss-interceptor-api-1.1.jar
jsr250-api-1.0.jar
lucene-core-2.9.1.jar
lucene-snowball-2.9.1.jar
mongo-java-driver-2.4.jar
mysql-connector-java-5.1.6.jar
protobuf-java-2.2.0.jar
proxytoys-1.0.jar
quartz-1.8.4.jar
commons-lang-2.5.jar
commons-httpclient-3.1.jar
commons-logging-1.0.4.jar
commons-codec-1.2.jar
nekohtml-1.9.7.jar
xercesImpl-2.8.1.jar
xml-apis-1.3.03.jar
gwtquickstarter.jar
solr-solrj-1.4.1.jar
commons-io-1.4.jar
commons-fileupload-1.2.1.jar
wstx-asl-3.2.7.jar
stax-api-1.0.1.jar
geronimo-stax-api_1.0_spec-1.0.1.jar
solr-core-1.4.1.jar
lucene-analyzers-2.9.3.jar
lucene-highlighter-2.9.3.jar
lucene-memory-2.9.3.jar
lucene-queries-2.9.3.jar
lucene-misc-2.9.3.jar
lucene-spellchecker-2.9.3.jar
solr-commons-csv-1.4.1.jar
wstx-asl-3.2.7.jar
morphia-0.98s10.jar
androidmarketapi-0.5.jar
openid4java-0.9.5.jar

---

1. You need to make sure that the logging properties file is deployed to .../yourWebapp/WEB-INF/classes/log4j.properties.

2. If there is a .../yourWebapp/WEB-INF/classes/log4j.xml file, this will override the properties file.

3. The location of the Log4j configuration file can also be overridden by system properties.

For more information on how Log4j gets its configuration, look at this section of the Log4j introduction.

---

If you are just looking to turn off logging completely from the Solr server , just visit http://(YOUR-IP):8080/solr/admin and navigate to the logging option seen on top list.

On that page , you could find the detailed table for the various logging levels defined for your Solr installation. Either change their parent logger to make them use the logger you are using in your application or turn them off entirely.

Hope this is what you were looking for. Thanks

---

Following link discusses how to configure Solr logging properly.

http://lucidworks.lucidimagination.com/display/solr/Configuring+Logging

If you want to make changes to logging temporarily, then it can done from the Solr admin page. For making changes permanently you will need to add a JDK logging properties file to Solr web application.

For tomcat following are the steps

1. Create the classes directory in the WB_INF foler for solr web application i.e. _%TOMCAT_INSTALL_DIR%/webapps/solr/WEB-INF/classes/logging.properties_
2. Create logging.properties file
3. Add following entries for setting log level (log levels can be set from FINEST to SEVERE for a class or an entire package)

org.apache.commons.digester.Digester.level = FINEST

org.apache.solr.level = WARNING

---

Having done this for log4j for SOLR I can tell you it was painful. Log4j doesn't work well with SOLR out the box. Anyway, here is what I did to make SOLR work with slf4j and log4j. If there is an easier way - awesome - do it.

In solr/lib

• Add slf4j-log4j12-1.5.11.jar ( make sure versions match to slf4j-api-??? )
• Remove slf4j-jdk14-1.5.5.jar (important - otherwise log4j doesnt get used)

in resources in your webapp add your log4j.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "/WEB-INF/log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/" >
  <appender class="org.apache.log4j.DailyRollingFileAppender" name="FILE" >
    <param name="file" value="logs/jetty.log" />
    <param name="datePattern" value="'.'yyyy-MM" />
    <param name="append" value="true" />
    <layout class="org.apache.log4j.PatternLayout" >
      <param name="ConversionPattern" value="%d [%t] %-5p %C{6} (%F:%L) - %m%n" />
    </layout>
  </appender>
  <root>
    <priority value="error"></priority>
    <appender-ref ref="FILE" />
  </root>
</log4j:configuration>

---

After long trials of placing the word WARNING in infinitely many files, it looks like I've finally managed to slow down Tomcat Solr logging. How would that be useful for your situation/environment is completely beyond me, since my brain is full of JAVA configs as of now. I'm writing it down anyway, with the hope of preventing some pain for someone in the future.

I may have completely disabled some logging along the way, so you may need to undo what I'll be describing here when debugging (or e.g. on the development server).

1. Create a file named logging.properties in your config folder, e.g. in /etc/tomcat... if it doesn't already exist.

2. If logging.properties already exists, comment everything out. You may choose not to comment out other lines though, in case you want some logging.

3. Append / insert the following to logging.properties:

   .level = WARNING
   

4. Create a file named setenv.sh in your $CATALINA_HOME/bin folder if it doesn't already exist. (Where is $CATALINA_HOME? Generally in /usr/share/tomcat... but type
   ps -ef | grep tomcat
   and hunt for the string catalina.home or something similar in the output to be sure.)

5. Append / insert the following to setenv.sh:

   export JAVA_OPTS="${JAVA_OPTS} -Djava.util.logging.config.file=/etc/tomcat.../logging.properties"
   

6. Restart Tomcat (e.g. /etc/init.d/tomcat... restart, depends on your OS)

(Replace tomcat... with your actual version of Tomcat)

---

In addition to what was written here in the recent versions of Solr, you have the option to configure slow query logging. You set the threshold above which a query will be considered long-running and such queries will be written in a separate log.

To do that add the slowQueryThresholdMillis to your solrconfig.xml file. For example, to consider each query running above 2 seconds as slow you would put the following to the solrconfig.xml:

<slowQueryThresholdMillis>1000</slowQueryThresholdMillis>