Comments (4)
I used VisualVM to do some profiling tonight.
With the MongoDbPatternLayoutAppender performance test increased to 10,000 loops and CPU profiling lowered to the minimum of 50 ms, I got the following results:
main thread - 4229 ms
org.log4mongo.MongoDbPatternLayoutAppender.append - 4062 ms
com.mongodb.DBCollection.insert - 2490 ms
org.log4mongo.MongoDbPatternLayout.format - 1036 ms (almost 1/2 spent in log4J code)
com.mongodb.util.JSON.parse - 505 ms
So about 36% was spent preparing the doc and most of the rest was spent inserting it. The overall times are long due to the impact of the profiler.
With the MongoDbAppender performance test increased to 10,000 loops and CPU profiling lowered to the minimum of 50 ms, I got the following results:
main thread - 8755 ms
org.log4mongo.BSONAppender.append - 8301 ms
org.log4mongo.MongoDbAppender.append - 5287 ms
com.mongodb.DBCollection.insert - 5268 ms
org.log4mongo.LoggingEventBsonifierImpl.bsonify - 3003 ms
2489 ms of the bsonify() method was spent getting location info, i.e., file, method, class and line, from Log4J and then adding it to the document. The next main contributor was bsonifyClassName.
Inserting the documents is about 2/3 of the execution time for this test for both appenders. Insert time is probably slower for MongoDbAppender because the documents are much larger.
Ideas for investigation:
- Add option to leave out location info in the bsonifier, since this would make a big difference in performance
- Look for options to speed up insert. I thought it was doing unsafe writes, but maybe not.
from log4mongo-java.
Regarding Unsafe writes, as far as I know, the current source code, picks up the WriteConcern from Collection object, which by default has a Normal Write Concern. I made changes in my repo to allow WriteConcerns to be specified in properties file. I am sure if we make it to unsafe, it would be a performance improvement.
I will try MongoDBPAtternLayoutAppender today, and see if I can get any performance improvement in Log4Mongo.
from log4mongo-java.
According to http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html, there are many paramters which takes more than usual to fetch the information. We used Custom Log4j PatternLayout, and we saw a substantial performance improvement.
Thanks Robert for looking into it.
from log4mongo-java.
Sure. The caller location info is definitely very slow. Usually you don't need the file if you are logging the class name. I skip the method name, too, since you can get it from the line number. The line number is the only location info that I find really valuable.
Also, I log the category instead of the class, but by convention I set the category to be the same as the class, with a few exceptions. Of course, this can lead to confusion if you copy and paste your logger declaration and forget to change the category.
Looking at log4j.properties.sample, I see that I have even labeled the key as "class" instead of "category". Technically, that's not correct. It is dependent on the user setting the category to be the same as the class. I think that is a common convention, though.
from log4mongo-java.
Related Issues (20)
- Timestamp is a String in CustomPatternLayout HOT 4
- MongoDbPatternLayout uses CustmPatternParser located in Test HOT 2
- LoggingEventBsonifierImpl.nullSafePut: supporting StringBuffer HOT 2
- rootLevelProperties only work with ExtendedMongoDbAppender
- Log4Mongo MongoDbAppender not working on 64-bit CentOS 6.4 x86_64 OS under JBoss 7.1.1 Final HOT 2
- How to add host information to ConversionPattern while i am using MongoDbPatternLayoutAppender ? HOT 2
- Update to mongo-java-driver 2.11.4 HOT 1
- Exception when a MDC property key contains a dot HOT 2
- Exception when logging an object
- c dxxc c +98*++80
- Switch from the deprecated Mongo class to MongoClient HOT 2
- Mongo-->MongoClient HOT 1
- how to use log4jmongo in springboot
- log4mongo is not compatible with latest version of mongodb-java-driver 3.4.1 HOT 2
- forbiden HOT 2
- JAXB classes not found HOT 1
- getDB method is deprecated and starting to cause trouble
- problem in glassfish deploy
- any plans to migrate to log4j2 ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from log4mongo-java.