Published on 00/00/0000
Last updated on 00/00/0000
Published on 00/00/0000
Last updated on 00/00/0000
Share
PRODUCT
3 min read
Published on 01/11/2022
Last updated on 06/18/2024
Logging impact application performance
Share
Compliant logging is highly important for application development, as it affects performance.
Here we see 70G of temporary memory consumption is used from the non-compliant logging.
This can cause garbage collector redundant heavy processing, which uses system memory and CPU.
Look on memory, after logging changes:
-> 70G of temporary memory consumption is removed.
Look on method profiling, before logging changes:
Here we see many of the methods invocations are redundant.
Look on method profiling, after logging changes:
-> Redundant methods invocations are removed.
Look on lock instances, before logging changes:
Here we see long processing time is spent busy with logging, which can be changed to debug.
Highlights
- non-compliant and excessive logging
- Real high scale execution recording showing the effect
- -> Why valid log writing is very important
Introduction
Consider the following code:parseMessage(List<Message> messages) {
log.debug("messages: " + messages);
}parseMessage(List<Message> messages) {
log.debug("messages: {}", messages.toString());
}Real high scale execution recording showing the effect
During high scale test analysis, resources were analyzed. Execution was with 100 parallel threads processing the data. With tools like memory heap dump and thread dump, such described phenomena do not comes to place, since it results in temporary high resources usage of memory allocation, CPU processing, and possibly file IO if this is written to log according to verbosity. As these are temporary, it is not reflects in the memory heap dump and other tools. This is where process recording analysis comes to place. Look on memory, before logging changes:
Here we see 70G of temporary memory consumption is used from the non-compliant logging.
This can cause garbage collector redundant heavy processing, which uses system memory and CPU.
Look on memory, after logging changes:
-> 70G of temporary memory consumption is removed.
Look on method profiling, before logging changes:
Here we see many of the methods invocations are redundant.
Look on method profiling, after logging changes:
-> Redundant methods invocations are removed.
Look on lock instances, before logging changes:
Here we see long processing time is spent busy with logging, which can be changed to debug.
Execution improvements after logging changes
after fixing non-compliant logging, and moving some excessive logging to debug, execution showed massive improvement. Reduced at least temporary 70G memory consumption per the recording from the non-compliant logging. Processing time down from 1~ minute to 25 seconds after fixing non-compliant logging, and moving some excessive logging to debug.Solution
Non-compliant logging
Solution is valid code writing which will cause toString() method invocation only when needed, for example when log is in debug verbosity:parseMessage(List<Message> messages) {
log.debug("messages: {}", messages);
}Excessive logging
Review logging, move unnecessary log to DEBUG or remove if not needed anymore post development.Conclusion
Fixing non-compliant logging, and reducing excessive logging can drastically improve performance, reduce processing time, reduce CPU and temporary memory usage and I/O for log file writing.
Subscribe to
the Shift!
Get emerging insights on emerging technology straight to your inbox.
