Multilevel analysis of the java virtual machine based on kernel and userspace traces
Introduction
Computer programs are becoming increasingly complex. Modern applications involve different interdependent components interacting together and running on optimized multi-core processors. In recent years, there has been an increasing interest in virtual machine based programming languages, because of their flexibility and ease of deployment. A well known example of these languages is Java. Java has been considered as one of the most widely used programming languages in industry (Cass, 2017). It is platform independent and it offers a great amount of advanced technologies like automatic memory management and Just-in-Time compilation.
However, the advanced mechanisms provided by Java made performance debugging more challenging. The developer is usually not fully aware of the internal behavior of the Java virtual machine (JVM) and, as a result, he is unable to pinpoint issues that may happen at runtime. For example, a misconfigured option can cause unnecessary garbage collection cycles that can be easily avoided once detected.
Unexpected behaviors are very hard to detect without proper tools. A wide range of analysis tools are available for developers to evaluate the performance of Java applications. Those tools may be classified into two main classes: monitoring tools and profilers. Monitoring tools provide high level information about the current state of the virtual machine. They present statistics about object allocation, garbage collection, active threads, etc. Profilers give additional information about the logic of the application. By periodically collecting the callstacks of the different threads, profilers can detect slow functions and the logic of the code behind them.
Existing performance analysis tools provide valuable information about the overall performance of Java applications. However, those tools have some limitations. First, the tracing mechanisms used to collect data are not always very efficient and introduce a significant overhead. Second, those tools do not provide a fully integrated analysis that correlates the data gathered from the different layers of the system.
In this paper, we propose a unified Java performance analysis framework that covers the whole software stack, from the user application down to the operating system. We achieve that by using a hybrid approach based on kernel and userspace tracing (Desnoyers, 2009) (Goulet, 2012). We use Kernel tracing to collect low-level information from the operating system (CPU scheduler, Block devices, network interfaces, etc) and userspace tracing to collect information from the different components of the Java virtual machine, such as the garbage collector and the JIT compiler. The traces are collected, synchronized and analyzed using a correlation model. By collecting data from multiple sources and layers, our tool offers full visibility of the system and helps in detecting problems that are very difficult to see otherwise.
We presented different use cases where our tool was able to detect low-level latencies that are difficult to analyze using traditional tools such as I/O and CPU contention. We also showed that our tool does not introduce a big overhead and can be used without altering the normal behavior of applications.
The main contributions are as follows:
- •
Instrumentation of the Hotspot(Griswold, 1998) virtual machine using LTTng(Desnoyers and Dagenais, 2006), a low-overhead tracer available in Linux.
- •
Multilevel performance analysis model that correlates and analyses trace data from different sources.
- •
A Visualization system that helps users understand and identify performance degradations.
The rest of the paper is organized as follows: in Section 2, we introduce the different components of the JVM and we discuss the existing Java performance analysis tools. Then, we describe the architecture of the proposed solution in sections 3 and 4. We present three use cases in Section 5 and we evaluate the efficiency of the framework in Section 6. The conclusion and future work are then presented.
Section snippets
Background
In this section, we provide a description of the different components of the Java Virtual Machine and we list the different studies related to the performance analysis of each. Then, we list some existing Java performance analysis tools and we discuss their limitations. Finally, we define tracing and we justify the choice of LTTng as the main tracer for our framework.
Proposed solution
Analyzing the runtime behavior of a Java application requires a deep understanding of the JVM and the environment on which it is running. To have full visibility of the application, our approach is based on collecting data from multiple layers: the user application, the Java virtual machine and the operating system. The collected traces are then synchronized and analyzed by inspecting the occurrence and the causality of the different events. The general architecture of the proposed analysis
Data model
One of the biggest challenges in tracing is the huge number of events generated, at a very high frequency. Problem detection has to be automated or at least semi-automated. Many approaches have been proposed to simplify data analysis. Ezzati-Jivan and Dagenais (2012) proposed a technique based on data abstraction to reduce trace size. Related low-level events are grouped into more meaningful higher level compound events. Another approach is to use machine learning techniques to automatically
Use cases
The main contribution of our tool is being able to provide Kernel and userspace information at the same time, providing low-level details about the behavior of Java applications. In this section, we present different use cases where our tool is able to solve challenging problems.
Evaluation
In this section, we evaluate the overhead introduced by our solution. Our goal is to have minimal impact to insure that tracing does not affect the normal behavior of the monitored application. We used DaCapo 2018 (Blackburn et al., 2006), a standard benchmarking suite, to generate different workloads, and we computed for each the tracing overhead and the analysis cost.
Conclusion
The Java virtual machine is a very powerful, yet complex application. The internal behavior of the JVM is not fully controlled by the developer and, as a result, many performance issues may happen without being noticed.
Existing analysis tools offer a limited visibility and do not efficiently collect data from the multiple layers of the system.
In this paper, we proposed a multilevel performance analysis tool for Java applications that covers the whole application stack. Tracing is used to
CRediT authorship contribution statement
Houssem Daoud: Conceptualization, Methodology, Software, Writing - original draft. Michel Dagenais: Supervision, Writing - review & editing.
Declaration of Competing Interest
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This research is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC), Prompt, Ericsson, Ciena and EfficiOS.
Houssem Daoud recieved his Master and PhD degree in computer engineering from Ecole Polytechnique de Montreal. His main interests are operating systems, parallel computing and low latency programming.
References (56)
Cling: A memory allocator to mitigate dangling pointers.
USENIX Security Symposium
(2010)- et al.
A survey of adaptive optimization in virtual machines
Proc. IEEE
(2005) - et al.
A comparative evaluation of parallel garbage collector implementations
International Workshop on Languages and Compilers for Parallel Computing
(2001) - et al.
Myths and realities: the performance impact of garbage collection
ACM SIGMETRICS Performance Evaluation Review
(2004) - et al.
The dacapo benchmarks: Java benchmarking development and analysis
ACM Sigplan Notices
(2006) - et al.
Mostly parallel garbage collection
ACM SIGPLAN Notices
(1991) - Brosseau, Y.,, 2020. A userspace tracing comparison: Dtrace vs LTTng ust. [10 February...
- et al.
Undangle: early detection of dangling pointers in use-after-free and double-free vulnerabilities
Proceedings of the 2012 International Symposium on Software Testing and Analysis
(2012) The 2017 top programming languages
IEEE Spectr
(2017)Low-impact operating system tracing
(2009)
The LTTng tracer: A low impact performance and behavior monitor for gnu/linux
OLS (Ottawa Linux Symposium)
User-level implementations of read-copy update
IEEE Trans. Parallel Distrib. Syst.
Problem solving with systemtap
Proc. of the Ottawa Linux Symposium
A stateful approach to generate synthetic events from kernel traces
Advances in Software Engineering
Virtual machines cpu monitoring with kernel tracing
Electrical and Computer Engineering (CCECE), 2014 IEEE 27th Canadian Conference on
Survey and analysis of kernel and userspace tracers on linux: design, implementation, and overhead
ACM Computing Surveys (CSUR)
Wait analysis of distributed systems using kernel tracing
IEEE Trans. Parallel Distrib. Syst.
Unified Kernel/User-Space Efficient Linux Tracing Architecture
Dtrace: Dynamic tracing in oracle solaris, mac OS x, and freebsd
The java hotspot virtual machine architecture
Sun Microsystems Whitepaper
Older-first garbage collection in practice
In The 2002 International Symposium on Memory Management (ISMM 2002
Purify: Fast detection of memory leaks and access errors
In Proc. of the Winter 1992 USENIX Conference
Error-free Garbage Collection Traces: How to Cheat and Not Get Caught
Proceedings of the 2002 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems
Generating object lifetime traces with merlin
ACM Trans. Program. Lang. Syst.
Cited by (3)
Design of Mobile Query System for Health Standard Based on J2ME Technology
2023, 2023 International Conference on Data Science and Network Security, ICDSNS 2023Visualization of profiling and tracing in CPU-GPU programs
2022, Concurrency and Computation: Practice and ExperienceDebugging of Performance Degradation in Distributed Requests Handling Using Multilevel Trace Analysis
2021, Wireless Communications and Mobile Computing
Houssem Daoud recieved his Master and PhD degree in computer engineering from Ecole Polytechnique de Montreal. His main interests are operating systems, parallel computing and low latency programming.
Michel Dagenais is a professor at Ecole Polytechnique de Montreal in Computer and Software Engineering. He authored or co-authored over one hundred scientific publications, as well as numerous free documents and free software packages in the fields of operating systems, distributed systems, and multicore systems, particularly in the area of tracing and monitoring Linux systems for performance analysis. In 1995–1996, during a leave of absence, he was the director of software development at Positron Industries and chief architect for the Power911, object oriented, distributed, fault-tolerant, call management system with integrated telephony, and databases. The Linux Trace Toolkit next generation, developed under his supervision, is now used throughout the world and is part of several specialised and general purpose Linux distributions.