Oracle Cloud Infrastructure Documentation

Configure APM Java Agent Hybrid Mode

The APM Java Agent hybrid mode, also referred to as APM Agent hybrid mode, allows the agent probe span and the application span to form a complete trace.

The APM Agent hybrid mode feature combines both bytecode instrumentation and OpenTracing/OpenTelemetry manual instrumentation to provide a complete picture of the application flow.

It provides a mechanism to make sure OpenTracing and OpenTelemetry libraries are loaded in a common ClassLoader accessible from the APM Agent and all the application. Therefore, spans can be created in the application, and they are linked to the spans created by APM Agent probe automatically.

Spans created by APM Agent probe are automatically linked to any activated span.

Prerequisites

  • The APM Agent is fully functional before enabling the property.
    For OpenTelemetry, use com.oracle.apm.agent.otel=true property.
    Note

    OpenTelemetry requires Java 8 and above.
  • The application should either get the OpenTracing tracer instance using GlobalTracer.get() or get the OpenTelemetry instance with GlobalOpenTelemetry.get() method.
  • Same library version.

    For OpenTracing, the APM Agent and the application use the same OpenTracing library version.

    For OpenTelemetry, the APM Agent and the application use the same OpenTelemetry library version. For details, see Configure APM Java Agent to Work with OpenTelemetry.

Enable APM Agent Hybrid Mode

The APM Agent hybrid mode is enable by default.

To disable it, use the property: com.oracle.apm.agent.hybrid=false. However, if the OpenTelemetry property: com.oracle.apm.agent.otel=true is set, then hybrid mode is always on.

Properties Type and Description Properties Example

AgentConfig.properties

Update the AgentConfig.properties file located under oracle-apm-agent/config/<version> directory.

com.oracle.apm.agent.hybrid com.oracle.apm.agent.hybrid=true

System Properties

Update the system properties.

 com.oracle.apm.agent.hybrid -Dcom.oracle.apm.agent.hybrid=true

Environment Variables

Update the environment variables.

 com_oracle_apm_agent_hybrid
For Windows: 
set com_oracle_apm_agent_hybrid=true
For Linux: 
export com_oracle_apm_agent_hybrid=true

Setting the property is the only step required for most appservers and microservices. However, some appservers may have a different classloading logics and class definition isolations. To ensure that the APM Agent uses the same OpenTracing and OpenTelemetry class definition the additional following step may be required:

  • Copy the OpenTracing and OpenTelemetry libraries from oracle-apm-agent/install/<version>/lib/ext directory to the oracle-apm-agent/bootstrap directory.

The above step ensures the APM Agent and all the applications will use the copied OpenTracing and OpenTelemetry libraries hence the same OpenTracing and OpenTelemetry class definitions.

APM Agent and OpenTracing and OpenTelemetry libraries: OpenTracing libraries version 0.33 and OpenTelemetry libraries version 1.38.0 are included with the APM Agent. If the application requires to use OpenTracing version 0.32 specific API, it is possible to copy the OpenTracing 0.32 libraries to the oracle-apm-agent/bootstrap directory. APM supports both OpenTracing 0.32 and 0.33 libraries. For application required to use different OpenTelemetry version, Configure APM Java Agent to Work with OpenTelemetry.

In general, the additional step is required for the following:

  • When OpenTracing libraries is explicitly included in microservices' classpath.

  • When appserver classloader does not prioritize classloading to parent classloader. For example, Tomcat.

Frequently Asked Questions

  1. Why did the Agent probe span such as JDBC span did not link to the application span even the JDBC calls were made within application span scope?

    A: In order for the Agent probe to create span linkage, the application span must be activated before JDBC calls.

  2. Why were Agent probe spans not linked to Helidon spans?

    A: If Helidon created spans are not activated, you can create a new span and activate with the parent SpanContext from ServerRequest. After that, all the Agent Probe spans will be children of the new span.

  3. How does the Agent hybrid mode work when my application has embedded OpenTracing version 0.32 while the APM Agent has OpenTracing version 0.33?

    A: In those cases, you should consider updating the application to use OpenTracing 0.33.

    If that's not possible, you can change the OpenTracing libraries in oracle-apm-agent/install/<version>/lib/ext to 0.32. The APM Agent is compatible with both OpenTracing versions 0.32 and 0.33.

  4. Why did the APM Agent and my application create separate traces on request after enabling Agent hybrid mode?

    A: Check the configuration steps and confirm that the property was set properly. For details, see Enable APM Agent Hybrid Mode.

    If the problem persists, you may need to copy OpenTracing libraries to oracle-apm-agent/bootstrap.

  5. Can I use Hybrid Mode from Agent (1.3+) with existing application built with APM Tracer (1.2 or below)?

    A: No. Both deployed APM Agent and APM Tracer used for application build must be at least 1.3.

  6. Why is the service name in AgentConfig.properties used for tracer with different service name setting when hybrid mode is enabled?

    A: In the case of hybrid mode, APM Agent starts a tracer and it is registered as the global tracer. If a tracer is started by the application or micro service, the tracer instance is actually the global tracer instance. Hence, it uses the same service name as defined in AgentConfig.properties file. In order to use different service name for different agent instances, one has to override the service name with the java system property or environment variable.

  7. How to override the service name in AgentConfig.properties file?

    A: One can override the service name by setting the java system property (com.oracle.apm.agent.service.name) or the environment variable (com_oracle_apm_agent_service_name).

  8. How to enable Opentracing hybrid mode for Micronaut applications ?
    1. Follow the instructions described above under Configure APM Java Agent Hybrid Mode.
    2. Add the APM Micronaut tracer in the application pom.xml file as a dependency. See Use APM Tracer in Micronaut for details.