Managing Remote JMX Metrics in ODH Components
Most of ODH components expose metrics over JMX. To access these metrics outside of this process, use the remoteJMX metrics utility helper function to manage remote JMX metrics on demand through a custom bootstrap script.
Enabling remoteJMX requires updating JVM arguments for java process of each ODH component. Each Ambari service has service-env
shell script template which deals with JVM arguments. This shell script template can be modified to enable and manage remoteJMX. To simplify management of remoteJMX there's a config property in Ambari for each component to enable/disable remoteJMX. For each component, go to the Configs section for each service in the Ambari UI, and then look for properties with following syntax:
odh.<service-name>.<component-name>.jmxremote
This property is exposed as a toggle in the Ambari UI and is used for enabling and disabling remoteJMX for respective service and component.
odh.<service-name>.<component-name>.jmxremote.port
This property denotes the remoteJMX port that the component JVM process exposes metrics.
The config toggle in Ambari UI enables remoteJMX for the component with default values as described below. To add JVM properties for remoteJMX, edit the <service>-env.sh
config template in the Ambari UI.
- -Dcom.sun.management.jmxremote=true
- -Dcom.sun.management.jmxremote.ssl=false
- -Dcom.sun.management.jmxremote.port=<port_value_for_component>
- -Dcom.sun.management.jmxremote.authenticate=false
The following is the complete list of all Ambari config properties to manage remote JMX:
Service | Component/Properties | Remote JMX Port | JVM args Config Property |
---|---|---|---|
HDFS | NAMENODE
|
18011 | HDFS_NAMENODE_OPTS in hadoop-env |
DATANODE
|
18012 | HDFS_DATANODE_OPTS in hadoop-env | |
JOURNALNODE
|
18013 | HADOOP_JOURNALNODE_OPTS in hadoop-env | |
ZKFC
|
18014 | HDFS_ZKFC_OPTS in hadoop-env | |
YARN | RESOURCEMANAGER
|
18021 | YARN_RESOURCEMANAGER_OPTS in yarn-env |
NODEMANAGER
|
18022 | YARN_NODEMANAGER_OPTS in yarn-env | |
APP_TIMELINE_SERVER
|
18023 | YARN_TIMELINESERVER_OPTS in yarn-env | |
MAPREDUCE2 | HISTORYSERVER
|
18031 | MAPRED_HISTORYSERVER_OPTS in mapred-env |
HIVE | HIVE_METASTORE
|
18041 | HADOOP_OPTS in hive-env (conditioned block) |
HIVE_SERVER
|
18042 | HADOOP_OPTS in hive-env (conditioned block) | |
HBASE | HBASE_MASTER
|
18051 | HBASE_MASTER_OPTS in hbase-env |
HBASE_REGIONSERVER
|
18052 | HBASE_REGIONSERVER_OPTS in hbase-env | |
ZOOKEEPER | ZOOKEEPER_SERVER
|
18061 | SERVER_JVMFLAGS in zookeeper-env |
SPARK3 | SPARK3_JOBHISTORYSERVER
|
18071 | SPARK_HISTORY_OPTS in spark3-env |
SPARK3_THRIFTSERVER
|
18072 | SPARK_THRIFT_OPTS in spark3-env | |
KAFKA | KAFKA_BROKER
|
18081 | KAFKA_OPTS in kafka-env |