Prometheus

Introduction

Prometheus offers a simple and easy get started monitoring infrastructure that works out of the box. It uses a pull method where the Prometheus server will poll each device for its status.

The documentation and download for setting up Prometheus is located below

https://prometheus.io/docs/introduction/first_steps/

Configuration

Configuration is simple, by pointing the Prometheus to the following URL on the FMADIO host system

https://<fmadio host IP>/metrics

Step 1) Set a target to FMADIO device

Typically this is done by editing the Prometheus configuration file located

/etc/prometheus/prometheus.yml

An example configuration is shown below, here the FMADIO system is located at 192.168.133.10 over TLS on port 443

fmadio@prometheus0:/etc/prometheus$ cat prometheus.yml
# my global config
global:
  scrape_interval: 5s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  query_log_file: /var/log/prometheus/query.log
  # scrape_timeout is set to the global default (10s).

# Alertmanager configuration
alerting:
  alertmanagers:
    - static_configs:
        - targets:
          # - alertmanager:9093

# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: "prometheus"

    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.

    static_configs:
      - targets: ["localhost:9090"]

  - job_name: "fmadio20v3-333"
    scrape_interval: 5s
    scrape_timeout: 5s
    scheme: https
    basic_auth:
      username: fmadio
      password: 100g

    static_configs:
      - targets: ["192.168.133.10:443"]
    tls_config:
      insecure_skip_verify: true
fmadio@prometheus0:/etc/prometheus$

The key part is the static_configs targets section, specifically

  - targets: ["192.168.133.10:443"]  

As FMADIO device uses the default /metrics URL only the IP location is required.

Step 2) restart Prometheus

Usually this is done with systemctl service such as below

fmadio@prometheus0:/etc/prometheus$ sudo systemctl restart prometheus.service
fmadio@prometheus0:/etc/prometheus$

Step 3) Confirm Target on the GUI

Then confirm the target is visible on the prometheus servers web page.

Example is plotting out

fmadio_capture_byte

As shown below the target we added “192.168.133.10” is visible in the list, as shown below

Dashboard

FMADIO Reference Dashboard

A reference dashboard using the FMADIO telemetry is located on the FMADIO public Github repo

https://github.com/fmadio/public/blob/master/prometheus/FMADIO.Prometheus-GrafanaDashboard.json

Which gives a good high and detailed overview of the system and provides a starting point for your own dashboarding.

Debugging

In the case of problems, a good base test is to curl the /metrics URL directly to confirm the output is valid.

For example.

 curl -k -u fmadio:100g https://192.168.133.10/metrics

Example full output of the all metrics is shown below.

$ curl -k -u fmadio:***** https://192.168.133.10/metrics
# fmadio_ver number
fmadio_ver 9927
# fmadio_cat_cpu_fetch{ioqueue="0"} number
fmadio_cat_cpu_fetch{ioqueue="0"} 0
# fmadio_cat_cpu_idle{ioqueue="0"} number
fmadio_cat_cpu_idle{ioqueue="0"} 0
# fmadio_cat_cpu_main{ioqueue="0"} number
fmadio_cat_cpu_main{ioqueue="0"} 0
# fmadio_cat_cpu_process{ioqueue="0"} number
fmadio_cat_cpu_process{ioqueue="0"} 0
# fmadio_cat_cpu_send{ioqueue="0"} number
fmadio_cat_cpu_send{ioqueue="0"} 0
# fmadio_cat_chunkid{ioqueue="0"} number
fmadio_cat_chunkid{ioqueue="0"} 0
# fmadio_cat_cat_0_CmdLine string
# fmadio_cat_enable{ioqueue="0"} boolean
fmadio_cat_enable{ioqueue="0"} 0
# fmadio_cat_cat_0_FilterBPF string
# fmadio_cat_iopriority{ioqueue="0"} number
fmadio_cat_iopriority{ioqueue="0"} 0
# fmadio_cat_cat_0_Mode string
# fmadio_cat_pending_byte{ioqueue="0"} number
fmadio_cat_pending_byte{ioqueue="0"} 0
# fmadio_cat_pkt_discard{ioqueue="0"} number
fmadio_cat_pkt_discard{ioqueue="0"} 0
# fmadio_cat_pkt_discard_total{ioqueue="0"} number
fmadio_cat_pkt_discard_total{ioqueue="0"} 0
# fmadio_cat_pkt_slice{ioqueue="0"} number
fmadio_cat_pkt_slice{ioqueue="0"} 0
# fmadio_cat_read_byte{ioqueue="0"} number
fmadio_cat_read_byte{ioqueue="0"} 0
# fmadio_cat_read_gbps{ioqueue="0"} number
fmadio_cat_read_gbps{ioqueue="0"} 0
# fmadio_cat_read_mpps{ioqueue="0"} number
fmadio_cat_read_mpps{ioqueue="0"} 0
# fmadio_cat_read_pkt{ioqueue="0"} number
fmadio_cat_read_pkt{ioqueue="0"} 0
# fmadio_cat_read_byte_total{ioqueue="0"} number
fmadio_cat_read_byte_total{ioqueue="0"} 0
# fmadio_cat_read_pkt_total{ioqueue="0"} number
fmadio_cat_read_pkt_total{ioqueue="0"} 0
# fmadio_cat_cat_0_StreamName string
# fmadio_cat_ts_pcap{ioqueue="0"} number
fmadio_cat_ts_pcap{ioqueue="0"} 0
# fmadio_cat_write_byte{ioqueue="0"} number
fmadio_cat_write_byte{ioqueue="0"} 0
# fmadio_cat_write_gbps{ioqueue="0"} number
fmadio_cat_write_gbps{ioqueue="0"} 0
# fmadio_cat_write_mpps{ioqueue="0"} number
fmadio_cat_write_mpps{ioqueue="0"} 0
# fmadio_cat_write_pkt{ioqueue="0"} number
fmadio_cat_write_pkt{ioqueue="0"} 0
# fmadio_cat_write_byte_total{ioqueue="0"} number
fmadio_cat_write_byte_total{ioqueue="0"} 0
# fmadio_cat_write_pkt_total{ioqueue="0"} number
fmadio_cat_write_pkt_total{ioqueue="0"} 0
# fmadio_cat_cpu_fetch{ioqueue="1"} number
fmadio_cat_cpu_fetch{ioqueue="1"} 0
# fmadio_cat_cpu_idle{ioqueue="1"} number
fmadio_cat_cpu_idle{ioqueue="1"} 0
# fmadio_cat_cpu_main{ioqueue="1"} number
fmadio_cat_cpu_main{ioqueue="1"} 0
# fmadio_cat_cpu_process{ioqueue="1"} number
fmadio_cat_cpu_process{ioqueue="1"} 0
# fmadio_cat_cpu_send{ioqueue="1"} number
fmadio_cat_cpu_send{ioqueue="1"} 0
# fmadio_cat_chunkid{ioqueue="1"} number
fmadio_cat_chunkid{ioqueue="1"} 0
# fmadio_cat_cat_1_CmdLine string
# fmadio_cat_enable{ioqueue="1"} boolean
fmadio_cat_enable{ioqueue="1"} 0
# fmadio_cat_cat_1_FilterBPF string
# fmadio_cat_iopriority{ioqueue="1"} number
fmadio_cat_iopriority{ioqueue="1"} 0
# fmadio_cat_cat_1_Mode string
# fmadio_cat_pending_byte{ioqueue="1"} number
fmadio_cat_pending_byte{ioqueue="1"} 0
# fmadio_cat_pkt_discard{ioqueue="1"} number
fmadio_cat_pkt_discard{ioqueue="1"} 0
# fmadio_cat_pkt_discard_total{ioqueue="1"} number
fmadio_cat_pkt_discard_total{ioqueue="1"} 0
# fmadio_cat_pkt_slice{ioqueue="1"} number
fmadio_cat_pkt_slice{ioqueue="1"} 0
# fmadio_cat_read_byte{ioqueue="1"} number
fmadio_cat_read_byte{ioqueue="1"} 0
# fmadio_cat_read_gbps{ioqueue="1"} number
fmadio_cat_read_gbps{ioqueue="1"} 0
# fmadio_cat_read_mpps{ioqueue="1"} number
fmadio_cat_read_mpps{ioqueue="1"} 0
# fmadio_cat_read_pkt{ioqueue="1"} number
fmadio_cat_read_pkt{ioqueue="1"} 0
# fmadio_cat_read_byte_total{ioqueue="1"} number
fmadio_cat_read_byte_total{ioqueue="1"} 0
# fmadio_cat_read_pkt_total{ioqueue="1"} number
fmadio_cat_read_pkt_total{ioqueue="1"} 0
# fmadio_cat_cat_1_StreamName string
# fmadio_cat_ts_pcap{ioqueue="1"} number
fmadio_cat_ts_pcap{ioqueue="1"} 0
# fmadio_cat_write_byte{ioqueue="1"} number
fmadio_cat_write_byte{ioqueue="1"} 0
# fmadio_cat_write_gbps{ioqueue="1"} number
fmadio_cat_write_gbps{ioqueue="1"} 0
# fmadio_cat_write_mpps{ioqueue="1"} number
fmadio_cat_write_mpps{ioqueue="1"} 0
# fmadio_cat_write_pkt{ioqueue="1"} number
fmadio_cat_write_pkt{ioqueue="1"} 0
# fmadio_cat_write_byte_total{ioqueue="1"} number
fmadio_cat_write_byte_total{ioqueue="1"} 0
# fmadio_cat_write_pkt_total{ioqueue="1"} number
fmadio_cat_write_pkt_total{ioqueue="1"} 0
# fmadio_cat_cpu_fetch{ioqueue="2"} number
fmadio_cat_cpu_fetch{ioqueue="2"} 0
# fmadio_cat_cpu_idle{ioqueue="2"} number
fmadio_cat_cpu_idle{ioqueue="2"} 0
# fmadio_cat_cpu_main{ioqueue="2"} number
fmadio_cat_cpu_main{ioqueue="2"} 0
# fmadio_cat_cpu_process{ioqueue="2"} number
fmadio_cat_cpu_process{ioqueue="2"} 0
# fmadio_cat_cpu_send{ioqueue="2"} number
fmadio_cat_cpu_send{ioqueue="2"} 0
# fmadio_cat_chunkid{ioqueue="2"} number
fmadio_cat_chunkid{ioqueue="2"} 0
# fmadio_cat_cat_2_CmdLine string
# fmadio_cat_enable{ioqueue="2"} boolean
fmadio_cat_enable{ioqueue="2"} 0
# fmadio_cat_cat_2_FilterBPF string
# fmadio_cat_iopriority{ioqueue="2"} number
fmadio_cat_iopriority{ioqueue="2"} 0
# fmadio_cat_cat_2_Mode string
# fmadio_cat_pending_byte{ioqueue="2"} number
fmadio_cat_pending_byte{ioqueue="2"} 0
# fmadio_cat_pkt_discard{ioqueue="2"} number
fmadio_cat_pkt_discard{ioqueue="2"} 0
# fmadio_cat_pkt_discard_total{ioqueue="2"} number
fmadio_cat_pkt_discard_total{ioqueue="2"} 0
# fmadio_cat_pkt_slice{ioqueue="2"} number
fmadio_cat_pkt_slice{ioqueue="2"} 0
# fmadio_cat_read_byte{ioqueue="2"} number
fmadio_cat_read_byte{ioqueue="2"} 0
# fmadio_cat_read_gbps{ioqueue="2"} number
fmadio_cat_read_gbps{ioqueue="2"} 0
# fmadio_cat_read_mpps{ioqueue="2"} number
fmadio_cat_read_mpps{ioqueue="2"} 0
# fmadio_cat_read_pkt{ioqueue="2"} number
fmadio_cat_read_pkt{ioqueue="2"} 0
# fmadio_cat_read_byte_total{ioqueue="2"} number
fmadio_cat_read_byte_total{ioqueue="2"} 0
# fmadio_cat_read_pkt_total{ioqueue="2"} number
fmadio_cat_read_pkt_total{ioqueue="2"} 0
# fmadio_cat_cat_2_StreamName string
# fmadio_cat_ts_pcap{ioqueue="2"} number
fmadio_cat_ts_pcap{ioqueue="2"} 0
# fmadio_cat_write_byte{ioqueue="2"} number
fmadio_cat_write_byte{ioqueue="2"} 0
# fmadio_cat_write_gbps{ioqueue="2"} number
fmadio_cat_write_gbps{ioqueue="2"} 0
# fmadio_cat_write_mpps{ioqueue="2"} number
fmadio_cat_write_mpps{ioqueue="2"} 0
# fmadio_cat_write_pkt{ioqueue="2"} number
fmadio_cat_write_pkt{ioqueue="2"} 0
# fmadio_cat_write_byte_total{ioqueue="2"} number
fmadio_cat_write_byte_total{ioqueue="2"} 0
# fmadio_cat_write_pkt_total{ioqueue="2"} number
fmadio_cat_write_pkt_total{ioqueue="2"} 0
# fmadio_cat_cpu_fetch{ioqueue="3"} number
fmadio_cat_cpu_fetch{ioqueue="3"} 0
# fmadio_cat_cpu_idle{ioqueue="3"} number
fmadio_cat_cpu_idle{ioqueue="3"} 0
# fmadio_cat_cpu_main{ioqueue="3"} number
fmadio_cat_cpu_main{ioqueue="3"} 0
# fmadio_cat_cpu_process{ioqueue="3"} number
fmadio_cat_cpu_process{ioqueue="3"} 0
# fmadio_cat_cpu_send{ioqueue="3"} number
fmadio_cat_cpu_send{ioqueue="3"} 0
# fmadio_cat_chunkid{ioqueue="3"} number
fmadio_cat_chunkid{ioqueue="3"} 0
# fmadio_cat_cat_3_CmdLine string
# fmadio_cat_enable{ioqueue="3"} boolean
fmadio_cat_enable{ioqueue="3"} 0
# fmadio_cat_cat_3_FilterBPF string
# fmadio_cat_iopriority{ioqueue="3"} number
fmadio_cat_iopriority{ioqueue="3"} 0
# fmadio_cat_cat_3_Mode string
# fmadio_cat_pending_byte{ioqueue="3"} number
fmadio_cat_pending_byte{ioqueue="3"} 0
# fmadio_cat_pkt_discard{ioqueue="3"} number
fmadio_cat_pkt_discard{ioqueue="3"} 0
# fmadio_cat_pkt_discard_total{ioqueue="3"} number
fmadio_cat_pkt_discard_total{ioqueue="3"} 0
# fmadio_cat_pkt_slice{ioqueue="3"} number
fmadio_cat_pkt_slice{ioqueue="3"} 0
# fmadio_cat_read_byte{ioqueue="3"} number
fmadio_cat_read_byte{ioqueue="3"} 0
# fmadio_cat_read_gbps{ioqueue="3"} number
fmadio_cat_read_gbps{ioqueue="3"} 0
# fmadio_cat_read_mpps{ioqueue="3"} number
fmadio_cat_read_mpps{ioqueue="3"} 0
# fmadio_cat_read_pkt{ioqueue="3"} number
fmadio_cat_read_pkt{ioqueue="3"} 0
# fmadio_cat_read_byte_total{ioqueue="3"} number
fmadio_cat_read_byte_total{ioqueue="3"} 0
# fmadio_cat_read_pkt_total{ioqueue="3"} number
fmadio_cat_read_pkt_total{ioqueue="3"} 0
# fmadio_cat_cat_3_StreamName string
# fmadio_cat_ts_pcap{ioqueue="3"} number
fmadio_cat_ts_pcap{ioqueue="3"} 0
# fmadio_cat_write_byte{ioqueue="3"} number
fmadio_cat_write_byte{ioqueue="3"} 0
# fmadio_cat_write_gbps{ioqueue="3"} number
fmadio_cat_write_gbps{ioqueue="3"} 0
# fmadio_cat_write_mpps{ioqueue="3"} number
fmadio_cat_write_mpps{ioqueue="3"} 0
# fmadio_cat_write_pkt{ioqueue="3"} number
fmadio_cat_write_pkt{ioqueue="3"} 0
# fmadio_cat_write_byte_total{ioqueue="3"} number
fmadio_cat_write_byte_total{ioqueue="3"} 0
# fmadio_cat_write_pkt_total{ioqueue="3"} number
fmadio_cat_write_pkt_total{ioqueue="3"} 0
# fmadio_cat_cpu_fetch{ioqueue="4"} number
fmadio_cat_cpu_fetch{ioqueue="4"} 0
# fmadio_cat_cpu_idle{ioqueue="4"} number
fmadio_cat_cpu_idle{ioqueue="4"} 0
# fmadio_cat_cpu_main{ioqueue="4"} number
fmadio_cat_cpu_main{ioqueue="4"} 0
# fmadio_cat_cpu_process{ioqueue="4"} number
fmadio_cat_cpu_process{ioqueue="4"} 0
# fmadio_cat_cpu_send{ioqueue="4"} number
fmadio_cat_cpu_send{ioqueue="4"} 0
# fmadio_cat_chunkid{ioqueue="4"} number
fmadio_cat_chunkid{ioqueue="4"} 0
# fmadio_cat_cat_4_CmdLine string
# fmadio_cat_enable{ioqueue="4"} boolean
fmadio_cat_enable{ioqueue="4"} 0
# fmadio_cat_cat_4_FilterBPF string
# fmadio_cat_iopriority{ioqueue="4"} number
fmadio_cat_iopriority{ioqueue="4"} 0
# fmadio_cat_cat_4_Mode string
# fmadio_cat_pending_byte{ioqueue="4"} number
fmadio_cat_pending_byte{ioqueue="4"} 0
# fmadio_cat_pkt_discard{ioqueue="4"} number
fmadio_cat_pkt_discard{ioqueue="4"} 0
# fmadio_cat_pkt_discard_total{ioqueue="4"} number
fmadio_cat_pkt_discard_total{ioqueue="4"} 0
# fmadio_cat_pkt_slice{ioqueue="4"} number
fmadio_cat_pkt_slice{ioqueue="4"} 0
# fmadio_cat_read_byte{ioqueue="4"} number
fmadio_cat_read_byte{ioqueue="4"} 0
# fmadio_cat_read_gbps{ioqueue="4"} number
fmadio_cat_read_gbps{ioqueue="4"} 0
# fmadio_cat_read_mpps{ioqueue="4"} number
fmadio_cat_read_mpps{ioqueue="4"} 0
# fmadio_cat_read_pkt{ioqueue="4"} number
fmadio_cat_read_pkt{ioqueue="4"} 0
# fmadio_cat_read_byte_total{ioqueue="4"} number
fmadio_cat_read_byte_total{ioqueue="4"} 0
# fmadio_cat_read_pkt_total{ioqueue="4"} number
fmadio_cat_read_pkt_total{ioqueue="4"} 0
# fmadio_cat_cat_4_StreamName string
# fmadio_cat_ts_pcap{ioqueue="4"} number
fmadio_cat_ts_pcap{ioqueue="4"} 0
# fmadio_cat_write_byte{ioqueue="4"} number
fmadio_cat_write_byte{ioqueue="4"} 0
# fmadio_cat_write_gbps{ioqueue="4"} number
fmadio_cat_write_gbps{ioqueue="4"} 0
# fmadio_cat_write_mpps{ioqueue="4"} number
fmadio_cat_write_mpps{ioqueue="4"} 0
# fmadio_cat_write_pkt{ioqueue="4"} number
fmadio_cat_write_pkt{ioqueue="4"} 0
# fmadio_cat_write_byte_total{ioqueue="4"} number
fmadio_cat_write_byte_total{ioqueue="4"} 0
# fmadio_cat_write_pkt_total{ioqueue="4"} number
fmadio_cat_write_pkt_total{ioqueue="4"} 0
# fmadio_cat_cpu_fetch{ioqueue="5"} number
fmadio_cat_cpu_fetch{ioqueue="5"} 0
# fmadio_cat_cpu_idle{ioqueue="5"} number
fmadio_cat_cpu_idle{ioqueue="5"} 0
# fmadio_cat_cpu_main{ioqueue="5"} number
fmadio_cat_cpu_main{ioqueue="5"} 0
# fmadio_cat_cpu_process{ioqueue="5"} number
fmadio_cat_cpu_process{ioqueue="5"} 0
# fmadio_cat_cpu_send{ioqueue="5"} number
fmadio_cat_cpu_send{ioqueue="5"} 0
# fmadio_cat_chunkid{ioqueue="5"} number
fmadio_cat_chunkid{ioqueue="5"} 0
# fmadio_cat_cat_5_CmdLine string
# fmadio_cat_enable{ioqueue="5"} boolean
fmadio_cat_enable{ioqueue="5"} 0
# fmadio_cat_cat_5_FilterBPF string
# fmadio_cat_iopriority{ioqueue="5"} number
fmadio_cat_iopriority{ioqueue="5"} 0
# fmadio_cat_cat_5_Mode string
# fmadio_cat_pending_byte{ioqueue="5"} number
fmadio_cat_pending_byte{ioqueue="5"} 0
# fmadio_cat_pkt_discard{ioqueue="5"} number
fmadio_cat_pkt_discard{ioqueue="5"} 0
# fmadio_cat_pkt_discard_total{ioqueue="5"} number
fmadio_cat_pkt_discard_total{ioqueue="5"} 0
# fmadio_cat_pkt_slice{ioqueue="5"} number
fmadio_cat_pkt_slice{ioqueue="5"} 0
# fmadio_cat_read_byte{ioqueue="5"} number
fmadio_cat_read_byte{ioqueue="5"} 0
# fmadio_cat_read_gbps{ioqueue="5"} number
fmadio_cat_read_gbps{ioqueue="5"} 0
# fmadio_cat_read_mpps{ioqueue="5"} number
fmadio_cat_read_mpps{ioqueue="5"} 0
# fmadio_cat_read_pkt{ioqueue="5"} number
fmadio_cat_read_pkt{ioqueue="5"} 0
# fmadio_cat_read_byte_total{ioqueue="5"} number
fmadio_cat_read_byte_total{ioqueue="5"} 0
# fmadio_cat_read_pkt_total{ioqueue="5"} number
fmadio_cat_read_pkt_total{ioqueue="5"} 0
# fmadio_cat_cat_5_StreamName string
# fmadio_cat_ts_pcap{ioqueue="5"} number
fmadio_cat_ts_pcap{ioqueue="5"} 0
# fmadio_cat_write_byte{ioqueue="5"} number
fmadio_cat_write_byte{ioqueue="5"} 0
# fmadio_cat_write_gbps{ioqueue="5"} number
fmadio_cat_write_gbps{ioqueue="5"} 0
# fmadio_cat_write_mpps{ioqueue="5"} number
fmadio_cat_write_mpps{ioqueue="5"} 0
# fmadio_cat_write_pkt{ioqueue="5"} number
fmadio_cat_write_pkt{ioqueue="5"} 0
# fmadio_cat_write_byte_total{ioqueue="5"} number
fmadio_cat_write_byte_total{ioqueue="5"} 0
# fmadio_cat_write_pkt_total{ioqueue="5"} number
fmadio_cat_write_pkt_total{ioqueue="5"} 0
# fmadio_cat_cpu_fetch{ioqueue="6"} number
fmadio_cat_cpu_fetch{ioqueue="6"} 0
# fmadio_cat_cpu_idle{ioqueue="6"} number
fmadio_cat_cpu_idle{ioqueue="6"} 0
# fmadio_cat_cpu_main{ioqueue="6"} number
fmadio_cat_cpu_main{ioqueue="6"} 0
# fmadio_cat_cpu_process{ioqueue="6"} number
fmadio_cat_cpu_process{ioqueue="6"} 0
# fmadio_cat_cpu_send{ioqueue="6"} number
fmadio_cat_cpu_send{ioqueue="6"} 0
# fmadio_cat_chunkid{ioqueue="6"} number
fmadio_cat_chunkid{ioqueue="6"} 0
# fmadio_cat_cat_6_CmdLine string
# fmadio_cat_enable{ioqueue="6"} boolean
fmadio_cat_enable{ioqueue="6"} 0
# fmadio_cat_cat_6_FilterBPF string
# fmadio_cat_iopriority{ioqueue="6"} number
fmadio_cat_iopriority{ioqueue="6"} 0
# fmadio_cat_cat_6_Mode string
# fmadio_cat_pending_byte{ioqueue="6"} number
fmadio_cat_pending_byte{ioqueue="6"} 0
# fmadio_cat_pkt_discard{ioqueue="6"} number
fmadio_cat_pkt_discard{ioqueue="6"} 0
# fmadio_cat_pkt_discard_total{ioqueue="6"} number
fmadio_cat_pkt_discard_total{ioqueue="6"} 0
# fmadio_cat_pkt_slice{ioqueue="6"} number
fmadio_cat_pkt_slice{ioqueue="6"} 0
# fmadio_cat_read_byte{ioqueue="6"} number
fmadio_cat_read_byte{ioqueue="6"} 0
# fmadio_cat_read_gbps{ioqueue="6"} number
fmadio_cat_read_gbps{ioqueue="6"} 0
# fmadio_cat_read_mpps{ioqueue="6"} number
fmadio_cat_read_mpps{ioqueue="6"} 0
# fmadio_cat_read_pkt{ioqueue="6"} number
fmadio_cat_read_pkt{ioqueue="6"} 0
# fmadio_cat_read_byte_total{ioqueue="6"} number
fmadio_cat_read_byte_total{ioqueue="6"} 0
# fmadio_cat_read_pkt_total{ioqueue="6"} number
fmadio_cat_read_pkt_total{ioqueue="6"} 0
# fmadio_cat_cat_6_StreamName string
# fmadio_cat_ts_pcap{ioqueue="6"} number
fmadio_cat_ts_pcap{ioqueue="6"} 0
# fmadio_cat_write_byte{ioqueue="6"} number
fmadio_cat_write_byte{ioqueue="6"} 0
# fmadio_cat_write_gbps{ioqueue="6"} number
fmadio_cat_write_gbps{ioqueue="6"} 0
# fmadio_cat_write_mpps{ioqueue="6"} number
fmadio_cat_write_mpps{ioqueue="6"} 0
# fmadio_cat_write_pkt{ioqueue="6"} number
fmadio_cat_write_pkt{ioqueue="6"} 0
# fmadio_cat_write_byte_total{ioqueue="6"} number
fmadio_cat_write_byte_total{ioqueue="6"} 0
# fmadio_cat_write_pkt_total{ioqueue="6"} number
fmadio_cat_write_pkt_total{ioqueue="6"} 0
# fmadio_cat_cpu_fetch{ioqueue="7"} number
fmadio_cat_cpu_fetch{ioqueue="7"} 0
# fmadio_cat_cpu_idle{ioqueue="7"} number
fmadio_cat_cpu_idle{ioqueue="7"} 0
# fmadio_cat_cpu_main{ioqueue="7"} number
fmadio_cat_cpu_main{ioqueue="7"} 0
# fmadio_cat_cpu_process{ioqueue="7"} number
fmadio_cat_cpu_process{ioqueue="7"} 0
# fmadio_cat_cpu_send{ioqueue="7"} number
fmadio_cat_cpu_send{ioqueue="7"} 0
# fmadio_cat_chunkid{ioqueue="7"} number
fmadio_cat_chunkid{ioqueue="7"} 0
# fmadio_cat_cat_7_CmdLine string
# fmadio_cat_enable{ioqueue="7"} boolean
fmadio_cat_enable{ioqueue="7"} 0
# fmadio_cat_cat_7_FilterBPF string
# fmadio_cat_iopriority{ioqueue="7"} number
fmadio_cat_iopriority{ioqueue="7"} 0
# fmadio_cat_cat_7_Mode string
# fmadio_cat_pending_byte{ioqueue="7"} number
fmadio_cat_pending_byte{ioqueue="7"} 0
# fmadio_cat_pkt_discard{ioqueue="7"} number
fmadio_cat_pkt_discard{ioqueue="7"} 0
# fmadio_cat_pkt_discard_total{ioqueue="7"} number
fmadio_cat_pkt_discard_total{ioqueue="7"} 0
# fmadio_cat_pkt_slice{ioqueue="7"} number
fmadio_cat_pkt_slice{ioqueue="7"} 0
# fmadio_cat_read_byte{ioqueue="7"} number
fmadio_cat_read_byte{ioqueue="7"} 0
# fmadio_cat_read_gbps{ioqueue="7"} number
fmadio_cat_read_gbps{ioqueue="7"} 0
# fmadio_cat_read_mpps{ioqueue="7"} number
fmadio_cat_read_mpps{ioqueue="7"} 0
# fmadio_cat_read_pkt{ioqueue="7"} number
fmadio_cat_read_pkt{ioqueue="7"} 0
# fmadio_cat_read_byte_total{ioqueue="7"} number
fmadio_cat_read_byte_total{ioqueue="7"} 0
# fmadio_cat_read_pkt_total{ioqueue="7"} number
fmadio_cat_read_pkt_total{ioqueue="7"} 0
# fmadio_cat_cat_7_StreamName string
# fmadio_cat_ts_pcap{ioqueue="7"} number
fmadio_cat_ts_pcap{ioqueue="7"} 0
# fmadio_cat_write_byte{ioqueue="7"} number
fmadio_cat_write_byte{ioqueue="7"} 0
# fmadio_cat_write_gbps{ioqueue="7"} number
fmadio_cat_write_gbps{ioqueue="7"} 0
# fmadio_cat_write_mpps{ioqueue="7"} number
fmadio_cat_write_mpps{ioqueue="7"} 0
# fmadio_cat_write_pkt{ioqueue="7"} number
fmadio_cat_write_pkt{ioqueue="7"} 0
# fmadio_cat_write_byte_total{ioqueue="7"} number
fmadio_cat_write_byte_total{ioqueue="7"} 0
# fmadio_cat_write_pkt_total{ioqueue="7"} number
fmadio_cat_write_pkt_total{ioqueue="7"} 0
# fmadio_cat_cat_EnableCnt number
fmadio_cat_cat_EnableCnt 0
# fmadio_cat_cat_LockCnt number
fmadio_cat_cat_LockCnt 0
# fmadio_cat_cat_ReadByte number
fmadio_cat_cat_ReadByte 0
# fmadio_cat_cat_ReadGbps number
fmadio_cat_cat_ReadGbps 0
# fmadio_cat_cat_ReadMpps number
fmadio_cat_cat_ReadMpps 0
# fmadio_cat_cat_ReadPkt number
fmadio_cat_cat_ReadPkt 0
# fmadio_cat_cat_ReadTotalByte number
fmadio_cat_cat_ReadTotalByte 0
# fmadio_cat_cat_ReadTotalPkt number
fmadio_cat_cat_ReadTotalPkt 0
# fmadio_cat_cat_WriteByte number
fmadio_cat_cat_WriteByte 0
# fmadio_cat_cat_WriteGbps number
fmadio_cat_cat_WriteGbps 0
# fmadio_cat_cat_WriteMpps number
fmadio_cat_cat_WriteMpps 0
# fmadio_cat_cat_WritePkt number
fmadio_cat_cat_WritePkt 0
# fmadio_cat_cat_WriteTotalByte number
fmadio_cat_cat_WriteTotalByte 0
# fmadio_cat_cat_WriteTotalPkt number
fmadio_cat_cat_WriteTotalPkt 0
# fmadio_cat_module string
# fmadio_cat_subsystem string
# fmadio_cat_timestamp number
# fmadio_cat_ver string
# fmadio_disk_valid{unit="hdd0"} boolean
fmadio_disk_valid{unit="hdd0"} 1
# fmadio_disk_valid{unit="hdd1"} boolean
fmadio_disk_valid{unit="hdd1"} 1
# fmadio_disk_valid{unit="hdd2"} boolean
fmadio_disk_valid{unit="hdd2"} 1
# fmadio_disk_valid{unit="os0"} boolean
fmadio_disk_valid{unit="os0"} 1
# fmadio_disk_valid{unit="par0"} boolean
fmadio_disk_valid{unit="par0"} 1
# fmadio_disk_valid{unit="scr0"} boolean
fmadio_disk_valid{unit="scr0"} 1
# fmadio_disk_valid{unit="scr1"} boolean
fmadio_disk_valid{unit="scr1"} 1
# fmadio_disk_valid{unit="ssd0"} boolean
fmadio_disk_valid{unit="ssd0"} 1
# fmadio_disk_valid{unit="ssd1"} boolean
fmadio_disk_valid{unit="ssd1"} 1
# fmadio_disk_error{unit="hdd0"} number
fmadio_disk_error{unit="hdd0"} 0
# fmadio_disk_error{unit="hdd1"} number
fmadio_disk_error{unit="hdd1"} 0
# fmadio_disk_error{unit="hdd2"} number
fmadio_disk_error{unit="hdd2"} 0
# fmadio_disk_error{unit="os0"} number
fmadio_disk_error{unit="os0"} 0
# fmadio_disk_error{unit="par0"} number
fmadio_disk_error{unit="par0"} 0
# fmadio_disk_error{unit="scr0"} number
fmadio_disk_error{unit="scr0"} 0
# fmadio_disk_error{unit="scr1"} number
fmadio_disk_error{unit="scr1"} 0
# fmadio_disk_error{unit="ssd0"} number
fmadio_disk_error{unit="ssd0"} 0
# fmadio_disk_error{unit="ssd1"} number
fmadio_disk_error{unit="ssd1"} 0
# fmadio_temperature{unit="hdd0"} number
fmadio_temperature{unit="hdd0"} 28
# fmadio_temperature{unit="hdd1"} number
fmadio_temperature{unit="hdd1"} 27
# fmadio_temperature{unit="hdd2"} number
fmadio_temperature{unit="hdd2"} 25
# fmadio_temperature{unit="os0"} number
fmadio_temperature{unit="os0"} 25
# fmadio_temperature{unit="par0"} number
fmadio_temperature{unit="par0"} 26
# fmadio_temperature{unit="scr0"} number
fmadio_temperature{unit="scr0"} 35
# fmadio_temperature{unit="scr1"} number
fmadio_temperature{unit="scr1"} 32
# fmadio_temperature{unit="ssd0"} number
fmadio_temperature{unit="ssd0"} 33
# fmadio_temperature{unit="ssd1"} number
fmadio_temperature{unit="ssd1"} 35
# fmadio_disk_free{unit="remote0"} number
fmadio_disk_free{unit="remote0"} 7896130000000
# fmadio_disk_free{unit="remote1"} number
fmadio_disk_free{unit="remote1"} 7896130000000
# fmadio_disk_free{unit="store0"} number
fmadio_disk_free{unit="store0"} 96389000000
# fmadio_disk_free{unit="store1"} number
fmadio_disk_free{unit="store1"} 1243846000000
# fmadio_disk_free{unit="system"} number
fmadio_disk_free{unit="system"} 1685000000
# fmadio_disk_module string
# fmadio_disk_subsystem string
# fmadio_disk_timestamp number
# fmadio_disk_ver string
# fmadio_counter_module string
# fmadio_counter_subsystem string
# fmadio_counter_timestamp number
# fmadio_counter_updateTS number
fmadio_counter_updateTS 1729674305131685120
# fmadio_counter_ver string
# fmadio_system_cpu number
fmadio_system_cpu 0
# fmadio_other_CacheSize number
fmadio_other_CacheSize 2048418906112
# fmadio_other_FDCnt number
fmadio_other_FDCnt 1681
# fmadio_other_MemBuffer number
fmadio_other_MemBuffer 524369920
# fmadio_other_MemCached number
fmadio_other_MemCached 25623552000
# fmadio_other_MemDirty number
fmadio_other_MemDirty 69632
# fmadio_other_MemErrorECC number
fmadio_other_MemErrorECC 0
# fmadio_system_memfree number
fmadio_system_memfree 47938289664
# fmadio_other_MemMapped number
fmadio_other_MemMapped 3728523264
# fmadio_other_PageInByte number
fmadio_other_PageInByte 0
# fmadio_other_PortConfig string
# fmadio_other_SerialNo string
# fmadio_other_StoreSize number
fmadio_other_StoreSize 12002352168960
# fmadio_other_UptimeHour number
fmadio_other_UptimeHour 166
# fmadio_other_Version string
# fmadio_other_WritebackB number
fmadio_other_WritebackB 0
# fmadio_other_WritebackDropG number
fmadio_other_WritebackDropG 0
# fmadio_other_WritebackDropTotalG number
fmadio_other_WritebackDropTotalG 0
# fmadio_other_WritebackPct number
fmadio_other_WritebackPct 0
# fmadio_other_module string
# fmadio_other_subsystem string
# fmadio_other_timestamp number
# fmadio_other_ver string
# fmadio_cpu_TotalCPU_Cnt number
fmadio_cpu_TotalCPU_Cnt 96
# fmadio_cpu_TotalCPU_IsolCnt number
fmadio_cpu_TotalCPU_IsolCnt 30
# fmadio_cpu_TotalCPU_LXCCnt number
fmadio_cpu_TotalCPU_LXCCnt 7
# fmadio_cpu_TotalCPU_LXCPct number
fmadio_cpu_TotalCPU_LXCPct 0
# fmadio_cpu_TotalCPU_Pct number
fmadio_cpu_TotalCPU_Pct 0
# fmadio_cpu_dSampleTS number
fmadio_cpu_dSampleTS 60212186112
# fmadio_cpu_module string
# fmadio_cpu_subsystem string
# fmadio_cpu_timestamp number
# fmadio_cpu_ver string
# fmadio_capture_byte{interface="all"} number
fmadio_capture_byte{interface="all"} 0
# fmadio_capture_drop{interface="all"} number
fmadio_capture_drop{interface="all"} 0
# fmadio_capture_enable boolean
fmadio_capture_enable 0
# fmadio_capture_fcs_error{interface="all"} number
fmadio_capture_fcs_error{interface="all"} 0
# fmadio_capture_CaptureName string
# fmadio_capture_pkt{interface="all"} number
fmadio_capture_pkt{interface="all"} 0
# fmadio_capture_byte{interface="cap0"} number
fmadio_capture_byte{interface="cap0"} 0
# fmadio_capture_fcs_error{interface="cap0"} number
fmadio_capture_fcs_error{interface="cap0"} 0
# fmadio_capture_pkt{interface="cap0"} number
fmadio_capture_pkt{interface="cap0"} 0
# fmadio_capture_byte{interface="cap1"} number
fmadio_capture_byte{interface="cap1"} 0
# fmadio_capture_fcs_error{interface="cap1"} number
fmadio_capture_fcs_error{interface="cap1"} 0
# fmadio_capture_pkt{interface="cap1"} number
fmadio_capture_pkt{interface="cap1"} 0
# fmadio_capture_byte{interface="cap2"} number
fmadio_capture_byte{interface="cap2"} 0
# fmadio_capture_fcs_error{interface="cap2"} number
fmadio_capture_fcs_error{interface="cap2"} 0
# fmadio_capture_pkt{interface="cap2"} number
fmadio_capture_pkt{interface="cap2"} 0
# fmadio_capture_byte{interface="cap3"} number
fmadio_capture_byte{interface="cap3"} 0
# fmadio_capture_fcs_error{interface="cap3"} number
fmadio_capture_fcs_error{interface="cap3"} 0
# fmadio_capture_pkt{interface="cap3"} number
fmadio_capture_pkt{interface="cap3"} 0
# fmadio_capture_byte{interface="cap4"} number
fmadio_capture_byte{interface="cap4"} 0
# fmadio_capture_fcs_error{interface="cap4"} number
fmadio_capture_fcs_error{interface="cap4"} 0
# fmadio_capture_pkt{interface="cap4"} number
fmadio_capture_pkt{interface="cap4"} 0
# fmadio_capture_byte{interface="cap5"} number
fmadio_capture_byte{interface="cap5"} 0
# fmadio_capture_fcs_error{interface="cap5"} number
fmadio_capture_fcs_error{interface="cap5"} 0
# fmadio_capture_pkt{interface="cap5"} number
fmadio_capture_pkt{interface="cap5"} 0
# fmadio_capture_byte{interface="cap6"} number
fmadio_capture_byte{interface="cap6"} 0
# fmadio_capture_fcs_error{interface="cap6"} number
fmadio_capture_fcs_error{interface="cap6"} 0
# fmadio_capture_pkt{interface="cap6"} number
fmadio_capture_pkt{interface="cap6"} 0
# fmadio_capture_byte{interface="cap7"} number
fmadio_capture_byte{interface="cap7"} 0
# fmadio_capture_fcs_error{interface="cap7"} number
fmadio_capture_fcs_error{interface="cap7"} 0
# fmadio_capture_pkt{interface="cap7"} number
fmadio_capture_pkt{interface="cap7"} 0
# fmadio_capture_CaptureRateGbps number
# fmadio_capture_CaptureRateMpps number
# fmadio_capture_rate_bps number
fmadio_capture_rate_bps 0
# fmadio_capture_rate_pps number
fmadio_capture_rate_pps 0
# fmadio_capture_module string
# fmadio_capture_subsystem string
# fmadio_capture_timestamp number
# fmadio_capture_ver string
# fmadio_temperature{unit="airin"} number
fmadio_temperature{unit="airin"} 22
# fmadio_temperature{unit="airout"} number
fmadio_temperature{unit="airout"} 0
# fmadio_temperature{unit="cpu0"} number
fmadio_temperature{unit="cpu0"} 36
# fmadio_temperature{unit="cpu1"} number
fmadio_temperature{unit="cpu1"} 47
# fmadio_temperature{unit="nic"} number
fmadio_temperature{unit="nic"} 54
# fmadio_temperature{unit="chipset"} number
fmadio_temperature{unit="chipset"} 39
# fmadio_temperature{unit="perhipherals"} number
fmadio_temperature{unit="perhipherals"} 22
# fmadio_temperature{unit="system"} number
fmadio_temperature{unit="system"} 35
# fmadio_temperature{unit="transceiver0"} number
fmadio_temperature{unit="transceiver0"} 38
# fmadio_temperature{unit="transceiver1"} number
fmadio_temperature{unit="transceiver1"} 40
# fmadio_temperature_module string
# fmadio_temperature_subsystem string
# fmadio_temperature_timestamp number
# fmadio_temperature_ver string
# fmadio_ptp_Clk156Offset number
fmadio_ptp_Clk156Offset 0
# fmadio_ptp_Clk156Period number
fmadio_ptp_Clk156Period 6
# fmadio_ptp_Clk250Offset number
fmadio_ptp_Clk250Offset 2
# fmadio_ptp_Clk250Period number
fmadio_ptp_Clk250Period 6
# fmadio_ptp_Clk322Offset number
fmadio_ptp_Clk322Offset 0
# fmadio_ptp_Clk322Period number
fmadio_ptp_Clk322Period 0
# fmadio_ptp_GMMaster string
# fmadio_ptp_GMOffset number
fmadio_ptp_GMOffset 0
# fmadio_ptp_GMSync boolean
fmadio_ptp_GMSync 0
# fmadio_ptp_GMUpTime number
fmadio_ptp_GMUpTime 0
# fmadio_ptp_NTPMaster string
# fmadio_ptp_NTPOffset number
fmadio_ptp_NTPOffset 0
# fmadio_ptp_NTPSync boolean
fmadio_ptp_NTPSync 0
# fmadio_ptp_NTPUptime number
fmadio_ptp_NTPUptime 0
# fmadio_ptp_PPSCnt number
fmadio_ptp_PPSCnt 0
# fmadio_ptp_PPSOffset number
fmadio_ptp_PPSOffset 0
# fmadio_ptp_PPSPeriod number
fmadio_ptp_PPSPeriod 0
# fmadio_ptp_PPSUptime number
fmadio_ptp_PPSUptime 0
# fmadio_ptp_PPSdPhase number
fmadio_ptp_PPSdPhase 0
# fmadio_ptp_PPSdZero number
fmadio_ptp_PPSdZero 0
# fmadio_ptp_SysOffset number
fmadio_ptp_SysOffset 0
# fmadio_ptp_SysSync boolean
fmadio_ptp_SysSync 0
# fmadio_ptp_SysUptime number
fmadio_ptp_SysUptime 0
# fmadio_ptp_TimeFPGA number
fmadio_ptp_TimeFPGA 1729674305130944000
# fmadio_ptp_TimeSYS number
fmadio_ptp_TimeSYS 1729674305130949120
# fmadio_ptp_iSysOffset number
fmadio_ptp_iSysOffset 0
# fmadio_ptp_iSysSync boolean
fmadio_ptp_iSysSync 1
# fmadio_ptp_iSysUptime number
fmadio_ptp_iSysUptime 597953
# fmadio_ptp_module string
# fmadio_ptp_subsystem string
# fmadio_ptp_timestamp number
# fmadio_ptp_ver string
# fmadio_disk_io_bps{unit="read"} number
fmadio_disk_io_bps{unit="read"} 0
# fmadio_disk_io_bps{unit="write"} number
fmadio_disk_io_bps{unit="write"} 0
# fmadio_io_module string
# fmadio_io_subsystem string
# fmadio_io_timestamp number
# fmadio_io_ver string
# fmadio_link_up{interface="cap0"} boolean
fmadio_link_up{interface="cap0"} 1
# fmadio_link_power_rx{interface="cap0"} number
fmadio_link_power_rx{interface="cap0"} 0
# fmadio_link_up{interface="cap1"} boolean
fmadio_link_up{interface="cap1"} 1
# fmadio_link_power_rx{interface="cap1"} number
fmadio_link_power_rx{interface="cap1"} 0
# fmadio_link_up{interface="cap2"} boolean
fmadio_link_up{interface="cap2"} 0
# fmadio_link_power_rx{interface="cap2"} number
fmadio_link_power_rx{interface="cap2"} 0
# fmadio_link_up{interface="cap3"} boolean
fmadio_link_up{interface="cap3"} 0
# fmadio_link_power_rx{interface="cap3"} number
fmadio_link_power_rx{interface="cap3"} 0
# fmadio_link_up{interface="cap4"} boolean
fmadio_link_up{interface="cap4"} 0
# fmadio_link_power_rx{interface="cap4"} number
fmadio_link_power_rx{interface="cap4"} 0
# fmadio_link_up{interface="cap5"} boolean
fmadio_link_up{interface="cap5"} 0
# fmadio_link_power_rx{interface="cap5"} number
fmadio_link_power_rx{interface="cap5"} 0
# fmadio_link_up{interface="cap6"} boolean
fmadio_link_up{interface="cap6"} 0
# fmadio_link_power_rx{interface="cap6"} number
fmadio_link_power_rx{interface="cap6"} 0
# fmadio_link_up{interface="cap7"} boolean
fmadio_link_up{interface="cap7"} 0
# fmadio_link_power_rx{interface="cap7"} number
fmadio_link_power_rx{interface="cap7"} 0
# fmadio_link_man0_dbyte_rx number
fmadio_link_man0_dbyte_rx 0
# fmadio_link_man0_dbyte_tx number
fmadio_link_man0_dbyte_tx 0
# fmadio_link_up{interface="man0"} boolean
fmadio_link_up{interface="man0"} 1
# fmadio_link_man10_dbyte_rx number
fmadio_link_man10_dbyte_rx 0
# fmadio_link_man10_dbyte_tx number
fmadio_link_man10_dbyte_tx 0
# fmadio_link_up{interface="man10"} boolean
fmadio_link_up{interface="man10"} 1
# fmadio_link_man11_dbyte_rx number
fmadio_link_man11_dbyte_rx 0
# fmadio_link_man11_dbyte_tx number
fmadio_link_man11_dbyte_tx 0
# fmadio_link_man1_dbyte_rx number
fmadio_link_man1_dbyte_rx 0
# fmadio_link_man1_dbyte_tx number
fmadio_link_man1_dbyte_tx 0
# fmadio_link_module string
# fmadio_link_phy0_dbyte_rx number
fmadio_link_phy0_dbyte_rx 0
# fmadio_link_phy0_dbyte_tx number
fmadio_link_phy0_dbyte_tx 0
# fmadio_link_phy10_dbyte_rx number
fmadio_link_phy10_dbyte_rx 0
# fmadio_link_phy10_dbyte_tx number
fmadio_link_phy10_dbyte_tx 0
# fmadio_link_phy11_dbyte_rx number
fmadio_link_phy11_dbyte_rx 0
# fmadio_link_phy11_dbyte_tx number
fmadio_link_phy11_dbyte_tx 0
# fmadio_link_phy1_dbyte_rx number
fmadio_link_phy1_dbyte_rx 0
# fmadio_link_phy1_dbyte_tx number
fmadio_link_phy1_dbyte_tx 0
# fmadio_link_subsystem string
# fmadio_link_timestamp number
# fmadio_link_ver string
# fmadio_power_status{psu="0"} boolean
fmadio_power_status{psu="0"} 0
# fmadio_power_status{psu="1"} boolean
fmadio_power_status{psu="1"} 1
# fmadio_power_watt number
fmadio_power_watt 200
# fmadio_power_module string
# fmadio_power_subsystem string
# fmadio_power_timestamp number
# fmadio_power_ver string
# fmadio_fan_system{interface="SYS0"} number
fmadio_fan_system{interface="SYS0"} 13500
# fmadio_fan_system{interface="SYS1"} number
fmadio_fan_system{interface="SYS1"} 33600
# fmadio_fan_system{interface="SYS2"} number
fmadio_fan_system{interface="SYS2"} 33450
# fmadio_fan_system{interface="SYS3"} number
fmadio_fan_system{interface="SYS3"} 13800
# fmadio_fan_system{interface="SYS4"} number
fmadio_fan_system{interface="SYS4"} 13650
# fmadio_fan_system{interface="SYS5"} number
fmadio_fan_system{interface="SYS5"} 13800
# fmadio_fan_system{interface="SYS6"} number
fmadio_fan_system{interface="SYS6"} 13800
# fmadio_fan_system{interface="SYS7"} number
fmadio_fan_system{interface="SYS7"} 13800
# fmadio_fan_module string
# fmadio_fan_subsystem string
# fmadio_fan_timestamp number
# fmadio_fan_ver string