Monitoring Tricks

Prev Next

Various one lines or small tricks to extract information from the logs

jq Log extraction

FMADIO relies heavily on JSON style logging, as such having an easy way to selectively extract data from the logs is useful.

Below extract 2 fields from the syslog, specifically the link status

cat messages | grep link | stripJSON | jq -c "{timestamp:(.timestamp|todate),cap0_link,cap1_link}"

Example output

fmadio@ny2-fmadio200v4-670:/mnt/store0/log$ cat messages | grep link | stripJSON | jq -c "{timestamp:(.timestamp|todate),cap0_link,cap1_link}" | head
{"timestamp":"2026-05-03T03:55:35Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:55:53Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:04Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:07Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:24Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:35Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:53Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:03Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:06Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:24Z","cap0_link":true,"cap1_link":true}

Extract Push PCAP Copy status

To list all fields on the push_pcap copy status

 cat messages | grep push_pcap | grep copy | stripJSON  | jq

Extract the Retry and Filename

 cat messages | grep push_pcap | grep copy | stripJSON  | jq -c "{timestamp:.timestamp|todate,Remote_Retry,RemoteFileName}"  

Example output

fmadio@ny2-fmadio200v4-670:/mnt/store0/log$  cat messages | grep push_pcap | grep copy | stripJSON  | jq -c "{timestamp:.timestamp|todate,Remote_Retry,RemoteFileName}"  | head
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/.../opra-05-1777778940.pcap.zst"}
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/../opra-03a-1777778940.pcap.zst"}
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/../opra-08b-1777778940.pcap.zst"}
.
.

Extract Failed Copys

To extract a list of failed remote copys. Using the NOT OK grep at the end selects any copy that did not pass

cat messages | grep push_pcap | grep copy | stripJSON  | jq -c "{timestamp:.timestamp|todate,Status,RemoteFileName}" | grep -v OK 

FMADIO Packet Capture systems