Various one lines or small tricks to extract information from the logs
jq Log extraction
FMADIO relies heavily on JSON style logging, as such having an easy way to selectively extract data from the logs is useful.
Below extract 2 fields from the syslog, specifically the link status
cat messages | grep link | stripJSON | jq -c "{timestamp:(.timestamp|todate),cap0_link,cap1_link}"Example output
fmadio@ny2-fmadio200v4-670:/mnt/store0/log$ cat messages | grep link | stripJSON | jq -c "{timestamp:(.timestamp|todate),cap0_link,cap1_link}" | head
{"timestamp":"2026-05-03T03:55:35Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:55:53Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:04Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:07Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:24Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:35Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:56:53Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:03Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:06Z","cap0_link":true,"cap1_link":true}
{"timestamp":"2026-05-03T03:57:24Z","cap0_link":true,"cap1_link":true}
Extract Push PCAP Copy status
To list all fields on the push_pcap copy status
cat messages | grep push_pcap | grep copy | stripJSON | jqExtract the Retry and Filename
cat messages | grep push_pcap | grep copy | stripJSON | jq -c "{timestamp:.timestamp|todate,Remote_Retry,RemoteFileName}" Example output
fmadio@ny2-fmadio200v4-670:/mnt/store0/log$ cat messages | grep push_pcap | grep copy | stripJSON | jq -c "{timestamp:.timestamp|todate,Remote_Retry,RemoteFileName}" | head
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/.../opra-05-1777778940.pcap.zst"}
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/../opra-03a-1777778940.pcap.zst"}
{"timestamp":"2026-05-03T03:56:07Z","Remote_Retry":0,"RemoteFileName":"/mnt/store1/s3_pcap/../opra-08b-1777778940.pcap.zst"}
.
.Extract Failed Copys
To extract a list of failed remote copys. Using the NOT OK grep at the end selects any copy that did not pass
cat messages | grep push_pcap | grep copy | stripJSON | jq -c "{timestamp:.timestamp|todate,Status,RemoteFileName}" | grep -v OK