RTP Offline Process

  • Updated on May 13, 2025
  • Published on May 8, 2025
  • 2 minute(s) read
  • aA
 Prev Next

To run the RTP Offline decode process follow the following steps

RTP Offline

Step 1) Start the decode processor

Inside the LXC container run the following

fmadio2pcap -i /opt/fmadio/queue/lxc_market2json_rtp_a | /opt/fmadio/app/market2json/bin/market2json --proto ./streaming/rtp --output-chbin-gap --chbin-host 127.0.0.1:9000 --chbin-user /opt/fmadio/app/market2json/etc/ch.credentials

The output looks similar to below

root@fmadio100v2-228U-market2json:~# fmadio2pcap -i /opt/fmadio/queue/lxc_market2json_rtp_a | /opt/fmadio/app/market2json/bin/market2json --proto ./streaming/rtp --output-chbin-gap --chbin-host 127.0.0.1:9000 --chbin-user /opt/fmadio/app/market2json/etc/ch.credentials
fmadio2pcap
FMAD Ring [/opt/fmadio/queue/lxc_market2json_rtp_a]
RING[/opt/fmadio/queue/lxc_market2json_rtp_a             ] Size   : 12595200 12595200 16777216
RING[/opt/fmadio/queue/lxc_market2json_rtp_a             ] Version:      100      100
RING[/opt/fmadio/queue/lxc_market2json_rtp_a             ] Path:/opt/fmadio/queue/lxc_market2json_rtp_a
RING[/opt/fmadio/queue/lxc_market2json_rtp_a             ] Put:103d 3d
RING[/opt/fmadio/queue/lxc_market2json_rtp_a             ] Get:103d 3d
Setup
LOG: [main                ] --proto
LOG: [main                ]    Protocol Name: [./streaming/rtp]
LOG: [main                ] --output-chbin-gap
LOG: [main                ]    CH-Gap output
LOG: [main                ] --chbin-host
LOG: [main                ]     127.0.0.1 : 9000  ratelimit: 0
LOG: [main                ] --chbin-user
LOG: [main                ]     ClickHouse binary-over-TCP user info: username default password <redacted>
LOG: [main                ] FMADIO Market Data Gap Detector

Step 2) Send traffic

Send traffic to the LXC from the host using the following command

 sudo stream_cat -v --ring /opt/fmadio/queue/lxc_market2json_rtp_a  rtp_red_20250304_20250508_1454

Example output looks like

fmadio@fmadio100v2-228U:$ sudo stream_cat -v --ring /opt/fmadio/queue/lxc_market2json_rtp_a  rtp_red_20250304_20250508_1454
Create FMAD Ring: 0 [/opt/fmadio/queue/lxc_market2json_rtp_a]
stream_cat ioqueue: 4
TimeStamp[0] --pcap
TimeStamp[1] --pcap
TimeStamp[2] --pcap
TimeStamp[3] --pcap
TimeStamp[4] --pcap
TimeStamp[5] --pcap
TimeStamp[6] --pcap
TimeStamp[7] --pcap
RING[/opt/fmadio/queue/lxc_market2json_cme             ] 00 : CPU:   0 FilterBPF:[(null)] FilterFrame:[(null)]
StartChunkID: 56156424
StartChunk: 56156424 Offset: 0 Stride: 1
StartChunk: 56156424
RING[/opt/fmadio/queue/lxc_market2json_cme             ] Size   : 12595200 16777216
RING[/opt/fmadio/queue/lxc_market2json_cme             ] Version:      100      100
RING[/opt/fmadio/queue/lxc_market2json_cme             ] Put:103d 3d 0x7f67f23f0000
RING[/opt/fmadio/queue/lxc_market2json_cme             ] Get:103d 3d 0x7f67f23f1000
RING[/opt/fmadio/queue/lxc_market2json_cme             ] thread:0
RING[/opt/fmadio/queue/lxc_market2json_cme             ] worker thread start
{"tstr":"20250508_150700", "timestamp":1746688020,"PktCnt":               0, "PktByte":               0, "ChunkID":56156424,"PCAPTS":"00:00:00.000.000.000","PendingB":28141420544,"Read_bps":0,"Read_pps":0,"Write_bps":0,"Write_pps":0,"FwdPct":0.000,""CPUIdle":0.000,"CPUFetch":0.683, "CPUSend":0.000}
{"tstr":"20250508_150703", "timestamp":1746688023,"PktCnt":            1149, "PktByte":         1572550, "ChunkID":56156437,"PCAPTS":"14:18:22.722.976.038","PendingB":28138012672,"Read_bps":4590299,"Read_pps":419,"Write_bps":9168543,"Write_pps":838,"FwdPct":2.000,""CPUIdle":0.000,"CPUFetch":0.001, "CPUSend":0.000}
{"tstr":"20250508_150704", "timestamp":1746688024,"PktCnt":           88797, "PktByte":       122311216, "ChunkID":56156903,"PCAPTS":"14:18:22.815.765.429","PendingB":28015853568,"Read_bps":965420572,"Read_pps":87604,"Write_bps":1928374373,"Write_pps":175207,"FwdPct":2.000,""CPUIdle":0.000,"CPUFetch":0.035, "CPUSend":0.000}
{"tstr":"20250508_150705", "timestamp":1746688025,"PktCnt":          172178, "PktByte":       237112472, "ChunkID":56157346,"PCAPTS":"14:18:22.905.519.631","PendingB":27899723776,"Read_bps":848948700,"Read_pps":77075,"Write_bps":1695762454,"Write_pps":154149,"FwdPct":2.000,""CPUIdle":0.000,"CPUFetch":0.029, "CPUSend":0.000}
{"tstr":"20250508_150706", "timestamp":1746688026,"PktCnt":          254995, "PktByte":       351130760, "ChunkID":56157786,"PCAPTS":"14:18:22.995.471.077","PendingB":27784380416,"Read_bps":910836112,"Read_pps":82698,"Write_bps":1819278627,"Write_pps":165396,"FwdPct":2.000,""CPUIdle":0.000,"CPUFetch":0.032, "CPUSend":0.000}
.
.
.

Step 3) Check the GUI

The URL Address is the FMADIO Capture system using port 3040, e.g

https://192.168.2.100:3040/

If no connection, please check the iptables forwarding setup

Example output looks like below

If unable to find any data, check the time range is correct / make it very wide to locate the data