--- title: "FMADIO Ring" slug: "fmadio-os-internal-fmadio-ring" updated: 2025-01-19T05:20:04Z published: 2025-01-19T05:20:04Z canonical: "docs.fmad.io/fmadio-os-internal-fmadio-ring" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.fmad.io/llms.txt > Use this file to discover all available pages before exploring further. # FMADIO Ring FMADIO Systems have a simple internal ring bus architecture. This Ring bus can be used for sending packet data within the system to containers all without dropping any packets. In the diagram below its a single Shared Memory lockless Ring. - Producer will write packets into the shared memory PUT slot + updated the PUT pointer - Consumer will read packets from the shared memory GET slot + update the GET pointer - Flow control ensures the PUT and GET pointers do not overflow or underflow - Lock free design. There are not mutexes or locks ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(83).png?sv=2026-02-06&spr=https&st=2026-10-29T08%3A37%3A11Z&se=2026-10-29T09%3A37%3A11Z&sr=c&sp=r&sig=j5asrFSpjhDP8DAH%2F%2FjPQPybxvIYTIV%2BcIs%2BR55WlXQ%3D) Its a simple system, by default the FMADIO Ring is only 1024 × 9K jumbo packets deep. This keeps the data hot in the L1, L2, L3 caches while still allow ebbs and flows of the processing. The location of the rings are in ```plaintext /opt/fmadio/queue/lxc_* ``` This allows an easy way to pass the the entire FMADIO Ring to an LXC container. For example the FShark2 container has the following in the lxc config ```plaintext #set lxc ring mount lxc.mount.entry = /opt/fmadio/queue/lxc_fshark2   opt/fmadio/queue/lxc_fshark2 none bind,create=file 0 0 lxc.prlimit.nofile = 65535 lxc.prlimit.memlock = unlimited ``` --- ## Monitoring Ring Status ### fmadiocli FMADIO CLI can be used to monitor the status of the ring using the command ```plaintext fmadiocli "show ring status" ``` Example output below: ```plaintext fmadio@fmadio200v4-636:~$ fmadiocli "show ring status" fmad fmadlua Oct 18 2024 (/opt/fmadio/bin/fmadiolua --nocal /opt/fmadio/bin/fmadiocli show ring status ) Disable cycle calibration [Tue Dec 17 19:09:17 2024] CmdLine [show ring status] [Tue Dec 17 19:09:17 2024] Cmd [show ring status] [Tue Dec 17 19:09:17 2024] Name                                     : Path                                                         :     Status :          Pkt Put :          Pkt Get : Pkt Queued : Desc [Tue Dec 17 19:09:17 2024] -----------------------------------------+--------------------------------------------------------------+------------+------------------+------------------+------------+------------------------------------ [Tue Dec 17 19:09:17 2024] lxc_fshark2                              : /opt/fmadio/queue/lxc_fshark2                                :     online :        3,137,624 :        3,137,624 :          0 : [Tue Dec 17 19:09:17 2024] -----------------------------------------+--------------------------------------------------------------+------------+------------------+------------------+------------+------------------------------------ done 0.036744Sec 0.000612Min fmadio@fmadio200v4-636:~$ ``` --- ## Producer : stream_cat The primary packet producer is the stream_cat utility. This will fetch data from the local storage, filter it and write to the specified ring ### Offline Capture Below is the command to write a PCAP without filtering into the ring ```plaintext sudo stream_cat -v --ring /opt/fmadio/queue/lxc_fshark2 ``` Example output looks like below ```plaintext fmadio@fmadio200v4-636:~$ sudo stream_cat -v --ring /opt/fmadio/queue/lxc_fshark2  wan0_20241217_1319 Create FMAD Ring: 0 [/opt/fmadio/queue/lxc_fshark2] RING[/opt/fmadio/queue/lxc_fshark2                     ] 00 : CPU:   0 FilterBPF:[(null)] FilterFrame:[(null)] StartChunk: 6338017 RING[/opt/fmadio/queue/lxc_fshark2                     ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_fshark2                     ] Version:      100      100 RING[/opt/fmadio/queue/lxc_fshark2                     ] Put:3ff 3ff 0x7f5af8334000 RING[/opt/fmadio/queue/lxc_fshark2                     ] Get:3ff 3ff 0x7f5af8335000 RING[/opt/fmadio/queue/lxc_fshark2                     ] thread:0 RING[/opt/fmadio/queue/lxc_fshark2                     ] worker thread start {"tstr":"20241217_190746", "timestamp":1734433666,"PktCnt":               0, "PktByte":               0,                                                                                "ChunkID":6338017,"PCAPTS":"00:00:00.000.000.000","PendingB":49860575232,                                                                               "Read_bps":0,"Read_pps":0,                                                                                                              "Write_bps":0,"Write_pps":0,                                                                                                            "FwdPct":0.000                 } {"tstr":"20241217_190747", "timestamp":1734433667,"PktCnt":          504489, "PktByte":       365485290,                                                                                "ChunkID":6339449,"PCAPTS":"20:00:03.678.433.179","PendingB":49485185024,                                                                               "Read_bps":2918460195,"Read_pps":503553,                                                                                                                "Write_bps":5829923357,"Write_pps":1007107,                                                                                                            "FwdPct":2.000                                                                                                                                           } {"tstr":"20241217_190748", "timestamp":1734433668,"PktCnt":         1000274, "PktByte":       797697754,                                                                                "ChunkID":6341128,"PCAPTS":"20:00:07.418.689.821","PendingB":49045045248,                                                                               "Read_bps":3456240418,"Read_pps":495576,                                                                                                                "Write_bps":6902900114,"Write_pps":991152,                                                                                                             "FwdPct":2.000                                                                                                                                           } {"tstr":"20241217_190749", "timestamp":1734433669,"PktCnt":         1513100, "PktByte":      1154918776,                                                                                "ChunkID":6342522,"PCAPTS":"20:00:11.297.875.278","PendingB":48679616512,                                                                               "Read_bps":2855335713,"Read_pps":512389,                                                                                                                "Write_bps":5703834082,"Write_pps":1024779,                                                                                                            "FwdPct":2.000                                                                                                                                           } {"tstr":"20241217_190750", "timestamp":1734433670,"PktCnt":         2030128, "PktByte":      1479828590,                                                                                "ChunkID":6343793,"PCAPTS":"20:00:15.832.633.068","PendingB":48346431488,                                                                               "Read_bps":2593419322,"Read_pps":515863,                                                                                                                "Write_bps":5180966837,"Write_pps":1031725,                                                                                                            "FwdPct":2.000                                                                                                                           . . . ``` ### Offline Filtered Capture By adding `--ring-filter-bpf <ring> BPF` to the command line a BPF filter can be applied to the traffic going into the FMADIO Ring, in this example filtering for `vlan 2` ```plaintext sudo stream_cat -v --ring /opt/fmadio/queue/lxc_fshark2 --ring-filter-bpf /opt/fmadio/queue/lxc_fshark2 "vlan 2"   ``` ### Realtime Capture Forwarding the captured data in realtime is achieved using the `--follow` command line option. Example below ```plaintext sudo stream_cat -v --follow --ring /opt/fmadio/queue/lxc_fshark2 ``` ### Realtime BPF Filtered Capture Sending BPF filtered capture data in realtime to the FMADIO Ring, Example below ```plaintext sudo stream_cat -v --follow --ring /opt/fmadio/queue/lxc_fshark2 --ring-filter-bpf /opt/fmadio/queue/lxc_fshark2 "vlan 2" ``` ### Realtime BPF Filtered Port Filtered Capture Finally filtering on the physical capture port and a BPF filter sending data into the FMADIO Ring. This uses the `--ring-filter-frame <ring> <frame filter>` functionality. Example below ```plaintext sudo stream_cat -v --follow --ring /opt/fmadio/queue/lxc_fshark2 --ring-filter-bpf /opt/fmadio/queue/lxc_fshark2 "vlan 2" --ring-filter-bpf /opt/fmadio/queue/lxc_fshark2 "capture.port=1" ``` --- ## Producer : pcap2fmadio In addition to stream_cat on the host system there is a utility `pcap2fmadio` . This does a similar function as stream_cat however it operates using PCAP format on STDIN. ### Unfiltered PCAP For example to send a pcap named `test.pcap` into the `lxc_fshark2` FMADIO Ring, shown below ```plaintext cat test.pcap  | pcap2fmadio   -i /opt/fmadio/queue/lxc_fshark2 ``` Example output: ```plaintext fmadio@fmadio200v4-636:~$ cat test.pcap  | pcap2fmadio   -i /opt/fmadio/queue/lxc_fshark2 Nano PCAP RING[/opt/fmadio/queue/lxc_fshark2                     ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_fshark2                     ] Version:      100      100 RING[/opt/fmadio/queue/lxc_fshark2                     ] Put:428935 135 0x7ffac3f9a000 RING[/opt/fmadio/queue/lxc_fshark2                     ] Get:428536 136 0x7ffac3f9b000 ``` ### Filtered PCAP A more complicated example can use zcat and tcpdump to filter the data before it goes into the ring buffer. In the example below it is - decompressing in gzip format - filtering for non VLAN tagged packets using tcpdump - writing the filtered packets into lxc_fshark2 ring ```plaintext zcat test.pcap.gz  | tcpdump -r - -w - "not vlan" | pcap2fmadio -i /opt/fmadio/queue/lxc_fshark2 ``` Example output: ```plaintext fmadio@fmadio200v4-636:~$ zcat test.pcap.gz  | tcpdump -r - -w - "not vlan" | pcap2fmadio -i /opt/fmadio/queue/lxc_fshark2 reading from file -, link-type EN10MB (Ethernet), snapshot length 16384 Nano PCAP RING[/opt/fmadio/queue/lxc_fshark2                     ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_fshark2                     ] Version:      100      100 RING[/opt/fmadio/queue/lxc_fshark2                     ] Put:488501 101 0x7efdd0c66000 RING[/opt/fmadio/queue/lxc_fshark2                     ] Get:488501 101 0x7efdd0c67000 ``` --- ## Consumer : fmadio2pcap On the otherside the utility `fmadio2pcap` will take the FMADIO Ring and convert that into standard PCAP format. While this is a suboptimal approach almost all network analysis tools have support for PCAPs. This enables a fast way to get up and running with almost any network analysis tool. ### tcpdump output Example piping the output into tcpdump ```plaintext fmadio2pcap -i /opt/fmadio/queue/lxc_fshark2  | tcpdump -r - -nn ``` Example output below: ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(84).png?sv=2026-02-06&spr=https&st=2026-10-29T08%3A37%3A11Z&se=2026-10-29T09%3A37%3A11Z&sr=c&sp=r&sig=j5asrFSpjhDP8DAH%2F%2FjPQPybxvIYTIV%2BcIs%2BR55WlXQ%3D) --- ## Library To enable processing there is a simple API, its part of our public code repo located below: [https://github.com/fmadio/platform/blob/main/include/fmadio_packet.h](https://github.com/fmadio/platform/blob/main/include/fmadio_packet.h) Full BSD license opensource library. Its a simple shared memory ring buffer making it simple for direct integration. Basic full source code example is shown below with the fmadio2pcap reference code. ### fmadio2pcap example The above fmadio2pcap example utility is located [https://github.com/fmadio/platform/tree/main/fmadio2pcap](https://github.com/fmadio/platform/tree/main/fmadio2pcap) The example fmadio2pcap utility shows how simple it is to get packets off the FMADIO Ring. Core code snippet below ```cpp while (!s_Exit) { u64 TS; PCAPPacket_t* Pkt = (PCAPPacket_t*)PktBuffer; u32 PktFlag = 0; // fetch packet from ring without blocking int ret = FMADPacket_RecvV1(s_RING, false, &TS, &Pkt->LengthWire, &Pkt->LengthCapture, NULL, &PktFlag, Pkt + 1); // if it has valid data if (ret > 0) { // count flaged FCS packets if (PktFlag & FMADRING_FLAG_FCSERR) { TotalPktFCS++; } // santize it assert(Pkt->LengthCapture > 0); assert(Pkt->LengthCapture < 16*1024); // convert 64b epoch into sec/subsec for pcap Pkt->Sec = TS / (u64)1e9; Pkt->NSec = TS % (u64)1e9; // write PCAP header and payload fwrite(PktBuffer, 1, sizeof(PCAPPacket_t) + Pkt->LengthCapture, FPCAP); // general stats TotalPkt += 1; TotalByte += ret; } // end of stream if (ret < 0) break; } fflush(stdout); ``` ---