--- title: "Getting Packets" slug: "fmadio-os-lxc-getting-packet" updated: 2025-10-04T08:46:11Z published: 2025-10-04T08:46:11Z canonical: "docs.fmad.io/fmadio-os-lxc-getting-packet" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.fmad.io/llms.txt > Use this file to discover all available pages before exploring further. # Getting Packets FMADIO LXC system high level architecture is shown below. ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image (1).avif?sv=2026-02-06&spr=https&st=2026-11-13T19%3A10%3A32Z&se=2026-11-13T20%3A10%3A32Z&sr=c&sp=r&sig=zyzpK7JMeCd%2BprqHibefYOfXzCxkgKZeFUTkbjNVyk0%3D) This provides a full lossless filtered version of data directly into the LXC container for processing. Backpressure is provided thru the LXC Ring structure, allowing the application to consume data as fast or slow as possible without dropping anything. --- ## Reference Example For packets to flow from the capture SSDs as shown above, into an LXC application additional configuration is required. In this example we are using a manual push into the LXC via an FMADIO Ring. Note, for automatic pushing of data into the LXC pls see: [https://docs.fmad.io/docs/fmadio-os-lxc-push-lxc](https://docs.fmad.io/docs/fmadio-os-lxc-push-lxc) The following steps are provided. ### Step 1) Create an LXC ring Code is as follows ```plaintext fmadiocli config ring add test1 ``` Example output shown below ```plaintext fmadio@fmadio100v2-228U:~$ fmadiocli config ring add lxc_test1 [Sat Oct  4 08:10:27 2025] CmdLine [config] [Sat Oct  4 08:10:27 2025] CmdLine [ring] [Sat Oct  4 08:10:27 2025] CmdLine [add] [Sat Oct  4 08:10:27 2025] CmdLine [lxc_test1] [Sat Oct  4 08:10:27 2025] Cmd [config ring add lxc_test1 ] [Sat Oct  4 08:10:27 2025] --------------------------------------------------------------------------------------------------------------------------------- [Sat Oct  4 08:10:27 2025] Created Ring named [lxc_test1] RING reset RING file [/opt/fmadio/queue/lxc_test1] RING[/opt/fmadio/queue/lxc_test1                       ] Size missmatch 0 12595200 RING[/opt/fmadio/queue/lxc_test1                       ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_test1                       ] Version:        0      100 RING[/opt/fmadio/queue/lxc_test1                       ] version wrong force reset RING[/opt/fmadio/queue/lxc_test1                       ] Put:0 0 0x7f830e3b7000 RING[/opt/fmadio/queue/lxc_test1                       ] Get:0 0 0x7f830e3b8000 [Sat Oct  4 08:10:28 2025] created ring [/opt/fmadio/queue/lxc_test1] fmadio@fmadio100v2-228U:~$ ``` This creates the FMADIO Ring in the following directory ```plaintext /opt/fmadio/queue/lxc_test1 ``` Example shown below ```plaintext fmadio@fmadio200v4-636:~$ ls -al /opt/fmadio/queue/lxc_test1 -rw-r--r--    1 root     root      12595200 Oct  4 08:26 /opt/fmadio/queue/lxc_test1 fmadio@fmadio200v4-636:~$ ``` ### Step 2) Configure the LXC By default the the LXC configuration does not setup FMADIO Rings inside an LXC. To configure edit the LXC configuration file, in this case we are using the LXC named security-20250912-0819, the configuration file is in ```plaintext /opt/fmadio/lxc//config ``` Example is shown below ```plaintext root@fmadio200v4-636:# cat /opt/fmadio/lxc/security-20250912-0819/config # lxc config generate by security install.lua # Sun Sep 14 13:14:39 2025 lxc.include = /usr/share/lxc/config/common.conf lxc.arch = x86_64 lxc.rootfs.path = dir:/opt/fmadio/lxc/security-20250912-0819/rootfs lxc.uts.name = fmadio200v4-636-security lxc.net.0.type = veth lxc.net.0.link = fmad0 lxc.net.0.flags = up lxc.net.0.ipv4.address = 192.168.255.60/24 lxc.net.0.ipv4.gateway = 192.168.255.2 # Mount Data Directory lxc.mount.entry = /opt/fmadio/lxc/security-20250912-0819/data/  mnt/data/ none bind,create=dir 0 0 # Mount Log Directory lxc.mount.entry = /opt/fmadio/lxc/security-20250912-0819/log/  mnt/log/ none bind,create=dir 0 0 #set lxc ring mount lxc.mount.entry = /opt/fmadio/queue/lxc_suricata   opt/fmadio/queue/lxc_suricata none bind,create=file 0 0 # mount test ring into the lxc lxc.mount.entry = /opt/fmadio/queue/lxc_test1   opt/fmadio/queue/lxc_test1 none bind,create=file 0 0 lxc.prlimit.nofile = 65535 lxc.prlimit.memlock = unlimited root@fmadio200v4-636:/mnt/store2/lxc/lib/lxc/security-20250912-0819# ``` The important lines are shown below, this allows the FMADIO Ring file to be passed though into the LXC. ```plaintext # mount test ring into the lxc lxc.mount.entry = /opt/fmadio/queue/lxc_test1   opt/fmadio/queue/lxc_test1 none bind,create=file 0 0 ``` ### Step 3) Restart LXC and validate The LXC is requires a restart for the new configuration to take effect. Typically this can be done with a stop/start process as shown below. ```plaintext sudo lxc-stop -n sudo lxc-start -n ``` NOTE: if the lxc-start fails, there might be a typeo or incorrect path in the configuration file. This can be debuged by writing the LXC log file to a location on the host system, example shown below. ```plaintext sudo lxc-start -n --logfile lxc_log ``` Then inspecting the `lxc_log` file for any problems Once the LXC has started, confirm the FMADIO Ring is present in the LXC. ```plaintext ls -al /opt/fmadio/queue/lxc_test1 ``` As shown below ```plaintext root@fmadio200v4-636:/opt/fmadio/lxc# lxc-attach -n security-20250912-0819 root@fmadio200v4-636-security:/# ls -al /opt/fmadio/queue/lxc_test1 -rw-r--r-- 1 root root 12595200 Oct  4 08:06 /opt/fmadio/queue/lxc_test1 root@fmadio200v4-636-security:/# ``` ### Step 4) Show the current Ring status Using the command below, check the status of the ring. This should online with no packet counts ```plaintext fmadiocli show ring status ``` Example output shows 0 packets (as expected) ```plaintext fmadio@fmadio100v2-228U:~$ fmadiocli show ring status [Sat Oct  4 08:12:20 2025] CmdLine [show] [Sat Oct  4 08:12:20 2025] CmdLine [ring] [Sat Oct  4 08:12:20 2025] CmdLine [status] [Sat Oct  4 08:12:20 2025] Cmd [show ring status ] [Sat Oct  4 08:12:20 2025] Name                                     : Path                                                         :     Status :          Pkt Put :          Pkt Get : Pkt Queued : Desc [Sat Oct  4 08:12:20 2025] -----------------------------------------+--------------------------------------------------------------+------------+------------------+------------------+------------+------------------------------------ [Sat Oct  4 08:12:20 2025] lxc_test1                                : /opt/fmadio/queue/lxc_test1                                  :     online :                0 :                0 :          0 : [Sat Oct  4 08:12:20 2025] -----------------------------------------+--------------------------------------------------------------+------------+------------------+------------------+------------+------------------------------------ fmadio@fmadio100v2-228U:~$ ``` ### Step 5) Start the LXC consuming data In this example we will just convert the ring data to PCAP data and output to tcpdump, for demonstration purposes. A real application will consume and process this data in some way. Running inside the ring ```plaintext fmadio2pcap -i /opt/fmadio/queue/lxc_test1  | tcpdump -r - -nn ``` Example shown below ```plaintext root@fmadio200v4-636:/opt/fmadio/lxc# lxc-attach -n security-20250912-0819 root@fmadio200v4-636-security:/# fmadio2pcap -i /opt/fmadio/queue/lxc_test1  | tcpdump -r - -nn FMAD Ring [/opt/fmadio/queue/lxc_test1] RING[/opt/fmadio/queue/lxc_test1                       ] Size   : 12595200 12595200 16777216 RING[/opt/fmadio/queue/lxc_test1                       ] Version:      100      100 RING[/opt/fmadio/queue/lxc_test1                       ] Path:/opt/fmadio/queue/lxc_test1RING[/opt/fmadio/queue/lxc_test1                       ] Put:0 0 RING[/opt/fmadio/queue/lxc_test1                       ] Get:0 0 ``` ### Step 6) Send data to the Ring Finally manually send data to the ring using the utility `stream_cat`. Example command is shown below For more options see: [https://docs.fmad.io/docs/fmadio-utils-stream-cat](https://docs.fmad.io/docs/fmadio-utils-stream-cat) ```plaintext sudo stream_cat -v --ring /opt/fmadio/queue/lxc_test1 ``` Example output shown below ```plaintext fmadio@fmadio200v4-636:~$ sudo stream_cat -v opra_trump_20250924_0718 --ring /opt/fmadio/queue/lxc_test1 Create FMAD Ring: 0 [/opt/fmadio/queue/lxc_test1] stream_cat ioqueue: 4 TimeStamp[0] --pcap TimeStamp[1] --pcap TimeStamp[2] --pcap TimeStamp[3] --pcap TimeStamp[4] --pcap TimeStamp[5] --pcap TimeStamp[6] --pcap TimeStamp[7] --pcap RING[/opt/fmadio/queue/lxc_test1                       ] 00 : CPU:   0 FilterBPF:[(null)] FilterFrame:[(null)] StartChunkID: 108797062 StartChunk: 108797062 Offset: 0 Stride: 1 StartChunk: 108797062 RING[/opt/fmadio/queue/lxc_test1                       ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_test1                       ] Version:      100      100 RING[/opt/fmadio/queue/lxc_test1                       ] Put:0 0 0x7f8cf16ac000 RING[/opt/fmadio/queue/lxc_test1                       ] Get:0 0 0x7f8cf16ad000 RING[/opt/fmadio/queue/lxc_test1                       ] thread:0 RING[/opt/fmadio/queue/lxc_test1                       ] worker thread start {"tstr":"20251004_082327", "timestamp":1759566207,"PktCnt":               0, "PktByte":               0, "ChunkID":108797062,"PCAPTS":"00:00:00.000.000.000","PendingB":226502377472,"Read_bps":0,"Read_pps":0,"Write_bps":0,"Write_pps":0,"FwdPct":0.000,"CPUIdle":0.000,"CPUFetch":0.604, "CPUSend":0.000} {"tstr":"20251004_082328", "timestamp":1759566208,"PktCnt":        13003297, "PktByte":      1953324528, "ChunkID":108805307,"PCAPTS":"14:30:01.610.367.686","PendingB":224341000192,"Read_bps":15625407131,"Read_pps":13002308,"Write_bps":15625407131,"Write_pps":13002308,"FwdPct":1.000,"CPUIdle":0.000,"CPUFetch":0.995, "CPUSend":0.000} {"tstr":"20251004_082329", "timestamp":1759566209,"PktCnt":        24546024, "PktByte":      3945222528, "ChunkID":108813610,"PCAPTS":"14:30:02.377.394.383","PendingB":222164418560,"Read_bps":15934572272,"Read_pps":11542284,"Write_bps":15934572272,"Write_pps":11542284,"FwdPct":1.000,"CPUIdle":0.000,"CPUFetch":0.995, "CPUSend":0.000} . . ``` ### Step 7) Confirm the LXC is receiving data In the LXC window per Step 5) the expected output is a stream of tcpdump ```plaintext root@fmadio200v4-636-security:/# fmadio2pcap -i /opt/fmadio/queue/lxc_test1  | tcpdump -r - -nn  | head -n 100 FMAD Ring [/opt/fmadio/queue/lxc_test1] RING[/opt/fmadio/queue/lxc_test1                       ] Size   : 12595200 12595200 16777216 RING[/opt/fmadio/queue/lxc_test1                       ] Version:      100      100 RING[/opt/fmadio/queue/lxc_test1                       ] Path:/opt/fmadio/queue/lxc_test1RING[/opt/fmadio/queue/lxc_test1                       ] Put:23516a6 2a6 RING[/opt/fmadio/queue/lxc_test1                       ] Get:23516a6 2a6 reading from file -, link-type EN10MB (Ethernet), snapshot length 65535 14:15:15.043316 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.043576 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 14:15:15.043883 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.044144 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 14:15:15.047041 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.047301 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 14:15:15.057104 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.057365 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 14:15:15.088217 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.088478 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 14:15:15.093018 IP 162.69.44.4.45004 > 224.0.204.4.45004: UDP, length 36 14:15:15.093279 IP 162.69.52.4.45004 > 224.0.206.4.45004: UDP, length 36 . . . ``` ### Step 7) Finished This is a reference walk thru on how to configure, send and received packets inside of the LXC container. --- ## Sending Data to LXC Offline During development its typically easiest to send data into the LXC container in a one time replay operation. This allows sending the same data repeatedly to the application allow the engineers to debug and test the code. In this example we will use the application "fmadio2pcap" which converts the LXC ring into a standard PCAP formatted stream of data. Its recommended to start the data consumer aka LXC application first, then start the replay. ### Consumer - fmadio2pcap tcpdump Run the following on the LXC container to generate TCPDUMP output thru the lxc ring named "general" ```plaintext /opt/fmadio/platform/fmadio2pcap  -i /opt/fmadio/queue/lxc_ring_general  | tcpdump  -r - -nn ``` It will look similar to the following on startup, it stops because no packets have been sent to the ring. ```plaintext root@fmadio40v3-440-centos7:/opt/fmadio/platform/fmadio2pcap# ./fmadio2pcap  -i /opt/fmadio/queue/lxc_ring_general  | tcpdump  -r - -nn fmadio2pcap FMAD Ring [/opt/fmadio/queue/lxc_ring_general] Ring size   : 12595200 12595200 16777216 Ring Version:      100      100 Ring Depth  :      400      400 Ring Mask   :      3ff      3ff RING: Put:1612f 12f RING: Get:1612f 12f ``` ### Producer - stream_cat sending to LXC ring On the FMADIO Host system find a capture you want to replay. In this example we are using capture named "inetsample_20230615_1513" filename can be found using "stream_dump" The command is as follows ```plaintext sudo stream_cat --ring /opt/fmadio/queue/lxc_ring_general  --ring-filter-bpf /opt/fmadio/queue/lxc_ring_general "icmp" -v inetsample_20230615_1513 ``` Example output of the command shown below ```plaintext fmadio@fmadio40v3-440:/mnt/store0/tmp2$ sudo stream_cat --ring /opt/fmadio/queue/lxc_ring_general  --ring-filter-bpf /opt/fmadio/queue/lxc_ring_general "icmp" -v inetsample_20230615_1513 Create FMAD Ring:0 [/opt/fmadio/queue/lxc_ring_general] found ring: [/opt/fmadio/queue/lxc_ring_general] id:0 FMAD Ring:0 [/opt/fmadio/queue/lxc_ring_general] FilterBPF[icmp] RING[/opt/fmadio/queue/lxc_ring_general                ] 00 : CPU:   0 FilterBPF:[icmp] FilterFrame:[(null)] StartChunkID: 766030 StartChunk: 766030 Offset: 0 Stride: 1 StartChunk: 766030 RING[/opt/fmadio/queue/lxc_ring_general                ] Size   : 12595200 16777216 RING[/opt/fmadio/queue/lxc_ring_general                ] Version:      100      100 RING[/opt/fmadio/queue/lxc_ring_general                ] Put:54a 14a 0x7f30c8cc1000 RING[/opt/fmadio/queue/lxc_ring_general                ] Get:54a 14a 0x7f30c8cc2000 RING[/opt/fmadio/queue/lxc_ring_general                ] thread:0 RING[/opt/fmadio/queue/lxc_ring_general                ] worker thread start 0M Offset:    0GB ChunkID:766030 TS:00:00:00.000.000.000 | Pending  31590 MB 0.000Gbps 0.000Mpps CPUIdle:0.000 CPUFetch:0.595 CPUSend:0.000 2M Offset:    2GB ChunkID:776263 TS:00:00:00.000.000.000 | Pending  29032 MB 21.164Gbps 2.311Mpps CPUIdle:0.000 CPUFetch:0.955 CPUSend:0.000 4M Offset:    4GB ChunkID:786596 TS:00:00:00.000.000.000 | Pending  26449 MB 21.374Gbps 2.300Mpps CPUIdle:0.000 CPUFetch:0.957 CPUSend:0.000 6M Offset:    7GB ChunkID:796886 TS:00:00:00.000.000.000 | Pending  23876 MB 21.283Gbps 2.318Mpps CPUIdle:0.000 CPUFetch:0.956 CPUSend:0.000 . . . ``` And on the LXC client side the output will show ICMP packets printed in tcpdump format as shown below ```plaintext root@fmadio40v3-440-centos7:/opt/fmadio/platform/fmadio2pcap# ./fmadio2pcap  -i /opt/fmadio/queue/lxc_ring_general  | tcpdump  -r - -nn | head -n 1000 fmadio2pcap FMAD Ring [/opt/fmadio/queue/lxc_ring_general] Ring size   : 12595200 12595200 16777216 Ring Version:      100      100 Ring Depth  :      400      400 Ring Mask   :      3ff      3ff RING: Put:29c1 1c1 RING: Get:29c1 1c1 reading from file -, link-type EN10MB (Ethernet) 13:39:53.729815 IP 130.128.35.9 > 210.175.175.21: ICMP echo request, id 4494, seq 1, length 64 13:39:53.731604 IP 210.175.175.21 > 130.128.35.9: ICMP echo reply, id 4494, seq 1, length 64 13:39:53.732577 IP 45.0.191.213 > 8.8.8.8: ICMP echo request, id 13542, seq 25606, length 40 13:39:53.733466 IP 202.17.221.191 > 216.58.200.196: ICMP host 202.17.221.191 unreachable, length 123 13:39:53.733501 IP 202.17.221.191 > 216.58.200.196: ICMP host 202.17.221.191 unreachable, length 123 13:39:53.733641 IP 8.8.8.8 > 45.0.191.213: ICMP echo reply, id 13542, seq 25606, length 40 13:39:53.734071 IP 130.128.255.176 > 103.235.46.39: ICMP echo request, id 65301, seq 2, length 64 13:39:53.744961 IP 130.128.255.81 > 210.175.175.21: ICMP echo request, id 4498, seq 1, length 64 13:39:53.746434 IP 210.175.175.21 > 130.128.255.81: ICMP echo reply, id 4498, seq 1, length 64 13:39:53.762392 IP 12.133.183.36 > 45.0.191.123: ICMP echo reply, id 46807, seq 6777, length 28 13:39:53.770100 IP 45.0.252.242 > 8.8.8.8: ICMP echo request, id 6172, seq 0, length 64 13:39:53.771160 IP 8.8.8.8 > 45.0.252.242: ICMP echo reply, id 6172, seq 0, length 64 13:39:53.775954 IP 130.128.5.7 > 172.21.32.22: ICMP echo request, id 63039, seq 32022, length 40 13:39:53.776217 IP 130.128.5.7 > 172.21.32.21: ICMP echo request, id 63039, seq 32278, length 40 13:39:53.776490 IP 130.128.5.7 > 172.21.32.20: ICMP echo request, id 63039, seq 32534, length 40 13:39:53.776777 IP 130.128.5.7 > 172.21.32.19: ICMP echo request, id 63039, seq 32790, length 40 13:39:53.790153 IP 45.0.191.229 > 8.8.8.8: ICMP echo request, id 34054, seq 39636, length 44 13:39:53.800539 IP 45.0.191.122 > 8.8.8.8: ICMP echo request, id 8265, seq 44218, length 64 13:39:53.801850 IP 8.8.8.8 > 45.0.191.122: ICMP echo reply, id 8265, seq 44218, length 64 . . . ``` --- ## Custom Application For a custom application to directly ingest data, the following reference code is provided Primary header file with all functions and strcutures inline [https://github.com/fmadio/platform/blob/main/include/fmadio_packet.h](https://github.com/fmadio/platform/blob/main/include/fmadio_packet.h) Core example code to retreive packets [https://github.com/fmadio/platform/blob/main/fmadio2pcap/main.c#L160-L185](https://github.com/fmadio/platform/blob/main/fmadio2pcap/main.c#L160-L185) Core loop snippet, this converts from LXC Ring into standard PCAP Packet format ```cpp while (!s_Exit) { u64 TS; PCAPPacket_t* Pkt = (PCAPPacket_t*)PktBuffer; // fetch packet from ring without blocking int ret = FMADPacket_RecvV1(s_RING, false, &TS, &Pkt->LengthWire, &Pkt->LengthCapture, NULL, Pkt + 1); // if it has valid data if (ret > 0) { // convert 64b epoch into sec/subsec for pcap Pkt->Sec = TS / (u64)1e9; Pkt->NSec = TS % (u64)1e9; // write PCAP header and payload fwrite(PktBuffer, 1, sizeof(PCAPPacket_t) + Pkt->LengthCapture, FPCAP); // general stats TotalPkt += 1; TotalByte += ret; } // end of stream if (ret < 0) break; } ```