In the previous example ( https://docs.fmad.io/docs/features-push-pcap-example-opra-96-multicast-groups ) we showed how to use BPF filters to split every single multicast group into its own 1minute PCAP file. While this offers great flexibility and usefulness, it consumes alot of hardware CPU resources and may have lag.
The configuration below simplifies this for maximum throughput / lowest lag of 1min PCAP generation.
This configuration has the following settings:
FeedA is split into 1min PCAPs containing multicast groups 0- 63
FeedA is split into 1min PCAPs containing multicast groups 64-96
FeedB is split into 1min PCAPs containing multicast groups 0 - 63
FeedB is split into 1min PCAPs containing multicast groups 64-96
The split is a little bit cumbersome, as the MC groups per PCAP are not evenly split (0-63 and 64-96).
It is done this way to:
Use 4 x stream_cat instances all running in parallel
The BPF filter itself is very fast, its uses a network filter
In addition the network filter BPF expression gets a fast path execution
Step 1) Configure 4 push_pcap configs
In this version of the code there are 4 seperate push_pcap configuration files, these are listed as
/opt/fmadio/etc/push_pcap_opra_A0.lua
/opt/fmadio/etc/push_pcap_opra_A1.lua
/opt/fmadio/etc/push_pcap_opra_B0.lua
/opt/fmadio/etc/push_pcap_opra_B1.luaThe files are shown below
push_pcap_opra_A0.lua
-- push_pcap_opra_A0.lua
local Config = {}
-- disable de-encapsulation
Config.Decap = false
-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_A0"
-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat = true
-- start from beginning of capture
Config.FollowStart = true
-- first quick filter (optional)
Config.FilterFrame = "capture.port==0"
-- bpf (Multicast groups 0-63)
Config.FilterBPF = "ip and dst net 224.0.204.0/26"
Config.Target = {}
table.insert(Config.Target,
{
Desc = string.format("pcap-opra-A0"),
Mode = "File",
Path = string.format("/mnt/store1/pcap/opra_a0/"),
Split = "--split-time 60e9 ",
FileName = "--filename-epoch-sec-startend",
})
return Config
push_pcap_opra_A1.lua
-- push_pcap_opra_A1.lua
local Config = {}
-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_A1"
-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat = true
-- start from beginning of capture
Config.FollowStart = true
-- first quick filter (optional)
Config.FilterFrame = "capture.port==0"
-- bpf (Multicast groups 64 - 128). yes OPRA only goes up to 96
Config.FilterBPF = "ip and dst net 224.0.204.64/26"
Config.Target = {}
table.insert(Config.Target,
{
Desc = string.format("pcap-opra-A1"),
Mode = "File",
Path = string.format("/mnt/store1/pcap/opra_a1/"),
Split = "--split-time 60e9 ",
FileName = "--filename-epoch-sec-startend",
})
return Configpush_pcap_opra_B0.lua
-- push_pcap_opra_B0.lua
local Config = {}
-- disable de-encapsulation
Config.Decap = false
-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_B0"
-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat = true
-- start from beginning of capture
Config.FollowStart = true
-- first quick filter
Config.FilterFrame = "capture.port==1"
-- bpf (Multicast groups 0-63)
Config.FilterBPF = "ip and dst net 224.0.206.0/26"
Config.Target = {}
table.insert(Config.Target,
{
Desc = string.format("pcap-opra-B0"),
Mode = "File",
Path = string.format("/mnt/store1/pcap/opra_b0/"),
Split = "--split-time 60e9 ",
FileName = "--filename-epoch-sec-startend",
})
return Configpush_pcap_opra_B1.lua
-- push_pcap_opra_B1.lua
local Config = {}
-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_B1"
-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat = true
-- start from beginning of capture
Config.FollowStart = true
-- first quick filter
Config.FilterFrame = "capture.port==1"
-- bpf
Config.FilterBPF = "ip and dst net 224.0.206.64/26"
-- then a slightly more granular filter
--Config.FilterBPF = "net 224.0.204.0/24"
--Config.FilterBPF = "net 224.0.204.0/24"
Config.Target = {}
table.insert(Config.Target,
{
Desc = string.format("pcap-opra-B1"),
Mode = "File",
Path = string.format("/mnt/store1/pcap/opra_b1/"),
Split = "--split-time 60e9 ",
FileName = "--filename-epoch-sec-startend",
})
return ConfigStep 2) Configure analytics
Analytics requires a different scheduler script to the standard setup. Here we run 4 parallel version of push_pcap as shown below
.png?sv=2022-11-02&spr=https&st=2025-11-26T05%3A10%3A23Z&se=2025-11-26T05%3A23%3A23Z&sr=c&sp=r&sig=93ayA%2FOZWgBBFAMvCOol7qN2mq4HrAimK7PbVcKew%2Bk%3D)
The analytics scheduler names are
NOTE: note the .lua suffix
push_pcap_opra_A0
push_pcap_opra_A1
push_pcap_opra_B0
push_pcap_opra_B1And create the following files in the directory below. These files are copied into /opt/fmadio/analytics on reboot automatically.
/opt/fmadio/etc/analytics.png?sv=2022-11-02&spr=https&st=2025-11-26T05%3A10%3A23Z&se=2025-11-26T05%3A23%3A23Z&sr=c&sp=r&sig=93ayA%2FOZWgBBFAMvCOol7qN2mq4HrAimK7PbVcKew%2Bk%3D)
push_pcap_opra_A0
#!/bin/sh
ulimit -n 10000000
# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_A0.luapush_pcap_opra_A1
#!/bin/sh
ulimit -n 10000000
# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_A1.luapush_pcap_opra_B0
#!/bin/sh
ulimit -n 10000000
# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_B0.luapush_pcap_opra_B1
#!/bin/sh
ulimit -n 10000000
# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_B1.luaNOTE: when debugging testing, these files can be copied to /opt/fmadio/analytics manually, instead of rebooting each time.
Step 3) Confirm system is running
As analytics schedule automatically starts all pushes, the log files to check are
/opt/fmadio/log/analytics_push_pcap_opra_A0.cur
/opt/fmadio/log/analytics_push_pcap_opra_A1.cur
/opt/fmadio/log/analytics_push_pcap_opra_B0.cur
/opt/fmadio/log/analytics_push_pcap_opra_B1.curAnd the following
/opt/fmadio/log/push_pcap_opra_A0_pcap-opra-A0.cur
/opt/fmadio/log/push_pcap_opra_A1_pcap-opra-A1.cur
/opt/fmadio/log/push_pcap_opra_B0_pcap-opra-B0.cur
/opt/fmadio/log/push_pcap_opra_B1_pcap-opra-B1.cur