Example - OPRA 96 Maximum throughput

Prev Next

In the previous example  ( https://docs.fmad.io/docs/features-push-pcap-example-opra-96-multicast-groups ) we showed how to use BPF filters to split every single multicast group into its own 1minute PCAP file. While this offers great flexibility and usefulness, it consumes alot of hardware CPU resources and may have lag.

The configuration below simplifies this for maximum throughput / lowest lag of 1min PCAP generation.

This configuration has the following settings:

  • FeedA is split into 1min PCAPs containing multicast groups 0- 63

  • FeedA is split into 1min PCAPs containing multicast groups 64-96

  • FeedB is split into 1min PCAPs containing multicast groups 0 - 63

  • FeedB is split into 1min PCAPs containing multicast groups 64-96

The split is a little bit cumbersome, as the MC groups per PCAP are not evenly split  (0-63 and 64-96).

It is done this way to:

  • Use 4 x stream_cat instances all running in parallel

  • The BPF filter itself is very fast, its uses a network filter

  • In addition the network filter BPF expression gets a fast path execution

Step 1) Configure 4 push_pcap configs

In this version of the code there are 4 seperate push_pcap configuration files, these are listed as

/opt/fmadio/etc/push_pcap_opra_A0.lua
/opt/fmadio/etc/push_pcap_opra_A1.lua
/opt/fmadio/etc/push_pcap_opra_B0.lua
/opt/fmadio/etc/push_pcap_opra_B1.lua

The files are shown below

push_pcap_opra_A0.lua

-- push_pcap_opra_A0.lua
local Config = {}

-- disable de-encapsulation
Config.Decap = false

-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_A0"

-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat  = true

-- start from beginning of capture
Config.FollowStart  = true

-- first quick filter (optional)
Config.FilterFrame  = "capture.port==0"

-- bpf (Multicast groups 0-63)
Config.FilterBPF  = "ip and dst net 224.0.204.0/26"

Config.Target       = {}
table.insert(Config.Target,
{
    Desc        = string.format("pcap-opra-A0"),
    Mode        = "File",
    Path        = string.format("/mnt/store1/pcap/opra_a0/"),
    Split       = "--split-time 60e9 ",
    FileName    = "--filename-epoch-sec-startend",
})

return Config

push_pcap_opra_A1.lua

-- push_pcap_opra_A1.lua
local Config = {}

-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_A1"

-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat  = true

-- start from beginning of capture
Config.FollowStart  = true

-- first quick filter (optional)
Config.FilterFrame  = "capture.port==0"

-- bpf (Multicast groups 64 - 128). yes OPRA only goes up to 96
Config.FilterBPF  = "ip and dst net 224.0.204.64/26"

Config.Target       = {}
table.insert(Config.Target,
{
    Desc        = string.format("pcap-opra-A1"),
    Mode        = "File",
    Path        = string.format("/mnt/store1/pcap/opra_a1/"),
    Split       = "--split-time 60e9 ",
    FileName    = "--filename-epoch-sec-startend",
})

return Config

push_pcap_opra_B0.lua

-- push_pcap_opra_B0.lua
local Config = {}

-- disable de-encapsulation
Config.Decap = false

-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_B0"

-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat  = true

-- start from beginning of capture
Config.FollowStart  = true

-- first quick filter
Config.FilterFrame  = "capture.port==1"

-- bpf (Multicast groups 0-63)
Config.FilterBPF  = "ip and dst net 224.0.206.0/26"

Config.Target       = {}
table.insert(Config.Target,
{
    Desc        = string.format("pcap-opra-B0"),
    Mode        = "File",
    Path        = string.format("/mnt/store1/pcap/opra_b0/"),
    Split       = "--split-time 60e9 ",
    FileName    = "--filename-epoch-sec-startend",
})

return Config

push_pcap_opra_B1.lua

-- push_pcap_opra_B1.lua
local Config = {}

-- required to match the analytics schdule name
Config.Instance = "push_pcap_opra_B1"

-- single cat highthroughput push
-- only a single target is supported
Config.IsSingleCat  = true

-- start from beginning of capture
Config.FollowStart  = true

-- first quick filter
Config.FilterFrame  = "capture.port==1"

-- bpf
Config.FilterBPF  = "ip and dst net 224.0.206.64/26"

-- then a slightly more granular filter
--Config.FilterBPF  = "net 224.0.204.0/24"
--Config.FilterBPF  = "net 224.0.204.0/24"

Config.Target       = {}
table.insert(Config.Target,
{
    Desc        = string.format("pcap-opra-B1"),
    Mode        = "File",
    Path        = string.format("/mnt/store1/pcap/opra_b1/"),
    Split       = "--split-time 60e9 ",
    FileName    = "--filename-epoch-sec-startend",
})

return Config

Step 2) Configure analytics

Analytics requires a different scheduler script to the standard setup. Here we run 4 parallel version of push_pcap as shown below

The analytics scheduler names are

NOTE: note the  .lua suffix

push_pcap_opra_A0
push_pcap_opra_A1
push_pcap_opra_B0
push_pcap_opra_B1

And create the following files in the directory below.  These files are copied into /opt/fmadio/analytics on reboot automatically.

/opt/fmadio/etc/analytics

push_pcap_opra_A0

#!/bin/sh
ulimit -n 10000000

# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_A0.lua

push_pcap_opra_A1

#!/bin/sh
ulimit -n 10000000

# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_A1.lua

push_pcap_opra_B0

#!/bin/sh
ulimit -n 10000000

# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_B0.lua

push_pcap_opra_B1

#!/bin/sh
ulimit -n 10000000

# default configuration file
/opt/fmadio/analytics/push_pcap.lua --config /opt/fmadio/etc/push_pcap_opra_B1.lua

NOTE: when debugging testing, these files can be copied to /opt/fmadio/analytics manually, instead of rebooting each time.

Step 3) Confirm system is running

As analytics schedule automatically starts all pushes, the log files to check are

/opt/fmadio/log/analytics_push_pcap_opra_A0.cur
/opt/fmadio/log/analytics_push_pcap_opra_A1.cur
/opt/fmadio/log/analytics_push_pcap_opra_B0.cur
/opt/fmadio/log/analytics_push_pcap_opra_B1.cur

And the following

/opt/fmadio/log/push_pcap_opra_A0_pcap-opra-A0.cur
/opt/fmadio/log/push_pcap_opra_A1_pcap-opra-A1.cur
/opt/fmadio/log/push_pcap_opra_B0_pcap-opra-B0.cur
/opt/fmadio/log/push_pcap_opra_B1_pcap-opra-B1.cur

Step 4) Finished