Example - OPRA 96 Multicast Groups

As part of our testing and system verification, FMADIO has a full FMADIO-5514-180T system deployed into NY2 receiving the full OPRA feed over 2 ×100G Cross connects from NY4. This system provides us with valuable real-world semi-PROD environment we can experiment and test our solution on.

Version 2

The configuration we have setup is shown below:

In the above example the system is splitting the OPRA A feed into a single 1Min PCAP per each Multicast group. This results in:

• Every 1 minute

• For each Multicast group

A generated  PCAP file on the local scratch disk.

The location of this config is:

/opt/fmadio/etc/push_pcap.lua

The actual config file:

local Config = {}

-- start processing from the start of the capture (instead of current write position)
Config.FollowStart = false

-- do not de-encapsulate the packets
Config.Decap = false

Config.Target = {}

-- for each of the 96 OPRA Multicast groups
for MCGroup=1,96 do

    -- generate a PCAP output
    table.insert(Config.Target,
    {
        Mode      = "File",
        -- unique name for each split
        Desc      = string.format("pcap-all-capture0-opra-%02i", MCGroup),

        -- promethues lablel for each metric
        Promethues= string.format('port="capture0",feed="A",group="%02i"', MCGroup),

        -- write location on the local disk
        Path      = string.format("/mnt/store1/pcap/capture0/opra-%02i-", MCGroup),

        -- copy location (e.g. remote S3 mount point)
        Copy      = string.format("/mnt/remote0/copy/capture0/opra-%02i-", MCGroup),

        -- split every 1min / 60 seconds
        Split     = "--split-time 60e9 ",

        -- filename format is epoch time
        FileName  = "--filename-epoch-sec-startend",

        -- BPF filter to select only the OPRA A feed destination multicast group
        FilterBPF = "ip and dst host 224.0.204."..MCGroup,


        -- optional additional (disabled)
        --PipeCmd     = "zstd -c -T8",
        --FileSuffix  = ".pcap.zst",
        --Post  = "/opt/fmadio/analytics/push_pcap_copy.lua",
    })
end

return Config

The above config general keeps running in pseudo-realtime during the day. During busier periods there are some delays at market open, which get quickly caught up within the first hour.  Capture is not impacted during this period.

IGMP Joins

In addition to capturing the FMADIO systems can issue IGMP joins from the capture interfaces itself. This allows the FMADIO device to subscribe to multicast groups (e.g. market data) directly without requiring a 3rd system or static joins on the switch.

For OPRA the following is the configuration file require. This is located in

/opt/fmadio/etc/igmp.lua

Joining all 96 Multicast groups x2 (one per Feed per capture port) shown below.

local Port0 = {}
for i=1,96 do
        table.insert(Port0, { Group = "224.0.204."..i, Port = 45000 + i, VLAN = nil }) --  "OPRA_001",
end

local Port1 = {}

for i=1,96 do
        table.insert(Port1, { Group = "224.0.206."..i, Port = 45000 + i, VLAN = nil }) --  "OPRA_001",
end

return { Port0, Port1, Interval = 30e9, }