MD Gap Detection

Prev Next

FMADIO Supports a software market data gap detection binary, which runs on the host system. This can be used with scripting to verify the integrity of captured data as well as generated PCAPs.

Supported Protocols

  • SIAC OPRA Pillar v6

  • CME MDP3 SBE v1

  • NASDAQ TotalView ITCH v5

  • NYSE Integrated Pillar 2.5

  • SET ITCH v5

  • EUREX EOBI T7 V12

  • EURONEXT OPTIQ SBE

  • SIAC CQS CTA

  • SIAC CTS CTA

  • BATS EDGX PITCH

  • TMX QUANTUMFEED XST (alpha)

  • CBOE BZX PITCH

  • CBOE CFE DEPTH PITCH

  • CBOE CFE TOP

  • UTP UQDF BINARY

  • UTP UTDF BINARY

  • IEX DEEP

  • MIAX PEARL TOM MACH

  • MIAX PEARL DOM MACH

  • FMX BIMP

  • MEMX MEMOIR DEPTH

  • ICE IMPACT MDF

  • INTELLIGENTCROSS IQX

Supported Exchanges

  • NYSE (NYSE, Arca, American, National, Texas)

  • NASDAQ (NASDAQ, NTX, PSX)

  • CBOE (BZX, EDGX, CFE Futures)

  • CME

  • MIAX Pearl

  • IEX

  • FMX

  • SIAC (OPRA, CQS, CTS)

  • UTP (UQDF, UTDF)

  • MEMX

  • ICE Futures Europe

  • IntelligentCross (ATS)

  • Eurex

  • Euronext (France, Netherlands, Belgium, Portugal, Ireland, Norway, Italy, ETLX)

  • SET (Thailand)

  • TMX

Operation

By default pcap_mdgap autodetects the exchange fee based on known subnet, port and IP configuration. This can be overridden by by per protocol command line options

Command line reference

fmadio@fmadio200v4-636:/mnt/store0/develop/tools$ ./pcap_mdgap  --help
Processing Args
  --help

FMADIO Market Data Gap detection (https://fmad.io)
(Fri May  1 10:23:59 2026)

pcap_mdgap

NOTE: Input PCAP`s are always read from STDIN

--cpu  <cpu id>                : bind specifically to a CPU
--log-gap                      : log each and every gap (default false)
--streams-file <file>          : load and save streams state to file
--state-seq <N>                : state sequence number for streams-file (required with --streams-file)
--json                         : print the stream statistics as JSON rows to stdout
--table                        : print the stream statistics as a table to stdout
--csv-seqnos <file>            : write all sequence numbers into file
--sort <column>                : sort table view by column: macsrc, dstport, mcgroup, gapcnt, dupcnt, msgcnt, pktcnt, exchmean (default mcgroup)
--filename                     : sets the PCAPFile field in the JSON output to the given filename
--disable-microburst           : disable microburst detection (default enabled)
--list-endpoints               : list all IP:PORT endpoints used for auto detection
--list-protocols               : list all the supported protocols
--timezone-utc                 : use UTC timezone, default is local timezone
--show-undetected              : show streams where the protocol was not detected
--arista7130                   : MetaMako/Arista 7130 footer enabled

Protocols:
(Defaults to automatic detection per stream if no flag is provided)

  --proto-siac-opra-v6                : decode data as SIAC OPRA v6
  --proto-cme-mdp3-sbe-v1             : decode data as CME MDP 3 SBE v1
  --proto-nasdaq-totalview-itch-v5    : decode data as NASDAQ TOTALVIEW ITCH v5
  --proto-set-itch                    : decode data as SET ITCH
  --proto-nyse-integrated-pillar      : decode data as NYSE INTEGRATED PILLAR
  --proto-sbe-optiq-euronext          : decode data as EURONEXT OPTIQ SBE
  --proto-siac-cts-cta                : decode data as SIAC CTS CTA
  --proto-siac-cqs-cta                : decode data as SIAC CQS CTA
  --proto-eurex-eobi-t7               : decode data as EUREX EOBI T7 V14
  --proto-tmx-quantumfeed-xmt         : decode data as TMX QUANTUMFEED XMT
  --proto-iex-deep                    : decode data as IEX DEEP
  --proto-cboe-bzx-pitch              : dcoder data as CBOE BZX PITCH
  --proto-bats-edgx-pitch             : decode data as BATS EDGX PITCH
  --proto-cboe-cfe-top                : decode data as CBOE CFE TOP
  --proto-memx-memoir-depth           : decode data as MEMX MEMOIR DEPTH
  --proto-ice-impact                  : decode data as ICE iMpact MDF
  --proto-rtp                         : decode data as RTP
  --proto-custom <proto-def.txt>      : decode data as it is defined in proto-def.txt

Example Usage

Gap Check FMADIO Capture file

Check for gaps of an FMADIO capture file

FMADIO Packet Capture systems