--- title: "Reference" slug: "market-data-gap-reference" updated: 2026-07-30T05:58:02Z published: 2026-07-30T05:58:02Z canonical: "docs.fmad.io/market-data-gap-reference" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.fmad.io/llms.txt > Use this file to discover all available pages before exploring further. # Reference ## Parquet Schema FMADIO MDGap analysis writes data into the Apache Parquet file format. This enables quick and easy searching for specific data using the duckdb command line. ### MDGap Summary This contains summary information of each PCAP file processed, its considered a high level enabling to locate specific PCAP files that are showing problems. | Field | Type | Unit | Description | | --- | --- | --- | --- | | ts_pcap_min | Int64 | Epoch NS | The minimum PCAP timestamp found in the PCAP | | ts_pcap_max | Int64 | Epoch NS | The maximum PCAP timestamp found in the PCAP | | source | String | Filename | Source PCAP file this was generated from | | vlan | Int32 | | VLAN ID (0 - for not present) | | mac_src | String | MAC Address | Source MAC Address | | proto | String | | Exchange Protocol used to decode the payload | | endpoint | String | | Exchange Line the data was decoded on | | mcgroup | String | IPv4 | Multicast address including destination port number | | port | String | Port | Destination UDP Port number | | gap_cnt | Int64 | | Total number of sequence number discontinuities e.g. number of times a gap was detected | | drop_cnt | Int64 | | Total number of missing Sequence Numbers | | dup_cnt | Int64 | | Total number of Duplicate Sequence Numbers | | msg_cnt | Int64 | | Total number of messages | | pkt_cnt | Int64 | | Total number of Packets | | pkt_byte | Int64 | Byte | Total number of bytes in this stream | | latency_exch_min | Int64 | Nanosec | Minimum Exchange Latency 0 - N/A | | latency_exch_max | Int64 | Nanosec | Maximum Exchange Latency 0 - N/A | | latency_exch_mean | Int64 | Nanosec | Mean Exchange Latency 0 - N/A | | latency_pb_min | Int64 | Nanosec | Minimum Packet Broker Latency 0 - N/A | | latency_pb_max | Int64 | Nanosec | Maximum Packet Broker Latency 0 - N/A | | latency_pb_mean | Int64 | Nanosec | Mean Packet Broker Latency 0 - N/A | ### Exchange Latency The `latency_exch_*` fields calculate the time between whats inside the business message, e.g. the exchange inserted a timestamp vs the FMADIO PCAP Capture Timestamp. Effective its ```plaintext FMADIO Capture Timestamp - Exchange Timestamp = Latency Exchange ``` Only available for Market Data protocols that have exchange timestamps ### Packet Broker Latency The `latency_pb_*` Packet broker latency uses the Arista 7130 Footer timestamp format to compare again the FMADIO Captured timestamp. This is a great way to monitor timekeeping drift on the Arista 7130 device, e.g. if the 7130 is being clock disciplined correctly. Effectively ```plaintext FMADIO Capture Timestamp - Arista 7130 Ingress Timestamp = Packet Broker Latency ``` Only available for data with appropriate Arista 7130 timestamp footer data. --- ## CLI Reference The following is a reference for command line options for using the tool directly. ```plaintext fmadio@fmadio200v4-636:/mnt/store1/pcap/20251027$ pcap_mdgap  --help FMADIO Market Data Gap detection (https://fmad.io) pcap_mdgap 1.26.6 (c0f5f92) built Jul  3 2026 02:25:51 NOTE: Input PCAP`s are always read from STDIN --version / -V                 : print pcap_mdgap version --log-gap                      : log each and every gap (default false) --log-reset                    : log each and every reset (default false) --streams-file          : load and save streams state to file --state-seq                : state sequence number for streams-file (required with --streams-file) --json                         : print the stream statistics as JSON rows to stdout --parquet-summary       : write high level summary info into parquet file --parquet-gap           : write all gaps into parquet file (independently of --log-gap) --parquet-seqno         : write all sequence numbers into parquet file --table                        : print the stream statistics as a table to stdout --table-time-unit      : sets the time resolution in the table (--table) views: us, ns (default us) --csv-seqnos            : write all sequence numbers into file --sort                : sort table view by column: macsrc, dstport, mcgroup, gapcnt, dupcnt, msgcnt, pktcnt, exchmean (default mcgroup) --filename                     : sets the PCAPFile field in the JSON output to the given filename --disable-microburst           : disable microburst detection (default enabled) --list-endpoints               : list all IP:PORT endpoints used for auto detection --list-protocols               : list all the supported protocols --timezone-utc                 : use UTC timezone, default is local timezone --show-undetected              : show streams where the protocol was not detected --arista7130                   : MetaMako/Arista 7130 footer enabled Protocols: (Defaults to automatic detection per stream if no flag is provided)  --proto-siac-opra-v6                : decode data as SIAC OPRA v6  --proto-cme-mdp3-sbe-v1             : decode data as CME MDP 3 SBE v1  --proto-nasdaq-totalview-itch-v5    : decode data as NASDAQ TOTALVIEW ITCH v5  --proto-set-itch                    : decode data as SET ITCH  --proto-nyse-integrated-pillar      : decode data as NYSE INTEGRATED PILLAR  --proto-sbe-optiq-euronext          : decode data as EURONEXT OPTIQ SBE  --proto-siac-cts-cta                : decode data as SIAC CTS CTA  --proto-siac-cqs-cta                : decode data as SIAC CQS CTA  --proto-eurex-eobi-t7               : decode data as EUREX EOBI T7 V14  --proto-tmx-quantumfeed-xmt         : decode data as TMX QUANTUMFEED XMT  --proto-iex-deep                    : decode data as IEX DEEP  --proto-cboe-bzx-pitch              : dcoder data as CBOE BZX PITCH  --proto-bats-edgx-pitch             : decode data as BATS EDGX PITCH  --proto-cboe-cfe-top                : decode data as CBOE CFE TOP  --proto-memx-memoir-depth           : decode data as MEMX MEMOIR DEPTH  --proto-ice-impact                  : decode data as ICE iMpact MDF  --proto-hkex-omd                    : decode data as HKEX OMD (OMD-C / OMD-D)  --proto-ose-itch                    : decode data as OSE ITCH  --proto-tse-flex                    : decode data as TSE FLEX Standard/Full  --proto-tse-flex-mbo                : decode data as TSE FLEX MBO  --proto-jnx-itch                    : decode data as JNX ITCH  --proto-miax-pearl-dom-mach         : decode data as MIAX PEARL DOM MACH  --proto-miax-pearl-tom-mach         : decode data as MIAX PEARL TOM MACH  --proto-iqx                         : decode data as IQX  --proto-bmf-sbe                     : decode data as BMF SBE  --proto-rtp                         : decode data as RTP  --proto-custom      : decode data as it is defined in proto-def.txt  --proto-perf-test                   : no decoder (internal usage) ``` ### —V / —version Outputs the current version Example Output ```plaintext fmadio@fmadio200v4-636:~$ pcap_mdgap -V pcap_mdgap 1.26.6 (c0f5f92) built Jul  3 2026 02:25:51 fmadio@fmadio200v4-636:~$ ``` ### —help Prints the current help listing all the command line options