Getting Started

FMADIO Microburst provides microburst analysis of captured data in pseudo real-time.

The application outputs 1 statistics print out for every 1 sec of PCAP time.

Command line arguments

fmadio@fmadio200v4-636:$ pcap_microburst  --help
Usage: pcap_microburst [path]

- runs microburst analysis on a PCAP; if no path is given, stdin is used
- writes one JSON object per line to stdout

Version: Jan 26 2025 14:01:04
Contact: support at fmad.io

Options:
  --help, -h                    | print this usage information
  --arista7130, -m              | use packet timestamp from 7130 footer
  --no-flush, -n                | don't flush stdout after every output
  --status, -S                  | print processing status updates
  --timebins, -t <ns> <ns> <ns> | sample packets into 3 time bins of <ns> long slices
                                | note: you can suffix time arguments with 'us' or 'ms'
                                | to multiply by 1,000 or 1,000,000 respectively

fmadio@fmadio200v4-636:$

Terminology

Name	Description
Sample	Summarized statistics for all buckets within the Sampling Time Window
Sampling Time Window	This is the frequency of statistics output
Bucket Interval	The time interval (in nanoseconds) for each Microburst bucket
Bucket	Microburst time interval with data statistics

Microbursts are calculated in the following way

There is a Sampling time window.
There are multiple Buckets in a Sampling Time Window
There is a time Bucket Interval (time interval window)
Each time Bucket has a bps rate
Each sampling time window, takes the maximum Bucket bps rate

A summary of all microburst calculations is printed every 1sec (by default). The image below shows a 1 second Sampling Time Window, with a 100usec microburst bucket window.

There are 1e9 (1 second) / 100e6 (100 usec) = 1000 time Buckets in a 1 second Sampling Time Window.

In the above the Buckets have time boundaries as follows

B0 : Packet time from 0 → 100usec

B1 : Packet time from 100usec → 200usec

B2 : Packet time from 200usec → 300usec

B999 : Packet time from 999900usec → 1000000usec

Bucket Calculation

For each Bucket the following is calculated

In the above example there are 3 packets in the 100usec Bucket

Packet0 : 128B
Packet1 : 1510B
Packet2 : 64B

Total amount of data transferred in the 100usec Bucket is:

128 + 1510 + 64 = 1702B

Data rate for this 100usec Bucket is

8 (Bytes to bits) * 1702 / 100sec
136,160,000 bps
1236.16 Mbps

Sampling Time Window Calculation

Once all the Buckets for a Time Window have been calculated, a Max and Mean calculation is done on each output.

In the above example

Max is 10Gbps (in bucket B1)
Mean is the average of (1.1Gbps, 10Gbps, 0.2Gbps, 5Gbps, … 5Gbps, 7Gbps, 2Gbps, 1Gbps)

The Max is typically used for hard microburst detection / alerting. As it typically indicates short line rate bursts of data.

Mean is typically used for context with respect to the Max. e.g. is the link heavily used, or lightly used (with occasional microbursts)

Multiple Buckets

By default 3 buckets with different time intervals are calculated. This provides context and “how bursty” the link actually is.

The default settings are

Bucket	Bucket Interval	Description
0	100usec	Detection of very short line rate bursting. This is classic “microburst”
1	1msec	Detection of moderate sustained bursting. Detection of queueing
2	100msec	Detection of link saturation

This is visualized in the below diagram

Output Format

The output format is one JSON line print per Sampling Time Window (usually 1 second). Example is shown below

{"IsJSON":true,"PCAPTS":1737383677000000000,"PCAPTSStr":"14:34:37.000.000.000","Wall_dT":"5.286908","Wall_bps":"18454522407","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567374560"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383678000000000,"PCAPTSStr":"14:34:38.000.000.000","Wall_dT":"5.270162","Wall_bps":"18513161863","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567493680"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383679000000000,"PCAPTSStr":"14:34:39.000.000.000","Wall_dT":"5.253906","Wall_bps":"18570440656","Sample_Byte":12195918698,"Sample_Pkt":15516436,"Sample_bps":97567349584,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567349584,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567349584,"Bucket2_bpsMax":97567376160"Bucket2_bpsMean":97567349584}
{"IsJSON":true,"PCAPTS":1737383680000000000,"PCAPTSStr":"14:34:40.000.000.000","Wall_dT":"5.270075","Wall_bps":"18513467484","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567500240"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383681000000000,"PCAPTSStr":"14:34:41.000.000.000","Wall_dT":"5.279707","Wall_bps":"18479692557","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567376160"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383682000000000,"PCAPTSStr":"14:34:42.000.000.000","Wall_dT":"5.246577","Wall_bps":"18596381905","Sample_Byte":12195918698,"Sample_Pkt":15516436,"Sample_bps":97567349584,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567349584,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567349584,"Bucket2_bpsMax":97567376160"Bucket2_bpsMean":97567349584}
{"IsJSON":true,"PCAPTS":1737383683000000000,"PCAPTSStr":"14:34:43.000.000.000","Wall_dT":"5.244264","Wall_bps":"18604586266","Sample_Byte":12195920249,"Sample_Pkt":15516438,"Sample_bps":97567361992,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567361992,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567361992,"Bucket2_bpsMax":97567374560"Bucket2_bpsMean":97567361992}
{"IsJSON":true,"PCAPTS":1737383684000000000,"PCAPTSStr":"14:34:44.000.000.000","Wall_dT":"5.274702","Wall_bps":"18497227360","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567376160"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383685000000000,"PCAPTSStr":"14:34:45.000.000.000","Wall_dT":"5.265835","Wall_bps":"18528371965","Sample_Byte":12195918698,"Sample_Pkt":15516436,"Sample_bps":97567349584,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567360040,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567352312,"Bucket2_bpsMax":97567374720"Bucket2_bpsMean":97567347538}
{"IsJSON":true,"PCAPTS":1737383686000000000,"PCAPTSStr":"14:34:46.000.000.000","Wall_dT":"5.284704","Wall_bps":"18462218916","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97596240000,"Bucket0_bpsMean":97567351697,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567359417,"Bucket2_bpsMax":97567500240"Bucket2_bpsMean":97567362684}
{"IsJSON":true,"PCAPTS":1737383687000000000,"PCAPTSStr":"14:34:47.000.000.000","Wall_dT":"5.283118","Wall_bps":"18467761302","Sample_Byte":12195920269,"Sample_Pkt":15516438,"Sample_bps":97567362152,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567362152,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567362152,"Bucket2_bpsMax":97567500240"Bucket2_bpsMean":97567362152}
{"IsJSON":true,"PCAPTS":1737383688000000000,"PCAPTSStr":"14:34:48.000.000.000","Wall_dT":"5.278272","Wall_bps":"18484714358","Sample_Byte":12195918778,"Sample_Pkt":15516437,"Sample_bps":97567350224,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567350224,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567350224,"Bucket2_bpsMax":97567494560"Bucket2_bpsMean":97567350224}
{"IsJSON":true,"PCAPTS":1737383689000000000,"PCAPTSStr":"14:34:49.000.000.000","Wall_dT":"5.276811","Wall_bps":"18489834393","Sample_Byte":12195920189,"Sample_Pkt":15516437,"Sample_bps":97567361512,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567361512,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567361512,"Bucket2_bpsMax":97567376160"Bucket2_bpsMean":97567361512}
{"IsJSON":true,"PCAPTS":1737383690000000000,"PCAPTSStr":"14:34:50.000.000.000","Wall_dT":"5.255768","Wall_bps":"18563861685","Sample_Byte":12195918775,"Sample_Pkt":15516437,"Sample_bps":97567350200,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567350200,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567350200,"Bucket2_bpsMax":97567374560"Bucket2_bpsMean":97567350200}
{"IsJSON":true,"PCAPTS":1737383691000000000,"PCAPTSStr":"14:34:51.000.000.000","Wall_dT":"5.237110","Wall_bps":"18630000427","Sample_Byte":12195920192,"Sample_Pkt":15516437,"Sample_bps":97567361536,"Bucket0_bpsMax":97590080000,"Bucket0_bpsMean":97567361536,"Bucket1_bpsMax":97577192000,"Bucket1_bpsMean":97567361536,"Bucket2_bpsMax":97567493760"Bucket2_bpsMean":97567361536}

Detailed description of each field is as follows

{
  "IsJSON": true,
  "PCAPTS": 1730903387000000000,
  "PCAPTSStr": "14:29:47.000.000.000",
  "Wall_dT": 0.000009,
  "Wall_bps": 6851555556,
  "Sample_Byte": 7708,
  "Sample_Pkt": 94,
  "Sample_bps": 61664,
  "Bucket0_bpsMax": 13120000,
  "Bucket0_bpsMean": 6702609,
  "Bucket1_bpsMax": 2624000,
  "Bucket1_bpsMean": 1121164,
  "Bucket2_bpsMax": 104960,
  "Bucket2_bpsMean": 61664
}

Field information

Field	Value	Description
IsJSON	true	Sentinel used for filtering out JSON data in the STDOUT output
PCAPTS	1730903387000000000	Nanosecond Epoch time value for the start of this status output
PCAPTSStr	"14:29:47.000.000.000"	Text string version of the timestamp
Wall_dT	0.000009	Number of seconds of CPU processing time to calculate this status line
Wall_bps	6851555556	CPU processing rate in bps
Sample_Byte	7708	Number of Bytes in the Sampling Time Window
Sample_Pkt	94	Number of Packets in the Sampling Time Window
Sample_bps	61664	bps rate in the Sampling Time Window
Bucket0_bpsMax	13120000	Maximum bps for all of Bucket Interval 0 in the Sampling Time Window
Bucket0_bpsMean	6702609	Mean average bps of all Bucket Interval 0 in the Sampling Time Window
Bucket1_bpsMax	2624000	Maximum bps for all of Bucket Interval 1 in the Sampling Time Window
Bucket1_bpsMean	1121164	Mean average bps of all Bucket Interval 1 in the Sampling Time Window
Bucket2_bpsMax	104960	Maximum bps for all of Bucket Interval 2 in the Sampling Time Window
Bucket2_bpsMean	61664	Mean average bps of all Bucket Interval 2 in the Sampling Time Window

Operating

pcap_microburst can be run with regular PCAPs or using FMADIO captured data natively.

Example regular PCAP file

This uses the default options

 zstdcat /mnt/remote1/pcap/market/opra/20241105_election/ny2-fmadio_20241106_0925.pcap.zst  | pcap_microburst  | head

Example output

fmadio@fmadio200v4-636:/mnt/store0/git/pcap_microburst$ zstdcat /mnt/remote1/pcap/market/opra/20241105_election/ny2-fmadio_20241106_0925.pcap.zst  | ./pcap_microburst  | head
Bandwidth time bins: 100000ns, 1000000ns, 100000000ns
{"IsJSON":true,"PCAPTS":1730903127000000000,"PCAPTSStr":"14:25:27.000.000.000","Wall_dT":0.000038,"Wall_bps":103578947,"Sample_Byte":492,"Sample_Pkt":6,"Sample_bps":3936,"Bucket0_bpsMax":6560000,"Bucket0_bpsMean":6560000,"Bucket1_bpsMax":1312000,"Bucket1_bpsMean":984000,"Bucket2_bpsMax":39360,"Bucket2_bpsMean":39360}
{"IsJSON":true,"PCAPTS":1730903128000000000,"PCAPTSStr":"14:25:28.000.000.000","Wall_dT":0.000012,"Wall_bps":5466666667,"Sample_Byte":8200,"Sample_Pkt":100,"Sample_bps":65600,"Bucket0_bpsMax":13120000,"Bucket0_bpsMean":6762887,"Bucket1_bpsMax":2624000,"Bucket1_bpsMean":1058065,"Bucket2_bpsMax":91840,"Bucket2_bpsMean":65600}
{"IsJSON":true,"PCAPTS":1730903129000000000,"PCAPTSStr":"14:25:29.000.000.000","Wall_dT":0.000011,"Wall_bps":5605818182,"Sample_Byte":7708,"Sample_Pkt":94,"Sample_bps":61664,"Bucket0_bpsMax":6560000,"Bucket0_bpsMean":6560000,"Bucket1_bpsMax":2624000,"Bucket1_bpsMean":1045153,"Bucket2_bpsMax":118080,"Bucket2_bpsMean":61664}
{"IsJSON":true,"PCAPTS":1730903130000000000,"PCAPTSStr":"14:25:30.000.000.000","Wall_dT":0.000011,"Wall_bps":6389818182,"Sample_Byte":8786,"Sample_Pkt":105,"Sample_bps":70288,"Bucket0_bpsMax":21920000,"Bucket0_bpsMean":6694095,"Bucket1_bpsMax":2624000,"Bucket1_bpsMean":1133677,"Bucket2_bpsMax":104960,"Bucket2_bpsMean":70288}
{"IsJSON":true,"PCAPTS":1730903131000000000,"PCAPTSStr":"14:25:31.000.000.000","Wall_dT":0.000017,"Wall_bps":7316705882,"Sample_Byte":15548,"Sample_Pkt":192,"Sample_bps":124384,"Bucket0_bpsMax":19200000,"Bucket0_bpsMean":7273918,"Bucket1_bpsMax":5120000,"Bucket1_bpsMean":1323234,"Bucket2_bpsMax":244800,"Bucket2_bpsMean":124384}
{"IsJSON":true,"PCAPTS":1730903132000000000,"PCAPTSStr":"14:25:32.000.000.000","Wall_dT":0.000017,"Wall_bps":6272000000,"Sample_Byte":13328,"Sample_Pkt":164,"Sample_bps":106624,"Bucket0_bpsMax":13120000,"Bucket0_bpsMean":6834872,"Bucket1_bpsMax":2624000,"Bucket1_bpsMean":1211636,"Bucket2_bpsMax":181120,"Bucket2_bpsMean":106624}

Example FMADIO Capture file

The following example prints only the JSON output traffic

 sudo stream_cat opra_election_20250125_2231 | ./pcap_microburst  | grep IsJSON

Example output

madio@fmadio200v4-636:/mnt/store0/git/pcap_microburst$ sudo stream_cat opra_election_20250125_2231 | ./pcap_microburst  | grep IsJSON | jq                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                06:12:22 [4182/9111]
stream_cat ioqueue: 4
calibrating...
0 : 2749996040           2.7500 cycles/nsec offset:0.004 Mhz
Cycles/Sec 2749996040.0000 Std:       0 cycle std(  0.00000000) Target:2.75 Ghz
StartChunkID: 51710785
StartChunk: 51710785 Offset: 0 Stride: 1
StartChunk: 51710785
Bandwidth time bins: 100000ns, 1000000ns, 100000000ns
{
  "IsJSON": true,
  "PCAPTS": 1730903127000000000,
  "PCAPTSStr": "14:25:27.000.000.000",
  "Wall_dT": 0.000024,
  "Wall_bps": 164000000,
  "Sample_Byte": 492,
  "Sample_Pkt": 6,
  "Sample_bps": 3936,
  "Bucket0_bpsMax": 6560000,
  "Bucket0_bpsMean": 6560000,
  "Bucket1_bpsMax": 1312000,
  "Bucket1_bpsMean": 984000,
  "Bucket2_bpsMax": 39360,
  "Bucket2_bpsMean": 39360
}
{
  "IsJSON": true,
  "PCAPTS": 1730903128000000000,
  "PCAPTSStr": "14:25:28.000.000.000",
  "Wall_dT": 0.000013,
  "Wall_bps": 5046153846,
  "Sample_Byte": 8200,
  "Sample_Pkt": 100,
  "Sample_bps": 65600,
  "Bucket0_bpsMax": 13120000,
  "Bucket0_bpsMean": 6762887,
  "Bucket1_bpsMax": 2624000,
  "Bucket1_bpsMean": 1058065,
  "Bucket2_bpsMax": 91840,
  "Bucket2_bpsMean": 65600
}
{
  "IsJSON": true,
  "PCAPTS": 1730903129000000000,
  "PCAPTSStr": "14:25:29.000.000.000",
  "Wall_dT": 0.000010,
  "Wall_bps": 6166400000,
  "Sample_Byte": 7708,
  "Sample_Pkt": 94,
  "Sample_bps": 61664,
  "Bucket0_bpsMax": 6560000,
  "Bucket0_bpsMean": 6560000,
  "Bucket1_bpsMax": 2624000,
  "Bucket1_bpsMean": 1045153,
  "Bucket2_bpsMax": 118080,
  "Bucket2_bpsMean": 61664
}