PCAP Flow2 is a command line utility for extracting half and full bi-directional flows, outputting the results into separate PCAP files.
fmadio@fmadio200v4-636:/mnt/store1/pcap_flow2$ pcap_flow2 --help
pcap_flow2 FMADIO build:May 29 2025 15:01:07
[--help]
fmad engineering all rights reserved
http://www.fmad.io
pcap_flow2 is a high performance PCAP flow extrator utility
cat /tmp/test.pcap | pcap_flow2 -o /mnt/store0/pcapflow2/
Command Line Arguments:
-o <output path> : location of the output files
-v : verbose output
--duplex-full : full duplex (bidirectional) output
--tiemout <nanos> : nanosecond timeout scienttific notation
sizeof(FlowRecord_t) = 3681
sizeof(FlowFullDuplex_t) = 64
fmadio@fmadio200v4-636:/mnt/store1/pcap_flow2$The command in invoked as
sudo stream_cat <capture name> | pcap_flow2 -o <output directory>Example shown below extracts full duplex flows into the directory /mnt/store1/pcap_flow2/
sudo stream_cat sg2_wan0_20250608_1810 | pcap_flow2 --duplex-full -o /mnt/store1/pcap_flow2/Example output
fmadio@fmadio200v4-636:/mnt/store1/pcap_flow2$ sudo stream_cat sg2_wan0_20250608_1810 | pcap_flow2 --duplex-full -o /mnt/store1/pcap_flow2/
pcap_flow2 FMADIO build:May 29 2025 15:01:07
[--duplex-full]
Full Duplex output
[-o]
Output PCAPs to [/mnt/store1/pcap_flow2/]
stream_cat ioqueue: 4
StartChunkID: 56603799
StartChunk: 56603799 Offset: 0 Stride: 1
StartChunk: 56603799
PCAP Nano
FlowIndex 256.000 MB
11:10:26.156.498.000 : PCAPTS:00:00:00.000.000.000 Total: 0.00 GB 0.000 Gbps | Mem 0.268 GB flowcnt: 0 -nan Pkts/Flow -nan MB/Flow
11:10:27.156.500.000 : PCAPTS:10:10:26.466.253.766 Total: 0.03 GB 0.276 Gbps | Mem 0.269 GB flowcnt: 273 337.905 Pkts/Flow 0.120 MB/Flow
11:10:28.174.983.000 : PCAPTS:10:10:50.973.094.302 Total: 0.07 GB 0.276 Gbps | Mem 0.270 GB flowcnt: 401 442.372 Pkts/Flow 0.166 MB/Flow
11:10:29.174.987.000 : PCAPTS:10:11:16.149.831.885 Total: 0.10 GB 0.256 Gbps | Mem 0.270 GB flowcnt: 494 523.945 Pkts/Flow 0.196 MB/Flow
11:10:30.186.693.000 : PCAPTS:10:11:41.120.295.651 Total: 0.13 GB 0.264 Gbps | Mem 0.271 GB flowcnt: 583 588.485 Pkts/Flow 0.221 MB/Flow
11:10:31.201.064.000 : PCAPTS:10:12:05.188.933.598 Total: 0.18 GB 0.323 Gbps | Mem 0.271 GB flowcnt: 675 639.788 Pkts/Flow 0.249 MB/Flow
11:10:32.205.942.000 : PCAPTS:10:12:29.194.087.034 Total: 0.21 GB 0.310 Gbps | Mem 0.271 GB flowcnt: 741 700.499 Pkts/Flow 0.277 MB/Flow
11:10:33.205.941.000 : PCAPTS:10:12:53.811.103.305 Total: 0.25 GB 0.273 Gbps | Mem 0.272 GB flowcnt: 837 723.405 Pkts/Flow 0.284 MB/Flow
11:10:34.205.948.000 : PCAPTS:10:13:17.654.353.516 Total: 0.28 GB 0.275 Gbps | Mem 0.272 GB flowcnt: 926 746.985 Pkts/Flow 0.292 MB/Flow
11:10:35.211.118.000 : PCAPTS:10:13:42.359.438.951 Total: 0.32 GB 0.255 Gbps | Mem 0.272 GB flowcnt: 1002 772.274 Pkts/Flow 0.300 MB/Flow
.
.
This results in the directory
/mnt/store1/pcap_flow2/Populated with files such as the following snippet below
.
.
flow_fed0073c_6d646168_8d1e6d7c_8807135b_63065c8a_1749377735417525381.pcap.json
flow_fed190d1_be6c34b2_e63bdd74_53f9fc2d_fc784d54_1749381050471724449.pcap
flow_fed190d1_be6c34b2_e63bdd74_53f9fc2d_fc784d54_1749381050471724449.pcap.json
flow_fed3e881_41aa7de6_fbd297aa_4d0cde4d_e4d5e8eb_1749377572525071507.pcap
flow_fed3e881_41aa7de6_fbd297aa_4d0cde4d_e4d5e8eb_1749377572525071507.pcap.json
flow_fed5a0e2_5ad4a892_21a3bf91_0b484fcd_cdc92d3c_1749378356950970509.pcap
flow_fed5a0e2_5ad4a892_21a3bf91_0b484fcd_cdc92d3c_1749378356950970509.pcap.json
flow_fed956e7_1d1ae787_a8ecb065_d997e140_b95409de_1749380768082049834.pcap
flow_fed956e7_1d1ae787_a8ecb065_d997e140_b95409de_1749380768082049834.pcap.json
flow_fee55252_64ad800f_908c22ed_05b802dd_23e5f432_1749381084141826522.pcap
flow_fee55252_64ad800f_908c22ed_05b802dd_23e5f432_1749381084141826522.pcap.json
flow_fee77f11_c4100395_6f031243_252c0f6b_0360bad4_1749378930337542798.pcap
flow_fee77f11_c4100395_6f031243_252c0f6b_0360bad4_1749378930337542798.pcap.json
flow_fef268ba_8fd9d3f0_f55af619_028c9e01_9f19d111_1749379108201308115.pcap
flow_fef268ba_8fd9d3f0_f55af619_028c9e01_9f19d111_1749379108201308115.pcap.json
flow_ff00f116_fcd30b8f_9797635c_e4a8a959_2e320411_1749380968153808023.pcap
flow_ff00f116_fcd30b8f_9797635c_e4a8a959_2e320411_1749380968153808023.pcap.json
flow_ff027c78_ee233dee_13b8be3f_e78bb80a_f0fcf97f_1749381130258874963.pcap
.
.The file naming convention is
flow_<SHA1 Hash of Flow>_<epoch timestamp of first packet>.pcapFull duplex flows are closed based on a timeout. Having the epoch timestamp of the first packet in the flow, allows the same flow to re-connect at a later time using a unique file name.
When a flow times out and is closed, a corresponding .json file is created with metadata associated with the flow. The creation of the .json file allows downstream processing of the .pcap file only when it has been closed/finished. By using the .json file as the signal the PCAP has been completed.
Example data in the .json file is shown below
fmadio@fmadio200v4-636:/mnt/store1/pcap_flow2$ cat flow_fed3e881_41aa7de6_fbd297aa_4d0cde4d_e4d5e8eb_1749377572525071507.pcap.json | jq
{
"FilePath": "/mnt/store1/pcap_flow2/flow_fed3e881_41aa7de6_fbd297aa_4d0cde4d_e4d5e8eb_1749377572525071507.pcap",
"frameProto": ":ether:ipv4:tcp:",
"hash_half": "91f70957_895ab3b2_75e2cdd5_23931478_23436c81",
"hash_full": "fed3e881_41aa7de6_fbd297aa_4d0cde4d_e4d5e8eb",
"pcap_ts_begin": 1749377572525071507,
"pcap_ts_begin_str": "20250608_181252.525.071.507",
"pcap_ts_end": 1749377573105830461,
"pcap_ts_end_str": "20250608_181253.105.830.461",
"pcap_ts_duration_ns": 580758954,
"total_pkt": 78,
"total_byte": 13786,
"ether_0_type": "0x0800",
"ether_0_dst": "f8:f2:1e:bc:cb:61",
"ether_0_src": "18:c0:4d:b4:0e:6c",
"ipv4_0_proto": 6,
"ipv4_0_proto_desc": "TCP",
"ipv4_0_fragment": 0,
"ipv4_0_dst": "192.168.90.110",
"ipv4_0_src": "192.168.2.225",
"tcp_0_port_dst": 9000,
"tcp_0_port_src": 44678,
"tcp_0_port_desc": ""
}
fmadio@fmadio200v4-636:/mnt/store1/pcap_flow2$