Provides general purpose scriptable way to download PCAP from the system
/api/v1/pcap/single
Downloads a single capture as a PCAP with optional time, port and BPF filtering
Address
/api/v1/pcap/single
Verb
GET
Notes
Arguments
Parameter | Type | Required | Value | Description |
---|---|---|---|---|
StreamName | String | Yes | Name of the capture | |
FilterBPF | String | BPF Filter to be applied to the stream. | ||
FilterBPFDecap | String | Enables or Disables FilterBPF run after de-encapsulation | ||
true | FilterBPF run after packet de-encapsluation (e.g. on inner payload) | |||
false (default) | FilterBPF run on the raw packet without de-encapsulation | |||
FilterFrame | String | Filter on the Packet Frame | ||
capture.port==<portid> | Selects FMADIO Capture FPGA Capture Port ID | |||
a7130.device==<device id> | Selects a specific Arista 7130 Device ID | |||
a7130.srcport==<port id> | Selects a specific Arista 7130 Ingress Port ID | |||
c3550.srcport==<portid> | Selects a specific Cisco 3550 Ingress Port ID | |||
TSMode | String | Sets the Timestamp of the PCAP | ||
nic (default) | FMADIO Capture FPGA timestamp (Default) | |||
arista7130 | Arista 7130 (Metamako) | |||
arista7150_overwrite | Arista 7150 FCS Overwrite | |||
arista7150_insert | Arista 7150 Insert 32bit | |||
arista7280_eth64 | Arista 7280 Ethernet 64bit header | |||
arista7280_mac48 | Arista 7280 SrcMAC 48bit Overwrite | |||
cisco_erspan3 | Cisco ERPSANv3 | |||
cisco3550 | Cisco 3550 (Exablaze) | |||
Compression | String | Compress the returned stream with gzip. | ||
none (default) | No compression | |||
fast | Fastest compression but not smallest | |||
Example
Example shows extracting TCP data with FilterBPF and outputing via tcpdump. Command is below
curl -u fmadio:xxx "http://192.168.2.75/api/v1/pcap/single?StreamName=TestCapture_20180702_1127&&FilterBPF=tcp" | tcpdump -r - -nn | head
Example output shown below
$ curl -u fmadio:xxx "http://192.168.2.75/api/v1/pcap/single?StreamName=TestCapture_20180702_1127&&FilterBPF=tcp" | tcpdump -r - -nn | head
11:33:08.000000 66:77:88:99:aa:bb > 00:44:44:44:44:44 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
0x0000: 0000 2c03 153a 2d03 153a 2e03 153a 2f03 ..,..:-..:...:/.
0x0010: 153a 3003 153a 3103 153a 3203 153a 3303 .:0..:1..:2..:3.
0x0020: 153a 3403 153a 3503 153a 3603 153a 3703 .:4..:5..:6..:7.
0x0030: 153a a878 4e26 .:.xN&
11:33:08.000000 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
0x0000: 0000 2c03 152a 2d03 152a 2e03 152a 2f03 ..,..*-..*...*/.
0x0010: 152a 3003 152a 3103 152a 3203 152a 3303 .*0..*1..*2..*3.
0x0020: 152a 3403 152a 3503 152a 3603 152a 3703 .*4..*5..*6..*7.
0x0030: 152a 7b57 491d .*{WI.
.
.
.
.
.
Example - StreamName only.
All arguments as defaults
curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/single?
StreamName=stream_test"
Example - StreamName and FilterBPF
With a FilterBPF (without de-encapsulation)
curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/single?
StreamName=stream_test_001"
-G --data-urlencode "FilterBPF=tcp"
Example - StreamName and Compression
With fast gzip compression
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/single?
StreamName=stream_test_001&
Compression=fast"
Example - StreamName, Compression and FilterBPF
Compression and BPF Filter
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/single?
StreamName=stream_test_001&
Compression=fast"
-G --data-urlencode "FilterBPF=tcp"
/api/v1/pcap/timerange
Downloads a BPF and Frame filtered PCAP using start / end time only. The system will merge multiple captures transparently across the Start/End time specified.
Address
/api/v1/pcap/timerange
Verb
GET
Notes
The system will merge multiple captures transparently across the Start/End time specified.
Arguments
Parameter | Type | Required | Value | Description |
---|---|---|---|---|
TSBegin | Integer | Yes | Start time in EPOCH units of TSUnit (default nanoseconds) | |
TSEnd | Integer | Yes | Stop time in EPOCH units of TSUnit (default nanoseconds) | |
TSUint | String | Time Unit used in TSBegin / TSEnd selection | ||
nanos (default) | Epoch is in Nanoseconds (1e9) | |||
msecs | Epoch is in Milliseconds (1e6) | |||
sec | Epoch is in Seconds | |||
YYYYMMDD_HHMMSS | Year Month Day Hour Min Second time format | |||
TSMax | Integer | Maximum nanosecond of packets to download. (this sets an upper bound on the TSMax value) | ||
FilterBPF | String | BPF Filter to be applied to the stream. | ||
FilterBPFDecap | String | Enables or Disables FilterBPF run after de-encapsulation | ||
true | FilterBPF run after packet de-encapsluation (e.g. on inner payload) | |||
false (default) | FilterBPF run on the raw packet without de-encapsulation | |||
FilterFrame | String | Filter on the Packet Frame | ||
capture.port==<portid> | Selects FMADIO Capture FPGA Capture Port ID | |||
a7130.device==<device id> | Selects a specific Arista 7130 Device ID | |||
a7130.srcport==<port id> | Selects a specific Arista 7130 Ingress Port ID | |||
c3550.srcport==<portid> | Selects a specific Cisco 3550 Ingress Port ID | |||
TSMode | String | Sets the Timestamp of the PCAP | ||
nic (default) | FMADIO Capture FPGA timestamp (Default) | |||
arista7130 | Arista 7130 (Metamako) | |||
arista7150_overwrite | Arista 7150 FCS Overwrite | |||
arista7150_insert | Arista 7150 Insert 32bit | |||
arista7280_eth64 | Arista 7280 Ethernet 64bit header | |||
arista7280_mac48 | Arista 7280 SrcMAC 48bit Overwrite | |||
cisco_erspan3 | Cisco ERPSANv3 | |||
cisco3550 | Cisco 3550 (Exablaze) | |||
Compression | String | Compress the returned stream with gzip. | ||
none (default) | No compression | |||
fast | Fastest compression but not smallest | |||
Example
Example shows extracting data from a specific start/end nanosecond epoch time.
curl -u fmadio:xxx "http://192.168.2.75/pcap/timerange?TSBegin=1530498788000000000&TSEnd=1530498789000000000&" | tcpdump -r - -nn | head
Example output shown below
curl -u fmadio:xxx "http://192.168.2.75/pcap/timerange?TSBegin=1530498788000000000&TSEnd=1530498789000000000&" | tcpdump -r - -nn | head
11:33:08.000000 66:77:88:99:aa:bb > 00:44:44:44:44:44 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
0x0000: 0000 2c03 153a 2d03 153a 2e03 153a 2f03 ..,..:-..:...:/.
0x0010: 153a 3003 153a 3103 153a 3203 153a 3303 .:0..:1..:2..:3.
0x0020: 153a 3403 153a 3503 153a 3603 153a 3703 .:4..:5..:6..:7.
0x0030: 153a a878 4e26 .:.xN&
11:33:08.000000 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
0x0000: 0000 2c03 152a 2d03 152a 2e03 152a 2f03 ..,..*-..*...*/.
0x0010: 152a 3003 152a 3103 152a 3203 152a 3303 .*0..*1..*2..*3.
0x0020: 152a 3403 152a 3503 152a 3603 152a 3703 .*4..*5..*6..*7.
0x0030: 152a 7b57 491d .*{WI.
.
.
.
Example - Nanosecond Epoch
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1621772572136996000&
TSEnd=1621774913584264000"
Example - Second Epoch
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSUnit=sec&
TSBegin=1621772572&
TSEnd=1621774913"
Example - Year Month Day Hour Min
Using the TSUnit option can use a more friendly time selection.
By default it uses the TimeZone configured on the system
Fetch PCAP from 3AM to 4AM on 2023 / 10 (October) / 1st
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSUnit=YYYYMMDD_HHMMSS&
TSBegin=20231001_030000&
TSEnd=20231001_040000"
Example - Year Month Day Hour Min with Timezone
Same as above but specifying the timezone.
NOTE: if using the full TSZone = Asia/Singapore for example, CURL will append a ? to the URL. Its recommended to use the City name only to avoid confusing CURL.
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSUnit=YYYYMMDD_HHMMSS&
TSZone=Singapore&
TSBegin=20231001_030000&
TSEnd=20231001_040000"
Example - Year Month Day Hour Min Timezone and BPF Filter with De-encapsulation
Specifying all of the above with a BPF Filter, with BPF De-encapsulation enabled
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSUnit=YYYYMMDD_HHMMSS&
TSZone=Singapore&
TSBegin=20231001_030000&
TSEnd=20231001_040000&
FilterBPFDecap=true&"
-G --data-urlencode "FilterBPF=icmp"
Example - Year Month Dat Hour Min Timezone with BPF bi-direction host filter
Extracts a bi-directional flow of traffic between 2 hosts 1.2.3.4 and 5.6.7.8
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSUnit=YYYYMMDD_HHMMSS&
TSZone=Singapore&
TSBegin=20231001_030000&
TSEnd=20231001_040000&
-G --data-urlencode "FilterBPF=host 1.2.3.4 and host 5.6.7.8"
Example - TSBegin, TSEnd and TSMax
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1621772572136996000&
TSEnd=1621774913584264000&
TSMax=100000
Example - TSBegin, TSEnd and FilterBPF
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1621772572136996000&
TSEnd=1621774913584264000"
-G --data-urlencode "FilterBPF=tcp"
Example - TSBegin, TSEnd and Compression
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1621772572136996000&
TSEnd=1621774913584264000&
Compression=fast"
Example - TSBegin, TSEnd, FilterBPF and Compression
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1621772572136996000&
TSEnd=1621774913584264000&
Compression=fast"
-G --data-urlencode "FilterBPF=tcp"
Example - Frame Filters Capture Port
Filter based on FMADIO Capture port number
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=capture.port==0"
| tcpdump -r - -nn
| head
Multiple capture ports
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=capture.port==0,1,2,3"
| tcpdump -r - -nn
| head
Excluding ports
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=capture.port!=0"
| tcpdump -r - -nn
| head
Example - Frame Filter Arista 7130
Filter for a specific 7130 Device 54932 (any port)
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=a7130.srcdevice==54932"
| tcpdump -r - -nn
| head
Filter for everything except a specific 7130 Device (not device id 54932)
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=a7130.srcdevice!=54932"
| tcpdump -r - -nn
| head
Filter for a specific 7130 Port number 1
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=a7130.srcport==1"
| tcpdump -r - -nn
| head
Filter for multiple 7130 Port numbers 1, 2, 3, 5, 10
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=a7130.srcport==1,2,3,5,10"
| tcpdump -r - -nn
| head
Filter for everything except 7130 Port number 10
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&"
-G --data-urlencode "FilterFrame=a7130.srcport!=10"
| tcpdump -r - -nn
| head
Filter on a specific 7130 Port number and use the 7130 Footer Timestamp as the PCAP timestamp. Overriding the current TimeStamp setting
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&
TSMode=arista7130"
-G --data-urlencode "FilterFrame=a7130.srcport!=10"
| tcpdump -r - -nn
| head
Example - Frame Filter Cisco 3550 (Exablaze)
Filter on a specific ingress port of the Cisco 3550, and use the Footer timestamp as the PCAP timestamp.
curl "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1658744408270221800&
TSEnd=1658744501189259300&TSMode=cisco3550"
-G --data-urlencode "FilterFrame=c3550.srcport==48"
| tcpdump -r - -nn
| head
Example - Encapsulation Debugging
Many times the exact packet encapsulation is unclear, the following uses a wireshark filter expression to extract and show the full encapsulation format of the packet. From this a high speed BPF filter can be used to process the data.
In the below example we are using the Wireshark filter "ip.addr == 192.168.1.1" on a historical capture.
curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1666706401000000000&
TSEnd=1666706401010000000"
| tshark -r - -T fields -e frame.protocols -e ip.src -e ip.dst
-Y "ip.addr == 192.168.1.1"
Alternatively running on the currently running capture via SSH on the fmadio box looks like the following. This example filters on any UDP traffic.
sudo stream_cat
| tshark -r - -T fields -e frame.protocols -e ip.src -e ip.dst -Y "udp"
| head
The output looks like the following
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.200 192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.201 192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 167.172.70.21 192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.200 192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.201 192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 167.172.70.21 192.168.133.10
.
.
The above output shows there is a single VLAN tag in the packet. Making the equivalent BPF filter
vlan and udp
With the final BPF filter using a CURL request
curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/timerange?
TSBegin=1671407102&
TSEnd=1671407752&
TSMode=sec&"
-G --data-urlencode "FilterBPF=vlan and udp"
| tcpdump -r - -n
| head
Output per below
23:47:45.409489 IP 106.10.186.201.123 > 192.168.133.10.123: NTPv4, Server, length 48
23:52:14.407364 IP 167.172.70.21.123 > 192.168.133.10.123: NTPv4, Server, length 48
23:55:42.405072 IP 106.10.186.200.123 > 192.168.133.10.123: NTPv4, Server, length 48