download

Prev Next

Provides general purpose scriptable way to download PCAP from the system


/api/v1/pcap/single

Downloads a single capture as a PCAP with optional time, port and BPF filtering

Address

/api/v1/pcap/single

Verb

GET

Notes

Arguments

Parameter

Type

Required

Value

Description

StreamName

String

Yes

Name of the capture

FilterBPF

String

BPF Filter to be applied to the stream.

FilterBPFDecap

String

Enables or Disables FilterBPF run after de-encapsulation

true

FilterBPF run after packet de-encapsluation (e.g. on inner payload)

false (default)

FilterBPF run on the raw packet without de-encapsulation

FilterFrame

String

Filter on the Packet Frame

capture.port==<portid>

Selects FMADIO Capture FPGA Capture Port ID

a7130.device==<device id>

Selects a specific Arista 7130 Device ID

a7130.srcport==<port id>

Selects a specific Arista 7130 Ingress Port ID

c3550.srcport==<portid>

Selects a specific Cisco 3550 Ingress Port ID

TSMode

String

Sets the Timestamp of the PCAP

nic (default)

FMADIO Capture FPGA timestamp (Default)

arista7130

Arista 7130 (Metamako)

arista7150_overwrite

Arista 7150 FCS Overwrite

arista7150_insert

Arista 7150 Insert 32bit

arista7280_eth64

Arista 7280 Ethernet 64bit header

arista7280_mac48

Arista 7280 SrcMAC 48bit Overwrite

cisco_erspan3

Cisco ERPSANv3

cisco3550

Cisco 3550 (Exablaze)

Compression

String

Compress the returned stream with gzip.

none (default)

No compression

fast

Fastest compression but not smallest

Example

Example shows extracting TCP data with FilterBPF and outputing via tcpdump. Command is below

 curl -u fmadio:xxx "http://192.168.2.75/api/v1/pcap/single?StreamName=TestCapture_20180702_1127&&FilterBPF=tcp" | tcpdump  -r - -nn | head

Example output shown below

$ curl -u fmadio:xxx "http://192.168.2.75/api/v1/pcap/single?StreamName=TestCapture_20180702_1127&&FilterBPF=tcp" | tcpdump  -r - -nn | head
11:33:08.000000 66:77:88:99:aa:bb > 00:44:44:44:44:44 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
        0x0000:  0000 2c03 153a 2d03 153a 2e03 153a 2f03  ..,..:-..:...:/.
        0x0010:  153a 3003 153a 3103 153a 3203 153a 3303  .:0..:1..:2..:3.
        0x0020:  153a 3403 153a 3503 153a 3603 153a 3703  .:4..:5..:6..:7.
        0x0030:  153a a878 4e26                           .:.xN&
11:33:08.000000 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
        0x0000:  0000 2c03 152a 2d03 152a 2e03 152a 2f03  ..,..*-..*...*/.
        0x0010:  152a 3003 152a 3103 152a 3203 152a 3303  .*0..*1..*2..*3.
        0x0020:  152a 3403 152a 3503 152a 3603 152a 3703  .*4..*5..*6..*7.
        0x0030:  152a 7b57 491d                           .*{WI.
.
.
.
.
.   

Example - StreamName only.

All arguments as defaults

curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/single?
    StreamName=stream_test"

Example - StreamName and FilterBPF

With a FilterBPF (without de-encapsulation)

curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/single?
    StreamName=stream_test_001" 
    -G --data-urlencode "FilterBPF=tcp"

Example - StreamName and Compression

With fast gzip compression

curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/single?
    StreamName=stream_test_001&
    Compression=fast"

Example - StreamName, Compression and FilterBPF

Compression and BPF Filter

curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/single?
    StreamName=stream_test_001&
    Compression=fast" 
    -G --data-urlencode "FilterBPF=tcp"


/api/v1/pcap/timerange

Downloads a BPF and Frame filtered PCAP using start / end time only. The system will merge multiple captures transparently across the Start/End time specified.

Address

/api/v1/pcap/timerange

Verb

GET

Notes

The system will merge multiple captures transparently across the Start/End time specified.

Arguments

Parameter

Type

Required

Value

Description

TSBegin

Integer

Yes

Start time in EPOCH units of TSUnit (default nanoseconds)

TSEnd

Integer

Yes

Stop time in EPOCH units of TSUnit (default nanoseconds)

TSUint

String

Time Unit used in TSBegin / TSEnd selection

nanos (default)

Epoch is in Nanoseconds (1e9)

msecs

Epoch is in Milliseconds (1e6)

sec

Epoch is in Seconds

YYYYMMDD_HHMMSS

Year Month Day Hour Min Second time format

TSMax

Integer

Maximum nanosecond of packets to download.

(this sets an upper bound on the TSMax value)

FilterBPF

String

BPF Filter to be applied to the stream.

FilterBPFDecap

String

Enables or Disables FilterBPF run after de-encapsulation

true

FilterBPF run after packet de-encapsluation (e.g. on inner payload)

false (default)

FilterBPF run on the raw packet without de-encapsulation

FilterFrame

String

Filter on the Packet Frame

capture.port==<portid>

Selects FMADIO Capture FPGA Capture Port ID

a7130.device==<device id>

Selects a specific Arista 7130 Device ID

a7130.srcport==<port id>

Selects a specific Arista 7130 Ingress Port ID

c3550.srcport==<portid>

Selects a specific Cisco 3550 Ingress Port ID

TSMode

String

Sets the Timestamp of the PCAP

nic (default)

FMADIO Capture FPGA timestamp (Default)

arista7130

Arista 7130 (Metamako)

arista7150_overwrite

Arista 7150 FCS Overwrite

arista7150_insert

Arista 7150 Insert 32bit

arista7280_eth64

Arista 7280 Ethernet 64bit header

arista7280_mac48

Arista 7280 SrcMAC 48bit Overwrite

cisco_erspan3

Cisco ERPSANv3

cisco3550

Cisco 3550 (Exablaze)

Compression

String

Compress the returned stream with gzip.

none (default)

No compression

fast

Fastest compression but not smallest

Example

Example shows extracting data from a specific start/end nanosecond epoch time.

curl -u fmadio:xxx "http://192.168.2.75/pcap/timerange?TSBegin=1530498788000000000&TSEnd=1530498789000000000&" | tcpdump  -r - -nn | head

Example output shown below

curl -u fmadio:xxx "http://192.168.2.75/pcap/timerange?TSBegin=1530498788000000000&TSEnd=1530498789000000000&" | tcpdump  -r - -nn | head
11:33:08.000000 66:77:88:99:aa:bb > 00:44:44:44:44:44 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
        0x0000:  0000 2c03 153a 2d03 153a 2e03 153a 2f03  ..,..:-..:...:/.
        0x0010:  153a 3003 153a 3103 153a 3203 153a 3303  .:0..:1..:2..:3.
        0x0020:  153a 3403 153a 3503 153a 3603 153a 3703  .:4..:5..:6..:7.
        0x0030:  153a a878 4e26                           .:.xN&
11:33:08.000000 66:77:88:99:aa:bb > 00:33:33:33:33:33 Null Information, send seq 22, rcv seq 1, Flags [Poll], length 54
        0x0000:  0000 2c03 152a 2d03 152a 2e03 152a 2f03  ..,..*-..*...*/.
        0x0010:  152a 3003 152a 3103 152a 3203 152a 3303  .*0..*1..*2..*3.
        0x0020:  152a 3403 152a 3503 152a 3603 152a 3703  .*4..*5..*6..*7.
        0x0030:  152a 7b57 491d                           .*{WI.
.
.
.

Example - Nanosecond Epoch

 curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSBegin=1621772572136996000&
  TSEnd=1621774913584264000"

Example - Second Epoch

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSUnit=sec&
  TSBegin=1621772572&
  TSEnd=1621774913"

Example - Year Month Day Hour Min

Using the TSUnit option can use a more friendly time selection.

By default it uses the TimeZone configured on the system

Fetch PCAP from 3AM to 4AM on 2023 / 10 (October) / 1st

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSUnit=YYYYMMDD_HHMMSS&
  TSBegin=20231001_030000&
  TSEnd=20231001_040000"

Example - Year Month Day Hour Min with Timezone

Same as above but specifying the timezone.

NOTE: if using the full TSZone = Asia/Singapore for example, CURL will append a ? to the URL. Its recommended to use the City name only to avoid confusing CURL.

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSUnit=YYYYMMDD_HHMMSS&
  TSZone=Singapore&
  TSBegin=20231001_030000&
  TSEnd=20231001_040000"

Example - Year Month Day Hour Min Timezone and BPF Filter with De-encapsulation

Specifying all of the above with a BPF Filter, with BPF De-encapsulation enabled

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSUnit=YYYYMMDD_HHMMSS&
  TSZone=Singapore&
  TSBegin=20231001_030000&
  TSEnd=20231001_040000&
  FilterBPFDecap=true&"
  -G --data-urlencode "FilterBPF=icmp"

Example - Year Month Dat Hour Min Timezone with BPF bi-direction host filter

Extracts a bi-directional flow of traffic between 2 hosts 1.2.3.4 and 5.6.7.8

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
  TSUnit=YYYYMMDD_HHMMSS&
  TSZone=Singapore&
  TSBegin=20231001_030000&
  TSEnd=20231001_040000&
  -G --data-urlencode "FilterBPF=host 1.2.3.4 and host 5.6.7.8"

Example - TSBegin, TSEnd and TSMax

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000&
    TSMax=100000

Example - TSBegin, TSEnd and FilterBPF

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000" 
    -G --data-urlencode "FilterBPF=tcp"  

Example - TSBegin, TSEnd and Compression

curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000&
    Compression=fast"

Example - TSBegin, TSEnd, FilterBPF and Compression

curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000&
    Compression=fast" 
    -G --data-urlencode "FilterBPF=tcp"

Example - Frame Filters Capture Port

Filter based on FMADIO Capture port number

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=capture.port==0" 
     | tcpdump -r - -nn 
     | head

Multiple capture ports

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=capture.port==0,1,2,3" 
     | tcpdump -r - -nn 
     | head

Excluding ports

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=capture.port!=0" 
     | tcpdump -r - -nn 
     | head

Example - Frame Filter Arista 7130

Filter for a specific 7130 Device 54932 (any port)

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=a7130.srcdevice==54932" 
     | tcpdump -r - -nn 
     | head

Filter for everything except a specific 7130 Device (not device id 54932)

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=a7130.srcdevice!=54932" 
     | tcpdump -r - -nn 
     | head

Filter for a specific 7130 Port number 1

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=a7130.srcport==1" 
     | tcpdump -r - -nn 
     | head

Filter for multiple 7130 Port numbers 1, 2, 3, 5, 10

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=a7130.srcport==1,2,3,5,10" 
     | tcpdump -r - -nn 
     | head

Filter for everything except 7130 Port number 10

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&"
     -G --data-urlencode "FilterFrame=a7130.srcport!=10" 
     | tcpdump -r - -nn 
     | head

Filter on a specific 7130 Port number and use the 7130 Footer Timestamp as the PCAP timestamp. Overriding the current TimeStamp setting

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&
    TSMode=arista7130"
     -G --data-urlencode "FilterFrame=a7130.srcport!=10" 
     | tcpdump -r - -nn 
     | head

Example - Frame Filter Cisco 3550 (Exablaze)

Filter on a specific ingress port of the Cisco 3550, and use the Footer timestamp as the PCAP timestamp.

curl "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1658744408270221800&
    TSEnd=1658744501189259300&TSMode=cisco3550"
     -G --data-urlencode "FilterFrame=c3550.srcport==48" 
     | tcpdump -r - -nn 
     | head

Example - Encapsulation Debugging

Many times the exact packet encapsulation is unclear, the following uses a wireshark filter expression to extract and show the full encapsulation format of the packet. From this a high speed BPF filter can be used to process the data.

In the below example we are using the Wireshark filter "ip.addr == 192.168.1.1" on a historical capture.

curl -u fmadio:xxxx "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1666706401000000000&
    TSEnd=1666706401010000000" 
    |  tshark -r - -T fields  -e frame.protocols -e ip.src -e ip.dst 
    -Y "ip.addr == 192.168.1.1"

Alternatively running on the currently running capture via SSH on the fmadio box looks like the following. This example filters on any UDP traffic.

sudo stream_cat 
    | tshark -r - -T fields  -e frame.protocols -e ip.src -e ip.dst -Y "udp" 
    | head

The output looks like the following

eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.200  192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.201  192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 167.172.70.21   192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.200  192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 106.10.186.201  192.168.133.10
eth:ethertype:vlan:ethertype:ip:udp:ntp 167.172.70.21   192.168.133.10
.
.

The above output shows there is a single VLAN tag in the packet. Making the equivalent BPF filter

vlan and udp

With the final BPF filter using a CURL request

curl -u fmadio:xxxxx "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1671407102&
    TSEnd=1671407752&
    TSMode=sec&" 
    -G --data-urlencode "FilterBPF=vlan and udp" 
    | tcpdump -r - -n 
    | head

Output per below

23:47:45.409489 IP 106.10.186.201.123 > 192.168.133.10.123: NTPv4, Server, length 48
23:52:14.407364 IP 167.172.70.21.123 > 192.168.133.10.123: NTPv4, Server, length 48
23:55:42.405072 IP 106.10.186.200.123 > 192.168.133.10.123: NTPv4, Server, length 48