Read Me First

Thank you for using the FMADIO Packet Capture systems, the following are some high level concepts about the system.

Operating System

FMADIO Uses a RAMDISK based operating system, this allows us to upgrade and downgrade the system easily without breaking things. Using a more traditional distribution results in stray files everywhere, with the end result of a fragile unverified operating environment.

One of the major differences is the root system / is on a RAMDISK not persistent media. This means most of the usual directories are:

• Volatile e.g. all data is lost on system reboot.

• It is easy to use all available system RAM resulting in a stalled system.

Specifically:

The /home directory is a volatile file system, and deleted on system reboot

The same for /etc directory.

Where are the PCAPs?

The FMADIO system by default does not write PCAP files to disk, instead FMADIO writes all incoming data into a SSD Storage array in a custom proprietary format. The advantage of this is no blocking process from the wire to the disk, results in 100% capture all the time.

Think of the SSD Array as a gigantic non-volatile FIFO.

For applications that read the SSD in real-time, the application processing performance may be lower than the data capture rate. In traditional capture systems this results in packet drops, as application processing speed can not keep up. In the FMADIO Capture System this only results processing lag, with every single packet processes with a little lag instead of dropped. Its a critical architectural difference to other capture systems.

The example dataflow is shown below:

Partitions

Default partitions are:

Boot drive

/mnt/system

This is where the Linux kernel, initfs boot parameters, EFI and FPGA images are saved.

Operating System

/mnt/store0

This is where the core persistent operating system files are stored.

Scratch disk

/mnt/store1/

For system configured with a scratch disk. This is used when working with PCAP’s generated from the captured data.

Remote Filesystem

/mnt/remote0

This is used for mounting remote file systems such as S3fs or NFS on the system.

Persistent Storage

Persistent storage is used for non-volatile (e.g across reboots) configure files, source files, utilities and other small data sized files

Please use the following path:

/mnt/store0/

This is the OS drive and suitable for everything except data storage. Its capacity is between 512GB to 2TB+ depending on the FMADIO SKU. A user directory can be made in this path, such as:

/mnt/store0/git/

Working Data Storage

Depending on the system, a working data partition can be created ( https://docs.fmad.io/docs/scratch-disk ) FMADIO terminology we call this the “Scratch Disk”. It is recommended to create a separate storage partition. When working with large PCAP files it is easy to fill up the entire disk. When using the OS disk for this, the OS will runs out of disk space and enter an unknown state, thus we recommend using a dedicated Scratch Disk for working with captured data.

For systems configured this is in:

/mnt/store1/

To access data in PCAP format you can use the following tools.

stream_cat

This utility takes the data on the SSD Array, converts it to standard PCAP format accessible for ingest using standard tool chains.

Example is piping output to tcpdump as shown below:

sudo stream_cat --follow | tcpdump -r - -nn

For more documentation see https://docs.fmad.io/docs/fmadio-utils-stream-cat

API

The JSON API allows for more  flexible approach. Under the hood, this uses stream_cat.

Example is to pipe a time range with a BPF filter to tcpdump as shown below. This example extracts ICMP traffic only.

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000" 
    -G --data-urlencode "FilterBPF=icmp"  | tcpdump -r - -nn

For more documentation see: https://docs.fmad.io/docs/fmadio-api-v1-pcap-download

Automatic PCAP generation

For a continuous stream of PCAP files to process, this can be achieved using our FMADIO Push PCAP function, as documented see: https://docs.fmad.io/docs/push-pcap-getting-started