Thank you for using the FMADIO Packet Capture systems, the following are some high level concepts about the system.

Operating System

FMADIO Uses a RAMDISK based operating system, this allows us to upgrade and downgrade the system easily without breaking things. Using a more traditional distribution results in stray files everywhere, with the end result of a fragile unverified operating environment.

One of the major differences is the root system / is on a RAMDISK not persistent media. This means most of the usual directories are:

Volatile e.g. all data is lost on system reboot.
It is easy to use all available system RAM resulting in a stalled system.

Specifically:

The /home directory is a volatile file system, and deleted on system reboot

The same for /etc directory.

Where are the PCAPs?

The FMADIO system by default does not write PCAP files to disk, instead FMADIO writes all incoming data into a SSD Storage array in a custom proprietary format. The advantage of this is no blocking process from the wire to the disk, results in 100% capture all the time.

Think of the SSD Array as a gigantic non-volatile FIFO.

For applications that read the SSD in real-time, the application processing performance may be lower than the data capture rate. In traditional capture systems this results in packet drops, as application processing speed can not keep up. In the FMADIO Capture System this only results processing lag, with every single packet processes with a little lag instead of dropped. Its a critical architectural difference to other capture systems.

Private

Drawing

https://app.diagrams.net/#G1Nr-0iWwZRA4untJd7_E87CfvmZf5EU8L#%7B%22pageId%22%3A%222I1EnZ2jc7x42J_IJAuF%22%7D

The example dataflow is shown below:

*There are no .pcap files on a regular file system on the device, all captured data is in proprietary format on the SSD Array.*

Partitions

Default partitions are:

Boot drive

/mnt/system

This is where the Linux kernel, initfs boot parameters, EFI and FPGA images are saved.

Operating System

/mnt/store0

This is where the core persistent operating system files are stored.

Scratch disk

/mnt/store1/

For system configured with a scratch disk. This is used when working with PCAP’s generated from the captured data.

Remote Filesystem

/mnt/remote0

This is used for mounting remote file systems such as S3fs or NFS on the system.

Persistent Storage

Persistent storage is used for non-volatile (e.g across reboots) configure files, source files, utilities and other small data sized files

Please use the following path:

/mnt/store0/

This is the OS drive and suitable for everything except data storage. Its capacity is between 512GB to 2TB+ depending on the FMADIO SKU. A user directory can be made in this path, such as:

/mnt/store0/git/

Working Data Storage

Depending on the system, a working data partition can be created ( https://docs.fmad.io/docs/scratch-disk ) FMADIO terminology we call this the “Scratch Disk”. It is recommended to create a separate storage partition. When working with large PCAP files it is easy to fill up the entire disk. When using the OS disk for this, the OS will runs out of disk space and enter an unknown state, thus we recommend using a dedicated Scratch Disk for working with captured data.

For systems configured this is in:

/mnt/store1/

Extracting PCAP files

To access data in PCAP format you can use the following tools.

stream_cat

This utility takes the data on the SSD Array, converts it to standard PCAP format accessible for ingest using standard tool chains.

Example is piping output to tcpdump as shown below:

sudo stream_cat --follow | tcpdump -r - -nn

For more documentation see https://docs.fmad.io/docs/fmadio-utils-stream-cat

API

The JSON API allows for more flexible approach. Under the hood, this uses stream_cat.

Example is to pipe a time range with a BPF filter to tcpdump as shown below. This example extracts ICMP traffic only.

curl -u fmadio:xxxx  "http://127.0.0.1/api/v1/pcap/timerange?
    TSBegin=1621772572136996000&
    TSEnd=1621774913584264000" 
    -G --data-urlencode "FilterBPF=icmp"  | tcpdump -r - -nn

For more documentation see: https://docs.fmad.io/docs/fmadio-api-v1-pcap-download

Automatic PCAP generation

For a continuous stream of PCAP files to process, this can be achieved using our FMADIO Push PCAP function, as documented see: https://docs.fmad.io/docs/push-pcap-getting-started

Logfiles

FMADIO OS is a custom distribution based on a RAM disk setup. As such the logfiles are not in their usual place.

syslog

The system logfile / syslog is located in

/mnt/store0/log/messages

SystemD does not run on the host, syslog messages goes to an old fashioned text file per above.

Most logfiles are in this directory both core linux services and the FMADIO processes running on the system.

Documentation Index