Remote S3 Mount

Writing PCAPs or processed PCAP files to an S3 mount point is a great way to get data off the system. The following demonstrates how to configure this.

The FMADIO system uses s3fs a FUSE based file system pre-installed, that uses libcurl to present an S3 bucket as a regular linux file system. Its a very convinient approach, however performance may be an issue for high data rates.

NOTE: Be careful as operations on this file system do incur AWS charges, e.g. excessive LIST commands may cause an issue

https://github.com/s3fs-fuse/s3fs-fuse

The following steps assume have the information

• S3 bucket Name

• S3 Region URL

• Access Key Name

• Access Key Secret

S3 Bucket Name

Example shown below, In this example the bucket name is “fmad-pcap-test1”

S3 Region URL

This is the AWS Region the bucket is located in. In the example below it is ap-southeast-1  which is in Singapore

Access Key Name

Depending on the AWS security policy, how to generate an Access Key may differ. Below is a simple example using IAM, we created a user fmad-push-test1 and created an access key per below

The Access key is AK******************

Access Key Secret

The Access Key secret is only shown one time on the AWS GUI. This need to be recorded and written down when the key is created.

Configuration (disk.lua)

Configuration prior to 2025Q4 firmware release (11600) manual configuration is required as follows

Step 1) Create a password file

All system configuration is stored in the directory

/opt/fmadio/etc

Create a file appropriately named, in this example we are using

password-s3fs-fmad-pcap-test1

NOTE: the password file must exactly match the S3 bucket name. in this case fmad-pcap-test1

Then write into this file

Access Key Name : Access Key Secret

Example shown below

AK******************:fO**************************************

Then change permissions so only the user can read it

chmod  0600 passwd-s3fs-fmad-pcap-test1

Then confirm permissions as follows

fmadio@fmadio200v4-636:/opt/fmadio/etc$ ls -al passwd-s3fs-fmad-pcap-test1
-rw-------    1 fmadio   staff           62 Jul  5 15:07 passwd-s3fs-fmad-pcap-test1
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 2) Edit the disk.lua file

The FMADIO system automatically mounts S3 mount points using the s3fs utility ( https://github.com/s3fs-fuse/s3fs-fuse ). It is not the highest performance S3 option, the convivence of a posix file system access to S3 is great.

If no section is named S3FS add it as follows

S3FS =  
{
}
,

Add in the follow line

S3FS =
{
 ["<bucket-name>"]             = { URL = "https://s3.<access point>.amazonaws.com",    Option = "", MountPoint = "<mount point>" },
}
,

The example above, uses the S3 bucket in Step 1). Each parameter is explained below

<bucket-name> = fmad-pcap-test1
<access point> = ap-southeast-1
<mount point> = s3_pcaptest   (it gets mounted on the system as /mnt/s3_pcaptest/

The completed example looks like

S3FS =
{
    ["fmad-pcap-test1"]             = { URL = "https://s3.ap-southeast-1.amazonaws.com",    Option = "", MountPoint = "s3_fmadtest" }
},

This mounts the s3://fmad-pcap-test1 bucket  to the /mnt/s3_fmadtest directory on the system

Step 3) Verify the disk.lua config

Verify the disk.lua config file has no syntax errors

sudo fmadiolua /opt/fmadio/etc/disk.lua

Example output

fmadio@fmadio200v4-636:/opt/fmadio/etc$ sudo fmadiolua /opt/fmadio/etc/disk.lua
fmad fmadlua Jun 18 2025 (fmadiolua /opt/fmadio/etc/disk.lua )
done 0.005533Sec 0.000092Min
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 4) Reboot the system

After reboot has completed, confirm the mount point has the correct files

e.g

ls -al /mnt/s3_fmadtest

Step 6) Finished

System is

Configuration (Legacy)

Prior to the 2025Q2 firmware release manual configuration is required as follows

NOTE: legacy configuration for older firmware

Step 1) Create a password file

All system configuration is stored in the directory

/opt/fmadio/etc

Create a file appropriately named, in this example we are using

password-s3fs-pcaptest

Then write into this file

Access Key Name : Access Key Secret

Example shown below

AK******************:fO**************************************

Then change permissions so only the user can read it

chmod  0600 passwd-s3fs-pcaptest

Then confirm permissions as follows

fmadio@fmadio200v4-636:/opt/fmadio/etc$ ls -al passwd-s3fs-pcaptest
-rw-------    1 fmadio   staff           62 Jul  5 15:07 passwd-s3fs-pcaptest
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 2) Edit the boot.lua file

The FMADIO system executes the boot.lua on startup once the system has initialized. This is where any custom initialization scripts or setup can be done. This is a LUA file, shell commands are executed using the command os.execute()

Add the following lines into the file

-- mount s3fs storage directory
os.execute([[mkdir /mnt/s3_<mount point>]])
os.execute([[s3fs <mount point> /mnt/s3_<mount point>/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-<mount point>  -o url=https://s3.ap-<S3 access point> -o use_path_request_style  -o allow_other ]])

Example for is shown below

-- mount s3fs storage directory
os.execute([[mkdir /mnt/s3_pushpcap]])
os.execute([[s3fs fmad-pcap-test1 /mnt/s3_pcaptest/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-pcaptest  -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style  -o allow_other ]])

Save the file

Step 3) Run the boot.lua manually

Next run the boot script manually. Usually it requires some modifications to get all the settings correct

sudo fmadiolua /opt/fmadio/etc/boot.lua

Example output

fmadio@fmadio200v4-636:/opt/fmadio/etc$ sudo fmadiolua /opt/fmadio/etc/boot.lua
fmad fmadlua Jun 18 2025 (fmadiolua /opt/fmadio/etc/boot.lua )
done 0.005533Sec 0.000092Min
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 4) Confirm directory is mounted

Check the contents of the s3 mountpoint

ls -al /mnt/s3_fmad-pcap-test1

Example output below

fmadio@fmadio200v4-636:/opt/fmadio/etc$ ls -al /mnt/s3_fmad-pcap-test1/
total 32684070
drwxrwxrwx    1 root     root          4096 Jan  1  1970 ./
drwxr-xr-x   15 root     root           320 Jul  5 16:08 ../
-rw-r--r--    1 root     root     16734237060 Mar 15  2024 interop.pcap
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 5) Reboot the system

This confirms on reboot the S3 mount point is automatically mounted

Step 6) Finished

Debugging

When configuring s3fs there maybe problems with the settings on host / cloud or in a number of other places. The s3fs logs get written to syslog, a good command to use while debugging is

tail -F /mnt/store0/log/messages | grep -v subsystem

This will show all messages except telemetry logging information, including any s3fs error messages

Alternatively greping for s3fs in the syslog can also help. Example below shows error message when the API Key information is incorrect

cat /mnt/store0/log/messages | grep s3fs

Example output

fmadio@fmadio200v4-636:~$ cat /mnt/store0/log/messages | grep s3fs
2025.08.15-14:01:33.736250 (+09:00) | fmadio200v4-636 | authpriv.notice | sudo      |   fmadio : TTY=pts/1 ; PWD=/home/fmadio ; USER=root ; COMMAND=/usr/local/bin/s3fs fmad-pcap-test1 /mnt/s3_fmad-pcap-test1/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-fmad-pcap-test1 -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style -o allow_other -o uid=1002 -o gid=50
2025.08.15-14:01:33.738694 (+09:00) | fmadio200v4-636 | user.info       | s3fs      | s3fs version 1.94(unknown) : s3fs -o passwd_file=/mnt/store0/etc/passwd-s3fs-fmad-pcap-test1 -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style -o allow_other -o uid=1002 -o gid=50 fmad-pcap-test1 /mnt/s3_fmad-pcap-test1/
2025.08.15-14:01:33.741267 (+09:00) | fmadio200v4-636 | user.info       | s3fs      | init v1.94(commit:unknown) with OpenSSL, credential-library(built-in)
2025.08.15-14:01:33.761152 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4487): The bucket region is not 'us-east-1'(default) for specified url(https://s3.ap-southeast-1.amazonaws.com), it is correctly 'ap-southeast-1'. You should specify url(http(s)://s3-ap-southeast-1.amazonaws.com) and endpoint(ap-southeast-1) option.
2025.08.15-14:01:33.761160 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4544): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option.
2025.08.15-14:01:33.781342 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4562): Failed to check bucket and directory for mount point : Invalid Credentials(host=https://s3.ap-southeast-1.amazonaws.com, message=The request signature we calculated does not match the signature you provided. Check your key and signing method.)
fmadio@fmadio200v4-636:~$

Specifically

In addition the automount function debug file can be checked

/mnt/store0/log/setup_storage.lua.log

Search for s3fs to find any relevant error messages