Remote S3 Mount

  • Updated on Aug 15, 2025
  • Published on Jul 5, 2025
  • 3 minute(s) read
  • aA
 Prev Next

Writing PCAPs or processed PCAP files to an S3 mount point is a great way to get data off the system. The following demonstrates how to configure this.

The FMADIO system uses s3fs a FUSE based file system pre-installed, that uses libcurl to present an S3 bucket as a regular linux file system. Its a very convinient approach, however performance may be an issue for high data rates.

NOTE: Be careful as operations on this file system do incur AWS charges, e.g. excessive LIST commands may cause an issue

https://github.com/s3fs-fuse/s3fs-fuse

The following steps assume have the information

  • S3 bucket Name

  • S3 Region URL

  • Access Key Name

  • Access Key Secret

S3 Bucket Name

Example shown below, In this example the bucket name is “fmad-pcap-test1”

S3 Region URL

This is the AWS Region the bucket is located in. In the example below it is ap-southeast-1  which is in Singapore

Access Key Name

Depending on the AWS security policy, how to generate an Access Key may differ. Below is a simple example using IAM, we created a user fmad-push-test1 and created an access key per below

The Access key is AK******************

Access Key Secret

The Access Key secret is only shown one time on the AWS GUI. This need to be recorded and written down when the key is created.

Configuration (Manual)

Prior to the 2025Q2 firmware release manual configuration is required as follows

Step 1) Create a password file

All system configuration is stored in the directory

/opt/fmadio/etc

Create a file appropriately named, in this example we are using

password-s3fs-pcaptest

Then write into this file

Access Key Name : Access Key Secret

Example shown below

AK******************:fO**************************************

Then change permissions so only the user can read it 

chmod  0600 passwd-s3fs-pcaptest

Then confirm permissions as follows

fmadio@fmadio200v4-636:/opt/fmadio/etc$ ls -al passwd-s3fs-pcaptest
-rw-------    1 fmadio   staff           62 Jul  5 15:07 passwd-s3fs-pcaptest
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 2) Edit the boot.lua file

The FMADIO system executes the boot.lua on startup once the system has initialized. This is where any custom initialization scripts or setup can be done. This is a LUA file, shell commands are executed using the command os.execute()

Add the following lines into the file

-- mount s3fs storage directory
os.execute([[mkdir /mnt/s3_<mount point>]])
os.execute([[s3fs <mount point> /mnt/s3_<mount point>/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-<mount point>  -o url=https://s3.ap-<S3 access point> -o use_path_request_style  -o allow_other ]])

Example for is shown below

-- mount s3fs storage directory
os.execute([[mkdir /mnt/s3_pushpcap]])
os.execute([[s3fs fmad-pcap-test1 /mnt/s3_pcaptest/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-pcaptest  -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style  -o allow_other ]])

Save the file

Step 3) Run the boot.lua manually

Next run the boot script manually. Usually it requires some modifications to get all the settings correct

sudo fmadiolua /opt/fmadio/etc/boot.lua

Example output

fmadio@fmadio200v4-636:/opt/fmadio/etc$ sudo fmadiolua /opt/fmadio/etc/boot.lua
fmad fmadlua Jun 18 2025 (fmadiolua /opt/fmadio/etc/boot.lua )
done 0.005533Sec 0.000092Min
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 4) Confirm directory is mounted

Check the contents of the s3 mountpoint

ls -al /mnt/s3_fmad-pcap-test1

Example output below

fmadio@fmadio200v4-636:/opt/fmadio/etc$ ls -al /mnt/s3_fmad-pcap-test1/
total 32684070
drwxrwxrwx    1 root     root          4096 Jan  1  1970 ./
drwxr-xr-x   15 root     root           320 Jul  5 16:08 ../
-rw-r--r--    1 root     root     16734237060 Mar 15  2024 interop.pcap
fmadio@fmadio200v4-636:/opt/fmadio/etc$

Step 5) Reboot the system

This confirms on reboot the S3 mount point is automatically mounted

Step 6) Finished

Debugging

When configuring s3fs there maybe problems with the settings on host / cloud or in a number of other places. The s3fs logs get written to syslog, a good command to use while debugging is

tail -F /mnt/store0/log/messages | grep -v subsystem

This will show all messages except telemetry logging information, including any s3fs error messages

Alternatively greping for s3fs in the syslog can also help. Example below shows error message when the API Key information is incorrect

cat /mnt/store0/log/messages | grep s3fs

Example output

fmadio@fmadio200v4-636:~$ cat /mnt/store0/log/messages | grep s3fs
2025.08.15-14:01:33.736250 (+09:00) | fmadio200v4-636 | authpriv.notice | sudo      |   fmadio : TTY=pts/1 ; PWD=/home/fmadio ; USER=root ; COMMAND=/usr/local/bin/s3fs fmad-pcap-test1 /mnt/s3_fmad-pcap-test1/ -o passwd_file=/mnt/store0/etc/passwd-s3fs-fmad-pcap-test1 -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style -o allow_other -o uid=1002 -o gid=50
2025.08.15-14:01:33.738694 (+09:00) | fmadio200v4-636 | user.info       | s3fs      | s3fs version 1.94(unknown) : s3fs -o passwd_file=/mnt/store0/etc/passwd-s3fs-fmad-pcap-test1 -o url=https://s3.ap-southeast-1.amazonaws.com -o use_path_request_style -o allow_other -o uid=1002 -o gid=50 fmad-pcap-test1 /mnt/s3_fmad-pcap-test1/
2025.08.15-14:01:33.741267 (+09:00) | fmadio200v4-636 | user.info       | s3fs      | init v1.94(commit:unknown) with OpenSSL, credential-library(built-in)
2025.08.15-14:01:33.761152 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4487): The bucket region is not 'us-east-1'(default) for specified url(https://s3.ap-southeast-1.amazonaws.com), it is correctly 'ap-southeast-1'. You should specify url(http(s)://s3-ap-southeast-1.amazonaws.com) and endpoint(ap-southeast-1) option.
2025.08.15-14:01:33.761160 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4544): Failed to connect by sigv4, so retry to connect by signature version 2. But you should to review url and endpoint option.
2025.08.15-14:01:33.781342 (+09:00) | fmadio200v4-636 | user.crit       | s3fs      | s3fs.cpp:s3fs_check_service(4562): Failed to check bucket and directory for mount point : Invalid Credentials(host=https://s3.ap-southeast-1.amazonaws.com, message=The request signature we calculated does not match the signature you provided. Check your key and signing method.)
fmadio@fmadio200v4-636:~$

Specifically