Scratch Disk

  • Updated on Nov 15, 2024
  • Published on Oct 8, 2024
  • 6 minute(s) read
  • aA
  • cK
 Prev Next

Scratch Disks are helpful for keeping temporary or persistent data outside of the FMADIO host capture system. Many times the OS disks are unable to be used due to capacity or location.

Typical disk allocation is below. The system is designed entirely for Capture

Modification of this to allow 3 of the capture drives to allocated as Scratch Disk is shown below

Scratch Disks can be run as a RAID0 or RAIDx group of drives, allows flexible allocation of the total storage capacity of the system between Capture vs Analytics.

Creating Scratch Disk

The steps to create a scratch disk are fairly straight forward. There are a few uses of a scratch, some of the more common examples are:

  • LXC container

  • PCAP local storage

  • Intermediate processing storage

  • Many more…

NOTE: Reallocating a disk from the capture array requires formatting / loss of all captured data.

Step 1) Assign physical disks

Start by editing the disk configuration file

/opt/fmadio/etc/disk.lua

In your preferred editor vi or nano edit the file

sudo vi /opt/fmadio/etc/disk.lua

Then moving serial numbers from the CacheDisk section to the ScratchDisk section.

For example moving a single disk (serial number "22443E9DC54E") from ssd8 -> scr0

Before:

  CacheDisk =
    {
        ["22443E9D204F"]        = "ssd0",
        ["22223AD5BFC3"]        = "ssd1",
        ["22443E9D3AFF"]        = "ssd2",
        ["22443E9DC543"]        = "ssd3",
        ["22443E9D2076"]        = "ssd4",
        ["22443E9D3B41"]        = "ssd5",
        ["22443E9D3B65"]        = "ssd6",
        ["22443E9D20A4"]        = "ssd7",
        ["22443E9DC54E"]        = "ssd8",
    }
    ,
    ParDisk =
    {
        ["22443E9D2087"]        = "par0",
    }
    ,
    RaidDisk =
    {
    }
    ,
    ScratchDisk =
    {
    }
    ,
    OSDisk =
    {
    ["50026B7685513F33"] = "os0",
    }
    ,

After: renaming ssd8 to scr0

    CacheDisk =
    {
        ["22443E9D204F"]        = "ssd0",
        ["22223AD5BFC3"]        = "ssd1",
        ["22443E9D3AFF"]        = "ssd2",
        ["22443E9DC543"]        = "ssd3",
        ["22443E9D2076"]        = "ssd4",
        ["22443E9D3B41"]        = "ssd5",
        ["22443E9D3B65"]        = "ssd6",
        ["22443E9D20A4"]        = "ssd7",
    }
    ,
    ParDisk =
    {
        ["22443E9D2087"]        = "par0",
    }
    ,
    RaidDisk =
    {
    }
    ,
    ScratchDisk =
    {
        ["22443E9DC54E"]        = "scr0",
    }
    ,
    OSDisk =
    {
    ["50026B7685513F33"] = "os0",
    }

Save the file.

This can be repeated for as many disks as you require, please keep the numbering sequential e.g. scr0, scr1, scr2, scr3  etc so the system can map it correctly.

Step 2) Reboot the system

Rebooting is required as the system needs to rename the mount points in /opt/fmadio/disk

Step 3) Create RAID partition

After reboot is completed confirm the allocated scratch disks are visible in:

/opt/fmadio/disk/scr*

e.g. in the example below 4 disks are allocated for scratch space:

fmadio@fmadio200v4-636aaaa:~$ ls -al /opt/fmadio/disk/scr*
lrwxrwxrwx    1 root     root            12 Sep 30 05:42 /opt/fmadio/disk/scr0 -> /dev/nvme1n1
lrwxrwxrwx    1 root     root            12 Sep 30 05:42 /opt/fmadio/disk/scr1 -> /dev/nvme2n1
lrwxrwxrwx    1 root     root            12 Sep 30 05:42 /opt/fmadio/disk/scr2 -> /dev/nvme3n1
lrwxrwxrwx    1 root     root            12 Sep 30 05:42 /opt/fmadio/disk/scr3 -> /dev/nvme4n1
fmadio@fmadio200v4-636aaaa:~$

After confirming disks are named correctly, run the following which will create and format scratch disk automatically:

sudo /opt/fmadio/bin/format_scratch.lua

Example output of a successful run is shown below:

fmadio@fmadio200v4-683:~$ sudo /opt/fmadio/bin/format_scratch.lua
fmad fmadlua Oct  7 2024 (/opt/fmadio/bin/fmadiolua /opt/fmadio/bin/format_scratch.lua )
calibrating...
0 : 2749997360           2.7500 cycles/nsec offset:0.003 Mhz
Cycles/Sec 2749997360.0000 Std:       0 cycle std(  0.00000000) Target:2.75 Ghz
------------------------------------------- umounting------------------------------------------
Cmd [sudo umount /mnt/store1]
umount: /mnt/store1: not mounted.
Cmd [sudo mdadm --stop /dev/md1]
mdadm: error opening /dev/md1: No such file or directory
------------------------------------------- create RAID0 ------------------------------------------
Scratch[1] Mountpoint: scr0
Total Drives: 1 [ /opt/fmadio/disk/scr0]
Cmd [/usr/bin/yes | sudo mdadm --create /dev/md1 --force --level=stripe --raid-devices=1 --chunk=64K  /opt/fmadio/disk/scr0]
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
sleep...0/5
sleep...1/5
sleep...2/5
sleep...3/5
sleep...4/5
    /dev/md1:
            Version : 1.2
      Creation Time : Tue Oct  8 17:47:08 2024
         Raid Level : raid0
         Array Size : 3750607168 (3576.86 GiB 3840.62 GB)
       Raid Devices : 1
      Total Devices : 1
        Persistence : Superblock is persistent

        Update Time : Tue Oct  8 17:47:08 2024
              State : clean
     Active Devices : 1
    Working Devices : 1
     Failed Devices : 0
      Spare Devices : 0

         Chunk Size : 64K

               Name : fmadio200v4-683:1  (local to host fmadio200v4-683)
               UUID : d5dde191:f70d2f80:046d1b8a:bd5c0abb
             Events : 0

        Number   Major   Minor   RaidDevice State
           0     259       12        0      active sync   /dev/nvme9n1
------------------------------------------- create file system ------------------------------------------
Cmd [/usr/bin/yes | sudo mkfs.ext4 /dev/md1 2>&1]
mke2fs 1.46.5 (30-Dec-2021)
Discarding device blocks: done
Creating filesystem with 937651792 4k blocks and 234414080 inodes
Filesystem UUID: 9019c1b3-f026-4425-a29e-49466632005e
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

Cmd [sudo mount /dev/md1 /mnt/store1 2>&1 ]
Cmd [sudo chown fmadio.staff /mnt/store1 2>&1]
------------------------------------------- done ------------------------------------------
    Total ScratchSize  : 3.751TB
    Scratch Mountpoint : /mnt/store1/

    Highly recommend to reboot the system. to confirm scratch disk is automatically mounted on reboot
fmadio@fmadio200v4-683:~$

Step 4) Reboot the system

Its highly recommended to reboot the system. This ensures on reboot the scratch disks are mounted correctly after reboot.

sudo reboot

Step 5) Confirm scratch disk is present

The scratch by default gets mounted to:

/mnt/store1/

There should be a logfile named scratch.log which is the log file from Step 3).

Step 6) Reformat the capture array

As the capture array has now lost a disk. Its unable to capture correctly.

Go to the TOOLS page on the WWW GUI, and click on the Quick Format menu item.

This will take approx. 5 minutes.  It is only formatting the custom FMADIO Filesystem (not every byte of capture data) thus its fairly fast to run.

The system will reboot once, going into “Safe Mode” so the format can be completed without interruption.

Then reboot again and back to regular operation.

Step 7) Finish

The scratch disk can now be used as a regular linux file system