--- title: "Network Topology" slug: "fmadio-os-network-topology" updated: 2025-11-20T10:16:42Z published: 2025-11-20T10:16:42Z canonical: "docs.fmad.io/fmadio-os-network-topology" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.fmad.io/llms.txt > Use this file to discover all available pages before exploring further. # Network Topology ## Introduction FMADIO Devices have a unique network topology. While we use the Linux operating system the combination of FPGA devices, LXCs, Time Synchronization and custom interfaces make the topology less standard. Below is a generic network topology diagram showing the standard configuration. ## FMADIO Network Topology ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(392).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) ## Description ### Linux Physical Network Interface ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(25).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) This indicates a regular linux network interface, typically using the in-kernel network driver. FMADIO network refer to these as “phy” physical interfaces, for example phy0. Typically in linux systems the physical network interface is the logical network interface, for example eth0. FMADIO systems its slightly more abstracted. Generally - phy0, phy1 are the low speed 1G RJ45 interfaces on the system - phy10, phy11 are the SFP/SFP28/QSFP/QSFP28 10G - 100G high speed management interfaces #### Example: - phy0 - phy1 - phy10 - phy11 ### Linux External Bridge Interface ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(26).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) Linux has the concept of a “bridge” network interface. It runs and operates exactly the same as a physical network interface but has not physical network drivers attached. FMADIO systems refer to these External Bridge interfaces as management interface, and named man0, man1, man10, man11. Typically a single physical interface is attached to a logical management interface. For example below is how phy0 and man0 are mapped. ```bash fmadio@fmadio100v2-228U:~$ sudo brctl show man0 bridge name     bridge id               STP enabled     interfaces man0            8000.e0d55e5d2150       no              phy0 fmadio@fmadio100v2-228U:~$ ``` In the above we see man0 as the logical bridge interface, which a single phy0 interface atached. The reason for this structure is described later, primary it allows an LXC Container to use bridged networking with its own IP address on the same physical network interface. #### Example: - man0 - man1 - man10 - man11 ### Linux Internal Bridge Interface ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(27).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) A linux internal bridge is a standard linux bridge but has no physical network ports attached. Its primary purpose is a private interface from the FMADIO Host system to the FMADIO LXC container system, where traffic will never egress from the box. This interfaces are named “fmad0” and have a static IP allowing FMADIO Host and LXC containers to have pre-defined network topology without worry of the external physical network topology. #### Example: - fmad0 ### FPGA Capture Interface ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(28).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) Finally the FPGA Capture interfaces are also shown in the above diagram. These interfaces have no linux networking connectivity and reside outside the usual linux network configuration. **FPGA Capture ports have no regular linux network interfaces** These interfaces are controlled entirely by the FMADIO OS, the usual Linux networking tools do not operate with them. #### Example: - cap0 - cap1 - cap2 - cap3 - cap4 - cap5 - cap6 - cap7 --- ## Interface List ### bmc The bmc interface is not present on the FMADIO Host system, however all systems have a physical BMC network interface which requires IP port, subnet, gateway etc configuration. ### phy0 This is the 1G RJ45 based copper physical network interface. ### phy1 This is the second 1G RJ45 based copper physical network interface. ### man0 This is the default 1G low speed management port. Typically its configured with a static IP address and the primary management interface. Its usually bridged with `phy0` physical interface ### man1 This can be a secondary interface, one typical use case is to run the PTPv2 network on the man1 interface so it does not interfere with the general management network ### phy10 High speed physical interface, typically an Intel 710 or 810 based 10/25/40/100G physical interface. ### phy11 High speed physical interface, typically an Intel 710 or 810 based 10/25/40/100G physical interface. ### man10 Usually a 10G, 40G or 100G high speed management interface. This provides high speed data transfer and access to the system. ### man11 Also a 10G, 40G, 100G high speed management interface. One typical example is man10 provides general core network access, while man11 provides dedicated cloud network access. ### fmad0 This is the internal bridged network, it runs on the subnet 192.168.255.0/24 ### cap0 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. In some scenarios this may be a lane within a QSFP28/MPO12 physical connection. ### cap1 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. In some scenarios this may be a lane within a QSFP28/MPO12 physical connection. ### cap2 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. ### cap3 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. ### cap4 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. ### cap5 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. ### cap6 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. ### cap7 FPGA capture port either a physical SFP/SFP28/QSFP28 interface. When running 8×10G mode the system can have 8 physical capture ports. --- ## LXC Port Forwarding Port forwarding on the host system into the LXC system is a convenient and simple to use way to access resources within an LXC container. This is configured using regular Linux iptables configuration located in ```plaintext /opt/fmadio/etc/iptables.conf ``` An example is forwarding port 8021 on the FMADIO Host system to port 3000 (Grafana) running inside an LXC Container on the system. This is described below. ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(29).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) An example iptables configuration file for this looks like. ```plaintext fmadio@fmadio100v2-228U:~$ cat /opt/fmadio/etc/iptables.conf # Generated by iptables-save v1.6.1 on Mon Feb  5 13:44:51 2024 *mangle :PREROUTING ACCEPT [5134902:12730600695] :INPUT ACCEPT [4690551:12653869027] :FORWARD ACCEPT [200:13048] :OUTPUT ACCEPT [4706895:872657173] :POSTROUTING ACCEPT [4706935:872658917] COMMIT # Completed on Mon Feb  5 13:44:51 2024 # Generated by iptables-save v1.6.1 on Mon Feb  5 13:44:51 2024 *nat :PREROUTING ACCEPT [448824:77026595] :INPUT ACCEPT [4501:295594] :OUTPUT ACCEPT [82233:11014821] :POSTROUTING ACCEPT [81566:10975253] # pcap2json -A PREROUTING -p tcp -m tcp --dport 3021 -j DNAT --to-destination 192.168.255.21:3000 -A POSTROUTING -o man0 -j MASQUERADE -A POSTROUTING -o fmad0 -j MASQUERADE COMMIT # Completed on Tue Nov 28 15:30:44 2023 # Generated by iptables-save v1.6.1 on Tue Nov 28 15:30:44 2023 *filter :INPUT ACCEPT [15028:3079580] :FORWARD ACCEPT [1651:509287] :OUTPUT ACCEPT [14446:4545136] COMMIT # Completed on Tue Nov 28 15:30:44 2023 fmadio@fmadio100v2-228U:~$ ``` All the usual iptables forwarding, nating, ACL options are avaliable on the system. ## LXC Virtual Network ![](https://cdn.document360.io/b9aab0c4-36d0-4616-9673-30b661455571/Images/Documentation/image(30).png?sv=2026-02-06&spr=https&st=2026-11-13T19%3A20%3A29Z&se=2026-11-13T20%3A20%3A29Z&sr=c&sp=r&sig=sBMDeH141JtC0q1jBta6URa47WFnd6Iq%2FjscSbSVMcc%3D) The above diagram shows how a logical bridge network can share the same physical network with the host. It is an convenient way to run multiple IP addresses on the same physical network however using a single physical network interface.