Layer 1 Trace in 40G Mode requires the following command
sudo fnic_test --trace-xlgmiiIt provides a list of options and starts by waiting for a trigger
fmadio@fmadio200v4-636:/mnt/store0/develop/fnic_test$ sudo ./fnic_test --trace-xlgmii
Port : 0 : --port <number>
Trigger Config
RUNT : : --runt
OVER : : --over
FCS Error : : --fcs-error
Sanitize : : --sanitize
Frame Error : : --frame-error
PCS Code Error : : --pcs-err
PCS Code Other : : --pcs-other
PCS Code SOP : : --pcs-sop
PCS Code SOP Error: : --pcs-sop-error (non 8Byte aligned)
PCS Code EOP : : --pcs-eop
FPGA Debug : : --fpga-debug
Triggert Delay : 8 cycles : --delay-cycle <counter>
wait trigger 0
wait trigger 1
wait trigger 2Layer 1 XLGMII 40G Linkup Debug
Debugging 40G link up issues can take time, with little visibility into what the error is. Using the Layer 1 40G XLGMII tracer improves the visibility by extracting small traces of the raw Layer 1 traffic.
This Layer 1 trace logic runs in parallel to the usual packet capture logic and has no impact on the capture.
For link up issues, the following command (change port number if required)
sudo fnic_test --trace-xlgmii --port 0 --fcs-error --sanitize --frame-error --pcs-error --pcs-other --fcs-error --runt --over > logExample output looks like the following
fmadio@fmadio200v4-636:~$ sudo fnic_test --trace-xlgmii --port 0 --fcs-error --sanitize --frame-error --pcs-error --pcs-other --fcs-error --runt --over > log
wait trigger 0
wait trigger 1
wait trigger 2
wait trigger 3
wait trigger 4
wait trigger 5
wait trigger 6
wait trigger 7
wait trigger 8
.
.
.The tracer is now waiting for a trigger event, in this case any PCS invalid code word 0xFE bit errors, invalid Frame errors, Other FCS codes such as SEQ and also trigger if a packet fails the FCS check, its a RUNT(less than 64B) or OVERSIZED(greater than 9K), generally anything that is a non standard link up 40G Ethernet packet.
Once the it gets triggered, the output is shown below
.
.
.
wait trigger 168
wait trigger 169
wait trigger 170
wait trigger 171
wait trigger 172
fmadio@fmadio200v4-636:~$Checking the log file shows
fmadio@fmadio200v4-636:~$ head -n 100 log
Port 0
Trigger FCS Error
Trigger Sanitize
Trigger Frame Error
Trigger PCS Code Error
Trigger PCS Code Other
Trigger FCS Error
Trigger RUNT
Trigger OVER
Port : 0 : --port <number>
Trigger Config
RUNT : Y : --runt
OVER : Y : --over
FCS Error : Y : --fcs-error
Sanitize : Y : --sanitize
Frame Error : Y : --frame-error
PCS Code Error : Y : --pcs-err
PCS Code Other : Y : --pcs-other
PCS Code SOP : : --pcs-sop
PCS Code SOP Error: : --pcs-sop-error (non 8Byte aligned)
PCS Code EOP : : --pcs-eop
FPGA Debug : : --fpga-debug
Triggert Delay : 8 cycles : --delay-cycle <counter>
WritePos: 00000081
WritePos: 00000081
[512:082:008] Rep:16777215 : XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
.
.
. (16777151 times)
.
.
at the end of the file the trigger condition can be seen with a full Layer 1 XLGMII trace data written to the log, as shown below
NOTE: ordering is the last cycle printed (at the end of the output) is the last cycle captured.
.
.
. (10049757 times)
.
.
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
XLGMII: Ctl:11111111_11111111 Dat:0707070707070707_0707070707070707 : iiiiiiiiiiiiiiii
[009:079:007] Rep: 8 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[008:07a:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[007:07b:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[006:07c:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[005:07d:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[004:07e:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[003:07f:007] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[002:080:008] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
[001:081:008] Rep: 1 : T XLGMII: Ctl:10000000_10000000 Dat:9c00000100000000_9c00000100000000 : ----------------
State
bram_index : 0
enable : 1
trigger : 1
zerodata : 0
trigger_disable : 0
trigger_runt : 0
trigger_over : 0
trigger_fcserr : 0
trigger_sanitize : 0
trigger_framerr : 0
trigger_pcserr : 0
trigger_pcsother : 1
trigger_pcssop : 0
trigger_pcssop_err : 0
trigger_pcseop : 0
trigger_back2back : 0
trigger_debug : 0
trigger_enb_runt : 1
trigger_enb_over : 1
trigger_enb_fcserr : 1
trigger_enb_sanitize : 1
trigger_enb_framerr : 1
trigger_enb_pcserr : 1
trigger_enb_pcsother : 1
trigger_enb_pcssop : 0
trigger_enb_pcssop_err : 0
trigger_enb_pcseop : 0
trigger_enb_back2back : 0
trigger_enb_debug : 0
PCS Codeword histograms
[ 0] SOF: 0 EOF: 0 ERR: 0 OTH: 16
[ 1] SOF: 16 EOF: 0 ERR: 0 OTH: 0
[ 2] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 3] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 4] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 5] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 6] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 7] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[ 8] SOF: 0 EOF: 0 ERR: 0 OTH: 16
[ 9] SOF: 16 EOF: 0 ERR: 0 OTH: 0
[10] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[11] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[12] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[13] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[14] SOF: 0 EOF: 0 ERR: 0 OTH: 0
[15] SOF: 0 EOF: 0 ERR: 0 OTH: 0In this case the PCS is generating SEQ errors (0×9C) highlighted below. The “T” highlighted below shows this is what triggered the trace logic to stop (based on the command line configuration)
.png?sv=2022-11-02&spr=https&st=2026-03-25T03%3A57%3A13Z&se=2026-03-25T04%3A27%3A13Z&sr=c&sp=r&sig=utvmepVJHe2vtaAFvwqq2clsyKftCpQXzRa8CPThRwE%3D)
The error type is usually seen are SEQ (0×9C) or ERROR (0xFE), contact support @ fmad with the layer 1trace further debug and investigation on the error.