It Was The Cable
Small one today. No big announcement, no build I've been sitting on for weeks, just a thing that happened in the homelab that I want written down somewhere because it made me stupidly happy for about twenty minutes.
I've got a Pi running NFS off bigbox, and for a while the throughput between them was stuck at around 350Mbps. Gigabit link on both ends. 350. Not terrible, doesn't matter for most day to day stuff, but when you're moving a couple hundred gigs a day across that link, the gap actually adds up, and it bugged me every time I saw the number.
First guess was the obvious one, the Pi's NIC just isn't great, everyone knows the onboard ethernet on these things tend to buckle under a real load. Fine. Reasonable theory. Except it didn't actually hold up once I started testing it properly.
So I did the boring thing. Swapped cables between bigbox's two NICs one at a time and watched what moved with what. If it's the Pi, the fault stays what ever route the Pi takes. If it's a cable or a port, the fault should follow the cable around when I move it. And that's exactly what happened. The 100Mb/s half duplex fault packed up and moved to whichever port I plugged that specific cable into, completely independent of which NIC on bigbox it was touching.
Long story short it was the cable. Not the Pi, not the switch, not some driver issue I'd have spent a weekend chasing through kernel logs for no reason. One bad cable, sitting in a rack, quietly capping everything that touched it.
Swapped it out. Ran iperf3 again. 874Mbps average, near enough to gigabit that I stopped caring about the last hundred and some megabits.
There's no lesson here beyond the one everyone already knows and nobody actually does, check the cheap dumb thing first before you start theorizing about NIC chipsets and driver quirks. I know that. I still went straight for the interesting explanation instead of the boring one, because the boring one is never as fun to think about. It was still the boring one.
Anyway. Gigabit's back. Onto the next thing.