NIC limited to 100Mb/s: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 06)

[pantalon]

Since switching from Core to Scale my NIC is only connected with 100Mb/s. I know, I know, Realtek isn't really recommended but I'm on mITX and my PCI slot is occupied by a SATA card.

Motherboard is: ASRock H87m-ITX

Can I fix this myself somehow? It worked fine and stable with 1Gbit/s on TrueNAS Core

 

[jgreco]

Unfortunately, completely switching OS's from FreeBSD to Linux means that some poorly supported devices such as the Realteks may change their "compatibility" level, which was never high to begin with. I would say it is somewhat unusual for Linux to support cheap Windows crappy devices like a Realtek worse than FreeBSD does, but it isn't unheard-of.

On the upside, you can definitely cruise around the Linux forums to see if anyone has had similar experiences. There's going to be at least ten times as many people who've had such experiences on Linux, because there's a larger userbase running on the least expensive hardware they can find. There are some different tweaks that can be applied to ethernet interfaces under Linux, but I'm not the right person to document those for you.

 

[awasb]

Don't know much about Scale. But since kernel-upgrade to > 4.19/5.x on some Linux boxes I had to enable flow control on some switch ports connected to Broadcom nics. Maybe you have a similar problem?

What is the output of

Code:

[sudo] ethtool eth0

? (Recommending ethtool here, because mii-tool seems deprecated.)

BTW: Did You know? Re-al-tek is a very old phrase from the Altaic languages meaning "now we got your as*, longnose!"

 

[pantalon]

Code:

Settings for enp3s0:
        Supported ports: [ TP    MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 100Mb/s
        Duplex: Full
        Auto-negotiation: on
        master-slave cfg: preferred slave
        master-slave status: slave
        Port: Twisted Pair
        PHYAD: 0
        Transceiver: external
        MDI-X: Unknown
        Supports Wake-on: pumbg
        Wake-on: d
        Link detected: yes

What exactly is flow control?

 

[jgreco]

Flow control starts and stops the flow of data at the request of the network (typically the switch).

 

[Arwen]

@pantalon - The output of "ethtool" indicates that your Ethernet switch does not support 1Gigabit per second speed. That's the "Link Partner advertised" section.

This can happen if your cable has only 2 pairs, (or 2 working pairs), or their is a fault in the 8p8c, (aka RJ-45 jack), connector. I've seen bent pins prevent traffic. And I've seen bad patch panel connections prevent Gigabit speed, limiting that connection to 100Megabit.

In the bad old days, some Ethernet switches did not implement Auto-negotiation properly, so some forced their connections to be what was needed. That should not be needed today, except as a trouble shooting step. And is too complex for me to walk you through it.

Flow control in "ethtool" is shown by "pause frame" lines. That won't affect your raw speed, but can affect data loss. (If one side sends so fast that the other side has to drop packets, flow control / pause frames can help prevent packet retries.)

 

[pantalon]

Problems with the cable are basically impossible. I didn't touch the machine other than upgrading from CORE to SCALE. I've been pushing files at 1Gbit/s (~100MB/s) regularly. It just showed up as 0.1Gbit/s since the upgrade to SCALE.

 

[awasb]

What is your SCALE machine connected to? Please give some details.

 

[pantalon]

It goes straight to my router (Fritz!Box 7530) and always has. There's an energy saving option to limit ports to 100Mbit, but I never used that. I tried setting them to 100Mbit and then back to Gbit, but no dice.

All my LAN devices go straight into the Fritz!Box (PC, Steam Link, Smart Home Hub, NAS). I feel like drawing a network diagram for that would be overkill

 

[awasb]

That's enough.

Simple solution: Get a proper switch (a small _managed_ one wouldn't be that expensive).

The Fritz!Box 74XX/75XX problems with 1 GbE-devices are quite common. Over here I couldn't connect a raspberry pi 3b+ (secondary DNS) and another pi 4b (OctoPrint) with 1GBit to the FB7590 until I added another Netgear GS108v3 to the cascade (the 3b+ needed flow control enabled because of the bus limitations).

 

[pantalon]

But why did it work for years on Core if that's the culprit? Just seems odd to me to now swap my router/get another switch

 

[awasb]

If the cable is sane, you could switch nics as well. But a „real“ switch will give you much more insight (and control). The FB is a black box in comparison.

Guessing reasons: Changes in the firmware? „Enhanced“ drivers? Fritz!OS-„Update(s)“? There is a plethora of possible reasons.

 

[gobygoby]

I am having either the same problem as pantalon, or at least a very similar problem. I was using TrueNAS Core for several years and the transfer speeds between my windows machine and my TrueNAS server were about 90MB/s in both directions.

As soon as I upgraded to TrueNAS Scale, the transfer speeds immediately dropped to 11MB/s in both directions and I cannot figure out why. Other than upgrading to Scale, nothing else in my network has changed.

 

[gobygoby]

I just ran the following command (which I found in another forum): "lshw -C network"
And here is the response:

*-network
description: Ethernet interface
product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
vendor: Realtek Semiconductor Co., Ltd.
physical id: 0
bus info: pci@0000:03:00.0
logical name: enp3s0
version: 0c
serial: 98:90:96:a2:dc:39
size: 1Gbit/s
capacity: 1Gbit/s
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress msix vpd bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
configuration: autonegotiation=on broadcast=yes driver=r8169 driverversion=5.10.70+truenas duplex=full firmware=rtl8168g-2_0.0.1 02/06/13 ip=192.168.1.138 latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s
resources: irq:18 ioport:d000(size=256) memory:f7e00000-f7e00fff memory:f0000000-f0003fff
*-network:0
description: Ethernet interface
physical id: 2
logical name: kube-bridge
serial: 2a:be:28:98:83:94
size: 10Gbit/s
capabilities: ethernet physical
configuration: autonegotiation=off broadcast=yes driver=bridge driverversion=2.3 firmware=N/A ip=172.16.0.1 link=yes multicast=yes speed=10Gbit/s
*-network:1
description: Ethernet interface
physical id: 3
logical name: kube-dummy-if
serial: ea:80:0f:b7:34:9d
capabilities: ethernet physical
configuration: broadcast=yes driver=dummy driverversion=5.10.70+truenas ip=172.17.22.245
*-network:2
description: Ethernet interface
physical id: 4
logical name: vethc0837077
serial: 56:6a:1e:96:6b:13
size: 10Gbit/s
capabilities: ethernet physical
configuration: autonegotiation=off broadcast=yes driver=veth driverversion=1.0 duplex=full link=yes multicast=yes port=twisted pair speed=10Gbit/s
*-network:3
description: Ethernet interface
physical id: 5
logical name: veth73fa0312
serial: 2a:4e:5f:d6:87:ca
size: 10Gbit/s
capabilities: ethernet physical
configuration: autonegotiation=off broadcast=yes driver=veth driverversion=1.0 duplex=full link=yes multicast=yes port=twisted pair speed=10Gbit/s
*-network:4
description: Ethernet interface
physical id: 6
logical name: veth53293329
serial: 9e:d2:a2:0e:c7:5d
size: 10Gbit/s
capabilities: ethernet physical
configuration: autonegotiation=off broadcast=yes driver=veth driverversion=1.0 duplex=full link=yes multicast=yes port=twisted pair speed=10Gbit/s

Does anyone know how to interpret this? I don't have any 10Gbit/s equipment in my network, so that part is especially confusing.

 

[gobygoby]

After researching some more, I think the problem may be driver related. See this page for more info:

How To get your Realtek RTL8111/RTL8168 working (updated guide)

Image Source: wikipedia.org A lot of people will remember my guide how to get a RTL8111/RTL8168 running under your Linux box. This guide is almost 5 years old now and I wanted to make a complete ov…

unixblogger.wordpress.com

 

[pantalon]

If the cable is sane, you could switch nics as well. But a „real“ switch will give you much more insight (and control). The FB is a black box in comparison.

Guessing reasons: Changes in the firmware? „Enhanced“ drivers? Fritz!OS-„Update(s)“? There is a plethora of possible reasons.

Last Fritz!OS update was on September 1. and as I said, it worked right up until I switched to Scale.

 

[awasb]

I understand, you don‘t wanna buy something, as long as the definite cause is unclear, it worked before (with BSD kernel, that is) etc. etc.

Could you then just borrow a managed switch, to test things out (and eventually see some status protocol)?

 

[pantalon]

Only way I could "borrow" one is to order it and send it back later. I've been thinking about a hardware upgrade anyway, so I might just invest in new MB+CPU+RAM. But it frustrates me to be stuck on 10% speed until then. I'm not sure if I trust myself switching drivers like gobygoy mentioned.

I'm gonna test out a different cable later and I do have an unmanaged Netgear switch lying around somewhere. Don't know if it could make a difference putting that in between the FritzBox and the NAS

 

[awasb]

Well ... it could. But it highly depends on default config (of the switch) and whether autonegotiation finds the right settings to adapt. It's hit and miss (as with the FB).

 

[Arwen]

@pantalon - It appears to be a difference in Ethernet Auto-negotiation between FreeBSD 12 & Linux kernel 5.10.

I've seen similar issues before, but nothing in the last 10 years or so.

You could try to force your TrueNAS SCALE to 1Gigabit per second speed. I don't have the details, but the manual page for Linux "ethtool" and google searching should get you an answer. Whence you've proven in concept that it works through a "root" shell, you could look at making it permanent.