If you unplug the Ethernet from the pc and wait a few minutes, does network connectivity return for the rest of the network? That would be a good test.
Added to the list!
That wouldn't necessarily be enough, briefly unplugging everything from the switch and replugging would though.
@dlakelan could you elaborate a bit -- Why do you think that may not be enough? Not doubting you, just curious what other things might be in play that might require unplugging most/all of the devices from the switch? Are you thinking that there might be a switching loop or something else that would keep circulating the packets even after the potential broadcast storm ends? Or that a broadcast storm may spawn responses from the other devices and thus have a long tail before the broadcast traffic settles down?
I'm thinking of a single device (the offending PC) that causes a storm and takes down the network as I have seen happen with some USB-C docking hubs. When they are unplugged, the network usually returns to normal in less than a minute. But maybe I'm missing some angle that also should be tested/explored.
Yes, if one device simply initiates a gigabit of broadcast as soon as it's plugged in, then unplugging it will be enough. But if what's going on is that the one device is somehow triggering other devices to send gigabits of broadcast packets as well, then you might need to remove them from the switch as well before you get control. And, yes, if there's some kind of loop/reflection issue then maybe you'd need to unplug more items to get it to stop.
One thing that's great about using smart-managed switches is that they often have configurable limitations on broadcast, multicast, and unknown-unicast packets. This makes it impossible for any device to take down the whole network by sending broadcasts.
For example on Tp-Link's switches, here I've limited Unknown-Unicast, Multicast, and Broadcast traffic to 8Mbps on all ports.
1 Like
Ok... yes, makes sense. Just a hunch here, but I suspect that the PC, if it really is causing a broadcast storm, will be the only offender, and that the network will return to normal once it is unplugged.
I agree about smart/managed switches -- they have a few different methods to potentially protect against these types of issues. Interestingly, though, some of the worst offenders (those USB-C docking hubs I referenced) seem to be really difficult to stop with most smart/managed switches, at least based on the reports from people I have helped (I don't have one of these devices, so can't test personally). My TP-Link T1600G-28PS has similar storm control features, but I've never had to use them.
Anyway, looking forward to learning from the OP what they learn with the various experiments.
I appreciate all the discussion and will definitely follow-up when it happens again. It's a watched kettle now though.
I just automatically configure them for all ports. I don't have any scenario on my network where more than a few megabits of broadcast or multicast is legit so I figure it can't hurt.
1 Like
There is a bug on some NIC that I think would cause a broadcast storm or something. I think it was something about jumbo frames? I wanna say Realtek but that is a guess. If it is a broadcast storm, it doesn’t cause an OOM but just maxes out the CPU. The Archer maxes around a gigabit. Also if it was a broadcast storm, it would keep your packets from getting to the router.
I would force the offending NIC to like 10 Mb/s so it won’t kill your network, or isolate it somehow. Then get a packet trace.
So it's recorded here, this one is a Intel 82579V Gigabit with the same (v12.18.9.23, Jun 10, 2020) driver that it's had since before the problem started, so it's probably not at fault alone. Perhaps some other aspect of Win10.
So, for some reason, these incidents slowed down and outright stopped for a while, but today returned. The trigger event was a reboot for a Windows Update (many times, it seems, that's the case): within a minute or two of that completing and being back in Windows, Internet was gone for that wired PC and other devices on the network, wireless and wired. The Internet was available for that first minute or two.
So they did the unplug test for the Ethernet cable of the suspect PC.
The result was that very quickly (half minute or so), not only did the other devices on the network that couldn't access Internet come back but so did the problem PC (once plugged back in, of course).
Rebooting the PC did not re-trigger the problem, but that's been the case before: you can't seem to trigger it when you want to, just when it wants to.
Would plugging into a switch alleviate the problem somehow now that we pretty much know that it's a broadcast storm issue?
Here are the available Advanced options for the NIC:
Jumbo Packet is on Disabled. Choices are 4088 Bytes and 9014 Bytes.
Complete settings:
In those over 4 years, have you never tried creating a Win10 media creation boot stick to reinstall the PC from scratch with recent drivers, without taking over old config? Or using a USB network adapter instead on the PC? (and checked for BIOS Updates?)
4 years? The installation is maybe a year and a half old. It's not that (what kind of Windows issue could even cause a broadcast storm?). And it's long since stopped having any BIOS updates.
Have not tried another NIC, but there might be other avenues to explore first, like whether not plugging into the router directly would be of any relevance, along with tweaking any of the esoteric NIC settings referenced above.
If it really is a broadcast storm, then plugging into a cheap managed switch with broadcast storm mitigation would mitigate the problem (it wouldn't prevent the storm but you could keep it down to a couple megabits).
The tp link sg108e has these broadcast, multicast, and unknown unicast storm control settings.
Is your TP Link switch configured with a static IP address via the switch's GUI? The default IP address on my switch is 192.168.0.1, which conflicts with my router's address (also 192.168.0.1). I had applied a static address for the TP Link switch in OpenWrt, but when I rebooted the switch, it reverted to the default address and that caused me problems.
I have now assigned the desired IP address directly on the switch and the problem is resolved.