2.5 - 10 Gbps internet

8gbps down and 4gbps up - at least this is what VMware says

A single one‐vCPU virtual machine can drive 8Gbps of traffic on the send path and 4Gbps traffic on the
receive path when using standard MTU (1500 byte) frames.

Yes. I expect to go that high..the problem is(and this is where i should have been a bit more clear) is that with one client to one server connection you will not reach that high.

If you have a few concurrent connections then you can, in fact, saturate the link.

Back to the topic at hand.. the best option is to build an x86 openwrt router and use 10g nics. Also.. a 10g switch is somewhere around 300$.. and then the pc NIC's... quite an expensive endeavour if you ask me :slight_smile:

yes, yes, everyone starts talking about the budget aspect but let's forget this for the moment as I believe 10g internet users have a high enough budget for this upgrade (or to justify the need)

Not wanting to derail your discussion, but in some countries "10G" internet access comes at not-crazy prices:
salt.ch: "10Gbps" 49.95 CHF/month (quotes, because 10Gbps XGS-PON uses some bits for FEC leaving < 10 Gbps for payloads)
init7: symmetric 10Gps 64.75 CHF/month or even symmetric 25Gbps for 64.75 CHF/month
free.fr: "10Gbps" 39.99 EUR/month

IMHO this will result in folks taking "10Gbps" internet that do not have their internal network readied for those speeds.

already covered that some time ago :slight_smile: 2.5 - 10 Gbps internet - #11 by maurer

2 Likes

Sorry, lost track, should not have posted without reading the full thread.

Considering that the cost of the 10g is low .. i would get it just because i can. And at worst i can upgrade later. For start you can get 10g nic at around 100€ for the pc and the rest later... just ..use cat6a... :slight_smile:

3 Likes

You can get used Broadcom 57810S based dual port 10gbE server NICs för $30 on ebay.

Cat5e works for 10gbit, when going shorter distances.

1 Like

So speaking of which: see https://threats.amnpardaz.com/en/2021/12/28/implant-arm-ilobleed-a/ for how a BMC (HP's iLO) was abused maliciously. Sure the culprit was iLO and not the multiplexing of iLO's access via the same network interface, but my point is unless BMC is to be used not having that capability seems not a loss.

LC simplex patch cable for 222 CHF? Damn, that must be made of gold.

On Ali 13 bucks:

You can go for Copperten cabling, but honestly 10Gbps on bare metal makes not much sense. Fibre optic can then more reliably handle the connection, but when looking for hardware on Ali that can be pretty expensive:

The 10Gbps on metal is the top speed that copper is actually able to handle - and this solution is more than 10 years old. Unless you want the 10Gbps connection for a data center, for an ordinary user it does not make any sense, whether it is only 2 euro or for free.

These 222CHF are for the patch cable as well as the SFP28 optic module... at least when I look that part up on flexoptic.net I would need to pay 368 EUR for the optic alone, so from that perspective 222 CHF does not sound too bad. On the other hand 25G SFP28 modules with similar specs apparently can also be had for less money (I have not researched whether these cheaper options actually are compatible though).

But they also offer bring-your-own:
" Compatibility requirements

There is no obligation on you to procure the hardware through us, and the hardware shown here is not the only possible hardware for you to use. There are also other compatible products, as long as the requisite «bi-di» fibre optic technology conforms to the following specifications (recommended: Flexoptix (1G, 10G, 25G), more router information):

  • Simplex single-mode cable as patch cable LC/PC to LC/APC, compatible with SFP optic
    • Fiber7-X2: 25G SFP28 BIDI LR, 10 km, TX1270/RX1330 nm, LC-Simplex, Singlemode
    • Fiber7-X: 10G SFP+ BIDI LR, 10 km, TX1270/RX1330 nm, LC-Simplex, Singlemode
    • Fiber7/Crossover7/Hybrid7 (P2P): 1 Gbit Singlefiber SFP Transceiver (10 km/TX1310/RX1490-1550 nm)

Longer cables (6m, 15m) are available on request."

Well, the SFP28 module is 40 bucks on ali plus the cable for 13 bucks that makes 53 bucks against 222 CHF.
I used to work for a telecom company for 10 years and we had re-branded chinese equipment purchased for 200 dollars and re-sold for 1500 dollars. Again fibre optic. The only difference was that the re-branded equipment was "quality assured" and re-sold with a big margin. That what makes the managers to drive a brand new BMW. The greed has been always with the telecom business and I see it stayed there even after I left the business.

Again I would state the same - running a 10Gbps network at home can be pretty expensive, not only in terms of cost of ownership, but in terms of energy consuption. Unless you are running high end servers on a backbone, you will not need it. The copper cable has reached its limits in terms of physical possibilities with 10Gbps and when you are not prepared to invest massively (3k-6k USD) in your equipment, then just leave it out.

Yes, but your original claim was "LC simplex patch cable for 222 CHF?" so you clearly had not followed that link and did proper research...

Yes, but as I cited, Init7 (whom I have no connection with what so ever) openly documents the specs and will happily have you bring your own devices, so this specific line of complaint is pretty much moot, independent of your own experience with unsanitary price-upmarking practices (which I do not doubt exist) in the business, no?

Fair enough.

I tend to not tell people in the forum, what they need or do not need a priori, I consider that a bit tricky without knowing the details of their requirements and use-cases...

That is a line of reasoning I have heard multiple times in the past about technologies that in spite of such claims kept on evolving and improving, like silicon was once claimed to not be able to reach high frequencies and we would need to switch to Gallium arsenide like a decade ago, or telephony wiring not allowing higher speeds than what ISDN offered... Given all the investment in existing structured copper wiring I would not count out copper just yet especially since 40Gbps copper ethernet is already a thing (probably pricy as hell, and limited to 30m).

IMHO, 10Gbps is not going to be the next consumer speed step, I expect 2.5 or 5 Gbps Ethernet to appear there first (simply because prices have reached a level enthusiasts are willing to pay) and then stay there for a while. But 10Gbps NICs are certainly already in enthusiast range as well (switches not so much). But with ISPs supplying 10Gbps links, they will also make sure to distribute CPE that have at least on LAN facing 10 Gbps port and so the "game might already be on".

1 Like

You might be true, that the game might be on, but the PC hardware is not there yet. With PCI express 4.0 you can handle 2000MB/s and 10Gbps symmetric connection means 2500MB/s. Even with the newest PC hardware on X64 intel or AMD you are barely able to handle the throughput.nowadays. This race with higher and higher speed resembles on the 3D TV screens and blu-rays, both of which were promising technologies and the marked killed them. But let us see, whether someone comes up with a router capable of handling 10Gbps speed and will cost only 100 bucks.

Mmmh so PCIe 4.0 gives (according to wikipedia:
16.0 GT/s 1x: 1.969 GB/s 2x: 3.938 GB/s 4x: 7.877 GB/s 8x: 15.754 GB/s 16x: 31.508 GB/s

I would respectfully recommend to use a 4x or 8x NIC instead, if a 1x does not cut it anymore...

I am not convinced that this is a veridical statement of the state about the art, sorry. Handling 10 Gbps is far from rocket science nowadays, just look at USB3... which scales from 5 to 20 Gbps.

"when" not "if/whether", first generations will rely heavily on accelerators (just as they already do for 1 Gbps). But in all honesty, 100$/EUR is a threshold for the trailing edge market, people seem to be willing to pay up to say 200-300 EUR/$ for capable routers even for speeds well below 10Gbps, so it will take a while for such devices to creep into the bargain bin.

3 Likes

Dear Friends,

After using OpenWRT during years (a decade), I migrated to Mikrotik for the sole reason of supporting 10Gbit, here is my feedback to the community. This is a home setup.

Please note that I am an ancient OpenWRT router (you can verify in my history), this is NOT a spam.

  • 10 Gbit copper RJ-45 is problematic. Connectors get very hot (around 90°c) and disconnect to heat down. The technology simply does not work. It means that if you are using a 10gbit switch with copper it will be spinning all day long and heating. I shipped back all my 10Gbe copper equipment. 10 Gbit copper is DEAD.

  • The only solution is either to use SFP+ fiber or SFP+ DAC passive cables (around 20 euros).

My setup:

Switches:
CRS112-8P-4S-IN
A very nice fanless 1Gbit switch with 10 Gbit uplink

CRS309-1G-8S+IN
A full SFP+ small fanless switch.

Both switches are interconnected with DAC cables.

On laptops, we are using Sonnet Solo10G SFP+ Fibre 10000 Mbit/s
with firewire.

As for routers, you have two solutions:

If you need 2.5 Gbit Internet,
RB5009UG+S+IN

If you need full 10Gbt Internet,
CCR2004-16G-2S+

I think using fifer/DAC and SFP+ is the way to go. I am dropping WIFI, unless good WIFI OpenWRT are available with at least 2.5 Gbit connectors.

All this is quite expensive for a low benefit as all Internet technology is mainly 1Gbit/s. But this was an experiment and a way to learn more about networks.

My building is made of concrete and all my neighbors are using WIFI 2.4Ghz and 5Ghz. So I am migrating to full fiber, which seems to be the long term solution. I am still using Gl.inet OpenWRT access points, but as secondary solutions.

Kind regards,
FFries

I'm not disagreeing with your post, but this particular issue is (hopefully) more of a design failure of this particular device/ vendor, than an issue of 10GBASE-T over copper in general. Yes, 10 GBASE-T needs considerably more power than <=2.5GBASE-T, even more so in older chipset designs, but keeping that at bay (and be it with active cooling) is the manufacturer's job - not a standards inherent problem.

The problem I have with condemning copper based >>1 GBit/s at large, is that we are starting to see copper based consumer/ prosumer devices with >>1 GBit/s in the field (routers, wifi6 APs, etc.), while SFP (especially SFP+) continues to be the rare exception outside the professional (enterprise) field. Yes, power consumption and heat dissipation are a problem there as well, that's why vendors seem to push 2.5GBASE-T instead of 5GBASE-T/ 10GBASE-T first (probably also as a means to get paid twice…), but we will have to deal with predominantly copper based networks for considerably time to come in the consumer and SOHO markets.

Generally speaking, there is also a much higher power consumption when using SFP+ to RJ45 10G adapters. Even on cisco devices there is a limit (maybe 10 i think or 20 of these on a 48 SFP+ port switch).

10 Gbit copper RJ-45 is problematic. Connectors get very hot (around 90°c) and disconnect to heat down. 

It all comes down to the cooling capabilities of the device you are using them. If there is not enough cooling they will go into thermal shutdown as you described. It can be true for SFP+ fiber modules in certain situations.
I would avoid as much as possible the SFP+ GLC T adapters...
If they are still available, and port number is not an issue: CRS305-1G-4S+IN (https://mikrotik.com/product/crs305_1g_4s_in) might be an option as well.

Dual psu, metal case, but the passive cooling means it will run pretty hot.

for SFP's and modules/fiber etc you can check out
www.fs.com and www.solid-optics.com. I am not sure they are selling these to end users though...I haven't tried to buy anything from them yet.

Well, yes, if we are going to make a big upgrade to network connections and enter the 10GB era, the logical path seems to migrate to optical connections.
More robust en many senses, less power consumption, no heating, no interferences, larger cable runs...
And you need new cabling anyway, so changing to fiber makes sense.

The only problem is price, but 10GB copper installations are not cheap either.

Price will drop down as more devices are available.
It makes sense for device developers to offer mainly fibre 10GB devices.
I hope they do and prices go down in a few years.

If manufacterers concentrate in fiber device production, the price will drop down earlier.

It should be "prohibited" to produce copper 10 GBps products xD

I think that nowadays if you are planning for more than 2.5 GBps network, you should go the fiber way.

May be you cannot use it everywhere (too expensive) but where ever you need such a quick connection.

I had thought of using fiber for installation in my new house, but it is too expensive yet, and I don't have really a need for more than 1 Gbps network connection.
And you need converters in almost any device you want to connect.
So for me is just a dream.

In the future when optical cards become the standard, I will need to lay new fiber cables where needed.

1 Like