Performance = v18/19/21/22?

Installing and Using OpenWrt
1

Hey fellas

I'm looking to get the most out of my little MT7620/7621/7628 based routers (like xiaomi).

I'd like to not having to reboot them so low resource, and higher throughput for the vpn services I'm running (like softether).

I'm running them for private use so I'm not all that concerned with security. So I'm just wondering. Which version of OpenWRT would be faster, use less resources etc?

I've only tried v19 which I'm fairly pleased with. I tested v21 but it seemed heavier and I read there's less throughput performance so I switched back.

Thanks

2

Run some tests and let us know!

2 Likes
3

And do reflect on the contradiction (and irony) of running a VPN on unmaintained and possibly insecure releases like 19.07 or 18.06.

6 Likes
4

It's not for security. It's to obtain different access/peering onto the internet. Like a proxy. But one that supports udp and everything.

I'm in a place where the network gets really congested during peak hours. Popping a 2usd vpn in the form of a xiaomi router in a friend's home (with better isp) can mitigate a lot of the trouble.

5

Neither botnets, nor ransomware, nor your neighbour's kid trying to get free wifi care about you being a private citizen or a company, they'll take your insecure gear as is and maximize their gain and your losses. In the modern world, non-security supported devices mustn't ever get on the ether (and that includes wifi, not 'just' the wide open internet), so anything but 21.02.3 or 22.03.0 should be out of the question. Performane is secondary (within reason), if it isn't sufficient any longer, you need look around for faster hardware (operating a device just at its performance limit isn't really workable in the long term anyways).

6

I appreciate the good intentions, but how remarkably unhelpful.

Why don't you guys just get openwrt to pull the download files instead of going after people who want to make full use of old hardware.

7

I didn't even consider v17 but now that I see it's still officially available I'll compare throughput on it too. I just had hoped that someone beat me to it. Where are all those users? :grinning: Hmm...

Xiaomi stock is based on OpenWRT v15 if I'm not mistaken. Still an improvement now isn't it? :joy:

8

Based on an OpenWrt 15.05 based SDK. :wink: By all means, feel free to decide for yourself. Nobody here is pushing you in a certain direction, we're just putting things in perspective. Do with it as you wish. That is your prerogative.

1 Like
9

You will get a lot of people here that believe that unless you are using the latest cutting edge made yesterday version you are in trouble. The idea that you want to maximize performance usually turns into "buy newer, faster, more expensive hardware".

I've been working with OpenWrt since Barrier Breaker days and the trend seems to be every new release takes more resources and doesn't really run any better. The SOC are faster which makes a big difference but is the code any better? I don't think so.

The security issue is always brought as a reason for switching to the latest version but that really valid? I don't remember people screaming that v18 was a total security risk at the time and things needed fixing right now. Same for v19 and v21. It is only after a new release is made do people start saying that the old versions are a risk.

I have produced thousands of different OpenWrt based firmware in use all around the world with probably 50,000 being used by commercial companies in the US alone. I have never hear a thing about security problems even though the version run from v18 to v21.

Yes, new exploits appear all the time but does that make the latest version safer? Not really. The code runs a ways behind the exploits so if a new one appeared tomorrow the fix would have to wait for months to appear.

In every release the kernel size becomes bigger and more default packages are added so there are a number of totally useful routers being cut out because of lack of memory. The Qualcomm AP147-010 based routers are a prime example. There are hundreds of different versions of this out and they are mostly dropped from later versions because of increased memory/kernel size. Yes, they aren't real fast but they still have a use and are still being bought in large numbers because of cost. In my world of routers and modems , when paired with a Cat 4 modem, they are a popular low cost router that almost every Rural ISP uses.

I think the only way you'll determine which version is better for you is to try each one on the same router. Watch for things like amount of flash used by firmware with a similar suite of packages and RAM usage when running. Checking the Load when something is running will tell a lot about what is going on under the hood. Does Load vary with different versions doing the same thing?

It will be interesting to see a good study of this being done as it would cut down on the newer is better thoughts.

2 Likes
10

I'm going to chime in here to say that I agree with @Borromini and @slh in that using older versions is a bad idea from a security perspective.

@Dairyman - your argument hinges on the idea that a vulnerability is known but not being actively exploited... in those cases, a patched version is obviously theoretically more secure than an unpatched one, but you would suggest that it is not much safer in practice... sure, I can give you that for security issues that are indeed not in the wild. However, earlier versions ( <=18.06) have known and actively exploited vulnerabilities, and are not suitable for the internet anymore as there really are tons of bots out there looking to compromise other network devices.

Put another way, if you found out your house key had been duplicated by some unauthorized person (maybe a sketchy valet at a restaurant), would you not change your locks as soon as you reasonably could? Maybe it is unlikely they'll hit your home in the next few days or so, but you wouldn't want to leave it too long.

Likewise, @alcatraz - you say that security isn't that important to you... but that's probably because you haven't had any serious issues to date. I'll make another analogy... many US states require annual auto inspections, and they'll look at things like your tire condition, remaining brake life, and other things that they deem necessary for safe operation of the vehicle. You may say that you don't drive your car fast and in situations that need exceptional traction, so what's the big deal if your tires are bald. You say that until you have a serious accident after losing traction unexpectedly, after which you realize just how important tires can be. Back to internet security -- the moment your computer is seriously comproimised and your data held for ransom, or your main email account(s) are hyjacked, subsequently making it difficult for you to access pretty much your entire online identity, you'll realize that security is important, even if your stuff has seemingly little value to others.

2 Likes
11

This. SO much this. The good old days of "if it ain't broke, don't fix it" are FAR behind us.

@Dairyman, you may be one of the luckiest people I've ever seen (or your customers are). Certainly agree that most (but not all!) exploits are not easily exploited against OpenWRT from the Internet so long as you aren't doing inbound allows straight to the device, but the moment ANY sort of malware appears on a device sitting behind it (and it propagates), the game is over...LAN attacks against most vulnerabilities are trivial. You really are playing with fire. Saying "it hasn't happened to me!" probably applies to heart attack, too...but in a very similar manner, it's just a matter of time. I hope you have a good lawyer to fend off the suits from your former customers once they are compromised.

3 Likes
12

The answer honestly is - it depends.

Some routers can now do software & hardware offloading in which performance is dramatically increased.

Others which are ROM/RAM/CPU constrained my have degraded performance due to the kernel growing and are now hitting limits.

Also "performance" can mean more stable due to bugs fixed, not just raw throughput.

13

Thanks for the time invested and for having the guts to speak your mind. Your post is very helpful to me.

I get it, when possible, avoid running old software. GETTING PASSED THAT, what are your observations of throughput loss after upgrading?

The Xiaomi Mini (MT7620) is supported from v17 all the way through v22. I find that to be an interesting opportunity to find out how much (if any) is the loss of staying up to date.

14

Nothing gets people stirred up here faster than not agreeing with them.

Another case of "My opinion is the only right one" syndrome.

So let's look back at the original post.

The question is about performance and not which is more secure. But instead of answering that people get on their hobbyhorse and start talking about which version is more secure. That was not the question.

If I asked you which car was faster, a Camaro or a Mustang, I'm not interested in hearing that I shouldn't drive the Camaro because the Mustang is much safer. I asked about speed, not safety. If you don't know the answer then say so. If you asked me the way to the bus station and I answered "Purple" you wouldn't be happy.

It is obvious that no one knows the answer to the question and only one person even addressed it. And those that are focused only on security are upset because everyone isn't the same.

And which world are you living in?

Is Linux sued because an exploit has compromised computers running it? Just because it may have been fixed at some point doesn't mean they aren't responsible for older versions, at least by your logic.

Look at all the routers being produced using some hacked up version of OpenWrt 15 or 17. There are thousands of these out there and have you ever heard of any of them being sued because of an exploit? Of course not. Or OpenWrt being held responsible for not retroactively fixing the exploit?

If you wish to focus strictly on router security then more power to you. It is something that needs to be done. But not everyone is going to be that way and getting upset because we aren't isn't going to help.

And getting back to the actual question here, you are so correct. There are many factors to take into consideration when looking at performance. And there seems to no test suite that can be used to measure it.

The only way in most cases is to try each version doing what you intend the router to do. If you want to pass huge amounts of data then test doing that. If Wifi performance is key then test doing that. If you use a VPN check how it handles that. In the world of routers and modems the most looked at test is download speed. Which firmware gives the best speeds on the same router.

I do too. This is something that just doesn't seem to have been looked at for whatever reason. A topic for discussion.

2 Likes
15

Windows 95 would scream on any modern hardware.... it would be far faster because it uses less resources (RAM, processor, storage). You can see where I'm going here -- older OpenWrt versions will be the same, in general. But in both cases, you would lose all of the security improvements over the years, and obviously functionally would have limits because of what existed (or more importantly: what had yet to be developed) at the time.

So, all else being equal, the older versions will probably be more performant for many things outside core routing/switching. Routing and switching, though, would likely be largely the same, except if HW or SW offloading improved performance in later versions (which sometimes is the case).

As a community, most of us feel that it is a moral imperative/responsible thing to steer people away from versions that have been deprecated and are unsupported and contain many security vulnerabilities. It is better for the user and better for the internet at large when modern security standards are used and older/insecure technologies are avoided. That is why you got the response you did. On that note, you have been warned -- you must accept the risk should you use an old version.

Also note that using anything pre-21.02 is technically unsupported. 19.07.10 went EOL a few months ago. Part of the 'unsupported' nature is obvious -- no further developments/patches. Other parts are that (as noted above) it is best to ensure that people move on from the older insecure technologies. And the final bit is that there are often significant changes over time with respect to syntax and other operating characteristics, so the volunteer community may not remember how to configure (or the nuances of) an older version. Therefore, should you go with anything pre 21.02, you should consider this unsupported and understand that you are on your own.

3 Likes
16

Well, if you ask a question in a public forum, be prepared to receive those answers others are willing to contribute. IMHO you should be:
a) thankful to everybody who chimed in a civil and polite tone
b) simply ignore the responses and lines of argument you consider tangential to your question.

If you would e.g. pay us to answer your questions you might be entitled to answers that are "on point".

Now i will volunteer a guess why you garnered so much headwind: your post was far from objective and friendly. If you post flame-bait, please do not be surprise if you get engaged responses, no?

Well, just as you seem to be concerned that security plays such a big role for others...

This forum tends to be one of the nicer and friendlier corners of the internet, not because we are inherently better/more civil/what ever, but mainly because we tend to get a grip on our selves and EDIT (even post-hoc) text we used that came out less friendly and civil than we did or should have intended. Your question falls under that category IMHO.

Again, this is because the requirements are not stable over time. Some years ago I did not care about ad blocking, but now I would not want to do without, so a realistic speedtest for me should now contain performance measurements with activated ad-blocker. And say, next year I decide I absolutely need WPA3 this again will change the test requirements (and likely the minimally required OpenWrt version as well).

Which sounds like a fools errand to me, because realistically you will need to repeat this test for all hardware ploatforms you are interested in.

Well, more of an opportunity for those interested in that question doing some experiments/tests and publishing the results. I expect that on that journey you might re-discover the performance hit when OpenWrt switched to its first kernel without the old Linux routing cache (which the kernel dropped because while it sped up a few things on small systems it was it was unfixably broken).

1 Like
17

Quite simply, if you've deployed 50,000 devices, you're a VAR. Linux doesn't get sued, but if you sold a package to a customer with known security deficiencies at the time of sale that get exploited, yes, you're in a difficult position legally. (In-box sales are quite another matter, but that's not your claim.)

It doesn't really matter if you take my word on anything, to be honest - you've aptly demonstrated by now that you're tilting at windmills and I'll leave you to that.

1 Like
18

We're currently under covid lockdown where I live (China) so I can't get to my devices at the moment. As soon as it's over and I've done some testing I'll present some numbers.

I'm not good at benchmarking OpenWRT by the standard I've seen others use, iperf/wan-lan etc. I'm just curious about the average throughput of a VPN tunnel so I'll present that.

19

The point being I'm not answering questions about security with my opinion on performance. My focus is on modems and I leave the security issues to those that are interested in it.

Which is realistic and if that is what is right for you then there is nothing wrong. But not everyone feels that way so presenting an opinion as the "Word of God" isn't. Flat out saying "v18 and v19 are evil and you will burn for using them" when the question is which is faster is counterproductive.

Remember, I'm not the one asking the question. I'm commenting on the replies.

Very likely but at this point there is no hard evidence one way or the other. We all know that as software develops it becomes bloated, eats more resources and gets slower. That seems to be how the software world works. OpenWrt is not exempt from this. But having more data on how performance has changed and what new features affect the performance will at least give a baseline to work from.

There may be some misunderstanding here. I haven't directly deployed that firmware. Instead I have sold custom firmware to companies that deploy it to their customers. I've also made factory firmware for 5 new routers from 3 different companies. No idea how many of those are out there.

The funny thing is, not one of those companies, reseller or manufacturer, has ever raised a concern about security. It seems a total nonissue to them. Until they are worried I'm not worried. And trust me, if they are worried I know about it.

And all of this is in addition to the Open Source ROOter project I run. That averages 10 to 15,000 downloads per month without any alarm about security. I think OpenWrt is one of the most secure firmware out there so problems tend not to arise.

20

My argument was that comparative testing and keeping relevant benchmarks for different OS versions get hard quickly if the is intended to adapt to changing usages, you will constantly have to reflash and re-run to be able to say that for a given usage patter OS version XY is currently fastest...

The same applies to comments as to questions, sometimes it is better to simply ignore than to engage...

Well then chalk this down as my prediction, thus was a pretty educated guess, since I still seem to remember quite similar discussions about performance and security of older versions when OpenWrt switched to its first inux kernel without that cache...

Yes and no, compared to the olden days the Linux kernel has grown considerably fatter (and slower under some conditions*), but it scales from low memory devices (like a 64 MB mips router) to hundreds or cores super computers. What I want to say, development over time has not been a constant decline, but over all we gained new relevant functions; sure we also introduced new bugs and regressions, but it is not that 10 year old OpenWrt releases offer the same capabilities as modern ones (which is on top of the now documented security issues with older releases).
Now, if you manage to compile reasonably up to date performance numbers over different OpwnWrt versions and different routers, nore power to you, I will happily look into such a database once it exists.

Yes, that in a nutshell is why the security situation is as dire as it is, once a device is sold we tend to let the manufacturers off the hook (I hope this is about to change with requiring security updates for longer periods after purchase), this is one of the fields where OpenSource OpenWrt actually shines IMHO, still supplying reasonably recent and bug-fixed OS versions for router hardware long since orphaned by its original seller.

Yes it is quite inconvenient that those that intend to or already do abuse security issues do not give proper notice to upstream projects about their ill intentions/actions...

Look I am not saying that consenting adults should not be allowed to run old software (or software of dubious security independent of age) if they so please and understand the trade-off they are making; but I am sure any description of doing so should come with appropriately large warnings so that nobody accidentally simply copies that approach without being aware of the potential consequences.

*) Sometimes the choices are between correctness and speed and fixes that improve correctness resilience occasionally have to sacrifice performance (think e.g. spectre counter measures)