Walking around the forum, I saw a lot of users talking about HighLoad on their routers with no apparent cause. Specifically when AUTOREFRESH is ON.
I just answered this on an specific topic here, and I'm creating this new topic so we can discuss it technically here (and not focused on the other guy problem that I replied)
And to permit us to discuss pros and cons of my proposal and/or analysis with the developers.
The Problem description: OpenWRT system get slow, presenting a very High Load while Luci interface is open and with AUTOREFRESH enabled.
Luci interface makes heavy use of CGI-BIN LUA Scripts. They are necessary to do almost everything on the interface. Some of them exists to update things in realtime, like indicators and dynamic text changes (just to mention a few). And when the page is opened on your browser, scripts are being called from time to time [via AUTOREFRESH] to retrieve updated informations and display them on the browser
[No news, everybody here knows how it works].
But why this causes High Load? (sometimes almost freezing the router out of resources?)
OpenWRT uses an embedded HTTP Server, called uhttp, that is responsible to process LuaScripts that reside on Luci interface and when AUTOREFRESH calls them.
The root of the problem:
uhttp comes pre-configured to run a maximum of 3 scripts at the same time in parallel. And Luci pages have calls to various different scripts. They keep running all the time to update indicators and graphs on Luci interface [when it's opened in borwser] and those scripts are getting stucked while uhttp is trying to run 3 of them at the same time!
This is the origin of the HUGE LOAD that occurs while autorefresh is ON. It´s uhttp cgi-bin subprocess that is originating the LOAD while running various scripts at the same time.
The proposal fix:
To fix it, we have to change the way uhttp deals with scripts processing, by changing uhttp configuration to run scripts sequentially [only 1 script at a time]. This will drop the load below 0.10 [and you can have as many opened Luci windows as you want (and with autorefresh ON)] .
By changing this behavior all of the opened Luci windows will continue updating things in realtime, and the load will be no more a problem for the router. CPU will also be idle by 95%+ even with Autorefresh ON and having many browsers windows opened and updating things.
By running 1 script at a time makes them run sequentially, and sequentially they run in milliseconds, this eliminates the HighLoad, also eliminates all the LAGs on the interface, and there is no side-effects (I extensively tested it and debuged this behavioral change. Did not find any problem until now) .
Changing it results in benefits and optimization.
Change option max_requests value to 1.
option max_requests '1'
Save the file, reboot the router.
No more high load average and you get free CPU cicles and free CPU interruption, to run other process and programs faster.
That parameter line refers to to SCRIPT REQUESTS, not HTTP connections. It determines the quantity of CGI-BIN Lua scripts that can be executed simultaneously. (HTTP max connection is another parameter, and don't need to be changed)
Analysis from the Application point of view [and usage]
- There´s no technical reason [and no necessity] to run more than 1 lua script at the same time on the web interface.
- By running them sequentially, they run in milliseconds.
- Others Luci interface script-calls that exists on the webpage gets automatically queued to be processed by uhttp in sequence [so no lost information, no lost time, no lost attempts].
- Being queued makes they run instantly after the before-one has finished [and we are talking about milliseconds here]. This means the returned new information [returned by the script to the interface] is populated on the Luci interface instantly, in milliseconds.
- For us humans, milliseconds means realtime, so AUTOREFRESH stays realtime at our practical human experience and view.
- Luci interface is informative, I love it, but it must not [it should be prohibited to] cause a High Load on the router. Router has many more important things and jobs to do with it's resources and CPU cycles.
- We love to see realtime things in Luci Interface, we need to see it, and sometimes we need to keep the interface opened for hours. It is common and it should not prejudice the router itself.
- We cannot have the luxury to waste the router precious [and limited] resources with a stuck script via the interface.
- And again, there is no necessity/need [as I investigated] for those scripts to run in parallel. (please correct me if I missed something ou if I'm wrong here)
Analysis from the Router point of view [and health]
- This modification completely eliminates the High Load and High CPU usage / High SYS usage (that Luci scripts causes)
- This modification allow more resources to be used for other processes and jobs
- It stops Luci interface from slowing down the router
- a faster router, faster applications running, faster throughput. Because Luci interface no more causes High Load and no more slows down the router while opened.
- a faster Luci interface rendering, with pleasure interface navigation and comfortable usage, hundreds times faster, yes, hundreds (as I measured).
Final personal observations:
Luci interface seems to be the origin for HighLoad on various OpenWRT routers. I looked around this forum and found similar topics from people suffering with the same problem, some topics from a long time ago, with no solution. My intention and focus on this analysis was to understand the origin of the problem and fix it. Beside this intention, when I found the cause, I questioned myself: "Why? Why do we need to run the interface scripts in parallel?"
For sure there may exist [technically] something wrong happening on a layer below the CGI-BIN script layer [of uhttp] that causes the Load.
- But again, do we need to run 3 scripts in parallel on the interface? Why 3? Why this number 3?
- Probably because 3 is the default value of uhttp basic/original configuration?
- For Luci interface, is there any reason needed to parallel script execution [and I don't know]?
- The solution provided above (considering there is no need to parallel execution) fixes 100% of the problem. No more High Load. Isn't it better to solve this problem this way and have a huge performance/load problem solved?
I work for decades with data networks, and this is similar to a simple data congestion problem, so let's solve it the same way QoS do for data networks, the same way disks do to write data fiscally and concurrently: Let's queue it to be processed super fast and keep things running smoothly!
Luci is an amazing interface, really good, beautiful and well done interface. With this modification, it becomes fast and very light for the router execute it.
By solving this big performance problem that affects a high percentage of OpenWRT users, my suggestion is to change it in trunk config file, to ship it adjusted like this, so users can have a much better solid OpenWRT product.
Do you guys see any problem in my fix suggestion proposal?
I'm looking forward for your replies,