Poor performance of rpcd or ubus

For Developers
1

Hello everyone, I am developing a network traffic monitoring program called luci-app-bandix to graphically display the internet speed of devices. The entire program call process is

browser -> ubus -> luci -> rpcd -> curl http_my_backend_core -> json_parse -> return

(I'm not sure if the path order between ubus and rpcd is correct )

I found that the entire process took a lot of time on ubus.

I used time ubus call luci.bandix getStatus>/dev/null for testing, which took about 0.4 seconds, but curl http_my_backend_core only took 0.02 seconds.

It seems that most of the time was spent on ubus calls, and the data size of this getStatus is 5KB. I want to know if this is a normal process for me? Later on, getStatus may return more data, such as 500KB. Additionally, does ubus have a maximum data processing limit, such as 1MB? I have seen some code with size limits in the ubus repository.

My device is cudy tr3000 128MB, openwrt 24.10.2

2

This ultimately calls the function get_device_status() in your RPC script:

The performance may be and probably is affected by what that function is doing. If you want to isolate RPC performance impact from your own code performance impact, replace that function with something like this:

get_device_status() { return 0; }

and measure the time it takes to execute.

Generally the code in that function is doing lots of json stuff which may (?) be slow'ish, and in addition it does a lot of writing to a temp file - which definitely has impact on the performance, and seems completely unnecessary, since that file is later dumped into STDOUT and then deleted. So you could simply print to STDOUT to begin with.

3

Hello, thank you for your suggestion. I modified get_device_status to return 0; or use mock data, and then tested with time, which takes about 0.02s. Then, based on the original code, I removed the writing to temporary files and optimized some repetitive code logic. The execution time changed from 0.4s to 0.3s (mostly 0.25s). So I think this time consumption is due to JSON parsing. I used time my_rust_json_parse and it took 0.02s (rust's internal timing is even shorter, on the nanosecond level).

  1. I am using jshn.sh for JSON parsing. Is this currently the best method?

  2. Can ubus transmit more data? It seems that if it exceeds 1MB, the rpcd service crashes, making it impossible to log into the system. This issue is mentioned here: https://github.com/openwrt/luci/issues/7905

4

Clearly it is not the best method, but it is probably the best you can get without creating a custom json processing binary and without giving up on json formatting altogether.

Look closely for ways to get rid of subshells and binary calls, especially inside loops.

I have no idea but you can always resort to writing data to file and reading that file in your Javascript code. Just avoid multiple file writes: write all the data in one go.

5

Thank you, I understand.

6

Hi everyone, I've moved my JSON processing logic into Rust code.

Now the performance has improved significantly.

If you're also experiencing performance issues, you can try processing the data using your own binary instead of using the shell.

Calling commands like cat and awk in the shell are all independent processes, which can consume a lot of time.

1 Like
7

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.