CPU frequency scaling driver for mvebu (WRT3200ACM etc.)

I compiled also a 4.14 version without CPU frequency scaling, and that naturally works ok.

With the frequency scaling patch, the router's clock seems to lag by some 40-50%, which is quite much and clearly indicates that something is calculated wrongly (and/or does not get adjusted along the CPU frequency and the real-time clock ticks at half-pace 933MHz while it is read like it would be running at full 1866 MHz speed).

Comparing the kernel bootlogs of 4.9 and 4.14 reveals that there is something different in clock handling. Apparently some things have moved from 32bit to 64bit and clock resolution dropped from 40ns to 1ns?

kernel 4.9:

[    0.000000] Switching to timer-based delay loop, resolution 40ns
[    0.000003] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 85899345900ns
[    0.000008] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
[    0.000113] Calibrating local timer... 933.19MHz.
[    0.060030] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=250000)

kernel 4.14:

[    0.000005] sched_clock: 64 bits at 933MHz, resolution 1ns, wraps every 4398046511103ns
[    0.000015] clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0x1ae5b571769, max_idle_ns: 881590513431 ns
[    0.000026] Switching to timer-based delay loop, resolution 1ns
[    0.000125] Ignoring duplicate/late registration of read_current_timer delay
[    0.000131] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
[    0.000245] Calibrating delay loop (skipped), value calculated using timer frequency.. 1866.00 BogoMIPS (lpj=9330000)

Not sure how that affects the CPU frequency scaling, but is likely part of the challenge. And just highlights why it would be great to have the frequency scaling functionality upstreamed in Linux itself.

It is also possible that something relevant has changed in the kernel config and/or devicetree files.

I have no large motivation into looking to fix the things right now, as I am not really a clock specialist (and mvebu is not my main router). I will leave the patches there in case somebody wants to experiment further.

for me the frequency driver was a nice to have; but as it wasn't fully developed, incorperating cpu-idle and various power tables, it didn't do much towards power consumption. Therefor we can easily live without it. Again your efforts are much appreciated.

It's a pity there is no 802.11AC device that is fully working though. the WRT devices have this issue, the netgear devices keep having errors in the logs on the wifi connections. Am still hoping that there will be a "perfect" device eventually.

Maybe we get help from Marvell opensource kernel:

But the latest kernel version is 4.4.52

@hnyman what if you revert this commit?
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/?h=v4.14.27&id=ca0a05a98f3090a77b678eaa778a30febc5795b8

1 Like

@hnyman Reverting those commits and disabling those config symbols restores the k4.9 timer state, so it probably fixes your time issue when using freq scaling patchset.

Thanks for the suggestion. Seems that reverting those config symbols is enough to make the clock behave again with the cpufreq driver. (The changes done to master today were not enough)

EDIT:
The new version with that upstream commit reverted:
https://github.com/hnyman/openwrt/commit/b72413ad5c32b6cb1f5a214041b899d45d61c9c6

I have just briefly tested it, but at the first glance, the router again keeps the correct time with CPU frequency scaling enabled.

2 Likes

After a recent commit in upstream Linux kernel 4.14, the patch 805-... is unnecessary and breaks compilation. The upstream commit did the same change as 805
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/commit/drivers/clk/mvebu/armada-38x.c?h=linux-4.14.y&id=37d8947c0b073a27198ae477d8a8a8b6cda61abf

Does this mean i can use your patch again, without having to fiddle with any other patches?

Yes.
I haven't yet updates the commit in this thread, so it creates six patches 801-806, but after the most recent kernel 4.14. version bump, the patch 805 needs to be deleted.

I will update this commits here sooner or later, but the previous message was just a heads up that 805 is unnecessary.

WHOOOHHHH! CPU Scaling for the 3200acm. I'll definitely give this one a spin later.

Only scales frequency, not power.

I'm wondering if the chipsets / driver blobs in the Linksys WRTs just don't have any available/enabled power saving options? I took a few readings on my 1900ACS with the stock firmware and it seems to make no difference if it's been idling for 1 second or 10 minutes or if one or both of the radios are turned off... it idles at 8 watts. This is the same power consumption I see at idle under OpenWRT... so it's not any worse, just surprising that you can't move the needle on it. (I've never seen a system where you can cut the clock rate in half and/or turn off wireless radios and not see some reduction)

If I remember correctly, the power save cpuidle mode is disabled, as it does not work in 38x. That was the reason for "wrt1900ac v1 crashes" 1-2 years ago, as it was not yet disabled for 370 (where it failed, too).

upstream mentions the broken cpu idle for other routers of the series:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/arm/mach-mvebu/pmsu.c?h=linux-4.14.y#n494

	 * Currently the CPU idle support for Armada 38x is broken, as
	 * the CPU hotplug uses some of the CPU idle functions it is
	 * broken too, so let's disable it

wrt1900ac v1 was fixed by disabling cpu idle also for it with https://github.com/openwrt/openwrt/commit/5faa9556b1c10a4b21208615bddb7480401fa213

I noticed pretty much the same (based on CPU temps, like discussed above), so I have not pushed this to be included in the Openwrt sources. Apparently the benefit is rather minimal.

1 Like

What i’ve read is that the hardware itself apparently is broken and that is why the code for it was abandoned. The chips refused to come out of powersave or something similar. Has been a few years, so can’t recollect the fine details anymore.

Actually works quite good on wrt1200.
Frequency switching isn't lagging at all.
Thank you for this =)

But it could stay longer in the max frequency.

i tried to play around with up_threshold and set it to 25 (Yes that low) works a bit better.
And maybe also change sampling_down_factor to some way higher value so the cpu stays a bit longer in the higher clock.

//edit
https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt

On my device cpuinfo_transition_latency is 1000000 (/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_transition_latency)

So 1000000 * 750 / 1000 = 750000 us or 750 ms ?

Is that what you mean by lag?

Yes, CPU frequency scaling works ok, but has not much impact on CPU temps, as the CPU idle functionality is disabled.

Why is it disabled? Because it doesn't work? Can't we force it ?

[    0.004119] mvebu-pmsu: CPU idle is currently broken on Armada 38x: disabling

Any update on getting this working in the main builds or with a package?

My WRT3200ACM averages around 70deg c!

I tried to remove the checks from the driver, but that didn't work.
It results in a boot loop. (Atleast my WRT1200AC does)