Thanks, interestingly this also helps other multi-core arm routers along (in my case an mvebu turris omnia).
@facboy searching in the various mess of the L2 cache i found THIS
port to linux-4.19 and running normally
the link is broken but LUCKLY i found it
Will check it and compare them... As we are finally putting some work in kernel 4.19 to master i think we can finally implement this...
With original pr ported for 4.19 i got this result
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07640 MiB: 32.00000 Copy: 418.856 MiB/s
AVG Method: DUMB Elapsed: 0.46355 MiB: 32.00000 Copy: 69.032 MiB/s
AVG Method: MCBLOCK Elapsed: 0.08433 MiB: 32.00000 Copy: 379.459 MiB/s
Modified the patch with your changes and
here the report
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.05278 MiB: 32.00000 Copy: 606.316 MiB/s
AVG Method: DUMB Elapsed: 0.49566 MiB: 32.00000 Copy: 64.561 MiB/s
AVG Method: MCBLOCK Elapsed: 0.05122 MiB: 32.00000 Copy: 624.724 MiB/s
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.06195 MiB: 32.00000 Copy: 516.577 MiB/s
AVG Method: DUMB Elapsed: 0.51844 MiB: 32.00000 Copy: 61.724 MiB/s
AVG Method: MCBLOCK Elapsed: 0.05544 MiB: 32.00000 Copy: 577.208 MiB/s
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.06237 MiB: 32.00000 Copy: 513.046 MiB/s
AVG Method: DUMB Elapsed: 0.49267 MiB: 32.00000 Copy: 64.952 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04944 MiB: 32.00000 Copy: 647.219 MiB/s
Also can you tell me why the mutex lock ?
I added a check as i notice that it could happen that new_l2_volt is never updated (and cause problem with voltage regulator set)
if (new_l2_volt > 0) {
ret = regulator_set_voltage_tol(l2_regulator,new_l2_volt,tol);
if (ret) {
goto l2_reg_fail;
}
}
Did you test without the new L2 scaling code? I remember the patch slowing down the router (throughput with SQM and cake) by 30% in my tests?
actually no... can you test for me?
Also the slow down was caused by a bug in the patch... I tested with max frequency and it does keep high results
With 4.19? I cannot take the router out of service without putting my life at risk
Is that the IF statement in the post above?
No test it with 4.14 (the patchset hasn't changed) (I mean in stock build without this patch)
The bug was the transition directly from an old frequency to the new one... It looks like there is a bug and we first need to set idle frequency and then set the new frequency.
The if statement fix a problem with kernel 4.19
I will do that later today. Assuming there are no major performance difference between the kernels, the results should be transferable.
I will test in 19.07-SNAPSHOT that I am running.
The stock seems to be faster....
I have this unscientific feeling that the higher throughput is observed when running on CPU0 and slower when running on CPU1. Is there a way to pin a process to the CPU?
echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
echo performance > /sys/devices/system/cpu/cpufreq/policy1/scaling_governor
echo 800000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 800000 > /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq
sleep 1
echo 1750000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 1750000 > /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq
while [ true ]; do ./mbw 32 | grep AVG; done
AVG Method: MEMCPY Elapsed: 0.05188 MiB: 32.00000 Copy: 616.839 MiB/s
AVG Method: DUMB Elapsed: 0.18568 MiB: 32.00000 Copy: 172.335 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04571 MiB: 32.00000 Copy: 700.046 MiB/s
AVG Method: MEMCPY Elapsed: 0.04487 MiB: 32.00000 Copy: 713.208 MiB/s
AVG Method: DUMB Elapsed: 0.19181 MiB: 32.00000 Copy: 166.831 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04665 MiB: 32.00000 Copy: 686.021 MiB/s
AVG Method: MEMCPY Elapsed: 0.04886 MiB: 32.00000 Copy: 654.923 MiB/s
AVG Method: DUMB Elapsed: 0.18831 MiB: 32.00000 Copy: 169.937 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04642 MiB: 32.00000 Copy: 689.294 MiB/s
AVG Method: MEMCPY Elapsed: 0.05121 MiB: 32.00000 Copy: 624.866 MiB/s
AVG Method: DUMB Elapsed: 0.20197 MiB: 32.00000 Copy: 158.439 MiB/s
AVG Method: MCBLOCK Elapsed: 0.05017 MiB: 32.00000 Copy: 637.890 MiB/s
AVG Method: MEMCPY Elapsed: 0.05931 MiB: 32.00000 Copy: 539.583 MiB/s
AVG Method: DUMB Elapsed: 0.24462 MiB: 32.00000 Copy: 130.814 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04943 MiB: 32.00000 Copy: 647.329 MiB/s
AVG Method: MEMCPY Elapsed: 0.04622 MiB: 32.00000 Copy: 692.332 MiB/s
AVG Method: DUMB Elapsed: 0.19109 MiB: 32.00000 Copy: 167.461 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04570 MiB: 32.00000 Copy: 700.242 MiB/s
AVG Method: MEMCPY Elapsed: 0.04622 MiB: 32.00000 Copy: 692.380 MiB/s
AVG Method: DUMB Elapsed: 0.19142 MiB: 32.00000 Copy: 167.171 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04497 MiB: 32.00000 Copy: 711.622 MiB/s
AVG Method: MEMCPY Elapsed: 0.04687 MiB: 32.00000 Copy: 682.801 MiB/s
AVG Method: DUMB Elapsed: 0.20309 MiB: 32.00000 Copy: 157.567 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04563 MiB: 32.00000 Copy: 701.282 MiB/s
AVG Method: MEMCPY Elapsed: 0.04839 MiB: 32.00000 Copy: 661.266 MiB/s
AVG Method: DUMB Elapsed: 0.18894 MiB: 32.00000 Copy: 169.368 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04539 MiB: 32.00000 Copy: 704.998 MiB/s
AVG Method: MEMCPY Elapsed: 0.04668 MiB: 32.00000 Copy: 685.542 MiB/s
AVG Method: DUMB Elapsed: 0.18796 MiB: 32.00000 Copy: 170.251 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04534 MiB: 32.00000 Copy: 705.783 MiB/s
AVG Method: MEMCPY Elapsed: 0.04464 MiB: 32.00000 Copy: 716.785 MiB/s
AVG Method: DUMB Elapsed: 0.18786 MiB: 32.00000 Copy: 170.341 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04567 MiB: 32.00000 Copy: 700.604 MiB/s
was think the same thing re: pin>cpu...
master-4.14-stock w interest performance run
[root@syno-rt2600ac /mbw-l2-scalingtest 54°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04440 MiB: 32.00000 Copy: 720.708 MiB/s
AVG Method: DUMB Elapsed: 0.19026 MiB: 32.00000 Copy: 168.190 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04474 MiB: 32.00000 Copy: 715.312 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 55°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04445 MiB: 32.00000 Copy: 719.938 MiB/s
AVG Method: DUMB Elapsed: 0.19087 MiB: 32.00000 Copy: 167.658 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04481 MiB: 32.00000 Copy: 714.193 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./set_scaling_governor.sh ondemand
[root@syno-rt2600ac /mbw-l2-scalingtest 53°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04533 MiB: 32.00000 Copy: 705.989 MiB/s
AVG Method: DUMB Elapsed: 0.19149 MiB: 32.00000 Copy: 167.107 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04530 MiB: 32.00000 Copy: 706.339 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04511 MiB: 32.00000 Copy: 709.421 MiB/s
AVG Method: DUMB Elapsed: 0.19192 MiB: 32.00000 Copy: 166.738 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04521 MiB: 32.00000 Copy: 707.860 MiB/s
>>>>>>>>>>>>>
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./set_scaling_governor.sh performance
[root@syno-rt2600ac /mbw-l2-scalingtest 54°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07416 MiB: 32.00000 Copy: 431.526 MiB/s
AVG Method: DUMB Elapsed: 0.20964 MiB: 32.00000 Copy: 152.643 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07472 MiB: 32.00000 Copy: 428.245 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07404 MiB: 32.00000 Copy: 432.189 MiB/s
AVG Method: DUMB Elapsed: 0.20878 MiB: 32.00000 Copy: 153.269 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07582 MiB: 32.00000 Copy: 422.069 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07639 MiB: 32.00000 Copy: 418.882 MiB/s
AVG Method: DUMB Elapsed: 0.20901 MiB: 32.00000 Copy: 153.104 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07409 MiB: 32.00000 Copy: 431.881 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 55°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07407 MiB: 32.00000 Copy: 432.005 MiB/s
AVG Method: DUMB Elapsed: 0.20792 MiB: 32.00000 Copy: 153.903 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07433 MiB: 32.00000 Copy: 430.509 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 55°]# ./set_scaling_governor.sh performance
[root@syno-rt2600ac /mbw-l2-scalingtest 54°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07436 MiB: 32.00000 Copy: 430.331 MiB/s
AVG Method: DUMB Elapsed: 0.20926 MiB: 32.00000 Copy: 152.919 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07646 MiB: 32.00000 Copy: 418.498 MiB/s
>>>>>>>>>>>>>
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./set_scaling_governor.sh ondemand
[root@syno-rt2600ac /mbw-l2-scalingtest 55°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04608 MiB: 32.00000 Copy: 694.392 MiB/s
AVG Method: DUMB Elapsed: 0.19306 MiB: 32.00000 Copy: 165.747 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04627 MiB: 32.00000 Copy: 691.629 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.07214 MiB: 32.00000 Copy: 443.589 MiB/s
AVG Method: DUMB Elapsed: 0.21197 MiB: 32.00000 Copy: 150.961 MiB/s
AVG Method: MCBLOCK Elapsed: 0.07222 MiB: 32.00000 Copy: 443.085 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 56°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04543 MiB: 32.00000 Copy: 704.331 MiB/s
AVG Method: DUMB Elapsed: 0.19170 MiB: 32.00000 Copy: 166.925 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04542 MiB: 32.00000 Copy: 704.490 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./set_scaling_governor.sh performance
[root@syno-rt2600ac /mbw-l2-scalingtest 54°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04475 MiB: 32.00000 Copy: 715.007 MiB/s
AVG Method: DUMB Elapsed: 0.19062 MiB: 32.00000 Copy: 167.869 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04494 MiB: 32.00000 Copy: 712.062 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04942 MiB: 32.00000 Copy: 647.570 MiB/s
AVG Method: DUMB Elapsed: 0.19690 MiB: 32.00000 Copy: 162.517 MiB/s
AVG Method: MCBLOCK Elapsed: 0.05529 MiB: 32.00000 Copy: 578.731 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04502 MiB: 32.00000 Copy: 710.822 MiB/s
AVG Method: DUMB Elapsed: 0.19053 MiB: 32.00000 Copy: 167.954 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04597 MiB: 32.00000 Copy: 696.158 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04478 MiB: 32.00000 Copy: 714.665 MiB/s
AVG Method: DUMB Elapsed: 0.19011 MiB: 32.00000 Copy: 168.326 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04504 MiB: 32.00000 Copy: 710.416 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04487 MiB: 32.00000 Copy: 713.143 MiB/s
AVG Method: DUMB Elapsed: 0.18991 MiB: 32.00000 Copy: 168.499 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04521 MiB: 32.00000 Copy: 707.863 MiB/s
[root@syno-rt2600ac /mbw-l2-scalingtest 57°]# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.04484 MiB: 32.00000 Copy: 713.687 MiB/s
AVG Method: DUMB Elapsed: 0.19013 MiB: 32.00000 Copy: 168.304 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04521 MiB: 32.00000 Copy: 707.752 MiB/s
this results are with?
If it's stock then it's confirmed that there is a problem with scaling... it's strange that we have 400 mb with performance gov
Also take notice that my test are done with a softethervpn running... i notice some time avg of 700MB but the good part is that i never experienced 300-400 mb
Can you pls use the steps from my post just above yours to set performance governor? There were posts in the past that the frequencies are not properly set unless a specific sequence of steps is followed?
on my test one is with ondemand... the other is with min frequency scaling set to maxium
I am confused: the stock seems to be delivering better memory throughput (700MB/s vs 600MB/s). What is the problem then?
we observed a worse performance with higher cpu freq... with ondemand we don't see this behavior because the cpu never goes to the max freq (i think)
yup... fantom method wins
PERFORMANCE
/sys/kernel/debug/clk/krait_l2_pri_mux/clk_rate: 1200000000
/sys/kernel/debug/clk/krait_l2_sec_mux/clk_rate: 384000000
/sys/kernel/debug/clk/hfpll_l2/clk_rate: 1200000000
AVG Method: MEMCPY Elapsed: 0.04513 MiB: 32.00000 Copy: 709.060 MiB/s
AVG Method: DUMB Elapsed: 0.18997 MiB: 32.00000 Copy: 168.449 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04532 MiB: 32.00000 Copy: 706.056 MiB/s
ONDEMAND
/sys/kernel/debug/clk/krait_l2_pri_mux/clk_rate: 384000000
/sys/kernel/debug/clk/krait_l2_sec_mux/clk_rate: 384000000
/sys/kernel/debug/clk/hfpll_l2/clk_rate: 1200000000
AVG Method: MEMCPY Elapsed: 0.04555 MiB: 32.00000 Copy: 702.502 MiB/s
AVG Method: DUMB Elapsed: 0.19677 MiB: 32.00000 Copy: 162.625 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04584 MiB: 32.00000 Copy: 698.127 MiB/s
PERFORMANCE
/sys/kernel/debug/clk/krait_l2_pri_mux/clk_rate: 1200000000
/sys/kernel/debug/clk/krait_l2_sec_mux/clk_rate: 384000000
/sys/kernel/debug/clk/hfpll_l2/clk_rate: 1200000000
AVG Method: MEMCPY Elapsed: 0.05064 MiB: 32.00000 Copy: 631.883 MiB/s
AVG Method: DUMB Elapsed: 0.19421 MiB: 32.00000 Copy: 164.766 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04514 MiB: 32.00000 Copy: 708.868 MiB/s
ONDEMAND
/sys/kernel/debug/clk/krait_l2_pri_mux/clk_rate: 384000000
/sys/kernel/debug/clk/krait_l2_sec_mux/clk_rate: 384000000
/sys/kernel/debug/clk/hfpll_l2/clk_rate: 1200000000
AVG Method: MEMCPY Elapsed: 0.04563 MiB: 32.00000 Copy: 701.302 MiB/s
AVG Method: DUMB Elapsed: 0.19714 MiB: 32.00000 Copy: 162.324 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04584 MiB: 32.00000 Copy: 698.024 MiB/s
PERFORMANCE
/sys/kernel/debug/clk/krait_l2_pri_mux/clk_rate: 1200000000
/sys/kernel/debug/clk/krait_l2_sec_mux/clk_rate: 384000000
/sys/kernel/debug/clk/hfpll_l2/clk_rate: 1200000000
AVG Method: MEMCPY Elapsed: 0.04533 MiB: 32.00000 Copy: 705.965 MiB/s
AVG Method: DUMB Elapsed: 0.19095 MiB: 32.00000 Copy: 167.581 MiB/s
AVG Method: MCBLOCK Elapsed: 0.04799 MiB: 32.00000 Copy: 666.825 MiB/s
opp-printk-while-above-is-running
[21314.953185] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21314.953249] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21314.963616] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21314.967553] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21314.970918] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21314.983041] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21314.990812] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21314.998342] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21315.009072] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21315.013154] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.037043] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.037172] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.047148] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21315.051095] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21315.059935] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.066401] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.089053] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.089176] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.099109] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21315.103108] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21315.112403] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.112961] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21315.118515] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.134930] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21315.144036] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21315.148889] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21315.148980] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21315.158983] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 800000000 Hz
[21315.165735] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21315.172973] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.187854] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21315.192480] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21315.194371] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.208371] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21315.219249] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.222672] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21315.224959] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21315.233281] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21315.249093] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.256073] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.276965] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.277117] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.331056] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21315.331238] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21315.339123] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21315.345508] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.354987] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.360957] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21315.364944] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21315.376290] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21315.385040] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.391516] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.411138] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.411293] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.422094] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21315.422982] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21315.425150] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21315.439025] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21315.449023] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.455763] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21315.462943] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21315.477018] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21315.509328] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.509459] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21315.517818] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21315.523514] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21316.542486] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21316.542548] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21316.550420] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21316.556806] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21324.931050] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21324.931120] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21324.931243] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21324.939705] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21324.949090] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21324.955176] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21324.960855] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21324.968856] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21324.991089] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21324.991223] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21325.003296] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.006024] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.015052] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.021372] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.043245] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.043386] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21325.053293] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.057297] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.067068] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.072682] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.095223] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.095360] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21325.105239] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.109246] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.115039] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21325.119645] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.125089] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21325.133493] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.145042] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.155317] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.161027] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21325.171285] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.177045] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.180216] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.186128] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.196552] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.203977] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21325.216663] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.237065] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.237191] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.307152] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21325.307294] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21325.311135] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21325.319187] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21325.321302] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.339157] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.349120] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21325.352019] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21325.359108] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21325.360876] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21325.367133] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.385247] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.390638] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.391135] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.418152] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.418271] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.429174] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21325.432168] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.437146] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21325.447676] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21325.457207] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21325.462605] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21325.487296] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.487462] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21325.497236] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21325.499253] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.501364] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21325.509661] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21325.519160] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.531990] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.540356] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21325.547375] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21325.645839] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.645986] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.657947] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21325.659924] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.711151] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.711272] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.728269] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.728328] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21325.739714] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.742785] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.763254] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.763367] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.793263] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1000000000 Hz
[21325.793317] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21325.805171] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21325.807522] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21325.816413] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21325.822695] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.853263] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21325.854051] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21325.865678] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21325.867690] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.899346] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21325.899490] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.913281] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21325.913336] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21325.935105] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21325.935214] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21325.983342] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21325.983462] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21325.993805] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21325.998053] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21326.045397] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21326.045519] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21326.059576] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21326.059632] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21326.083165] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1000000000 Hz
[21326.083281] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21326.095986] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21326.097317] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21326.107216] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21326.113416] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21326.117168] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1000000000 Hz
[21326.130045] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21326.133136] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21326.141310] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21326.142961] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21326.151495] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21326.151916] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21326.151929] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21326.163343] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21326.173602] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21326.173618] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21326.188936] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21326.198684] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21326.206338] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21326.215176] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21326.221310] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21326.227342] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21326.234600] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21326.257538] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21326.264769] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21326.273020] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21326.279664] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21326.288055] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21326.294652] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21326.416146] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21326.416265] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21329.956654] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21329.956715] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21329.957225] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21329.991141] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21330.150218] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21330.150769] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21330.161503] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21330.164248] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21330.349167] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21330.349296] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21330.359171] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21330.363222] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21330.373205] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21330.378655] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21331.113469] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21331.113592] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21331.517240] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21331.517249] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21331.517263] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21331.517377] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21331.541192] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21331.546992] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21331.567222] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21331.567357] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21331.581287] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21331.581336] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21331.609396] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21331.609534] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21333.383232] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21333.383354] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.289314] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21334.289431] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.301458] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21334.303278] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.313222] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21334.318599] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.459342] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.459459] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.469799] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21334.473424] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.483267] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21334.489046] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.511363] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.511475] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.521834] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21334.526355] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.553312] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.553429] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.575628] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1000000000 Hz
[21334.575680] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21334.627749] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 800000000 Hz
[21334.627997] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.639687] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21334.643590] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.693304] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.693532] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.711495] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21334.711747] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.735324] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 800000000 Hz
[21334.735452] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.754358] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21334.754415] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.777791] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1000000000 Hz
[21334.777914] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21334.789668] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1400000000 Hz
[21334.792164] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.801597] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21334.807411] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.833701] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.833844] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.844525] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21334.847917] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.877335] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21334.877450] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21334.887937] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1725000000 Hz
[21334.891700] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21334.959604] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21334.959723] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21334.959920] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1000000000 Hz
[21334.977463] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21334.983267] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21334.983308] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21334.988893] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21334.997242] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21335.013459] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21335.021271] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.029470] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21335.034951] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21335.055309] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21335.055442] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21335.067317] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21335.069861] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21335.078039] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21335.079290] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21335.084890] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21335.101331] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21335.109262] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21335.115333] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.171501] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21335.171637] cpu cpu1: _set_opp_voltage: voltages (mV): 1025000 1025000 1025000
[21335.172259] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21335.186092] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21335.193603] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1000000000 Hz
[21335.194786] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21335.201548] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21335.210912] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.219369] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21335.231764] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21335.248182] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1400000000 Hz
[21335.248327] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21335.255886] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21335.263032] cpu cpu1: _set_opp_voltage: voltages (mV): 900000 900000 900000
[21335.271586] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21335.272223] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1000000000 Hz
[21335.277593] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.286189] cpu cpu1: _set_opp_voltage: voltages (mV): 950000 950000 950000
[21335.302901] cpu cpu1: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21335.308042] cpu cpu1: _set_opp_voltage: voltages (mV): 1100000 1100000 1100000
[21335.309064] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 1000000000 Hz
[21335.324544] cpu cpu0: _set_opp_voltage: voltages (mV): 902500 950000 997500
[21335.333460] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1000000000 Hz --> 1725000000 Hz
[21335.338373] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.361509] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21335.361664] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21335.371525] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21335.375622] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21335.384581] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21335.390952] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.413496] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
[21335.413641] cpu cpu0: _set_opp_voltage: voltages (mV): 855000 900000 945000
[21335.423550] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 800000000 Hz --> 1400000000 Hz
[21335.427620] cpu cpu0: _set_opp_voltage: voltages (mV): 973750 1025000 1076250
[21335.437363] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1400000000 Hz --> 1725000000 Hz
[21335.443815] cpu cpu0: _set_opp_voltage: voltages (mV): 1045000 1100000 1155000
[21335.465473] cpu cpu0: dev_pm_opp_set_rate: switching OPP: 1725000000 Hz --> 800000000 Hz
The original post about the scaling issues.
There is some frequency setting bug in ipq8065's drivers. If you change frequency from 1GHz-1.4GHz to max, you get a lot worse latency than when changing from 600-800MHz. I suspect L2 is not done right?
@Ansuel can you run your test in 4.19 using this way of switching to performace governor?
can you explain better how to test this?
Run these commands below and then run bwm the way you did.
echo performance > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
echo performance > /sys/devices/system/cpu/cpufreq/policy1/scaling_governor
echo 800000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 800000 > /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq
sleep 1
echo 1750000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo 1750000 > /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq
Before
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.05528 MiB: 32.00000 Copy: 578.877 MiB/s
AVG Method: DUMB Elapsed: 0.45141 MiB: 32.00000 Copy: 70.889 MiB/s
AVG Method: MCBLOCK Elapsed: 0.06016 MiB: 32.00000 Copy: 531.895 MiB/s
after
root@No-Lag-Router:/tmp# ./mbw 32 | grep AVG
AVG Method: MEMCPY Elapsed: 0.05853 MiB: 32.00000 Copy: 546.753 MiB/s
AVG Method: DUMB Elapsed: 0.39838 MiB: 32.00000 Copy: 80.326 MiB/s
AVG Method: MCBLOCK Elapsed: 0.06192 MiB: 32.00000 Copy: 516.767 MiB/s
I think we should find a way to trigger the error and check if it does show also with this patch...
Both 19.07 and master seem to faster by ~30% according to this metric...