[RETRACTED] compressed-memory: this project is officially retracted, take note about it's future

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

Hi,

While zram writeback and zstd requires recent kernel, zram as a kmod was introduced in 3.1x.

I do understand that it's intended for NAS and generally more powerful devices. What I'm saying is that it has potential to benifit a far wider range of users. Maybe you could split it into two, one for general purpose and another for NAS. That's just a suggestion.

I'm glad you got good lz4hc results. I actually never used lz4hc because according to lz4's own github page, hc is like 12.5x slower than zstd but with actually lower compression ratio.

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

Yours looks much more polished than the existing one. Thanks for the effort.

Hey, I think you should make a pull request! You'll get way more attention that way and your code deserves that. Looks great. Would be awesome to have an option to choose compression algorithm though. I think your tests demonstrate quite well that lz4/lzo is much better?

median speed/ratio
zstd: 2/2.566
lzo: 6.9/2.061
lz4: 6.3/1.9
lz4hc: 2.2/2.07

max speed/ratio
zstd: 5.8/10.937
lzo: 27.7/9.373
lz4: 24.1/9.503
lz4hc: 6/10.476

I'd rather use lz4 or lzo based on this (and that was my opinion before looking at your data too), no idea why you would ever want zstd and lz4hc for zram and zswap. Those two should never be the default for anything (as you suggest in the OP), it's a massive speed drop for a marginal increase in compression ratio. lzo and lz4 are on average 3x faster with 0.2x lower ratio, in the best case scenario they are about 4x faster and still 0.1x lower ratio. Your tests that seem to match the benchmark on the zstd github and your real world screenshots as well - although it's kind of meaningless screenshots.

I get the impression you don't work with statistics a lot from your excel sheets, normally you wouldn't use that many percentiles and just stick to 3 or 4 :slight_smile: Percentiles aren't particularly useful here either, it would show how the distribution of the data is with regards to it's compressibility. Median is definitely the right thing to look for when comparing these results. Variance would be nice too, along with the median. I've approximated it by taking the max results too.

Your point that lzma compression is a terrible idea is absolutely right, everything should basically be lz4. I wonder if lzma is still used for some reason, maybe it's common to have hardware support. Otherwise I see no reason to keep it based on this https://catchchallenger.first-world.info/wiki/Quick_Benchmark:_Gzip_vs_Bzip2_vs_LZMA_vs_XZ_vs_LZ4_vs_LZO.

But your code is cool! Make a pull request!

2 Likes

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

2 Likes

Quick reply! Yeah I've been reading through your code and eating my hat. It does everything I want it to. I still don't get the logic of your percentile system, but who cares if it works.

Hey, you updated the OP again. Why did you remove the patches? I'd appreciate it if you put up a new link as I'm not running your specific system :slight_smile: Had to get them from your edit, still applies cleanly.

I think the most correct way to do the patch and make it available for everyone is like this: https://openwrt.org/docs/guide-developer/feeds

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

uhh okay. I don't see the point of what you're doing. you're upset at people discussing your patch with you on github, so you make the patches inaccessible for other people? seems like a silly way to get revenge. very dramatic.

oversimplifying the efforts of a developer and practical reasons for decommissioning code to serve your interests?

i've tested this code... so i've time invested in it too... dissappointing to see more practicable means forward were not facilitated for whatever reason... it was quality work that touched areas deeper in the os / life cycle... thus drawing high levels of scrutiny/obstacle...

he wrote it...he can remove it... simple...

[RETRACTED] This post is officially retracted. Please take note that compressed-memory and luci-app-compressed-memory will be actively maintained, but I will not longer provide them to OpenWrt.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.