Shell-scheduler: parallelization library for shell scripts

Hi all,

here is another side project of mine intended mainly for fellow shell coders. The title is fairly self-descriptive. This is a generalized version of the job scheduler I wrote for adblock-lean (and it's a part of adblock-lean code for a long time now). Now that I'm working on another adblock-lean feature that should benefit from parallelization, naturally I wanted to reuse the same code, and if I'm generalizing it already, I decided to package it as a separate library so other projects can benefit from it as well.

While the initial implementation was entirely coded by hand, I used a couple of clankers for this refactoring. That said, this is not vibe-coded. The clankers mainly helped me to find bugs, write tests, examples and the README. ~95% of the code is still written by hand and the other 5% went through a thorough review. The testing code and the README are not vibe-coded either: I held clanker's hand at every point and manually fixed what it messed up when writing the code and the text (which was a lot of work).

This library was specifically designed to work with Busybox ash but since Bash supports all required features, it works just as well on Bash.

If you use or want to use this library with your OpenWrt project, let me know. If at some point there are enough projects that integrate with it, perhaps it will make sense to make an apk package out of it and have it accepted into the official repo, so rather than every project keeping its own copy of parallelization code, we have a single, well-tested and reusable library.

Feature requests and/or bug reports are welcome as well.

2 Likes

Update:

  • fixed some bugs
  • added interface to easily and safely set and fetch job-specific parameters (README updated)
  • refactored the test suite for better readability
  • added many more tests (tests related to params setting and fetching are being worked on, should be available later today)
1 Like

Thanks Anton looks really useful :+1:

1 Like