How to handle large script size?

For Developers
1

Hi all, this is another question about adjusting my project to OpenWRT.

The project was initially written in Bash, then eventually I converted it to POSIX-compliant code. There is still no Luci interface, just the shell code. The total size of all essential scripts is currently about 170KB. Out of that, about 80KB is taken by scripts which are used for installation, uninstallation and post-installation configuration in case the user wants to adjust settings. The core functionality scripts which run after installation take the additional 90KB. That size is expected to grow as I'm planning to add support for nftables (currently the project only supports iptables).

So I'd like some input on what methods are preferable to reduce the size.

One thing that I could trim down is code comments. There are a lot of them in the code. Probably around 15-20% of the total size. I don't really want to do that since this will make the code less readable for someone who wants to understand what it's doing, and harder to maintain for me. If there is no choice, I suppose I could go for it.

There is also some code used for debugging embedded in the scripts which is not needed for the project to function but makes it much easier to maintain and develop. It takes probably 2-3% of the size. Not sure if removing it makes sense, at the expense of increased maintenance burden.

A third thing is code which processes command-line arguments and outputs a nice help message when the arguments don't make sense. Almost all scripts have that. That takes probably around 5-7% of the size.

Another thing that is possible to remove is error checking logic, which is a huge part of the code, I'd say at least 50%. Looking at some other projects around, I notice that they don't implement much error checking. In my opinion, this reduces the reliability of the code, more so because it's shell code, and again makes it more difficult to maintain. Does it make sense to strip it down?

Theoretically, I could also merge some scripts which should reduce the size of arguments handling code and error checking code without compromising reliability and user-friendliness. The downside would be that this would make the code less modular and more rigid, and the merged scripts will probably become quite long and thus harder to read. Currently the project is comprised of 14 scripts, 1 of them won't be used for OpenWRT and 2 are library scripts which won't benefit from merging. The additional 11 scripts are basically modules which call each other as needed.

Your input will be appreciated.

2

It's hard to give advice about a code I cannot see...

1 Like
3

Thank you for your interest. Here's the link. This is not the POSIX version yet as that branch is still local on my machine. But fundamentally the code is about the same.

https://github.com/blunderful-scripts/geoblocker-bash

4

If its a single file, you could gzip it, then execute with
zcat myscript.gz | sh

Mutliple files, you could tar.gzip it, then execute with
tar -zxf myproject.tgz myscript.sh -O | sh

edit1: if you need another shell, then replace sh with ash or bash

edit2: quick look at the code, start using more functions and read up on case statements

This:

case "$action_run" in
	add) action_apply="add" ;;
	remove) action_apply="remove" ;;
	update) action_apply="add" ;;
	apply) action_apply="add" ;;
	"") usage; die "Specify action in the 1st argument!" ;;
	*) usage; err1="Error: Unsupported action: '$action_run'."; err2="Specify action in the 1st argument!"; die "$err1" "$err2" ;;
esac

Can be this:

case "$action_run" in
	add|update|apply) action_apply="add" ;;
	remove) action_apply="remove" ;;
	"") usage; die "Specify action in the 1st argument!" ;;
	*) usage; err1="Error: Unsupported action: '$action_run'."; err2="Specify action in the 1st argument!"; die "$err1" "$err2" ;;
esac
3 Likes
5

Thank you for your suggestions. You are right, some bits of that code can certainly be condensed. My shell coding skills have been improving since I started working on this project. Although I regularly go back and apply better style to older code, some older bits (like this one) still stick around.

Regarding functions, if the idea is to reduce code duplication then I am of course keeping this in mind. Most of the functions which can be re-used across multiple scripts reside in the *-common script. Otherwise, most scripts have internal functions as well where this helps to avoid code duplication. Perhaps there is some more code which can be converted into functions to reduce duplication but honestly I don't think it's a lot.

As to gzipping files, I've been considering this, but I can't imagine how to implement this without introducing a lag when a user wants to call a script. Perhaps extracting all scripts to /temp the first time any of them gets called and then letting them sit there till reboot? Does it sound reasonable?

6

Np, we've all been there.

If you use multiple scripts, you can create a file for just for functions. Then reference to it with your script. I think its called sourcing. Bash uses "source" but basic shell scripting uses the dot "."
(OpenWRT scripting is a perfect example, just look at their functions file.)

Example:

myfunctions.file

example(){ echo this is an example;}

example.sh

!#/bin/sh
. myfunctions.file
example

Good luck :+1:

3 Likes
7

That's a good advice but as mentioned above, my project already implements this exactly, that's what the -common script is for - it's sourced in most other scripts.

source "$script_dir/${suite_name}-common" || { err="$me: Error: Can't source ${suite_name}-common."; echo "$err" >&2; \
	[[ ! "$nolog" ]] && logger "$err"; exit 1; }

(The POSIX-compliant version uses the . syntax which you referred to)

8

Yeah, I saw but the main takeaway was looking at the OpenWRT scripts. Those are really really well written, and their functions are short and to the point.

9

Ah cool, I'll take a look at them indeed.