Ahh, the solution is even easier than I thought.
I decided to delete my own /mnt directory to see the behaviour-
root@TonyOpenWrt:~# cd /
root@TonyOpenWrt:/# rmdir mnt
root@TonyOpenWrt:/# mkdir mnt
mkdir: can't create directory 'mnt': I/O error
Now the overlay that creates/appends the writeable partition to the read-only root filesystem is here, notice the mnt folder is marked in some way to "hide" it in the main filesystem and it is this character device (shows as purple in my putty terminal session) that is blocking the ability to recreate the directory again.
root@TonyOpenWrt:/# ls -al /overlay/upper/
drwxr-xr-x 8 root root 608 Nov 28 18:03 .
drwxr-xr-x 4 root root 360 Nov 4 06:25 ..
drwxr-xr-x 19 root root 2632 Nov 4 06:25 etc
drwxr-xr-x 4 root root 288 Nov 4 06:25 lib
c--------- 1 root root 0, 0 Nov 28 18:03 mnt
drwxr-xr-x 4 root root 352 Nov 19 17:24 root
drwxr-xr-x 2 root root 288 Nov 27 02:07 sbin
drwxr-xr-x 7 root root 480 Nov 4 06:25 usr
drwxr-xr-x 3 root root 232 Nov 4 06:25 www
So simply delete the "character" file/node and remake that mnt directory-
root@TonyOpenWrt:/overlay/upper# rm mnt
root@TonyOpenWrt:/overlay/upper# ls
etc lib root sbin usr www
root@TonyOpenWrt:/overlay/upper# mkdir mnt
root@TonyOpenWrt:/overlay/upper# ls -alstr
0 drwxr-xr-x 3 root root 232 Nov 4 06:25 www
0 drwxr-xr-x 7 root root 480 Nov 4 06:25 usr
0 drwxr-xr-x 4 root root 288 Nov 4 06:25 lib
0 drwxr-xr-x 19 root root 2632 Nov 4 06:25 etc
0 drwxr-xr-x 4 root root 360 Nov 4 06:25 ..
0 drwxr-xr-x 4 root root 352 Nov 19 17:24 root
0 drwxr-xr-x 2 root root 288 Nov 27 02:07 sbin
0 drwxr-xr-x 2 root root 160 Nov 28 18:05 mnt
0 drwxr-xr-x 9 root root 608 Nov 28 18:05 .
And bingo, directory is back-
root@TonyOpenWrt:/overlay/upper# cd /
root@TonyOpenWrt:/# ls -alstr
0 dr-xr-xr-x 105 root root 0 Jan 1 1970 proc
0 dr-xr-xr-x 11 root root 0 Jan 1 1970 sys
0 drwxr-xr-x 1 root root 232 Nov 4 06:25 www
0 lrwxrwxrwx 1 root root 3 Nov 4 06:25 var -> tmp
snip
0 drwxr-xr-x 4 root root 360 Nov 4 06:25 overlay
0 drwxr-xr-x 1 root root 352 Nov 19 17:24 root
0 drwxr-xr-x 1 root root 288 Nov 27 02:07 sbin
0 drwxr-xr-x 6 root root 1880 Nov 28 15:16 dev
0 drwxrwxrwt 23 root root 680 Nov 28 18:04 tmp
0 drwxr-xr-x 1 root root 160 Nov 28 18:05 mnt
So the "bug" is that any directory that is in the root (read only) filesystem, if deleted will create a character file to highlight not to show it, however, will throw an IO error if attempting to rebuild again, solution is to rebuild it in the /overlay/upper area.
Ahh, an even easier way to restore it if it exists in the readonly filesystem only and not necessarily in the writeable upper area is to delete the mnt character file in /overlay/upper, and then
mount -o remount /
and the /mnt directory comes back. This quirk/bug only seems to only impact directories that were in the original read only root filesystem. If I create a directory in "/" I can delete and re-create no issues at all. I guess this means that it is very important to only have directories that are necessary 100% of the time in the rootfs build, any superfulous ones that may be purged from time to time (e.g. pki directory following the openVPN guide to reset & rebuild certs etc) should probably not sit in the readonly image.
I suppose I never saw this behaviour deleting files (rather than directories) from the readonly filesystem because they are also marked with a character file pointer in the /overlay/upper area, but when recreating them, there is no IO issues overwriting a character file with real file, but overwriting a character file with directory of the same name gives the IO error.
Yep-
root@TonyOpenWrt:/# cd /overlay/upper/
root@TonyOpenWrt:/overlay/upper# ls
etc lib mnt root sbin usr www
root@TonyOpenWrt:/overlay/upper# mknod tonytest c 0 0
root@TonyOpenWrt:/overlay/upper# ls -alstr
snip
0 drwxr-xr-x 2 root root 160 Nov 28 18:48 mnt
0 crw-r--r-- 1 root root 0, 0 Nov 28 19:15 tonytest
0 drwxr-xr-x 9 root root 680 Nov 28 19:15 .
root@TonyOpenWrt:/overlay/upper# mkdir /tonytest
mkdir: can't create directory '/tonytest': I/O error
root@TonyOpenWrt:/overlay/upper# touch /tonytest
root@TonyOpenWrt:/overlay/upper# ls -alstr
snip
0 drwxr-xr-x 2 root root 160 Nov 28 18:48 mnt
0 -rw-r--r-- 1 root root 0 Nov 28 19:17 tonytest
0 drwxr-xr-x 9 root root 680 Nov 28 19:17 .
This to me seems like a bug that can be fixed, just need to allow a directory to overwrite a 0,0 node character file of same name if it belongs to upper overlay?