Use x86_64 since you have a 64 bit CPU. Use a "combined" image, see below. The choice of ext4 or squashfs is how the file systems are organized, either one will work. The choice of EFI or not depends on whether your virtual BIOS supports EFI booting.
OpenWrt does not have an installer like most Linux OS. It is distributed as a direct disk image. When uncompressed with gzip and written block by block (raw) to a virtual disk, there will be a partition table and two partitions. Boot this disk as a legacy OS.
Maybe this news is coming too late (I just had some little time to play with it last night), but for someone interested to do it on your Synology you can check this out:
I am using Synology DS1621+, with Mellanox ConnectX-3 314 dual 10G SFP+ port (those China manufactured one, officially no such model), tested with OpenWrt 23.05.2 x86-64 image.
First thing, the 2nd comment about using OpenWrt with Docker, yes it works, I didn't tried with Docker but LXC, they are similar thing, however no matter it's docker or LXC, you should be using "rootFS" only to boot (I tested with R6S + DietPi + Proxmox 7).
For Synology DSM, the x86-64 combined image is a DISK IMAGE, not CD-ROM ISO, of course it won't boot when you try to fake the system as CD-ROM. You only need to IMPORT DISK, with the combined image, I tested with non-EFI image with "Legacy BIOS" option under Synology VMM and it works well for me.