Let's take "Xiaomi Redmi Router AX6000 (OpenWrt U-Boot layout)" as an example, it not only provides the sysupgrade and recovery image, it also has preloader and uboot image.
What is the preloader image for?
Will the sysupgrade update the uboot and preloader image when a new version of openwrt is released?
If not, can I update them manually and how?
I don't see how the page can answer my questions. What can I tell is the preloader and uboot should be flashed to BL2 and FIP partition respectively.
they're required for the ubootmod, the preloader is probably replaced, so you can update u-boot.
and no, you usually don't update uboot and preloader when you update openwrt.
if you want details, I suggest you try the device specific thread Add OpenWrt support for Xiaomi "Redmi AX6000"