OpenLDAP: custom build with overlays

I'm trying to make a custom package of openldap-server with memberof overlay enabled to make it easier to use for user authentication.

Build fails for Rewrite/Remap overlay which is required for most other overlays with:

make[7]: Entering directory '/build/openwrt/build_dir/target-mipsel_24kc_musl/openldap-2.6.6/servers/slapd/overlays'
ld -r -o rwm_x.o rwm.o rwmconf.o rwmdn.o rwmmap.o
ld: rwm.o: relocations in generic ELF (EM: 8)
ld: rwm.o: relocations in generic ELF (EM: 8)
ld: rwm.o: relocations in generic ELF (EM: 8)
ld: rwm.o: error adding symbols: file in wrong format

OpenLDAP builds fine without overlays. Any help would be appreciated.