You are trying to reach the webinterface of your router from your client (-computer, which means ca-bundle or ca-certificates installed on the router don't apply - those are only needed for the router to authenticate remote https resources). Your router's SSL certificate presented to your clients are generated at firstboot and self-signed, meaning no CA has signed them for you (how could they, without according configuration, a public domain, etc.) - in other words, you need to accept it (once) on your client(s).
Previous versions never defaulted to using HTTPS, it was a manual choice.
Is the cert exception being saved in your browser? Your browser might be setting a temporary exception instead of adding a permanent one. Look up instructions on how to add/verify self-signed certificates in your browser.
Firefox for example will remove any saved certificate exemptions if you clear your history and have site preferences checked. Same if you auto-clear your history on closing the browser, or use Private mode.
Per the release notes:
In addition, LuCI is now available over HTTPS in addition to HTTP. There is no automatic redirection to HTTPS on a fresh OpenWrt 21.02 installation; however, redirection will be enabled after upgrading from OpenWrt 19.07 to OpenWrt 21.02.
It is always possible to activate or deactivate the redirection to HTTPS like this:
uci set uhttpd.main.redirect_https=1 # 1 to enable redirect, 0 to disable redirect
uci commit uhttpd
service uhttpd reload