Difficult (maybe impossible?) nowadays, because most sites use HTTPS so the router never gets to see the full URL, only the IP (and hostname if your router is the DNS server). If you want to block URLs you'll probably need something at the browser level.
I use this option in dnsmasq.conf to completely block youtube (NXDOMAIN is returned for the listed domains and their subdomains):
# Block Youtube
However, I think this requires two dnsmasq instances (one for kids' devices, one for everything else) if you want to apply this rule only to kids' devices. That's how I have it set up anyway - I have another dnsmasq instance running on a different AP, with the above rule in its dnsmasq.conf file, and I give the address of that DNS server to my kids devices, like this:
# Devices use the local DNS server (192.168.1.1) by default
# This uses OpenDNS upstream servers which are set up to filter porn, malware, gore sites etc.
# Kids devices use 192.168.1.2
dhcp-mac=set:kidsdevice,XX:XX:XX:XX:XX:XX # Kids device 1
dhcp-mac=set:kidsdevice,XX:XX:XX:XX:XX:XX # Kids device 2
dhcp-mac=set:kidsdevice,XX:XX:XX:XX:XX:XX # Kids device 3
# Devices with completely unrestricted DNS (22.214.171.124)
dhcp-mac=set:unrestrict,XX:XX:XX:XX:XX:XX # Unrestricted device 1
dhcp-mac=set:unrestrict,XX:XX:XX:XX:XX:XX # Unrestricted device 2
dhcp-mac=set:unrestrict,XX:XX:XX:XX:XX:XX # Unrestricted device 3
I also have a script that pulls a list of DoH servers from here and adds them to an ipset, running every night to keep the list up to date. I then have a firewall rule that blocks any traffic to those IPs, and another rule that redirects all forwarded traffic on port 53 to the local DNS server (except for the totally unrestricted devices).
I'm sure a time will come when the kids work out how to get around all of this, but it's good enough to stop them accidentally stumbling upon most bad stuff. I figure that once they're actively looking for porn it's going to be very difficult to stop them anyway.