Does unbound do recursive name resolution (default) or did you configure unbound to use forward servers?
The first resulotion can take some time,
as unbound has to ask all nameserves involved in the chain.
Also network latency plays are role here (where the target dns server is located).
But looking at the results...Exactly 1sec looks more like a time out happend there.
Cached dns name resolution should be quite fast.
Does the test script use random dns names?
//edit
Oops. I missed that you use forwarding.
What TLS forward servers does unbound use?
Maybe some providers throttle/block users when they exceed a certain query limit...
I monitor my RAM pretty often and there's always quite a bit left.. I even have a SWAP partition setup just in case it would run out.
I really don't think this is the issue as I've tried rebooting and I was getting the same results.
I use the paid service of Cleanbrowsing for my DNS. I forward to their custom TLS DNS. It's the only ZONE I have active at the moment.
My testscript I got uses the same servers to perform the tests. They are DNS servers of public DNS.
When my issue was occurring I did try switching to another provider such as 1.1.1.1 and 8.8.8.8 and I was getting the same results. I basically had to wait it out for it to start functioning properly again. The only setting I had changed was disabling DNSSEC which I don't know cause trigger such an issue.