Hey, so '\|^address=|!d;\|/#$|d;q' is sent to $rogue_element, if $rogue_element =/= zero then there is an issue. Looks quite tidy really.
I do want to tidy up the first check more eg what do you think about this? A lot like the second pass filter but opposite. It would pick up more than currently.
if start with address=(/ must follow)(alnum's must follow)(alnum's or .- must follow)(must end with /#)
Then second pass would just clean out random bits eg if there just happened to be a random text line.
Having said that, haven't seen a bad line in oisd yet. But checks are a must for running!
But I do appologise I might be a bit quiet for the next ~10 days. RL work (finance) is about to get pretty crazy unfortunately with a few things due.
Actually one isn't better than the other, it depends.
I have seen some cases where since 0.0.0.0 is an answer it continually tries to keep connecting to 0.0.0.0 like a server is down or unresponsive. Apple recommends using NXDOMAIN for their devices.
In other cases, NXDOMAIN will cause a retry or fall back like you mentioned with some Android devices.
It depends on the software & application. You can find answers for both ways.
Cool. I have it running all the time now and it updates every 5am just fine. I like that this service script solution: 'adblock-oisd' - inspired by this thread - has a tiny footprint at less than 3KB and I believe more sanity checks and safeguards than the heavier alternatives. And of course it can be improved further with time.
@Lynx I've been running your script for a week or so - thanks!
Also noticed that simple-adguard 22.03.2 package is now updated to include dnsmasq config (oisd dnsmasq) option. So can install direct from the software tab
Couple of thoughts see what you think:
Is there a way to get the file size using wget, before downloading the file? As apposed to downloading file, then checking size. I'm guessing currently using wget as it's included in openwrt?
New sed filter, could possibly use this as 1st pass only, no need for 2nd pass: -E '\~^address=/[[:alnum:]][[:alnum:].-]+/#$|^#|^\s*$~d'
Output any malformed 'address=/...../#' lines, random text except comment # lines, exclude whitespace/blank lines. Therefore is now a lot stricter, but keeping in mind that this is only designed to catch the .00000001% chance of a malformed oisd dnsmasq file. So the following would be caught, plus all others previously mentioned:
Buffalowings
address=/test.com/1.1.1.1
etc, etc
Eg Buffalowings could be any random text that isn't # commented out (which WILL cause dnsmasq to stop)
So if this strict test passes, then can just use the oisd.txt file exactly as is...
So I think your point is that if the first pass completes with your new sed then we don't need second pass. That sounds good.
But part of the original idea was to use first pass to check for any rogue elements and not continue of those exist.
I think it's useful to distinguish between rogue elements (stop with this dangerous file) or just typographical errors (keep going since it's just a silly error).
And that necessitates two checks - the first to see if rogue elements exist and then we stop if any exist.
The second to clear out any typographical errors.
You see if we conflate the two into one then we will stop on just a typographical error which seems to harsh.
What do you think? Would you be able to revise for first pass and second pass? I'm open minded.
Hope financial world was OK by the way. I had two patent hearings last week and am rather tired from those. I'm looking forward to some time off at Christmas.
Well, budgets for next 6 months complete! So huge relief, and yes looking forward to a little time off over Xmas. Glad we both made it through...
I'm getting the hang of this sed business:
for a two pass:
-E '\~^address=~!d;\~^address=/[[:alnum:]][[:alnum:].-]+/#$~d'
(Output anything that starts with 'address=', and is also malformed. So is stricter - but will catch absolutely everything)
-E '\~^address=/[[:alnum:]][[:alnum:].-]+/#$|^#|^\s*$~!d'
(Same as one-pass check, but using '!d' to delete all undesirables)
Either 1-step or 2-step will do the job. I can't find anything at all that gets past these now, so I'd be comfortable with either option...
So just to check since my sed reading ability is still limited - is the intention that 1st filter shows up any rogue elements to abandon ship and the 2nd filter is to ready the ship because we're definitely sailing?
All correct. This is just a refined 2-step, same as original plan The first step will catch any malformed address= (moreso than previous versions). And the second step is very similar, but will now also keep the comments/header, which is what I was aiming for. I like to have those in there, just in case one-day for troubleshooting etc.
Hiya, yes I think done.... for now. I've had a quick look over the code etc and high level all seems good. And it's running daily and updating etc. I wouldn't mind have a more thorough look over the next few weeks with some more spare time etc coming up.
Wonder if this could get turned into a package which is updatable etc?
The first step will catch any malformed address= (moreso than previous versions). And the second step is very similar, but will now also keep the comments/header, which is what I was aiming for. I like to have those in there, just in case one-day for troubleshooting etc.