This seems to anticipate end devices set up to use proxy and hence all the certificate jazz that renders this technique too fussy to use in practice - I don’t want to go configuring all my end devices including smart TVs etc.
But isn’t there a way to intercept YouTube.com traffic in a manner that doesn’t require any configuration of end devices whatsoever? So end device doesn’t even know data is passed through proxy?
So closer to a true man in the middle attack in that everything is as seamless as possible?
In practice, proxies are a dead technology (https, hsts and hardcoded certificate pinning, hello google) for almost[0] all uses, but especially filtering. Yes, it is true that you 'can' still work around that with custom wildcard certs, but at that point you can (and should) just do the filtering on your hosts, in your browser (which is going to be easier and more sensible than importing custom certs, configuring the proxy, etc. pp.).
I realize that enterprise environments are still insisting on this, as they can manage their fleet that way (and force their man-in-the-middle wildcard certificate down their users' throats), but for 'normal' users with typical consumer devices (which will often not allow access to the cert pool, at least not easily) that is much less convenient compared to local filtering, while at the same time being a huge security risk (you do need to manage your wildcard certificates extremely well, as those hold the keys to all of your traffic, including banking, credit cards, etc.)
--
[0] until very recently, I used squid to cache Debian packages (plain http sources), with some attention to details that (can still-) works well, if you need the same files more often (needs much less storage space than a (partial-) mirror, with almost all the benefits). I only stopped that, because my WAN speed is now on par (or faster-) than that of the (spinning 2.5") HDD in the server providing the proxy services, and while upgrading that to an SSD is on the todo list, it's neither that high on the list, nor that likely to help much in isolation (an Atom j1900 board has other bottlenecks as well).
There is no magic way to decrypt traffic and remove ads without MITM in this scenario. You must install certificate to achieve that task. Luckily it is a common practice in corporate environment and devices advertised for that market are expected to support custom certs installation.
I would like to better understand this as it seems I’m missing something. Why can’t the router pass the traffic from end device through proxy then return traffic to end device without the end device having been configured to use a proxy? The router has the decrypted data so why can’t it just pass it back whether in some reconstructed way or not. I can’t see why that wouldn’t be possible one way or another.
Is this because of the desire to maintain an https connection between end device and router? Isn’t it possible to make that unencrypted and reconstruct the https data sent through proxy in unencrypted form and send it back to client device? I’m not understanding why it’s not possible to prevent any client configuration.
I found this post from @dlakelan but on Reddit discussing this topic:
The need to maintain https from the device to the router (which in this case acts as a proxy or a transparent proxy) is there because the client wants to connect to the original site using https and cannot be taught any other way.
How so? Surely client knows nothing about how the data is dealt with at the router. Why can’t the router intercept YouTube traffic, do what needs to be done over https, then return in unencrypted form?
The client only knows how to request and read encrypted data. For the Android app, there is simply no HTTP support at all. For the browser, the links are generated by heavily obfuscated JavaScript (so we can't really modify it), and it says "use https".
OK I think I’m getting there in terms of my slow understanding here thanks to the posts here. Router could be arranged to redirect traffic to YouTube.com to random.com such that when user visits YouTube.com they instead see content from random.com. So why can’t the interception still work despite this configuration relating to JavaScript at least for the browser case (I get the app may be more problematic).
How about this. I remember configuring ssh ages ago such that I could browse from my computer in my college from the engineering department. Couldn’t this be pushed over a browser somehow such that rather than opening browser session sent over ssh the same X session(?) is sent over browser protocol?
No, you misunderstood again. To issue a redirect, you still need to allow the initial connection to succeed, which is HTTPS, so you need a certificate. Therefore, by redirecting to random.com instead of filtering YouTube traffic, you gain nothing.
I still have a VM image (not OpenWrt) with a similar setup but for anti-virus purposes (that were mandated by CyberEssentials in the UK). Would it be acceptable if I spin up a VM for you to connect to and see how it works?
I’ll take you up but later perhaps as I’m about to get ready to go out.
My point about the redirection is that I don’t understand if YouTube.com can be redirected to something completely random then why can’t it be redirected to some sort of interface that passes through the filtered YouTube.com traffic.
Regarding initial connection must succeed I am not sure what to make of that. Say with the case when at uni I would fire up in console X over ssh session and browse as if from my college room computer. Couldn’t something like that be used but instead the X session is passed to a website that is the interface to which YouTube.com is redirected?
