MultiWAN and yt-dlp

I occasionally download with yt-dlp. I've recently learned that it includes the feature:

  • Multi-threaded fragment downloads: Download multiple fragments of m3u8/mpd videos in parallel. Use --concurrent-fragments (-N) option to set the number of threads used

Seems useful with a MultiWAN configuration. While it does download multiple fragments, it seems to stick with one ISP for all of them regardless of the number I choose. I did try forcing all downloads to IPv4 (it's an ISP issue I have), and that didn't help.

Is this a Windows limitation, a yt-dlp thing, or is there something I'm missing?

Could you perhaps track how many connections are opened by yt-dlp to the server you're downloading from, with and without this option?
My guess is there is still the same number of TCP connections opened, and this option just enables multiple simultaneous downloads through that unique pipe. If that's the case, it'd be expected for MultiWAN to have no effect.

How do you check the connections?

Not really an answer but...

  • Try launching multiple downloads to use all the bandwidth?: Can the 1 computer use all the bandwdth to roll over to the other wan? yt-dlp usually averages 30mbps-175mbps. What is your expected multiwan bandwidth/load demand?

  • Try using the yt-dlp "datebefore/dateafter" args to break up bigger channels? (or filesize args) More unique instances may get better distributed across the available wan ifaces.

  • Make unique mac's VM's as more than 4-5 connections from one mac may get refused.

  • Use multiple vpn routers? :white_check_mark:

  • Become a yt premium member(s).

With aria2 you can download videos super fast using yt-dlp, on a single connection.

yt-dlp --external-downloader aria2c --external-downloader-args '-c -j 3 -x 3 -s 3 -k 1M' URL

I found videos to download in minutes vs over an hour.

1 Like

Can you tell me what the aria2 arguments do?

-c - Continue downloading a partially downloaded file. Use this option to resume a download started by a web browser or another program which downloads files sequentially from the beginning. Currently this option is only applicable to HTTP(S)/FTP downloads.

-j 3 - maximum number of parallel downloads (in this case 3)

-x 3 - maximum connections per server (also 3 in this case)

-s 3 - Download a file using 3 connections

-k 1M - minimum split size (1 MB in this case)

Thanks. I already have aria2, but rarely use it. I'll give it a go and see if that solves the problem.

Edit: Doesn't seem to. Changed all the '3' in the args to '4's, but still stuck on one ISP.
Btw, arguments need double quotes, not single.

Edit 2: So I tried adding '--disable-ipv6 true' to aria2c, even though I have -4 on yt-dlp... no change.
Started a download and everything was still on wana. Disabled wana and the download continued uninterrupted on wanb. Re-enabled wana and everything stayed on wanb.

Weird.

What is the config for your multiwan? That would control how the connections are balanced between interfaces.

Got sick of this issues. Rebooted Netgear that had OpenWrt, the Linksys running Fresh Tomato as a dumb access point, and the modems...
And now everything works just fine. Weird, but happy now.

1 Like

I tried the -n that helped with the bigger file downloads. :white_check_mark:

But big channels with lots of smaller files were still slow.

So then I tried to test how to download lots and lots of smaller files on a single yt channel more rapidly:

Split a download in 2 chronologically at the midpoint (with datebefore and dateafter args) and got it roughly twice as fast on one router.

However yt-dlp cannot immediately ascertain the midpoint - it loads each file sequentially for date comparison which takes around ~1.5s per file on the before arg. The after arg started downloading right away. -n worked on big files but didn't seem to help as much on smaller files. Smaller files could never spool up to hit the max download speed.

Sample arguments:

yt-dlp -f b -n 10 --download-archive downloaded.txt --datebefore 20170601 https://www.youtube.com/@Gazzz696/videos

=1,946 items, totalling 22.5 GB = 4 hours


yt-dlp -f b --download-archive downloaded.txt -n 10 --dateafter 20170531 https://www.youtube.com/@Gazzz696/videos

= 3,899 items, totalling 62.3 GB = 3 hours

Total time elapsed around 4 hours. Before it would have been 7-8h. Test performed on a 240mbps VPN/ linksys EA8300.

***still haven't tried aria2 because yt-dlp is just so damn good for my use case and limited testing experience.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.