TL-WR1043ND snapshot images - High download numbers - Spanish users needed


#21

Hmm.
And it has it been a rather stable load since then? Or is it still growing?
(the public history only shows the current year 2019, so it is hard to see if there has been growth in Sep/Oct/Nov/Dec/Jan.

If it is rather stable, it reduces the probability of gradual firmware updates by individual customers, or something like that, (causing a growing load along the growth the installed base of the new fiirmware with this test download site).

That being the last day of a quarter (or one day before the first of a new quarter), I have a guess:
some kind of SLA that has been automatically tested since 1.10.2018. E.g. automatic connection speed testing every few hours, run from customer data centers (or modems).
(or the remotely controlled test URL has been changed on 30.9.2018 to point to our site)


#22

Mind the different scaling...

grafik
grafik
grafik
grafik


#23

In my opinion it seems like some kind of local software system chosen by consumers (for instance a transportation app), that connects to the internet using which ever provider they are using. When those consumers travel to other places in Spain it will still connect, but in much lower numbers than in the prime area.


#24

So, it really started at one day, and has then stayed rather stable at 15-16k downloads per day. Strange.

Looks like somebody toggled a centrally managed test URL on that day.


#25

It shouldn't be that hard to gather ranges given the amount of data and feed those a dummy file instead, someone will eventually complain :wink:


#26

Madrid is the capital, and "the greater Madrid" is where most ISPs have their headquarters, what you are seeing are the default locations.

I'll try to contact Euskaltel and ask them.


#27



Localized variant of ^?

Guifi + Movi has muchomicro's.......

Top 2 ~ volume correlation... ( to 10 ) of this page look strange;
https://downloads.openwrt.org/stats/awstats.downloads2.lede-project.org.errors404.html


#28

I have just opened a thread on Euskaltel's forum: http://foro.euskaltel.com/Thread-Inexplicable-aumento-de-las-descargas-desde-Euskaltel-hacia-OpenWrt, will keep you informed.


#29

Rename target file/dir and redirect to warning page describing the problem.


#30

They removed all the links so it will be hard for them to find the info :confused:

Regarding the source I now noticed that they seem to be from fixed connections, not mobile. There are more sources than Euskatel, though, so I don't think it's specific to that ISP. From what I understand RIMA is Telefonica/Movistar and ONO is Vodafone fiber/ADSL.


#31

Numbers after todays updates (added cities to IPs with >1000 downloads, prefering those which can be found from September through December; approx. 50% of downloads with city now)

Barakaldo, Bilbao, Burgos, Castro Urdiales are the Top 4.

grafik

For comparison: numbers from yesterday evening: TL-WR1043ND snapshot images - High download numbers - Spanish users needed

Download size per netname (only relevant shown):
grafik


#32

Perhaps I interpreted the data wrongly: there is a heavy bias towards the Euskadi region (Madrid is there because of the reason I explained before, and the rest of the connections is noise), and that produces a bias towards the Euskaltel ISP, because Euskaltel has a notable presence in Euskadi, and not because the issue is related to Euskaltel at all.

So, there seems to be some software that was mass-installed on 30/09/2018 in the Euskadi region, across all providers, on devices running 24/7... each provider uses their own customized routers, I would discard a firmware update or a remote attack.

I do not see a correlation between the number of downloads and the working calendar in Euskadi.

I would like to see some results restricted to the Euskadi region: how many different IPs? does that match the number of schools in Euskadi, for example?


#33

I am also seeing that the number of downloads per city does not match the number of inhabitants, even in the Euskadi region... that seems weird, too.


#35

@tmomas Do you think you could share the raw data from the statistics (without compromising anybody's privacy, obviously)? I have worked in data-mining, and I think that I could get more info if I have access to the raw data and do my own analysis.


#36

Only now I see a weekly pattern: see graphics above

October: Peaks on 8 / 15 / 22 / 28 (28 being the odd man out) -> 7day period, with the peaks on Mondays
November: 5 / 12 / 19 / 26 -> 7day period, with the peaks on Mondays
December: 3 / 10 / 17 / 24 -> 7day period, with the peaks on Mondays, but much lower than in November.


#37

Further above I mentioned that 99,7% of the downloads come from Spain.
grafik

Let's take a look at the rest:
grafik

Oh, Romania is sticking out. Let's take an even closer look:
grafik

Where are those IPs located?
grafik

-> all are within a line of 75km length, so relatively close to each other.

My interpretation:

  • A single IP (A,B,C,H) each day, over long periods of time -> one single user
  • When there are two IPs per day, it might be the same user that hast just changed his (dynamic) IP

Now, Romania is quite a bit off from Spain (roughly 3000km apart), isn't it?

How comes a single user in Romania shows the same download behaviour as the spanish users (which are 99,7% of all download requests) 3000km away? What is the link between them?


#38

While pretty interesting, that unfortunately doesn't help a lot (unless it's a static, directly assigned IP with a clear text address (GDPR/ DSGVO) or 'hack-back' capability/ authority).

Reasons for this might include the obvious, someone having bought the device in spain and moved there (respectively ebay) - or the testbed of a hired (remote) consultant/ developer for a spanish ISP (or botnet admin).


#39

whatismyipaddress.com/ip/ thinks all those IPs are static. Not sure how reliable this information is...


#40

Further above I mentioned:

75% Wget; 25% uclient-fetch, both without version as "usual" user agents

  1. Does any distribution other than OpenWrt use uclient-fetch?
  2. Does any distribution other than OpenWrt use Wget or uclient-fetch without version in the user-agent string?
  3. If No: Can we assume that those devices are OpenWrt driven (and are not IPTVs or such stuff)?

#41

uclient-fetch has been developed by OpenWrt and is, in the grand scheme of things (relative to most vendor SDKs), rather new (old versions used busybox wget), that makes it rather likely that the culprit is based on LEDE/ OpenWrt - but... it's opensource, small, has advantages some (mostly) proprietary projects might appreciate as well.