MediaTek and VLAN 802.1ad on ethernet

Dear all,
I have a MediaTek-based router (YouHua WR1200JS) and I'm having problems with VLAN 802.1ad on cabled connections.
Basically, I set up an interface with VLAN 802.1ad (instead, 802.1q goes through surprisingly well) on the router on top of eth0.1 (eth0-1.17) and on eth0.2 (eth0-2_17), then I did the same on my laptop enp0s25 and tried to get ping going through these interfaces, and it didn't get through.
For the original report, see the LibreMesh development mailing list here.

Here goes what I tested and how to reproduce.

I just flashed OpenWrt snapshot, stopped and disabled the firewall, and installed tcpdump-full both on a MediaTek-based YouHua WR1200JS and on an Atheros-based TP-Link WDR3600 (for checking that in this case everything works fine).

I created VLAN 802.1ad interfaces on top of the "yellow ports" eth0.1 and on the "blue port" eth0.2:

openwrt# ip link add link eth0.1 name eth0-1_17 type vlan proto 802.1ad id 17; ip link set eth0-1_17 up; ip address add 10.2.1.1/24 dev eth0-1_17
openwrt# ip link add link eth0.2 name eth0-2_17 type vlan proto 802.1ad id 17; ip link set eth0-2_17 up; ip address add 10.3.1.1/24 dev eth0-2_17

And on the laptop:

laptop# ip link add link enp0s25 name enp0s25.17 type vlan proto 802.1ad id 17; ip link set enp0s25.17 up; ip address add 10.2.1.2/24 dev enp0s25.17; ip address add 10.3.1.2/24 dev enp0s25.17

On the MediaTek router, the ping does not work neither on eth0-1_17 nor on eth0-2_17 in neither of the directions.

When I use tcpdump on the router or Wireshark on the laptop, I paste here the first two ping packets pinging the laptop from the router (ping 10.2.1.2), as captured on various interfaces (at the same time, they are the same two packets):

  • On the router, on eth0-1_17:
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 45 00   T.uz..._%.~...E.
0010   00 54 46 ef 40 00 40 01 dd b3 0a 02 01 01 0a 02   .TF.@.@.........
0020   01 02 08 00 3a bd f9 06 00 00 e9 04 db 36 00 00   ....:........6..
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00                                             ..

Second packet:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 45 00   T.uz..._%.~...E.
0010   00 54 47 41 40 00 40 01 dd 61 0a 02 01 01 0a 02   .TGA@.@..a......
0020   01 02 08 00 92 65 f9 06 00 01 82 5b ea 36 00 00   .....e.....[.6..
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00                                             ..

Both the first and the second ping packets are ok.

  • On the router, on eth0.1:
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 45 00 00 54 46 ef 40 00 40 01 dd b3 0a 02   ..E..TF.@.@.....
0020   01 01 0a 02 01 02 08 00 3a bd f9 06 00 00 e9 04   ........:.......
0030   db 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .6..............
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00                                 ......

Second packet:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 45 00 00 54 47 41 40 00 40 01 dd 61 0a 02   ..E..TGA@.@..a..
0020   01 01 0a 02 01 02 08 00 92 65 f9 06 00 01 82 5b   .........e.....[
0030   ea 36 00 00 00 00 00 00 00 00 00 00 00 00 00 00   .6..............
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00                                 ......

Both the first and the second ping are correctly tagged with VLAN 802.1ad.

  • On the router, on eth0:
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 81 00 00 01   T.uz..._%.~.....
0010   88 a8 00 11 08 00 45 00 00 54 46 ef 40 00 40 01   ......E..TF.@.@.
0020   dd b3 0a 02 01 01 0a 02 01 02 08 00 3a bd f9 06   ............:...
0030   00 00 e9 04 db 36 00 00 00 00 00 00 00 00 00 00   .....6..........
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00 00 00 00 00                     ..........

Second packet:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 81 00 00 01   T.uz..._%.~.....
0010   88 a8 00 11 08 00 45 00 00 54 47 41 40 00 40 01   ......E..TGA@.@.
0020   dd 61 0a 02 01 01 0a 02 01 02 08 00 92 65 f9 06   .a...........e..
0030   00 01 82 5b ea 36 00 00 00 00 00 00 00 00 00 00   ...[.6..........
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00 00 00 00 00                     ..........

Both the first and the second ping are correctly tagged both with VLAN 802.1q and VLAN 802.1ad.

  • On the laptop, on enp0s25:
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 08 00 3a bd f9 06 00 00 e9 04 dd b3 00 00   ....:...........
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00                                 ......

Just after the definition of the VLAN 802.1ad (0xE to 0x11), which is ok, the IP header information is broken.

Second packet:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 08 00 92 65 f9 06 00 01 82 5b dd 61 00 00   .....e.....[.a..
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 63 65 04 5f 74 63 70 05 6c 6f 63 61 6c 00   ..ce._tcp.local.
0060   00 10 80 01 00 00                                 ......

From the second ping packet on, additionally to be broken the IP header, at the end of the packet there is some leak from maybe some cache in the switch?
In this case it can be read, more often it does not have a string representation.

  • On the laptop, on enp0s25.17:
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   3a bd f9 06 00 00 e9 04 dd b3 00 00 00 00 00 00   :...............
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00                                             ..

Here the VLAN 802.1ad header has been removed, but the IP header is still broken.

Second packet:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   92 65 f9 06 00 01 82 5b dd 61 00 00 00 00 00 00   .e.....[.a......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 63 65   ..............ce
0050   04 5f 74 63 70 05 6c 6f 63 61 6c 00 00 10 80 01   ._tcp.local.....
0060   00 00                                             ..

And in the second packet we can still see the weird content leaked from somewhere (usually not a string, but just weird hex content).


When pinging the router from my laptop (ping 10.2.1.1), I receive broken ping replies.

On the router, the reply being sent looks good when captured on any interface, here I report only the packet as captured on eth0.1:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 45 00 00 54 9e ec 00 00 40 01 c5 b6 0a 02   ..E..T....@.....
0020   01 01 0a 02 01 02 00 00 56 42 2e 87 00 1f 14 12   ........VB......
0030   4c 5d 00 00 00 00 51 d5 0a 00 00 00 00 00 10 11   L]....Q.........
0040   12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21   .............. !
0050   22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31   "#$%&'()*+,-./01
0060   32 33 34 35 36 37                                 234567

And this is how the same packet reaches my laptop on enp0s25, broken:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 00 00 56 42 2e 87 00 1f 14 12 c5 b6 00 00   ....VB..........
0020   00 00 51 d5 0a 00 00 00 00 00 10 11 12 13 14 15   ..Q.............
0030   16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25   .......... !"#$%
0040   26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35   &'()*+,-./012345
0050   36 37 9e b9 26 c3 af 0a a2 86 d6 da 13 18 e5 13   67..&...........
0060   09 66 fc b6 54 68                                 .f..Th

If I connect on the "blue port" (ping 10.3.1.2) the ping does not get through neither in any direction.

So, when connecting to the blue WAN port and pinging my laptop from the router (ping 10.3.1.2), the packets reach my laptop with the broken header as when connecting to yellow LAN ports but the weird content at the end of the packet is not always present.

As captured on my laptop on enp0s25:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ad 88 a8 00 11   T.uz..._%.~.....
0010   08 00 08 00 99 fa 3c 07 00 3e 1b a2 e7 07 00 00   ......<..>......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0060   00 00 00 00 00 00                                 ......

When pinging the router on the WAN port from my laptop (ping 10.3.1.1), I receive broken ping replies on the laptop.

Also in this case, the packets captured on the router looks good, both the incoming request and the outcoming reply.

As captured on enp0s25 on my laptop:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ad 88 a8 00 11   T.uz..._%.~.....
0010   08 00 00 00 2d 6f 29 61 00 54 b8 0e d0 ab 00 00   ....-o)a.T......
0020   00 00 dc 9c 09 00 00 00 00 00 10 11 12 13 14 15   ................
0030   16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25   .......... !"#$%
0040   26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35   &'()*+,-./012345
0050   36 37 00 00 00 00 00 00 00 00 00 00 00 00 00 00   67..............
0060   00 00 00 00 00 00                                 ......

If I do the same with the TP-Link WDR3600 router, the ping just works.

Any idea of what could be going on?
Can anyone reproduce on other MediaTek routers?
Thanks,
Ilario

1 Like

For helping understanding where the leaked content comes from (seems from the router, maybe a cache in the switch or directly from RAM?) I paste a few more packets where the content contains a string:

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 88 a8 00 11   T.uz..._%.~.....
0010   08 00 08 00 28 c7 75 07 00 16 c7 be dd a3 00 00   ....(.u.........
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0050   00 00 72 69 6f 40 69 67 6c 61 70 74 6f 70 09 5f   ..rio@iglaptop._
0060   70 72 65 73 65 6e                                 presen

where ilario@iglaptop is my username and hostname.

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   37 88 a5 06 00 0f 5f 4e 73 6a 00 00 00 00 00 00   7....._Nsj......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 34 2d   ..............4-
0050   73 68 61 32 35 36 2c 64 69 66 66 69 65 2d 68 65   sha256,diffie-he
0060   6c 6c                                             ll
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   bb 1d a5 06 00 10 cc b7 73 60 00 00 00 00 00 00   ........s`......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 63 6f   ..............co
0050   6d 2c 75 6d 61 63 2d 36 34 40 6f 70 65 6e 73 73   m,umac-64@openss
0060   68 2e                                             h.
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   71 b3 a5 06 00 11 06 21 73 34 00 00 00 00 00 00   q......!s4......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 68 2e   ..............h.
0050   63 6f 6d 2c 68 6d 61 63 2d 73 68 61 32 2d 32 35   com,hmac-sha2-25
0060   36 2c                                             6,
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ac 08 00 08 00   T.uz..._%.~.....
0010   54 12 a5 06 00 7a d1 52 5f 6b 00 00 00 00 00 00   T....z.R_k......
0020   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ................
0040   00 00 00 00 00 00 00 00 00 00 00 00 00 00 cb d9   ................
0050   39 bc 60 96 3f d3 64 6e 73 2d 73 64 04 5f 75 64   9.`.?.dns-sd._ud
0060   70 05                                             p.
0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ad 08 00 00 00   T.uz..._%.~.....
0010   f4 23 2c 17 00 02 8e 10 7c f9 00 00 00 00 43 82   .#,.....|.....C.
0020   03 00 00 00 00 00 10 11 12 13 14 15 16 17 18 19   ................
0030   1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29   ...... !"#$%&'()
0040   2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 66 f6   *+,-./01234567f.
0050   7c 51 ac c8 55 d9 2c 2d 2e 2f 30 31 32 33 34 35   |Q..U.,-./012345
0060   36 37                                             67

here part of the packet repeats.

0000   54 ee 75 7a c2 1f d4 5f 25 eb 7e ad 08 00 00 00   T.uz..._%.~.....
0010   96 11 2c 17 00 07 93 10 7b f4 00 00 00 00 99 8f   ..,.....{.......
0020   06 00 00 00 00 00 10 11 12 13 14 15 16 17 18 19   ................
0030   1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29   ...... !"#$%&'()
0040   2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 65 73   *+,-./01234567es
0050   65 6e 63 65 04 5f 74 63 70 05 6c 6f 63 61 6c 00   ence._tcp.local.
0060   00 0c                                             ..
0030   aa aa 64 68 63 70 20 31 2e 32 38 2e 34 0c 07 4f ..dhcp 1.28.4..O
0040   70 65 6e 57 72 74                               penWrt

of this one I don't have the full content but here appears 1.28.4 which is the Busybox version.