Connecting to my ISP using PPPoE. OpenWRT is v4.9.152. Machine is RPi 3B+. Connection to ISP is fine, and setting wan6 ifname to eth0 gives me a global IPv6 address as well as IPv4. But when I change the ifname to @wan ('at' wan), I lose ppoe-wan and have no connection to my ISP, either v4 or v6. logread shows that I get as far as "pppd 2.4.7 started by root, uid 0" but no further - I don't get to "PPP session is 33737, Connected to 00:2a:10:c3:6f:53 via interface eth0", etc.
I want to use @wan because otherwise the DHCP6 Solicit messages go out on the eth0 interface rather than the ppoe-wan interface, and get ignored. My config/network is
@wan potentially conflicts with Linux naming conventions. Unless you’re intending the @ to have its Linux meaning, a different name may resolve your breakage.
root@lede:/# cat /etc/config/network
config interface wan
option ipv6 1 # only required for PPP-based protocols
...
config interface wan6
option ifname eth1 # use same ifname as in wan-section or "@wan"
option proto dhcpv6
I just tried changing wan to wan1 throughout, no difference
Second I don't understand what is your problem exactly here. With ifname eth0 it works fine. You have IPv6 on pppoe-wan and br-lan.
So what exactly is not working?
Thanks @trendy! Removing the gateway option seems to have fixed it.
With the iface set to eth0, the problem was that I was sending out ethernet packets, not PPoE packets. To give more context: I'm trying to do prefix delegation. My ISP tells me I have a /56, so I include the prefix request in the wan6 interface by option reqprefix 56. I run tcpdump into wireshark using ssh root@192.168.3.254 tcpdump -i eth0 -U -s0 -w - ip6 |sudo wireshark -k -i -
and can see the DHCP6 Solicitation ethernet packets do include the prefix delegation request. That part is working okay.
But I never saw router adverts from my ISP on the eth0 interface. That's because they are coming in on the ppoe-wan interface. They include no mention of a prefix, only the /64 address assigned to my router. I was sending out ethernet packets when I should be sending out PPPoE packets and my ISP was ignoring them. I saw this in Issue 13599 Problems with native IPv6 VDSL connection, where the solution is to set the wlan6 iface to @wan.
Now that @wan is working, I see DHCP6 Solicit XID packets from me on ppoe-wan with a prefix request and DHCP6 Advertise XID packets coming back from the ISP with a prefix assignment. Success - now on to the next part of distributing them
Sorry, it seems inconsistent behaviour fooled me into thinking the solution from @trendy worked. I wish it did!
It seems that openwrt is only occasionally sending out PADI PPPoE Discovery packets. Until it does, there is no PPPoE connection, and no pppoe-wan device. Once a PADI packet is sent out to the ISP's modem it initiiates a PADO/PADR/PADS session and everything is then okay: I get a pppoe-wan device with IPv4 and IPv6 addresses.
My environment now adds a hub between the router and the modem, and I am monitoring all ethernet packets using wireshark on a PC connected to the hub. Also, on the router I have added option pppd_options debug to the wan interface of /etc/config/network. If I don't see a PADI packet on wireshark, I restart the network using /etc/init.d/network restart. According to logread, openwrt sends many PADI packets
I go through this restart upto 10 times! Wireshark does not see most of those as ethernet packets. When it finally does see one, it also sees the PPoE session initiation as described above.
So where are the PADI packets that pppd is producing going to? pppd is version 2.4.7 and rp-pppoe.so is version 3.8p. OpenWRT is 18.06.2
Have you tried to change that to 1 ? (or manual in Luci)
You'll then have to create the wan6 interface yourself, rather than having it automatically created by openwrt. Maybe it will you more leverage on setting the options you want.
Thanks @trendy, yes I changed it back to one. There was a posting PPPoE addressing and prefix... which suggested auto which is why I tried it. my current network config is
Making progress - at identifying the problem, not found the solution yet!
Running tcpdump -i eth0 ether[0x0c:2] == 0x8863 on the router shows that a PADI packet is sent, but is immediately followed by
tcpdump: pcap_loop: The interface went down
No packet appears on the hub between the router and ISP modem. So I keep restarting the interface /etc/init.d/network restart. Eventually the interface stays up, and then a PADI packet comes out to the modem, the ISP responds with PADO, followed by PADR/PADS and then the pppoe-wan interface is created and everything is fine.
So the question is now, not "where are the PADI packets going to," but "why is the interface going down?"
Notes: each time the interface goes down, tcpdump terminates. So I run it in a shell loop
while true
do
tcpdump -i eth0 ether[0x0c:2] == 0x8863
done
The output is
...
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:15:43.524301 PPPoE PADI [Service-Name]
tcpdump: pcap_loop: The interface went down
1 packet captured
1 packet received by filter
0 packets dropped by kernel
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:15:49.814308 PPPoE PADI [Service-Name]
tcpdump: pcap_loop: The interface went down
1 packet captured
1 packet received by filter
0 packets dropped by kernel
tcpdump: eth0: That device is not up
tcpdump: eth0: That device is not up
tcpdump: eth0: That device is not up
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
09:15:52.514365 PPPoE PADI [Service-Name]
09:15:57.519739 PPPoE PADI [Service-Name]
09:15:57.523511 PPPoE PADO [Service-Name] [AC-Name "*********"] [AC-Cookie *******************]
09:15:57.523908 PPPoE PADR [Service-Name] [AC-Cookie **********************]
09:15:57.542984 PPPoE PADS [ses 0xee0c] [Service-Name] [AC-Cookie ***********************]
5 packets captured
5 packets received by filter
0 packets dropped by kernel
The cable has been okay so far. I changed it anyway for a new one, and no change to the behaviour. lastlog gives info that can be divided into 2 parts: when the packet fails to be delivered and the interface goes down, and when it does get delivered and the interface stays up. This part is common to both:
Wed Feb 27 09:14:21 2019 daemon.notice netifd: Interface 'wan' is enabled
Wed Feb 27 09:14:21 2019 daemon.notice netifd: Interface 'wan' is setting up now
Wed Feb 27 09:14:21 2019 kern.info kernel: [ 76.474986] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
Wed Feb 27 09:14:21 2019 daemon.err insmod: module is already loaded - slhc
Wed Feb 27 09:14:21 2019 daemon.err insmod: module is already loaded - ppp_generic
Wed Feb 27 09:14:21 2019 daemon.err insmod: module is already loaded - pppox
Wed Feb 27 09:14:21 2019 daemon.err insmod: module is already loaded - pppoe
Wed Feb 27 09:14:21 2019 kern.info kernel: [ 76.551284] device eth0 entered promiscuous mode
Wed Feb 27 09:14:21 2019 daemon.info pppd[1972]: Plugin rp-pppoe.so loaded.
Wed Feb 27 09:14:21 2019 daemon.info pppd[1972]: RP-PPPoE plugin version 3.8p compiled against pppd 2.4.7
Wed Feb 27 09:14:21 2019 daemon.notice pppd[1972]: pppd 2.4.7 started by root, uid 0
Wed Feb 27 09:14:21 2019 daemon.debug pppd[1972]: Send PPPOE Discovery V1T1 PADI session 0x0 length 4
Wed Feb 27 09:14:21 2019 daemon.debug pppd[1972]: dst ff:ff:ff:ff:ff:ff src b8:27:eb:10:52:a1
Wed Feb 27 09:14:21 2019 daemon.debug pppd[1972]: [service-name]
Wed Feb 27 09:14:22 2019 daemon.notice netifd: Network device 'eth0' link is down
Wed Feb 27 09:14:22 2019 daemon.notice netifd: Interface 'wan' has link connectivity loss
Wed Feb 27 09:14:22 2019 daemon.notice netifd: Network device 'eth0' link is up
Wed Feb 27 09:14:22 2019 daemon.notice netifd: Interface 'wan' has link connectivity
Wed Feb 27 09:14:22 2019 daemon.notice netifd: Interface 'wan' is setting up now
Wed Feb 27 09:14:22 2019 kern.info kernel: [ 77.984118] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xD1E1
Then in the first case, after about 5 seconds it goes down:
Wed Feb 27 09:14:27 2019 daemon.notice netifd: Interface 'wan' is now down
Wed Feb 27 09:14:27 2019 kern.info kernel: [ 82.596506] smsc95xx 1-1.1:1.0 eth0: hardware isn't capable of remote wakeup
Wed Feb 27 09:14:27 2019 daemon.notice netifd: Interface 'wan' is disabled
Wed Feb 27 09:14:27 2019 kern.info kernel: [ 82.615517] device eth0 left promiscuous mode
This behaviour repeats until it stays up and then attention (correctly) switches to setting up lan:
...
Wed Feb 27 09:14:35 2019 daemon.notice netifd: Interface 'wan' has link connectivity
Wed Feb 27 09:14:35 2019 daemon.notice netifd: Interface 'wan' is setting up now
Wed Feb 27 09:14:35 2019 kern.info kernel: [ 90.464081] smsc95xx 1-1.1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xD1E1
CHANGES FROM HERE
Wed Feb 27 09:14:37 2019 daemon.notice netifd: Interface 'lan' is now down
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.743819] br-lan: port 2(wlan0) entered disabled state
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.752349] br-lan: port 1(eth1) entered disabled state
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.769722] device eth1 left promiscuous mode
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.777304] br-lan: port 1(eth1) entered disabled state
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.795002] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.804015] device wlan0 left promiscuous mode
Wed Feb 27 09:14:37 2019 kern.info kernel: [ 92.811451] br-lan: port 2(wlan0) entered disabled state
Wed Feb 27 09:14:37 2019 daemon.info dnsmasq[1026]: read /etc/hosts - 4 addresses
Wed Feb 27 09:14:37 2019 daemon.info dnsmasq[1026]: read /tmp/hosts/odhcpd - 0 addresses
Wed Feb 27 09:14:37 2019 daemon.info dnsmasq[1026]: read /tmp/hosts/dhcp.cfg01411c - 2 addresses
ALL OKAY FROM HERE
I don't see anything wrong. Just to be safe before taking any further action, backup the system, test the backup file, try to install the firmware from scratch without saving the configuration, and restore one step at a time the configuration.
If it still gets unsolved, unless someone else has a better idea, I'd suggest to take it to developers for a potential bug.
Thanks @trendy you've been a tremendous help on this. I wouldn't have been able to get so far without your pointers.
It is starting to look like developer issue rather than a configuration issue. One other thing I have noticed so far: when the interface goes down, tcpdump almost immediately can pick it up again. But in the last instance, when the interface restarts and stays up, tcpdump has about 6 attempts to see the device which all fail, but then succeeds. Then the PADI message goes out and the interface stays up. Looks like the kernel taking time to get the interface set up properly - but that is speculation at the moment!
Update. I have two ethernet ports on the RP: the original one eth0, and another one on a USB-to-ethernet adapter as eth1. I have been using eth0 as my WAN connection, eth1 as my LAN. With that setup, I have been getting problems getting PPoE PADI packets out to the WAN. So, I just tried swapping them around: the USB ethernet adapter is now my WAN port, the builtin ethernet is now my LAN port. I've rebooted the RPi at least 6 times now, and each time pppoe-wan has come up immediately, and I can see the IA prefixes coming up okay.
Assuming this behaviour stays consistent, it more and more looks like a issue with the device drivers, perhaps. Yes, I know that on the RPi, ethernet hangs off the USB bus. So two USB-ethernet adapters, one apparently ok, one not. Hmmmm!
Try installing ethtool and see how the ethernet NIC reports to be configured. I had problems in the past with EEE (energy efficient ethernet) settings, some years ago, so probably not your issue but worth looking at as it should be simple
Thanks @moeller0. Installed ethtools and generally got: no link detected, speed 10Mbps, half-duplex, repeated until a connection stayed up and then switched to link detected, 100Mbps, full duplex.
So then I changed to another RPi. Same SSD card, same cables, same configuration. Basically the same behaviour, but with wrinkles.
a) Sometimes the link is alive, and then the interface goes down, with zero packets sent.
b) Sometimes the link is alive and PADI/PADO/PADR get sent and received and then the link goes down
c) Something in between such as PADI gets sent, link goes down.
Looking for reports on RPi ethernet drivers (smsc95xx), I'm not seeing many complimentary posts, but none reporting this kind of behaviour that I can see. I would guess at driver, h/w issues, except that when the PPoE link gets set up, it stays up.
