Understanding ipv6 and How to Route It on AT&T Fiber

Hello, I'm trying to get ipv6 working the way I want on my network. My ISP, AT&T Fiber, gives me 8 /64 prefixes. I have a linux server that hosts containers through docker and I'm trying to do 2 things:

  1. Have a publicly routable ipv6 address assigned to my host server that is used for all ipv6 traffic, including outgoing. Right now, the address for outgoing uses the address my WAN is given for all LAN devices instead of unique addresses for each LAN device. The WAN address seems automatically assigned by AT&T and there doesn't seem to be a way to change it. It also has a different prefix than the prefix I specifically request.
  2. Have each docker container on my linux server use a publicly routable ipv6 address. Not really sure how to go about this one, maybe ip6table rules? I need the address the container is given specifically to be publicly routable so I don't know if that will work.

I think you didn't mention at all where is the OpenWrt in this setup and how is it configured.

Please run the following commands (copy-paste the whole block) and paste the output here, using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have

ubus call system board; \
uci export network; \
uci export dhcp; uci export firewall; \
ip -6 addr ; ip -6 ro li tab all ; ip -6 ru
1 Like

My (wired only) router is openwrt. I'm not super familiar and mostly stick to only luci so I didn't run the commands, but I can run them now:

ubus call system board
{
        "kernel": "5.4.188",
        "hostname": "gg",
        "system": "ARMv8 Processor rev 3",
        "model": "Raspberry Pi 4 Model B Rev 1.2",
        "board_name": "raspberrypi,4-model-b",
        "release": {
                "distribution": "OpenWrt",
                "version": "21.02.3",
                "revision": "r16554-1d4dea6d4f",
                "target": "bcm27xx/bcm2711",
                "description": "OpenWrt 21.02.3 r16554-1d4dea6d4f"
        }
}
uci export network

package network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'ddc9:e21f:b2f3::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1'
        list ports 'eth2'
        list ports 'eth3'

config interface 'lan'
        option proto 'static'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option device 'br-lan'
        option ipaddr '192.168.2.1'
        option ip6ifaceid 'eui64'

config interface 'wan'
        option device 'eth0'
        option proto 'dhcp'

config interface 'wan6'
        option proto 'dhcpv6'
        option reqaddress 'none'
        option device 'eth0'
        option reqprefix '64'
uci export dhcp; uci export firewall

package dhcp

config dnsmasq
        option domainneeded '1'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option localservice '1'
        option ednspacket_max '1232'
        option noresolv '1'
        list server '0::1#5453'
        list server '127.0.0.1#5453'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv4 'server'
        option ra 'server'
        list ra_flags 'none'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'
        list ra_flags 'none'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

config host
        option name 'server'
        option ip '192.168.2.199'
        option mac 'REDACTED'

config domain
        option ip '192.168.2.1'
        option name 'router.extd'

config domain
        option ip '192.168.2.199'
        option name 'server.extd'

config host
        option name 'server'
        option dns '1'
        option duid 'REDACTED'
        option hostid '337'

config host
        option name 'pikvm'
        option ip '192.168.2.171'
        option mac 'REDACTED'

config dhcp 'wan6'
        option interface 'wan6'
        option ignore '1'
        option master '1'
        list ra_flags 'none'

package firewall

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone 'lan'
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'lan'

config zone 'wan'
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option masq6 '1'
        option masq6_privacy '1'
        list network 'wan'
        list network 'wan6'

config forwarding 'lan_wan'
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config rule
        option name 'Support-UDP-Traceroute'
        option src 'wan'
        option dest_port '33434:33689'
        option proto 'udp'
        option family 'ipv4'
        option target 'REJECT'
        option enabled '0'

config include
        option path '/etc/firewall.user'

config rule 'wg'
        option name 'Allow-WireGuard'
        option src 'wan'
        option dest_port '51820'
        option proto 'udp'
        option target 'ACCEPT'

config include 'nat6'
        option path '/etc/firewall.nat6'
        option reload '1'

config redirect
        option target 'DNAT'
        option name 'Bittorrent'
        option src 'wan'
        option dest 'lan'
        list proto 'tcp'
        list proto 'udp'
        option dest_ip '192.168.2.199'
        option src_dport '51413-51418'
        option dest_port '51413-51418'

config rule
        option name 'test'
        option family 'ipv6'
        option target 'ACCEPT'
        option src 'wan'
        option dest 'lan'
        option dest_port '51417'
        list dest_ip '{}::337'
ip -6 addr ; ip -6 ro li tab all ; ip -6 ru

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2600:1700:XXXX:2c00:aaaa:bbbb:cccc:dddd/64 scope global dynamic noprefixroute
       valid_lft 3118sec preferred_lft 3118sec
    inet6 2600:1700:XXXX:2c00:eeee:ffff:gggg:hhhh/64 scope global secondary dynamic //This is the public ipv6 address my devices get
       valid_lft 2670sec preferred_lft 2670sec
    inet6 fe80::1111:2222:3333:4444/64 scope link
       valid_lft forever preferred_lft forever
11: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2600:1700:XXXX:2c0f:{eui64}/64 scope global dynamic noprefixroute
       valid_lft 2729sec preferred_lft 2729sec
    inet6 ddc9:e21f:b2f3:0:{eui64}/60 scope global noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::{eui64}/64 scope link
       valid_lft forever preferred_lft forever
23: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 500
    inet6 2a05:dfc7:45:babe::1000/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::834b:32c3:304b:cbdf/64 scope link flags 800
       valid_lft forever preferred_lft forever
30: ifb4eth0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 32
    inet6 fe80::{REDACTED}/64 scope link
       valid_lft forever preferred_lft forever
default from 2600:1700:XXXX:2c00::/64 via fe80::abcd:efgh:hijk:lmn dev eth0  metric 512
default from 2600:1700:XXXX:2c0f::/64 via fe80::abcd:efgh:hijk:lmn dev eth0  metric 512
2600:1700:XXXX:2c00::/60 from 2600:1700:XXXX:2c00::/64 via fe80::abcd:efgh:hijk:lmn dev eth0  metric 384
2600:1700:XXXX:2c00::/60 from 2600:1700:XXXX:2c0f::/64 via fe80::abcd:efgh:hijk:lmn dev eth0  metric 384
2600:1700:XXXX:2c00:{REDACTED} dev br-lan  metric 1024
2600:1700:XXXX:2c00:{REDACTED} dev br-lan  metric 1024
2600:1700:XXXX:2c00:{REDACTED} dev br-lan  metric 1024
2600:1700:XXXX:2c00:qwer:rrrr:iiii:ffff dev br-lan  metric 1024
2600:1700:XXXX:2c00:{REDACTED} dev br-lan  metric 1024
2600:1700:XXXX:2c00:{REDACTED} dev br-lan  metric 1024
2600:1700:XXXX:2c00::/64 dev eth0  metric 256
unreachable 2600:1700:XXXX:2c00::/64 dev lo  metric 2147483647
2600:1700:XXXX:2c0f::/64 dev br-lan  metric 1024
unreachable 2600:1700:XXXX:2c0f::/64 dev lo  metric 2147483647
2a05:dfc7:45:babe::/64 dev tun0  metric 256
ddc9:e21f:b2f3:0:qwer:rrrr:iiii:ffff dev br-lan  metric 1024
ddc9:e21f:b2f3::/64 dev br-lan  metric 1024
unreachable ddc9:e21f:b2f3::/48 dev lo  metric 2147483647
fe80::/64 dev eth0  metric 256
fe80::/64 dev br-lan  metric 256
fe80::/64 dev tun0  metric 256
fe80::/64 dev ifb4eth0  metric 256
default dev tun0  metric 1024
default via fe80::abcd:efgh:hijk:lmn dev eth0  metric 1024  expires 0sec
local ::1 dev lo table local  metric 0
anycast 2600:1700:XXXX:2c00:: dev eth0 table local  metric 0
local 2600:1700:XXXX:2c00:{REDACTED} dev eth0 table local  metric 0
local 2600:1700:XXXX:2c00:{REDACTED} dev eth0 table local  metric 0
anycast 2600:1700:XXXX:2c0f:: dev br-lan table local  metric 0
local 2600:1700:XXXX:2c0f:{eui64} dev br-lan table local  metric 0
anycast 2a05:dfc7:45:babe:: dev tun0 table local  metric 0
local 2a05:dfc7:45:babe::1000 dev tun0 table local  metric 0
anycast ddc9:e21f:b2f3:: dev br-lan table local  metric 0
local ddc9:e21f:b2f3:0:{eui64} dev br-lan table local  metric 0
anycast fe80:: dev eth0 table local  metric 0
anycast fe80:: dev br-lan table local  metric 0
anycast fe80:: dev tun0 table local  metric 0
anycast fe80:: dev ifb4eth0 table local  metric 0
local fe80::{eui64} dev br-lan table local  metric 0
local fe80::834b:32c3:304b:cbdf dev tun0 table local  metric 0
local fe80::{REDACTED} dev ifb4eth0 table local  metric 0
local fe80::{REDACTED} dev eth0 table local  metric 0
multicast ff00::/8 dev br-lan table local  metric 256
multicast ff00::/8 dev eth0 table local  metric 256
multicast ff00::/8 dev tun0 table local  metric 256
multicast ff00::/8 dev ifb4eth0 table local  metric 256
0:      from all lookup local
32766:  from all lookup main
4200000000:     from 2600:1700:XXXX:2c0f:{eui64}/64 iif br-lan lookup unspec unreachable
4200000001:     from all iif lo lookup unspec 12
4200000002:     from all iif eth0 lookup unspec 12
4200000002:     from all iif eth0 lookup unspec 12
4200000011:     from all iif br-lan lookup unspec 12

Perhaps this has something to do with it, going through the output above I noticed I had nat6 turned on due to prior configuration at my old ISP. I believe I installed it according to the instructions here: https://openwrt.org/docs/guide-user/network/ipv6/ipv6.nat6?rev=1622822402 but I am unsure how to uninstall it precisely.

I think @_FailSafe had a how-to for getting multiple prefixes from ATT on OpenWrt. You don't want any nat66 at all.

I've heard Dockers networking config is really bad at Ipv6. I mostly haven't done a lot with docker. The few docker containers I'm running are actually running through podman.

Ideally a docker containers would use a veth and the other side outside the container would be attached to a bridge that had an Ethernet port in it and docker would just get RA announcements and make itself a SLAAC address or a tokenized address.

2 Likes

Do you know how to remove it? I enabled it according to the instructions above about a year and a half ago but I'm not sure how to uninstall.

@Thickness0448 Not sure about the docker configuration, sorry. But here’s a wiki post I put together for how to configure DHCPv6-PD with ATT fiber:

Thanks for the mention, @dlakelan!

2 Likes

Depending on how much customization you've got perhaps a reset to default?

1 Like

Yeah, I wanted to avoid that but it might be the only way.

Take a backup first, maybe restore just the customizations you really want

1 Like

Alright the new install is up and running, upgraded to 22.03 as well. Everything ipv6 related is on default, and while I'm getting individual ipv6 addresses per device on my lan I can't seem to get it to use the static address for all ipv6 traffic.

This is the output of "ip -6 route show dev eth0" on my server:

2600:1700:XXXX:2c0f::8a7 proto kernel metric 100 pref medium
2600:1700:XXXX:2c0f::/64 proto ra metric 100 pref medium
fd75:8791:cb96::8a7 proto kernel metric 100 pref medium
fd75:8791:cb96::/64 proto ra metric 100 pref medium
fd75:8791:cb96::/48 via fe80::aaaa:bbbb:cccc:dddd proto ra metric 100 pref medium
fe80::/64 proto kernel metric 100 pref medium
default via fe80::aaaa:bbbb:cccc:dddd proto ra metric 100 pref medium

And which address is the server using?

1 Like

Comment out ula_prefix so there is no ULA involved.

It looks like you have only one /64 prefix active, this is the one for the WAN itself. Since it is a /64 you could use DHCP relay to put all of your LAN devices also in this /64 and they could use the Internet with no NAT on that same prefix.

But the better way to do it is have the other GUA prefixes appear each on your LANs. The ISP returns packets from the Internet to any of those prefixes to your house, then your router dispatches them to the proper LAN.

If they are static (AT&T will never change them on you), you could set them statically directly onto the LAN-like interfaces. Otherwise you have to count on receiving them via dhcpv6 and being delegated.

1 Like

Well, due to reqprefix=64 in the wan6 configuration, we don't really know what prefix AT&T could provide.

It is built into new versions without needing additional packages-- all that is needed is to put option masqv6 1 in the wan firewall zone. If you don't set that option, NAT66 is not active.

1 Like

First of all, I am assuming you followed the setup in the wiki I posted--is that true?

Assuming the answer is yes, could you post your updated /etc/config/network config?

1 Like

For all ipv6 internet traffic it uses 2600:1700:XXXX:2c0f:{dynamic postfix}.

I've decided this is fine though. The important part is getting a full /64 prefix to my server, and I think if I can get it all the way through to my server I can get docker to use it.

AT&T will only provide /64 to anything downstream of their proprietary gateway, but they do give 8 /64 prefixes.

Ok I can remove ULA.

I assume you're talking about using macvlan for this. Can I use that to successfully get one of AT&T's /64 prefixes all the way to my server? If needed, I do have extra NIC interfaces I could plug in and physically route separately to my server (rather than my server being behind the LAN NIC).

Haven't gotten a chance to do that yet. I'm wondering if it might be better to just use a physical separate NIC for the server and delegate a new prefix to that.