(solved) Coova chilli failing DNS sometimes and dropping tunnels

(Firewall was the root of the cause)

using OpenWrt 18.06.2, r7676-cddd7b4c77

coovachilli 1.3.0+20141128-6

any ideas whats wrong with my coova chilli install

so the network seems to work ok user can connect fine authenticating with radius either over wifi or on the local LAN but soon loose internet.

here's a traceroute

traceroute to google.com (172.217.169.46), 30 hops max, 38 byte packets
 1  10.0.0.1 (10.0.0.1)  0.244 ms  0.233 ms  0.230 ms
 2  164.39.173.253 (164.39.173.253)  0.899 ms  0.858 ms  1.598 ms
 3  51.219.94.84 (51.219.94.84)  5.979 ms  5.958 ms  6.001 ms
 4  *  *  *
 5  164.39.242.0 (164.39.242.0)  12.799 ms  12.945 ms  12.854 ms
 6  *  *  *
 7  *  *  *
 8  172.253.68.218 (172.253.68.218)  15.439 ms  108.170.232.96 (108.170.232.96)  13.725 ms  108.170.232.104 (108.170.232.104)  15.451 ms
 9  172.253.66.89 (172.253.66.89)  14.497 ms  108.170.246.176 (108.170.246.176)  14.095 ms  108.170.246.143 (108.170.246.143)  14.392 ms
10  lhr48s08-in-f14.1e100.net (172.217.169.46)  13.662 ms  13.656 ms  13.681 ms

from inside the network on 10.0.1.1/24
after looking at it it seems if you traceroute google.com it will timeout on some hops but if i'm using a tunnel connection gauthenticated by chili and radius desk it gives mixed results sometimes this

on the 10.0.10.1/24 it fails

traceroute to google.com (172.217.169.46), 30 hops max, 38 byte packets
1 10.0.0.1 (10.0.0.1) 0.244 ms 0.233 ms 0.230 ms
2 164.39.173.253 (164.39.173.253) 0.899 ms 0.858 ms 1.598 ms
3 51.219.94.84 (51.219.94.84) 5.979 ms 5.958 ms 6.001 ms
4 10.0.10.1

and sometimes this

Tracing route to eurocarparts.com [194.74.180.83]

over a maximum of 30 hops:

  1     1 ms     1 ms     3 ms  10.0.10.1

  2     1 ms     1 ms     1 ms  10.0.0.1

  3  10.0.10.1  reports: Destination protocol unreachable.

Trace complete.

this is the config

root@OpenWrt:/etc# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.0.0.1        0.0.0.0         UG    0      0        0 eth0.2
10.0.0.0        *               255.255.255.0   U     0      0        0 eth0.2
10.0.1.0        *               255.255.255.0   U     0      0        0 br-lan
10.0.10.0       *               255.255.255.0   U     0      0        0 tun0
config chilli
    # disable to running chilli. remove this option before running.
    #option disabled 1

    # name of TUN device name. required.
    option tundev 'tun0'

    # name of network interfacs
    option network 'lan'      

    # Include this flag if process is to run in the foreground
    #option fg

    # Include this flag to include debug information.
    option debug 9

    # Re-read configuration file at this interval. Will also cause new domain
    # name lookups to be performed. Value is given in seconds.
    #option interval 3600

    # File to store information about the process id of the program.
    # The program must have write access to this file/directory.
    #option pidfile /var/run/chilli.pid

    # Directory to use for nonvolatile storage.
    # The program must have write access to this directory.
    # this option is currently ignored
    #option statedir ./


    # TUN parameters

    # IP network address of external packet data network
    # Used to allocate dynamic IP addresses and set up routing.
    # Normally you do not need to uncomment this option.
    option net 10.0.10.0/24

    # Dynamic IP address pool
    # Used to allocate dynamic IP addresses to clients.
    # If not set it defaults to the net tag.
    # Do not uncomment this option unless you are an experienced user!
    option dynip 10.0.10.0/24


    # Static IP address pool
    # Used to allocate static IP addresses to clients.
    # Do not uncomment this option unless you are an experienced user!
    # option statip 10.0.10.0/24


    # Primary DNS server.
    # Will be suggested to the client.
    # If omitted the system default will be used.
    # Normally you do not need to uncomment this option.
    option dns1 10.0.1.1

    # Secondary DNS server.
    # Will be suggested to the client.
    # If omitted the system default will be used.
    # Normally you do not need to uncomment this option.
    option dns2 10.0.0.1

    # Domain name
    # Will be suggested to the client.
    # Normally you do not need to uncomment this option.
    # option domain motorsport-tools.com

    # Script executed after network interface has been brought up.
    # Executed with the following parameters: <devicename> <ip address>
    # <mask>
    # Normally you do not need to uncomment this option.
    #option ipup /etc/chilli.ipup

    # Script executed after network interface has been taken down.
    # Executed with the following parameters: <devicename> <ip address>
    # <mask>
    # Normally you do not need to uncomment this option.
    #option ipdown /etc/chilli.ipdown


    # Radius parameters

    # IP address to listen to
    # Normally you do not need to uncomment this option.
    #option radiuslisten 127.0.0.1

    # IP address of radius server 1
    # For most installations you need to modify this option.
    option radiusserver1 10.0.0.240

    # IP address of radius server 2
    # If you have only one radius server you should set radiusserver2 to the
    # same value as radiusserver1.
    # For most installations you need to modify this option.
    option radiusserver2 10.0.0.240

    # Radius authentication port
    # The UDP port number to use for radius authentication requests.
    # The same port number is used for both radiusserver1 and radiusserver2.
    # Normally you do not need to uncomment this option.
    #option radiusauthport 1812

    # Radius accounting port
    # The UDP port number to use for radius accounting requests.
    # The same port number is used for both radiusserver1 and radiusserver2.
    # Normally you do not need to uncomment this option.
    #option radiusacctport 1813

    # Radius shared secret for both servers
    # For all installations you should modify this option.
    option radiussecret **************

    # Radius NAS-Identifier
    # Normally you do not need to uncomment this option.
    option radiusnasid public001

    # WISPr Location ID. Should be in the format: isocc=<ISO_Country_Code>,
    # cc=<E.164_Country_Code>,ac=<E.164_Area_Code>,network=<ssid/ZONE>
    # Normally you do not need to uncomment this option.
    #option radiuslocationid isocc=us,cc=1,ac=408,network=ACMEWISP_NewarkAirport

    # WISPr Location Name. Should be in the format:
    # <HOTSPOT_OPERATOR_NAME>,<LOCATION>
    # Normally you do not need to uncomment this option.
    #option radiuslocationname ACMEWISP,Gate_14_Terminal_C_of_Newark_Airport
    # option ssid *****************

    # Radius proxy parameters

    # IP address to listen to
    # Normally you do not need to uncomment this option.
    #option proxylisten 10.0.0.1

    # UDP port to listen to.
    # If not specified a port will be selected by the system
    # Normally you do not need to uncomment this option.
    #option proxyport 1645

    # Client(s) from which we accept radius requests
    # Normally you do not need to uncomment this option.
    #option proxyclient 10.0.0.1/24

    # Radius proxy shared secret for all clients
    # If not specified defaults to radiussecret
    # Normally you do not need to uncomment this option.
    #option proxysecret *************


    # DHCP Parameters

    # Ethernet interface to listen to.
    # This is the network interface which is connected to the access points.
    # In a typical configuration this option should be set to eth1.
    option dhcpif eth1.1

    # Use specified MAC address.
    # An address in the range  00:00:5E:00:02:00 - 00:00:5E:FF:FF:FF falls
    # within the IANA range of addresses and is not allocated for other
    # purposes.
    # Normally you do not need to uncomment this option.
    #option dhcpmac 00:00:5E:00:02:00

    # Time before DHCP lease expires
    # Normally you do not need to uncomment this option.
    #option lease 600


    # Universal access method (UAM) parameters

    # URL of web server handling authentication.
    option uamserver 'http://10.0.0.240/cake3/rd_cake/dynamic-details/chilli-browser-detect/'

    # URL of welcome homepage.
    # Unauthenticated users will be redirected to this URL. If not specified
    # users will be redirected to the uamserver instead.
    # Normally you do not need to uncomment this option.
    #option uamhomepage http://192.168.182.1/welcome.html

    # Shared between chilli and authentication web server
    option uamsecret *************

    # IP address to listen to for authentication requests
    # Do not uncomment this option unless you are an experienced user!
    option uamlisten 10.0.10.1

    # TCP port to listen to for authentication requests
    # Do not uncomment this option unless you are an experienced user!
    #option uamport 3990

    # Comma separated list of domain names, IP addresses or network segments
    # the client can access without first authenticating.
    # It is possible to specify this option multiple times.
    # Normally you do not need to uncomment this option.
    option uamallowed 10.0.0.0/24,10.0.1.0/24,10.0.10.0/24 


    # the client can access without first authenticating.
    # It is possible to specify this option multiple times.
    # Normally you do not need to uncomment this option.
    option uamdomain .motorsport-tools.com,.motamec.com

    # If this flag is given unauthenticated users are allowed to use
    # any DNS server.
    # Normally you do not need to uncomment this option.
    # option uamanydns
    

    # MAC authentication

    # If this flag is given users will be authenticated only on their MAC
    # address.
    # Normally you do not need to uncomment this option.
    #option macauth

    # List of MAC addresses.
    # The MAC addresses specified in this list will be authenticated only on
    # their MAC address.
    # this option is ignored if the macauth tag is given.
    # It is possible to specify this option multiple times.
    # Normally you do not need to uncomment this option.
    #option macallowed E8-94-F6-EC-32-0B,14-CC-20-4F-C0-04
    option macallowlocal E8-94-F6-EC-32-0B,14-CC-20-4F-C0-04

    # Password to use for MAC authentication.
    # Normally you do not need to uncomment this option.
    #option macpasswd password

    # Suffix to add to MAC address in order to form the username.
    # Normally you do not need to uncomment this option.
    #option macsuffix suffix
root@OpenWrt:~# cat /etc/config/network 

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdde:c9f5:ae34::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth1.1'
	option proto 'static'
	option ipaddr '10.0.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option delegate '0'
	option dns '10.0.0.1'

config interface 'wan'
	option ifname 'eth0.2'
	option proto 'dhcp'
	option delegate '0'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 4 0t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '5 6t'

root@OpenWrt:~# cat /etc/config/firewall 

config rule
	option target 'ACCEPT'
	option src 'wan'
	option proto 'tcp'
	option dest_port '80'
	option name 'Web'

config rule
	option target 'ACCEPT'
	option src 'wan'
	option proto 'tcp'
	option dest_port '22'
	option name 'SSH'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config include
	option path '/etc/firewall.user'
root@OpenWrt:/etc# ifconfig
br-lan    Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D6  
          inet addr:10.0.1.1  Bcast:10.0.1.255  Mask:255.255.255.0
          inet6 addr: fdde:c9f5:ae34::1/60 Scope:Global
          inet6 addr: fe80::c24a:ff:fef6:ccd6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16919 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20763 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2140151 (2.0 MiB)  TX bytes:22320003 (21.2 MiB)

eth0      Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D7  
          inet6 addr: fe80::c24a:ff:fef6:ccd7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27530 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25486 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23744929 (22.6 MiB)  TX bytes:4905245 (4.6 MiB)
          Interrupt:4 

eth0.2    Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D7  
          inet addr:10.0.0.8  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::c24a:ff:fef6:ccd7/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27530 errors:0 dropped:817 overruns:0 frame:0
          TX packets:25483 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:23249389 (22.1 MiB)  TX bytes:4803330 (4.5 MiB)

eth1      Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D6  
          inet6 addr: fe80::c24a:ff:fef6:ccd6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9 errors:0 dropped:4 overruns:0 frame:0
          TX packets:195 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:834 (834.0 B)  TX bytes:25172 (24.5 KiB)
          Interrupt:5 

eth1.1    Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D6  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:180 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:21993 (21.4 KiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:20 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:1899 (1.8 KiB)  TX bytes:1899 (1.8 KiB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.0.10.1  P-t-P:10.0.10.1  Mask:255.255.255.0
          inet6 addr: fe80::5bcf:da41:913a:da72/64 Scope:Link
          inet6 addr: fe80::5bcf:da41:913a:da72/64 Scope:Link
          UP POINTOPOINT RUNNING  MTU:1500  Metric:1
          RX packets:5348 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7251 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:829199 (809.7 KiB)  TX bytes:5162665 (4.9 MiB)

wlan0     Link encap:Ethernet  HWaddr C0:4A:00:F6:CC:D6  
          inet6 addr: fe80::c24a:ff:fef6:ccd6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16917 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20880 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2376977 (2.2 MiB)  TX bytes:22707630 (21.6 MiB)