On my WRT32x as OpenWrt
and lots of custom scripts(especially emailing)
and the Lighttpd for luci and proxy and webdav
and fail2ban and other services with many modified '.conf's...
From few days ago,
the system keep started reboot without caution when I try to SSH or web connection
(or even I'm not using it sometime)
Any idea to check or monitor system status?
Please share any of know-how to shoot down the problem.
I need any of and all of your knowledge.. until take down the problem.
(you should assume me that newbie when you advice. please.)
P.S1 : Possibly file description limit error in certain point of view. But who knows.
P.S2 : Below are basic system information.
#free
total used free shared buffers cached
Mem: 511948 255660 256288 776 24140 34968
-/+ buffers/cache: 196552 315396
Swap: 999996 0 999996 (<<-- this swap on mounted SSD)
#df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/root 3072 3072 0 100% /rom
tmpfs 255972 776 255196 0% /tmp
/dev/ubi0_1 100232 37136 58256 39% /overlay
overlayfs:/overlay 100232 37136 58256 39% /
tmpfs 512 0 512 0% /dev
Log into the router over ssh, "tail" the log into the terminal window, wait for failure and go from what those logs reveal.
Run your services on something other than your router. Given that, from what I've read, the WRT32X is reasonably stable, it is probably the load you're putting on it.
Well....once I thought that was a memory lack issue,
so that put the swap for the test since OpenWrt is linux.
By the way...the issue seem that not the simple memory lack issue..
No, couldn't. since once the situation happened cannot use SSHD(dropbear) due to sudden disconnection. And many of time reboot automatically.(only once not rebooted that luci console worked to reboot manually via Luci console..funny huh?)
Any, more idea please.
You'll need to create a shell script that logs temp and memory, say every 10s, to a logfile on a USB drive (I'd also recommend tailing the system log to a logfile as well), and set it to run at boot; Else you'll need to configure an external systog server, as until you have the cause of the crash, you can't address it.
Will give it try.
Beside, here is some wired log from 'System Log' that memory address showed.
What is this should be? Is this normal??
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.444638] ------------[ cut here ]------------
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.449303] WARNING: CPU: 0 PID: 6726 at backports-2017-11-01/net/mac80211/ieee80211_i.h:1412 0xbf2111fc [mac80211@bf1f8000+0x5e000]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.461277] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_ipv6 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_state xt_policy xt_nat xt_multiport xt_mark xt_mac xt_limit xt_esp xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc rfcomm nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat_ftp nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack_ftp nf_conntrack mwifiex_sdio mwifiex iptable_mangle iptable_filter ipt_ah ip_tables hidp hci_uart crc_ccitt btusb btmrvl_sdio btmrvl btintel bnep bluetooth hid evdev input_core mwlwifi mac80211 cfg80211 compat xt_set ip_set_list_set ip_set_hash_netiface ip_set_hash_netport ip_set_hash_netnet ip_set_hash_net
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.532731] ip_set_hash_netportnet ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables ipcomp6 xfrm6_tunnel xfrm6_mode_tunnel xfrm6_mode_transport xfrm6_mode_beet esp6 ah6 ipcomp xfrm4_tunnel xfrm4_mode_tunnel xfrm4_mode_transport xfrm4_mode_beet esp4 ah4 tunnel6 tunnel4 af_key xfrm_user xfrm_ipcomp xfrm_algo ecdh_generic sha256_generic jitterentropy_rng drbg md5 kpp hmac echainiv ecb cmac cbc authenc fuse uas gpio_button_hotplug
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.591893] CPU: 0 PID: 6726 Comm: hostapd Tainted: G W 4.14.63 #0
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.599233] Hardware name: Marvell Armada 380/385 (Device Tree)
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.605181] Function entered at [<c010f500>] from [<c010b1d0>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.611038] Function entered at [<c010b1d0>] from [<c061a8a8>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.616895] Function entered at [<c061a8a8>] from [<c0121d40>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.622752] Function entered at [<c0121d40>] from [<c0121e0c>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.628610] Function entered at [<c0121e0c>] from [<bf2111fc>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.634472] Function entered at [<bf2111fc>] from [<bf1d14f8>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.640331] Function entered at [<bf1d14f8>] from [<c0550eb8>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.646187] Function entered at [<c0550eb8>] from [<c0550098>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.652045] Function entered at [<c0550098>] from [<c0550bc0>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.657901] Function entered at [<c0550bc0>] from [<c054f86c>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.663757] Function entered at [<c054f86c>] from [<c054fd10>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.669614] Function entered at [<c054fd10>] from [<c04ff744>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.675472] Function entered at [<c04ff744>] from [<c04ffc78>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.681328] Function entered at [<c04ffc78>] from [<c0500a78>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.687185] Function entered at [<c0500a78>] from [<c0107880>]
Sat Nov 24 12:25:50 2018 kern.warn kernel: [ 8540.693080] ---[ end trace 114085cd0ae6e14e ]---
Sat Nov 24 12:25:57 2018 kern.warn kernel: [ 8546.917777] ------------[ cut here ]------------
Do you have any idea to update NEW wifi driver?
(I have removed ppp packages by the way 'opkg remove kmod-ppp kmod-pppoe kmod-pppox ppp-mod-pppoe ppp #luci-proto-ppp')
I have ubuntu VirtualBox which compile environment of OpenWrt to build new version of nginx. Can I manually rebuild and install the WiFi driver? Then how??
See the Marvell WiFi section in the WRT AC Series wiki, or the official Marvell compile instructions on kaloz's GitHub, to compile the most up to date version.
You can verify the current driver version via: opkg list-installed | grep mwlwifi
Just a general FYI, there's always been issues with the 88W8964 Marvell drivers/firmware on the WRT3200ACM and WRT32X since the day they were released. I haven't followed discussions on the issues with the 88W8964 drivers/firmware for a while, but last I heard, it was something users were just going to have to manage with.
Out of curiosity, is the wifi stable on the firmware the WRT32X ships with, as I thought I heard it was running OpenWrt out of the box.
Thanks for the inform. Will give it try for the driver part.
FYI, my OpenWrt divice situation are that
before last night, my entire system suddenly halt without notice when I tried to 'opkg list-upgradable' and upgrade(especially when iface package update, system stopped/never booted again).
So I have done hard reset and restore all configurations.
With lighttpd, I using complicated multiple server+ssl+auth+redirect+connection control configuration. With the lighttpd, when I do 'service lighttpd restart(or stop/start)' system going to reboot. So today I'm going to port to NginX which custom build.
Will comback to this thread for update of situation for original question.
[sit-wrap]
I have updated all the custom build packages(by 'git clone https://github.com/openwrt/openwrt.git') as below, and will monitor it whether the symptom improve (but here weird mwlwifi packages will be asked below).
[Q]
In the device WRT32x seem that has 'Marvell 88W8964' and 'Marvell 88W8887' as below system information on Luci consol. Not that mentioned as
mwlwifi-firmware-88w8964 - 2018-10-29-382700ce-1
mwlwifi-firmware-88w8864 - 2018-10-29-382700ce-1.
Is there anything I'm missing to understand? Which mean that OpenWrt support different device firmware(or maybe WRT32x has different revision).
Please give an insight about this matter.
You have to be extremely careful using opkg upgrade, as you cannot upgrade system critical packages required by OpenWrt. Since you know how to compile, you may want to consider creating your own build environment and compiling your own images.
Many packages are kernel version dependent, and once buildbot compiles new packages against an updated kernel version, the new packages cannot be upgraded to - instead, an updated image must be flashed.
I noticed in menuconfig the default selection for any WRT AC Series router is for two different radio firmwares to be selected, and I'm not sure why this is.
The WRT32X has three radios? Unless the 88W8887 is a BT 4.2 radio?