OK it starting to make me crazy!
Before flashing, I tried soft factory resetting as below. The result is that new setting (e.g. SSID and security) weren't saved after I restart. It made me doubt that I forgot to apply the settings before restarting (PC is away from the main router, so I have to keep moving OpenWrt router over when I need to connect with a cable). Anyway, after few times of applying the settings and losing thme after restarting, I just re-flashed (it's the same version I had actually, except that I used sysupgrade copy of course).
umount /overlay && firstboot && reboot
Anyway, after re-flashing, the problem of the OpenWrt router has no internet connection and not being able to ping the main router form LuCI (LAN to LAN connection), even though I can access internet when connected to OpenWrt AP! I also tried a cross-over cable, thinking that maybe one of the two router isn't smart enough to recognize that at the other side it's a router not a terminal, but that didn't seem to make a difference.
So I had to connect the cable to OpenWrt via WAN (still DHCP client, different subnet though) in order to download the packages I wanted to download. But that's not an ideal situation for two reasons:
1- I wanted both router to be on the same subset for convenience.
2- I wanted OpenWrt WAN to be configured for my PPPoE, for easy swap when the other router if and when I need to do so.