That's quite doable. Create a bridge and add the WAN port and the LAN port going to the secondary router. That take cares of the PPPoE relay, on the secondary router you can plug the cable to the WAN port and configure PPPoE as usual.
As for LAN, create a VLAN (802.1q) device, with the Base device being the same port that connects to secondary router. On br-lan, add this new device to the bridge. On the secondary router, similarly create VLAN, but this time of course with WAN port as the Base device and just as in primary, add it to br-lan.
Side note: I trust you know how to manage DHCP as you're bridging LAN between the two routers.