Bonding + X10SBA-F almost dropping every packet

Hey friends how are you doing.
I've build up a new router for my 1Gbps connection and i still had an unused X10SBA lying around.
So i wanted to use this board, because it's really nice, due to it's dual NIC, perfect for bonding.
So that's what i've done, i have compiled OpenWrt almost with default settings, only added bonding and LuCi support.
Unfortunately i have a quite strange behaviour.
After establishing the LAG on the device it doesn't respond correctly anymore to any kind of network traffic. It's almost impossible to SSH into the device, nor access the web interface.
That's how the pings look like to the device.
What might be the issue here?

On the switch's side everything is okay. Sometimes strange behaviour on the device side, one port get's removed and re-added again.

25-Dec-2021 21:01:32 %LINK-I-Up:  gi18, aggregated (1)
25-Dec-2021 21:01:34 %LINK-I-Up:  gi17, aggregated (1)
25-Dec-2021 21:01:38 %TRUNK-I-PORTADDED: Port gi18 added to Po3
25-Dec-2021 21:01:38 %LINK-I-Up:  Po3, aggregated (1)
25-Dec-2021 21:01:42 %STP-W-PORTSTATUS: Po3: STP status Forwarding, aggregated (1)
25-Dec-2021 21:06:07 %TRUNK-W-PORTREMOVED: Port gi18 removed from Po3
25-Dec-2021 21:06:07 %LINK-W-Down:  gi18, aggregated (1)
25-Dec-2021 21:06:07 %LINK-W-Down:  Po3, aggregated (1)
25-Dec-2021 21:06:08 %LINK-W-Down:  gi17, aggregated (1)
25-Dec-2021 21:06:11 %LINK-I-Up:  gi18, aggregated (1)
25-Dec-2021 21:06:12 %LINK-I-Up:  gi17, aggregated (1)
25-Dec-2021 21:06:16 %TRUNK-I-PORTADDED: Port gi18 added to Po3, aggregated (1)
25-Dec-2021 21:06:16 %LINK-I-Up:  Po3


Thanks in advance!

64 bytes from 192.168.3.250: icmp_seq=162 ttl=64 time=0.597 ms
64 bytes from 192.168.3.250: icmp_seq=163 ttl=64 time=0.629 ms
64 bytes from 192.168.3.250: icmp_seq=164 ttl=64 time=0.594 ms
64 bytes from 192.168.3.250: icmp_seq=165 ttl=64 time=0.589 ms
64 bytes from 192.168.3.250: icmp_seq=167 ttl=64 time=0.591 ms
64 bytes from 192.168.3.250: icmp_seq=169 ttl=64 time=0.579 ms
64 bytes from 192.168.3.250: icmp_seq=171 ttl=64 time=0.584 ms
64 bytes from 192.168.3.250: icmp_seq=172 ttl=64 time=0.598 ms
64 bytes from 192.168.3.250: icmp_seq=173 ttl=64 time=0.602 ms
64 bytes from 192.168.3.250: icmp_seq=174 ttl=64 time=0.576 ms
64 bytes from 192.168.3.250: icmp_seq=175 ttl=64 time=0.622 ms
64 bytes from 192.168.3.250: icmp_seq=177 ttl=64 time=0.601 ms
64 bytes from 192.168.3.250: icmp_seq=179 ttl=64 time=0.593 ms
64 bytes from 192.168.3.250: icmp_seq=196 ttl=64 time=0.580 ms
64 bytes from 192.168.3.250: icmp_seq=200 ttl=64 time=0.587 ms
64 bytes from 192.168.3.250: icmp_seq=202 ttl=64 time=0.602 ms
64 bytes from 192.168.3.250: icmp_seq=204 ttl=64 time=0.594 ms
64 bytes from 192.168.3.250: icmp_seq=206 ttl=64 time=0.579 ms
64 bytes from 192.168.3.250: icmp_seq=209 ttl=64 time=0.569 ms
64 bytes from 192.168.3.250: icmp_seq=211 ttl=64 time=0.566 ms
64 bytes from 192.168.3.250: icmp_seq=214 ttl=64 time=0.593 ms
64 bytes from 192.168.3.250: icmp_seq=216 ttl=64 time=0.596 ms
64 bytes from 192.168.3.250: icmp_seq=218 ttl=64 time=0.568 ms
64 bytes from 192.168.3.250: icmp_seq=220 ttl=64 time=0.558 ms
64 bytes from 192.168.3.250: icmp_seq=221 ttl=64 time=0.585 ms
64 bytes from 192.168.3.250: icmp_seq=222 ttl=64 time=0.586 ms
64 bytes from 192.168.3.250: icmp_seq=223 ttl=64 time=0.588 ms
64 bytes from 192.168.3.250: icmp_seq=225 ttl=64 time=1.21 ms
64 bytes from 192.168.3.250: icmp_seq=227 ttl=64 time=0.582 ms
64 bytes from 192.168.3.250: icmp_seq=230 ttl=64 time=0.591 ms
64 bytes from 192.168.3.250: icmp_seq=232 ttl=64 time=0.575 ms
64 bytes from 192.168.3.250: icmp_seq=234 ttl=64 time=0.557 ms
64 bytes from 192.168.3.250: icmp_seq=236 ttl=64 time=0.556 ms
64 bytes from 192.168.3.250: icmp_seq=238 ttl=64 time=0.570 ms
64 bytes from 192.168.3.250: icmp_seq=240 ttl=64 time=0.597 ms
64 bytes from 192.168.3.250: icmp_seq=242 ttl=64 time=0.587 ms
64 bytes from 192.168.3.250: icmp_seq=244 ttl=64 time=0.607 ms
64 bytes from 192.168.3.250: icmp_seq=246 ttl=64 time=0.574 ms
64 bytes from 192.168.3.250: icmp_seq=248 ttl=64 time=0.583 ms
64 bytes from 192.168.3.250: icmp_seq=250 ttl=64 time=0.566 ms
64 bytes from 192.168.3.250: icmp_seq=252 ttl=64 time=0.580 ms
64 bytes from 192.168.3.250: icmp_seq=254 ttl=64 time=0.604 ms
64 bytes from 192.168.3.250: icmp_seq=256 ttl=64 time=0.606 ms