Yes, that is what several of us have been trying to explain, especially in that IPv6 addressing is very different than NAT-ed IPv4 addressing.
The people on this thread trying to help you likely understand that whether you call it a "modem" or you call it a "router" it has the same function of transporting packets to and from your ISP and delivering them appropriately.
You need to consider in your thinking that your IPv4 addressing is completely independent of your IPv6 addressing.
For that matter, there is no such thing as "sharing" of a "public IP Adress on the internet". Public addresses are required to be on no more than one, globally unique interface within the public, routable Internet space.
When you think about IPv6, you need to disassociate from your experience with NAT-ed IPv4.
In your topology, every node will have a link-local address (required for all nodes in IPv6). Router A's upstream interface will likely have an public IPv6 address assigned by your ISP. If, and only if your ISP provides IPv6 transport for multiple devices, then it will have a Prefix Delegation (PD) also assigned by your ISP. The ISP's router will set up a router for the PD via the upstream interface of Router A. Router A will also "ensure" that destination IPs in the PD are not routed back to the ISP (often a "blackhole" route for the entire PD). Typically Router A would advertise a portion of that PD on its downstream interface, as well as potentially assign one of those addresses to its downstream interface (though neither is required).
The IPv6 address assigned to the modem is not "yours", nor does it necessarily have anything at all to do with address(es) or a PD potentially assigned to you. Your default route may well be a link-local address, which is completely "fine" under IPv6.
Router B would be similar, with "ISP" in the forgoing becoming "Router A". Like the ISP assigning you a PD, if you chose to take a portion of your PD that your ISP gave you and assign it to be downstream of Router B, Router A would need a route installed for that address space via the upstream interface of Router B (which, like any on-link connection, only needs a link-local address, not a publicly routable address.
Edit:
While it may be convenient for router interfaces to be assigned a public IPv6 address, routing, being a next-hop process, can be accomplished with only link-local addresses. Technically, as long as a router knows the next-hop address for a given packet, that is sufficient. Given that return routes need not be the same as "outbound" routes, this knowledge need not even be reciprocal.