BTHH5A 22.03 LVM2 on LUKS: EXT4/F2FS problems

Hello everyone, i've recently upgraded my Bt home hub 5a to 22.03 with a clean setup and manual reconfiguration. Most things work like in the previous release but USB storage.
I've setup an external 256Gb USB 3.0 flash drive as disk storage. More precisely the disk is GPT formatted and encrypted with LUKS. On top of LUKS i've setup LVM2 with 1 physical volume, 1 volume group and 3 logical partitions on the volume group. Up until here it all looks and works fine, the disk is properly decrypted and the logical volume management works fine too. The problem shows up when i try to format the partitions. I've tried to use both F2FS and EXT4 without much luck. The partitions get properly formatted but when i try to mount them it shows errors. When trying to mount F2FS partitions it completely fails with the following message:

mount: mounting /dev/mapper/vg0-sambaprivate on /mnt/samba-private/ failed: No error information

The kernel log after the mount attempt is the following:

F2FS-fs (dm-4): Mismatch valid blocks 0 vs. 141
F2FS-fs (dm-4): Failed to initialize F2FS segment manager (-135)

This is after a fresh mkfs.f2fs succesfully completed.
With F2FS something even more weird happens: the above happens if i try to mount a large partiton (~200Gib). When mounting a smaller partition (~100Mib) the mount is successful but when i cd into the /mnt directory (where i have the directiories used as mount points) and try an "ls" command it fails with a segmentation fault and with the following kernel log:

do_page_fault(): sending SIGSEGV to ls for invalid read access from 00000014
epc = 77e141cc in libc.so[77d94000+a9000]
ra  = 77e14154 in libc.so[77d94000+a9000]

If i cd into the mounted directory and create some other directory, then get back to the root of the mount point the "ls" command works fine.

If instead i try to use EXT4, i can successfully mount the partition but corruption happens during usage with messages like the following:

EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 5248 with max blocks 48 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:6: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 5296 with max blocks 192 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 5488 with max blocks 208 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 5695 with max blocks 190 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:6: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:6: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs: 6 callbacks suppressed
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 6551 with max blocks 191 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:0: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 6742 with max blocks 361 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:0: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 7103 with max blocks 284 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:0: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 7387 with max blocks 305 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost 
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:0: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 7692 with max blocks 249 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:5: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs: 12 callbacks suppressed
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 9398 with max blocks 216 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error: 1 callbacks suppressed
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:5: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 9614 with max blocks 189 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:0: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 9803 with max blocks 288 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:5: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 10091 with max blocks 277 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs (dm-4): Delayed block allocation failed for inode 13 at logical offset 10367 with max blocks 321 with error 77
EXT4-fs (dm-4): This should not happen!! Data will be lost
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:5: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:2: bg 1: bad block bitmap checksum
EXT4-fs error (device dm-4): ext4_validate_block_bitmap:390: comm kworker/u4:1: bg 1: bad block bitmap checksum

The above happens while transferring a file over a samba share. When creating directories other kind of EXT4-fs errors show up.
I've tried different USB flash drives and tested the drives on my Linux machine where they work perfectly fine.
I don't know which part of the chain is broken (usb drivers, luks, lvm2 or the in-kernel filesystem drivers). I've tried to use the drive without LUKS and LVM2 and f2fs worked fine as also ext4 which lead me to think there might be some issues with LUKS or LVM2.

Wild guess: you might be running out of RAM. Try adding swap (possibly in the form of zram).

I already have zram swap enabled and also i've watched RAM usage closely (especially during samba transfer tests) and i didn't see any problem or OOM reaper logs. I don't think it's a RAM problem due to the fact that on 21.02.3 with the same packages and settings it worked fine.
You might be fooled about the "allocation" word in the logs, but that refers to the block allocation on the USB drive itself by the ext4 driver (somehow the data on the disk gets corrupted and the filesystem internal metadata which gets used for allocations too).

I wanted to update on this:
I am now using a USB flash drive without LUKS and without LVM, GPT formatted with 3 partitions and with f2fs filesystems without any issue. My guess now is that the problems has to be with LUKS or LVM which causes filesystem metadata corruption. Initially i thought my USB flash drives were draining too much power from the USB port and as suggested on the Wiki for the Bt Home Hub 5a i've put an old sd card like drive with 1Gib which for sure doesn't draw much power (and is old USB 2.0). Then i tested this new drive with LUKS and LVM2 and had the same corruption issues like with the previous drives. Without encryption and LVM it all works good. I will try again in a later release to see if the problem persists or not.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.