Sorry for the necro'ing, I've pondered creating a new identical topic, and chose the revival.
I'm in a quite similar boat, but I think the outcome may not be the same.
The kernel config modification may not be acceptable by all other boards from the same target.
I'm investigating the loss of some available RAM on a GnuBee PC1 board.
The board was previously running Neil Brown's firmware, and I installed a recent OpenWRT snapshot.
Then I saw the following difference in available RAM and started investigating:
Original:
dmesg-neilb-5.6.3: Linux version 5.6.3+ (root@gnubeast.brown) (gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)) #3 SMP Sun Apr 12 18:27:54 AEST 2020
dmesg-neilb-5.6.3: HighMem [mem 0x000000001c000000-0x0000000023ffffff]
dmesg-neilb-5.6.3: Early memory node ranges
dmesg-neilb-5.6.3: node 0: [mem 0x0000000000000000-0x000000001bffffff]
dmesg-neilb-5.6.3: node 0: [mem 0x0000000020000000-0x0000000023ffffff]
dmesg-neilb-5.6.3: Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
dmesg-neilb-5.6.3: On node 0 totalpages: 131072
dmesg-neilb-5.6.3: HighMem zone: 16384 pages, LIFO batch:3
dmesg-neilb-5.6.3: Memory: 499300K/524288K available (6727K kernel code, 249K rwdata, 1464K rodata, 11004K init, 242K bss, 24988K reserved, 0K cma-reserved, 65536K highmem)
Versus
dmesg-owrt-6.6.44: Linux version 6.6.44 (builder@buildhost) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 13.3.0 r27119-c609c3bd6e) 13.3.0, GNU ld (GNU Binutils) 2.42) #0 SMP Sun Aug 11 16:56:27 2024
dmesg-owrt-6.6.44: Early memory node ranges
dmesg-owrt-6.6.44: node 0: [mem 0x0000000000000000-0x000000001bffffff]
dmesg-owrt-6.6.44: node 0: [mem 0x0000000020000000-0x0000000023ffffff]
dmesg-owrt-6.6.44: Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
dmesg-owrt-6.6.44: On node 0, zone Normal: 16384 pages in unavailable ranges
dmesg-owrt-6.6.44: Memory: 441424K/458752K available (8179K kernel code, 636K rwdata, 1756K rodata, 1256K init, 227K bss, 17328K reserved, 0K cma-reserved)
It looks like an highmem-enabled kernel can access a 64MB chunk of RAM that a no-highmem can't.
That's more than 10% on a RAM-poor NAS-oriented board, probably worth the hassle to get it back.
I built & flashed a current OpenWRT snapshot, without any modifications, wich gave the following output:
dmesg-owrt-6.6.45-custom-nomodifs: Linux version 6.6.45 (builder@buildhost) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 13.3.0 r27140-ccc06f6716) 13.3.0, GNU ld (GNU Binutils) 2.42) #0 SMP Tue Aug 13 10:22:33 2024
dmesg-owrt-6.6.45-custom-nomodifs: Early memory node ranges
dmesg-owrt-6.6.45-custom-nomodifs: node 0: [mem 0x0000000000000000-0x000000001bffffff]
dmesg-owrt-6.6.45-custom-nomodifs: node 0: [mem 0x0000000020000000-0x0000000023ffffff]
dmesg-owrt-6.6.45-custom-nomodifs: Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
dmesg-owrt-6.6.45-custom-nomodifs: On node 0, zone Normal: 16384 pages in unavailable ranges
dmesg-owrt-6.6.45-custom-nomodifs: Memory: 441424K/458752K available (8180K kernel code, 636K rwdata, 1756K rodata, 1256K init, 227K bss, 17328K reserved, 0K cma-reserved)
And then rebuilt & flashed, after adding CONFIG_HIGHMEM=y
dmesg-owrt-6.6.45-custom-highmem: Linux version 6.6.45 (builder@buildhost) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 13.3.0 r27140-ccc06f6716) 13.3.0, GNU ld (GNU Binutils) 2.42) #0 SMP Tue Aug 13 10:22:33 2024
dmesg-owrt-6.6.45-custom-highmem: HighMem [mem 0x0000000020000000-0x0000000023ffffff]
dmesg-owrt-6.6.45-custom-highmem: Early memory node ranges
dmesg-owrt-6.6.45-custom-highmem: node 0: [mem 0x0000000000000000-0x000000001bffffff]
dmesg-owrt-6.6.45-custom-highmem: node 0: [mem 0x0000000020000000-0x0000000023ffffff]
dmesg-owrt-6.6.45-custom-highmem: Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
dmesg-owrt-6.6.45-custom-highmem: On node 0, zone HighMem: 16384 pages in unavailable ranges
dmesg-owrt-6.6.45-custom-highmem: Memory: 506352K/524288K available (8187K kernel code, 637K rwdata, 1756K rodata, 1248K init, 251K bss, 17936K reserved, 0K cma-reserved, 65536K highmem)
The lost RAM is back usable.
Now, I think this kernel config modification will not be acceptable for the whole ramips/mt7621 family of boards.
What should I do ?
I would like to contribute that back to OpenWRT so that GnuBee users can enjoy their full RAM.