OpenWrt Forum Archive

Topic: mpd ignores filenames with special characters

The content of this topic has been archived on 27 Mar 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Dear all,

I have seen a ticket about this earlier https://dev.openwrt.org/ticket/10121
Still, I wonder if there is really no fix.

My configuration:
- I run Backfire 10.03.1 final (and tried rc5 and rc6 earlier)
- System is an Nslu2
- CIFS is mounted
  mount -t cifs //192.168.178.23/share /mnt/lsminicifs -o username=myuser,password=mypass,nounix,noserverinfo,iocharset=utf8,codepage=cp850
- mpd is set to use utf-8 in /etc/mpd.conf
  filesystem_charset        "UTF-8"

Problem description:
Mounting the cifs share works fine. Also, I can read correctly all special characters in the shell (which is set to UTF8). However, when creating the database, mpd is skipping all files with special characters, such as äöü.

Is there a fix? Either by configuration or by compiling my own mpd packet?

Any hint would be greatly appreciated.

Best wishes & merry x-mas everyone!

Michael

PS: These are the installed packages

alsa-lib - 1.0.24.1-1
base-files - 43.32-r29592
busybox - 1.15.3-3.4
cifsmount - 1.5-2
dnsmasq - 2.55-6.1
dropbear - 0.53.1-5
fconfig - 20080329-1
firewall - 2-34.8
glib2 - 2.26.1-2
hotplug2 - 1.0-beta-3
iptables - 1.4.6-3.1
iptables-mod-conntrack - 1.4.6-3.1
iptables-mod-nat - 1.4.6-3.1
ixp4xx-microcode - 2.4-2
kernel - 2.6.32.27-1
kmod-crc-ccitt - 2.6.32.27-1
kmod-fs-cifs - 2.6.32.27-1
kmod-input-core - 2.6.32.27-1
kmod-ipt-conntrack - 2.6.32.27-1
kmod-ipt-core - 2.6.32.27-1
kmod-ipt-nat - 2.6.32.27-1
kmod-ipt-nathelper - 2.6.32.27-1
kmod-madwifi - 2.6.32.27+r3314-5
kmod-nls-base - 2.6.32.27-1
kmod-nls-cp437 - 2.6.32.27-1
kmod-nls-cp850 - 2.6.32.27-1
kmod-nls-iso8859-1 - 2.6.32.27-1
kmod-nls-utf8 - 2.6.32.27-1
kmod-ppp - 2.6.32.27-1
kmod-pppoe - 2.6.32.27-1
kmod-sound-core - 2.6.32.27-1
kmod-usb-audio - 2.6.32.27-1
kmod-usb-core - 2.6.32.27-1
kmod-usb-ohci - 2.6.32.27-1
libaudiofile - 0.2.7-1
libc - 0.9.30.1-43.32
libcurl - 7.21.7-1
libfaad2 - 2.7-1
libflac - 1.2.1-1
libgcc - 4.3.3+cs-43.32
libiconv - 5
libid3tag - 0.15.1b-3
libip4tc - 1.4.6-3.1
libiwinfo - 18
libiwinfo-lua - 18
liblua - 5.1.4-7
libmad - 0.15.1b-3
libmms - 0.6-1
libmpdclient - 2.4-1
libogg - 1.1.4-2
libopenssl - 0.9.8r-1
libpthread - 0.9.30.1-43.32
librt - 0.9.30.1-43.32
libshout - 2.2.2-2
libsndfile - 1.0.21-1
libspeex - 1.2rc1-1
libtheora - 1.0-3
libuci - 12012009.7-4
libuci-lua - 12012009.7-4
libvorbis - 1.2.3-1
libvorbisidec - 1.0.2+svn14261-1
libwrap - 7.6-1
libxtables - 1.4.6-3.1
lua - 5.1.4-7
luci - 0.10.0-1
luci-app-firewall - 0.10.0-1
luci-i18n-english - 0.10.0-1
luci-lib-core - 0.10.0-1
luci-lib-ipkg - 0.10.0-1
luci-lib-lmo - 0.10.0-1
luci-lib-nixio - 0.10.0-1
luci-lib-sys - 0.10.0-1
luci-lib-web - 0.10.0-1
luci-mod-admin-core - 0.10.0-1
luci-mod-admin-full - 0.10.0-1
luci-proto-core - 0.10.0-1
luci-proto-ppp - 0.10.0-1
luci-sgi-cgi - 0.10.0-1
luci-theme-base - 0.10.0-1
luci-theme-openwrt - 0.10.0-1
mpc - 0.20-1
mpd - 0.16.1-1
mtd - 13
opkg - 576-2
ppp - 2.4.4-16.1
ppp-mod-pppoe - 2.4.4-16.1
uci - 12012009.7-4
udevtrigger - 106-1
uhttpd - 28
wireless-tools - 29-4
wpad-mini - 20111103-2
zlib - 1.2.3-5

(Last edited by mseewald on 24 Dec 2011, 15:46)

When I use strace to look for problems, I come across these error messages:

open("/usr/lib/charset.alias", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/locale.alias", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)

This seems to point towards libiconv or libiconv-full. However, when manually inspecting these packages (not installing), they do not seem to provide these files.

Any idea?

Best wishes

Hi,

I also have noticed this problem (Nslu and DIR-825) also.

I did a lot of tests and unfortunately I had to downgrade to rc3 to make Mpd able to read accent on filenames on a cifs mount.

I would really apreciate if someone can help resolve it.

Thanks for all.

Regards.

It remains a big surprise to me, that few others seem to have the problem. I assumed a lot of people would be running mpd on openwrt, e.g. to stream music over a USB soundcard. It is a low-cost solution providing high quality sound. At the same time, this comes along with convenience (MPD plus iphone app MPOD), similar commercial solutions like sonos cost a fortune...

Anyway, I think I have to reinvest to get a slightly larger solution for this setup. Voyage Linux seems to be a good choice:
http://store.voyage.hk/KMPD2fwn.php?id=52

Best wishes,
Michael

glibc should be compiled with a special flag set. I did it and it worked for me. The problem is I do not remember what this option was.

That's at least a start. Do you still have the corresponding source tree?

If not: Do you think, it makes sense to compare glibc source trees for rc3 and rc4 and check for flag differences?

ok, I managed to find it in the #opnewrt channel logs

flux242  > xMff: maybe the uclib are misses uclib_has_lacale flag during compilation?
xMff > flux242: that could be
xMff > $ grep LOCALE build_dir/toolchain-mips_r2_gcc-linaro_uClibc-0.9.32/uClibc/.config
xMff > # UCLIBC_HAS_LOCALE is not set
xMff > $ grep LOCALE toolchain/uClibc/config-0.9.32/common
xMff > # UCLIBC_HAS_LOCALE is not set

iirc it's enough to add UCLIBC_HAS_LOCALE into the uClibc/.config

I added "UCLIBC_HAS_LOCALE=y" to the .config, compiled the image and flashed the NSLU2 with it.

-> It's the same as before. sad

However, I am not sure, I have received a fully coherent system. Kernel and modules are freshly compiled (based on r29592 release = 10.3.1), that should have replaced uClibc. But, mpd itself is pulled from the package repository.

Do I need to recompile mpd so that it will become aware of the changed flag during compilation time?

Apologies for these basic questions... and again thanks for any help!

Best wishes,
Michael

mseewald wrote:

Do I need to recompile mpd so that it will become aware of the changed flag during compilation time?

Almost certainly. Most packages don't know about libc compilation flags at all: instead they probe offered libc binaries to see if they support certain features or not. And this happens at compile time. If later you'll try to run program with more capable libc version - it'll ignore advanced features, if you'll try to run it using less capable libc version - it'll not even start.

mseewald wrote:

I added "UCLIBC_HAS_LOCALE=y" to the .config, compiled the image and flashed the NSLU2 with it.

what .config did you change? You need to change the build_dir/toolchain-*/uClibc/.config like in the example. The toolchain* dir will be different in your case. After that you need to call make dirclean and then built everything again. You also need to update the mpd database.

There is something else I did but I'm not sure if it is needed (it could be activated by default)
I used --enable-libiconv flag (or similar. don't remember the name exactly) in the mpd/Makefile. But I do not have sources at hand. Try what I suggested above first and if it doesn't work I'll try to find my old mpd/Makefile and post it here

flux wrote:

what .config did you change? You need to change the build_dir/toolchain-*/uClibc/.config like in the example. The toolchain* dir will be different in your case. After that you need to call make dirclean and then built everything again. You also need to update the mpd database.

What I did is the following:
- make menuconfig
- make world (-> before that, the uClibc .config file is not available)
- edit the .config ("./build_dir/toolchain-armeb_v5te_gcc-4.3.3+cs_uClibc-0.9.30.1_eabi/uClibc-0.9.30.1/.config")
- make world
- flash NSLU with image and setup
- connect to repository and pull mpd
- created mpd database to see if it includes mp3 with special characters

Based on your post:
- I wonder now if I need to edit further upstream. When I run "make dirclean", this particular .config is deleted. This suggests there should be a better place for patching.
- I probably need to recompile mpd... (as also suggested by khim)

Thx!

mseewald wrote:

- connect to repository and pull mpd
- created mpd database to see if it includes mp3 with special characters

you mean you installed mpd from repositories? No, it won't work. You need to compile mpd with the uClibc with the locale activated. Functions from the uClibc are statically linked

flux wrote:

You need to compile mpd with the uClibc with the locale activated.

Ok, I got it. These custom images are best set up at compilation time, not with opkg... Makes a lot of sense.

RTFM was also helpful - http://wiki.openwrt.org/doc/howto/build

Now waiting for the compiler run and next test results...

...and it worked! smile

Thanks everybody for the help!

Setting "UCLIBC_HAS_LOCALE=y" and then recompiling everything including mpd made this a success.

How can I contribute this as a patch? The .config file is generated from somewhere else. This cannot be patched in the source tree...

(Last edited by mseewald on 1 Jan 2012, 14:38)

See https://dev.openwrt.org/changeset/29633 - in menuconfig, go to "Global build settings" and enable "Compile with full language support" to activate the locale awareness. Obviously you have to rebuild everything, including the toolchain afterwards.

jow wrote:

See https://dev.openwrt.org/changeset/29633 - in menuconfig, go to "Global build settings" and enable "Compile with full language support" to activate the locale awareness. Obviously you have to rebuild everything, including the toolchain afterwards.

only in trunk? I'm personally not using trunk - it doesn't really work for me

btw why the zoneinfo package is missing in the backfire branch for 10.03.1?

Zoneinfo is missing because it has no users, we do not support eglibc or glibc in backfire.

jow wrote:

Zoneinfo is missing because it has no users, we do not support eglibc or glibc in backfire.

I've checked and seems like it's indeed not needed. Seems like luci has timezone info hardcoded and it sets timezone property. I just thought it's done by the zoneinfo package

jow wrote:

See https://dev.openwrt.org/changeset/29633 - in menuconfig, go to "Global build settings" and enable "Compile with full language support" to activate the locale awareness. Obviously you have to rebuild everything, including the toolchain afterwards.

Will this be included in the next rc? mpd is not very useful without support of non-ascii characters. I don't know about other affected programs.

Thanks for the people to have find a solution!
Is there a way to apply the modification to a 10.03.1 version?

I tried to build the trunk version and change the parameter in make menuconfig, but it did notcorrect the problem.
And more, the mpd on the trunk version is a mpd-mini without alsa output.

My wish would be to build the 10.03.1 with the modification. But how?

owcpt wrote:

Thanks for the people to have find a solution!
Is there a way to apply the modification to a 10.03.1 version?

What I did was the following
- download 10.03.1 via svn
- download packages via scripts mechanism
- configure via "make menuconfig" to make sure everything is included in the image (mpd, kmods..)
- first compiler run "make world" to create .config in uClibc directory
- go to uClibc directory with .config file, run "make menuconfig" to set UCLIBC_HAS_LOCALE and two other dependent variables)
- second compiler run "make world"
- flash image

Thanks, I will try right now!

Hello,

I did the following:

svn checkout svn://svn.openwrt.org/openwrt/tags/backfire_10.03.1
cd backfire_10.03.1
./scripts/feeds update -a
./scripts/feeds install -a
make menuconfig
  remove dnsmasq, ppp
  add vim, tcpdump, cifsmount, mpd, libsndfile, libwrap, kmod-crypto-core, kmod-crypto-hmac, kmod-crypto-md5, kmod-nls-utf8, kmod-fs-cifs, kmod-usb-audio, kmod-usb-uhci
make world

When finished:
cd  build_dir/toolchain-armeb_v5te_gcc-4.3.3+cs_uClibc-0.9.30.1_eabi/uClibc/
make menuconfig
  add
    Locale Support
      Use Pre-generated Locale Data
        Automagically Download the Pre-generated Locale Data (if necessary)
      Extended Locale Support (experimental/incomplete)
cd -
make world

Then I did flash my nslu2 and try mpd but it did not work. (on rc3 it was fine)

I certainly miss something but I do not know what.
Any clues?
Can you post your .config file?

The discussion might have continued from here.