[SOLVED] Need help with EoL mismatch on .patch

For Developers
#1

I cant figure out why a .patch for the softethervpn-server package from my feed fails to apply with:

Applying ./patches/001-disable-aes_ni.patch using plaintext:
patching file src/Mayaqua/Encrypt.c
Hunk #1 FAILED at 148 (different line endings).
Hunk #2 FAILED at 4273 (different line endings).
Hunk #3 FAILED at 4287 (different line endings).

Here is my workflow:

  1. All my git cloned openwrt sdk and feeds show CRLF endings for .c/.h files and LF for textfiles.
  2. If i create a diff i get a mixed CRLF and LF .patch file. (CRLF for diff sections and LF for metadata)
  3. I commit the .patch to my repo and force LF via .gitattributes and * text=auto eol=lf.
  4. I test the feed, all .patch (text) files are changed to LF and i get the above error only for the softethervpn-server package.
  5. If i use the org. (before commit) mixed CRLF+LF .patch all works.

The funny thing is that's exactly the same workflow as my samba4 package and there the patches work, i also checked other packages and all .patch files have only LF endings?
So whats going on, why is this one package special?
Do i need to force .patch files to binary's so mixed endings are preserved?

thx
Andy

#2

I think whatever you're using to edit is trying to be smart and auto-convert files between DOS and UNIX which in the end makes the files inconsistent.

Softethervpn uses DOS as standard so for consistency I wouldn't change that.

Easiest would be to use something that doesn't convert line endings for you but otherwise I'd use sed, dos2unix or whatever you perfer and apply that to the file you want to edit or the whole tree. Do your changes and then change it back.

I wouldn't bother with git text conversions at all but I'm a noob :wink:

Since it's a small change I just tried nano which worked fine and kept the DOS endings (without displaying).

Also, you probably want to use codeload instead of pulling a tree that's ~300+Mbyte for each update.

#3

Interesting reading on the "old" and "new" approaches to git and how it handles line endings at

I think

is the root of the "problem"

#4

I use notepad++ on windows and mcedit on linux. As far as notepad++ goes, it never changes existing EoL unless i manually convert those. So i can reload the edited file and still get the mixed EoL. So the actual conversion happen via git.
Yet this does not explain why every other .patch file that was converted to LF only works, while Softether only works with mixed endings.

I also could not find any source repo where .c/.h files are not CRLF only, so any diff created should result in a mixed .patch file initially, if the editor left the CRLF endings?

PS: Thanks for the tip on codeload.

#5

I'm also using Notepad++ and at least my version is a bit buggy regarding line conversion. I don't recall exactly right now how to reproduce but sometimes I need to select conversion an extra time, v7.5.1 (32-bit) so it's probably a bit old.

#6

Ok i found the problem and its a stupid one :stuck_out_tongue:
I often access my linux VM via sftp and on one of my main machines i use Total Commander with the sftp plugin out of convenience and on the others i use WinSCP.

Under linux in the VM i did run: find src/ -exec file {} \; | grep CRLF and indeed all softether source files show up with CRLF:

./src/Mayaqua/Network.h: C source, ASCII text, with very long lines, with CRLF line terminators
./src/Mayaqua/Table.h: C source, ASCII text, with CRLF line terminators
./src/Mayaqua/Encrypt.c: C source, ASCII text, with CRLF line terminators

If i run the same command on the samba4 source, nothing shows up. Yet if i opened/copied source files to my windows host all files had CRLF endings, that is if i used the TC sftp plugin. If i use WinSCP the samba4 files have LF endings.

So on the PC with TC installed, the sftp plugin silently converted LF only files to CRLF, but not mixed CRLF/LF files, thats why the org. mixed .patch file showed up "correctly". On the other PC with WinSCP no conversions ever happens. It never occurred to me that a sftp operation could change line endings, but after running sha1sum i noticed different sha1 hashes between linux/windows source files.

Now i just have to figure out how to handle softether's actual CRLF endings.

PS: Just found the actual changelog of the TC sftp plugin:

Supported functions:
convert text files to/from Unix format during transfers
#7

Heh, that's a very interesting feature. :slight_smile:
Glad that you managed to figure it out.

#8

oki switching to https://github.com/billziss-gh/sshfs-win solves those cross-platform EoL problems and keeps the convenience for anyone interested.

Additionally .gitattributes can be set to: * -text this way no conversion ever happens and files are added with whatever line-ending is present or even a mix. This is because many windows git installations use core.autocrlf true and the above suppresses the default conversion feature.

PS: I ended up with this general cross-platform git/repo setup after extensive testing.
My repos get:
.gitattributes

* text=auto eol=lf
*.patch -text

git itself gets: git config --global core.autocrlf false

This disables git automatic conversion logic, so i rather handle this myself if needed without getting confused again. For my own feed repo i force LF on anything except .patch files, so i don't accidentally add a CRLF file, while still retaining the mixed .patch files as needed.

#9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.