开发者

Charset conversion from XXX to utf-8, command line

开发者 https://www.devze.com 2022-12-28 03:13 出处:网络
I have a bunch of text files that are开发者_如何学编程 encoded in ISO-8851-2 (have some polish characters). Is there a command line tool for linux/mac that I could run from a shell script to convert t

I have a bunch of text files that are开发者_如何学编程 encoded in ISO-8851-2 (have some polish characters). Is there a command line tool for linux/mac that I could run from a shell script to convert this to a saner utf-8?


Use iconv, for example like this:

iconv -f LATIN1 -t UTF-8 input.txt > output.txt

Some more information:

  • You may want to specify UTF-8//TRANSLIT instead of plain UTF-8. To quote the manpage:

    If the string //TRANSLIT is appended to to-encoding, characters being converted are transliterated when needed and possible. This means that when a character cannot be represented in the target character set, it can be approximated through one or several similar looking characters. Characters that are outside of the target character set and cannot be transliterated are replaced with a question mark (?) in the output.

  • For a full list of encoding codes accepted by iconv, execute iconv -l.

  • The example above makes use of shell redirection. Make sure you are not using a shell that mangles encodings on redirection – that is, do not use PowerShell for this.


recode latin2..utf8 myfile.txt

This will overwrite myfile.txt with the new version. You can also use recode without a filename as a pipe.


GNU 'libiconv' should be able to do the job.

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号