Java String contains extraneous data

I have an input which is in UTF16LE encoding. By the time this input reaches my code its been through a FileInputStream encased in a FileReader encased in a LineNumberReader.

The first line read gives a string like:

"1 piece of data like a string"

However, looking into this String the value will be something along the lines of:

[, 1, p, i, ...]

Notice the empty element to start.

No this string is passed through a couple of functions here and there, converted to Object and basically is being put through its paces. At a certain point, what shoul开发者_如何转开发d only be the first part of the String (the 1 or in my case any number including with decimals) is passed to a function which has to parse it to an actual Number.

The content of this String appears to be "1" but in the value it says:

[, 1, p, i, ...]

so the whole string is still in there.

In any case it returns a ParseException and I print the unparseable number to an exception messages and my logging tels me that "1" is an unparseable number.

The real problem appears to be the leading empty element as subsequent lines show similar behavior except for the leading empty element and they parse.

---

A String (at least the implementation in the OpenJDK) stores a char[], an offset and a count. The actual content of the String are the characters in the char[] with the indices offset up to offset+count.

That means that the char[] can hold more characters than the String actually represents.

This is done in order to be able to share char[]s between different String instances.

For example, if you have a String with the value foobar and you call .substring(3) on it, then the resulting String will represent bar, but they may actually reference the same char[]. The second String will just have an offset that 3 bigger than the originals String and a count that's 3 smaller.

All of this only works because String objects are truly immutable: since no single String will ever modify it's char[] in any way, it's perfectly safe for them to share it.

This means that inspecting a String object in the debugger might give a false impression. Therefore the safest thing to do if you want to inspect a String character-for-character is to call either toCharArray() or to call charAt() in a loop.

---

I think I have the answer. The encoding was not UTF16LE. It was set to UTF16LE by an auto char detection algorithm. The encoding is utf16 with a BOM. However since the various classes thought the encoding was UTF16LE they were not getting rid of the BOM which shouldn't be there in the LE version.