fix #6540: utf16(s) for binary data, etc.

[stevengj]

This patch adds utf16(s) for binary data, an is_valid_utf16(s) function analogous to is_valid_utf8, and restricts IOBuffer(s::String) to ByteStrings as discussed in #6540.

[stevengj]

@vtjnash, @nolta, does this look good to you?

[JeffBezanson]

lgtm

[vtjnash]

i'm not that well versed in UTF16 strings.

should the convert(::Type{UTF16String}, data::Array{Uint16}) function also check for the BOM?

[vtjnash]

is this copy necessary?

[stevengj]

Not strictly necessary, but my thinking was that whether or not this function makes a copy of the data should not depend on the contents of the data.

[ivarne]

Should we have a converting copy?

eg.

function copy(T::Type, x)
    a = convert(T, x)
    if a === x
        a = copy(a)
    end
    a
end

[stevengj]

You could always do convert(T, copy(x)) if you need to force a copy in a particular case. In any case, === won't work here since I don't think the UTF16String will compare equal to an Array{Uint16} even if they share the same data.

[stevengj]

Note that if the user has an array containing a UTF-16 string in the native byte order, with no BOM, and it is important to convert it to a string without making a copy, she can always force that behavior by doing utf16(reinterpret(Uint16, array)). So, I don't think the copy here could pose a big problem in practice.

[stevengj]

@vtjnash, my feeling was that if the user supplies an Array{Uint16} directly, then we should assume that they are really Uint16s, i.e. that they are in the native byte order, and that they want a string from the raw data. (A BOM, if any, would then be treated as a zero-width non-breaking space.)