Check UTF-8 for text files that are uploaded without a selected character set
|Reported by:||Nicklas Nordborg||Owned by:||Nicklas Nordborg|
When uploading files to BASE the "Character set" option is typically set to
-n/a- unless the user manually changes the encoding (which is typically very rare).
This can cause problems later on since parsing the text file have to rely on the
defaultCharset setting in base.config which have a default value of ISO-8859-1.
While it is not possible to check every possible character encoding, checking if a file is UTF-8 is possible and I think we should implement a check in the file upload. If the file is detected to be UTF-8 compatible we set that, otherwise we keep the character set empty.