Javascript & # 8594; Upload CSV file encoded in ISO-8859-1 / Latin1 / Windows-1252
I put together a small tool to extract shipping data from Amazon CSV order data. it still works. here's a simple version as JS Bin: http://output.jsbin.com/jarako
To print shipping stamps / tags, I need a file to upload to Deutsche Post and other parcel services. I used a little function saveTextAsFile
I found on stackoverflow. Everything is fine so far. Special characters (äöüß ...) were displayed incorrectly in the output text area or loaded files.
All these German post / post services sites only accept latin1 / iso-8859-1 encoded files for download. But my uploaded file is always utf-8. If I load it all the special characters (äöüß ...) go wrong.
How can I change this? I still searched a lot. I tried ie:
Setting tool encoding to iso-8859-1:
<META http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
But the result: I now have the wrong special characters still in the output text area and in the loaded file. If I upload it to the site, I still get more wrong characters. Also if I check the encoding in the CODA editor it still says that the downloaded file is UTF-8.
The function saveTextAsFile
uses var textFileAsBlob = new Blob([textToWrite], {type:'text/plain'});
. Maybe there are ways to set the encoding for the download there ??
function saveTextAsFile()
{
var textToWrite = $('#dataOutput').val();
var textFileAsBlob = new Blob([textToWrite], {type:'text/plain'});
var fileNameToSaveAs = "Brief.txt";
var downloadLink = document.createElement("a");
downloadLink.download = fileNameToSaveAs;
downloadLink.innerHTML = "Download File";
if (window.webkitURL != null)
{
// Chrome allows the link to be clicked
// without actually adding it to the DOM.
downloadLink.href = window.webkitURL.createObjectURL(textFileAsBlob);
}
else
{
// Firefox requires the link to be added to the DOM
// before it can be clicked.
downloadLink.href = window.URL.createObjectURL(textFileAsBlob);
downloadLink.onclick = destroyClickedElement;
downloadLink.style.display = "none";
document.body.appendChild(downloadLink);
}
downloadLink.click();
}
Anyway, there must be a way to load the files in a different encoding as the site uses itself. Amazon site where I download CSV file from UTF-8 encoded. But the uploaded CSV file from there is Latin1 (iso-8859-1) if I check it in CODA ...
source to share
LOOK FOR UPDATE FOR REAL SOLUTION!
Since I received no answer, I searched more and more. It looks like there is no SOLUTION in Javascript. Each I'v test load generated in javascript was UTF-8 encoded. It looks like Javascript is only for UNICODE / UTF-8, or a different encoding (maybe) only applies if the data is sent again using the old HTTP transport. But for Javascript that runs on the client, no additional HTTP transport happens because the data is still on the client.
Now I have helped me create a small PHP Script on my server to which I post data via GET or POST request. It converts the encoding to latin1 / ISO-8859-1 and downloads it as a file. This is an ISO-8859-1 file with properly encoded special characters that I can upload to the specified post and post services and it looks good.
latin-download.php: (VERY IMPORTANT to save the PHP file itself also in ISO-8859-1 for it to work!)
<?php
$decoded_a = urldecode($_REQUEST["a"]);
$converted_to_latin = mb_convert_encoding($decoded_a,'ISO-8859-1', 'UTF-8');
$filename = $_REQUEST["filename"];
header('Content-Disposition: attachment; filename="'.$filename.'"; content-type: text/plain; charset=iso-8859-1;');
echo $converted_to_latin;
?>
in my javascript code i am using:
<a id="downloadlink">Download File</a>
<script>
var mydata = "this is testdata containing äöüß";
document.getElementById("downloadlink").addEventListener("click", function() {
var mydataToSend = encodeURIComponent(mydata);
window.open("latin-download.php?a=" + mydataToSend + "&filename=letter-max.csv");
}, false);
</script>
for large amounts of data you have to switch from GET to POST ...
UPDATE 08-Feb-2016
Six months later, I found a solution in PURE JAVASCRIPT. Using inexorabletash / text-encoding . This is a polyfill for the Encoding Living Standard . The standard includes decoding for older encodings such as latin1 ("windows-1252"), but it prohibits encoding into these older encoding types. So if you use a browser implemented function window.TextEncoder
, it only offers UTF encoding. BUT, polyfill solution offers a legacy mode that ALLOW also encodes into older encodings such as latin1.
I use it like this:
<!DOCTYPE html>
<script>
// 'Copy' browser build in TextEncoder function to TextEncoderOrg (because it can NOT encode windows-1252, but so you can still use it as TextEncoderOrg() )
var TextEncoderOrg = window.TextEncoder;
// ... and deactivate it, to make sure only the polyfill encoder script that follows will be used
window.TextEncoder = null;
</script>
<script src="lib/encoding-indexes.js"></script> // needed to support encode to old encoding types
<script src="lib/encoding.js"></script> // encording polyfill
<script>
function download (content, filename, contentType) {
if(!contentType) contentType = 'application/octet-stream';
var a = document.createElement('a');
var blob = new Blob([content], {'type':contentType});
a.href = window.URL.createObjectURL(blob);
a.download = filename;
a.click();
}
var text = "Es wird ein schöner Tag!";
// Do the encoding
var encoded = new TextEncoder("windows-1252",{ NONSTANDARD_allowLegacyEncoding: true }).encode(text);
// Download 2 files to see the difference
download(encoded,"windows-1252-encoded-text.txt");
download(text,"utf-8-original-text.txt");
</script>
The encoding-indexes.js file is about 500kb in size because it contains all the encoding tables. Since I only need windows-1252 encoding, I have removed other encodings in this file for my use. so now only 632 bytes are left.
source to share
You cannot force the web server to send you data in a given encoding, just ask it politely. Your approach to simply converting to the format you want is the right way to go.
If you want to avoid a PHP script, you might be lucky to specify the encoding as a parameter when creating Blob
:
var textFileAsBlob = new Blob(textToWrite, {
type: 'text/plain;charset=ISO-8859-1',
encoding: "ISO-8859-1"
});
See Setting blob encoding in Google Chrome for details .
source to share