Code pages, batch files and localization

PrimalScript 2011 treats everything as Unicode files, so when you open an ASCII file it will be converted to Unicode and then converted back to ASCII when saving. Any conversion to and from Unicode requires a code page, which tells the converter which characters get converted to what. The default Windows code page is 1252 (http://en.wikipedia.org/wiki/Windows-1252) and usually this back and forth conversion goes unnoticed.

An ASCII VBScript file for example which gets saved with this conversion will not produce any problem, as CScript and WScript use the same code page when writing to the console. So basically the characters you see in the script will be the same that you see in the output.

Batch and CMD files are a different story. The default code page for a CMD shell is 437, at least here in the US. So if you save a file and the conversion uses code page 1252 you will get some unexpected characters.

Consider the following batch file:

image

Running this batch file in a normal cmd shell produces the following output:

image

As you can see the accented characters are replaced by letters from the Greek alphabet.

CMD.EXE expects command and batch files to be encoded with the current code page used in the console. So with PrimalScript 2011 Service Build 141 files with the extensions .BAT or .CMD will be saved and loaded using the OEM character code page and this is no longer any issue.

image

If you are only concerned about files on your system you could stop reading here. But if you need to create files that are using a code page different from the default on your system, continue on.

PrimalScript 2011 Service Build 141 also introduces a way to force the encoding of a file to a specific code page:
Simply type “REM %ForceCodePage%=437” in your batch file and save the file.

image

Now PrimalScript uses the specified code page when saving the file and, even better, with the comment in place, PrimalScript knows which code page to use when loading the file and uses the proper conversion.

Last but not least, with Service Build 141 we made it easier to change the encoding of an open file. If you remember, we recently added a status bar indicator showing the encoding of the active file. With this build now you can just right click on this status bar indicator and select the encoding.

image