Saved ps1 files appear corrupted

Support for all customers who have purchased a SAPIEN PowerShell Studio product license. This forum does not offer a response time guarantee.
Forum rules
DO NOT POST SUBSCRIPTION NUMBERS, LICENSE KEYS OR ANY OTHER LICENSING INFORMATION IN THIS FORUM.
Only the original author and our tech personnel can reply to a topic that is created in this forum. If you find a topic that relates to an issue you are having, please create a new topic and reference the other in your post.

Any code longer than three lines should be added as code using the 'Select Code' dropdown menu or attached as a file.
User avatar
Squidge123
Posts: 4
Joined: Fri Nov 25, 2016 6:59 am

Saved ps1 files appear corrupted

Post by Squidge123 » Fri Oct 05, 2018 8:39 am

**BUG**
After working for some time with certain powershell files, after copying the files to remote machines for execution, sometimes I will get syntax/parsing errors such as below.
  1. At E:\WGT\WGT_GetFarmsByServer2.ps1:577 char:108
  2.  
  3. + ... ypeInformation -Depth 10).Save("$PSScriptRoot\WGT_FarmsByServer.xml")
  4.  
  5. +                                                                        ~~
  6.  
  7. The string is missing the terminator: ".
  8.  
  9. At E:\WGT\WGT_GetFarmsByServer2.ps1:281 char:2
  10.  
  11. +     {
  12.  
  13. +     ~
  14.  
  15. Missing closing '}' in statement block or type definition.
  16.  
  17. At E:\WGT\WGT_GetFarmsByServer2.ps1:580 char:1
  18.  
  19. +
  20.  
  21. The Try statement is missing its Catch or Finally block.
  22.  
  23. At E:\WGT\WGT_GetFarmsByServer2.ps1:83 char:16
  24.  
  25. + $MainJobScript={
  26.  
  27. +                ~
  28.  
  29. Missing closing '}' in statement block or type definition.
  30.  
  31.     + CategoryInfo          : ParserError: (:) [], ParseException
  32.  
  33.     + FullyQualifiedErrorId : TerminatorExpectedAtEndOfString
Even taking out the last edited portion of the script does not fix the problem. Quite a few times I have literally given up and reverted to a previously saved version. Today I realized that the underlying issue is due to the file being saved without BOM. This is very strange, as the file stays open in the editor and can be saved and copied many times and successfully executed. Every now and then it seems to change the encoding and breaks. I am not sure if this behaviour has anything to do with Version Recall running?

I have spent many hours trying to find syntactical errors that do not exist .. opening the file in Notepad++ and changing the encoding to 'Encode in UTF-8-BOM' fixes the issue, but the file originally was set that way.


Product, version and build: PowerShell Studio 2018 v5.5.154
32 or 64 bit version of product: 64
Operating system: Windows 10
32 or 64 bit OS: 64

*** Please add details and screenshots as needed below. ***

DO NOT POST SUBSCRIPTIONS, KEYS OR ANY OTHER LICENSING INFORMATION IN THIS FORUM

User avatar
Alexander Riedel
Posts: 6684
Joined: Tue May 29, 2007 4:43 pm

Re: Saved ps1 files appear corrupted

Post by Alexander Riedel » Fri Oct 05, 2018 10:40 am

I am not sure what would cause the encoding to change, I will talk to the team as soon as possible. VersionRecall really should not change the encoding either.
The ONLY thing that I could think of is that maybe signing the file could mess with it. Applying a digital signature is done by actually modifying the file by means of a Windows API.
While we have no real control over what that does or does not do, it is the only place where we cannot check the code to find out.
So please let us know if you have code signing set up in PowerShell Studio and/or VersionRecall so we know if we have to investigate that angle.
Alexander Riedel
SAPIEN Technologies, Inc.

User avatar
Squidge123
Posts: 4
Joined: Fri Nov 25, 2016 6:59 am

Re: Saved ps1 files appear corrupted

Post by Squidge123 » Mon Oct 08, 2018 4:12 am

No signing has ever been applied to scripts. This has affected a number of my scripts going back maybe 18 months and always seems to be when they are pretty complex involving codeblocks and jobs (which themselves make syntax checking a pain). After trying unsuccessfully to find the latest reported syntactical error, I downloaded and setup the 'PSScriptAnalyzer'. Running that against my script showed in the first couple of lines the BOM issue.

TBH it makes no sense to me either, 99% of the time a save, copy file to remote platform and execute is fine. When this does happen though, the local file is what is affected and if you are not aware of the BOM issue, further attempts at fixing syntax are unsuccessful. My normal recourse was to recover an old version of the file .. though not through version recall as sometimes that is affected too.

User avatar
Alexander Riedel
Posts: 6684
Joined: Tue May 29, 2007 4:43 pm

Re: Saved ps1 files appear corrupted

Post by Alexander Riedel » Mon Oct 08, 2018 5:16 am

In the absence of digital signing, VersionRecall will store the file as is, meaning if it is somehow corrupted, a version of said corrupted file is created. So that actually makes sense, even though it is not what you want. You would likely have to examine the file's history more closely to find the point in time where this happened.

This sounds a little bit like another BOM (Byte Order Mark) issue a user just discovered. He was using a Copy command to concatenate scripts, which introduced BOMs in the middle of the file.
Needless to say, many tools cannot handle this kind of scenario.
What I am trying to say, UTF and unicode files with a BOM are not text files in the old ASCII type of sense. So many tools and commands that worked on ASCII files may wreak havoc on UTF or unicode files.

To really examine the issue it would be best if we had one of these corrupted files.
Can you attach it here or upload it here: https://www.sapien.com/support/upload ?
While we may not be able to find out what is causing it, we might be able to detect and fix the condition if encountered.
The next build of PrimalScript for example now detects and removes errant BOMs in the middle of a file. Maybe we can do something similar here.
Alexander Riedel
SAPIEN Technologies, Inc.

User avatar
Squidge123
Posts: 4
Joined: Fri Nov 25, 2016 6:59 am

Re: Saved ps1 files appear corrupted

Post by Squidge123 » Tue Oct 09, 2018 9:55 am

Unfortunately, all those files that were affected I recovered (and overwrote) the file with a backup. If it happens again then I will post it to this thread.

I guess this is not a game changer now that I realize what is happening it is easily rectified. But it is probably good to flag to the wider community - after all, I would not want other users chasing their tails for hours trying to find a syntactical error that does not exist...

User avatar
Alexander Riedel
Posts: 6684
Joined: Tue May 29, 2007 4:43 pm

Re: Saved ps1 files appear corrupted

Post by Alexander Riedel » Tue Oct 09, 2018 10:17 am

Yeah, I agree, it can be very frustrating to hunt an issue like that down. So please send a file when it happens again.
Alexander Riedel
SAPIEN Technologies, Inc.