Multiple computers writing to a domain hosted file

Ask your Windows PowerShell-related questions, including questions on cmdlet development!
Forum rules
Do not post any licensing information in this forum.

Any code longer than three lines should be added as code using the 'Select Code' dropdown menu or attached as a file.
PaschalIT
Posts: 32
Joined: Tue Apr 02, 2019 4:37 pm

Multiple computers writing to a domain hosted file

Post by PaschalIT » Wed Apr 03, 2019 12:43 pm

Hi everyone!
New to the forums, so let me know if I do anything wrong.

I'm writing a script for my company which runs on every necessary computer on our domain at once, updating a piece of software if needed, and then writes that software's version to a file on a network drive. The issue I'm encountering is that since they're all running the same task, the computers all finish the process at about the same time and try to write to the file at the same time, effectively overwriting or nulling each others' efforts, since Powershell can't replace text in a file without completely rewriting it (to my knowledge).

So my dilemma is, is there a way to "lock" the file a computer is working on it and prevent other computers from accessing it until that one is done? I don't have a problem with the task continuing to run in the background until they get access to the file.

User avatar
mxtrinidad
Site Admin
Posts: 303
Joined: Sun Mar 03, 2013 12:42 pm

Re: Multiple computers writing to a domain hosted file

Post by mxtrinidad » Wed Apr 03, 2019 12:49 pm

We need more information!
It will help if you tell us what cmdlet you're using to write the file to the share drive.
:)

User avatar
jvierra
Posts: 13623
Joined: Tue May 22, 2007 9:57 am
Contact:

Re: Multiple computers writing to a domain hosted file

Post by jvierra » Wed Apr 03, 2019 1:11 pm

Simultaneous write to a file cannot be done by normal means. You will need to open the file as shared write so the locking is cooperative. This is normal Windows behavior.

A file in Windows can only be opened as "create" or "Append". When appending the writes always get added to the existing file at the end. TO do an update you would have to open a stream in binary mode and seek to the location to change. If the write buffer is not resized then the edit will not require a complete rewrite. If the edit is an insert or delete of bytes then the file will be rewritten. This is only an issue with very large files.

PowerShell CmdLets only allow new files or Append.

User avatar
jvierra
Posts: 13623
Joined: Tue May 22, 2007 9:57 am
Contact:

Re: Multiple computers writing to a domain hosted file

Post by jvierra » Wed Apr 03, 2019 1:14 pm

A quick fix to your issue will be to have each computer write to a separate file. If you make the files CSV files then it will be easy to concatenate all files and sort/filter the results.


# catenate
$csv = Import-Csv *.csv

# all records from all system for today
$csv = Import-Csv *.csv | sort Mydate | Where{ $_.Mydate -gt [datetime]::Today }

PaschalIT
Posts: 32
Joined: Tue Apr 02, 2019 4:37 pm

Re: Multiple computers writing to a domain hosted file

Post by PaschalIT » Wed Apr 03, 2019 1:18 pm

mxtrinidad wrote:
Wed Apr 03, 2019 12:49 pm
We need more information!
It will help if you tell us what cmdlet you're using to write the file to the share drive.
:)
Sorry! I'm using get-content and set-content to read and write. My code reads in the file, checks to see if that computer's version is already present, appends it if it's not there, or replaces the version if that computer is already present.

Currently it seems like the reads and writes are stepping on each others' toes, as if I watch the file while the scripts are running the entire text occasionally disappears, or values get lost in the mix.

My current code for finding and replacing is:

Code: Select all

if ($filetext -match $hostname) # Find comp name in file
{
$filetext -replace $find, $replace | Set-Content $file # Replace current data with new data
}
else
{
Add-Content $file $replace # Append data if comp name not found
}

User avatar
mxtrinidad
Site Admin
Posts: 303
Joined: Sun Mar 03, 2013 12:42 pm

Re: Multiple computers writing to a domain hosted file

Post by mxtrinidad » Wed Apr 03, 2019 1:44 pm

I would suggest to use PowerShell existing commands as it was designed to such purpose.

As JVierra shown one way using the Import-Csv cmdlet, you can also try using the Out-File cmdlet which has the parameters '-NoClobber' which will not override the file, and in combination with the '-Append' the output is appended to the existing file.

This is a much simple way to collect information.

Try them both to see which give you the better result!
:)

User avatar
jvierra
Posts: 13623
Joined: Tue May 22, 2007 9:57 am
Contact:

Re: Multiple computers writing to a domain hosted file

Post by jvierra » Wed Apr 03, 2019 1:46 pm

PaschalIT wrote:
Wed Apr 03, 2019 1:18 pm

My current code for finding and replacing is:

Code: Select all

if ($filetext -match $hostname) # Find comp name in file
{
$filetext -replace $find, $replace | Set-Content $file # Replace current data with new data
}
else
{
Add-Content $file $replace # Append data if comp name not found
}
This will not work from multiple computers. If you open the file and then append the item you will not lose data except when the open fails which you will have to detect as an exception and retry until you are successful.

Code: Select all

if(Get-Content $filename |Where {$_ -match $computername }){
       # computer record exists
}else{
      Try{
           $text | Out-File $filename -Append -ErrorAction Stop
      }
      Catch{
            # file in use
      }
}
Trying to rewrite the complete file each time will only make the situation worse. You will also lose all write between the time you test the file and when you rewrite it. PowerShell CmdLets do not support advanced locking. For this you will need to use a file stream.

To have an updateable record store we would use a database file like MSAccess and use the Access data provider (ACE). This provides all locking semantics and allows individual records to be updated with no loss.

User avatar
jvierra
Posts: 13623
Joined: Tue May 22, 2007 9:57 am
Contact:

Re: Multiple computers writing to a domain hosted file

Post by jvierra » Wed Apr 03, 2019 1:49 pm

"NoClobber" will not avoid an exception if the file is in transition. Another process can intercept and the NoClobber will not preserve the file. It can work under some circumstances but not when you try to change a single line in the file. Updates are not cooperative in PS. Only an Append will work.

User avatar
jvierra
Posts: 13623
Joined: Tue May 22, 2007 9:57 am
Contact:

Re: Multiple computers writing to a domain hosted file

Post by jvierra » Wed Apr 03, 2019 1:54 pm

Here is a good blurb on what "NoClobber" actually does.
noclobber.png
noclobber.png (36.17 KiB) Viewed 1020 times

PaschalIT
Posts: 32
Joined: Tue Apr 02, 2019 4:37 pm

Re: Multiple computers writing to a domain hosted file

Post by PaschalIT » Wed Apr 03, 2019 1:56 pm

I've seen a few posts and also heard from my colleagues that a database is the best option, so I will try to look into that for the long run. For now, it sounds like separate files are my best bet, so I think that's what I'll go with for the moment. Thank you everyone for your help! (I'll be back if I can't get it to work :P )

Locked