StreamWriter - The process cannot access the file because it is being used by another process
I am trying to rewrite the Add-Content
script as version StreamWriter
because the file is ~ 140MB and is Add-Content
too slow.
This is my version Add-Content
that goes through each line until it finds a header line starting with FILE|
and creates a new file with the filename of the second delimited (piped) value on that line. Add-Content
works as intended but is very slow. This takes 35-40 minutes:
Param(
[string]$filepath = "\\fileserver01\Transfer",
[string]$filename = "sourcedata.txt"
)
$Path = $filepath
$InputFile = (Join-Path $Path $filename)
$Reader = New-Object System.IO.StreamReader($InputFile)
while (($Line = $Reader.ReadLine()) -ne $null) {
if ($Line -match 'FILE\|([^\|]+)') {
$OutputFile = "$($matches[1]).txt"
}
Add-Content (Join-Path $Path $OutputFile) $Line
}
I researched what StreamWriter
should be faster. Here is my attempt, but I am getting the error
The process cannot access the file '\ fileserver01 \ Transfer \ datafile1.txt' because it is in use by another process.
Param(
[string]$filepath = "\\fileserver01\Transfer",
[string]$filename = "sourcedata.txt"
)
$Path = $filepath
$InputFile = (Join-Path $Path $filename)
$Reader = New-Object System.IO.StreamReader($InputFile)
while (($Line = $Reader.ReadLine()) -ne $null) {
if ($Line -match 'FILE\|([^\|]+)') {
$OutputFile = "$($matches[1])"
}
$sw = New-Object System.IO.StreamWriter (Join-Path $Path $OutputFile)
$sw.WriteLine($line)
}
I guess it has something to do with using it in my loop.
Sample data:
FILE | datafile1 | 25/04/17 25044 | 0001 | 37339 | 10380 | TT75 25045 | 0001 | 37339 | 10398 | TT75 25046 | 0001 | 78711 | 15940 | TT75 FILE | datafile2 | 25/04/17 25047 | 0001 | 98745 | 11263 | TT75 25048 | 0001 | 96960 | 13011 | TT84 FILE | datafile3 | 25/04/17 25074 | 0001 | 57585 | 13639 | TT84 25075 | 0001 | 59036 | 10495 | TT84 FILE | datafile4 | 25/04/17 25076 | 0001 | 75844 | 13956 | TT84 25077 | 0001 | 17430 | 01111 | TT84
The desired result is 1 file per FILE|
heade line , using the second separator value as the filename.
source to share
You create the author inside the loop while
without closing it, so your code tries to reopen the already opened output file with each iteration. Close the existing writer and open a new one every time the filename changes:
while (($Line = $Reader.ReadLine()) -ne $null) {
if ($Line -match 'FILE\|([^\|]+)') {
if ($sw) { $sw.Close(); $sw.Dispose() }
$sw = New-Object IO.StreamWriter (Join-Path $Path $matches[1])
}
$sw.WriteLine($line)
}
if ($sw) { $sw.Close(); $sw.Dispose() }
Note that this assumes that you will not open the same file twice. If the same output file may appear multiple times in the input file, you need to open the file to append. In this case, replace
$sw = New-Object IO.StreamWriter (Join-Path $Path $matches[1])
from
$sw = [IO.File]::AppendText((Join-Path $Path $matches[1]))
Note also that the code does not handle errors (for example, the input file does not start on a line FILE|...
, the input file is empty, etc.). You can change this.
source to share