Import-Csv - A participant is already presenting a problem
I need to combine multiple CSV files into one. Each CSV has a header. One of the column headings is identical. Ideally, the final file (all_out.csv) should have one header.
I am running the PowerShell code:
Import-Csv out_1_result.csv,out_2_result.csv,out_3_result.csv,out_4_result.csv,out_5_result.csv,out_6_result.csv,out_7_result.csv,out_8_result.csv,out_9_result.csv,out_10_result.csv,out_11_result.csv,out_12_result.csv,out_13_result.csv,out_14_result.csv,out_15_result.csv,out_16_result.csv,out_17_result.csv,out_18_result.csv,out_19_result.csv,out_20_result.csv,out_21_result.csv,out_22_result.csv,out_23_result.csv,out_24_result.csv,out_25_result.csv,out_26_result.csv,out_27_result.csv,out_28_result.csv | Export-Csv all_out.csv -NoType
and I get the error
Import-Csv: The "URL" member is already present.
Is there a way to ignore / fix this?
source to share
One of the column headings is identical
Does this mean that each CSV has two header columns 'URL'? Import-Csv
creates objects where each title becomes a property name, eg. @{Id=10; Url='example.com'}
and the same name will collide again.
There is no way for this to work cleanly without modifying the csv files as there is no way to say "use different column names" and also "skip the header line" just with the Import-Csv cmdlet.
The lightest change I can think of is to drop the header line from each one, for example:
$CsvFiles = 'out_1_result.csv','out_2_result.csv','out_3_result.csv','out_4_result.csv','out_5_result.csv','out_6_result.csv','out_7_result.csv','out_8_result.csv','out_9_result.csv','out_10_result.csv','out_11_result.csv','out_12_result.csv','out_13_result.csv','out_14_result.csv','out_15_result.csv','out_16_result.csv','out_17_result.csv','out_18_result.csv','out_19_result.csv','out_20_result.csv','out_21_result.csv','out_22_result.csv','out_23_result.csv','out_24_result.csv','out_25_result.csv','out_26_result.csv','out_27_result.csv','out_28_result.csv'
$NewFileNames = $CsvFiles | ForEach-Object {
$NewFileName = $_ + "_noheader.csv"
Get-Content $_ | Select-Object -Skip 1 | Set-Content $NewFileName -Encoding UTF8
$NewFileName # write new name to output stream
}
And then when they don't have a header line, import them all and give the header line as a parameter
Import-Csv -Path $NewFileNames -Header 'Col1', 'Col2', 'Url1', 'Url2' | Export-Csv ...
source to share