Read csv with double quotes with lumenwork csv reader
I am reading a csv file using Lumenworks csv reader. Below is an example entry
"001-0000265-003"|"Some detail"|"detal1"|"detail2"|"detal3"|"detail4"|"detail5"|"detail6"
I created a class with below constructor to read this file
using (var input = new CsvReader(stream, true, '|'))
{
//logic to create an xml here
}
This works great when there are no double quotes inside the parts. But when scenarios like this are like
"001-0000265-003"|"Some " detail"|"detal1"|"detail2"|"detal3"|"detail4"|"detail5"|"detail6"
the reader throws an exception
An unhandled exception of type 'LumenWorks.Framework.IO.Csv.MalformedCsvException' occurred in LumenWorks.Framework.IO.dll
So, I used the CsvReader constructor which takes 7 arguments,
CsvReader(stream, true, '|', '"', '"', '#', LumenWorks.Framework.IO.Csv.ValueTrimmingOptions.All))
But still I am getting the same error. Please submit any suggestions.
I am reading a complex file like this:
"001-0000265-003"|"ABC 33"X23" CDE 32'X33" AAA, BB'C"|"detal1"|"detail2"|"detal3"|"detail4"|"detail5"|"detail6"
source to share
I tested it with your sample data and it is quite difficult to fix this wrong line (from Catch
-block for example ). So I would not use quotes, but instead just use the pipe separator and remove "
later via csv[i].Trim('"')
.
Here's a method that parses the file and returns all lines:
private static List<List<string>> GetAllLineFields(string fullPath)
{
List<List<string>> allLineFields = new List<List<string>>();
var fileInfo = new System.IO.FileInfo(fullPath);
using (var reader = new System.IO.StreamReader(fileInfo.FullName, Encoding.Default))
{
Char quotingCharacter = '\0'; // no quoting-character;
Char escapeCharacter = quotingCharacter;
Char delimiter = '|';
using (var csv = new CsvReader(reader, true, delimiter, quotingCharacter, escapeCharacter, '\0', ValueTrimmingOptions.All))
{
csv.DefaultParseErrorAction = ParseErrorAction.ThrowException;
//csv.ParseError += csv_ParseError; // if you want to handle it somewhere else
csv.SkipEmptyLines = true;
while (csv.ReadNextRecord())
{
List<string> fields = new List<string>(csv.FieldCount);
for (int i = 0; i < csv.FieldCount; i++)
{
try
{
string field = csv[i];
fields.Add(field.Trim('"'));
} catch (MalformedCsvException ex)
{
// log, should not be possible anymore
throw;
}
}
allLineFields.Add(fields);
}
}
}
return allLineFields;
}
Testing and output with a file that contains your sample data:
List<List<string>> allLineFields = GetAllLineFields(@"C:\Temp\Test\CsvFile.csv");
foreach (List<string> lineFields in allLineFields)
Console.WriteLine(string.Join(",", lineFields.Select(s => string.Format("[{0}]", s))));
[001-0000265-003],[Some detail],[detal1],[detail2],[detal3],[detail4],[detail5],[detail6]
[001-0000265-003],[Some " detail],[detal1],[detail2],[detal3],[detail4],[detail5],[detail6]
source to share