Import to SQL Server with format file and bulk insert

Hi everyone, I have a tricky problem using a command BULK INSERT

when trying to import data from a text file.

I have found many articles and examples on the Internet about importing using the program BULK INSERT

or BCP

, but they do not help me.

Problem:
I am doing an export from Oracle to a column {#}

delimited <#>

and row delimited text file and import it into SQL Server.

Table (SQL Server):

CREATE TABLE my_DATA
(
ID_PK     NUMERIC(30)  NOT NULL ,
BEGIN_TIME    DATETIME  NULL ,
END_TIME      DATETIME  NULL
);

      

for Oracle:

CREATE TABLE my_DATA
(
ID_PK     NUMBER(30)  NOT NULL ,
BEGIN_TIME    TIMESTAMP  NULL ,
END_TIME      TIMESTAMP  NULL
);

      

Delimited file:

ID_PK{#}BEGIN_TIME{#}END_TIME<#>296167{#}01/01/2012 01:30:00.000{#}01/01/2012 02:00:00.000<#>296178{#}01/01/2012 02:00:00.000{#}01/01/2012 02:30:00.000<#>

      

File format:

9.0                         
3                           
1   SQLNUMERIC  0   19  {#} 1   ID_PK   ""
2   SQLDATETIME 0   8   {#} 2   BEGIN_TIME  ""
3   SQLDATETIME 0   8   <#> 3   END_TIME    ""

      

So when I used the command:

BULK INSERT my_DATA 
FROM 'D:\my_DATA.txt' 
WITH 
(CODEPAGE = '1251',
 FIELDTERMINATOR = '{#}', 
 FIRSTROW = 2, 
 ROWTERMINATOR = '<#>' );

      

This works, but when I try to use the format file it doesn't work:

BULK INSERT my_DATA 
FROM 'D:\my_DATA.txt' 
WITH (CODEPAGE = '1251',
      FORMATFILE ='D:\format_file.txt');

      

Mistake:

Bulk load data conversion failed (type mismatch or invalid character for specified code page) for line 2, column 2 (start_time).

I was looking for this problem, trying to change the date and time to smalldate, trying to change the length to 23 or 24. It won't work. So I am trying to import another table without date, I am using numeric columns and char columns, but faced the same problem with numeric column:

Bulk load data conversion error (type mismatch or invalid character for specified code page) for line 2, column 1 (id_pk).

Table:

CREATE TABLE unit_table
(
id_pk  NUMERIC(30)  NOT NULL ,
name             NVARCHAR(200)  NULL ,
name_full        NVARCHAR(200)  NULL ,
CONSTRAINT  PK_unit_table_2C1 PRIMARY KEY (bule_biz_unit_level_id_pk)
);

      

file format:

9.0
3
1       SQLNUMERIC    0       1      "{#}"      1     id_pk                              ""
2       SQLNCHAR      0       11     "{#}"      2     name                                              ""
3       SQLNCHAR      0       11     "{#}"      3     name_full                                         ""

      

The data file only contains 3 lines (the bulk insert command is the same):

ID_PK{#}NAME{#}NAME_FULL<#>1{#}factory{#}factory<#>2{#}station{#}station<#>

      

I wonder if the file only contains 2 lines:

ID_PK{#}NAME{#}NAME_FULL<#>1{#}factory{#}factory<#>

      

Bulk insert result:

(0 lines (lines) affected)

I am also trying to make an example http://msdn.microsoft.com/en-us/library/ms178129.aspx but am running into an error:

Unable to bulk load because file "D: \ myTest.txt" could not be read. Operating system error code (null).

And lastly, I tried to use this with a program BCP

and a buggy face too.

Can anyone help me with my problem or give me at least one working example with bulk insert and formatted files.

PS I am using MS SQL Server 2005 updated to the latest version. OS - Windows 7 x64.

+3


source to share


2 answers


Thanks everyone. But the problem was that MS SQL Server requires SQLCHAR type when importing from a text file, and it doesn't matter what type of column is in the database. In my problem, the solution replaces any column types like SQLNUMERIC, SQLDATETIME with SQLCHAR in the format file.



+3


source


This is not what you asked for, but I tested it and should get you where you need to go. As far as I know, the following needs to be done from the server that hosts the SQL Server. Perhaps you can set up a scheduled task or something more complex in the future.

First you need to create a format file. In this example, I have placed different data types to show how they should be used.

<?xml version="1.0"?>
<BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <RECORD>
  <FIELD ID="1" xsi:type="CharTerm" TERMINATOR="{#}" MAX_LENGTH="24"/>
  <FIELD ID="2" xsi:type="CharTerm" TERMINATOR="{#}" MAX_LENGTH="35" COLLATION="SQL_Latin1_General_CP1_CI_AS"/>
  <FIELD ID="3" xsi:type="CharTerm" TERMINATOR="{#}" MAX_LENGTH="12"/>
  <FIELD ID="4" xsi:type="CharTerm" TERMINATOR="{#}" MAX_LENGTH="41"/>
  <FIELD ID="5" xsi:type="CharTerm" TERMINATOR="<#>" MAX_LENGTH="30"/>
 </RECORD>
 <ROW>
  <COLUMN SOURCE="1" NAME="RpPrdEndDt" xsi:type="SQLDATETIME"/>
  <COLUMN SOURCE="2" NAME="ContrCustNum" xsi:type="SQLVARYCHAR"/>
  <COLUMN SOURCE="3" NAME="ClimateZone" xsi:type="SQLINT"/>
  <COLUMN SOURCE="4" NAME="BCMsrQty" xsi:type="SQLDECIMAL" PRECISION="16" SCALE="3"/>
  <COLUMN SOURCE="5" NAME="RMsrPricePerUnit" xsi:type="SQLMONEY"/>
 </ROW>
</BCPFORMAT>

      

Second, you need to create an SQL statement INSERT

for your data:



INSERT INTO TableName
            (RpPrdEndDt,
             ContrCustNum,
             ClimateZone,
             BCMsrQty,
             RMsrPricePerUnit)
SELECT RpPrdEndDt,
       ContrCustNum,
       ClimateZone,
       BCMsrQty,
       RMsrPricePerUnit
FROM   OPENROWSET(BULK 'C:\ImportFileName.csv',
                  FORMATFILE='C:\FormatFile.xml' ) AS t1; 

      

And thirdly, if you want to run this from the command line like BCP, you can use this:

sqlcmd -S ServerName\InstanceName -i C:\SavedSQLFile.sql

      

+1


source







All Articles