How do I add EOF to a file using Perl or Python?
I am trying to bulk insert data into SQL Express Express. When running bcp from the Windows XP command line, the following error appears:
C: \ temp> bcp in -T -f -S Starting copy ... SQLState = S1000, NativeError = 0 Error = [Microsoft] [SQL Native Client] Unexpected EOF encountered in BCP data-file 0 rows copied. Network packet size (bytes): 4096 Clock Time (ms.) Total: 4391
So the problem with EOF. How can I add the correct EOF character to this file using Perl or Python?
source to share
EOF is the end of the file. What has probably happened is that the file is incomplete; the software expects data, but there will be no more data.
These things happen when:
- export is aborted (exit dump during dump)
- when copying dumpfile, interrupt copying
- disk full during dump
such things.
By the way, although EOF is usually just the end of a file, there is an EOF character. This is used because terminal (command line) input does not end like a file, but sometimes it is necessary to pass EOF to such a utility. I don't think it was used in real files, at least not to indicate the end of the file. The file system knows perfectly well when the file is finished, it doesn't require a pointer.
EDIT shamelessly copied from a comment provided by John Machin
This can happen (unarmed) in real files. All it needs is (1) a data-entry user who types Ctrl-Z by mistake, sees nothing on the screen, types in an alleged Shift-Z and keeps running, and (2) verification software (written, for example, by his nephew company president) who happily accepts Ctrl-anykey on text fields, and your database has a little bomb in it just waiting for someone to query a flat file.
source to share
Unexpected EOF means that the bcp reader encountered an EOF when it was expecting more data. This EOF can be:
(1) actual physical end of file (more bytes to read). This means that you have incorrectly formatted data. Check incomplete file write for incomplete write.
OR
(2) on Windows where you are, programs that read a file in text mode respect the ancient convention inherited through MS-DOS from CP / M regarding Ctrl-Z (aka ^ Z aka \ 'x1A' aka SUB aka SUBSTITUTE) in as an end-of-file marker when reading from ANY file, not just the terminal. This includes Python - the behavior is defined by the C stdlib. Check for \\\\\\\\\\\\\ \
Update comment response in a clear manner:
In Notepad ++, you can make it display unusual characters by doing View / Show Symbol / Show All Characters. You can perform a search by executing Ctrl-F, typing \ x1a in the Find box and selecting the Advanced radio button in the search bar.
Or you can, with a little Python, get the line number of the first Ctrl-Z:
bytes = open('bcp.dat', 'rb').read()
zpos = bytes.find('\x1a')
# if zpos is -1, no Ctrl-Z in file
print 1 + bytes[:zpos].count('\r\n')
If your .dat was created, it doesn't matter. An unintentional Ctrl-Z can happen anywhere in a file created on any operating system. This is where it reads like a text file that matters - Windows? Explosions!
source to share