MarkLogic: error while importing files using MLCP
I tried to import files with MarkLogic Content Pump on MarkLogic Server running in production using the following command.
C:\Users\Admin\Desktop\mlcp-1.3-3\bin>mlcp.bat import -host localhost -port 8891 -username admin -password admin -mode local -input_file_type archive -input_file_path /d:/NewFolder/
The input file path has both binary and XML files.
"D:\NewFolder\20150626200126+0800-000000-BINARY.zip"
"D:\NewFolder\20150626200126+0800-000001-XML.zip"
When I tried to run the import command on the command line. I got the following answer, which I am not familiar with.
15/06/29 16:53:11 INFO contentpump.ContentPump: Hadoop library version: 2.6.0
15/06/29 16:53:11 INFO contentpump.LocalJobRunner: Content type: XML
15/06/29 16:53:11 ERROR contentpump.ContentPump: Error running a ContentPump job
java.lang.RuntimeException: Error while running command to get file permissions
: ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
715)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:582)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)
at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:620)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)
at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)
java.lang.RuntimeException: Error while running command to get file permissions
: ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
715)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:582)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)
at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
loadPermissionInfo(RawLocalFileSystem.java:620)
at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
getPermission(RawLocalFileSystem.java:557)
at org.apache.hadoop.fs.LocatedFileStatus.<init>(LocatedFileStatus.java:
42)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1699)
at org.apache.hadoop.fs.FileSystem$4.next(FileSystem.java:1681)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedL
istStatus(FileInputFormat.java:303)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(File
InputFormat.java:264)
at com.marklogic.contentpump.FileAndDirectoryInputFormat.getSplits(FileA
ndDirectoryInputFormat.java:80)
at com.marklogic.contentpump.ArchiveInputFormat.getSplits(ArchiveInputFo
rmat.java:56)
at com.marklogic.contentpump.LocalJobRunner.run(LocalJobRunner.java:128)
at com.marklogic.contentpump.ContentPump.runJobLocally(ContentPump.java:
307)
at com.marklogic.contentpump.ContentPump.runCommand(ContentPump.java:204
)
at com.marklogic.contentpump.ContentPump.main(ContentPump.java:67)
Can anyone help me with this issue?
Thank.
source to share
The error message is similar to the one below: Hadoop job failed with error
The resolution was to increase the heap size. Add -Xmx512m
to an environment variable called JVM_OPTS
..
NTN!
source to share
I ran into this same error while doing the MarkLogic University XQuery II course. The VMs they had available for the course were very slow, so I am setting up my own Windows 7 VM in Parallels.
When I ran this from module 6:
mlcp.bat import -host localhost -port 8012 -username admin -password admin -input_file_path C:\mls-developer-2\socialmedia\content\enriched -mode local -input_file_pattern "disqus.*\.xml" -output_uri_replace "C:/mls-developer-2/socialmedia/content/enriched, 'socialmedia/disqus'"
I got the same error as you ... I tried setting _JAVA_OPTIONS to increase the memory available to the JVM as some posts suggested, but it had no effect.
I ended up getting the command to work, changing it to:
mlcp.bat import -host localhost -port 8012 -username admin -password admin -input_file_path "C:\mls-developer-2\socialmedia\content\enriched\*.xml" -mode local -output_uri_replace "C:/mls-developer-2/socialmedia/content/enriched, 'socialmedia/disqus'"
There was something like input_file_pattern "disqus. *. Xml" which he didn't like. It doesn't look like it's the same thing in your case, but I thought I posted my problem here for others.
source to share