Renaming PIG Output Portion Files
I have a requirement to change the naming convention of part files after starting my PIG job. I want to part-r-0000
be userdefinedName-r-0000
.
Any possible solution? I avoid the hadoop -cp and hadoop -mv commands .
thank
+3
Aviral kumar
source
to share
2 answers
These files are generated by the map zoom jobs generated by Pig. Therefore, you need to configure Apache Map-reduce. Corresponding propertymapreduce.output.basename
You can define any Hadoop property directly in the pig script:
SET mapreduce.output.basename 'custom-name';
+3
Mikhail Golubtsov
source
to share
A pig run like this will do the same
pig -Dmapreduce.job.queuename=my-queue -Dmapreduce.output.basename=my-outputfilename;
+1
rbyndoor
source
to share