Clever Geek Handbook

Renaming PIG Output Portion Files

I have a requirement to change the naming convention of part files after starting my PIG job. I want to part-r-0000

be userdefinedName-r-0000

.

Any possible solution? I avoid the hadoop -cp and hadoop -mv commands .

thank

+3


source to share


2 answers


These files are generated by the map zoom jobs generated by Pig. Therefore, you need to configure Apache Map-reduce. Corresponding propertymapreduce.output.basename

You can define any Hadoop property directly in the pig script:



SET mapreduce.output.basename 'custom-name';
+3


source


A pig run like this will do the same

 pig -Dmapreduce.job.queuename=my-queue -Dmapreduce.output.basename=my-outputfilename;


+1


source






More articles:


All Articles