Hadoop Pig cannot store in existing folder
I created a folder to remove the result file from the Pig process using the Store command. It works the first time, but the second time, it claims the folder already exists. What's the best practice for this situation? The documentation is sparse on this topic.
The next step is to rename the folder to its original filename to reduce the impact of this. Any thoughts?
+3
source to share
1 answer
You can run fs commands from inside Pig and should be able to remove the directory by sending fs -rmr before running the STORE command:
fs -rmr dir
STORE A into 'dir' using PigStorage();
The only subtle thing is that the fs command does not expect quotes around the directory name, while the store command expects quotes around the directory name.
+8
source to share