Clever Geek Handbook

R: write to CSV and continue piping

Is there a way to include the call to write.csv in the dplyr pipe line?

library(dplyr)
mtcars %>% 
  filter(cyl == 4) %>% 
  write.csv(file = "firstStage.csv") %>% 
  group_by(carb) %>% 
  summarise(hp.sum = sum(hp)) %>%
  write.csv(file = "secondStage.csv")

I could create my own function:

csv2go <- function(x, ...) {
  write.csv(x, ...)
  x
}

but was wondering if there is something that does this in base or dplyr. Maybe function write_csv()

in library(readr)

can accept this option?

+3


source to share


2 answers


Very simple!!! All you have to do is add one "T" character to convert the normal pipe operator%>% to Tee-pipe operator% T>% like this:

library(dplyr)
mtcars %>% 
   filter(cyl == 4) %T>%                  # <== Tee-pipe operator 
   write.csv(file = "firstStage.csv") %>% 
   group_by(carb) %>% 
   summarise(hp.sum = sum(hp)) %>%
   write.csv(file = "secondStage.csv")


% T>% is your new friend

+9


source


Is something like this what you need? This is how I write csvs on HDFS.



write.csv(data, file=pipe("hadoop dfs -put - /tmp/test.csv"))
+1


source






More articles:


All Articles