How to convert vector of factors in POSIXct to ff or ffbase
After reading in a large dataset with read.csv.ffdf
one of the columns, the time is. For example, 2014-10-18 00:01:02
for 1 million rows in this column. This column is a factor. How do I convert it to POSIXct
supported ff
? Just using as.POSIXct()
just turns the values intoNA
Or when I read into the dataset at the beginning, can I specify what the column will be POSIXct
?
My goal is to get the month and days (or even an hour). Therefore, I am open to solutions other than converting to POSIXct
.
For example, we have table 9 through 2,
test <- read.csv.ffdf(file="test.csv", header=T, first.rows=-1)
The two columns are ID (numeric class) and time (factor-class)
Dput is located here
structure(list(virtual = structure(list(VirtualVmode = c("integer",
"integer"), AsIs = c(FALSE, FALSE), VirtualIsMatrix = c(FALSE,
FALSE), PhysicalIsMatrix = c(FALSE, FALSE), PhysicalElementNo = 1:2,
PhysicalFirstCol = c(1L, 1L), PhysicalLastCol = c(1L, 1L)), .Names = c("VirtualVmode",
"AsIs", "VirtualIsMatrix", "PhysicalIsMatrix", "PhysicalElementNo",
"PhysicalFirstCol", "PhysicalLastCol"), row.names = c("ID", "time"
), class = "data.frame", Dim = c(9L, 2L), Dimorder = 1:2), physical = structure(list(
ID = structure(list(), physical = <pointer: 0x000000000821ab20>, virtual = structure(list(), Length = 9L, Symmetric = FALSE), class = c("ff_vector",
"ff")), time = structure(list(), physical = <pointer: 0x000000000821abb0>, virtual = structure(list(), Length = 9L, Symmetric = FALSE, Levels = c("10/17/2003 0:01",
"12/5/1999 0:02", "2/1/2000 0:01", "3/23/1998 0:01", "3/24/2013 0:00",
"5/29/2004 0:00", "5/9/1985 0:01", "6/14/2010 0:01", "6/25/2008 0:02"
), ramclass = "factor"), class = c("ff_vector", "ff"))), .Names = c("ID",
"time")), row.names = NULL), .Names = c("virtual", "physical",
"row.names"), class = "ffdf")
source to share
You can use with ffbase package as shown below on toys example. Best.
require(ff)
x <- data.frame(id = 1:100000, timepoint = seq(from = Sys.time(), by = "sec", length.out = 100000))
x$timepoint <- as.factor(x$timepoint)
xff <- as.ffdf(x)
class(xff)
require(ffbase)
xff$time <- with(xff, as.POSIXct(as.character(timepoint)), by = 10000)
ramclass(xff$time)
[1] "POSIXct" "POSIXt"
str(xff[1:10, ])
'data.frame': 10 obs. of 3 variables:
$ id : int 1 2 3 4 5 6 7 8 9 10
$ timepoint: Factor w/ 100000 levels "2014-10-20 09:14:10",..: 1 2 3 4 5 6 7 8 9 10
$ time : POSIXct, format: "2014-10-20 09:14:10" "2014-10-20 09:14:11" "2014-10-20 09:14:12" "2014-10-20 09:14:13" ...
source to share