What is the most efficient way to store a Jota-Time object `DateTime`?
I have written some Java software that saves and retrieves Joda-Time DateTime
from Redis very often . I am just serializing and deserializing objects currently. Software reads objects about 50 times more often than it writes. I haven't profiled the serialization / deserialization of Joda-Time objects, but the software scaled well, computationally, under load, and I'm happy with the performance.
What scales well is memory usage. The Joda-Time serialized objects are quite large and a decently sized sample Redis instance can take about 3 days of customer data before I have to pull it out of the relational database to disk. A secondary issue is that Redis' own backup mechanisms seem to be more complex to manage a larger dataset ...
After giving up the temptation to throw more RAM on the problem, I was already thinking of the following ideas:
- then shrink the objects before saving
- saved as ISO date format string
- saved as some other Joda-compatible string formats
I'll try and analyze them before I solve them, but I'm wondering if anyone could think of a more efficient way to reduce the memory footprint of Joda's surviving objects without breaking the computational bank?
source to share
ISO 8601
I don't know anything about Redis yet ... Generally speaking, the simplest and most efficient way to serialize Joda-Time objects is to use built-in support for sane, unambiguous, standard ISO 8601 string formats for date and time values.
For a time value specified in a loop, the standard provides a format YYYY-MM-DDTHH:MM:SS.SSS±HH:SS
such as 2014-10-24T21:17:30+02:00
or 2014-10-24T19:17:30Z
( Z
for Zulu
stands for 00:00 offset from UTC ).
The various Joda-Time 2.5 classes use ISO 8601 as their defaults to parse and render String representations of date and time values.
Generating strings
For DateTime
just call its toString
method explicitly or implicitly.
String output = DateTime.now( DateTimeZone.forID( "America/Montreal" ) ).toString();
It is generally best to work with UTC while storing date and time values. Joda-Time makes it easy to customize UTC.
DateTime nowMontreal = DateTime.now( DateTimeZone.forID( "America/Montreal" ) );
DateTime nowUtc = nowMontreal.withZone( DateTimeZone.UTC );
String output = nowUtc.toString();
Another example.
DateTime output = DateTime.now( DateTimeZone.UTC ).toString();
Parsing strings
Parsing is just as simple. The only problem is the time zone. If you omit the timezone, usually Joda-Time will assign the current timezone to the JVMs. It is usually best if you explicitly specify the desired time zone.
DateTime dateTimeMontreal = new DateTime( "2014-10-24T19:17:30Z", DateTimeZone.forID( "America/Montreal" ) );
or, for UTC ...
DateTime dateTimeUtc = new DateTime( "2014-10-24T19:17:30Z", DateTimeZone.UTC ) );
java.time
Another alternative is the new java.time package built into Java 8. Inspired by Joda-Time, java.time is much the same, but one difference is that java.time generates string representations by default, extending the ISO 8601 standard to add time zone name. While the standard format has an offset from UTC, you lose time zone information. (Time zone is offset plus rules for daylight saving time and other anomalies in the present, future, and past.)
On the other hand, it is usually best to store the datetime in UTC. If you really care about the time zone during data entry, it is usually best to keep this information separate, in addition to the UTC-adjusted value.
In java.time, the class Instant
represents a moment on the timeline in UTC format.
Instant instant = Instant.parse( "2014-10-24T19:17:30Z" );
String outputInstant = instant.toString();
2014-10-24T19: 17: 30z
To customize in time zone, specify ZoneId
to receive . ZonedDateTime
ZoneId zoneId = ZoneId.of( "America/Montreal" );
ZonedDateTime zdt = ZonedDateTime.ofInstant( instant , zoneId );
String outputZdt = zdt.toString();
2014-10-24T15: 17: 30-04: 00 [America / Montreal]
source to share
Try to analyze the distribution of your time objects. If it happens that they are relatively close to each other, then you can do the "magic":
1) you can enter a constant constant "starting point" and then store the actual date as an offset of a few days from the constant - this will be an integer value (~ 8 bytes on a 64-bit arch without compression)
2) do you need the actual time? if not, just throw the time away; if yes - you can store hours + minutes + seconds in one int variable (~ 8 more bytes on a 64-bit arch uncompressed)
3) parse the results - chances are you can fit both: date (shift) and time in one int variable
4) introduce a caching mechanism that will dramatically increase the serialization / deserialization performance of your objects
source to share
Store millis from the beginning of the era. This is the only long value. If you need a timezone value, also store the timezone id as a string. Serialization and parsing of the string representation will always take up more resources, including RAM, so much data processing inside, some regex, split calls that allocate more memory.
Use this constructor to retrieve the value: public BaseDateTime(long instant, DateTimeZone zone)
It's so lightweight because it stores what's under the hood of each DateTime instance:
public BaseDateTime(long instant, Chronology chronology) {
super();
iChronology = checkChronology(chronology);
iMillis = checkInstant(instant, iChronology);
adjustForMinMax();
}
source to share