Merging multiple RRDs over time
I have an old RRD file that was set up to only track 1 year of history. I decided that more story would be enjoyable. I resized the rrdtool and the RRD is bigger now. I have old backups of this RRD file and I would like to merge the old data so that the updated RRD also has historical data.
I've tried rrd contrib "merged-rrd.py" but it gives:
$ python merged-rrd.py ../temperature-2010-12-06.rrd ../temperature-2011-05-24.rrd merged1.rrd
merging old:../temperature-2010-12-06.rrd to new:../temperature-2011-05-24.rrd. creating merged rrd: merged1.rrd
Traceback (most recent call last):
File "merged-rrd.py", line 149, in <module>
mergeRRD(old_path, new_path, mer_path)
File "merged-rrd.py", line 77, in mergeRRD
odict = getXmlDict(oxml)
File "merged-rrd.py", line 52, in getXmlDict
cf = line.split()[1]
IndexError: list index out of range
Also tried "rrd_merger.pl":
$ perl rrd_merger.pl --oldrrd=../temperature-2010-12-06.rrd --newrrd=../temperature-2011-05-24.rrd --mergedrrd=merged1.rrd
Dumping ../temperature-2010-12-06.rrd to XML: /tmp/temperature-2010-12-06.rrd_old_8615.xml
Dumping ../temperature-2011-05-24.rrd to XML: /tmp/temperature-2011-05-24.rrd_new_8615.xml
Parsing ../temperature-2010-12-06.rrd XML......parsing completed
Parsing ../temperature-2011-05-24.rrd XML...
Last Update: 1306217100
Start processing Round Robin DB
Can't call method "text" on an undefined value at rrd_merger.pl line 61.
at rrd_merger.pl line 286
at rrd_merger.pl line 286
Is there a tool for merging or merging RRDs that works?
source to share
This is the fixed rrdtool-merge.pl for me:
< my $xff = $new_rra->first_child( 'xff' )->text;
---
> my $xff = $new_rra->first_child_text( 'xff' );
From XML :: Twig documentation:
first_child_text ($optional_condition)
Return the text of the first child of the element, or the first child
matching the $optional_condition If there is no first_child then returns ''. This
avoids getting the child, checking for its existence then getting the text for trivial
cases.
source to share
The rrdmerge.pl utility included with Routers2 in the / extras directory can do this. Build the latest Routers2 from http://www.steveshipway.org/software/rrd/
This is a utility I wrote to merge multiple zipped MRTG RRD files that sound exactly the same as the situation you mention.
This is probably too late for the OP, but will hopefully be helpful for later people who come here. It can concatenate any RRD files, even with different DS, RRA or spacing, and can generate XML or RRDs, and will pick the best available data from the component's RRD files to make an output.
Example:
rrdmerge.pl --rrd --libpath $RRDLIBDIR --output /tmp/merge.rrd --rows 12000 $FROMDIR/file.rrd $ARCHIVE/*.rrd
source to share
Looking at the XML file generated by rrdtool, there is a simple logic error in the Perl script. The AVERAGE elements are simple enough, but the tag is contained within a tag with text inside.
<cf> AVERAGE </cf>
<pdp_per_row> 1 </pdp_per_row> <!-- 300 seconds -->
<params>
<xff> 5.0000000000e-01 </xff>
</params>
The parsing needs to be tweaked a bit, and when it works, the fix comes back here (where it's easy for Google) and also for the author of the script to fix it.
source to share