Exporting Neo4j.rb nodes with associations as JSON
I have an application that uses Neo4j.rb in a Rails application in a small database with several thousand nodes / relationships and now I am writing a batch upload and script upload. Mainly for backup / migration purposes. At the moment I have 2 nodes - terms and permission groups. It's pretty easy to export each node using properties only, but I'm missing the associations inside each node. Is there a way to export each node with its own properties and associations?
An example in a database with two nodes that have communication with each other:
# Create blank holding array
2.0.0-p353 :001 > bulk = []
=> []
# Iterate through each node and drop into array
2.0.0-p353 :002 > Term.all.each do |t|
2.0.0-p353 :003 > bulk << t
2.0.0-p353 :004?> end
=> [#<Term access_designation: nil, created_at: Mon, 20 Jul 2015 18:12:39 +0000, created_by: nil, data_availability: nil, definition: "New York", name: "Awesome Term", notes: nil, possible_values: nil, sensitivity_access_notes: nil, sensitivity_classification: nil, source_system: nil, updated_at: Mon, 20 Jul 2015 18:12:39 +0000, updated_by: nil>]
2.0.0-p353 :005 > PermissionGroup.all.each do |pg|
2.0.0-p353 :006 > bulk << pg
2.0.0-p353 :007?> end
=> [#<PermissionGroup created_at: Mon, 20 Jul 2015 18:14:29 +0000, created_by: nil, name: "my group", updated_at: Mon, 20 Jul 2015 18:14:29 +0000, updated_by: nil>]
# output result to json
2.0.0-p353 :008 > bulk.to_json
=> "[{\"term\":{\"name\":\"Awesome Term\",\"definition\":\"New York\",\"source_system\":null,\"possible_values\":null,\"notes\":null,\"data_availability\":null,\"sensitivity_classification\":null,\"access_designation\":null,\"sensitivity_access_notes\":null,\"created_at\":\"2015-07-20T18:12:39+00:00\",\"updated_at\":\"2015-07-20T18:12:39+00:00\",\"created_by\":null,\"updated_by\":null,\"id\":\"ed820017-24d9-4895-aea2-2a899c83987a\"}},{\"permission_group\":{\"name\":\"my group\",\"created_at\":\"2015-07-20T18:14:29+00:00\",\"updated_at\":\"2015-07-20T18:14:29+00:00\",\"created_by\":null,\"updated_by\":null,\"id\":\"7f6d87cf-a2d3-488e-be03-ca4087e48986\"}}]"
The resulting JSON prettected looks like this:
[
{
"term": {
"name": "Awesome Term",
"definition": "New York",
"source_system": null,
"possible_values": null,
"notes": null,
"data_availability": null,
"sensitivity_classification": null,
"access_designation": null,
"sensitivity_access_notes": null,
"created_at": "2015-07-20T18:12:39+00:00",
"updated_at": "2015-07-20T18:12:39+00:00",
"created_by": null,
"updated_by": null,
"id": "ed820017-24d9-4895-aea2-2a899c83987a"
}
},
{
"permission_group": {
"name": "my group",
"created_at": "2015-07-20T18:14:29+00:00",
"updated_at": "2015-07-20T18:14:29+00:00",
"created_by": null,
"updated_by": null,
"id": "7f6d87cf-a2d3-488e-be03-ca4087e48986"
}
}
]
But this is the absence of associations. I know that I have them when I request
2.0.0-p353 :010 > t = Term.first
=> #<Term access_designation: nil, created_at: Mon, 20 Jul 2015 18:12:39 +0000, created_by: nil, data_availability: nil, definition: "New York", name: "Awesome Term", notes: nil, possible_values: nil, sensitivity_access_notes: nil, sensitivity_classification: nil, source_system: nil, updated_at: Mon, 20 Jul 2015 18:12:39 +0000, updated_by: nil>
2.0.0-p353 :011 > t.permission_group
=> #<PermissionGroup created_at: Mon, 20 Jul 2015 18:14:29 +0000, created_by: nil, name: "my group", updated_at: Mon, 20 Jul 2015 18:14:29 +0000, updated_by: nil>
2.0.0-p353 :012 > t.permission_group.name
=> "my group"
So my question is, how can I display all nodes with their properties and associations? Is there a fancy neo4j.rb magic method or do I have to manually write out the cypher request?
source to share
That's a good question. You just reminded me that I wanted to create methods to get identifiers for associations. I just put together some working code that will go into the master (still needs tests) that provides the following:
t.permission_group_id # UUID
t.permission_group_neo_id # Neo4j ID
pg.term_ids
pg.term_neo_ids
Obviously it won't. It also needs to_json
to ActiveNode
be changed to output these IDs to JSON (possibly based on an option). ActiveRel models are also discussed.
Everything I said, I'm not sure if this is the best way to make a backup. The problem is that if you go to repair, you will have models on both sides of the relationship that are trying to create that relationship, which means you get two connections. You can do MERGE
, but there are also times when you need two relationships;)
So I would suggest using Neo4j's built-in backup tools. There is a tool neo4j-backup
:
http://neo4j.com/docs/stable/re04.html
Though I think it might only be in an enterprise (I don't see it in my community setup).
There is also a command dump
from Cypher:
http://neo4j.com/docs/stable/shell-commands.html#_dumping_the_database_or_cypher_statement_results
I mark in the docs for an experimental feature.
Does this indicate your problem? If not, I can look into other solutions;)
source to share
I still think that backing up via Neo4j directly is better. Since you said that your database is not very large, you should be able to make simple Cypher queries like this:
MATCH n RETURN n
MATCH (a)-[r]->(b) RETURN a, r, b
Depending on what information you return for r
, you can only do:
MATCH ()-[r]->() RETURN r
In Neo4j.rb, you can run a Cypher query like this:
Neo4j::Session.current.query('MATCH n RETURN n')
or like this:
Neo4j::Session.current.query.match(:n).pluck(:n)
source to share