Cypher Batch update forever

I was doing a POC on a publicly available Twitter dataset for our project. I was able to create a Neo4j database for it using the Michael Hunger Batch Inserter utility and it was relatively fast (it took only 2 hours and 53 minutes to complete). In total there were 15,203,731 Nodes with 2 properties (name, address) 256,147,121 Relationships, with 1 property

I have now created a Cypher query to update the Twitter database. I added a new property (age) to Node and a new relationship property (FollowedSince) to CSV. Now things are starting to look bad. The relationship update request (see below) is executed forever.

USING PERIODIC COMMIT 100000
LOAD CSV WITH HEADERS FROM {csvfile} AS row FIELDTERMINATOR '\t'
MATCH (u1:USER {name:row.`name:string:user`}), (u2:USER {name:row.`name:string:user2`})
MERGE (u1)-[r:Follows]->(u2)
ON CREATE SET r.Property=row.Property, r.FollowedSince=row.FollowedSince
ON MATCH SET r.Property=row.Property, r.FollowedSince=row.FollowedSince;

      

I already pre-created the index by running

CREATE INDEX ON :USER(name); 

      

My neo4j property:

allow_store_upgrade=true
dump_configuration=false
cache_type=none
use_memory_mapped_buffers=true
neostore.propertystore.db.index.keys.mapped_memory=260M
neostore.propertystore.db.index.mapped_memory=260M
neostore.nodestore.db.mapped_memory=768M
neostore.relationshipstore.db.mapped_memory=12G
neostore.propertystore.db.mapped_memory=2048M
neostore.propertystore.db.strings.mapped_memory=2048M
neostore.propertystore.db.arrays.mapped_memory=260M

node_auto_indexing=true

      

I would like to know what should I do to speed up the Cypher query? At the time of this writing, more than an hour and a half have passed, and my relationship (10,000 747) has not yet ended. Node (15,203,731), which finished earlier, got 34 minutes, which I think is too long. (The Batch Inserter processed the entire Node in just 5 minutes!)

I tested my queries on a small dataset to try first before tackling a larger dataset and it actually worked.

My Neo4j lives on a server machine, so the hardware isn't an issue here.

Any advice please? Thank.

+3
neo4j cypher


source to share


No one has answered this question yet

Check out similar questions:

17
In Cypher, how can I create a relationship if it doesn't exist; update property if it is
1
Neo4j: create a link between two or more nodes sharing the same label
0
Unexpected behavior in Neo4j Cypher
0
Getting nodes in Neo4j using Cypher, determining by properties
0
Cypher query - creating a graph from a csv file
0
StackOverflowError while making cypher requests through Neo4J REST API
0
How to quickly create 100k nodes and relationships in neo4j?
0
Neo4j Cypher request fails and returns with unknown error
0
Cypher query equivalent for neo4j-import
0
Creating relationship for other properties skipped if property is Null in Cypher query



All Articles
Loading...
X
Show
Funny
Dev
Pics