Most efficient way to batch load objects and all their children in Doctrine2

I have one type of entity. I will call the parent one-to-many with the entity that I will call the child (many children belong to the same parent).

What I would like to do is load - as efficiently as possible - the parent and all of its children with as few requests as possible.

By default, by doctrine, if I do something like load 100 of my parent, then loop through them and do operations on the children, it will be at least 101 requests, one to load the parent and one to get all the children for each parent. If I try to write a query that loads all the objects in one pass, it gets incredibly slow, loading what I can only assume is a Cartesian object with an entire string of all parent and child properties for each child. This problem becomes even more devastating if I have multiple child objects that I am making.

The only solution I can think of is that you query all parents and all children yourself, and then chain them in a nested loop. This cuts it down to two queries, but it seems ... not correct. Anyone have any insight for me? Here, in essence, I think:

//First, select an array of all parent objects
$parents = "SELECT p FROM parent p WHERE 1 LIMIT 100";

//Select all child elements
$children = "SELECT c FROM children c WHERE c.parent IN ($parents)";

//Loop through all elements and assign them to parents, and parents to children
foreach($children as $child){
    foreach($parents as $parent){
        //All children will have loaded references to their parent object
        if ($child->getParent()->getId() === $parent->getId()){
            //This child belongs to this parent
            $parent->addChild($child);
            $child->setParent($parent);
            continue;
        }
    }
}

      

Edit 1:

In response to Tom Corrigan's comment below:

Using the query builder is the same as your suggestion:

$qb->select(['p', 'c'])
   ->from('Parent', 'p')
   ->leftJoin('p.children', 'c')
;

      

I do load the whole object, but the performance hit is ridiculous. Using the above query to load two parent objects for a total of about 12 child objects took almost 40 seconds. If I load the parent THEN, load two children, it takes about 5ms.

Edit 2:

OK, the problem with incredibly slow load times only occurs when I query based on a many-to-many concatenated table. If I make a simple choice like above, it works pretty fast, so thanks Tom, you answered my question perfectly.

What I don't understand is a very different question, but I'll cover it here because it is related at some level - which is why when I make multiple selections it takes an INCREDIBLY long time if I filter one from the selected subclasses.

This example adds two more objects to the group, groups (many with people) and application (one to many with groups).

$qb->select(['p', 'c', 'g', 'a'])
   ->from('Parent', 'p')
   ->leftJoin('p.children', 'c')
   ->join('p.groups','g')
   ->join('g.application','a')
   ->where('a.id = :applicationId')
   ->addGroupBy('p')
   ->setMaxResults(1)
;

      

This request will take almost a minute, whereas if I do the same without additional selections, for example:

$qb->select('p')
   ->from('Parent', 'p')
   ->leftJoin('p.children', 'c')
   ->join('p.groups','g')
   ->join('g.application','a')
   ->where('a.id = :applicationId')       
   ->setMaxResults(1)
;

      

It takes several MS and adding additional objects is a few ms more.

Any thoughts?

+3


source to share


2 answers


It's very simple in Doctrine (as you would expect), but my search for the documentation hasn't come up with a clear explanation of how to do it. However, Guilherme Blanco's fantastic talk explains how to do it. (see from slide 22)

I note that the example you provided used DQL, but it is also possible to do this using the Query Builder.

In DQL:

SELECT p, c
FROM Parent p
LEFT JOIN p.children c

      



Using QueryBuilder

$qb->select(['p', 'c'])
   ->from('Parent', 'p')
   ->leftJoin('p.children', 'c')
;

      

Running any of the above queries will drop the database just once and return an array of parent objects with all their children fully moistened. If you need to join more entities, you can do so, but just remember if you want the doctrine to actually hyrdrate those entities for you, then you must add the merged entity to the select statement. For example:

$qb->select(['p', 'c', 'gc'])
   ->from('Parent', 'p')
   ->leftJoin('p.children', 'c')
   ->leftJoin('c.grandchildren', 'gc')
;

      

+4


source


If you know which Entity data you need and which you don't, you can use the PARTIAL

DQL keyword .



0


source







All Articles