Problem creating nested xml from flat xml

It's a little longer; I try to appeal to everything that caught my attention.

Existing templates

First, let's break down what's going on with your own XSL code.

<xsl:template match="/table">
    <table name="ecatalogue">
        <collection>
            <xsl:apply-templates select="tuple[not(tuple[@name='AssParentObjectRef']/atom[@name='irn'])]"/>
        </collection>
    </table>
</xsl:template>

      

        
        
        
      

    

• You have <collection>
  
  here in the template that matches /table
  
  . Since there is only one match /table
  
  , you will only have one <collection>
  
  in the output.

• Also, your selection of top-level items tuple
  
  ( tuple
  
  which are not related to the parent) can be simplified. Instead:

  select="tuple[not(tuple[@name='AssParentObjectRef']/atom[@name='irn'])]"
  
        
  
          
          
          
        
  
      

  you could just say:

  select="tuple[not(tuple/*)]"
  
        
  
          
          
          
        
  
      

  as you can tell from your example that the top level tuple
  
  only contains an empty self-closing tag <tuple>
  
  .

Next bit:

<xsl:key name="child" match="tuple" use="tuple[@name='AssParentObjectRef']/atom[@name='irn']" />

      

        
        
        
      

    

So, we have matching keys to tuple

and using the parent identifier tuple

.

<xsl:template match="tuple">
    <tuple>
        <xsl:copy-of select="atom"/>

        <!-- If this `tuple` is a parent (i.e. if it included in
             the list of parent IDs in the key), then we add a
             wrapper for the children and process the children.  -->
        <xsl:if test="key('child', atom[@name='irn'])">
            <tuple name="children">
                <!-- Now we apply templates to the `tuple`s 
                     in the key -->
                <xsl:apply-templates select="key('child', atom[@name='irn'])"/>
            </tuple>
        </xsl:if>
    </tuple>
</xsl:template>

      

        
        
        
      

    

It basically works. Comparing the results of this to your sample of the desired output, the bits you're missing are the wrapper tag <collection>

(see above) and the parent and top-level predecessor names and names (for which you don't have XSL code).

You indicate

"And while this is grouping records, the output will only be one of those collections. So from a file of 3524 records, I get one set of 24 records."

I can assume that the XML structure of the rest of your actual input may be different from what your XSL is targeting. But without seeing your actual input, I cannot tell why this might be the case.

Your edit

You describe the addition in the following template:

<xsl:template match="node()|@*">
    <xsl:copy>
        <xsl:apply-templates select="node()|@*"/>
    </xsl:copy>
</xsl:template>

      

        
        
        
      

    

This is the "identity" pattern, so called because it just copies the elements the same way. The duplication you see is expected: the XSLT processor goes through flat typing tuple

in the input file and copies it, and then processes some of it in the context of a specially matched tuple

template, which applies the templates to tuple

that match the key (nested children).

An alternative approach

The template is table

not too different:

<xsl:template match="table">
    <xsl:copy>
        <xsl:copy-of select="@*"/>
        <!-- Target only parent-level `tuple`s.  This excludes
            child-level tuples, helping to prevent duplicates.-->
        <xsl:apply-templates select="tuple[not(tuple/*)]" mode="top"/>
    </xsl:copy>
</xsl:template>

      

        
        
        
      

    

Remarkably, we are not doing anything with <collection>

here - we want to wrap each top level tuple

(and its children) in <collection>

, so we need to add <collection>

at the level tuple

.

<!-- Add the `collection` wrapper only to top-level tuples -->
<xsl:template match="tuple" mode="top">
    <collection>
        <!-- Pass on this tuple to the main `tuple` template -->
        <xsl:apply-templates select="."/>
    </collection>
</xsl:template>

      

        
        
        
      

    

I'm matching tuple

, but using special mode - we only want to add a wrapper <collection>

for the top level tuple

s. Then I have a separate template that handles all tuple

s:

<!-- This is the main template for processing `tuple` elements.
    Most of the changes needed are common to all `tuple`s, so it
    makes sense to keep all the logic in one place. -->
<xsl:template match="tuple">
    <tuple>
        <!-- Copy each existing `atom` child -->
        <xsl:copy-of select="atom"/>
        <!-- Add in metadata about parent and top-level ancestor titles and IDs -->
            <xsl:choose>
                <!-- If this is a top-level item, just use its own values -->
                <xsl:when test="not(tuple/*)">
                    <atom name="TopTitle"><xsl:value-of select="atom[@name='EADUnitTitle']"/></atom>
                    <atom name="TopID"><xsl:value-of select="atom[@name='irn']"/></atom>
                </xsl:when>
                <!-- If this is a descendant, we need to find its parent and its top-level ancestor -->
                <xsl:when test="tuple/*">
                    <atom name="ParentTitle"><xsl:value-of select="tuple/atom[@name='EADUnitTitle']"/></atom>
                    <atom name="ParentID"><xsl:value-of select="tuple/atom[@name='irn']"/></atom>
                <!-- For convenience, grab the top-level ancestor `tuple` and stuff it in a variable.
                    This is vaguely annalogous to your use of `key`. -->

                <!-- Finding the top-level `tuple` is complicated by the fact that the ID values in 
                    `<atom name="irn">` do not have a standardized format, other than that the whole
                    strings appear to consist of atomic values separated by single periods, with 
                    descendant `irn` values appending to the precedent values.  Examples:
                      Top:        `da.04`
                      Descendant: `da.04.11.02`
                      Top:        `D42.PL.05`
                      Descendant: `D42.PL.05.01`
                    So chunking the ID values is a problematic approach, since we don't know how many
                    chunks comprise the initial non-numeric portion: `da`, or `D42.PL`, or ... ???.
                    Top-level elements *do* also have empty `<tuple name="AssParentObjectRef">` elements.
                    So we _can_ find all the top-level elements, and then look in those for the one that
                    has an `irn` value that matches the start of the `irn` value of this current `tuple`. -->
                <xsl:variable name="top" select="/table/tuple[tuple[@name='AssParentObjectRef'][not(*)]]
                    ['The above statement grabs all the `tuple`s that have an empty `tuple[@name=`AssParentObjectRef``.
                      The below statement then goes through all those `tuple`s to find the ones where the `irn`
                      values match the start of the `irn` value of the current `tuple`.']
                    [starts-with(current()/atom[@name='EADUnitID'], atom[@name='EADUnitID'])]"/>
                    <!-- Now we can reference that variable to get the top-level ancestor values -->
                    <atom name="TopTitle"><xsl:value-of select="$top/atom[@name='EADUnitTitle']"/></atom>
                    <atom name="TopID"><xsl:value-of select="$top/atom[@name='irn']"/></atom>
                </xsl:when>
            </xsl:choose>
        <!-- Process any children of this tuple, based on `irn` values.
            Basically, we look for any other `tuple`s in the `table`
            that point to this current `tuple` `irn` value. -->
        <xsl:if test="/table/tuple[tuple/atom[@name='irn'] = current()/atom[@name='irn']]">
            <tuple name="children">
                <xsl:apply-templates select="/table/tuple[tuple/atom[@name='irn'] = current()/atom[@name='irn']]"></xsl:apply-templates>
            </tuple>
        </xsl:if>
    </tuple>
</xsl:template>

      

        
        
        
      

    

I cannot speak to your complete dataset of 3524 records, but by doing the above against your sample XML input, my output is identical to your desired output (except for one error in your XML input example about a reference irn

value mentioned in a comment on your original message).

Suggestions for the desired XML output

As a data format, the desired output XML has a number of aspects that strike me as a bit odd.

• Wrapper <collection>
  
  
  This seems overkill; it is easy enough to see if a given tuple is 1) top level and 2) has children.

• Headers and IDs Parent
  
  and Top
  
  
  They seem overkill too. As long as your data is structured hierarchically, it all becomes clear without any need to specifically include it. Including this metadata simply duplicates the information you already have.

• Wrapper <tuple name="children">
  
  
  This can also be omitted without losing information. The mere presence of one element tuple
  
  nested within another is enough children.

I don't know if you have any control over the design or influence of the output XML file format, but if you do, I would suggest a shorter and more ordered structure such as:

<table name="ecatalogue">
    <tuple>
        <atom name="irn">2470</atom>
        <atom name="EADUnitID">da.01</atom>
        <atom name="EADUnitTitle">Some title</atom>
        <tuple>
            <atom name="irn">5416</atom>
            <atom name="EADUnitID">da.01.01</atom>
            <atom name="EADUnitTitle">Child of Some title</atom>
        </tuple>
        <tuple>
            <atom name="irn">6</atom>
            <atom name="EADUnitID">da.01.02</atom>
            <atom name="EADUnitTitle">Child of Some title 2</atom>
            <tuple>
                <atom name="irn">8</atom>
                <atom name="EADUnitID">da.01.02.01</atom>
                <atom name="EADUnitTitle">3rd Generation</atom>
            </tuple>
        </tuple>
    </tuple>
</table>

      

        
        
        
      

    

In this structure, a tuple

can only contain atom

or other tuple

s. We can identify a collection by simply finding any tuple

top level that contains other tuple

s. We can identify children by simply finding anyone tuple

who has a parent tuple

. We can find the top level and parent level headers and IDs by simply selecting tuple

and looking further down the element tree.

This structure is simpler, avoids data duplication, and is arguably clearer and easier to process. However, you know your needs! Do what works for you. :)

Please go through the code and comments, and let me know if you have any lingering questions.

Update 2017-06-15: issues

I added these to your previous XML data sample and tried to apply my previous XSL code. When looking at the problematic tuples, two things happened:

• My previous code uses a feature tokenize
  
  that is only available in XSL 2.0 and later. Your post tags didn't specify XSL 1.0, and I didn't notice that you specified that in your example XSL header.

  I reworked the XSL code above (template that matches tuple
  
  ) to rely only on XSL 1.0 features.

• The values EADUnitID
  
  in the original XML input sample are not representative, so any attempt to code specifically for that sample will fail when applied to your full, invisible input XML.

  Your original sample includes only EADUnitID
  
  format values da.XX
  
  , where XX
  
  are numbers, and the pattern .XX
  
  can be repeated. I've made some assumptions on how to trick this string and compare shapes.

  However, your tuples have values ​​in a EADUnitID
  
  very different format, which looks like DXX.PL.XX
  
  where again XX
  
  are numbers, and the pattern .XX
  
  at the end may be repeated. This means that relying on chunk matching between periods is not a safe approach.

  I reworked the XSL code to fit the entire line front instead irn
  
  , which works reliably.

Take a look at the code and comments, and let me know if anything remains unclear or non-functional.