Clever Geek Handbook

How to parse non-traditional XML files

I need to take what I consider to be a "non-traditional" XML file that I ended up in and convert it to what I consider to be "regular". The XML examples below have been "stripped" of proprietary names or values.

This is what a "non-traditional" XML file looks like:

    <?xml version="1.0" encoding="UTF-8"?>
    <dataView name="document/aDocument" xmlns="http://component.mycompany.com/entity/ds" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <dataItems>
        <dataItem name="item1" location="item1" dataItemType="Long">
            <value xsi:type="xs:string">1</value>
        </dataItem>
        <dataItem name="item2" location="item.seconditem(some item).ref" dataItemType="String">
            <value xsi:type="xs:string">ABC2DEF3</value>
        </dataItem>
    </dataItems>
    <dataRepeaters>
        <dataRepeater name="SomeLevel1" runtimeInfomration="SomeLevel11984371030">
            <dataGroups>
                <dataGroup name="SomeLevel2" location="SomeLevel:(a level).thelevel" keyValue="SomeLevel119843710300" runtimeInformation="19843710300">
                    <dataItems>
                        <dataItem name="someItem1" location="someLevel.aLevel(another item)" dataItemType="String">
                            <value xsi:type="xs:string">12345</value>
                        </dataItem>
                        <dataItem name="someItem2" location="someLevel.aLevel(another item2)" dataItemType="Integer"/>
                        <dataItem name="someItem3" location="someLevel.aLevel(another item3)" dataItemType="ObjectReference">
                            <value xsi:type="xs:string">11:20012:3536</value>
                        </dataItem>
                        <dataItem name="someItem4" location="someLevel.aLevel(another item4)" dataItemType="String"/>
                        <dataItem name="someItem5" location="someLevel.aLevel(another item5)" dataItemType="String">
                            <value xsi:type="xs:string">someItem5 Data</value>
                        </dataItem>
                        <dataItem name="someItem6" location="someLevel.aLevel(another item6)" dataItemType="CurrencyAmount">
                            <value xsi:type="xs:string">123456:USD</value>
                        </dataItem>
                    </dataItems>
                </dataGroup>
                <dataGroup name="SomeLevel3" keyValue="SomeLevel31984371030" runtimeInformation="1984371030">
                    <dataRepeaters>
                        <dataRepeater name="SomeLevel4" runtimeInfomration="SomeLevel4">
                            <dataGroups>
                                <dataGroup name="SomeLevel5" location="anotherLevel.level5(SomeLevel5):someLevel5" keyValue="SomeLevel51984371030-11521863690" runtimeInformation="1984371030-11521863690">
                                    <dataItems>
                                        <dataItem name="someItem6" location="someLevel.aLevel(another item6)" dataItemType="BigDecimal">
                                            <value xsi:type="xs:string">303.149</value>
                                        </dataItem>
                                        <dataItem name="someItem7" location="someLevel.aLevel(another item7)" dataItemType="CurrencyAmount">
                                            <value xsi:type="xs:string">117:USD</value>
                                        </dataItem>
                                        <dataItem name="someItem8" location="someLevel.aLevel(another item8)" dataItemType="String">
                                            <value xsi:type="xs:string">117.000000:USD</value>
                                        </dataItem>
                                        <dataItem name="someItem9" location="someLevel.aLevel(another item9)" dataItemType="CurrencyAmount">
                                            <value xsi:type="xs:string">117:USD</value>
                                        </dataItem>
                                    </dataItems>
                                    <dataRepeaters>
                                        <dataRepeater name="SomeLevel6" runtimeInfomration="someLevel6">
                                            <dataGroups/>
                                            </dataRepeater>
                                    </dataRepeaters>
                                </dataGroup>
                                <dataGroup name="SomeLevel7" location="anotherLevel.level5(SomeLevel7):someLevel7" keyValue="SomeLevel71984371030-11521863690" runtimeInformation="1984371030-11521863690">
                                    <dataItems>
                                        <dataItem name="someItem10" location="someLevel.aLevel(another item10)" dataItemType="BigDecimal">
                                            <value xsi:type="xs:string">292.741</value>
                                        </dataItem>
                                        <dataItem name="someItem11" location="someLevel.aLevel(another item11)" dataItemType="CurrencyAmount">
                                            <value xsi:type="xs:string">165:USD</value>
                                        </dataItem>
                                        <dataItem name="someItem12" location="someLevel.aLevel(another item12)" dataItemType="String">
                                            <value xsi:type="xs:string">165.000000:USD</value>
                                        </dataItem>
                                        <dataItem name="someItem13" location="someLevel.aLevel(another item13)" dataItemType="CurrencyAmount">
                                            <value xsi:type="xs:string">165:USD</value>
                                        </dataItem>
                                    </dataItems>
                                    <dataRepeaters/>
                                </dataGroup>
                                <dataGroup name="SomeLevel8" location="anotherLevel.level5(SomeLevel8):someLevel8" keyValue="SomeLevel81984371030-11521863690" runtimeInformation="1984371030-11521863690">
                                    <dataItems>
                                        <dataItem name="someItem14" location="someLevel.aLevel(another item14)" dataItemType="BigDecimal">
                                            <value xsi:type="xs:string">369.075</value>
                                        </dataItem>
                                        <dataItem name="someItem15" location="someLevel.aLevel(another item15)" dataItemType="CurrencyAmount">
                                            <value xsi:type="xs:string">598:USD</value>
                                        </dataItem>
                                        <dataItem name="someItem16" location="someLevel.aLevel(another item16)" dataItemType="String">
                                            <value xsi:type="xs:string">598.000000:USD</value>
                                        </dataItem>
                                    </dataItems>
                                    <dataRepeaters>
                                        <dataRepeater name="SomeLevel9" runtimeInfomration="someLevel9">
                                            <dataGroups>
                                                <dataGroup name="SomeLevel10" location="ownedAuto.AgreementActual(Liability).ConstantRole(Policy Form):policyForm" keyValue="aomeLevel101984371030-11521863690-115218636900" runtimeInformation="1984371030-11521863690-115218636900">
                                                    <dataItems>
                                                        <dataItem name="someItem17" location="someLevel.aLevel(another item17)" dataItemType="String">
                                                            <value xsi:type="xs:string">Some Data | ABC 123</value>
                                                        </dataItem>
                                                        <dataItem name="someItem18" location="someLevel.aLevel(another item18)" dataItemType="Integer">
                                                            <value xsi:type="xs:string">2</value>
                                                        </dataItem>
                                                    </dataItems>
                                                </dataGroup>
                                            </dataGroups>
                                        </dataRepeater>
                                        <dataRepeater name="SomeLevel10" runtimeInfomration="someLevel11">
                                            <dataGroups/>
                                        </dataRepeater>
                                    </dataRepeaters>
                                </dataGroup>
                            </dataGroups>
                        </dataRepeater>
                    </dataRepeaters>
                </dataGroup>
            </dataGroups>
        </dataRepeater>
    </dataRepeaters>
</dataView>

I am trying to convert it to something like this:

    <?xml version="1.0" encoding="UTF-8"?>
<dataView name="document/aDocument" xmlns="http://component.mycompany.com/entity/ds" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <item1>1</item1>
  <item2>ABC2DEF3</item2>
  <SomeLevel1>
    <SomeLevel2>
      <someItem1>12345f</someItem1>
      <someItem2></someItem2>     
      <someItem3>11:20012:3536</someItem3>
      <someItem4>12345f</someItem4>
      <someItem5>someItem5 Data</someItem5>
      <someItem6>123456:USD</someItem6>
    </SomeLevel2>
    <SomeLevel3>
      <SomeLevel4>
        <SomeLevel5>
          <someItem7>303.149</someItem7>
          <someItem8>117:USD</someItem8>
          <someItem9>117.000000:USD</someItem9>
          <someItem10>117:USD</someItem10>
          <SomeLevel6></SomeLevel6>
        </SomeLevel5>
        <SomeLevel7>
          <someItem11>292.741</someItem11>
          <someItem12>165:USD</someItem12>
          <someItem13>165.000000:USD</someItem13>
          <someItem14>165:USD</someItem14>
        </SomeLevel7>
        <SomeLevel8>
          <someItem15>369.075</someItem15>
          <someItem16>598:USD</someItem16>
          <someItem17>598.000000:USD</someItem17>
          <SomeLevel9>
            <SomeLevel10>
              <someItem18>Some Data | ABC 123</someItem18>
              <someItem19>2</someItem19>
            </SomeLevel10>
          </SomeLevel9>
          <SomeLevel11></SomeLevel11>
        </SomeLevel8>
      </SomeLevel4>
    </SomeLevel3>
  </SomeLevel1>
</dataView>

So basically I am trying to take the "name" attribute and make it an XML node / tag and a value between the <value>

node / tag and use that as the "value" for the XML node / tag.

I've tried several different parsing / writing "routines", and below two are the ones that have become closer to me. So I leave the rest, because they will just be "noise."

Here's what I've tried so far:

private Document xmlDocument;

public void convertXML() {
      xmlDocument = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new StringReader("DSExample.xml")));

      DocumentBuilderFactory docFactory = DocumentBuilderFactory.newInstance();
      DocumentBuilder docBuilder = docFactory.newDocumentBuilder();

      Node dsNode = xmlDocument.getDocumentElement();

      xmlDocument = docBuilder.newDocument();

      // Attempt 1:   
      doSomethingDS(dsNode); // This will convert the XML but with NO nesting - creates example output 1
      // Attempt 2:
      traverseDS(dsNode); // This will create an XML with ONLY the root dataItems - Don't know why - creates example output 2
}

public void doSomethingDS(Node node) {  
    System.out.println(node.getNodeName());
    Element xmlElement = null;

    // This will be the "root" element/node "dataView":
    if(node.getNodeName() != null && node.getNodeName().equalsIgnoreCase("dataView")) {

      // Debugging: Print the node
      printNode(node);

      String nodeName = node.getNodeName().trim();
      rootElement = xmlDocument.createElement(nodeName);
      mPreviousElement = rootElement;

      addAllAttributesToRootElement((Element) node, rootElement);

      xmlDocument.appendChild(rootElement);
    }
    else {

        // Debugging: Print the node
        printNode(node);

        // If has "name" property - create element/node
        if(node.getAttributes() != null && node.getAttributes().getNamedItem("name") != null) {
          xmlElement = createElement(xmlDocument, node);

          if(xmlElement !=null && xmlDocument.getFirstChild() != null) {
            xmlDocument.getFirstChild().appendChild(xmlElement);  
          }
        }

    }    

    NodeList nodeList = node.getChildNodes();
    for(int i = 0; i < nodeList.getLength(); i++) {
      Node currentNode = nodeList.item(i);
      if(currentNode.getNodeType() == Node.ELEMENT_NODE) {
        // recursively call this method for all the children which are of type Element
        doSomethingDS(currentNode);
      }
    }
} // End of doSomethingDS() 


public void traverseDS(Node parentNode) {
    // This will create an XML with ONLY the root dataItems - Don't know why.

    Element xmlElement = null;

    //****************************************************************
    // This will be the "root" element/node "dataView":
    if(parentNode.getNodeName() != null && parentNode.getNodeName().equalsIgnoreCase("dataView")) {

      // Debugging: Print the node
      printNode(parentNode);

      String nodeName = parentNode.getNodeName().trim();
      rootElement = xmlDocument.createElement(nodeName);
      mPreviousElement = rootElement;

      if(!isRootNodeSet) {        
        mRootNode = parentNode;
        isRootNodeSet = true;
      }

      addAllAttributesToRootElement((Element) parentNode, rootElement);

      xmlDocument.appendChild(rootElement);

      // traverse children
      Node theNode = parentNode.getFirstChild();
      if(theNode != null) {
        theNode = theNode.getNextSibling();
      }
      else if (isRootNodeSet) {
        theNode = mRootNode.getNextSibling();
      }

      traverseDS(theNode);
    }
    else {

      // traverse all nodes that belong to the parent
      for(Node theNode = parentNode.getFirstChild(); theNode != null; theNode = theNode.getNextSibling()) {

        // Debugging: Print the node
        printNode(theNode);

        // If has "name" property - create element/node
        if(theNode.getAttributes() != null && theNode.getAttributes().getNamedItem("name") != null) {
          // Create new Element/Node
          xmlElement = createElement(xmlDocument, theNode);
          if(xmlElement !=null && xmlDocument.getFirstChild() != null) {
            xmlDocument.getFirstChild().appendChild(xmlElement);  
          }
          else {
            System.out.println(" not a node we wanted?");
          }
        }

        // traverse children     
        traverseDS(theNode);
      }
    }
} // End of traverseDS()
private Element createElement(Document aDoc, Node aNode) {
    Element xmlElement = null;
    String elementName = "";

    NamedNodeMap dataNodeMap = aNode.getAttributes();
    if(dataNodeMap != null && dataNodeMap.getNamedItem("name") != null) {

      elementName = dataNodeMap.getNamedItem("name").getTextContent();
      xmlElement = aDoc.createElement(elementName);

      // if node = "dataItem" - walk node to get <value> node
      // Note: some "dataItem" nodes also have a <previousValue> node - what to do with these????
      if(aNode.getNodeName() != null && aNode.getNodeName().equalsIgnoreCase("dataItem")) {
        // Walk Node to get <value> node
        NodeList childNodes = aNode.getChildNodes();
        int childNodesLength = childNodes.getLength();
        for(int x = 0; x < childNodesLength; x++) {
          Node childNode = childNodes.item(x);
          if(childNode.getNodeName() != null && childNode.getNodeName().equalsIgnoreCase("value")) {
            xmlElement.setTextContent(childNode.getTextContent());
            break;
          }
        }
      }
    }
    return xmlElement;
} // End of createElement()

Sample output 1 (generated by doSomethingDS ()):

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<dataView xmlns="http://component.mycompany.com/entity/ds" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="document/aDocument">
    <item1>1</item1>
    <item2>ABC2DEF3</item2>
    <SomeLevel1/>
    <SomeLevel2/>
    <someItem1>12345</someItem1>
    <someItem2/>
    <someItem3>11:20012:3536</someItem3>
    <someItem4/>
    <someItem5>someItem5 Data</someItem5>
    <someItem6>123456:USD</someItem6>
    <SomeLevel3/>
    <SomeLevel4/>
    <SomeLevel5/>
    <someItem6>303.149</someItem6>
    <someItem7>117:USD</someItem7>
    <someItem8>117.000000:USD</someItem8>
    <someItem9>117:USD</someItem9>
    <SomeLevel6/>
    <SomeLevel7/>
    <someItem10>292.741</someItem10>
    <someItem11>165:USD</someItem11>
    <someItem12>165.000000:USD</someItem12>
    <someItem13>165:USD</someItem13>
    <SomeLevel8/>
    <someItem14>369.075</someItem14>
    <someItem15>598:USD</someItem15>
    <someItem16>598.000000:USD</someItem16>
    <SomeLevel9/>
    <SomeLevel10/>
    <someItem17>Some Data | ABC 123</someItem17>
    <someItem18>2</someItem18>
    <SomeLevel11/>
</dataView>

Sample Output 2 (Created by traverseDS () method):

    <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<dataView xmlns="http://component.mycompany.com/entity/ds" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" name="document/aDocument">
    <item1>1</item1>
    <item2>ABC2DEF3</item2>
</dataView>

Now here's the catch .... What I get is dynamic, I could get a Mac truck once and Toyota the next, or a mixture of the two. And the XML size I get can be small or large. So it's hard to just use XPath or go directly to specific elements as I don't know what I will get. This is why I basically walk / traverse the DOM a node at a time to only extract what can be used.

So my question is: What am I doing wrong in my parsing / writing the XML file so that it doesn't look the way I am trying to make it look?

+3


source to share


2 answers


It's (almost) trivially easy with XSLT:

<xsl:stylesheet 
  version="1.0" 
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:ds="http://component.mycompany.com/entity/ds" 
  xmlns:xs="http://www.w3.org/2001/XMLSchema" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://component.mycompany.com/entity/ds" 
>
  <xsl:output indent="yes" />
  <xsl:strip-space elements="*" />

  <xsl:template match="ds:dataView" priority="1">
    <xsl:copy>
      <xsl:apply-templates />
    </xsl:copy>
  </xsl:template>

  <xsl:template match="ds:*[@name]">
    <xsl:element name="{@name}">
      <xsl:apply-templates />
    </xsl:element>
  </xsl:template>

  <xsl:template match="text()">
    <xsl:value-of select="normalize-space()" />
  </xsl:template>

</xsl:stylesheet>

applies your input gives you:

<dataView xmlns="http://component.mycompany.com/entity/ds" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <item1>1</item1>
  <item2>ABC2DEF3</item2>
  <SomeLevel1>
    <SomeLevel2>
      <someItem1>12345</someItem1>
      <someItem2 />
      <someItem3>11:20012:3536</someItem3>
      <someItem4 />
      <someItem5>someItem5 Data</someItem5>
      <someItem6>123456:USD</someItem6>
    </SomeLevel2>
    <SomeLevel3>
      <SomeLevel4>
        <SomeLevel5>
          <someItem6>303.149</someItem6>
          <someItem7>117:USD</someItem7>
          <someItem8>117.000000:USD</someItem8>
          <someItem9>117:USD</someItem9>
          <SomeLevel6 />
        </SomeLevel5>
        <SomeLevel7>
          <someItem10>292.741</someItem10>
          <someItem11>165:USD</someItem11>
          <someItem12>165.000000:USD</someItem12>
          <someItem13>165:USD</someItem13>
        </SomeLevel7>
        <SomeLevel8>
          <someItem14>369.075</someItem14>
          <someItem15>598:USD</someItem15>
          <someItem16>598.000000:USD</someItem16>
          <SomeLevel9>
            <SomeLevel10>
              <someItem17>Some Data | ABC 123</someItem17>
              <someItem18>2</someItem18>
            </SomeLevel10>
          </SomeLevel9>
          <SomeLevel10 />
        </SomeLevel8>
      </SomeLevel4>
    </SomeLevel3>
  </SomeLevel1>
</dataView>

Notes:

After the numbered element names ( someItem1

thru someItem6

) will eventually come back and bite you. XML elements have natural ordering (or you can have a "number" attribute), casting a counter to a name is bad and incorrect. Take my advice and don't (*) .



To understand the solution here, do a quick crash:

  • XSLT processors run XML nodes through templates, finding the best match for the XML node.
  • The process starts at the root node.
  • Since there is no pattern explicitly matching the root node ( <xsl:template match="/">

    ), the default rule is to output nothing, but find patterns for child nodes.
  • The first (and only) child of a node is the document element ( <ds:dataView>

    ).
  • Pattern # 1 matches this node, tells the processor to copy it and process its children ( <xsl:apply-templates />

    ).
  • Each element with @name

    matches pattern # 2, which renders the element with that name and processes its children again.
  • Every other element (such as those <value>

    nodes) is - in the absence of a matching template - processed according to the specified default standard.
  • Text nodes are matched with pattern # 3, which outputs their truncated value.

Additional Notes:

  • The expression in curly braces is called the attribute value pattern.
  • The template priority="1"

    ensures that this template is selected for the element <ds:dataView>

    , as the other will match.
  • Your final document is still in "http://component.mycompany.com/entity/ds"

    . This may or may not be correct. I suspect this is not correct, technically speaking, but you decide.

(*)<xsl:element name="{translate(@name, '0123456789', '')}"

would be one of the options to lose counters.

+5


source


XSLT was created for just such a task. As long as your Source-XML (non-traditional xml) is well-formed (correct xml syntax) to be read by a standard XML parser like the DOM, you can use it.

It works like this: you create an xsl-template file containing a descriptive description of how the original xml should be converted to a different format. The transformation itself is performed by the XSL-Transformer. Xsl is also an xml format, so it should be easy to read as long as the conversion isn't too complicated.



It explains how this can be implemented: http://docs.oracle.com/javase/tutorial/jaxp/xslt/transformingXML.html

0


source






More articles:


All Articles