Defining a file format using XML and XML Schema (XSD) in C#/Java - IV

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

In the previous posts I, II, and III we defined an XML schema and generated XML files which complied with the schema, read/write the XML file with C# programming language and finally, compressed bulk dataset into ASCII-encoded string of binary data. Quite often we have sensible data which is not intended to be published for viewing or modification. In today's post, we will encrypt this data with our private key, so that only the application can decrypt and understand the real meaning of the data. Data encryption is also an integral part of secure exchange of structured data.

Encryption
W3C has a recommendation for encryption of data and representing the data in XML which is hosted at XML Encryption Syntax and Processing. The recommendation is independent of encryption algorithm which is left to the implementing application.

.Net framework provides encryption facilities in System.Security.dll assembly which includes several classes in System.Security.Cryptography namespace for general purpose encryption and some classes in System.Security.Cryptography.Xml namespace to save this encrypted data to XML complying with W3C recommendation for XML encryption.
In our encryption, we will use symmetric encryption algorithm. The .Net implementation of this algorithm resides in RijndaelManaged class. It needs a secret key and an initialization vector for the symmetric algorithm. You can generate a secret key in this web address: https://www.bigbiz.com/genkey.html

For a detailed description of the XML encryption and decryption facilities look at the following MSDN documentation:
http://msdn.microsoft.com/en-us/library/sb7w85t6.aspx

private string m_encrpytionKey = "Wx8Ti1pc0pX"; 
public void Encrypt(XmlElement element) 
{ 
 byte[] encryptionKey = Encoding.ASCII.GetBytes(m_encrpytionKey);
 byte[] iv = Encoding.ASCII.GetBytes(m_encrpytionKey);
 Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(m_encrpytionKey, iv);
 iv = keyGenerator.GetBytes(16);
 encryptionKey = keyGenerator.GetBytes(32);
 RijndaelManaged key = new RijndaelManaged() { Key = encryptionKey, IV = iv };
 EncryptedXml eXml = new EncryptedXml();

 byte[] encryptedElement = eXml.EncryptData(element, key, false);
 EncryptedData edElement = new EncryptedData();
 edElement.Type = EncryptedXml.XmlEncElementUrl;

 string encryptionMethod = null;
 if (key is Rijndael)
 {
  switch (key.KeySize)
  {
  case 128:
   encryptionMethod = EncryptedXml.XmlEncAES128Url;
   break;
  case 192:
   encryptionMethod = EncryptedXml.XmlEncAES192Url;
   break;
  case 256:
   encryptionMethod = EncryptedXml.XmlEncAES256Url;
   break;
  }
 }
 else
 {
  // Throw an exception if the transform is not in the previous categories
  throw new CryptographicException("The specified algorithm is not supported for XML Encryption.");
 }

 edElement.EncryptionMethod = new EncryptionMethod(encryptionMethod);
 edElement.CipherData.CipherValue = encryptedElement;
 EncryptedXml.ReplaceElement(element, edElement, false);
}

Decryption uses the same key. The technique is to load the XML fragment with EncryptedXml.DecryptData method and then replace the data of encrypted XML element with this decrypted data.

public void Decrypt(XmlElement encryptedElement, SymmetricAlgorithm Alg)
{ 
 // Check the arguments.
 if (encryptedElement == null)
  throw new ArgumentNullException("encryptedElement");

 byte[] encryptionKey = Encoding.ASCII.GetBytes(m_encrpytionKey);
 byte[] iv = Encoding.ASCII.GetBytes(m_encrpytionKey);
 
 Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(m_encrpytionKey, iv);
 iv = keyGenerator.GetBytes(16);
 RijndaelManaged key = new RijndaelManaged() { Key = encryptionKey, IV = iv };
 
 // If the EncryptedData element was not found, throw an exception.
 if (encryptedElement == null)
 {
  throw new XmlException("The EncryptedData element was not found.");
 }
 
 // Create an EncryptedData object and populate it.
 EncryptedData edElement = new EncryptedData();
 edElement.LoadXml(encryptedElement);
 
 // Create a new EncryptedXml object.
 EncryptedXml exml = new EncryptedXml();
 
 // Decrypt the element using the symmetric key.
 byte[] rgbOutput = exml.DecryptData(edElement, key);
 
 // Replace the encryptedData element with the plaintext XML element.
 exml.ReplaceData(encryptedElement, rgbOutput);
}

We will encrypt the following XML fragment:

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="false">
        <ValueBulk>.0</ValueBulk>
        <ValueBulk>1.0</ValueBulk>
        <ValueBulk>2.0</ValueBulk>
        <ValueBulk>3.0</ValueBulk>
        <ValueBulk>4.0</ValueBulk>
        <ValueBulk>5.0</ValueBulk>
        <ValueBulk>6.0</ValueBulk>
        <ValueBulk>7.0</ValueBulk>
        <ValueBulk>8.0</ValueBulk>
        <ValueBulk>9.0</ValueBulk>
    </ValueBulks>
</Settings2>

Our encryption method generates the following XML fragment:

<Settings2>
    <ValueX>30</ValueX>
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"  
            xmlns="http://www.w3.org/2001/04/xmlenc#">
        <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc" />
        <CipherData>
        <CipherValue>zVymBnB8d344SyxPMAonVj ... HlVxsxupUNg=</CipherValue>
        </CipherData>
    </EncryptedData>
</Settings2>

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.

Defining a file format using XML and XML Schema (XSD) in C#/Java - III

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

Compressing Data
In the first post of this series we have defined an XML schema and created an example file complied with the schema while in the second post we have written C# code to manipulate this XML file and validate it with the predefined schema.

 The tag-based verbose representation of XML files was a matter of debates due to increasing overhead during serialization and transfer over networks. There were number of groups involved in addressing and effectively overcoming this problem. W3C Efficient XML Interchange (EXI) Working Group has only recently published the recommendation for binary coded XML on March, 2011. There are only few well developed tools which implemented this recommendation. Two open source projects, both programmed in Java, are actively working on it - EXIficient and OpenEX. efficientXML of Agile Delta is a popular XML compression commercial alternative.

In today's post, we will use some general compression mechanisms like Zip to compress large datasets into binary datasets, encode them to Text form and save into the XML file. The problem with such compression is that you need to rework the existing schema otherwise document might be incompatible with the schema. The following code snippet will return the compressed XML fragment as a string:

public string GetXmlFragmentForValueBulks()
{
    XmlDocument xmlDocument = new XmlDocument();
    XmlElement xmlElementCompress = xmlDocument.CreateElement("ValueBulks");
    xmlDocument.AppendChild(xmlElementCompress);
           
    m_bulk.ForEach(f =>
    {
        XmlElement xmlElementChild = xmlDocument.CreateElement("ValueBulk");
        xmlElementChild.InnerText = f.ToString("#.0", Thread.CurrentThread.CurrentUICulture);
        xmlElementCompress.AppendChild(xmlElementChild);
    });

    if (m_compress)
    {
        byte[] byteArray = Encoding.Unicode.GetBytes(xmlElementCompress.InnerXml);
        MemoryStream stream = new MemoryStream(byteArray);
        MemoryStream outStream = new MemoryStream();
        using (GZipStream compression = new GZipStream(outStream, CompressionMode.Compress, true))
        {
            compression.Write(byteArray, 0, byteArray.Length);
            compression.Close(); // we must close it
            byteArray = new byte[outStream.Length];
            outStream.Position = 0;
            outStream.Read(byteArray, 0, byteArray.Length);
        }
        outStream.Close();
        return Convert.ToBase64String(byteArray);
    }

    return xmlElementCompress.InnerXml;
}

Now load the compressed XML fragment and add it to the document in uncompressed form:

XmlDocument xmlDocument = new XmlDocument();
xmlDocument .Load(fileName);

XmlElement xmlElementChild = xmlDocument.SelectSingleNode("ValueBulks") as XmlElement;

byte[] byteArray = Convert.FromBase64String(xmlElementChild.InnerText);
MemoryStream instream = new MemoryStream();
MemoryStream stream = new MemoryStream(byteArray);
stream.Position = 0;
using (GZipStream decompression = new GZipStream(stream, CompressionMode.Decompress))
{
    decompression.CopyTo(instream);
}
byteArray = new byte[instream.Length];
instream.Position = 0;
instream.Read(byteArray, 0, byteArray.Length);
value = Encoding.Unicode.GetString(byteArray);
xmlElementChild.InnerXml = value;

The following uncompressed XML fragment (... shows for continuity):

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="false">
        <ValueBulk>.0</ValueBulk>
        <ValueBulk>1.0</ValueBulk>
        ...
        ...
        <ValueBulk>98.0</ValueBulk>
        <ValueBulk>99.0</ValueBulk>
    </ValueBulks>
</Settings2>

will be converted to the following compressed fragment:

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="true">H4sIAAAAAAAEAOy9B2A...ppD8CFQAA</ValueBulks>
</Settings2>

My upcoming post will be last in this series where I will write about how to encrypt your sensitive data and still comply with the schema.

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.