keensocial.com has been discontinued

My keensocial.com has been deactivated. Some of my blog posts will not work properly since they need files from keensocial.com. I'm terribly busy now, so will not be able to fix it in the next days. Sorry for the inconvenience.

Defining a file format using XML and XML Schema (XSD) in C#/Java - V

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

In the previous four posts (I, II, III, IV), we developed an XML file format on .Net platform. While developing a new application using this technique I needed some advanced features of XSD 1.1 like Type Alternatives and Assertion which are not yet supported by Microsoft.

The type alternatives provides a mechanism to choose a particular type for an element based on the value of an attribute. The element xs:alternative does the trick. Following is an XSD snippet for alternative data type:

<xs:element name="Animal" type="AnimalType">
    <xs:alternative test="@kind eq 'dog'" type="DogType" />
    <xs:alternative test="@kind eq 'cat'" type="CatType" />
</xs:element>

<xs:complexType name="AnimalType">
    <xs:sequence>
        <xs:element> name="Name" type="xs:string" />
        <xs:element> name="Talk" type="xs:string" />
    </xs:sequence>
    <xs:attribute name="kind" />
</xs:complexType>

<xs:complexType name="DogType">
    <xs:complexContent>
        <xs:extension base="AnimalType">
            <xs:sequence>
                <xs:element> name="Veterinarian" type="xs:string" />
            </xs:sequence>
        </xs:extension>
    </xs:complexContent>
</xs:complexType>

<xs:complexType name="CatType">
    <xs:complexContent>
        <xs:extension base="AnimalType">
            <xs:sequence>
                <xs:element> name="Owner" type="xs:string" />
            </xs:sequence>
        </xs:extension>
    </xs:complexContent>
</xs:complexType>

The corresponding XML can be the following:

<Animal kind="dog">
    <Name>Woof</Name>
    <Talk>Woof! Woof!</Talk>
    <Veterinarian>Mike</Veterinarian>
</Animal>

Assertion, as the name suggests, is a mechanism to put constraints on a particular element. It is achieved through the xs:assert element.

<xs:complexType name="DogType">
    <xs:complexContent>
        <xs:extension base="AnimalType">
            <xs:sequence>
                <xs:element> name="Veterinarian" type="xs:string" />
            </xs:sequence>
        </xs:extension>
    </xs:complexContent>
    <xs:assert test="(Talk eq 'Woof!') or (Talk eq 'Woof! Woof!')" />
</xs:complexType>

While validating my XML file with the new schema using my C# program, the validation was failing. After some investigation I have found that Microsoft doesn't support XSD 1.1 yet. So I have to look for alternatives. Apache Xerces-C++ didn't implement XSD1.1 support yet, but Xerces-J did.

After some googling in the Internet, it was clear that Java 7 has taken over the implementation from Xerces-J. I created a Java desktop application in Eclipse and recreate the C# program. Compressing and encrypting were also done using (GZIPOutputStream) java.util.zip and Apache security (com.sun.org.apache.xml.internal.security) packages respectively. To validate against XSD1.1 you need to use SchemaFactory and all your source schema to this factory. Following code snippet shows how to validate and parse an XML file:

DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
Path xmlFile = Paths.get(fileName);
Document xmlDoc = null;
try (BufferedInputStream in = new BufferedInputStream(Files.newInputStream(xmlFile))){

 SchemaFactory sf = SchemaFactory.newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
 sf.setErrorHandler(new ErrorHandler() {

  @Override
  public void error(SAXParseException arg0) throws SAXException {
   System.out.println("Error occurred: " + arg0.getMessage());
  }

  @Override
  public void fatalError(SAXParseException arg0)
    throws SAXException {
   System.out.println("Fatal error occurred: " + arg0.getMessage());
  }

  @Override
  public void warning(SAXParseException arg0) throws SAXException {
   System.out.println("Warning: " + arg0.getMessage());
  }
 });

 //StreamSource schemaDocument = new StreamSource(schemaName);
 //Schema s = sf.newSchema(schemaDocument);
 
 // we want to validate against the following schemas; already existing in our res folder
 sf.setResourceResolver(new ResourceResolver());
 Schema s = sf.newSchema(new StreamSource[] { 
   new StreamSource(this.getClass().getResourceAsStream("res/xmldsig-core-schema.xsd"), "xmldsig-core-schema.xsd"),
            new StreamSource(this.getClass().getResourceAsStream("res/xenc-schema.xsd"), "xenc-schema.xsd"),
            new StreamSource(this.getClass().getResourceAsStream("res/set.xsd"), "set.xsd")
   }
 );     
 
 Validator v = s.newValidator();
 StreamSource instanceDocument = new StreamSource(fileName);
 v.validate(instanceDocument);
 
 xmlDoc = builder.parse(in);
  
} catch(SAXException | IOException e) {
  e.printStackTrace();
  System.exit(1);
}

The set.xsd file has been extended to work with compressed and encrypted elements.

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xenc="http://www.w3.org/2001/04/xmlenc#" elementFormDefault="qualified">

  <xsd:import namespace="http://www.w3.org/2001/04/xmlenc#"></xsd:import>

  <xsd:element name="XmlFileFormat">
    <xsd:complexType>
      <xsd:sequence>
        <xsd:element name="Value1" type="xsd:integer"></xsd:element>
        <xsd:element name="Value2" type="xsd:string"></xsd:element>
        <xsd:element name="Value3" type="xsd:float"></xsd:element>
        <xsd:element name="Settings2">
          <xsd:complexType>
            <xsd:sequence minOccurs="0" maxOccurs="1">
              <xsd:element name="ValueX" type="xsd:integer">
              </xsd:element>
              <xsd:element name="ValueBulks">
                <xsd:complexType mixed="true">
                  <xsd:sequence>
                    <xsd:element name="ValueBulk" type="xsd:float"
                        minOccurs="0" maxOccurs="unbounded"/>
                    <xsd:element ref="xenc:EncryptedData" minOccurs="0" maxOccurs="1" />
                  </xsd:sequence>
                  <xsd:attribute name="Compress" use="optional" type="xsd:boolean" />
                  <xsd:attribute name="Encrypt" use="optional" type="xsd:boolean" />
                </xsd:complexType>
              </xsd:element>
            </xsd:sequence>
          </xsd:complexType>
        </xsd:element>
      </xsd:sequence>
    </xsd:complexType>
  </xsd:element>
</xsd:schema>

The complete source code can be downloaded from the following link:
http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat-j.zip

Advanced Developer Conference

Last week, I attended the "Advanced Developers Conference 2001" in Frankenthal, Germany. There were some interesting presentations on upcoming version of .Net Framework 4.5, Visual Studio 11 and Windows 8. Since only the preview versions of these products are currently available, the speakers were forced to only present overviews and expected features.

Among nearly 40 presentations from renowned .Net speakers in Germany, I have listened to the following:

KEYNOTE: Cloud-Strategie und Zukunft der Cloud (Cloud Strategy and the future of cloud)
Christian Liensberger



Trennung tut weh: UI und Anwendung mit MVVM und Flow Design trennen (Separation hurts: Separate UI and Application with MVVM and Flow Design)
Stefan Lieser



Parallelisierung - Einführung (Parallelization - Introduction)
Parallelisierung - am Beispiel (Parallelization - Example)
Bernd Marquardt

Parallelisierung - im User Interface (Parallelization - in User Interface)
Bernd Marquardt

Garbage Collector in .NET 4.5 - Teil 1 (Garbage Collector in .NET 4.5 - Part 1)
Garbage Collector in .NET 4.5 - Teil 2 (Garbage Collector in .NET 4.5 - Part 1)
Patric Boscolo

Visual Studio 11 Application Lifecycle Management
Christian Binder, Neno Loje

Neues von der Microsoft BUILD-Konferenz: Ausblick auf Windows 8 (New from the Microsoft BUILD-Conference: Prospects for Windows 8)
Hannes Preishuber

Was ist neu in Visual Studio 2011 und .NET 4.5 - Preview (What is new in Visual Studio 2011 and .Net 4.5 - Preview)
Dr. Holger Schwichtenberg, Neno Loje      


Software Design mit C# 4 - Patterns und fortgeschrittene Sprachelemente (Software Design with C#4 - Patterns und forg
Manfred Steyer

.NET Deep Dive
David Tielke  

Entwickeln von "Metro-Style-Apps" für Windows 8 (Development of "Metro-Style-Apps" for Windows 8)
Hannes Preishuber 

Testbare Systeme mit Dependency Injection und Mocking Framework (Testable System with Dependency Injection and Mocking Framework)
Manfred Steyer  

10 Minute Guide to DirectX 11 Programming in C++

DirectX is already 16 years old. There are tons of resources available online for DirectX programming. This post will give you just some idea how quickly a simple Direct3D based application can be built. This will serve as a starting point for my future posts on XNA Game Studio or SlimDX library for managed 3D visualization on .Net platform.

Prerequisites
1. DirectX Software Development Kit. It will include DirectX End-User Runtimes (June 2010) which is necessary to run your DirectX application on client computer.
2. Some knowledge of COM programming
3. Visual C++ 2010 Express IDE. If you are comfortable with building C++ application from command line, you don't need this IDE.

Concepts

1. Direct3D: Used for all 3D graphics in DirectX
2. Rendering pipeline: DirectX processes build the scene in multiple layers called rendering pipeline.

3. Swap Chain - This is ideally buffers for the rendering. Double buffering with 1 front buffer and 1 back buffer is a popular technique to reduce flicker effect.
   
4. Rendering: Send your buffer to the output.
   

Implementation

1. Create a Win32 Project using Visual C++ 2010 Express "New Project" Wizard like the following and let's name the project RenderDirectX:
   
   
   
   
   
   
   
2. Right click your project in the solution explorer and select properties. Select to C/C++ from the tree and add <directx-installation-dir>\Include to "Additional Include Directories" Field. Replace <directx-installation-dir< with the installation directory of your DirectX.
   
   
   
   
   
3. Now select Linker from the tree in the property pages and add <directx-installation-dir>\Lib\X86 to "Additional Library Directories" Field. Replace <directx-installation-dir> with the installation directory of your DirectX.
   
   
   
   
   
4.  Now select Linker->Input and add d3d11.lib;d3dx11.lib;dxerr.lib; to "Additional Dependencies" Field.
   
   
   
   
   
5. Include the following headers in RenderDirectX.cpp file:
   

   #include<d3d11.h>
   #include<d3dx11.h>
   #include<dxerr.h>
   

6. Initialize the Direct3D device and swap chain with intended feature level and driver types:
   

   ID3D11Device* piD3dDevice;
   D3D_FEATURE_LEVEL eFeatureLevel;
   ID3D11DeviceContext* piD3dContext;
   D3D_FEATURE_LEVEL aFeatureLevel[] =
   {
           D3D_FEATURE_LEVEL_11_0,
           D3D_FEATURE_LEVEL_10_1,
           D3D_FEATURE_LEVEL_10_0
   };
   
   DXGI_SWAP_CHAIN_DESC stSwapChainDesc; // initialize the swap description 
   IDXGISwapChain* piSwapChain;
   D3D11CreateDeviceAndSwapChain( 0, D3D_DRIVER_TYPE_HARDWARE, 0, 0, aFeatureLevel, 
            ARRAYSIZE(aFeatureLevel),  D3D11_SDK_VERSION, &stSwapChainDesc, 
            &piSwapChain, &piD3dDevice, &eFeatureLevel, &piD3dContext );
   

7. Create a render target view and set it to the device context:
   

   ID3D11Device* piD3dDevice;
   ID3D11Texture2D* piTexture;
   piSwapChain->GetBuffer(0, __uuidof( ID3D11Texture2D ),    ( LPVOID* )&piTexture);
   ID3D11RenderTargetView* piTargetView;
   piD3dDevice->CreateRenderTargetView(bufferTexture, 0, &piTargetView);
   piD3dContext->OMSetRenderTargets( 1, &piTargetView, 0 );
   //Bind viewports to rasterizer stage of pipeline
   piD3dContext->RSSetViewports(1, &viewport);
   

8. Change your windows message so that it can update the screen even if no event occurs and draw your image:
   

   ID3D11Device* piD3dDevice;
   while( msg.message != WM_QUIT )
   {
           if( PeekMessage( &msg, 0, 0, 0, PM_REMOVE ) )
           {
               TranslateMessage( &msg );
               DispatchMessage( &msg );
           }
           else
           {
               // Update and Draw 
               float afColor[4] = { 1.0f, 0.0f, 0.0f, 1.0f };
               piD3dContext->ClearRenderTargetView(piTargetView, afColor);
               piSwapChain->Present(0, 0);
           }
   }
   

9. Uninitialize the device and release resources
   

   ID3D11Device* piD3dDevice;
   piTargetView->Release( );
   piSwapChain->Release( );
   piD3dContext->Release( );
   piD3dDevice->Release( );
   

This is rather quick and dirty way of coding with no error handling and object orientation. For real application you should proceed in more object oriented way and use smart pointers to automatically release the resources. DirectX programming is complex and one needs years of experience to master it due to multidisciplinary knowledge required to create attractive visualization effects. To render even a simple content, you have to understand Geometry, Vertex & Vertex Buffer and Pixel Shader. In future posts, I will give more insights into the wonderful world of Visualization, Animation, Gaming on DirectX.

I hope this tutorial will give you a starting point to play with 3D visualization on Windows platform. The complete code for this post can be downloaded from http://keensocial.freeiz.com/blogs/directx/renderdirectx/renderdirectx.zip

Defining a file format using XML and XML Schema (XSD) in C#/Java - IV

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

In the previous posts I, II, and III we defined an XML schema and generated XML files which complied with the schema, read/write the XML file with C# programming language and finally, compressed bulk dataset into ASCII-encoded string of binary data. Quite often we have sensible data which is not intended to be published for viewing or modification. In today's post, we will encrypt this data with our private key, so that only the application can decrypt and understand the real meaning of the data. Data encryption is also an integral part of secure exchange of structured data.

Encryption
W3C has a recommendation for encryption of data and representing the data in XML which is hosted at XML Encryption Syntax and Processing. The recommendation is independent of encryption algorithm which is left to the implementing application.

.Net framework provides encryption facilities in System.Security.dll assembly which includes several classes in System.Security.Cryptography namespace for general purpose encryption and some classes in System.Security.Cryptography.Xml namespace to save this encrypted data to XML complying with W3C recommendation for XML encryption.
In our encryption, we will use symmetric encryption algorithm. The .Net implementation of this algorithm resides in RijndaelManaged class. It needs a secret key and an initialization vector for the symmetric algorithm. You can generate a secret key in this web address: https://www.bigbiz.com/genkey.html

For a detailed description of the XML encryption and decryption facilities look at the following MSDN documentation:
http://msdn.microsoft.com/en-us/library/sb7w85t6.aspx

private string m_encrpytionKey = "Wx8Ti1pc0pX"; 
public void Encrypt(XmlElement element) 
{ 
 byte[] encryptionKey = Encoding.ASCII.GetBytes(m_encrpytionKey);
 byte[] iv = Encoding.ASCII.GetBytes(m_encrpytionKey);
 Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(m_encrpytionKey, iv);
 iv = keyGenerator.GetBytes(16);
 encryptionKey = keyGenerator.GetBytes(32);
 RijndaelManaged key = new RijndaelManaged() { Key = encryptionKey, IV = iv };
 EncryptedXml eXml = new EncryptedXml();

 byte[] encryptedElement = eXml.EncryptData(element, key, false);
 EncryptedData edElement = new EncryptedData();
 edElement.Type = EncryptedXml.XmlEncElementUrl;

 string encryptionMethod = null;
 if (key is Rijndael)
 {
  switch (key.KeySize)
  {
  case 128:
   encryptionMethod = EncryptedXml.XmlEncAES128Url;
   break;
  case 192:
   encryptionMethod = EncryptedXml.XmlEncAES192Url;
   break;
  case 256:
   encryptionMethod = EncryptedXml.XmlEncAES256Url;
   break;
  }
 }
 else
 {
  // Throw an exception if the transform is not in the previous categories
  throw new CryptographicException("The specified algorithm is not supported for XML Encryption.");
 }

 edElement.EncryptionMethod = new EncryptionMethod(encryptionMethod);
 edElement.CipherData.CipherValue = encryptedElement;
 EncryptedXml.ReplaceElement(element, edElement, false);
}

Decryption uses the same key. The technique is to load the XML fragment with EncryptedXml.DecryptData method and then replace the data of encrypted XML element with this decrypted data.

public void Decrypt(XmlElement encryptedElement, SymmetricAlgorithm Alg)
{ 
 // Check the arguments.
 if (encryptedElement == null)
  throw new ArgumentNullException("encryptedElement");

 byte[] encryptionKey = Encoding.ASCII.GetBytes(m_encrpytionKey);
 byte[] iv = Encoding.ASCII.GetBytes(m_encrpytionKey);
 
 Rfc2898DeriveBytes keyGenerator = new Rfc2898DeriveBytes(m_encrpytionKey, iv);
 iv = keyGenerator.GetBytes(16);
 RijndaelManaged key = new RijndaelManaged() { Key = encryptionKey, IV = iv };
 
 // If the EncryptedData element was not found, throw an exception.
 if (encryptedElement == null)
 {
  throw new XmlException("The EncryptedData element was not found.");
 }
 
 // Create an EncryptedData object and populate it.
 EncryptedData edElement = new EncryptedData();
 edElement.LoadXml(encryptedElement);
 
 // Create a new EncryptedXml object.
 EncryptedXml exml = new EncryptedXml();
 
 // Decrypt the element using the symmetric key.
 byte[] rgbOutput = exml.DecryptData(edElement, key);
 
 // Replace the encryptedData element with the plaintext XML element.
 exml.ReplaceData(encryptedElement, rgbOutput);
}

We will encrypt the following XML fragment:

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="false">
        <ValueBulk>.0</ValueBulk>
        <ValueBulk>1.0</ValueBulk>
        <ValueBulk>2.0</ValueBulk>
        <ValueBulk>3.0</ValueBulk>
        <ValueBulk>4.0</ValueBulk>
        <ValueBulk>5.0</ValueBulk>
        <ValueBulk>6.0</ValueBulk>
        <ValueBulk>7.0</ValueBulk>
        <ValueBulk>8.0</ValueBulk>
        <ValueBulk>9.0</ValueBulk>
    </ValueBulks>
</Settings2>

Our encryption method generates the following XML fragment:

<Settings2>
    <ValueX>30</ValueX>
    <EncryptedData Type="http://www.w3.org/2001/04/xmlenc#Element"  
            xmlns="http://www.w3.org/2001/04/xmlenc#">
        <EncryptionMethod Algorithm="http://www.w3.org/2001/04/xmlenc#aes256-cbc" />
        <CipherData>
        <CipherValue>zVymBnB8d344SyxPMAonVj ... HlVxsxupUNg=</CipherValue>
        </CipherData>
    </EncryptedData>
</Settings2>

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.

Defining a file format using XML and XML Schema (XSD) in C#/Java - III

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

Compressing Data
In the first post of this series we have defined an XML schema and created an example file complied with the schema while in the second post we have written C# code to manipulate this XML file and validate it with the predefined schema.

 The tag-based verbose representation of XML files was a matter of debates due to increasing overhead during serialization and transfer over networks. There were number of groups involved in addressing and effectively overcoming this problem. W3C Efficient XML Interchange (EXI) Working Group has only recently published the recommendation for binary coded XML on March, 2011. There are only few well developed tools which implemented this recommendation. Two open source projects, both programmed in Java, are actively working on it - EXIficient and OpenEX. efficientXML of Agile Delta is a popular XML compression commercial alternative.

In today's post, we will use some general compression mechanisms like Zip to compress large datasets into binary datasets, encode them to Text form and save into the XML file. The problem with such compression is that you need to rework the existing schema otherwise document might be incompatible with the schema. The following code snippet will return the compressed XML fragment as a string:

public string GetXmlFragmentForValueBulks()
{
    XmlDocument xmlDocument = new XmlDocument();
    XmlElement xmlElementCompress = xmlDocument.CreateElement("ValueBulks");
    xmlDocument.AppendChild(xmlElementCompress);
           
    m_bulk.ForEach(f =>
    {
        XmlElement xmlElementChild = xmlDocument.CreateElement("ValueBulk");
        xmlElementChild.InnerText = f.ToString("#.0", Thread.CurrentThread.CurrentUICulture);
        xmlElementCompress.AppendChild(xmlElementChild);
    });

    if (m_compress)
    {
        byte[] byteArray = Encoding.Unicode.GetBytes(xmlElementCompress.InnerXml);
        MemoryStream stream = new MemoryStream(byteArray);
        MemoryStream outStream = new MemoryStream();
        using (GZipStream compression = new GZipStream(outStream, CompressionMode.Compress, true))
        {
            compression.Write(byteArray, 0, byteArray.Length);
            compression.Close(); // we must close it
            byteArray = new byte[outStream.Length];
            outStream.Position = 0;
            outStream.Read(byteArray, 0, byteArray.Length);
        }
        outStream.Close();
        return Convert.ToBase64String(byteArray);
    }

    return xmlElementCompress.InnerXml;
}

Now load the compressed XML fragment and add it to the document in uncompressed form:

XmlDocument xmlDocument = new XmlDocument();
xmlDocument .Load(fileName);

XmlElement xmlElementChild = xmlDocument.SelectSingleNode("ValueBulks") as XmlElement;

byte[] byteArray = Convert.FromBase64String(xmlElementChild.InnerText);
MemoryStream instream = new MemoryStream();
MemoryStream stream = new MemoryStream(byteArray);
stream.Position = 0;
using (GZipStream decompression = new GZipStream(stream, CompressionMode.Decompress))
{
    decompression.CopyTo(instream);
}
byteArray = new byte[instream.Length];
instream.Position = 0;
instream.Read(byteArray, 0, byteArray.Length);
value = Encoding.Unicode.GetString(byteArray);
xmlElementChild.InnerXml = value;

The following uncompressed XML fragment (... shows for continuity):

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="false">
        <ValueBulk>.0</ValueBulk>
        <ValueBulk>1.0</ValueBulk>
        ...
        ...
        <ValueBulk>98.0</ValueBulk>
        <ValueBulk>99.0</ValueBulk>
    </ValueBulks>
</Settings2>

will be converted to the following compressed fragment:

<Settings2>
    <ValueX>30</ValueX>
    <ValueBulks Compress="true">H4sIAAAAAAAEAOy9B2A...ppD8CFQAA</ValueBulks>
</Settings2>

My upcoming post will be last in this series where I will write about how to encrypt your sensitive data and still comply with the schema.

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.

Defining a file format using XML and XML Schema (XSD) in C#/Java - II

The complete implementation of XML file format is divided into the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

 In the previous post, we have defined an XML schema and an example XML file based on the schema. In today's post, we will read/write/validate the XML file against the schema in C# using Visual Studio.

1. Create a Windows Forms application.
2. Add 3 buttons
• Load
• Load and Validate
• Save

Following is a code snippet for creating an XML document:

string  schemaName = "set.xsd";
XmlDocument document = new XmlDocument();
document.Load(fileName);
XmlElement xmlNode = doc.CreateElement("XmlFileFormat");

XmlAttribute attr = doc.CreateAttribute("xmlns:xsi");
attr.Value = "http://www.w3.org/2001/XMLSchema-instance";
xmlNode.SetAttributeNode(attr);

attr = doc.CreateAttribute("xsi:noNamespaceSchemaLocation");
attr.Value = m_schemaName;
xmlNode.SetAttributeNode(attr);

doc.AppendChild(xmlNode);
doc.Save(fileName);

And code snippet for loading this document:

string  schemaName = "set.xsd";
XmlDocument document = new XmlDocument();  
document.Load(fileName);

The validation is done using XmlSchema class:

string  schemaName = "set.xsd";
string  fileName = "set.xml";

XmlReaderSettings settings = new XmlReaderSettings();
settings.Schemas.Add(string.Empty, schemaName);
settings.ValidationType = ValidationType.Schema;

XmlReader reader = XmlReader.Create(fileName, settings);
XmlDocument document = new XmlDocument();
document.Load(reader);

ValidationEventHandler eventHandler = new ValidationEventHandler(OnValidation);
// the following call to Validate succeeds.
document.Validate(eventHandler);
reader.Close();

If anything is changed in the document we can always call document.Validate() with the OnValidation event handler: 

private void OnValidation(object sender, ValidationEventArgs e)
{
  switch (e.Severity)
  {
  case XmlSeverityType.Error:
    Console.WriteLine(e.Message);
    break;
  case XmlSeverityType.Warning:
    Console.WriteLine(e.Message);
    break;
  }
}

Very frequently we come to problems in XML data when we need to represent large data sets. Due to verbose tags, we need several byte even to write a single data byte. This causes an extreme overhead during serialization and data transfer over slower networks.. There were number of efforts seen in the open source and research communities to overcome this problem. In the next post, we will implement one mechanism in C# to compress large datasets.

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.

Defining a file format using XML and XML Schema (XSD) in C#/Java - I

Occasionally I need a custom file type for new application. I prefer to use OLE/COM Structured Storage since it offers me a compact file-system like hierarchy with Storages and Streams to save application data in binary format. My open source application "UVFSEditor" can view a compound file along with other file formats. The storages are shown in a tree like structure at the left and the streams are displayed in a HexEditor.

For quite a long time Microsoft is not doing anything around OLE/COM structured storage and the company itself is moving away from this file format for its own applications in favor of some XML formats. I have also been looking for an alternative for new applications. XML is a viable alternative since the information can be viewed and edited with simple text editor, validated using XML schema validator, the data can easily be exported to other file formats.

I divided the complete work in the following steps:

1. XML Schema (XSD) definition
• Let an extension for my file type: .xef (XML example file-type)
• Generate an XML file with data compatible to previously defined schema
2. Validate my XML with the schema
3. Compress huge data fragments
4. Encrypt sensitive data
5. Implementation in Java (Compliance with XSD 1.1)

Schema definition
A file type has a definite structure which helps the application to save and load data in a systematic way. An XML schema helps to define the type and size of data along with their relationship with each other which creates a predefined structure of a generated file. XML schema serves the same purpose as a relational schema serves for a database. The World Wide Web Consortium (W3C) has a recommendation for defining XML schema called XML Schema Definition Language (XSD). The most recent recommendation of XSD 1.1 is in candidate status. The language itself is in XML.

Let's start with my sample file-type schema:

<?xml version=“1.0“?>

<xsd:schema xmlns:xsd=“http://www.w3.org/2001/XMLSchema“>
 <xsd:element name=“Document“>
    <xsd:complexType>
     <xsd:sequence>
       <xsd:element name=“Value1“ type=“xsd:integer“></xsd:element>
       <xsd:element name=“Value2“ type=“xsd:string“></xsd:element>
       <xsd:element name=“Value3“ type=“xsd:float“></xsd:element>
       <xsd:element name=“Settings2“>
         <xsd:complexType>
           <xsd:sequence minOccurs=“0“ maxOccurs=“1“>
             <xsd:element name=“ValueX“ type=“xsd:integer“>
             </xsd:element>
             <xsd:element name=“ValueBulks“>
               <xsd:complexType>
                 <xsd:sequence>
                   <xsd:element name=“ValueBulk“ type=“xsd:float“ minOccurs=“0“ maxOccurs=“unbounded“/>
                 </xsd:sequence>
                 <xsd:attribute name=“Compress“ use=“optional“ type=“xsd:boolean“></xsd:attribute>
               </xsd:complexType>
             </xsd:element>
           </xsd:sequence>
         </xsd:complexType>
       </xsd:element>
     </xsd:sequence>
    </xsd:complexType>
 </xsd:element>
</xsd:schema>

Extension for New File Format
XML is a text format, so is XSD. A file in Windows OS almost always has an extension which provides information to the Windows Shell which application should be started once a file of particular type is, for example, double clicked in Windows explorer. For my test application, I take chosen .xef (XML example file-type) as file extension

Sample XML file definition

 10
 Test File Format
 101
 
    30
    
     1.0
     2.0
    
 

I prefer Visual Studio IDE and C# whenever I need to try some demo application which has a user interface. The next post of this blog will include how to read & write an XML file and validate it against our predefined schema using .Net framework classes.

The complete code can be downloaded from: http://keensocial.freeiz.com/blogs/xmlfileformat/xmlfileformat.zip. The project file can only be opened in Visual Studio 2010. If you have other versions of Visual Studio, you have to create a new solution and add the extracted files from the zip file to the solution.

Parabox

Last weekend was lonely and I felt nostalgic for the paranoid game that we used to play during university days. We were all mad for it and spent hours playing it. For few days I was experimenting with HTML5 and CSS3 capabilities in various browsers. I took the opportunities and programmed our beloved paranoid and integrated it as facebook application. It needs chrome, firefox, IE9, safari and opera browsers to run.
The facebook apps is at:
                                      Parabox Game
You can play it here:

Journey to an Unknown Land

Previous 1 2 

You have a feel of safety everywhere. Car drivers don't pass on red signal, pedestrians don't cross the roads here and there. You don't need to be afraid of some pick pocketers or hijackers. Streets are free of dirt. You are relieved from chaos and rush. Until you are in a supermarket, you will see calmness all around.

Life is charming and beautiful. It is cold outside but with room heaters on you don't feel it from inside. You don't need to care for seniors when doing anything. The people are friendly and warm. If you ask for help, most stand by you eagerly. Unlike many friends during my University life, they will not point you to the wrong direction when you ask them for a street or house number.

My company was bankrupt before arriving here. I got a great feel of job safety when a new job was confirmed just the next day after entering into this land. There was a round of walking through the company compound to introduce myself to other colleagues on the first day. Everyone seems very much friendly. The office was located in a nice place nearby to the city. The forest was just beside the office building and the city on the other.

Everything was going perfect. Life is going to settle down in an unknown land without any hassle. It seems I was born in the wrong place and now migrated to my home.

Previous 1 2