Deep copy & Shallow copy - Why basics matter

Recently, we ran into an issue in our application, a weird bug that was not easily reproducible. It goes like this:

When I try to look at data for User X, the application sometimes pulls up the details of X and sometimes it gets me the data for some other random User Y. There was no relation between X and Y. This was happening more under load and it was pretty difficult to reproduce.

After digging through the freaking code for hours and hours we finally figured out the problem.It happened for two reasons:

First Reason:
The programmer who wrote the code was careless or lacked a clear understanding of programming fundamentals.

Second Reason:
A shallow copy of the data was being returned instead of a deep copy.

Explanation:
We were using the memory cache to cache the common user metadata from the database so as to cut down the load on the database servers. We then created a class (a Singleton) to encapsulate the access to the cache. This class has an instance of a cache object that talks to the actual memory. The getters in this instance returned shallow copies of data i.e actual references to memory. It was the responsibility of the Singleton class to create a deep copy of the data before returning it to the calling code.

This critical piece of code was missing and so the Singleton was essentially returning a reference to the data in cache.The code that called the Singleton was then modifying the shallow copy of the metadata, unaware that it was actually modifying the values in cache. So when User Y requested data, since the class was a singleton(and therefore using the same cache instance as well), it was returning a modified version of the metadata. When this metadata was combined with other data, specific to user Y, it resulted in weird values and as a result the user ended up seeing some other data.

Solution:
The fix was very simple. In the Singleton class, we had to create a deep copy of the object returned from the cache, before passing it to the calling code, so any changes made to that object did not impact the original values in cache.

The State Design Pattern

Sometime ago I was given the task of designing a module for creating and managing work orders for one of our clients. For those of you wondering what a work order is..let me give you an example..

Let's say you live in an apartment and your air conditioning system is not working. what do you do? You immediately call up the apartment office to report it. So what you are doing is initiating a work order process. The office assistant takes your request and creates a work order for repairing the a/c. This work order is then sent to a technician who sends an estimate to the assistant. The assistant or the assistant's supervisor approves the cost estimate and the technician gets to work. The technician completes the job and reports back to the assistant. The assistant then closes the work order(after confirming with you obviously). This is a typical work order workflow process.

I had to develop a user interface to facilitate this entire process. I decided to browse through the GoF design patterns to see if I could find some pattern that would serve the purpose. I obviously ruled out the creational and structural patterns since I was not looking along those lines. So I started looking at the behavioral patterns wherein I came across the State Design Pattern. After carefully studying the pattern I decided that I was going to go with it.

Why the State Design Pattern?
The work order process goes through different stages and the behavior changes at every step. It is the same work order that appears to take on different forms at different stages. We can use a single object and keep altering it's underlying behavior with every step in the work flow process. We can tie the user interface to the underlying behavioral changes so that, as the flow changes, the UI will also change accordingly.

The UML is as follows:

How it works:

• The UI interacts with the State Manager which is the object that changes it's underlying behavior
• The State Manager changes it's underlying behavior through an instance of  the Abstract State class, which happens to be one of its member fields.
• This instance points to a different concrete instance of type State at different stages of the work order process. As this happens, it appears as if the State Manager is changing it's underlying behavior.
• The UI also changes from step to step depending on the properties of the State Manager.

 Code:

public class WorkOrderUI: System.Web.UI.Page
{
   StateManager objStateManager;
   protected void Page_Load(object sender, EventArgs e)
  {
     objStateManager = new StateManager();
     DisplayUI();
   }
  
/*render the UI depending on the state manager's underlying state*/
private void DisplayUI()
{
    if(objStateManager.IsApproved) //do so and so
      btnApprove.visible = false;
}

  /*save handler, calls the statemanager that will change it    behavior as a result of this save*/
   protected void Save(object sender, EventArgs e)
  { 
     objStateManager.SaveState();
   }
}

public class StateManager
{
   State objState;
   public StateManager()
  {    
     objState = GetState();
  }

 private State GetState()
{
  /*depending on various properties this method returns the current state*/
   return new WorkOrderState();//example
}

public void SaveState()
{
   //saves state
    objState.SaveData();
}

}

public abstract class State
{
   protected abstract State GetData();
   protected abstract void SaveData();
}

public class WorkOrderRequest:State
{
   protected State GetData()
  {
     //implement functionality specific to this state
   }

  protected void SaveData()
  {
     //implement the save specific to this state
  }
}

other state classes are implemented in a similar manner.

Pros and Cons of using XML data type in SQL Server 2005

Recently I developed a web-based tool using ASP.NET and C#, that enables users to create custom forms in a matter of minutes.Every form can have any number of controls like Textboxes, Radiobuttons,Checkboxes,Labels etc.Since the form size can vary from say 5 fields to 20 fields to 50 fields,there was no way I could create a flat table structure to store this data. So I decided to use the XML data type in SQL Server 2005 to store the form fields configuration.

The rendering part of the tool parses this xml configuration(using LINQ to XML) and renders the form.Let us assume that this form is going to be used for survey purposes. The survey users can fill in the data and complete the survey. All this works just fine and the survey is successfully completed.

Then comes the reporting(SSRS). The management wants to see the results grouped and sorted in a multitude of ways. SQL Server does not have too much of an API for querying XML columns. So I had to use whatever was available(the query,value,exist,modify and nodes methods) along with some SQL cursors to come up with the desired results in the desired format.

The problem with this approach was that this parsing(which was happening at runtime) took time and the time taken was directly proportional to the number of survey instances, so as the data grew, the reporting became slower and slower. So while XML was very convenient way to store variable length data, it was not very easy to work with, in terms of reporting or analysis purposes.

I managed to solve the issue with reporting by creating a flat table structure for storing the survey data(since the configuration is constant once the form is created).I then created a SQL agent that runs as a scheduled task during the non-peak hours and performs the time consuming task of parsing instance xml data and populating the flat table. I then pointed the reports to the flat table. This speeded up the reporting process.

So while XML data type is suitable for storing variable length data, the developers need to note that it is not very easy to work with the XML, with the limited API in SQL Server. One way to solve the problem would be to get the data out of SQL and use the powerful .NET API to get the desired result. The other way would be to create SQL agent(as described above) to pre-populate data into flat tables and then work with those tables. But all said and done, the XML data type is a very powerful feature and provides tremendous flexibility. I would definetely recommend it.