Process Monitor a.k.a Procmon

Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity. You can get more information here

A couple of months ago, I ran into a very strange issue with some data update functionality involving two separate but related applications, one a WPF client and the other a console application running as a scheduled task.

The data update happened in two ways. In the first case, the user explicitly requested for an update using one of the context menu options inside the WPF Client. In the second case, the data update request was automatically queued up by the same Client when certain state changes happened. These queued requests were then processed by the scheduled console application at a later time.The code to perform the actual data update was in a separate dll and both these applications were referencing the same dll.

What was strange was that the manual request was updating the data successfully whereas the queued up request was failing to do so.

I stepped through the code but I was not able to find anything different. so I decided to use the process monitor to troubleshoot this issue. I ran procmon and monitored the manual data update request. Procmon shows you all the activities like registry access and files being loaded. I didn't notice any assembly loading failures or any other access failures.

I then repeated the same process with the console process. In this case, I noticed that there was an assembly loading failure. The console process was trying to load a particular assembly and it wasn't able to locate it after trying several locations. Procmon shows you all the paths that the process is looking at. The assembly that the process was trying to load was missing and that was the reason for the data update failure. Once we dropped the missing dll into a specific location, the process was able to load it and the data process worked just fine.

We later found out that the install team had forgotten to include that particular dll while building their installers. Once the installer got fixed, the data update issue went away.

Using the Thread Pool in .NET

The Microsoft.Net framework provides a Thread Pool, which is a pool of threads available for applications to use. There is one Thread Pool per CLR and all the Application Domains managed by the CLR will have access to the threads in that Thread Pool.

The applications do not need to go through the hassle of creating and managing the threads as all that is taken care of by the Thread Pool. Using the threads in the Thread Pool is more efficient in terms of resources since the threads can be reused. Using the Thread Pool threads will also lead to an improvement in performance since the creation of threads is a time consuming operation that could negatively impact performance.

One way for applications to use the threads in the Thread Pool is to call the ThreadPool.QueueUserWorkItem method in the System.Threading namespace. This method takes a method and related parameters as input and adds an entry into the Thread Pool's global queue.

//we are adding the method ProcessCreditCard to the ThreadPool queue
System.Threading.ThreadPool.QueueUserWorkItem(ProcessCreditCard, 100);

//this method will be executed by a thread pool thread
private void ProcessCreditCard(object input)

{

   //do something here

}

When a thread becomes available in the Thread Pool, it will take this entry from the queue and execute it. The drawback of using this method is that there is no way to interact with the Thread Pool thread that is executing your task. We do not know if the method completed successfully or if there were any exceptions. We also do not have the ability to get a return value from the method. To fix these limitaions Microsoft introduced the concept of Tasks.

Task is a class in the System.Threading.Tasks namespace. It provides an alternative for scheduling tasks for execution by the Thread Pool.

//create a task that will schedule a delegate(ProcessAmount) that returns a bool
 Task t1 = new Task(ProcessAmount);

//this line queues up the delegate for execution by a Thread Pool thread
  t1.Start(); 

//we are waiting for the execution to complete
  t1.Wait(); 

//we are printing the result from the execution
  Console.WriteLine(t1.Result); 


//this is the delegate that will be executed by a thread pool thread
 private bool ProcessAmount()

 {

        return true;

 }

As you can see, we can schedule a task that returns a bool, wait for it to complete and then get the result back.

We also have the ability to cancel the task if necessary.

 CancellationTokenSource source = new CancellationTokenSource();

//this line queues up the delegate for execution by a Thread Pool thread
Task t1 = new Task(()=>ProcessAmount(source.Token));

 t1.Start(); 
source.Cancel();

 private static bool ProcessAmount(CancellationToken token)
  {

    //some periodic operation

   //check if the task has been cancelled, 
   //if yes, this method will throw an OperationCanceledException
      token.ThrowIfCancellationRequested();
      return true;

   }

The calling code can capture the exception thrown by the Task on cancellation and do whatever is necessary.

The System.Threading.Tasks namespace also contains a static class called Parallel that provides two methods For and ForEach.

Both of them internally use Task objects to execute the code using the Thread Pool threads

//regular For

for(int i=0;i<100 i="" p="">//do something


//Parallel's For

Parallel.For(0,100, SomeDelegate);

When we call the For method of the Parallel class, the thread pool thread's perform this task in parallel resulting in improved performance.It is ideal for situations where you want to execute a large number of tasks in parallel OR if there are long-running tasks. Few operations or short operations are not good for executing in parallel. This is also not an ideal option if the processing needs to happen sequentially since the Thread Pool threads execute in no particular order. This option is also not ideal if the operations share some data since this would require some kind of synchronization that will negatively impact the performance.

Parallel.ForEach and Parallel.Invoke also work similarly.

Parallel LINQ

When dealing with collections via LINQ, we can use Tasks(a.k.a the Thread Pool threads) to execute the operations on the collections in parallel in order to improve performance. This is possible via the AsParallel method of the System.Linq.ParallelEnumerable class. This method can be used to convert sequential queries based on IEnumerable or IEnumerable to parallel queries as follows:

var filteredList = GetItems.AsParallel().where(//do some filtering here);


private IEnumerable GetItems()
{

   //return a list

}

As is the case with Parallel.For and Paralle.ForEach and Parallel.Invoke, Parallel LINQ is ideal for situations where you want to execute a large number of tasks in parallel OR if there are long-running tasks. Few operations or short operations are not good for executing in parallel. This is also not an ideal option if the processing needs to happen sequentially since the Thread Pool threads execute in no particular order. This option is also not ideal if the operations share some data since this would require some kind of synchronization that will negatively impact the performance.

(Reference: CLR via C# by Jeffrey Richter)

Traits in Scala

Scala is a popular programming language that combines the best features of object-oriented and functional programming languages. It has some really cool features that are not available in other languages like C# or Java. Traits is one such feature that provides a viable solution for the problems associated with multiple inheritance in other languages.

Multiple inheritance is not allowed in Java/C# and the only other way to build common functionality among a set of related classes is via interfaces. But interfaces are abstract and require concrete implementation. The other option is to create a class that implements the desired interfaces and then inherit from that class. This is not very convenient.

Following are some of the cool features of a trait:

• A Trait is much like an abstract class in C#. It can contain abstract members as well as concrete implementations
• A Trait can contain abstract fields as well as concrete fields
• A class in Scala can only have one base class but it can implement any number of Traits. So a lot of concrete implementations are readily available for use. 
• A Trait can extend other Traits.
• A Trait can extend other classes
• An instance of a class can implement a Trait. So, not the class, but the desired instances of that class can implement Traits as needed. 

Following are some code samples of Traits:

An abstract Trait called Runner that has an abstract method called Run

trait Runner
{
   def Run(destination:String) //a string input param called "destination"

}

A class called Athlete that provides the implementation for the Trait:
class Athlete extends Runner
{
  //override keyword is not necessary while implementing an abstract method of a trait
  def Run(destination:string) {println(destination)}
}

A concrete Trait called Runner that has a concrete method called Run
trait Runner
{
  def Run(destination:String)  //a string input param called "destination"
  {
    println(destination)

   }
}

A class called Athlete that implements the Trait and calls the Run method. In Scala slang "The Runner functionality is mixed in with the Athlete class"

class Athlete extends Runner
{
   def PrintDestination(destination:String)
{
     Run(destination);
   }
}

A trait called MarathonRunner that extends the trait Runner

trait MarathonRunner extends Runner
{
  def PrintMarathonDestination(destination:String)
  {
   Run(destination)
   }
}

A class called Athlete that implements both Runner and MarathonRunner

class Athlete extends Runner with MarathonRunner
{
}



NOTE : In Scala, the first trait is implemented using the extends keyword and the second trait is implemented using the with keyword

Let us say that there is a trait called HalfMarathonRunner, an instance of the class Athlete can implement this trait:
val objAthlete = new Athlete with HalfMarathonRunner

A trait with an abstract field
trait Runner
{
 val MaxDistance:int //no initial value means abstract
}

a trait with a concrete field
trait Runner
{
  val MaxDistance = 15
}

Randomizing Display Ads

I was working on the home page of a website that had a small area on the top right corner that was set aside for advertisements. The real-estate had to be shared by four different Ads provided they satisfied the criteria for display. Each had a different criteria that determined their eligibility for display. Criteria included the logged-in user's age, postal code and gender.

So it was possible that at any given time there could be one, two, three or four Ads eligible for display.Then the Ads that were eligible for display had to be randomized to ensure that each had an equal likelihood of appearing on the page.

The model for the page was as follows:

 public class MyHomeModel
  {
   public bool ShowNexusAd {get;set;}
   public bool ShowIpadAd {get;set;}
   public bool ShowSurfaceAd {get;set;}
   public bool ShowKindleAd {get;set;}
  }

So when we returned the model to the view, each of the properties had to have a value of true or false, that would be used in the view to determine whether to hide or show the particular Ad as follows:

@if(model.ShowNexusAd)
{
  //show Nexus Ad
}

For randomization, I decide to use the Random class in .NET to generate a random number in a specified range (1-9) and then use the value generated to determine which ad to show. The code would like this:

if(ShowNexusAd && ShowIpadAd && ShowSurfaceAd && ShowKindleAd)
{
   //generate a number between 1 and 9(including 1)
   Random randomNumber = new Random(1,9) 
   if(randomNumber <=2)
      ShowNexusAd = true;
   if(randomNumber >2 && randomNumber <=4)
      ShowIpadAd = true;
   if(randomNumber >4 && randomNumber <=6)
      ShowSurfaceAd = true;
   if(randomNumber >6 && randomNumber <=8)
      ShowKindleAd = true;
}

But there is a problem here.......

what if only three of the ads were eligible for display or two or just one? we would end up with code as follows:

if(ShowNexusAd && ShowIpadAd && ShowSurfaceAd && ShowKindleAd)

{
 Random randomNumber = new Random(1,9)
 if(randomNumber <=2) 
  ShowNexusAd = true;
 if(randomNumber >2 && randomNumber <=4) 
  ShowIpadAd = true;
 if(randomNumber >4 && randomNumber <=6) 
  ShowSurfaceAd = true;
 if(randomNumber >6 && randomNumber <=8) 
  ShowKindleAd = true;
}else
if(ShowNexusAd && ShowIpadAd && ShowSurfaceAd )
{
 Random randomNumber = new Random(1,7)
 if(randomNumber <=2) 
  ShowNexusAd = true;
 if(randomNumber >2 && randomNumber <=4) 
  ShowIpadAd = true;
 if(randomNumber >4 && randomNumber <=6) 
  ShowSurfaceAd = true;
}else
if(ShowNexusAd && ShowIpadAd && ShowKindleAd)

{
 Random randomNumber = new Random(1,7)
 if(randomNumber <=2) 
  ShowNexusAd = true;
 if(randomNumber >2 && randomNumber <=4) 
  ShowIpadAd = true;
 if(randomNumber >4 && randomNumber <=6)
  ShowKindleAd= true;
}

.......................and so on with all the possible permutations and combinations

As the number of Ads increase(or even for the existing four Ads) it will become a nightmare to extend this logic and even maintain it. So how do we make the code smart enough to handle this?

I hit upon the following idea(which might not be the greatest idea, but is pretty decent)

I created the following classes:

public abstract class AdDisplayBase
{
 public MyHomeModel ViewModel { get; set; }
 public abstract bool ShowAd { set; }
}

public class NexusAdDisplay : AdDisplayBase
{
 public override bool ShowAd
 {
  set { ViewModel.ShowNexusAd = value; }
 }
}

public class IpadAdDisplay : AdDisplayBase
{
 public override bool ShowAd
 {
  set { ViewModel.ShowIpadAd = value; }
 }
}

public class SurfaceAdDisplay : AdDisplayBase
{
 public override bool ShowAd
 {
  set { ViewModel.ShowSurfaceAd = value; }
 }
}

public class KindleAdDisplay : AdDisplayBase
{
 public override bool ShowAd
 {
  set { ViewModel.ShowKindleAd = value; }
 }
}

My goal was to convert the simple value types(the boolean properties) into reference types, so that I could treat them as objects. This would enable me to add the objects to a collection and handle them in a generic manner. To achieve this, I created an abstraction around the boolean properties by creating a class for each type of Ad.

I then refactored the unmanageable conditional logic above to the following method:

void ManageAdsDisplay(MyHomeModel model)
{
  int randomNum;
  var lstAdsToDisplay = new List();

  if (model.ShowNexusAd ) 
     lstAdsToDisplay.Add(new NexusAdDisplay () { ViewModel = model });
  if (model.ShowIpadAd ) 
     lstAdsToDisplay.Add(new IpadAdDisplay () { ViewModel = model });
  if (model.ShowSurfaceAd ) 
     lstAdsToDisplay.Add(new SurfaceAdDisplay () { ViewModel = model });
  if (model.ShowKindleAd ) 
     lstAdsToDisplay.Add(new KindleAdDisplay () { ViewModel = model });
  
  switch (lstAdsToDisplay.Count)
  {
    case 1:
 lstAdsToDisplay[0].ShowAd = true;
 break;
    case 2:
 randomNum = new Random().Next(1, 9);
 lstAdsToDisplay[0].ShowAd = (randomNum <= 4);
 lstAdsToDisplay[1].ShowAd = (randomNum > 4);
 break;
   case 3:
 randomNum = new Random().Next(1, 10);
 lstAdsToDisplay[0].ShowAd = (randomNum <= 3);
 lstAdsToDisplay[1].ShowAd = (randomNum > 3 && randomNum <= 6);
 lstAdsToDisplay[2].ShowAd = (randomNum > 6 && randomNum <= 9);
 break;
  case 4:
 randomNum = new Random().Next(1, 9);
 lstAdsToDisplay[0].ShowAd = (randomNum <= 2);
 lstAdsToDisplay[1].ShowAd = (randomNum > 2 && randomNum <= 4);
 lstAdsToDisplay[2].ShowAd = (randomNum > 4 && randomNum <= 6);
 lstAdsToDisplay[3].ShowAd = (randomNum > 6 && randomNum <= 8);
 break;
  }
}

By doing this we avoid the need to write complicated and confusing conditional statements and it is pretty easy to extend this design if there are more Ads in the future. To be honest, I am not completely happy with the Switch/Case part of the solution and I am trying to find a better way to do this. It will definitely end up as another blog post if I do.

Adapter Design Pattern

The Adapter design pattern is one of the most basic structural design patterns and it's definition is as follows:

"Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn't otherwise because of incompatible interfaces."

Last week I was working on a part of a web application that pulled a list of users from the Database. This application has a pretty good amount of traffic and as the traffic increased it started putting a lot of load on the DB and slowing down the application. In an attempt to find a viable solution, we decided to try out an in-memory cache to store the user data and speed up access. About 25% of the traffic would get the data from the Cache and the rest would continue to go to the Database.

The code changes for the cache repository had to fit seamlessly into the existing design and the changes had to be made without touching the existing implementation for DB calls. The Adapter pattern turned out to be a perfect choice for the requirement.

Following is the code with just the call to the Database:

public class DbRepository
{
  public IEnumerable<user> GetUsers(int userId)
  {
    //call DB and get data
  }
}

//code calling the repository
public class UserService
{
  public IEnumerable<user> GetAllFriendsFor(int loggedInUserId)
  {
    //call DB repository to get data
   DbRepository repository = new DbRepository();
   return repository.GetUsers(loggedInUserId);    
  }
}

Following is the code after implementing the Adapter design pattern:

public interface IRepository
{
  IEnumerable<user> GetUsers(int userId);
}

public class CacheRepository:IRepository
{
  public IEnumerable<user> GetUsers(int userId)
  {
    //call cache and get data
  }
}

//the adapter
public class DbRepositoryAdapter:DbRepository,IRepository
{
}

public class RepositoryFactory
{ 
  //a variable that determines if the user is in test
  bool UserInCacheTest {get;set;}
  
  //return the appropriate repository
  public IRepository GetRepository()
  {
    if(UserInCacheTest)
   return new CacheRepository();
 
 return new DbRepositoryAdapter();
}
}

//code calling the repository
public class UserService
{
  public IEnumerable<user> GetAllFriendsFor(int userId)
  {
    //call the factory to get the appropriate repository
   IRepository repository = new RepositoryFactory().GetRepository();
   return repository.GetUsers(userId);    
  }
}

Explanation:

• We create a new interface called IRepository that has a single method. This method will have the same signature as the method in the exisiting DbRepository since we are going to be adapting that repository to fit into the new design. NOTE: Using the same signature as in the existing DbRepository is just a matter of convenience, it is not necessary to do so.
• We then create a new repository called DbRepositoryAdapter that will inherit from the existing DbRepository and implements the IRepository interface.The DbRepository already has a method with the same signature as the method in IRepository, so it is safe to say that the DbRepositoryAdapter implements the IRepository interface. NOTE: If the signature of the interface method was different, we would need to implement that method and call the DbRepository's method from inside it.
• We create a new class called CacheRepository that also implements IRepository. 
• We create a factory class called RepositoryFactory that will return a repository of type IRepository depending on whether the user is in the cache test. 
• The CacheRepository and the DbRepositoryAdapter both implement IRepository, so we can use interface polymorphism within the UserService to return the appropriate repository.
• We have incorporated the existing DbRepository into our new design by creating an adapter class on top of it. This way the existing DbRepository class and the new CacheRepository class can work well together.(NOTE: we have not touched the existing DbRepository in any way.). This is the advantage of using the Adapter design pattern.

Generic Contravariance in C#

In my previous post .NET interfaces Part 3 I had written about the IEqualityComparer interface which has the following signature:

 public interface IEqualityComparer<in T>
 {
        bool Equals(T x, T y);        
        int GetHashCode(T obj);
 }
 

The keyword "in" before the type T indicates that this interface is contravariant.

In the same post we were calling the Contains method on a collection of type FootballStar as shown below:

FootballStars.Contains(Peyton2,new StarComparer())

We were passing in an instance of type IEqualityComparer<Star> to a method that was expecting an IEqualityComparer<FootballStar> i.e we were able to pass a less derived type than was specified by the type parameter.

This was possible because the interface IEqualityComparer<in T> is a contravariant interface that allows us to pass a less derived type than the specified type parameter.

Even though we are passing in a less derived type, everything works perfectly because the instances that ultimately get passed to the Equals method of the IEqualityComparer method are still of type FootballStar(since the collection is of type FootballStar), which derives from Star.

If the interface was not Contravariant, we would have to create a new IEqualityComparer of type FootballStar and pass that to the contains method. The compiler will not have it any other way. We would also have to repeat this for every type that derived from Star, assuming that all instances of type Star(e.g. FootballStar, BaseballStar) wish to use the same logic to determine the equality of their instances.

We would end up writing code like this:

//a comparer for FootballStar instances
public class FootballStarComparer:IEqualityComparer<FootballStar>
{
}

//that will be passed to contains
FootballStars.Contains(Peyton2,new FootballStarComparer())



//a comparer for BaseballStar instances
public class BaseballStarComparer:IEqualityComparer<BaseballStar>
{
}

//that will be passed to contains
BaseballStars.Contains(bbStar1,new BaseballStarComparer())

So instead of using the type inheritance heirarchy and polymorphism we would be writing a lot of repititive code to perform similar tasks.

.NET interfaces Part 3

IEqualityComparer<T>

 public interface IEqualityComparer<in T>
 {
        bool Equals(T x, T y);        
        int GetHashCode(T obj);
 }
 

When to use it:
Let us say you have a custom Type and you plan to use instances of that type in a collection. If you want to use methods like List<T>.contains or Dictionary<T1,T2>.Add, you will need to way to check if an item in your collection equals the item we are trying to find.To facilitate this you can implement the IEqualityComparer<T>.

How is it different from IEquatable<T>?
For every custom type that you wish to check for Equality, you will need to implement the IEquatable<T> interface. But if you have a set of related base/child classes that share a common Equality check functionality, you need to create just one IEqualityComparer<T> where T is the Base type and use it with all other types.

public abstract class Star
    {
        public int Age { get; set; }
        public int GamesPlayed { get; set; }
        public int PointsScored { get; set; }
    }

    public class FootballStar : Star
    {
       
    }

    public class BaseballStar : Star
    {

    }

    //This comparer can be used by all instances of type Star 
    public class StarComparer: IEqualityComparer<Star>
    {
        //strongly typed input parameter
        public bool Equals(Star input1, Star input2)
        {
            if (input1.GamesPlayed == input2.GamesPlayed && 
                  input1.PointsScored == input2.PointsScored) 
               return true;
            return false;
        }
        
        public int GetHashCode(Star input)
        {
            //the XOR value
            return input.GamesPlayed ^ input.PointsScored;
        }
    }

    //calling code
    class Program
    {
        static void Main(string[] args)
        {
     FootballStar Brady = new FootballStar() 
                   { Age = 35, GamesPlayed = 900, PointsScored = 55000 };
            FootballStar Peyton = new FootballStar() 
                   { Age = 35, GamesPlayed = 500, PointsScored = 30000 };
            FootballStar Rodgers = new FootballStar() 
                   { Age = 28, GamesPlayed = 500, PointsScored = 31000 };
            FootballStar Griffin = new FootballStar() 
                   { Age = 28, GamesPlayed = 400, PointsScored = 20000 };

            List FootballStars = new List() 
                   {Brady, Peyton, Rodgers, Griffin};
            
            FootballStar Peyton2 = new FootballStar() 
                  { Age = 35, GamesPlayed = 500, PointsScored = 30000 };

            /* the Equals method of the comparer will be used to 
                  determine the equality
               */
    Console.WriteLine(FootballStars.Contains(Peyton2,new StarComparer()));

            /* Add internally calls GetHashCode of the StarComparer 
                  and will not let you add two keys with the same HashCode
               */
         Dictionary StarDictionary = new Dictionary(3,new StarComparer());
            StarDictionary.Add(Brady, "Patriots");
            StarDictionary.Add(Peyton, "Denver");
            StarDictionary.Add(Rodgers, "Greenbay");
            StarDictionary.Add(Griffin, "Washington");

            //Contains and ContainsKey will both internally call 
            //comparer's Equals and GetHashCode methods
            Console.WriteLine(StarDictionary.ContainsKey(Peyton2));
   Console.WriteLine(StarDictionary.Contains(new KeyValuePair(Peyton2, "Denver")));
 }
}

NOTE:
Since the type T of the comparer is of type Star, any types that derive from it can use the same equality comparer as long as they wish to use the same criteria for determining equality. e.g. if the instances of the type BaseballStar were used in a collection, we can use the same EqualityComparer to test for Equality since BaseballStar inherits from Star