Verifying dynamic methods

While tinkering with reflection emit, the biggest problem is that there is no way to verify a dynamic method easily. After getting frustrated with VerificationException for the millionth time I did some searching and all I found was to write the IL in an .il, compile it with ilasm.exe and then verify it with peverify. Another route was to create a complete dynamic assembly and save it, then verify that dll with peverify.

Needless to say, I didn't want to do that each time manually. I started looking for a wrapper for PEVerify and found AssemblyVerifier. Because PEVerify is written in unmanaged code, the only way was to parse the output from the command line and throw custom verification exceptions, not ideal but it works most of the time.

I had to alter the code a bit because it couldn't locate peverify (so I embedded the version from .NET 4.0.

Right, so now I could have an assembly and verify it in code, dynamic methods can't be saved to an assembly (as far as I know), so in order be able to verify that method I had to save the same method with an MethodBuilder to a dynamic assembly.

This meant defining the same method (return type, parameters, attributes,..) using an MethodBuilder and saving the emitted IL using the dynamicMethod.GetILGenerator() to an methodBuilder.GetILGenerator(). Sadly I couldn't find a way to just bulk copy the entire emitted part so I had to write a wrapper.

This wrapper wraps an ILGenerator (to present the same methods) and takes both the methodbuilder and dynamic method in its constructor. All methods that are in the ILGenerator are then delegated to both the methodbuilder and dynamic method. I used a dictionary to map the original labels and localbuilders that I return (in methods such as DefineLabel) to their methodbuilder equivalents.

This is probably more clear with this snippet:

// Map labels from dmGen to mbGen
private Dictionary<Label, Label> labelMapping = new Dictionary<Label, Label>();

/// <summary>
/// Defines a label on the dynamic method
/// </summary>
public Label DefineLabel()
{
  Label lblDm = dmGen.DefineLabel();
  Label lblMb = mbGen.DefineLabel();
  labelMapping.Add(lblDm, lblMb);
  return lblDm;
}


public void Emit(OpCode opcode, Label label)
{
  mbGen.Emit(opcode, labelMapping[label]);
  dmGen.Emit(opcode, label);
}

This allows me to write the exact same IL for a dynamic method as for a methodbuilder.

One annoyance is the private constructor of an ILGenerator. Because of that I couldn't inherit from an ILGenerator to create a true adapter pattern construct (and which also disallows you to generalize the wrapper in existing code to an ILGenerator).

The rest was pretty easy to get an Verifiable IL Generator from a dynamic method:
- Build dynamic assembly
- Build module
- Build dynamic type
- Build method for dynamic type using methodbuilder
- Create the wrapper with the newly constructed methodbuilder and dynamic method (and assembly and such needed to verify it)
- Return the wrapper which has the same methods as a normal IL Generator.

And after all the generator.Emit shenanigans is done I can call the generator.Verify() method which passes the assembly builder to the AssemblyVerifier.Verify method.

To make it even more useful, this method that encapsulated all that can be defined as an extension method on a DynamicMethod itself, which allows you to do the following:

// define dynamic method and stuff
// ...

VerifiableILGenerator generator = dymMethod.GetILVerifiableGenerator();

// .. emitting ho!

generator.Emit(OpCodes.Newobj, cInfo);
generator.Emit(OpCodes.Ret);

// ...

// verify the emitted IL
generator.Verify();

// create delegate from dynamic method

Can't get much easier in usage than that.

There's only 1 slight problem with this approach: dynamic methods allow you to skip JIT visibility checks (using skipRestrictedVisibility=true, so you can access private members). However methodbuilder doesn't have a similar parameter and PEVerify will curse you to death with 'Field not visible' if you try to verify the IL. I did searching on how to allow this (or at least skip the checks in PEVerify), but couldn't really find an answer. Something with ReflectionPermissionFlags.MemberAccess came up but either I used it wrong or it doesn't help. If anyone has some insight in this, please let me know.

I've uploaded the altered source code for the AssemblyVerifier here for the moment. I've also contacted the original author so he could add it (if he wants to do so). Aside from that extra class I had to modify some stuff here and there to allow verification of dynamic assemblies. I also fixed a bug where the path of the assembly was constructed with new Uri(.. CodeBase) rather than EscapedCodeBase (which crapped out on my 'C# Projects' folder name.

However, if the above code ever dissappears or the author disallows my spreading an altered version, this is the full code for the extra class that allows verifying dynamic methods:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Reflection.Emit;
using System.Reflection;
using System.Threading;
using System.Security.Permissions;

namespace AssemblyVerifier
{
 public static class DynamicMethodVerification
 {

  private static AssemblyBuilder asmBuilder = null;
  private static ModuleBuilder modBuilder = null;

  /// <summary>
  /// Returns an IL generator for the dynamic method which allows the emitted IL to be verified
  /// </summary>
  /// <param name="dm">The dynamic method to write the IL for</param>
  /// <returns>IL generator for the dynamic method</returns>
  public static VerifiableILGenerator GetILVerifiableGenerator(this DynamicMethod dm)
  {
   if (asmBuilder == null)
   {
    AssemblyName assemblyName = new AssemblyName();
    assemblyName.Name = "TestAssembly";
    AppDomain thisDomain = Thread.GetDomain();
    asmBuilder = thisDomain.DefineDynamicAssembly(assemblyName,
        AssemblyBuilderAccess.RunAndSave);

    ReflectionPermission p = new ReflectionPermission( ReflectionPermissionFlag.MemberAccess);
    
    var caBuilder = new CustomAttributeBuilder(typeof(ReflectionPermission).GetConstructor(new Type[] { typeof(ReflectionPermissionFlag) }),
               new object[] { ReflectionPermissionFlag.MemberAccess | ReflectionPermissionFlag.AllFlags });
    asmBuilder.SetCustomAttribute(caBuilder);

    modBuilder = asmBuilder.DefineDynamicModule(
        asmBuilder.GetName().Name, "testdll.dll", false);
   }

   TypeBuilder tb = CreateType(modBuilder, "TestClass");


   MethodBuilder mb = tb.DefineMethod("TestMethod", 
    MethodAttributes.Static | MethodAttributes.Public, 
    dm.ReturnType, 
    dm.GetParameters().Select(p => p.ParameterType).ToArray());   
   VerifiableILGenerator vilGen = new VerifiableILGenerator(asmBuilder, tb, mb, dm);

   return vilGen;
  }

  private static TypeBuilder CreateType(ModuleBuilder modBuilder, string typeName)
  {
   TypeBuilder typeBuilder = modBuilder.DefineType(typeName,
      TypeAttributes.Public |
      TypeAttributes.Class |
      TypeAttributes.AutoClass |
      TypeAttributes.AnsiClass |
      TypeAttributes.BeforeFieldInit |
      TypeAttributes.AutoLayout,
      typeof(object));

   return typeBuilder;
  }


 }

 /// <summary>
 /// Wrapper for IL generator. Delegates the methods to the generator for the dynamic method and
 /// also to a method that will be built into an assembly, which in turn will be validated by PEVerify
 /// </summary>
 public class VerifiableILGenerator
 {
  private TypeBuilder tb;
  private AssemblyBuilder asmBuilder;

  private MethodBuilder mb;
  private DynamicMethod dm;

  private ILGenerator mbGen;
  private ILGenerator dmGen;

  internal VerifiableILGenerator(AssemblyBuilder asmBuilder, TypeBuilder tb, MethodBuilder mb, DynamicMethod dm)
  {
   this.mb = mb;
   this.dm = dm;

   this.tb = tb;
   this.asmBuilder = asmBuilder;

   mbGen = mb.GetILGenerator();
   dmGen = dm.GetILGenerator();
  }

  public void BeginCatchBlock(Type exceptionType)
  {
   mbGen.BeginCatchBlock(exceptionType);
   dmGen.BeginCatchBlock(exceptionType);
  }

  // Map labels from dmGen to mbGen
  private Dictionary<Label, Label> labelMapping = new Dictionary<Label, Label>();

  // Map localbuilder from dmGen to mbGen
  private Dictionary<LocalBuilder, LocalBuilder> localBuilderMapping = new Dictionary<LocalBuilder, LocalBuilder>();

  public void BeginExceptFilterBlock()
  {
   mbGen.BeginExceptFilterBlock();
   dmGen.BeginExceptFilterBlock();
  }

  public Label BeginExceptionBlock()
  {
   Label lblDm = dmGen.BeginExceptionBlock();
   Label lblMb = mbGen.BeginExceptionBlock();
   labelMapping.Add(lblDm, lblMb);
   return lblDm;
  }

  public void BeginFaultBlock()
  {
   mbGen.BeginFaultBlock();
   dmGen.BeginFaultBlock();
  }

  public void BeginFinallyBlock()
  {
   mbGen.BeginFinallyBlock();
   dmGen.BeginFinallyBlock();
  }

  public void BeginScope()
  {
   mbGen.BeginScope();
   dmGen.BeginScope();
  }

  /// <summary>
  /// Defines a label on the dynamic method
  /// </summary>
  /// <returns></returns>
  public Label DefineLabel()
  {
   Label lblDm = dmGen.DefineLabel();
   Label lblMb = mbGen.DefineLabel();
   labelMapping.Add(lblDm, lblMb);
   return lblDm;
  }

  public LocalBuilder DeclareLocal(Type localType)
  {
   LocalBuilder lbDm = dmGen.DeclareLocal(localType);
   LocalBuilder lbMb = mbGen.DeclareLocal(localType);
   localBuilderMapping.Add(lbDm, lbMb);
   return lbDm;
  }

  public LocalBuilder DeclareLocal(Type localType, bool pinned)
  {
   LocalBuilder lbDm = dmGen.DeclareLocal(localType, pinned);
   LocalBuilder lbMb = mbGen.DeclareLocal(localType, pinned);
   localBuilderMapping.Add(lbDm, lbMb);
   return lbDm;
  }

  public void Emit(OpCode opcode)
  {
   mbGen.Emit(opcode);
   dmGen.Emit(opcode);
  }

  public void Emit(OpCode opcode, byte arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, ConstructorInfo con)
  {
   mbGen.Emit(opcode, con);
   dmGen.Emit(opcode, con);
  }

  public void Emit(OpCode opcode, double arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, FieldInfo field)
  {
   mbGen.Emit(opcode, field);
   dmGen.Emit(opcode, field);
  }

  public void Emit(OpCode opcode, float arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, int arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, Label label)
  {
   mbGen.Emit(opcode, labelMapping[label]);
   dmGen.Emit(opcode, label);
  }

  public void Emit(OpCode opcode, Label[] labels)
  {
   mbGen.Emit(opcode, labels.Select(l => labelMapping[l]).ToArray());
   dmGen.Emit(opcode, labels);
  }

  public void Emit(OpCode opcode, LocalBuilder local)
  {
   mbGen.Emit(opcode, localBuilderMapping[local]);
   dmGen.Emit(opcode, local);
  }

  public void Emit(OpCode opcode, long arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, MethodInfo meth)
  {
   mbGen.Emit(opcode, meth);
   dmGen.Emit(opcode, meth);
  }

  public void Emit(OpCode opcode, short arg)
  {
   mbGen.Emit(opcode, arg);
   dmGen.Emit(opcode, arg);
  }

  public void Emit(OpCode opcode, SignatureHelper signature)
  {
   mbGen.Emit(opcode, signature);
   dmGen.Emit(opcode, signature);
  }

  public void Emit(OpCode opcode, string str)
  {
   mbGen.Emit(opcode, str);
   dmGen.Emit(opcode, str);
  }

  public void Emit(OpCode opcode, Type cls)
  {
   mbGen.Emit(opcode, cls);
   dmGen.Emit(opcode, cls);
  }

  public void EmitCall(OpCode opcode, MethodInfo methodInfo, Type[] optionalParameterTypes)
  {
   mbGen.EmitCall(opcode, methodInfo, optionalParameterTypes);
   dmGen.EmitCall(opcode, methodInfo, optionalParameterTypes);
  }

  public void EmitCalli(OpCode opcode, CallingConventions callingConvention, Type returnType, Type[] parameterTypes, Type[] optionalParameterTypes)
  {
   mbGen.EmitCalli(opcode, callingConvention, returnType, parameterTypes, optionalParameterTypes);
   dmGen.EmitCalli(opcode, callingConvention, returnType, parameterTypes, optionalParameterTypes);
  }

  public void EmitCalli(OpCode opcode, System.Runtime.InteropServices.CallingConvention unmanagedCallConv, Type returnType, Type[] parameterTypes)
  {
   mbGen.EmitCalli(opcode, unmanagedCallConv, returnType, parameterTypes);
   dmGen.EmitCalli(opcode, unmanagedCallConv, returnType, parameterTypes);
  }

  public void EmitWriteLine(FieldInfo fld)
  {
   mbGen.EmitWriteLine(fld);
   dmGen.EmitWriteLine(fld);
  }

  public void EmitWriteLine(LocalBuilder localBuilder)
  {
   mbGen.EmitWriteLine(localBuilderMapping[localBuilder]);
   dmGen.EmitWriteLine(localBuilder);
  }

  public void EmitWriteLine(string value)
  {
   mbGen.EmitWriteLine(value);
   dmGen.EmitWriteLine(value);
  }

  public void EndExceptionBlock()
  {
   mbGen.EndExceptionBlock();
   dmGen.EndExceptionBlock();
  }

  public void EndScope()
  {
   mbGen.EndScope();
   dmGen.EndScope();
  }

  public int ILOffset
  {
   get
   {
    // mbGen.ILOffset should be the same
    return dmGen.ILOffset;
   }
  }

  public void MarkLabel(Label loc)
  {
   mbGen.MarkLabel(labelMapping[loc]);
   dmGen.MarkLabel(loc);
  }

  public void MarkSequencePoint(System.Diagnostics.SymbolStore.ISymbolDocumentWriter document, int startLine, int startColumn, int endLine, int endColumn)
  {
   mbGen.MarkSequencePoint(document, startLine, startColumn, endLine, endColumn);
   dmGen.MarkSequencePoint(document, startLine, startColumn, endLine, endColumn);
  }

  public void ThrowException(Type excType)
  {
   mbGen.ThrowException(excType);
   dmGen.ThrowException(excType);
  }

  public void UsingNamespace(string usingNamespace)
  {
   mbGen.UsingNamespace(usingNamespace);
   dmGen.UsingNamespace(usingNamespace);
  }


  /// <summary>
  /// Verifies the generated IL from the dynamic method
  /// </summary>
  public void Verify()
  {
   Type t = tb.CreateType();

   AssemblyVerifier.AssemblyVerification.Verify(asmBuilder);
  }

 }
}

Now to put it to good use :)

Fast operation on Values of Dictionary in C#

At work I encountered a significant slowdown mapping data objects onto domain objects because it was implemented using reflection. I'm sure if you've worked with reflection before you know that it's not the fastest toolset available. Especially not if it's used to map properties from 1 object onto another, with those objects being in a dictionary about with a count of 10000 ~ 1000000.

Saying it was slow is quite an understatement, it was unbearably slow, with delays up to 30 seconds and that was just the mapping, let alone displaying it in a grid and/or performing some calculations on it.

I thought it could be a lot faster, I've encountered code fragments that cache the PropertyInfo objects, which speeds things up considerably, but performing GetValue() and SetValue on them is still quite slow. I've played with HyperDescriptor before (which uses a custom implementation of TypeDescriptor that emits the IL equivalent of the get & sets), which is a great performance boost, but I thought I could still somehow do better.

The last few days I analyzed how the IL statements work and tried a few things until I got the hang of it.

Perhaps let me first explain what is required to be done:
- Our custom data engine reads from the database and stores the objects in a Dictionary.
- According to our DSDM method, we have several modules with their own domain, GUI and adapter.
- The translation between the module objects and the data engine objects is done using an xml schema that describes the mapping between those 2.

This allows my colleagues to work in their isolated module, without having to worry about other objects related to other modules. Some objects can be shared between modules (e.g. a Country object, where module A uses CountryName, CountryCode and module B uses CountryName and CountryCitizenName).

The bottleneck was, as you can expect, the mapping of module objects with data objects.

The exact mapping that had to be done was: Dictionary <----> Dictionary, which was implemented like this:

   var pCopyDic = new Dictionary<string, PersonCopy>(nrOfObjects);

   foreach (var p in personsDic)
   {
    PersonCopy pCopy = new PersonCopy();
    foreach (var pair in propMappings)
    {
     object value = pair.Key.GetValue(p.Value, null);
     pair.Value.SetValue(pCopy, value, null);
    }
    pCopyDic.Add(p.Key, pCopy);
   }

Which was not very fast (the code is a sample from my speed test), compared to a direct copy it was ungodly slow. The direct code used to compare is:

pCopyDic = new Dictionary<string, PersonCopy>(nrOfObjects);
   foreach (var p in personsDic)
   {
    PersonCopy pCopy = new PersonCopy();
    pCopy.DateOfBirth = p.Value.DateOfBirth;
    pCopy.FirstName = p.Value.FirstName;
    pCopy.Function = p.Value.Function;
    pCopy.GUID = p.Value.GUID;
    pCopy.IsEmployee = p.Value.IsEmployee;
    pCopy.LastName = p.Value.LastName;
    pCopy.ParentGUID = p.Value.ParentGUID;
    pCopy.Rating = p.Value.Rating;
    pCopyDic.Add(p.Key, pCopy);
   }

Running both code fragments on a dictionary with 1 million random generated Person objects, I got the following times on average (using StopWatch):

Map props with reflection, 1000000 persons: 32208ms / 57522777ticks
Direct copy 1000000 persons: 821ms / 1467400ticks

The first thing I tried out is emitting those get & set of properties in IL. It was my first time writing actual IL so it was a lot of trial and error. I started off with code I found here and modified it to work with the backing fields rather than with the properties (iirc accessing fields is faster than calling the get & set methods of a property to set the field, although it seems the JIT optimizes in this greatly on my machine). Thus this became the following code:

  /// <summary>
  /// Map compiler generated properties (e.g. public string LastName { get; set; }) to another object's properties.
  /// </summary>
  /// <typeparam name="From"></typeparam>
  /// <typeparam name="To"></typeparam>
  /// <param name="myObject"></param>
  /// <param name="mapping"></param>
  /// <param name="flags"></param>
  /// <returns></returns>
  public static To MapProperties<From, To>(this From myObject, Dictionary<string, string> mapping, BindingFlags flags = System.Reflection.BindingFlags.Instance  System.Reflection.BindingFlags.Public  System.Reflection.BindingFlags.NonPublic)
  {
   MetaData data;
   if (!store.TryGetValue(typeof(From), out data))
   {
    data = new MetaData();
    store.Add(typeof(From), data);
   }

   if (data.Map == null)
   {
    // Create ILGenerator
    DynamicMethod dymMethod = new DynamicMethod("DoMap", typeof(To), new Type[] { typeof(From) }, true);
    ConstructorInfo cInfo = typeof(To).GetConstructor(new Type[] { });

    ILGenerator generator = dymMethod.GetILGenerator();

    LocalBuilder lbf = generator.DeclareLocal(typeof(To));
    //lbf.SetLocalSymInfo("_temp");

    generator.Emit(OpCodes.Newobj, cInfo);
    generator.Emit(OpCodes.Stloc_0);
    foreach (PropertyInfo prop in typeof(From).GetProperties(flags))
    {
     if ((prop.GetGetMethod(true) ?? prop.GetSetMethod(true)).IsDefined(typeof(CompilerGeneratedAttribute), false))
     {
      FieldInfo fromField = GetFieldInfo<From>(prop.Name);

      if (!mapping.ContainsKey(prop.Name))
       throw new ArgumentException("The mapping dictionary does not contain a mapping for " + prop.Name);

      FieldInfo toField = GetFieldInfo<To>(mapping[prop.Name]);

      if (fromField.FieldType != toField.FieldType)
       throw new InvalidOperationException("Mapping between '" + prop.Name + "' from type '" + typeof(From).Name + "' and '" + mapping[prop.Name] + "' from type '" + typeof(To).Name + "' is invalid. The properties have different underlying types");

      // Load the new object on the eval stack... (currently 1 item on eval stack)
      generator.Emit(OpCodes.Ldloc_0);
      // Load initial object (parameter)    (currently 2 items on eval stack)
      generator.Emit(OpCodes.Ldarg_0);
      // Replace value by field value    (still currently 2 items on eval stack)
      generator.Emit(OpCodes.Ldfld, fromField);
      // Store the value of the top on the eval stack into the object underneath that value on the value stack.
      //  (0 items on eval stack)
      generator.Emit(OpCodes.Stfld, toField);
     }
    }

    // Load new constructed obj on eval stack -> 1 item on stack
    generator.Emit(OpCodes.Ldloc_0);
    // Return constructed object.   --> 0 items on stack
    generator.Emit(OpCodes.Ret);

    data.Map = dymMethod.CreateDelegate(typeof(Func<From, To>));
   }
   return ((Func<From, To>)data.Map)(myObject);
  }

  private static FieldInfo GetFieldInfo<From>(string propName)
  {
   Type t = typeof(From);
   FieldInfo fromField = t.GetField(string.Format("<{0}>k__BackingField", propName), BindingFlags.NonPublic  BindingFlags.Instance  BindingFlags.FlattenHierarchy);

   while (fromField == null && t != null)
   {
    t = t.BaseType;
    fromField = t.GetField(string.Format("<{0}>k__BackingField", propName), BindingFlags.NonPublic  BindingFlags.Instance  BindingFlags.FlattenHierarchy);
   }

   return fromField;
  }

Wrapping this code in a cached delegate and calling it 1 million times in a loop gave me:


Map props with emitted method 1000000 persons: 1037ms / 1852797ticks



Hey not bad, it approaches the direct copy method, at least it's faster than reflection (hint: at first I had +- 400-500ms more, but that's because I forgot to initialize the target dictionary with the initial capacity, which in turn made unnecessary resizes, so keep this in mind if you ever copy 1 collection over to another one).

I could have probably settled for this, it's acceptable, but I thought that calling a delegate a million times would still provide some overhead, so I moved creating the dictionary and adding the new pairs in the IL code. Several hours and much cursing later (at least I learned a lot from it) I got it working, but there was not a lot of gained speed. It was slightly faster but only by 20-30ms so I was somewhat disappointed.

Then I thought of the dictionary structure, the keys stay the same, only the values are mapped onto their respective counterparts. Adding the objects into the new dictionary however calculates the hash code and stores the index to a pair in a bucket (the actual pairs are stored in an array Entry<TKey,TValue>[] entries). These calculations are unnecessary, the entire dictionary structure stays the same, the only thing that is different are the value properties of each Entry object.

Thus, the next thing I did was create a new dictionary and copy over all relevant private fields of the original dictionary, to have the same structure). Next I instantiated the entries[] of the target dictionary with the same size as the original dictionary, and finally instead of Adding each pair I iterated over the original entries[] and made a new Entry object with the target type as value and stored it into the same index in the entries array of the target dictionary, effectively bypassing all hashcode calculations. This is also entirely done in IL.

After I got it working, I tested its speed and got:

Direct on dic. values with emitted method 1000000 persons: 614ms /1097640ticks

That's fast! That's even faster than the direct copy method!

Here's the code:

  /// <summary>
  /// Map compiler generated properties (e.g. public string LastName { get; set; }) to another object's properties.
  /// </summary>
  /// <typeparam name="From"></typeparam>
  /// <typeparam name="To"></typeparam>
  /// <param name="myObject"></param>
  /// <param name="mapping"></param>
  /// <param name="flags"></param>
  /// <returns></returns>
  public static Dictionary<string, To> MapProperties<From, To>(this Dictionary<string, From> fromDic, Dictionary<string, string> mapping, BindingFlags flags = System.Reflection.BindingFlags.Instance | System.Reflection.BindingFlags.Public | System.Reflection.BindingFlags.NonPublic)
  {
   MetaData data;
   if (!store.TryGetValue(typeof(From), out data))
   {
    data = new MetaData();
    store.Add(typeof(From), data);
   }

   if (data.MapDictionary == null)
   {
    // Create ILGenerator
    DynamicMethod dymMethod = new DynamicMethod("DoDictionaryMap", typeof(Dictionary<string, To>), new Type[] { typeof(Dictionary<string, From>) }, true);
    ConstructorInfo newTo = typeof(To).GetConstructor(Type.EmptyTypes);
    ConstructorInfo newDictionaryTo = typeof(Dictionary<string, To>).GetConstructor(new Type[] { typeof(int) });
    FieldInfo fldEntries = typeof(Dictionary<string, From>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance);

    FieldInfo fldEntryKey = fldEntries.FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
    FieldInfo fldEntryValue = fldEntries.FieldType.GetElementType().GetField("value", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);

    ILGenerator generator = dymMethod.GetILGenerator();

    Label loopConditionCheck = generator.DefineLabel();
    Label insideLoop = generator.DefineLabel();


    generator.DeclareLocal(typeof(Dictionary<string, To>)); // toDic , 0
    generator.DeclareLocal(fldEntries.FieldType.GetElementType()); // pair entry<string, from>, 1
    generator.DeclareLocal(typeof(int)); // i, 2
    generator.DeclareLocal(typeof(int)); // count, 3;
    generator.DeclareLocal(typeof(To)); // to, 4;
    generator.DeclareLocal(typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType()); // entry<to> 5;

    // store count 
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Callvirt, typeof(Dictionary<string, From>).GetProperty("Count").GetGetMethod());
    generator.Emit(OpCodes.Stloc_S, 3);

    generator.Emit(OpCodes.Ldloc_S, 3); // load count and pass it as capacity parameter for toDic
    generator.Emit(OpCodes.Newobj, newDictionaryTo); // toDic = new ...
    generator.Emit(OpCodes.Stloc_0);

    // COPY Dictionary fields to toDic
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));

    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("comparer", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("comparer", BindingFlags.NonPublic | BindingFlags.Instance));

    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("count", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("count", BindingFlags.NonPublic | BindingFlags.Instance));

    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("freeCount", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("freeCount", BindingFlags.NonPublic | BindingFlags.Instance));

    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("freeList", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("freeList", BindingFlags.NonPublic | BindingFlags.Instance));

    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, fldEntries);
    generator.Emit(OpCodes.Ldlen);
    generator.Emit(OpCodes.Newarr, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType());
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance));

    // End COPY

    generator.Emit(OpCodes.Ldc_I4_0); // i = 0
    generator.Emit(OpCodes.Stloc_2);

    generator.Emit(OpCodes.Br, loopConditionCheck); // perform loop test
    generator.MarkLabel(insideLoop);

    generator.Emit(OpCodes.Ldarg_0); // load fromDic on stack
    generator.Emit(OpCodes.Ldfld, fldEntries); // load entries field from dic on stack
    generator.Emit(OpCodes.Ldloc_2); // load i
    generator.Emit(OpCodes.Ldelem, fldEntries.FieldType.GetElementType()); // load fromDic.entries[i]
    generator.Emit(OpCodes.Stloc_1); // pair = fromDic.eEntries[i];

    generator.Emit(OpCodes.Newobj, newTo); // To t = new To();
    generator.Emit(OpCodes.Stloc_S, 4);

    foreach (PropertyInfo prop in typeof(From).GetProperties(flags))
    {
     if ((prop.GetGetMethod(true) ?? prop.GetSetMethod(true)).IsDefined(typeof(CompilerGeneratedAttribute), false))
     {
      FieldInfo fromField = GetFieldInfo<From>(prop.Name);

      if (!mapping.ContainsKey(prop.Name))
       throw new ArgumentException("The mapping dictionary does not contain a mapping for " + prop.Name);

      FieldInfo toField = GetFieldInfo<To>(mapping[prop.Name]);

      if (fromField.FieldType != toField.FieldType)
       throw new InvalidOperationException("Mapping between '" + prop.Name + "' from type '" + typeof(From).Name + "' and '" + mapping[prop.Name] + "' from type '" + typeof(To).Name + "' is invalid. The properties have different underlying types");

      generator.Emit(OpCodes.Ldloc_S, 4); // load 'to'

      // load pair.Value.<Field> on stack
      generator.Emit(OpCodes.Ldloc_1);
      generator.Emit(OpCodes.Ldfld, fldEntryValue); // load value from pair on stack
      generator.Emit(OpCodes.Ldfld, fromField);

      ////// save to to.<tofield>
      generator.Emit(OpCodes.Stfld, toField);
     }
    }


    // bypass add & insert manually into entries

    //entryTo = new Entry<,>();
    generator.Emit(OpCodes.Ldloca_S, 5);
    generator.Emit(OpCodes.Initobj, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType());


    // entryTo.key = entryFrom.key
    generator.Emit(OpCodes.Ldloca_S, 5);
    generator.Emit(OpCodes.Ldloc_1);
    generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

    // entryTo.hashCode = entryFrom.hashCode
    generator.Emit(OpCodes.Ldloca_S, 5);
    generator.Emit(OpCodes.Ldloc_1);
    generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("hashCode", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("hashCode", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

    // entryTo.next = entryFrom.next
    generator.Emit(OpCodes.Ldloca_S, 5);
    generator.Emit(OpCodes.Ldloc_1);
    generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("next", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("next", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

    // entryTo.value = 'to'
    generator.Emit(OpCodes.Ldloca_S, 5);
    generator.Emit(OpCodes.Ldloc_S, 4);
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("value", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));


    generator.Emit(OpCodes.Ldloc_0); // load entries[]
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Ldloc_2); // load i
    generator.Emit(OpCodes.Ldloc_S, 5);
    generator.Emit(OpCodes.Stelem, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType()); // save element

    generator.Emit(OpCodes.Ldc_I4_1); // load 1
    generator.Emit(OpCodes.Ldloc_2); // load i
    generator.Emit(OpCodes.Add); // i + 1
    generator.Emit(OpCodes.Stloc_2); // i = i+1

    generator.MarkLabel(loopConditionCheck);

    generator.Emit(OpCodes.Ldloc_2); // load i
    generator.Emit(OpCodes.Ldloc_S, 3); // load count
    generator.Emit(OpCodes.Blt, insideLoop); // i < fromDic.entries.Length

    generator.Emit(OpCodes.Ldloc_0); // return toDic;
    generator.Emit(OpCodes.Ret);

    data.MapDictionary = dymMethod.CreateDelegate(typeof(Func<Dictionary<string, From>, Dictionary<string, To>>));
   }
   return ((Func<Dictionary<string, From>, Dictionary<string, To>>)data.MapDictionary)(fromDic);
  }

I'm still a novice at writing IL (and have no assembly background whatsoever), so I'm sure there are probably some tweaks that can be done to make it even faster (if you know any, please let me know :)).

Well that's it for mapping properties, the last optimization there could be generalized to provide a way to quickly generate a new dictionary with a Func delegate to transform the values to a new type. Think of examples like quickly casting each value in a dictionary (which is usually done with the .ToDictionary(p => p.Key, p => (To)p.Value) Linq extension method, but lacks performance because of the unnecessary hashcode calculations).

So this is the code you were probably looking for when you read the title of this post:

 /// <summary>
  /// Cast values from a dictionary using the specified delegate. This results in a new dictionary
  /// </summary>
  public static Dictionary<string, To> CastValues<From, To>(this Dictionary<string, From> fromDic, Func<From, To> cast)
  {
   MetaData data;
   if (!store.TryGetValue(typeof(From), out data))
   {
    data = new MetaData();
    store.Add(typeof(From), data);
   }


   if (data.CastDictionaryValues == null)
   {
    // Create ILGenerator
    DynamicMethod dymMethod = new DynamicMethod("DoDictionaryCastValues", typeof(Dictionary<string, To>), new Type[] { typeof(Dictionary<string, From>) }, true);
    ConstructorInfo newDictionaryTo = typeof(Dictionary<string, To>).GetConstructor(new Type[] { typeof(int) });
    FieldInfo fldEntries = typeof(Dictionary<string, From>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance);

    FieldInfo fldEntryKey = fldEntries.FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
    FieldInfo fldEntryValue = fldEntries.FieldType.GetElementType().GetField("value", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);

    ILGenerator generator = dymMethod.GetILGenerator();

    // define labels for loop
    Label loopConditionCheck = generator.DefineLabel();
    Label insideLoop = generator.DefineLabel();

    // define local variables
    generator.DeclareLocal(typeof(Dictionary<string, To>)); // toDic , 0
    generator.DeclareLocal(fldEntries.FieldType.GetElementType()); // pair entry<string, from>, 1
    generator.DeclareLocal(typeof(int)); // i, 2
    generator.DeclareLocal(typeof(int)); // count, 3;
    generator.DeclareLocal(typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType()); // entry<to> 4;
    generator.DeclareLocal(typeof(int)); // bucketLength, 5;
    generator.DeclareLocal(typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance).FieldType); // newbuckets, 6;

    // store count
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Callvirt, typeof(Dictionary<string, From>).GetProperty("Count").GetGetMethod());
    generator.Emit(OpCodes.Stloc_S, 3);

    generator.Emit(OpCodes.Ldloc_S, 3); // load count and pass it as capacity parameter for toDic
    generator.Emit(OpCodes.Newobj, newDictionaryTo); // toDic = new ...
    generator.Emit(OpCodes.Stloc_0);

    // COPY Dictionary fields to toDic

    // toDic.buckets = fromDic.buckets;

    // This is incorrect, changes on old dictionary will influence the new one because the reference is copied!!!
    //generator.Emit(OpCodes.Ldloc_0);
    //generator.Emit(OpCodes.Ldarg_0);
    //generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
    //generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));

    // This is the correct way to copy the bucket array
    // bucketLength = fromDic.buckets.Length
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Ldlen);
    generator.Emit(OpCodes.Stloc_S, 5);
    // newbuckets = new Int32[bucketLength]
    generator.Emit(OpCodes.Ldloc_S, 5);
    generator.Emit(OpCodes.Newarr, typeof(Int32));
    generator.Emit(OpCodes.Stloc_S, 6);

    // Array.Copy(fromDic.buckets, newbuckets, bucketLength)
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Ldloc_S, 6);
    generator.Emit(OpCodes.Ldloc_S, 5);
    generator.Emit(OpCodes.Call, typeof(Array).GetMethod("Copy", new Type[] { typeof(Array), typeof(Array), typeof(Int32) }));

    // toDic.buckets = newbuckets
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldloc_S, 6);
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));


    // toDic.comparer = fromDic.comparer;
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("comparer", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("comparer", BindingFlags.NonPublic | BindingFlags.Instance));

    // toDic.count = fromDic.count;
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("count", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("count", BindingFlags.NonPublic | BindingFlags.Instance));

    // toDic.freeCount = fromDic.freeCount;
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("freeCount", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("freeCount", BindingFlags.NonPublic | BindingFlags.Instance));

    // toDic.freeList = fromDic.freeList;
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("freeList", BindingFlags.NonPublic | BindingFlags.Instance));
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("freeList", BindingFlags.NonPublic | BindingFlags.Instance));

    // toDic.entries = new Entry<,>[fromDic.entries.Length];
    generator.Emit(OpCodes.Ldloc_0);
    generator.Emit(OpCodes.Ldarg_0);
    generator.Emit(OpCodes.Ldfld, fldEntries);
    generator.Emit(OpCodes.Ldlen);
    generator.Emit(OpCodes.Newarr, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType());
    generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance));

    // End COPY

    // i = 0;
    generator.Emit(OpCodes.Ldc_I4_0);
    generator.Emit(OpCodes.Stloc_2);

    generator.Emit(OpCodes.Br, loopConditionCheck); // perform loop test
    generator.MarkLabel(insideLoop);
    {
     // pair = fromDic.entries[i];
     generator.Emit(OpCodes.Ldarg_0); // load fromDic on stack
     generator.Emit(OpCodes.Ldfld, fldEntries); // load entries field from dic on stack
     generator.Emit(OpCodes.Ldloc_2); // load i
     generator.Emit(OpCodes.Ldelem, fldEntries.FieldType.GetElementType()); // load fromDic.entries[i]
     generator.Emit(OpCodes.Stloc_1);

     // bypass add & insert manually into entries from toDic

     // entryTo = new Entry<,>();
     generator.Emit(OpCodes.Ldloca_S, 4);
     generator.Emit(OpCodes.Initobj, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType());

     // entryTo.key = entryFrom.key
     generator.Emit(OpCodes.Ldloca_S, 4);
     generator.Emit(OpCodes.Ldloc_1);
     generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
     generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("key", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

     // entryTo.hashCode = entryFrom.hashCode
     generator.Emit(OpCodes.Ldloca_S, 4);
     generator.Emit(OpCodes.Ldloc_1);
     generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("hashCode", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
     generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("hashCode", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

     // entryTo.next = entryFrom.next
     generator.Emit(OpCodes.Ldloca_S, 4);
     generator.Emit(OpCodes.Ldloc_1);
     generator.Emit(OpCodes.Ldfld, fldEntries.FieldType.GetElementType().GetField("next", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));
     generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("next", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

     // entryTo.value = 'to'
     generator.Emit(OpCodes.Ldloca_S, 4);
     // call cast(pair.value)
     generator.Emit(OpCodes.Ldloc_1);
     generator.Emit(OpCodes.Ldfld, fldEntryValue); // load value from pair on stack
     generator.Emit(OpCodes.Call, cast.Method);
     // and store the to value into the new entry
     generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType().GetField("value", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public));

     // toDic.entries[i] = entryTo;
     generator.Emit(OpCodes.Ldloc_0); // load entries[]
     generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance));
     generator.Emit(OpCodes.Ldloc_2); // load i
     generator.Emit(OpCodes.Ldloc_S, 4);
     generator.Emit(OpCodes.Stelem, typeof(Dictionary<string, To>).GetField("entries", BindingFlags.NonPublic | BindingFlags.Instance).FieldType.GetElementType()); // save element

     generator.Emit(OpCodes.Ldc_I4_1); // load 1
     generator.Emit(OpCodes.Ldloc_2); // load i
     generator.Emit(OpCodes.Add); // i + 1
     generator.Emit(OpCodes.Stloc_2); // i = i+1
    }
    generator.MarkLabel(loopConditionCheck);

    generator.Emit(OpCodes.Ldloc_2); // load i
    generator.Emit(OpCodes.Ldloc_S, 3); // load count
    generator.Emit(OpCodes.Blt, insideLoop); // i < fromDic.entries.Length

    generator.Emit(OpCodes.Ldloc_0); // return toDic;
    generator.Emit(OpCodes.Ret);

    data.CastDictionaryValues = dymMethod.CreateDelegate(typeof(Func<Dictionary<string, From>, Dictionary<string, To>>));
   }
   return ((Func<Dictionary<string, From>, Dictionary<string, To>>)data.CastDictionaryValues)(fromDic);
  }

This can be used like:

 pCopyDic = personsDic.CastValues<Person, PersonCopy>(p =>
    {
     PersonCopy pCopy = new PersonCopy();
     pCopy.DateOfBirth = p.DateOfBirth;
     pCopy.FirstName = p.FirstName;
     pCopy.Function = p.Function;
     pCopy.GUID = p.GUID;
     pCopy.IsEmployee = p.IsEmployee;
     pCopy.LastName = p.LastName;
     pCopy.ParentGUID = p.ParentGUID;
     pCopy.Rating = p.Rating;
     return pCopy;
    });

To check out how a Dictionary is implemented I used .NET Reflector, to check how I had to write IL statements I searched on various websites and wrote the equivalent in a dummy method in C#, compiled it then disassembled it with Reflector.

I know micro optimization like this is usually not needed, so don't always rely that it will work well. This optimization can improve speed a lot but won't fix anything if you have a structural problem (that scales exponentially). That said, it's awesome to tinker with IL and improve framework related issues :).

Edit: It seems I had a slight oversight. Copying the buckets from 1 dictionary to another copied the reference, so any changes on the source dictionary reflected on the new dictionary as well! This is the correct code (sigh now to change it in the already html marked up code above):

 // toDic.buckets = fromDic.buckets;
 // This is incorrect, changes on old dictionary will influence the new one because the reference is copied!!!
 //generator.Emit(OpCodes.Ldloc_0);
 //generator.Emit(OpCodes.Ldarg_0);
 //generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
 //generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));

 // This is the correct way to copy the bucket array
 // bucketLength = fromDic.buckets.Length
 generator.Emit(OpCodes.Ldarg_0);
 generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
 generator.Emit(OpCodes.Ldlen);
 generator.Emit(OpCodes.Stloc_S, 5);
 // newbuckets = new Int32[bucketLength]
 generator.Emit(OpCodes.Ldloc_S, 5);
 generator.Emit(OpCodes.Newarr, typeof(Int32));
 generator.Emit(OpCodes.Stloc_S, 6);

 // Array.Copy(fromDic.buckets, newbuckets, bucketLength)
 generator.Emit(OpCodes.Ldarg_0);
 generator.Emit(OpCodes.Ldfld, typeof(Dictionary<string, From>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));
 generator.Emit(OpCodes.Ldloc_S, 6);
 generator.Emit(OpCodes.Ldloc_S, 5);
 generator.Emit(OpCodes.Call, typeof(Array).GetMethod("Copy", new Type[] { typeof(Array), typeof(Array), typeof(Int32) }));

 // toDic.buckets = newbuckets
 generator.Emit(OpCodes.Ldloc_0);
 generator.Emit(OpCodes.Ldloc_S, 6);
 generator.Emit(OpCodes.Stfld, typeof(Dictionary<string, To>).GetField("buckets", BindingFlags.NonPublic | BindingFlags.Instance));

This reminds me that I really need to do some layout changes, because copy pasting code in here is a pain.

2D Tree generation algorithm

I've been working on a tree generation algorithm since yesterday (after having a couple of ideas I wanted to try for a week now). 

It's based on a particle engine rather than fractals (which seem to be the most common method of procedural tree generation I encountered online). I searched if anyone had a similar approach but couldn't exactly find one (there is one that uses particles for the leaf textures, but not for the entire tree building).

So here goes, it's a little crude and it would probably still need some fine tuning to remove obvious artifacts and oddities but I think it works rather well in 2D that is. I hope it will scale well to 3D, I think it should due to the nature of particles but until I have finished that and found some acceptable results I won't take it for granted.

I made a quick step by step of the algorithm, or rather the behaviour of the different particles used.

And here are some results:












The algorithm is a hybrid between cellular automata and a particle engine. While it uses particles to draw the tree, they can also be interpreted as individual cells with behaviour like cellular automata do.

--Dwight

Using datasets on the C# Compact Framework

I am in the middle of developing a CRM client for mobiles in C# 3.5SP1 with Visual studio 2008 and the after a few weeks of implementation I noticed the performance of the program is bad, and I mean very bad. At first I thought it was the WM5.0 Device emulator that was slowing things down, it uses a ARM cpu so it needs to emulate it completely.

However I had access to a WM5.0 Smartphone device last week and it appears that it the performance is equally bad, so I started investigating what could be the cause of the slowdown. I read on msdn that creating forms with a lot of nested controls could cause a significant slowdown, so I applied the suggested optimizations (creating controls top->down with setting Parent property of the childs and use the Bounds property instead of a combination of Size and Location) but I saw little difference in speed.

I'm using a SQL CE 3.5SP1 Compact database and I already figured that the queries would be the cause, so I started testing the speed and came to some very bad results:

A query SELECT * FROM Entities, with about 5 columns and 3 records took a whopping 1500ms. I was using a generated dataset for the queries, so I checked if the speed would improve if I used the SqlCeConnection with SqlCeCommand manually.

My first implementation was even slower with 2000ms, but this was because I opened & closed the connection each time I called the method, leaving a lot to be desired on speed. SQL CE doesn't support connection pooling, so opening and closing each time was a huge slowdown.

After I wrote a connection pool singleton class (basically a stack with available connections made thread safe) with 2 methods Aqcuire() and Release() which pop & push a connection. I opened them if they were closed at Aqcuire and I leave them opened. I changed my retrieve method with this approach and I got 400ms, which is still pretty bad but a huge optimization from where I started.

I don't know why datasets are so much slower, but they are. So write your queries yourself instead of relying on a dataset with generated code, it's much faster. If you have a lot of queries or a lot of tables in the dataset write your own generator that generates the code for the classes.

It's not much more code if you group the same code parts (such as reading from the SQLCeDataReader) than if you used a datatable, so it's definately worth using if you're focused on performance.