Also available at

Also available at my website http://tosh.me/ and on Twitter @toshafanasiev

Wednesday, 15 August 2012

C++ ENSURE macro

C++ doesn't provide a finally clause for structured exception handling and it doesn't need to. The deterministic object destruction of C++ provides the perfect way to ensure that your cleanup code is executed - tying your resource management to an object's lifetime ensures that it will take place when the memory that object occupies is reclaimed; for local objects this corresponds to control hitting the end of the lexical scope in which the object is declared, whether that scope is left via normal flow or due to an exception.
There are numerous Scope Guard or Scope Exit constructs around, including one in the Boost library and the D Language's scope(exit) construct, that aim to tidy up this pattern, making it quicker to write and easier to read. My good friend Steve recently sent me his version of this idea - it was characteristically brilliant and I decided to have a go myself.
Using the example of a wrapper for FormatMessage (a Windows function which I personally can never remember how to call without consulting documention), we can see how RAII can be gainfully employed. The flags I'm passing in tell FormatMessage to allocate a buffer big enough for the data returned, that allocation is done using LocalAlloc and it's up to us to clean up by using LocalFree. Here's how you might go about this (console output added for illustration)

#include <tchar.h>
#include <string>

typedef std::basic_string<TCHAR> tstring;

namespace manual
{
  tstring MessageFromHR( HRESULT hr )
  {
    LPTSTR ptr = NULL;
    struct cleanup {
      LPTSTR& p;
      ~cleanup() {
        cout << "about to free pointer 0x" << hex << (int)p << endl;
        LocalFree( p );
        cout << "done" << endl;
      }
    } cu = { ptr };

    FormatMessage(
        FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM
      , NULL
      , hr
      , MAKELANGID( LANG_NEUTRAL, SUBLANG_NEUTRAL )
      , (LPTSTR)&ptr
      , 0
      , NULL
      );

    return tstring( ptr );
  }
}

This approach is very wordy but it does nicely put the cleanup code right next to the declaration of the variable that represents the leak-prone resource. This lexical locality has a huge effect on readability and hence ultimately on reliability.
What can be done about the wordiness though? Any syntactic sweetener for this pattern will have to simplify usage, obviously, or what's the point? Well, thanks to some of the new features introduced with C++ 11 we can wrap the whole affair up nicely in a macro that's simple to write and clear when read. Let's build it up bit by bit.
The main thing we want to achieve is ease of use, and hence readability (and then onto reliability) - this means keeping the number of parameters to a minimum. So what is the minimum? Arbitrary cleanup code could involve an arbitrary number of parameters and so to avoid hand-cranking each and every reference we'd employ a lambda, closing around any variables we need. This neatly brings the maximum number of parameters to our macro down to one - the body of that lambda.
We have no way of knowing, up front, what the type of such a construct will be called, and so we turn to the tools of type inference, auto and decltype. These keywords allow us to declare a variable of fixed but unknown type and, crucially, refer to that type later on

#include <tchar.h>
#include <string>

typedef std::basic_string<TCHAR> tstring;

namespace lambda
{
  tstring MessageFromHR( HRESULT hr )
  {
    LPTSTR ptr = NULL;
    auto __l = [&]() {
      cout << "about to free pointer 0x" << hex << (int)ptr << endl;
      LocalFree( ptr );
      cout << "done" << endl;
    };
    struct cleanup {
      decltype( __l )& l;
      ~cleanup() { l(); }
    } __i = { __l };

    FormatMessage(
        FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM
      , NULL
      , hr
      , MAKELANGID( LANG_NEUTRAL, SUBLANG_NEUTRAL )
      , (LPTSTR)&ptr
      , 0
      , NULL
      );

    return tstring( ptr );
  }
}

Turning it into a macro is pretty simple

#define SIMPLE_ENSURE( routine ) auto __l = [&]() { routine; };\
                                 struct __s {\
                                   decltype( __l )& l;\
                                   ~__s() { l(); }\
                                 } __i = { __l };

But we're not quite done here - given that we're defining named types and instances we'll be limited to one per lexical scope. We can fix this by using the __LINE__ macro to generate unique names. Note the ID macro and inner implementation ENSURE_ - this is necessary due to the way macro expansion works. Note also that the instance variable r in the struct is a reference to the lambda.

#define ID( base, unique ) base##unique
#define ENSURE_( routine, unique ) auto ID( r, unique ) = [&]() { routine; };\
                                    struct ID( e, unique ) {\
                                      decltype( ID( r, unique ) )& r;\
                                      ID( ~e, unique )() { r(); }\
                                    } ID( ei, unique ) = { ID( r, unique ) };
#define ENSURE( routine ) ENSURE_( routine, __LINE__ )

Here's the FormatMessage wrapper again, this time using the ENSURE macro

#include <tchar.h>
#include <string>

typedef std::basic_string<TCHAR> tstring;

namespace macro
{
  tstring MessageFromHR( HRESULT hr )
  {
    LPTSTR ptr = NULL;
    ENSURE(
      cout << "about to free pointer 0x" << hex << (int)ptr << endl;
      LocalFree( ptr );
      cout << "done" << endl;
    )

    FormatMessage(
        FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM
      , NULL
      , hr
      , MAKELANGID( LANG_NEUTRAL, SUBLANG_NEUTRAL )
      , (LPTSTR)&ptr
      , 0
      , NULL
      );

    return tstring( ptr );
  }
}

All these examples yield the same output (pointers may vary, but notice it's no longer NULL by the time it's freed - that's because the lambda captures by reference)

about to free pointer 0x4888e0
done

And all return the correct message

No such interface supported

And here's an example of multiple instances in the same scope

int main( int argc, const char* argv[] )
{
  try
  {
    cout << "one" << endl;
    ENSURE( cout << "four" << endl )
    ENSURE( cout << "three" << endl )
    cout << "two" << endl;
    throw std::exception();
  }
  catch ( const std::exception& e )
  {
    cout << "five" << endl;
  }
}

This illustrates how the execution of ENSURE blocks is deferred until the end of the current scope and also the fact that they run in reverse order, as per normal object destruction. And here's what the preprocessor chucks out (line breaks added)

int main( int argc, const char* argv[] )
{
  try
  {
    cout << "one" << endl;
    auto r141 = [&]() { cout << "four" << endl; }; struct e141
    { decltype( r141 )& r; ~e141() { r(); } } ei141 = { r141 };
    auto r142 = [&]() { cout << "three" << endl; }; struct e142
    { decltype( r142 )& r; ~e142() { r(); } } ei142 = { r142 };
    cout << "two" << endl;
    throw std::exception();
  }
  catch ( const std::exception& e )
  {
    cout << "five" << endl;
  }
}

Of course in production code you wouldn't have all the console output and it would look like this

#include <tchar.h>
#include <string>

typedef std::basic_string<TCHAR> tstring;

namespace production
{
  tstring MessageFromHR( HRESULT hr )
  {
    LPTSTR ptr = NULL;
    ENSURE( LocalFree( ptr ) )

    FormatMessage(
        FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM
      , NULL
      , hr
      , MAKELANGID( LANG_NEUTRAL, SUBLANG_NEUTRAL )
      , (LPTSTR)&ptr
      , 0
      , NULL
      );

    return tstring( ptr );
  }
}

I should mention that this sort of in-line cleanup code is not always appropriate - sometimes what you need is a proper class, defined in its own file, that wraps up a resource and manages its life time. This gives you testability and separation of concerns. However, as with all things, it's a trade off and I feel that the lexical locality wins out in the case of that one little call to LocalFree.

Wednesday, 18 July 2012

An Ode to Python in C#

One of my favourite features of Python is the List Comprehension - a feature that allows a list (Python's fundamental sequential collection type) to be declared in a literal-like way using the definition of its contents, rather than their enumeration. What's the difference?

Well, in Python an enumerative list literal looks like this

>>> numbers = [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
>>> numbers
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>

Each item in the list is included in the literal - they are enumerated. This is not a huge problem for 0 to 9 but will not scale particularly well, as you can imagine. The list comprehension allows you to declare a list literal using an expression that describes the contents of the list rather than, for want of a better word, listing them:

>>> numbers = [ i for i in range( 10 ) ]
>>> numbers
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>>

Actually that's not quite right - it's not really the list comprehension that I'm out to emulate here - that's just one use for it, and the one people usually encounter first. My real goal is to reproduce the bit that makes the list comprehension work - the generator expression between the square brackets. Generators can be used outside the context of lists, for example

>>> sum( i for i in range( 10 ) )
45
>>>

Here I've passed a generator to a function whose required parameter is simply something that can be used in a for loop - that's a Python for loop, the C# equivalent would be a foreach loop so I guess the C# equivalent of a generator would be a construct that allows us to define arbitrary IEnumerable<T> instances in as literal-like a way as possible.

It's no coincidence that the IEnumerable<T> interface (in the System.Collections.Generic namespace) is the fundamental building block of LINQ - in the .NET world it represents the essence of a sequence of items. Having a terse, expressive, literal-like way of defining arbitrary IEnumerable<T> objects allows us to decouple control flow structures from the operations being controlled while also giving us the option to define operations inline, without the need for separate class definitions that could hurt readability.

If I just wanted to stick to trivial examples like the one above, all the tools I need are actually present in the Enumerable class (System.Linq namespace - the one that defines all those great IEnumerable<T> extension methods) - it has a Range function similar to Python's and of course the ToList() extension

using System;
using System.Linq;

// ...

var numbers = Enumerable.Range( 0, 10 ).ToList();

But what about a more interesting list of numbers? Python's range function allows you to additionally specify a step parameter (to use it, you also have to specify the start value)

>>> numbers = [ i for i in range( 0, 10, 2 ) ]
>>> numbers
[0, 2, 4, 6, 8]
>>>

I suppose we could achieve this effect in C# by scaling a step 1 range by projection

var numbers = Enumerable.Range( 0, 5 ).Select( i => i * 2 ).ToList();

But this is already less readable and has only pushed the problem back - this is all based on a fairly specialised sequence, and while that sequence can be projected to anything you like using the Select() extension, it does require you to know up front how long the sequence will be. And what about access to the previous value in the sequence?

What we need is to be able to take any for loop

for ( var i = 0; i < 10; ++i )
 Console.WriteLine( i );

and package it up and pass it around like any other data structure, having it play nicely with all the IEnumerable<T> based goodness in .NET - including deferred execution.

To do it we need something like this

public static class Generator
{
 public static IEnumerable<T> For<T>( Func<T> setup, Predicate<T> test, Func<T, T> adv )
 {
  for ( var i = setup(); test( i ); i = adv( i ) )
  yield return i;
 }

 public static IEnumerable<T> For<T>( T start, Predicate<T> test, Func<T, T> adv )
 {
  return For( () => start, test, adv );
 }
}
which can be used like so
var numbers = Generator.For( 0, i => i < 10, i => ++i );
or
var pow2 = Generator.For( 1, i => i < 1000, i => i * 2 );
or
var letters = Generator.For( 'a', c => c <= 'z', c => ++c );
var capitals = letters.Select( c => Char.ToUpper( c ) );
etc.

Note: I've declared the setup parameter as an operation in the actual implementation, for flexibility, while providing a value overload for convenience.

Just as you'd expect from LINQ-like expressions these generators are not evaluated until they are iterated over - deferred execution. This is quite cool - you want the alphabet in a list? On one terse line?

var alphabet = Generator.For( 'a', c => c <= 'z', c => ++c ).ToList();

As is often the case though, there's a gotcha. Let's take an aside to explore this. Many developers, particularly those who've not done much C++, will write a for loop like this

for ( int i = 0; i < 10; i++ )
 // ...

And while there's nothing wrong with the semantics of this, the performance of this loop can be less than optimal if the integer is replaced with an STL iterator for instances of which copying and destruction can be non-trivial.

// C++
std::vector<int> ints;

// populate the vector

// iterator is unnecessarily copied by i++
for ( auto i = ints.begin(), e = ints.end(); i != e; i++ )
 // ...

For this reason C++ programmers will usually write their loops like this

for ( int i = 0; i < 10; ++i )
 // ...

using the prefix increment operator even for integers, just out of habit.

With the Generator class though, the semantics become important. A for loop's header consists of three clauses representing three distinct operations - setup, test, advance. Not values, operations. The advance operation does not yield a value that replaces that of the control variable - the for loop is not smart enough for that - the advance operation is responsible for modifying the control variable directly. Hence in the case of a language level for loop (performance considerations and the STL aside) it does not matter whether you choose the pre- or postfix increment operator - the resulting value is discarded and the variable ends up in the correct state.

The Generator.For<T> method takes three functions as arguments to setup, test and advance the control variable respectively. For a value type like an int, each invocation of the advance operation results in a local copy of the control variable, the modification of which is not visible outside of the function. For this reason, it is the value yielded by the advance operation that is important - this is used to overwrite the control variable.

So these are ok

var letters = Generator.For( 'a', c => c <= 'z', c => ++c );
var numbers = Generator.For( 0, i => i < 1000, i => i + 5 );
But this
var endless = Generator.For( 0, i => i < 5, i => i++ ); // don't do this
will give you a sequence of zeros that never ends. Remember, the postfix increment operator yields the value of the operand before the increment.
i == i++ // think about it

Ok great, for loops are covered. But I think there's also value in supporting while and do-while.

For instance, say you want to accept a password from a user and store it in a SecureString object. Let's also say that you want to be able to test the code that does this without having to sit there hammering away at the keys. What you need is an API that accepts an arbitrary sequence of characters

public static class Extensions
{
  public static SecureString ToSecureString( this IEnumerable<char> characters )
  {
    return characters.Aggregate(
        new SecureString()
      , ( ss, c ) => { ss.AppendChar( c ); return ss; }
    );
  }
}

public class Safe
{
  public void Unlock( IEnumerable<char> passwordChars )
  {
    var pwd = passwordChars.ToSecureString();

    // use SecureString here ....
  }
}

and then some terse, readable way of driving it from, for instance, the console
public static class Generator
{

  // ... For<T> methods omitted

  public static IEnumerable<T> YieldWhile<T>( Func<T> source, Predicate<T> condition )
  {
    var value = source();

    while ( condition( value ) )
    {
      yield return value;

      value = source();
    }
  }
}

// ...

// ReadKey( true ) suppresses key output
var fromPrompt = Generator.YieldWhile(
    () => Console.ReadKey( true )
  , k => k.Key != ConsoleKey.Enter
  ).Select( k => k.KeyChar );


// production code

var safe = new Safe();
safe.Unlock( fromPrompt );


// test code

var safe = new Safe();
safe.Unlock( "1234" );

So YieldWhile<T> lets us enumerate a sequence of values while they continue to satisfy some condition. What about the reverse - enumerating a sequence of values only while some condition is satisfied?


public static class Generator
{
  // other methods omitted

  public static IEnumerable<T> WhileYield( Func<bool> condition, Func<T> source )
  {
    while( condition() )
      yield return source();
  }
}


which can be use like so

var input = Generator.WhileYield( () =>
    {
      Console.Write( "more values? (y/n): " );
      return Console.ReadLine() == "y";
    }
  , () => Console.ReadLine()
  ).ToList();


These simple functions allow you to define arbitrary and possibly large or unpredictable sequences of values in a terse, readable and flexible way. You can use the results with LINQ. You can have an enumerable data structure with a human being on the end of it supplying the values. You can even create a list of the first ten integers, just like a Python programmer :)

A complete listing of the Generator class follows:


public static class Generator
{
  public static IEnumerable<T> For<T>( Func<T> setup, Predicate<T> test, Func<T, T> adv )
  {
    for ( T i = setup(); test( i ); i = adv( i ) )
      yield return i;
  }

  public static IEnumerable<T> For<T>( T start, Predicate<T> test, Func<T, T> adv )
  {
    return For( () => start, test, adv);
  }

  public static IEnumerable<T> YieldWhile<T>( Func<T> source, Predicate<T> condition )
  {
    var value = source();

    while ( condition( value ) )
    {

      yield return value;
      value = source();
    }
  }

  public static IEnumerable<T> WhileYield<T>( Func<bool> condition, Func<T> source )
  {
    while ( condition() )
      yield return source();
  }
}

Friday, 29 June 2012

An Ode to Ruby in C#

I love going off and exploring new languages, particularly when it allows (forces) me to think differently about the kind of stuff I generally take for granted. Take Ruby for instance. In Ruby you can do this

10.times do |i|
  puts "The square of #{i} is #{i * i}"
end
Which outputs
The square of 0 is 0
The square of 1 is 1
The square of 2 is 4
The square of 3 is 9
The square of 4 is 16
The square of 5 is 25
The square of 6 is 36
The square of 7 is 49
The square of 8 is 64
The square of 9 is 81

This is cool - the iteration construct reads like a sentence (perhaps not uttered in the speaker's first language, but a sentence all the same). The do ... end bit is called a block and it's a bit of code you can pass to a function to be executed at that function's discretion; the i between the pipe symbols is the parameter list for the block and the funky #{ ... } construct allows string interpolation - the evaluation of expressions inside a string literal.

Now I'm not going to get carried away here - the string interpolation can probably wait for a later post - but I can see value in being able to pass a lump of code, in a very readable way, to another function. The trite example above doesn't really do justice to the power this construct offers; features such as logging, timing, thread synchronisation, iteration, error protection, deterministic cleanup, inversion of control etc. are all candidates for this sort of code injection - with heavy emphasis on readability.

Now, I say to myself, modern C# supports lambdas and extension methods mean that you can simulate Ruby's ability to re-open and add to a class definition, I can do that; and this is what I come up with

    public static class Extensions
    {
        public static void Times(this int number, Action<int> action)
        {
            for (var i = 0; i < number; ++i)
                action(i);
        }
    }

Which is used like this

    10.Times(
        i => Console.WriteLine("The square of {0} is {1}", i, i * i)
    );

That's not bad, I think, it's pretty terse, but I can almost hear my Ruby friends tut as they catch sight of the lone punctuation on the last line. I can do better, but I'll need a new class.

    public sealed class Block<T>
    {
        private readonly Action<Action<T>> wrapper;
        public Block(Action<Action<T>> wrapper)
        {
            if (wrapper == null)
                throw new ArgumentNullException("wrapper");

            this.wrapper = wrapper;
        }
        public Action<T> Do { set { wrapper(value); } }
    }

    public static class Extensions
    {
        public static Block<int> Times(this int number)
        {
            return new Block<int>(action =>
            {
                for (var i = 0; i < number; ++i) action(i);
            });
        }
    }

Having set this up I can now call it like this

    10.Times().Do = i =>
        Console.WriteLine("The square of {0} is {1}", i, i * i);

I like this. Assigning the lambda to the write-only property has the effect of passing it to whatever wrapper the Block was initialised with. The Block class is a reusable little nugget of orchestration that's totally decoupled from whatever logic you choose to use it with, but the best part is it's quite declarative at the call site; and if you squint a bit you don't even see the assignment and lambda operators :)

Ok, but this seems pretty much geared to enumeration/iteration, what about a more generalised code injection scenario?

    public sealed class Block
    {
        private readonly Action wrapper;
        public Block(Action wrapper)
        {
            if (wrapper == null)
                throw new ArgumentNullException("wrapper");

            this.wrapper = wrapper;
        }
        public Action Do { set { wrapper(value); } }
    }

    public static class Wrap
    {
        public static Block With(Action wrapper)
        {
            return new Block(wrapper);
        }
    }

It's the same class! Well, nearly - it's the Block equivalent to Action<T>'s non-generic counterpart (I'd love to find a way to coalesce the two). The Wrap class here doesn't really add anything other than making the call site a bit more sentencey, as shown below

        var withLogging = Wrap.With(action =>
            {
                Console.Write("1. Before\n2. During: ");
                try
                {
                    action();
                    Console.WriteLine("3. After");
                }
                catch (Exception ex)
                {
                    Console.WriteLine("3. Exception: {0}", ex.Message);
                }
                finally
                {
                    Console.WriteLine("4. Cleanup");
                }
            });

With that variable in scope I can do this

        withLogging.Do = () => Console.WriteLine("This is going ok.");

        Console.WriteLine();

        withLogging.Do = () =>
        {
            Console.WriteLine("This might not go so well...");
            throw new Exception("KERBLAM!");
        };

With the following output

1. Before
2. During: This is going ok.
3. After
4. Cleanup

1. Before
2. During: This might not go so well...
3. Exception: KERBLAM!
4. Cleanup

All in all I'm pretty pleased with this. The idea could also be extended to functions returning values by means of the Func<T> family of delegates.

Afterthought: is this an instantiation of a named pattern?

Wednesday, 8 February 2012

C++/CLI literal keyword bug

In C# you have two similar but importantly different options for defining constants; const and static readonly, as shown:

// names.cs
// compile with csc /t:library names.cs
public static class Names {
  public const string First = "Tosh";
  public static readonly string Last  = "Afanasiev";
}

The compiler generates the following IL for this code:

.assembly names

// lots of guff omitted

.class public abstract auto ansi sealed beforefieldinit Names
       extends [mscorlib]System.Object
{
  .field public static literal string First = "Tosh"
  .field public static initonly string Last
  .method private hidebysig specialname rtspecialname static 
          void  .cctor() cil managed
  {
    // Code size       11 (0xb)
    .maxstack  8
    IL_0000:  ldstr      "Afanasiev"
    IL_0005:  stsfld     string Names::Last
    IL_000a:  ret
  } // end of method Names::.cctor

} // end of class Names

Both Names.First and Names.Last are constants in the sense that you can consume but not modify their values but the way they are bound is the crucial difference. Notice how the First symbol is bound to its value in metadata - it is used as an alias for the literal (hence the literal flag) string 'Tosh'; while the Last symbol is declared but assigned to in the type initialiser (or static constructor, if you prefer), thereby making its value only available once the Names class has been loaded in the current app domain. Hence the value of First is knowable at compile time which the value of Last can only be known at run time.

This difference is most clearly illustrated by examining the code that consumes these values; here are two C# programs and the IL generated for them:


// firstname.cs
// compile with csc /r:names.dll firstname.cs
using System;

static class prog {
  static void Main() {
    Console.WriteLine( "First name: {0}", Names.First );
  }
}


.assembly firstname

// guff

.class private abstract auto ansi sealed beforefieldinit prog
       extends [mscorlib]System.Object
{
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       18 (0x12)
    .maxstack  8
    IL_0000:  nop
    IL_0001:  ldstr      "First name: {0}"
    IL_0006:  ldstr      "Tosh"
    IL_000b:  call       void [mscorlib]System.Console::WriteLine(string,
                                                                  object)
    IL_0010:  nop
    IL_0011:  ret
  } // end of method prog::Main

} // end of class prog


// lastname.cs
// compile with csc /r:names.dll lastname.cs
using System;

static class prog {
  static void Main() {
    Console.WriteLine( "Last name: {0}", Names.Last );
  }
}


.assembly extern names
{
  .ver 0:0:0:0
}
.assembly lastname

// guff

.class private abstract auto ansi sealed beforefieldinit prog
       extends [mscorlib]System.Object
{
  .method private hidebysig static void  Main() cil managed
  {
    .entrypoint
    // Code size       18 (0x12)
    .maxstack  8
    IL_0000:  nop
    IL_0001:  ldstr      "Last name: {0}"
    IL_0006:  ldsfld     string [names]Names::Last
    IL_000b:  call       void [mscorlib]System.Console::WriteLine(string,
                                                                  object)
    IL_0010:  nop
    IL_0011:  ret
  } // end of method prog::Main

} // end of class prog

The difference that immediately jumps out is that the firstname program's code contains a copy of the literal value Names.First, as defined in the metadata of that assembly; while lastname loads the static field Names.Last in order to evaluate the constant. A less obvious difference is that lastname declares a dependency on the names assembly while firstname does not.

So what are the implications of this? Firstly, since Names.Last is evaluated at runtime by loading another assembly and accessing a static field, the value can be changed after deployment without recompiling and redistributing the consuming code - great when you have numerous clients, possibly not your own, that need to be kept up to date. Assemblies consuming the Names.First value would be totally unaware any change to this value unless they were themselves recompiled.

The other side of this coin is that the consuming assembly has a runtime dependency on names.dll (firstname will run fine on its own while lastname fails miserably with names.dll out of reach) and, more subtly, the use of the constant Names.Last is limited to runtime contexts. This last point may not be a problem but consider the case where attribute values are centralised, the following would result in a compile error:


[FicticiousAttribute(Names.Last)]
public class ...

While this would not:

[FicticiousAttribute(Names.First)]
public class ...

So, when choosing how to declare a shared constant you need to think about how shared and how constant it actually is.

Now for the bug.

The keywords in C++/CLI resemble their IL counterparts more closely than in C# - literal is used to denote compile time constants while static initonly is used for those dynamically evaluated runtime constants (obviously they couldn't use const).

Here's where I noticed a bug in the VS2005 and VS2008 C++/CLI compilers (not VS2010 though). If you define a literal (i.e. metadata based, class level compile time constant ) value on an abstract sealed class (that's a static class in C#) you get a compile error:


// values.cpp
// compile with cl /clr values.cpp
using namespace System;

  public ref class Names abstract sealed {
  public:
    static initonly String^ First = "Tosh";
    literal String^ Last = "Afanasiev";
  };

int main() {
  Console::WriteLine( "Hi, my name is {0} {1}!", Names::First, Names::Last );

  return 0;
}

VS2005:

C:\code>cl /clr values.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 14.00.50727.762
for Microsoft (R) .NET Framework version 2.00.50727.5420
Copyright (C) Microsoft Corporation.  All rights reserved.

values.cpp
values.cpp(9) : error C4693: 'Names': a sealed abstract class cannot have any in
stance members 'Last'

VS2008:

C:\code>cl /clr values.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 15.00.30729.01
for Microsoft (R) .NET Framework version 2.00.50727.5420
Copyright (C) Microsoft Corporation.  All rights reserved.

values.cpp
values.cpp(9) : error C4693: 'Names': a sealed abstract class cannot have any in
stance members 'Last'

VS2010:

C:\code>cl /clr values.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 16.00.30319.01
for Microsoft (R) .NET Framework version 4.00.30319.1
Copyright (C) Microsoft Corporation.  All rights reserved.

values.cpp
Microsoft (R) Incremental Linker Version 10.00.30319.01
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:values.exe
values.obj

C:\code>values
Hi, my name is Tosh Afanasiev!

The compiler complains of instance members on an abstract sealed class when it's not an instance member we're adding.

To get around this you have to suppress error 4693. I'd suggest making the scope of this as tight as possible and clearly documenting your reasons, here's an example:


// values.cpp
// compile with cl /clr values.cpp
using namespace System;

  public ref class Names abstract sealed {
  public:
    static initonly String^ First = "Tosh";
// disabling error for VS2005 and VS2008 compilers
// search http://blog.tosh.me/ for 'literal keyword bug' for details
#pragma warning( push )
#pragma warning( disable: 4693 )
    literal String^ Last = "Afanasiev";
#pragma warning( pop )
  };

int main() {
  Console::WriteLine( "Hi, my name is {0} {1}!", Names::First, Names::Last );

  return 0;
}

There you go, happy constants all round.

Monday, 30 January 2012

Copy directory structure MSBuild

When using MSBuild you may find your self wanting to copy some existing directory structure somewhere. If you have a directory structure, the output of a solution for example, like the following:

output_dir/
 - bin/
   - thing.dll
   - thing.pdb
 - lib/
   - thing.lib
 - inc/
   - thing.h

You'll notice that doing this:

<ItemGroup>
  <out_files Include='output_dir/**/*'/>
</ItemGroup>

<Target Name='copy_files'>
  <Copy SourceFiles='@(out_files)' DestinationFolder='deployment_dir'/>
</Target>

Will flatten your output file structure to this:

deployment_dir/
 - thing.dll
 - thing.pdb
 - thing.lib
 - thing.h

In order to preserve your file structure you could create separate specifications for the bin/, lib/ and inc/ directories (which would not only add a maintenance burden in the case that these directories changed, but also require you to type them out in the first place) or you could simply take advantage of the metadata that MSBuild attaches to the path information it returns from a recursive directory search. The RecursiveDir property is attached to each file name in the array yielded by the search; it holds the value of the recursive directory directive (**) as evaluated for each file found, so doing this:

<ItemGroup>
  <out_files Include='output_dir/**/*'/>
</ItemGroup>

<Target Name='copy_files'>
  <Copy SourceFiles='@(out_files)' DestinationFolder='deployment_dir/%(out_files.RecursiveDir)'/>
</Target>

Will give you the structure you were after:

deployment_dir/
 - bin/
   - thing.dll
   - thing.pdb
 - lib/
   - thing.lib
 - inc/
   - thing.h

You've got loads of control over what gets captured too, say you want to exclude debug symbols, you'd do this:

<ItemGroup>
  <out_files Include='output_dir/**/*' Exclude='output_dir/**/*.pdb'/>
</ItemGroup>

<Target Name='copy_files'>
  <Copy SourceFiles='@(out_files)' DestinationFolder='deployment_dir/%(out_files.RecursiveDir)'/>
</Target>

To get this:

deployment_dir/
 - bin/
   - thing.dll
 - lib/
   - thing.lib
 - inc/
   - thing.h

Using these constructs allows you to manage builds of multiple solutions from a central MSBuild project without requiring that that project know all the solutions' intimate details - it gives you good separation of concerns and hence fewer headaches.

Tuesday, 24 January 2012

Overloading new and delete in C++

While writing C++ you might want to be involved in the details of dynamically allocated memory yourself; to ensure that certain types of object sit together in memory; to minimise space wasted on overhead for arrays of small objects; to improve allocation speed (at the cost of memory held) or to satisfy a fixed memory requirement. Whatever the reason (and once the trade-offs are fully understood), the basic plumbing you would need looks something like this:
#include <iostream>

using std::cout;
using std::endl;

class Thing {
public:
  Thing() {
    cout << "new Thing: " << sizeof( Thing ) << " bytes" << endl;
  }
  ~Thing() {
    cout << "~Thing()" << endl;
  }
  static void* operator new( size_t size ) {
    cout << "allocating " << size << " bytes in operator new" << endl;

    return ::operator new( size );
  }
  static void operator delete( void* mem ) {
    cout << "operator delete" << endl;

    ::operator delete( mem );
  }
  static void* operator new[]( size_t size ) {
    cout << "allocating " << size << " bytes in operator new[]" << endl;

    return ::operator new[]( size );
  }
  static void operator delete[]( void* mem ) {
    cout << "operator delete[]" << endl;

    ::operator delete[]( mem );
  }
private:
  unsigned char value_;
  // not worrying about copy/move for this example
  Thing( const Thing& );
  Thing( Thing&& );
  void operator=( const Thing& );
  void operator=( Thing&& );
};

int main( int argc, char* argv[] ) {

  cout << "creating an object ..." << endl;
  auto* t = new Thing;

  delete t;

  const int SIZE = 5;
  cout << "creating a " << SIZE << " element array ..." << endl;
  auto* a = new Thing[SIZE];

  delete[] a;

  return 0;
}

In this example my overloaded operators defer to the default provided versions, so nothing really interesting is being done in terms of memory allocation, but it does serve as an illustration of the mechanism, as the output shows:
creating an object ...
allocating 1 bytes in operator new
new Thing: 1 bytes
~Thing()
operator delete
creating a 5 element array ...
allocating 9 bytes in operator new[]
new Thing: 1 bytes
new Thing: 1 bytes
new Thing: 1 bytes
new Thing: 1 bytes
new Thing: 1 bytes
~Thing()
~Thing()
~Thing()
~Thing()
~Thing()
operator delete[]
It also illustrates the importance of pairing up the scalar and vector forms of new and delete properly - the additional 4 bytes allocated for the array are for tracking the size of the array for deletion - using the wrong form of delete causes that information to be ignored/erroneously read, depending on which way round you get it wrong.