VSoft Technologies Blogs

Introducing Delphi Mocks

Tue, 06 Nov 2012 15:36:53 GMT

Delphi has had Unit Testing support (in the form of DUnit) for many years, but until now there very little in the way of Automatic Mocking. By contrast the .NET and Java worlds have plenty of mocking frameworks to choose from.

So what are Automatic Mocks anyway? Simply put, they are mock objects that you don't have to hand code. What's wrong with hand coding mock objects you might ask? Well nothing, really, but consider the following example, where we pass an instance of our hand coded mock object to the object under test.

[code:delphi] type IFoo = interface function Bar(param : integer) : string; end; procedure TTestSomething.SimpleTest; var something : ISomething; mockFoo : IFoo; begin something := TSomething.Create; mockFoo := TMockFoo.Create; something.UseFoo(mockFoo); end; [/code]

What does our mockFoo mock object actually do, what is it's behavior, how many times will Bar be called and what will it return? It's not obvious from looking at the above code, so you would have to go off and find the TMockFoo

implementation. Not so hard, but what happens when someone else comes along and adds a new unit test that also uses TMockFoo, what happens if they change the TMockFoo implementation, will that break your existing tests. This is where automatic
mocks can help, because you define the behavior and expectations on you mock object right there in the unit test method :

[code:delphi] procedure TTestSomething.SimpleTest; var something : ISomething; mockFoo : TInterfaceMock begin something := TSomething.Create; //create our auto mock mockFoo := TInterfaceMock.Create; //define the behavior of IFoo.Bar mockFoo.Setup.WillReturn('hello something').When.Bar(1); //define our expectations of how many times TSomething will call IFoo.Bar mockFoo.Setup.Expect.Once.When.Bar(1); //now lets test ISomething.UseFoo something.UseFoo(mockFoo); //Now verify that ISomething used IFoo correctly mockFoo.Verify; end; [/code]

The auto mock object is defined right there in our unit test, and the behavior is not going to change unless we change it in our unit test. This sort of mocking makes it simple to focus our testing effort on TSomething rather than TFoo (our real IFoo implementation) or TMockFoo.

One of the reasons auto mocking hasn't really happened for Delphi is the lack of detailed and easy to use runtime type information (Rtti) and the lack of runtime code generation (yes it's always been possible, but not through a well defined interface like Reflection.Emit in .NET). Auto mocking involves creating types on the fly, in the case of interfaces, creating a type and implementing an interface. That's not an easy thing to do. There are some examples of this in Delphi's SOAP code, but they are not easy to follow. Delphi XE2 introduces some new features in the RTTI that make creating interface proxies simple. TVirtualInterface creates an implementation of an interface at runtime and marshals the method calls to the OnInvoke method.

Delphi-Mocks - https://github.com/VSoftTechnologies/Delphi-Mocks is an attempt to create an Auto Mocking framework for Delphi. It makes use of Generics and Fluent style interfaces. Currently only interfaces can be mocked, and it supports only Delphi XE2 at this time. I hope to add support for TObject mocks and earlier versions (D2009+).

The quest for the perfect publish/subscribe mechanism in Delphi

Thu, 03 Jun 2010 14:00:00 GMT

FinalBuilder's IDE is message driven. When you select an action in an actionlist for example, the actionlist view publishes a message to which other parts of the IDE can subscribe. The publish/subscribe mechanism used in the IDE has gone through a few revisions over the years, in the quest for the perfect publish/subscribe mechanism...

Of course when I say "perfect", what I'm really after is the architecture that is the easiest to use, and more importantly, the easiest to maintain. In earlier versions of the IDE, we used simple interfaces with an event args object that can carry various payloads :

 
type
IEventArgs = interface
  function EventID : integer;
  ..
  property IntfData : IInterface..
  property StringData : string...
  property IntData : integer..
  ...
end;
ISubscriber = interface
  procedure OnIDEEvent(const AEventArgs : IEventArgs);
end;

This has worked well for a few revisions, but with FinalBuilder 7 we have a new IDE, the number of messages being published has doubled, and code was starting to look pretty ugly. Each subcriber's OnIDEEvent method was one long case statement... and we were always having to refer back to where the message was sent from to confirm exactly what the payload was. Not at all scalable. So I started looking for new ideas. In a c# app that a colleague here wrote, he used interfaces with generics to create a publish/subscribe mechanism :

 
public interface IConsumer { }
public interface IConsumerOf : IConsumer where T : IMessage
{
  void Consume(T message);
}

Well, Delphi 2010 has generics support and I'm already making extensive use of them (hard to imagine now how I did without them!), so I figured I'd see if I could use the same technique in delphi :

 
IMessage = interface
['{BCBD228C-F184-461F-B4EE-2FC7A757C0AC}']
end;

ISubScriber = interface
['{B0D49727-272F-4D51-98B3-AA6E0708DD44}']
end;

//note the constraint so only message objects can be published
ISubScriberOf = interface(IConsumer)
//No guid for generic interfaces!
  procedure Consume(const message : T);
end;

TMyMessage = class(TIntefacedObject, IMessage)
end;
 
TMyOtherMessage = class(TIntefacedObject, IMessage)
end;
 
  
TMySubscriber = class(TIntefacedObject,ISubScriberOf, ISubScriberOf)
protected
  procedure Consume(const message : TMyMessage);overload;
  procedure Consume(const message : TMyOtherMessage);overload;
  ....
end;

Works perfectly.... except now I have a bunch of methods named Consume... and my class needs to implement an interface per message. According to my well thumbed copy of Delphi in a Nutshell, a class can implement up to 9999 interfaces so that's not a problem, but, it's just not quite as neat as I'd hoped. It worked well in the c# app as there were only a few message types. Most of my subscribers are handling 20+ messages, navigating 20+ Consume methods isn't all that maintainable.

Back to looking at the mother of all case statements in my existing architecture, it reminded me of a WndProc method and that got me thinking. How do message handlers work in the VCL? You know, this sort of thing :

 
procedure WMActivate(var Message: TWMActivate); message WM_ACTIVATE;

I had always just assumed it was compiler magic... (back to Delphi in a Nutshell again), well they are dynamic methods, which are invoked via TObject.Dispatch. This method takes a message record, and based on the message id, finds the matching method in the object's dynamic method table (which is compiler generated), if not found then it looks up the class heirachy and then eventually calls DefaultHandler if no match was found. Delphi's windows controls use this mechansim to dispatch windows messages, but there's really nothing windows specific about it and it looks like it could be used for any messages. The key is that the messages must be records because TObject.Dispatch treats them as such, and it looks at the first 4 bytes (DWoRD) as the message id. If you look in Messages.pas you will see that most messages are typically 16 bytes long, and they are packed records. In my initial tests everything worked fine just keeping the first 4 bytes for the message id, however in my IDE strange things happened (random av's). It turns out the messages really do need to be at least 16 bytes. So my message types look like this :

 
type
  TMyMessage = packed record
    MsgID : Cardinal;
    Unused : array[1..12] of Byte;
    MyPayload : whatever;
    .....
    constructor Create(APayload : whatever);
  end;

Note that I use a constructor on the record. Constructors on records are really just psuedo constructors.. but they serve a purpose here.. this is where I make sure the message gets assigned the correct message id (from a constant). Without the constructor we would have to assign it to the message before it is sent.. that opens the possibility of used the wrong id by mistake, which could cause random access violations :

 
constructor TMyMessage.Create(APayload : whatever);
begin
  MsgID := IDE_MYMESSAGE; //constant 
  MyPayLoad := APayload;
end;

One caveat with constructors on records is that they must have at least one parameter, so if your message has no payload then just use a dummy parameter. Our Subscriber interface now looks like this :

 
type
  ISubscriber = interface
    procedure Dispatch(var Message);
  end;

The Dispatch message on the interface is declared the same as TObject.Dispatch, and TObject.Dispatch is our actual implementation on our subscriber class so we don't need to do much other than declare that it implements the interface, and then provide the message handlers :

 
TMySubscriber = class(TInterfacedObject,ISubscriber,...)
protected
  procedure DoMyMessage(var Message : TMyMessage);message IDE_MYMESSAGE;
  ....
end;

Our publisher interface is quite simple too :

 
IPublisher = inteface
  procedure SendMessage(var Message);
  procedure Subscribe(const subscriber : ISubscriber; const filter : integer = 0);
  procedure UnSubscribe(const subscriber : ISubscriber);
end;

The SendMessage method takes an untype var parameter (just like Dispatch), so any message type can be passed to it. The Subscribe method has an extra parameter, filter, which allows you to specify which messages a subscriber is interested in. If it's not specified then all messages will be passed to the subscribers Dispatch method. The filter isn't strictly neccessary, but it does provide an opportunity for a small optimisation if the subscriber really only handles a few messages.

So is this the "perfect" publish/subscribe mechanism for delphi? Probably not. It's kinda neat how it's using something that's been there in TObject probably back as far as Delphi 1 (I will have to dig out my D1 source disk and have a look!). We spent a day replacing our old mechanism with this one throughout the entire FB7 IDE, and it's performing flawlessly. I've read that dynamic methods are a bit slower than non dynamic methods.. but to be honest we haven't noticed any change in performance.. what we have noticed is how much easier our code is to read and maintain.

A sample D2010 app with full source is available here.

WeakRefence in Delphi - solving circular interface references

Fri, 26 Mar 2010 14:00:00 GMT

Using interfaces and reference counting in Delphi works great for the most part. Its a feature I use a lot, I'm a big fan of using interfaces to tightly control what parts of a class a consumer has access to. But, there is one big achillies heel with reference counting in Delphi, you cannot keep circular references, at least not easily, without causing memory leaks.

Consider this trivial example :

 
IChild = interface;
IParent = interface
['{62DC70E1-8D82-4012-BF01-452EB0F7F45A}']
  procedure AddChild(const AChild : IChild);
end;

IChild = interface
['{E1DB1DA0-55D6-408E-8143-072CA433412D}']
end;

TParent = class( TInterfacedObject, IParent )
private
  FChild : IChild;
  procedure AddChild(const AChild : IChild);
public
  destructor Destroy; override;
end;

TChild = class( TInterfacedObject, IChild )
private
  FParent : IParent;
public
  constructor Create( AParent : IParent );
  destructor Destroy; override;
end;

implementation
	
constructor TChild.Create(AParent: IParent);
begin
  inherited Create;
  FParent := AParent;
  AParent.AddChild(Self);
end;

destructor TChild.Destroy;
begin
  FParent := nil;
  inherited;
end;
	
procedure TParent.AddChild(const AChild: IChild);
begin
  FChild := AChild;
end;

destructor TParent.Destroy;
begin
  if Assigned( FChild ) then
    FChild := nil;
  inherited;
end;

procedure Test;
var 
  MyParent : IParent;
  MyChild : IChild;
begin
  MyParent := TParent.Create;
  MyChild := TChild.Create(MyParent);
  MyChild := nil;
  MyParent := nil;
end;

Both parent and child are now orphaned and we have no reference to them and no way to free them! Ideally, the parent would control the life of the child, but the child would not control the life parent.

So how can we get around this? Well a technique that I have used a lot in the past is to not hold a reference to the parent in the child, but rather just a pointer to the parent.

 
TChild = class(TInterfacedObject,IChild) 
private 
  FParent : Pointer; 
  ... 
end;

constructor TChild.Create(AParent : IParent); 
begin 
	FParent := Pointer(AParent); 
end; 
function TChild.GetParent : IParent; 
begin 
  result := IParent(FParent); // if the parent has been released the we are passing out a bad reference! 
                              // a nil reference would be preferable as it's easy to check. 
end;

This works well for the most part, but it does have the potential for access voilations if you do not understand or at least know how the child is referencing the parent.

For example :

 
var child : IChild; 
parent : IParent 
begin 
  parent := TParent.Create; 
  child := TChild.Create(parent): 
  parent := nil; //parent will now be freed, since nothing has a reference to it.
  ....... 
  parent := child.GetParent; //kaboom 
end;

One of my collegues kindly pointed out that C# doesn't suffer from this problem and he uses circular references all the time without even thinking about it. While discussing this, he mentioned the WeakReference class in .NET. It basically allows you to hold a reference to a object without affecting it's lifecycle (ie, not influencing when it will be garbage collected). I figured there must be a way to do this in Delphi, and so set about creating a WeakReference class for Delphi.

I wasn't able to find a reliable way to do this with any old TInterfacedObject descendant, however by creating a TWeakReferencedObject class and the use of generics on Delphi 2010 I did manage to implement something that works well and is not too cumbersome. Lets take a look at our Child/Parent example using a weak reference.

The important part in this is the use of the WeakReference to the parent in the Child class. So instead of declaring

FParent : IParent;

we have

FParent : IWeakReference<IParent>;

We create it using

FParent := TWeakReference<IParent>.Create(parent); //value is an IParent instance

This is how our TChild.GetParent /SetParent methods look now :

 
function TChild.GetParent: IParent; 
begin 
  if FParent <> nil then 
    result := FParent.Data as IParent 
  else 
    result := nil; 
end; 

procedure TChild.SetParent(const value: IParent); 
begin 
  if (FParent <> nil) and FParent.IsAlive then 
    FParent.Data.RemoveChild(Self); 
  FParent := nil; 
  if value <> nil then 
    FParent := TWeakReference<IParent>.Create(value); 
end;

Note the use of the IsAlive property on our weak reference, this tells us whether the referenced object is still available, and provides a safe way to get a concrete reference to the parent.

I still think this is something that could be solved in a better way by the delphi compiler/vcl guys n girls.

Hopefully someone will find this useful, the code is available for download here - Updated Sunday 28/3/2010

Feedback welcolme, I'm about to start making extensive use of this code, so if you see any holes then please do let me know!

Delphi: Workaround for TThread Synchronize/WaitFor Deadlock

Wed, 23 Apr 2008 14:00:00 GMT

FinalBuilder is written in a combination of Delphi and C#. This is a blog post about a Delphi RTL threading issue that I recently ran into and had to work around.

We recently had some bug reports of FinalBuilder deadlocking at shutdown under certain circumstances. I could reproduce the problem, but I was baffled for a long time because - according to the call stacks - the program should have been fine.

The issue I'd run into has been around for years, and is summed up nicely in this Borland QC note: Synchronize and WaitFor methods of TThread can lead to a DeadLock in logically correct applications.

The Problem

The simple summary is: If you ever have a call stack which includes both Synchronize and WaitFor, and the thread you are waiting on also calls Synchronize at some point, you will occasionally get a deadlock.

Explaining why is kind of tricky, but if you read the QC case you'll get the gist of it. The problematic sequence of events looks like this:

Thread A calls Synchronize(MethodA)

Thread B calls Synchronize(MethodB)

Then, inside the context of the Main Thread:

Main thread calls CheckSynchronize() while processing messages
CheckSynchronize is implemented to batch-process all waiting calls(*). So it picks up the queue of waiting calls (containing MethodA and MethodB) and loops through them one by one.
MethodA executes in the main thread's context. Assume MethodA calls ThreadB.WaitFor
WaitFor calls CheckSynchronize to process any waiting calls to Synchronize
In theory, this should then process ThreadB's Synchronize(MethodB), allowing Thread B to complete.
However, MethodB is already a possession of the first CheckSynchronize call, so it never gets called.

DEADLOCK!

The Solution

The ideal solution is to refactor your program so it will never call CheckSynchronize or WaitFor from inside an already-synchronized method call. For FinalBuilder, this was not a good option - all it takes is one Synchronized method call to call ProcessMessages and you're letting in dozens of potential deadlocks.

The solution I came up with is to write a "SafeSynchronize" method that only allows one thread at a time to get on the Synchronize method queue. This comes with a performance hit, but prevents the deadlock.

Here's a new class which implements the SafeSynchronize method:

  TSafeSyncThread = class(TThread)
  private
    procedure SafeSynchronizeInner;
  protected
    procedure SafeSynchronize(const AThreadMethod: TThreadMethod);
  end;

implementation

{ TSafeSyncThread }

var
  FSingleSynchronize : TCriticalSection;
  FSingleSynchronizeTarget : TThreadMethod;

procedure TSafeSyncThread.SafeSynchronize(const AThreadMethod: TThreadMethod);
begin
  FSingleSynchronize.Acquire;
  FSingleSynchronizeTarget := AThreadMethod;
  Synchronize(SafeSynchronizeInner);
end;

procedure TSafeSyncThread.SafeSynchronizeInner;
var
  targetMethod : TThreadMethod;
begin
  targetMethod := FSingleSynchronizeTarget;
  FSingleSynchronize.Release; // Now we're inside a CheckSynchronize call, it's safe to let another thread join the queue
  targetMethod;
end;

Basically, this serializes access to the Synchronize queue. The performance hit is potentially quite large if you have a lot of concurrent SafeSynchronize calls and a busy main thread. FinalBuilder is designed in a way which minimizes this, so it's less of an issue for us.

I put together a quick test project which demonstrates the issue (at least on Delphi 2007.) You can get it here if you want to have a look. It also contains SafeSyncThread.pas if you want to incorporate the workaround into your own projects. This code comes with no guarantees and no warranty, obviously.

If anyone has a better (preferably faster) solution to this problem, which doesn't require rewriting the RTL, please let us know. :).

PS The test project example contains some fairly awful cut-and-paste code reproduction, which - for the record - I normally avoid like the plague. I used it here because the methods are very simple, and I couldn't see an easy way to rearchitect it.

(*) FWIW, I actually quite like the way that CheckSynchronize is implemented, using an InterlockedExchange to swap the old queue object with a brand new one. That way you can process the entire old queue at once, without granular locking. Check the source code if you want to see what I mean. It's just a shame about the deadlock. (back)