I’ve been interfacing a third-party library for a project at work. The third party library is a DLL with a C interface. Now most libraries for Windows usually have a __stdcall calling conventions and for some reason I assumed that would be case with the C library. I guess it was because the demo application had not done anything to give a hint otherwise. Since the client component was going to be in .net component, I wrote a nice little C++/CLI wrapper around the unmanaged native library, so that I could easily call it from C#. I decided against using P/Invoke because I don’t like using meta-information in source code to identify implementation of interfaces. In my opinion using P/Invoke client code too dependent on a specific library. That’s a topic for another post.
One of the functions of the library needed a function pointer to a callback function. The prototype of the function was something like this, wrapped with linkage.
<span style="color:blue;">extern </span><span style="color:#a31515;">"C"
</span>{
<span style="color:blue;">typedef void </span>(*SomeCallbackType)(<span style="color:blue;">int</span>, <span style="color:blue;">char</span>*, <span style="color:blue;">float</span>);
<span style="color:blue;">void </span>SetCallBackInNativeFunction(SomeCallbackType p){<span style="color:green;">/*...*/</span>}
}
Since I wanted the callback mechanism to hook up to .Net delegates I did something as follows,
<span style="color:blue;">using namespace </span>System;
<span style="color:blue;">using namespace </span>System::Runtime::InteropServices;
<span style="color:blue;">delegate void </span>FooDelegate(<span style="color:blue;">int </span>arg1, <span style="color:blue;">char</span>* arg2, <span style="color:blue;">float </span>arg3);
<span style="color:blue;">ref class </span>AClass
{
<span style="color:green;">///...
</span><span style="color:blue;">public</span>:
<span style="color:blue;">void </span>FooCallback(<span style="color:blue;">int </span>arg1, <span style="color:blue;">char </span>*arg2, <span style="color:blue;">float </span>arg3){<span style="color:green;">/*...*/</span>}
<span style="color:green;">///..
</span>};
<span style="color:blue;">static </span>GCHandle globalGCHandle;
<span style="color:blue;">void </span>SetCallback()
{
AClass ^theObj = <span style="color:blue;">gcnew </span>AClass();
FooDelegate ^del = <span style="color:blue;">gcnew </span>FooDelegate(theObj, &AClass::FooCallback);
<span style="color:green;">//We want to do this because the callback will be called many times.
//and may live through many garbage collection sprees.
//This would be freed at the end of the program.
</span>globalGCHandle = GCHandle::Alloc(del);
IntPtr ptr2Func = Marshal::GetFunctionPointerForDelegate(del);
SetCallBackInNativeFunction((SomeCallbackType)ptr2Func.ToPointer());
}
Now every time time I ran my program, it would start up normally, but then would mysteriously crash with really strange return values like 0xC00005, 0xC0000013, etc. I recognized these return codes that indicated access violation and stack corruption exceptions. I turned on exception handlers in the debugger but the debugger refused to catch those exceptions. I was mystified. I thought that the exceptions were happening because the garbage collector was moving around my delegate, which is why I used the globalGCHandle = GCHandle::Alloc(del). This didn’t help. Turned out the program was crashing because Marshal::GetFunctionPointerForDelegate returns a function pointer with __stdcall calling convention. Now as everyone knows __stdcall and __cdecl (the default in C compilers) are incompatible, and it causes really confusing errors because the stack always ends up corrupted. The compiler (at least VC compiler) would have helped had I been passing a native function pointer. With managed code, the compiler happily let me shoot myself in the foot. The solution was to add the following attribute to the delegate declaration
[UnmanagedFunctionPointer(CallingConvention::Cdecl)] <span style="color:blue;">delegate void </span>FooDelegate(<span style="color:blue;">int </span>arg1, <span style="color:blue;">char</span>* arg2, <span style="color:blue;">float </span>arg3);
which I got from here
The program worked flawlessly after that (I mean it did what I wanted it to do).

Posted by bybitsandbytes
Posted by bybitsandbytes