Marshalling data from unmanaged code

[jlkinchantilly]

I have a VB6 application that calls a method from a custom DLL that I think was written in C (or C++). Unfortunately, I dont have the original source code, so I cannot examine the method signature for date types etc.

Here is what worked in VB6:
Public Declare Function xyz_function Lib "xyz.dll" _
(ByVal x As Integer, ByVal y As Integer, refArray() As Byte) As Integer

Dim refData(0 To 31) As Byte
If xyz_function(CInt(0), CInt(2), refData()) = 0 Then
...
End If


After reading several articles, here is what I tried in VB.NET:
<DllImport("xyz.dll")> _
Public Shared Function xyz_function(ByVal x As Integer, ByVal y As Integer, _
ByVal refArray As IntPtr) As Integer
End Function

Dim refData(31) As Byte
refData.Initialize()
Dim dataLength As Integer = refData.Length + 1
Dim unmanagedMem As IntPtr = Marshal.AllocHGlobal(dataLength)
Marshal.Copy(refData, 0, unmanagedMem, dataLength)
If xyz_function(CShort(0), CShort(2), unmanagedMem) = 0 Then
Marshal.Copy(unmanagedMem, refData, 0, dataLength)
Marshal.FreeHGlobal(unmanagedMem)
...
End If

I dont receive any exceptions, however at the end of the operation, all slots of refData are zero. Actually after trying several different things, I discovered that if I replace refData.Initialize() with pre-initialization of the slots to random values, those values are what I get back after calling the external method. So evidently, the external method is not changing the values in the memory location.

My question is: Is there something obviously wrong with this code that would make it not work? Does anyone have a suggestion for how to get it to work keeping in mind that I have no way of obtaining the original source?

 

[PlausiblyDamp]

Try

Public Declare Function xyz_function Lib "xyz.dll" _
(ByVal x As Short, ByVal y As Short, refArray() As Byte) As Short

 

[jlkinchantilly]

I did change the Integers to Short - strange that the vb6 to .NET converter did not catch that - but it produced no change in the output from the dll.

 

[PlausiblyDamp]

After you changed the declare are you still using the Marshall class to pass information in and out? If so just try using an array of bytes like in the original VB6 version.

 

[jlkinchantilly]

That definitely does not work. The array of bytes comes back as one byte.

 

[PlausiblyDamp]

Try the following for a declaration

Public Declare Function xyz_function Lib "xyz.dll" (ByVal x As Short, ByVal y As Short, ByRef refArray() As Byte) As Short

and see if using a method body of

Dim refData(31) As Byte

If xyz_function(CShort(0), CShort(2), refData) = 0 Then
...
End If

works