Subclassing without crashing - (Testers required.)

[Jaafar Tribak]

Hi dear excellers,

I have put together this project which subclasses userforms (can be extended to other windows) and I would like to know if it works as expected for trapping window messages. I am particularly interested to know if the project is stable enough. Meaning: It dosen't shuts down the entire excel application if an unhandled error occurs while the form(s) is (are) subclassed or when pressing the Break, Reset or the Design Mode buttons in the VBE ... Subclassing is notoriously limited in vba if not outright inoperational. Hopefully, with this technique, the crashings will be overcome.

Clicking the 'Raise Error' button on the first form or right-clicking its titlebar should cause an intentional error to help the testings.

Basically, the project makes use of two dlls that I have written and compiled in TwinBasic (one dll is for x32bit excel and the other one is for x64bit) ... I have embedded the dlls binary data in two modules as base64 strings, just like reources for portability reasons ... The code automatically takes care of everything from decoding the base64 strings to extracting the dll bytes and saving them to the temp directory as dll files.

Please, try resizing, moving the forms as well as right-clicking the form's titlebar to see the action.

Thanks.


File for download:
SubclassDll_ VBA_x32_x64.xlsm



Here is the event handler for the first form :

Private Sub oSubclass_MessageReceived( _
    hwnd As LongPtr, _
    uMsg As Long, _
    wParam As LongPtr, _
    lParam As LongPtr, _
    dwRefData As LongPtr, _
    bDiscardMessage As Boolean, _
    lReturnValue As LongPtr _
)
    Const WM_SETCURSOR = &H20, WM_GETMINMAXINFO = &H24, WM_CONTEXTMENU = &H7B
    Select Case uMsg
        Case WM_SETCURSOR      'Hover the mouse over the form.
            Debug.Print Me.Name, Format(Now, "hh:mm:ss")
        Case WM_GETMINMAXINFO  'Resize the form.
            Call CheckMinMaxInfo(lParam, bDiscardMessage)
        Case WM_CONTEXTMENU    'Right-click the form titlebar to raise error..
            Debug.Print 1 / 0  'Raising an error shows that excel doesn't crash (doesn't shut down) !!
    End Select
   
End Sub

For the second form :

Private Sub oSubclass_MessageReceived( _
    hwnd As LongPtr, _
    uMsg As Long, _
    wParam As LongPtr, _
    lParam As LongPtr, _
    dwRefData As LongPtr, _
    bDiscardMessage As Boolean, _
    lReturnValue As LongPtr _
)
   
    Const WM_SYSCOMMAND = &H112, SC_MOVE = &HF012&, WM_NCLBUTTONUP = &HA2
    Const WM_SETCURSOR = &H20, WM_NCMOUSELEAVE = &H2A2
    Select Case uMsg
        Case WM_SYSCOMMAND             'Move the form
            If wParam = SC_MOVE Then
                Label1 = "Sorry, this form is unmovable."
                bDiscardMessage = True 'abort message.
            End If
        Case WM_NCLBUTTONUP            'move the form
            Label1 = ""
        Case WM_SETCURSOR
            If GetAsyncKeyState(VBA.vbKeyLButton) = 0& Then
                Label1 = ""
            End If
         Case WM_NCMOUSELEAVE
            If GetParent(WndFromPoint(hwnd)) <> hwnd Then
               Label1 = "": Call MonitorMouse
            End If
    End Select
   
End Sub

 

[Jaafar Tribak]

Hi. This is great - try as I might, I could not get it to crash!

I'm on Windows 11.

Thanks for the feedback and glad it is crash proof as I had hoped.

 

[sancarn]

Interesting approach Jaafar. Though I'd think if you're already creating a DLL on disk, it'd be better to simply pass the VBA subclasser object directly to the DLL and handle all the hooking etc. within the DLL, and then use COM calls into the object? Pseudocode below:

'Subclasser from VBA side:
class CSubclasser
  Event MessageReceived( _
      hwnd As LongPtr, _
      uMsg As Long, _
      wParam As LongPtr, _
      lParam As LongPtr, _
      dwRefData As LongPtr, _
      bDiscardMessage As Boolean, _
      lReturnValue As LongPtr _
  )
  Sub protRaiseEvent(hwnd as LongPtr, uMsg as Long, wParam as LongPtr, lParam as LongPtr, dwRefData as LongPtr, bDiscardMessage as Boolean, lReturnValue  as LongPtr)
    RaiseEvent MessageReceived(...)
  End Sub
  Sub Class_Initialize
    Call buildDLL()
    Dim me_IDispatch as Object: set me_IDispatch = me
    Call registerSubclass(hWnd, me_IDispatch)
  End Sub
end class

'Subclasser from DLL code:
module bas_SubclasserDLL
  private dispatchObj as object
  Public Sub registerSubclass(ByVal hwnd as LongPtr, ByVal dispatchObj as Object)
    ...
  End Sub
  Function SubclassEntry(hwnd as LongPtr, uMsg as Long, wParam as LongPtr, lParam as LongPtr) as lReturnValue  as LongPtr
    Call dispatchObj.protRaiseEvent(...)
    ...
  End Function
end module

Maybe I'm wrong, but I wouldn't expect the above to crash, as long as calls are routed through COM

P.S. I couldn't crash either, but same system and version as Dan_W.

 

[Jaafar Tribak]

@sancarn

Though I'd think if you're already creating a DLL on disk, it'd be better to simply pass the VBA subclasser object directly to the DLL and handle all the hooking etc. within the DLL, and then use COM calls into the object? Pseudocode below:

Thanks for the interest, feedback and testing.

Unless I misunderstood you, I think I am basically doing just that. I am casting the subclasser object to the IEvent interface then passing the interface pointer to the dll. ( The reason I am using an interface is to keep the event raiser WndProc Private so it can't be seen by the subclasser client [ie:= by the calling form] in the vbe intellisense )

The heavy lifting hooking is all happening within the dll already.


In Red, is how I am passing the subclasser object:

'Section in the CSubclasser Class module inside the SubclassWnd Method:
        hLib = LoadDll(sDllOutputPath)
        If hLib Then
            If Not IsWndSubclassed(hwnd, dwRefData) Then
                Set Interface = Me
                uIdSubclass = ObjPtr(Interface)
                hProcAddr = GetProcAddress(hLib, "SubclassWnd")
                If hProcAddr Then
                    If bas_API.DllStdCall(hProcAddr, vbBoolean, hwnd, uIdSubclass, dwRefData) Then
                        SubclassWnd = SetProp(hwnd, "Sunbclassed", True)
                    End If
                End If
            End If
        End If

 

[sancarn]

@sancarn


Thanks for the interest, feedback and testing.

Unless I misunderstood you, I think I am basically doing just that. I am casting the subclasser object to the IEvent interface then passing the interface pointer to the dll. ( The reason I am using an interface is to keep the event raiser WndProc Private so it can't be seen by the subclasser client [ie:= by the calling form] in the vbe intellisense )

View attachment 117644

The heavy lifting hooking is all happening within the dll already.


In bold, is how I am passing the subclasser object:

'In the CSubclasser Class module inside the SubclassWnd Method:

hLib = LoadDll(sDllOutputPath)
        If hLib Then
            If Not IsWndSubclassed(hwnd, dwRefData) Then
                Set Interface = Me
                uIdSubclass = ObjPtr(Interface)
                hProcAddr = GetProcAddress(hLib, "SubclassWnd")
                If hProcAddr Then
                    If bas_API.DllStdCall(hProcAddr, vbBoolean, hwnd, uIdSubclass, dwRefData) Then
                        SubclassWnd = SetProp(hwnd, "Sunbclassed", True)
                    End If
                End If
            End If
        End If

Ahh I see RE: IEvent fair.

I think what through me off was the following:

lHook = SetWindowsHookEx(WH_CBT, AddressOf HookProc, NULL_PTR, GetCurrentThreadId)
...

Seemed like this could be code which is handled in the DLL, but perhaps not.

Also... Why are you redirecting Sleep? Doesn't look like sleep is used at all in the project...? Is this protection against a potential crash vector?

 

[Jaafar Tribak]

Ahh I see RE: IEvent fair.

I think what through me off was the following:

lHook = SetWindowsHookEx(WH_CBT, AddressOf HookProc, NULL_PTR, GetCurrentThreadId)
...

Seemed like this could be code which is handled in the DLL, but perhaps not.

No. That HookProc must run before the dll is loaded.

 

[sancarn]

No. That HookProc must run before the dll is loaded.

Alas, that is unfortunate.