How to get handle of menu in external application, and find button where no text is part of class.

[Galaxea]

@Jaafar Tribak - I have seen that you have extensive experience in this topic so I am hoping that you can help me please.

I have an external application that I can launch and control from Excel, but only up to a point.
As soon as I click on the SEND TO button and the menu pops up I can't go any further as I cannot get the handle or any other info.
Even bringing up the SPY++ tool will make the menu disappear as soon as the focus is changed off the menu.

Please contact me, even off the forum if necessary as I really need to get this working! I can't see a way to contact someone directly on this forum.

I have another similar issue with the latest version of the same program (I am in the process of upgrading to it) where I can't find enough info to know what to click to open a company file (see below).
Thanks. Rob.

 

[John_w]

Microsoft Inspect Tool is better than Spy++ for analysing UI controls and elements.

See this page for installing Inspect:

How to install the Inspect tool on Windows 10?

Is Inspect a part of the Windows 10 standalone SDK? Is there any alternative application to "inspect" for windows 10? I want to install "inspect" tool on my machine running windows 10 build 10586 ...

stackoverflow.com

You could control the external application from Excel with UIAutomation. Example code in these threads:

Using UIAutomation to automate the Windows 10 Calculator

As a learning exercise, I decided to write VBA code to automate the Windows 10 Calculator using the UIAutomation library, after seeing this thread which controls the old Windows 7 Calc.exe program...

www.mrexcel.com

Using UIAutomationClient to automate the Save As file download in IE11

Here is VBA code which automates the complete 'Save As' file download procedure in IE11, when a download is offered by IE's Download Notification Bar. To use for your own web site downloads you would automate IE and do whatever is necessary to make the Download Notification Bar appear at the...

www.mrexcel.com

 

[Jaafar Tribak]

Hi Rob,

I second John's siggestion to explore Inspect.exe... SPY++ will only detect UI elements that have a hwnd whereas Inspect.exe can detect (and act upon to a certain extent) some windowless user interface elements such as menus. This is done via the IAccessibility and IUIAutomation interfaces as long as the server application implements those interfaces.

It is also worth noting that sometimes it is difficult to access some UI elements which are only created after a user action ha been performed ( like the 3 submenus that appear in the dropdown after clicking the SendTo menu shown on your first image shot). In such cases, the client code will need to setup a windows or accessibility hook or a windows timer beforehand in order to detect the creation of the submenus.

 

[Galaxea]

Thank you both for the suggestions. I am not familiar with UIAutomation so it will take some time and reading for me to see if I understand it.
If you have any examples that might be simple and relevant to the issue I am trying to resolve, please post them to help me speed up the process.

If I posted some pics of what INSPECT is showing me for those windows would it help you to tell whether it could solve my problem?

 

[Galaxea]

This is what I am seeing. I don't have a clue how to now navigate to this UI element and click it.
It looks like the INSPECT tool has been replaced by this new version. Does this screen give you enough info to guide me on what to do next?
It looks like it can get the focus which is good, but how do I click on it with UIAutomation please?

 

[John_w]

This is what I am seeing. I don't have a clue how to now navigate to this UI element and click it.
It looks like the INSPECT tool has been replaced by this new version. Does this screen give you enough info to guide me on what to do next?
It looks like it can get the focus which is good, but how do I click on it with UIAutomation please?

Inspect has been replaced by Accessibility Insights, however I prefer Inspect and find it easier to use.

It will be difficult for me to help you to automate your application using UIAutomation because I don't have the app, so I can only suggest starting with the Calculator code as an example.

Start by finding the main app window:

    'Find Calculator window on Desktop
    
    Set Desktop = UIAuto.GetRootElement
    Set CalcWindow = Desktop.FindFirst(TreeScope_Children, _
                                       UIAuto.CreateAndCondition(UIAuto.CreatePropertyCondition(UIA_ControlTypePropertyId, UIA_WindowControlTypeId), UIAuto.CreatePropertyCondition(UIA_NamePropertyId, "Calculator")))

that gives you a handle to the app's window, which you use to get the app's automation element:

    'Get the Calculator automation element from its window handle
    
    Set UIAuto = New CUIAutomation
    Set Calc = UIAuto.ElementFromHandle(ByVal CalcHwnd)

From there, to find a specific element use a combination of CreatePropertyCondition, CreateAndCondtion and FindFirst with TreeScope_Descendants or TreeScope_Children. Typically, you use CreateAndCondition to find an element by its ControlType and Name - see Calculator code.

To click an element, use its InvokePattern - see Calculator code.

 

[Galaxea]

Inspect has been replaced by Accessibility Insights, however I prefer Inspect and find it easier to use.

It will be difficult for me to help you to automate your application using UIAutomation because I don't have the app, so I can only suggest starting with the Calculator code as an example.

To click an element, use its InvokePattern - see Calculator code.

Thanks John. I have the Calculator code working so I will go through that and your suggestions above and see how far I can get.

 

[Galaxea]

I have downloaded the Inspect tool (old one) as suggested.
I am trying to get to the "Send To" - "Disk" option (see OP above).

I can see that the popup menu is not a child of the "Send To" button, but rather a top level window (green arrow).
Do you still think it is possible to find this element using UIAutomation (or standard APIs)?
Is there anything in the right-hand window below that I can use to get to it?
Please suggest how to go about it. Thanks!

 

[Galaxea]

This is what I can see with the Inspect tool for the newer version of the app:

 

[John_w]

I can see that the popup menu is not a child of the "Send To" button, but rather a top level window (green arrow).
Do you still think it is possible to find this element using UIAutomation (or standard APIs)?
Is there anything in the right-hand window below that I can use to get to it?
Please suggest how to go about it. Thanks!

Search for "Context menu" in the IE11 automation code - it has the same situation as your popup menu, whereby it is a child of the Desktop rather than a child of the IE window as one might expect.

The answer is to search the Desktop root element, something like this for your case to get the 3rd menu item (I haven't shown the lines which define/get UIAutomation and DesktopRoot):

    Dim UIAutomation As IUIAutomation
    Dim NameAndType As IUIAutomationCondition
    Dim ControlType As IUIAutomationCondition
    Dim ContextMenu As IUIAutomationElement
    Dim MenuItem3 As IUIAutomationElement
     
    With UIAutomation
        Set NameAndType = .CreateAndCondition(.CreatePropertyCondition(UIA_NamePropertyId, "Context"), _
                                              .CreatePropertyCondition(UIA_ControlTypePropertyId, UIA_MenuControlTypeId))
    End With

    Set ContextMenu = DesktopRoot.FindFirst(TreeScope_Children, NameAndType)
    
    Set ControlType = UIAutomation.CreatePropertyCondition(UIA_ControlTypePropertyId, UIA_MenuItemControlTypeId)
    Set MenuItems = ContextMenu.FindAll(TreeScope_Children, ControlType)
   
    Set MenuItem3 = MenuItems.GetElement(3)  'the 3rd menu item