How to get handle of menu in external application, and find button where no text is part of class.

[Galaxea]

@Jaafar Tribak - I have seen that you have extensive experience in this topic so I am hoping that you can help me please.

I have an external application that I can launch and control from Excel, but only up to a point.
As soon as I click on the SEND TO button and the menu pops up I can't go any further as I cannot get the handle or any other info.
Even bringing up the SPY++ tool will make the menu disappear as soon as the focus is changed off the menu.

Please contact me, even off the forum if necessary as I really need to get this working! I can't see a way to contact someone directly on this forum.

I have another similar issue with the latest version of the same program (I am in the process of upgrading to it) where I can't find enough info to know what to click to open a company file (see below).
Thanks. Rob.

 

[Galaxea]

I see that some of those "File" windows are not visible so maybe not worth trying to link to.
I also determined that there is a TAB in the "Open... Ctrl+O" menu, but I have tried all versions of that name and still cannot set that menu object. ("Open", "Open...", "Open...spaceCtrl+O","Open...tabCtrl+O", etc.)

What about using SendKeys to send "Ctrl+O" to the MenuBar? Really ugly solution but I am running out of hope here.

 

[Galaxea]

Last post for the day:

Back to the new version of the app I am automating:

I am trying to open the last company file that was opened - the first or second "data panel" row.
I can get down to the row quickly, but I cannot click/invoke/select it using the methods as per my code below.
Please advise some other things I can try. Thanks.

 

[John_w]

The Accelerator Key option really does seem like a great way to do what I need without traversing the menu, so if you have any ideas on how to implement this please do let me know.

I've successfully used InvokePattern.Invoke to 'press' the Accelerator Key or Access Key, first getting the related element. For example, in one of my macros I have:

    'Run Tools -> Partial Match using the Ctrl+P keys
    'Name:  "Partial Match"
    'ControlType:   UIA_ButtonControlTypeId (0xC350)

    Dim InvokePattern As IUIAutomationInvokePattern
    Dim AccelKey As IUIAutomationElement
    Dim PartialMatchCond As IUIAutomationCondition
   
    With UIAuto
        Set PartialMatchCond = .CreateAndCondition(.CreatePropertyCondition(UIA_NamePropertyId, "Partial Match"), _
                                                   .CreatePropertyCondition(UIA_ControlTypePropertyId, UIA_ButtonControlTypeId))
    End With

    Set AccelKey = Exp.FindFirst(TreeScope_Descendants, PartialMatchCond)
    Set InvokePattern = AccelKey.GetCurrentPattern(UIA_InvokePatternId)
    InvokePattern.Invoke
    DoEvents
    Sleep 100

You can also see below that the Ancestors have 3 different "File" objects. Should I not be traversing them all perhaps to get to the "Open" menu item?

Use TreeWalker to traverse the nodes. There's an example in the IE11 automation code.

 

[John_w]

I am trying to open the last company file that was opened - the first or second "data panel" row.
I can get down to the row quickly, but I cannot click/invoke/select it using the methods as per my code below.
Please advise some other things I can try. Thanks.

The element's LegacyIAccessible.DefaultAction is "Focus", and I guess Invoke does the same, so maybe you need to do one of those first and then something with the validationBlinkyColumn thing to open that file item.

More generally, I've done as much as possible to help you and pretty much exhausted the techniques that have worked for me with different applications. As you've found, a lot of it is trial and error and it's essential that you've found and referenced the correct element before doing anything with it. Make sure you refresh the Inspect tree regularly, as it can leave old elements in the tree which may mean you're looking at the wrong element.

 

[Galaxea]

Thanks for the above tips John.

Your line below highlights the issue that I currently have - I cannot get the element yet, so I cannot use this method.

Perhaps you could educate me on which "name" I need to use when referring to an element (see below).
Would it be the "Name" (EXIT) or the LegacyIAccessible one, and if the latter, how do I treat all the trailing spaces?

 

[John_w]

Perhaps you could educate me on which "name" I need to use when referring to an element (see below).
Would it be the "Name" (EXIT) or the LegacyIAccessible one, and if the latter, how do I treat all the trailing spaces?

Using the normal UIA properties you would look for the Name "Exit" and Control Type UIA_MenuItemControlTypeId.

Using the Legacy IAccessible properties you would look for the Name "Exit " and Role &HC (VBA syntax for hex 0xC). I'm not sure whether you need to specify the exact number of trailing spaces or none.

Here's how my macro finds and clicks the "File" menu using both methods:

    Dim UIAuto As IUIAutomation
    Dim Exp As IUIAutomationElement
    Dim FileMenuCond As IUIAutomationCondition
    Dim FileMenu As IUIAutomationElement
    Dim FileMenuLegacy As IUIAutomationElement
    Dim InvokePattern As IUIAutomationInvokePattern
    Dim LegacyPattern As IUIAutomationLegacyIAccessiblePattern

    '---- Code to define/get UIAuto and Exp objects removed ----


    'Find File menu using normal UIA properties
    'Name:          "File"
    'ControlType:   UIA_MenuItemControlTypeId (0xC35B)
    
    With UIAuto
        Set FileMenuCond = .CreateAndCondition(.CreatePropertyCondition(UIA_NamePropertyId, "File"), _
                                               .CreatePropertyCondition(UIA_ControlTypePropertyId, UIA_MenuItemControlTypeId))
    End With
    Set FileMenu = Exp.FindFirst(TreeScope_Descendants, FileMenuCond)
    
    'Click File menu
    
    Set InvokePattern = FileMenu.GetCurrentPattern(UIA_InvokePatternId)
    InvokePattern.Invoke
    DoEvents


    'Find File menu using Legacy IAccessible properties
    'LegacyIAccessible.Name:    "File"
    'LegacyIAccessible.Role:    menu item (0xC)
    
    'NOTE - With UIA_LegacyIAccessibleRolePropertyId, the value argument must be passed as CLng, otherwise Run-time error 5 'Invalid procedure call or argument' occurs
    
    With UIAuto
        Set FileMenuCond = .CreateAndCondition(.CreatePropertyCondition(UIA_LegacyIAccessibleNamePropertyId, "File"), _
                                               .CreatePropertyCondition(UIA_LegacyIAccessibleRolePropertyId, CLng(&HC)))
    End With
    Set FileMenuLegacy = Exp.FindFirst(TreeScope_Descendants, FileMenuCond)
        
    'Click File menu using Legacy pattern (possible because IsLegacyIAccessiblePatternAvailable: true)
  
    Set LegacyPattern = FileMenuLegacy.GetCurrentPattern(UIA_LegacyIAccessiblePatternId)
    FileMenuLegacy.SetFocus
    LegacyPattern.DoDefaultAction
    DoEvents

 

[Galaxea]

John, I have found a difference between what Inspect and Spy++ tells me and Spy++ looks more correct.

There is a popup menu that appears to select a customer, and I can't get to any of the items on that screen (like the Easy Add button).
Spy++ tells me that it is a top-level window off the desktop (which I believe), but Inspect tells me that it is a descendant of the main application
even though it doesn't appear in the left window pane when I hover over any elements.
The right window pane shows the descendants and I can't get to the buttons by following that chain.
pnlBottomPanel cannot be found when searching off any of the ancestors of the main application.

Have you come across this before?

 

[John_w]

Sorry, I've never used Spy++ to look at UI elements, always Inspect, so I don't know which one is correct.

One unexpected thing I noticed was with the IE11 automation, where the Save/Save as/Save and open dropdown is a child of the Desktop, not a child of the main IE window or any of its intermediate elements.

 

[Galaxea]

After some testing I can confirm that Spy++ was the more accurate one.
I can find and select the button that I need by searching for the top level Class name (with a partial match, as the full name is dynamic and changes every time).
From there I can press the button using UIA with the pattern.
I find that I have to use Spy++ together with UIA to get the full picture of what is going on.

Anyway, my current issue is using UIA to get to a grid/table where the elements are not available (programmatically) until the mouse-click happens on the first row, and even then only one edit box is available for that field clicked on, not for the rest, so I am using Sendkeys with the TAB key to get from one element to the next. Not a pretty solution but the only one that I have that works right now. Do you have any experience with objects like this, or suggestions? Thanks again.

 

[John_w]

No, I've never come across any objects like that and I can't think of anything that would help.