Window Dissection Technology

Overview

AutoMate's Window Dissection™ technology can be used to precisely identify any window by effectively "dissecting" the objects and controls that are inside it. Windows are frequently identified by their title, however, in certain occasions, this may not be enough. If, for example, there are several open windows with the same title, it is necessary to specify additional criteria to identify one window from another. Window Dissection alleviates such an issue by allowing the specification of a window based on its class and handle as well as any child objects, controls, or text inside it. Multiple objects may be specified which, when combined together, form a unique window on the system.

Where Window Dissection is Utilized

Window Dissection can be applied to the Window trigger, an object used to start a task upon occurrence of a window-related event (i.e. window opens, closes, is focused or loses focus). It can also be found in all activities that takes a window as a parameter, such as all Window actions and several conditional activities , as well as activities that enable interactive automation to be performed on a UI  interactive  within a number of interactive actions used for extracting or setting data, manipulating controls and directly interacting with other applications, such as Press, Move mouse to object or Get text. The available Window Dissection parameters may vary depending on the activity. The Window Dissection parameters shown below appear in the properties of the Move mouse to object activity.

Window Trigger & Window Actions

In addition to the Window Trigger, the following window related actions contain Window Dissection parameters.

Minimize Window
Maximize Window

Restore Window

Focus Window

Close Window

Hide Window

Unhide Window

Resize Window

Move Window

Wait for a Window

If Window Exists

Loop Windows

Interactivity Actions

The following are interactive actions that contain the Windows Dissection parameters. These actions are found in the Task Builder's Available Actions pane under the Interactivity folder.

Move Mouse to Object

Get Text
Set Text

Check
Press

Select Item

Get Selected List Item

Get Selected Tree Item

Select List Item

Select Tree Item

Accessibility engines

Window Dissection actually uses a variety of combined technologies designed to provide a single reliable source of UI information. This enables automated tasks the ability to programmatically gather accurate information about a user interface in order to interact with it, whether it be Windows, Java or browser-based.

Microsoft Active Accessibility

Microsoft Active Accessibility is a technology that provides a standard, consistent mechanism for any window control, module, or application to expose information about its user interface. For example, Active Accessibility provides these individual pieces of information:

Java Access Bridge

In order for existing assistive technologies available on Microsoft Windows systems to provide access to Java applications, they need some way to communicate with Java Accessibility Utilities. Java Access Bridge supports this communication. Java Access Bridge is a technology that enables certain Java applications and applets to be visible to assistive technologies on Microsoft Windows systems. An assistive technology application running on Microsoft Windows (for example a screen reader) communicates with Java Access Bridge DLLs, which in turn communicates with the Java Virtual Machine through Java Access Bridge Java libraries. These Java libraries communicate with Java Accessibility Utilities. Java Accessibility Utilities collects information about what is happening in the Java application, which it forwards to the screen reader through Java Access Bridge.

Internet Explorer DOM

DOM, short for Document Object Model, is the specification for how objects in a web page (i.e. text, images, headers, links, etc.) are represented.

Setting Window Dissection properties

Window Dissection preferences are accessible from the action editor of qualifying actions/activities under the Properties tab.

Parameters

Window Dissection parameters are most easily set by dragging the provided magnifier icon from the properties dialog box and releasing it over the desired window/object. This populates available parameters with the proper specifications. A green border indicates successful detection.

Property

Type

Required

Default

Markup

Description

By window title

Text

No

(Empty)

WINDOWTITLE="theTitle"

If enabled, the window title will be used in the automation process.

By window class

Text

No

(Empty)

WINDOWCLASS="theClass"

If enabled, the window class will be used in the automation process. A window class is a set of attributes that the system uses as a template to create a window. Every window is a member of a window class. All window classes are process specific.

By window handle

Number

No

(Empty)

WINDOWHANDLE="number"

If enabled, the window handle will be used in the automation process. A window handle is a unique identifier that is assigned to each window created.

Accessibility engine

text

no

Auto

  1. ACCESSIBILITYENGINE="auto"

  2. ACCESSIBILITYENGINE="active"

  3. ACCESSIBILITYENGINE="java"

  4. ACCESSIBILITYENGINE="ie"

The accessibility engine that the activity will use to programmatically gather accurate information about a user interface in order to interact with it. The available options are:

  • Auto - AutoMate will detect the best accessibility engine to use based on current properties.
  • Active - Active Accessibility will be used. Active Accessibility is Microsoft's standard for exposing information about its user interface.
  • Java - Java Access Bridge will be used, which is a technology that enables certain Java applications and applets to be visible to assistive technologies on Microsoft Windows systems.
  • Internet Explorer - Internet Explorer DOM will be used, which is the Document Object Model used to represent objects in a web page (i.e. text, images, headers, links, etc.).

Window must contain the following object(s)

Yes/No

No

No

CONTAINSOBJECT="YES"

Specifies whether or not objects are to be included in the window identification criteria. All other object specification parameters are valid only if this is set to YES.

Object type must be

Yes/No

No

No

CHECKOBJECTTYPE="YES"

If set to YES, specifies that the object type is to be used to identify the object. If set to NO, the object type will not be used.

Object type

Text

No

(Empty)

OBJECTTYPE="ComboBox"

The type of the object. Valid only if the Object type must be parameter is set to YES.

Object name must be

Yes/No

No

No

CHECKOBJECTNAME="YES"

If set to YES, specifies that the object name is to be used to identify the object. If set to NO, the object name will not be used.

Object name

Text

No

(Empty)

OBJECTNAME="Font:"

The name of the object. Valid only if the Object name must be parameter is set to YES.

Object class must be

Yes/No

No

No

CHECKOBJECTCLASS="YES"

If set to YES, specifies that the object class is to be used to identify the object. If set to NO, the object class will not be used.

Object class

Text

No

(Empty)

OBJECTCLASS="MsoCommandBar"

The class of the object. Valid only if the Object class must be parameter is set to YES.

Object value must be

Yes/No

No

No

CHECKOBJECTVALUE="YES"

If set to YES, specifies that the object value is to be used to identify the object. If set to NO, the object value will not be used.

Object value

Text

No

(Empty)

OBJECTVALUE="Arial"

The value of the object. Valid only if the Object value must be parameter is set to YES.

Object position must be

Yes/No

No

No

CHECKOBJECTPOSITION="YES"

If set to YES, specifies that the object position is to be used to identify the object. If set to NO, the object position will not be used.

Object X Position

Number

No

(Empty)

OBJECTXPOS="30"

Specifies the X pixel coordinate location on the screen for the upper left corner of the object. Valid only if the Object position must be parameter is set to YES.

Object Y Position

Number

No

(Empty)

OBJECTYPOS="158"

Specifies the Y pixel coordinate location on the screen for the upper left corner of the object. Valid only if the Object position must be parameter is set to YES.

Examples

NOTE: The sample AML code below can be copied and pasted directly into the Steps panel of the Task Builder.

Sample 1

Maximize the window "Untitled - Notepad".  The window must contain the objects: names = "Application", values = "", classes = "Notepad", types = "MenuBar", X = "229", Y = "317", Check names = "YES", check values = "", check classes = "YES", check types = "YES", check positions = "YES".   

<AMWINDOWMAXIMIZE WINDOWTITLE="Untitled - Notepad" CONTAINSOBJECT="YES" OBJECTNAME="Application" OBJECTCLASS="Notepad" OBJECTTYPE="MenuBar" CHECKOBJECTNAME="YES" CHECKOBJECTCLASS="YES" CHECKOBJECTTYPE="YES" OBJECTXPOS="229" OBJECTYPOS="317" CHECKOBJECTPOSITION="YES" />

Sample 2

Get text from the control: (class = "Edit", type = "SelectableText", value = "Hello World!", X = "231", Y = "339", check class = "YES", check name = "NO", check type = "YES", check value = "YES", check position = "YES") and populate the variable "theVar".  The object must be in the window : (title = "Untitled - Notepad", class = "Notepad", handle = "58263186").  

<AMGETTEXT WINDOWTITLE="Untitled - Notepad" WINDOWCLASS="Notepad" WINDOWHANDLE="58263186" OBJECTVALUE="Hello World!" OBJECTCLASS="Edit" OBJECTTYPE="SelectableText" CHECKOBJECTVALUE="YES" CHECKOBJECTCLASS="YES" CHECKOBJECTTYPE="YES" CHECKOBJECTPOSITION="YES" OBJECTXPOS="231" OBJECTYPOS="339" RESULTVARIABLE="theVar" />