Friday, 28 October 2016

Robot Framework and Windows GUI Automation with AutoItLibrary

Web test scenarios sometimes involve native Windows components that Selenium cannot access. I show how to use AutoItLibrary to automate Windows components inside a web test scenario written in Robot Framework and Selenium Library.

Usage of Robot’s external test libraries should be considered at first place. Alternatively AutoIT or other tools can be independently used to automate their parts and call these scripts from Robot code. That is one option when the library does not support the needed functionality. However it is an advantage to have all the control under single framework.

Certain dialogs, file browsing or sending email are examples of Windows components that may appear as part of the web test scenario. My example test case performs an XML validation for a file at To pick up a file to be uploaded, an operating system/browser specific file browser component is launched. In Windows environment, AutoIt functionality is needed to take control over that window and type in the file name and click Open button. After that Selenium Library keywords are used to continue the web test flow.

AutoITLibrary and installation instructions can be found hereIt is a wrapper like keyword library to AutoIt, a free tool for automating the Windows GUI. Keyword documentation can be found locally from C:\RobotFramework\Extensions\AutoItLibrary\AutoItLibrary.html and in practise the function specs at AutoIt pages are an essential reference for the usage, for example to use keyword Control Click, see

AutoIt comes with an inspection tool, Au3Info.exe (installed under C:\RobotFramework\Extensions\AutoItLibrary), that can be used to get the information needed to locate the Windows UI elements such as text fields and buttons. Drag the Finder Tool over the area to be inspected. The tool is used here to see that classname ”Button1” is a proper identifier for the Open button.

Similarly it can be inspected that "Edit1" is an identifier for the text field where file name is inputted. In the Robot code we first wait that the window named "Open" is detected, then input the file name and click Open and continue by clicking the validation button at the web page. Finally the success message is expected.

In this example the UI elements can be inspected and uniquely identified. X and Y offset values can be used to click certain position of the identified area. Proper UI mapping should be made instead of hard coding. In case no elements cannot be inspected, tools based on image recognition like Sikuli are an option to consider. I demonstrate that one at the next post. By the way, in the video there are a few sleep commands added for demonstration purposes.

Tuesday, 27 September 2016

Executing Remote Admin Tasks with Robot Framework's SSH Library

Technical testing or devops in general often requires administrative tasks to be done at test environments or remote systems under test as part of the testing process. Transferring test data files, cleaning up test server environment or restoring test database are examples of such tasks. Recurring admin tasks naturally are good candidates to be automated.
There are certainly various ways to execute commands and scripts at remote machine and in this example the objective is to keep execution and verification of results at Robot Framework code level, under single flow of control. Alternatively Run commands from OperatingSystem library could be used to launch local scripts taking care of the remote actions or Java or Python code could be run by keywords implemented as a custom test library.
Method demonstrated here is to use Robot Framework’s SSH Library to execute commands/scripts at remote machine and read the results to be asserted. I have two examples, firstly a single command executing a remote script expected to return with code 0. The second example deals with executing a sequence of remote commands as well as reading the results from output.
See SSH Library documentation for installation instructions. By using pip the required dependencies are also installed and the keywords are usable from Robot FW code after importing the SSHLibrary itself. 

Note that by default Execute Command returns the stdout output. To get the return code instead, we explicitly set both return_rc argument as True and return_stdout as False.
Commands run by Execute Command or Start Command keywords are always run in a new shell and hence the changes (for example cd command) made by previous keywords are not visible to the next keywords when executing multiple commands. To execute a sequence of commands at a remote shell we use Write and Read keywords as in the following example. Assume an asynchronous/background process is started and we want to wait until expected indication about the completion of the task can be read from output, for example ”Result = OK.” or ”Result = FAIL.” written by the custom script in this example. Output is read until ”Result =” can be read from the output and then read until a dot character to get the result string for assertion. Note that this should happen within the timeout set when importing the SSH Library, 10 minutes in this example.

A bit of string manipulation is done here with Strip String and Fetch From Left keywords to get the exact string representing the result. See also related keywords Read, Read Until Prompt and Read Until Regexp to find the most suitable one depending on the expected output.

Friday, 19 August 2016

API Testing with Robot Framework

This example shows how to start API testing with Robot Framework. Handling of both XML and JSON responses are demonstrated. Example API used is Google Maps Geocoding API and request I make is a location search with a Finnish postal number. My test verifies that the expected city name can be fetched from the response. Robot Framework’s RequestsLibrary is used to make the requests.
Example request:,Finland

Write it to browser to get the response. In my test I investigate elements ‘status’ to verity that the response is OK and ‘formatted_address’ to fetch the expected city name, Helsinki in this case.
In case of XML response, Robot Framework’s XML Library is used to parse the response. Method is powerful since xpath expressions can be used to parse the element structure, especially in case of nested or otherwise complex response.
Similar request with JSON response:,Finland

Name/value pairs at JSON response realise as Python dictionary containing lists and dictionaries in this case. The top level of entries can hence be conveniently accessed e.g. with keyword Get From Dictionary. Parsing more deeply requires either traversing the data structure (like ${response.json()['results'][0]['formatted_address']} in the example), developing parsing functionality in Python or using some of the JSON libraries available. At least straightforward JSON structures can be referred with the existing library implementations but the actual complexity level they can support is another topic.
This was an example how to start API testing with Robot Framework and a few suitable libraries. Powerful cases are already possible without Python coding. Of course, the response as a file, as a string, or as whatever data structure can be parsed in numerous ways depending on the response structure and particular need.

*** Settings ***
Library    RequestsLibrary
Library    XML
Library    String
Library    Collections

Suite Setup    Create Session    googleapi
Suite Teardown    Delete All Sessions

*** Test Cases ***

Demonstrate API testing and XML response
    Postcode should match city in XML response    00500    Helsinki
    Postcode should match city in XML response    33100    Tampere

Demonstrate API testing and JSON response
    Postcode should match city in JSON response    20250    Turku
    Postcode should match city in JSON response    02200    Espoo

*** Keywords ***
Postcode should match city in XML response
    [Arguments]    ${postal_code}    ${expected_city}

    # Make the request and verify return code
    ${response}=    Get Request     googleapi    maps/api/geocode/xml?address=${postal_code},Finland
    Should Be Equal    ${response.status_code}     ${200}

    # Parse response to XML element structure
    ${xml_response}=    Parse XML    ${response.text}

    # Get and assert element 'status' from the XML structure
    ${status_element}=    Get Element Text    ${xml_response}    status
    Should Be Equal As Strings    ${status_element}    OK

    # Verify that the expected city name can be found from the 'formatted_address' element
    ${returned_address}=     Get Element Text    ${xml_response}     result/formatted_address
    ${returned_city}=    Fetch City From Formatted Address    ${returned_address}

    Should Be Equal As Strings    ${returned_city}    ${expected_city}

Postcode should match city in JSON response
    [Arguments]    ${postal_code}    ${expected_city}

    # Make the request and verify return code
    ${response}=    Get Request     googleapi    maps/api/geocode/json?address=${postal_code},Finland
    Should Be Equal    ${response.status_code}    ${200}

    # Get and assert element 'status' from the JSON dictionary
    ${status_element}=    Get From Dictionary       ${response.json()}    status
    Should Be Equal As Strings    ${status_element}    OK

    # Verify that the expected city name can be found from the 'formatted_address' element
    ${returned_address}=    Set Variable    ${response.json()['results'][0]['formatted_address']}
    ${returned_city}=    Fetch City From Formatted Address    ${returned_address}

    Should Be Equal As Strings    ${returned_city}    ${expected_city}

Fetch City From Formatted Address
    [Arguments]    ${raw_address}
    ${addr}=    Fetch From Left    ${raw_address}    ,
    ${addr}=    Fetch From Right    ${addr}    ${SPACE}
    [Return]    ${addr}