Sikuli Selenium Robot Framework Tutorial
Introduction
In this tutorial I will demonstrate how to integrate Sikuli (open source image based test automation tool) and Selenium (object based web automation framework) into Robot Framework (keyword and data driven test automation framework). This is not a Robot Framework Tutorial nor a Sikuli Tutorial. For more information about these technologies please visit their respective websites as indicated below:
- Sikuli website http://sikuli.org
- Selenium (web driver) Robot Framework library: https://github.com/rtomac/robotframework-selenium2library
- Robot Framework: http://code.google.com/p/robotframework
The following sections outline the detailed steps needed to perform the integration on Windows machine. Installation on different operating systems should be similar though it might not be exactly the same.
Python Installation
Robot Framework runs on Python so we need to install it. Please follow the steps below:
- Go to http://python.org
- Click the download link on the left side of the page
- Select windows installer “Python 2.7.2 Windows Installer” or you can directly access the link: http://python.org/ftp/python/2.7.2/python-2.7.2.msi You can always select the latest production version.
- Follow the instructions and use default settings
- To make sure Python was properly installed open a console window then type: “python” You should get the Python command prompt
- Type: “exit()” to quit
- You might consider setting PYTHONPATH environment variable if needed.
Java Run Time Installation
Sikuli is Java based so if you do not have Java Run Time (JRE) already installed then download the 32 bit package from oracle website and install it. You can obtain the package from: http://www.oracle.com/technetwork/java/javase/downloads/index.html. Select the version that matches your OS. In this tutorial I selected JRE6. Please note that Sikuli runs only using the JRE6 32 bit version.
Jython Installation
Jython is a Java implementation of the Python programming language. The reason why we need to install Jython is because we will be calling Sikuli Java API using Python scripts. You can also access Sikuli Java API from a native Java application if you wish. Please follow the steps below:
- Go to http://www.jython.org
- Click the download link on the left side of the page
- Download the most stable version. Clicking the download link takes you to http://sourceforge.net
- Double click the jar file to start the installer. Select default settings
- To make sure Jython is properly installed, open a console window then type: C:\\path\\to\\jython\\bin\\jython.bat. You should get the jython command prompt
- Type “exit()” to quit
- You might consider setting JYTHONPATH environment variable if needed
- If you encounter virtual memory related problems you might need to edit [C:\\path\\to\\jython\\bin\\jython.bat] and increase the value in [set _JAVA_MEM=-Xmx512m]
- If you encounter encoding issues for example using Sikuli to type international characters then you might need to add [-Dfile.encoding=utf-8] in [set _FULL_CMD]
Installing Easy Install
As the name indicates it is used for easy installations. Please follow the steps below:
- Go to: http://pypi.python.org/pypi/setuptools
- Download the file “setuptools-0.6c11.win32-py2.7.exe”
- Start the installer and use default settings
Note that there is an alternative installation tool called pip if you want to give it a try. Here is how to install pip on Windows:
- Download http://peak.telecommunity.com/dist/ez_setup.py to C:\\Python\\Scripts
- cd C:\\Python\\Scripts
- Type: easy_install pip
- Type: python ez_setup.py
Setting Environment Variables
In order to call programs from the command prompt directly we need to add some directories to the system PATH environment variable. Please follow the steps below:
- On Windows go to: Start > Settings > Control Panel > System > Advanced > Environment Variables
- Add the following directories to PATH:
- Path to python installation for example: C:\\python27
- Path to python scripts directory for example: C:\\Python27\\Scripts
- Path to jython bin directory for example: C:\\jython2.5.2\\bin
- Path to Java for example C:\\Program Files (x86)\\Java\\jre6\\bin
Install Robot Framework
Robot Framework provides high level automation services such as test execution and reporting. You can visit Robot Framework download page on Google Code or you may follow the steps below to install it:
- Open a new command window
- Type: easy_install robotframework
- To verify installation type: pybot –version
- If that does not work go to: C:\\python27\\scripts then type: robot_postinstall.py
Installing wxPython
This is a GUI toolkit for Python programming language. It is used by Robot Framework RIDE (will be installed in the next section)
- From the command line type: easy_install wxPython
- If this does not work go to: http://www.wxpython.org/download.php
- Download an installer for your OS
- Start the installer and follow the instructions using default settings
Note: For some reason the 64 bit version did not work with me. I tried the 32 bit version and it worked fine.
Robot Framework RIDE Installation
Robot Framework Ride is an awesome Integrated Development Environment (IDE) for Robot Framework test case development. Follow the steps below:
- Open a new command window
- Type: easy_install robotframework-ride
- To make sure Robot Framework RIDE is working fine type: python ride.py
- In Windows you can associate python files with the python executable. In that case you just need to only type ride.py from the command line
Sikuli Installation
Sikuli is used for visual testing. This installation is needed to get a copy of sikuli-script.jar which provides image based test API. The installation also includes an IDE that can help in debugging visual tests. Follow the steps below:
- Go to : http://sikuli.org/download.shtml
- Depending on your OS download the appropriate version. Use the Self-extracting installer for easy installation “Sikuli-X-1.0rc3 (r905)-win32.exe” You can always select the latest stable version
- Start the installer and follow the instructions using default settings
Selenium2Library Installation
This is a Robot Framework Selenium test library which provides Selenium 2 (web driver) test keywords. Follow the steps below:
- Open a command line console
- Type: easy_install robotframework-selenium2library
Sikuli Python and Selenium Robot Framework in Action
You can find below all the steps needed to develop and execute a Robot Framework test case using both Sikuli and Selenium at the same time. Note that Selenium library runs on Python (does not support Jython) however Sikuli needs Jython so using one Python interpreter is not going to work for both at the same time. The trick is to use Python (works fine with Selenium library) and run Jython on a remote machine (localhost simulating remote machine). This way we execute Selenium keywords using Python on the local machine and call Sikuli keywords on the remote machine using RPC-xml sever.
-
- Create the following directories (you can do it the way you prefer but this is just to make it easy to follow the steps) [C:\\robot] [C:\\robot\\data] [C:\\robot\\libs] [C:\\robot\\suites]
- Download the Python Remote Server and save it into [C:\\robot\\libs] You can find it here: http://robotframework.googlecode.com/hg/tools/remoteserver/robotremoteserver.py
- Copy the Sikuli Remote Library code (you can find it at the bottom of the page) and save it as: C:\\robot\\libs\\SikuliRemoteLibrary.py
- Copy C:\\Program Files\\Sikuli X\\sikuli-script.jar into C:\\robot\\libs. Note that this step is only to put things in one place but you can skip this step and only point to Sikuli jar file
- Add C:\\robot\\libs\\sikuli-script.jar to your java class path. You need to set the CLASSPATH environment variable. Make sure there is no extra spaces otherwise problems may arise.
- Run the remote server. Go to: C:\\robot\\libs and type: jython.bat SikuliRemoteLibrary.py
- Launch RIDE. From the command line type: ride.py
- Create a new project: File -> New project then set the parent directory to C:\\robot\\suites and give the suite a name for example “test”
- Switch to text edit view and paste the following text:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
*** Settings *** Documentation Integrating Selenium, Sikuli into Robot Framework Test Setup Login To Yahoo Mail ${user_name} ${password} Test Teardown Tear Test Down # Selenium library Library Selenium2Library 15 # Sikuli Library Remote http://localhost:${port} # Taking screenshots when a test fails Library Screenshot *** Variables *** # Yahoo mail URL ${url} http://mail.yahoo.com # Browser ${browser} ff # User name ${user_name} user_name # Password ${password} password # Default port number for the remote server ${port} 8270 # Sikuli images ${data_path} c:\\robot\\data # Used in Sikuli image comparison ${similarity} 0.90 # Time to wait for objects ${timeout} 10 *** Testcases *** login To Yahoo Mail Test Case Wait Until Page Contains Element yuhead-sform-searchfield ${timeout} Input Text yuhead-sform-searchfield ENGLISH Object Exists ${data_path}\\search_box_left.png ${similarity} *** Keywords *** Login To Yahoo Mail [Arguments] ${user} ${pass} [Documentation] This keyword logs user into Yahoo mail Open Browser ${url} ${browser} Wait Until Page Contains Element username Input Text username ${user} Input Password passwd ${pass} Click Button .save Wait Until Page Contains Element toolbar Click Object ${data_path}\\maximize.png Sleep 1 Tear Test Down Run Keyword If Test Failed Take Screenshot Close All Browsers |
Here is the python code for Sikuli Remote Library:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
import sys from robotremoteserver import RobotRemoteServer from org.sikuli.script import * class SikuliRemoteLibrary: def __init__(self): self.SS = Screen() self.PT = Pattern() def _wait(self, imgFile, timeOut, similarity): try: self.PT = Pattern(imgFile) self.PT = self.PT.similar(float(similarity)) self.SS.wait(self.PT, float(timeOut)) except FindFailed, err: print "ERR: _wait" raise AssertionError(err) def click_object(self, imgFile, timeOut, similarity): try: self._wait(imgFile, timeOut, similarity) self.SS.click(imgFile) except FindFailed, err: raise AssertionError("Cannot click [" + imgFile + "]") def object_exists(self, imgFile, similarity, timeOut): try: self._wait(imgFile, timeOut, similarity) except FindFailed, err: raise AssertionError("Could not find [" + imgFile + "]") def type_at_object(self, imgFile, txt, timeOut, similarity): try: self._wait(imgFile, timeOut, similarity) self.SS.type(imgFile, txt) except FindFailed, err: raise AssertionError("Cannot type at [" + imgFile + "]") def paste_at_object(self, imgFile, txt, timeOut, similarity): try: self._wait(imgFile, timeOut, similarity) self.SS.paste(imgFile, txt) except FindFailed, err: raise AssertionError("Cannot paste at [" + imgFile + "]") if __name__ == '__main__': SRL = SikuliRemoteLibrary() RobotRemoteServer(SRL, *sys.argv[1:]) |
Continuous Integration
We explained how to execute the test suite using RIDE but what if we want to execute it automatically in a CI (Continuous Integration) environment. In order to do that we should be able to invoke the test suite from the command line for example Windows command prompt or UNIX terminal. Let us do that:
-
- Create a directory called [C:\\robot\\arg]. This is where Robot argument files are stored. Recall that argument files are used to pass parameters to the framework at runtime.
- Create an argument text file for example C:\\robot\\arg\arg.txt
- Paste the following into the argument file:
1 2 3 |
--variable user_name:user_name --variable password:pass_word --outputdir ..\\out |
As you can see we are passing two parameters (user name and password) and specifying the output directory relative to the directory where the test suite is located.
-
- Create a directory called C:\\robot\\run
- Create a batch file called run.bat
- Paste the following into the batch file:
1 |
call pybot.bat --argumentfile C:\\robot\arg\\arg.txt C:\\robot\\suites\\test.txt |
This command executes the test suite using an argument file. Note that variables passed in the argument file override whatever variables defined in the test suite if they have the same name. The reason why we pass parameters in an argument file is because we might need to keep the variables in the test suite for development and debugging purposes when we use RIDE. It is up to test developer to articulate the test cases in the way he or she prefers.
So far we are ready to invoke robot from the command line using a windows bat file. We will not be doing that manually but instead we will use Jenkins. We are still working on the Windows OS.
Install Jenkins
- Go to the download page at: http://jenkins-ci.org
- You can download the windows installer then run Jenkins as a windows service. The windows service in this case will not be able to interact with the GUI which causes selenium to fail opening the browser. The solution is to run Jenkins from the war file. Now go ahead and download the war file jenkins.war
- Run Jenkins as follows: java -jar jenkins.war –httpPort=80
Install Jenkins Plugin
Robot Framework has a Jenkins plugin for aggregating test execution results and linking to reports and logs. Follow the steps below to install the plugin:
- Go to: http://code.google.com/p/robotframework-javatools/downloads/list/
- Download the file: robotframework.hpi
- Make sure Jenkins is up otherwise launch using the command: java -jar jenkins.war –httpPort=80
- Open a browser window and point to http://localhost:8080
- Click the link “Manage Jenkins”
- Click the link “Manage Plugins”
- Click the “Advanced” tab
- Go to “Upload Plugin” and browse to the robotframework.hpi file then upload it
Note: You might need to restart Jenkins in order to see robot framework plugin listed in the post build action drop down.
Create Jenkins Job
Let us now create and configure a Jenkins job to automatically execute robot framework test suite. Follow the steps below:
-
- Click the link “New Job”
- Give it a name and select “Build a free-style software project”
- Click “Add build step” drop down
- Click “Execute windows batch command”
- You can paste something like:
1 2 |
cd C:\\robot\\run call run.bat |
- Click the drop down “Add post-build action”
- Click “Publish Robot Framework test results”
- Set “Directory of Robot output” as “..\\out”
- Set “Log/Report link” as “log.html”
- Set “Output xml name” as “output.xml”
- Set “Report html name” as “report.html”
- Set “Log html name” as “log.html”
That is it we are now ready to kick off the Jenkins job. Make sure Sikuli remote library is running and then click to build the job. As the job executes you can go to the console and watch Jenkins invoking Robot Framework in a similar way to invoking Robot from the command line. We can configure Jenkins to run the job on a slave machine but this is beyond the scope of this article.
Tips and Tricks
-
- Firefox Profiles
Go to robot framework installation directory for example:
1 |
C:\\Python27\\Lib\site-packages\\robotframework_selenium2library-1.0.1-py2.7.egg\\Selenium2Library\\resources |
You will find Firefox browser profiles which are used by selenium library. It might be very useful to update the provided profile with whatever Firefox profile you wish. This is a great advantage and saves allot of test setup time if you need the profile to be configured in a certain way instead of programmatically doing that. You could maintain several copies of different profiles there and then copy the one you need at run time just before invoking robot. Here is a sample run.bat file that does that:
1 2 3 4 5 6 |
@echo off @rmdir %SEL2RES%\firefoxprofile /q /s @mkdir %SEL2RES%\firefoxprofile @xcopy %SEL2RES%\my_ff_profile %SEL2RES%\firefoxprofile /e /f /y @echo on call pybot.bat --argumentfile c:\\robot\\arg\\arg.txt C:\\robot\\suites\\test.txt |
Where %SEL2RES% is an environment variable that points to the resources directory in robot framework selenium library.
-
- Window Minimized
Sikuli remote server and Jenkins should start minimized in order not to interfere with Sikuli while image based testing is executing. To do that create a windows shortcut for both Sikuli remote server batch file and Jenkins batch file then right click the link and go to properties. There you can chose to run the command as minimized.
That is it for today, If you have questions about the code please contact me as the main focus of this article was on the integration side not on the actual test cases. Thanks for reading.
hi, this is a really interesting article. only one thing I could not find was the SikuliRemoteLibrary.py – can you help and provide a download link. Thank you.
It is at the bottom of the article
For the step:
Run the remote server. Go to: C:\robot\libs and type: jython.bat SikuliRemoteLibrary.py
I seem to be getting an error on line 8: self.SS = Screen() saying that Screen is a NoneType object and is not callable. All the other steps worked fine for me.
Also may I suggest you to add correct indents to SikuliRemoteLibrary.py for the try, except blocks.
Thanks!
Hi,
I got an error when running SikuliRemoteLibrary.py :
” File “SikuliRemoteLibrary.py”, line 3, in
from org.sikuli.script import *
ImportError: No module named sikuli ”
Could you pls correct it ?
Thanks a lot in advance!
@Mohammed Abualrob Thanks in advance
It works well 😛
@everyone
You can download file SikuliRemoteLibrary.py at :
https://docs.google.com/file/d/0B_VTRRrTuA3PckdIZTBwS1dHVkU/edit?pli=1.
The problem is that those data contained a lot of tabs,spaces…To fix it you need:
— Download IDLE Python
— Open your file with IDLE. Press key: Alt + 6 then input value 2. Then OK.
At the moment I want to build TestNG Framework. I want to use pure JAVA language. TestNG Framework doesn’t support remote execution. Selenium Grid supports it. To can run remote execution with Sikuli I use XML-RPC technique.
I had the same problem as Henry Lu.
” File “SikuliRemoteLibrary.py”, line 3, in
from org.sikuli.script import *
ImportError: No module named sikuli ”
I eventually got it fixed, and it turned out I missed some extra environment variables.
I put the following into jythonsikuli.bat and run that instead of jython.bat to make it work:
@echo off
set JYTHONPATH=c:\sikuli\sikuli-script.jar\Lib
set CLASSPATH=c:\sikuli\sikuli-script.jar
set SIKULI_HOME=c:\sikuli\
set PATH=%PATH%c:\sikuli\libs;
jython.bat %*
I saved this bat-file to my jython2.5.2\bin directory.
Hope this helps!
Hello
According your article, on xp-32bit system it works well.
On Win7-64 bit system, when I run jython.bat robotremoteserver.py, it pops up importError:No module named inspect. I add inpect.py to the folder, there will pops up other error.
Get an error for the last line of test.txt:-
Click Object ${data_path}\maximize.png
the error says, image not found on disk, null error,
please advise
Hi vinod,
Try to use the DOUBLE backslash in your path, like C:\\datapath\\maximize.png
Thanks a lot Hahadali !!! It works for me 🙂
Thanks for the detail description, I like the way you explain all the implementation. I owe you bier, just call me If you visit Dusseldorf Germany!
where is the SikuliRemoteLibrary.py? i can’t find it everywhere.
I have a sikuli script for opening a firefox browser by doubleclick icon on my windows7 desktop and I am able to run this from command line. Its working as expected.
I want to run the same script from robot framework without using selenium library.
I have integrated both sikuli and RFW. But I am not getting the keywords part. What are the keywords we need to use so that both RFW and sikuli script will run simutaneously. Please help me on how to do this.
This Example in one system or Two systems ?
Is this in one System or Two ??
command: pybot.bat –argumentfile c:\users\ee208473\appdata\local\temp\RIDEfsjogn.d\argfile.txt –listener C:\Python27\lib\site-packages\robotide\contrib\testrunner\TestRunnerAgent.py:52992:False C:\RoboTrailAnjali\suites\tester2
unexpected error: [ ERROR ] Suite ‘Tester2’ contains no tests. I am getting this error just before doing continous integration what todo
command: C:\jython2.5.4rc1\bin>jythonsikuli.bat c:\robot\libs\SikuliRemoteLibrary.py
Usage: java [-options] class [args…]
(to execute a class)
or java [-options] -jar jarfile [args…]
(to execute a jar file)
where options include:
-client to select the “client” VM
-server to select the “server” VM
-hotspot is a synonym for the “client” VM [deprecated]
The default VM is client.
-cp
-classpath
A ; separated list of directories, JAR archives,
and ZIP archives to search for class files.
-D=
set a system property
-verbose[:class|gc|jni]
enable verbose output
-version print product version and exit
-version:
require the specified version to run
-showversion print product version and continue
-jre-restrict-search | -jre-no-restrict-search
include/exclude user private JREs in the version search
-? -help print this help message
-X print help on non-standard options
-ea[:…|:]
-enableassertions[:…|:]
enable assertions
-da[:…|:]
-disableassertions[:…|:]
disable assertions
-esa | -enablesystemassertions
enable system assertions
-dsa | -disablesystemassertions
disable system assertions
-agentlib:[=]
load native agent library , e.g. -agentlib:hprof
see also, -agentlib:jdwp=help and -agentlib:hprof=help
-agentpath:[=]
load native agent library by full pathname
-javaagent:[=]
load Java programming language agent, see java.lang.instrument
-splash:
show splash screen with specified image
HOW TO GET PORT AND SERVER IMFORMATION?
Hi,
I’m sharing this library (SikuliXRobotLibrary) to anyone interested. The libary uses SikuliXv 1.1.0 methods. The installation instructions and how to run the demo test are in the following github link as well:
https://github.com/jaredfin/SikuliXRobotLibrary
jythonsikuli.bat file content is below.
@echo off
set JYTHONPATH=c:\sikuli\sikuli-script.jar\Lib
set CLASSPATH=c:\sikuli\sikuli-script.jar
set SIKULI_HOME=c:\sikuli\
set PATH=%PATH%c:\sikuli\libs;
jython.bat %*
I executed like below I got following error.
C:\jython2.7.0\bin>jythonsikuli.bat C:\robot\libs\SikuliRemoteLibrary.py
The input line is too long.
C:\jython2.7.0\bin>
I want to use SIKULI + Robot Framework for automating a Windows application, I do not want any Selenium Library.. I have followed your perfect tutorial and installed everything.
ROBOT framework 3.0
Python 2.7
JYTHON
RIDE
Could you help me in integrating these without the issue of python and selenium not being able to use same interpreter.
I actually tried a lot to follow this tutorial
http://blog.mykhailo.com/2011/02/how-to-sikuli-and-robot-framework.html
but its very complex and i dont understand.
PS: I came from quora 🙂
How use keywords from Sikuli in RIDE?
how can I import selenium to sikuli ide ? If I dont want to further make the system complex?
I am new to python desktop based robot framework RIDE .
using RIDE, on desktop how can I click on vnc_viewer icon?
Hello
Thank you very much by this article quite understandable and clear, would you have an idea on what was changed since, or it is still relevant, for example the new versions, new tools …
Kind regards