Monday, January 25, 2010

Automated test in Android by Sikuli

There is a very friendly and pretty cool project called 'Sikuli' and the authors are from Taiwan too. What is Sikuli? I copy these texts from its website. Sikuli is a visual technology to search and automate graphical user interfaces (GUI) using images (screenshots). The first release of Sikuli contains Sikuli Script, a visual scripting API for Jython, and Sikuli IDE, an integrated development environment for writing visual scripts with screenshots easily. Sikuli Script automates anything you see on the screen without internal API's support. You can programmatically control a web page, a desktop application running on Windows/Linux/Mac OS X, or even an iphone application running in an emulator.

How I use Sikuli for Android? Since we can catch things from screenshots, it means we can run Android emulator in my Ubuntu machine. Furthermore, I can run a vnc server in Android device, then run a vnc viewer in Ubuntu. Therefore, I can see Android HOME screen in my Desktop. What it can do for me? ha, it can run automated tests and then we don't need SQA to verify phone basic functionalities. Like detect GSM signal, dial out a phone call, enable Bluetooth, connect to WiFi hotspot....etc.

Launch Sikuli IDE editor in Ubuntu

sh Sikuli-IDE/

Launch Android in Ubuntu

There is another tool "androidscreencast" I am using for this automated test. It can allow me to control my Android Dev phone remotely. We can have keyboard and mouse input! This is very important for Sikuli scripts, coz I can say "Click" something or "type" something. :)

javaws androidscreencast.jnlp

Let's write some examples! Please check these draft videos!

Build Sikuli in Ubuntu 9.10 (64 bit)

1. download the source code (you'll need to install bzr, if you don't have it)
> bzr branch lp:sikuli

2. check java version (use java-6-sun)
erin@midnight:~/code/sikuli$ file /etc/alternatives/java
/etc/alternatives/java: symbolic link to `/usr/lib/jvm/java-6-sun/jre/bin/java'
erin@midnight:~/code/sikuli$ file /etc/alternatives/javac
/etc/alternatives/javac: symbolic link to `/usr/lib/jvm/java-1.5.0-sun/bin/javac'

3. install some libraries
tesseract-ocr - Command line OCR tool
tesseract-ocr-dev - Development files for the tesseract command line OCR tool
maven2 - Java software project management and comprehension tool
libcv1 - computer vision library
libcvaux1 - computer vision extension library
libhighgui1 - computer vision GUI library

4. build code
>cd sikuli-script

g++ ScreenMatchProxy.o ocr-matcher.o template-matcher.o myocr.o screendiff.o -o ../../../target/lib/ `pkg-config --libs opencv` -shared -L/usr/local/lib -I/usr/lib/jvm/java-6-sun/lib -ltesseract_full -ltiff
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/libtesseract_full.a(libtesseract_full.o): relocation R_X86_64_32 against `BLOCK_zapper(ELIST_LINK*)' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/4.4.1/../../../../lib/libtesseract_full.a: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[1]: *** [../../../target/lib/] Error 1

how i solved: i removed -ltesseract_full, then rebuilt it!
>cd src/main/native/
>g++ ScreenMatchProxy.o ocr-matcher.o template-matcher.o myocr.o screendiff.o -o ../../../target/lib/ `pkg-config --libs opencv` -shared -L/usr/local/lib -I/usr/lib/jvm/java-6-sun/lib -ltiff

>make install-jar (it would build sikuli-script.jar, that is what we need!)
>make release (it would build for all platforms, then zip to a file.)

erin@midnight:~/project/sikuli/sikuli-script$ make release
mkdir -p target/sikuli-script
rm -rf target/sikuli-script/*
cp target/sikuli-script.{bat,sh} target/sikuli-script
cp: cannot stat `target/sikuli-script.{bat,sh}': No such file or directory
make: *** [release] Error 1

how i solved: modify two lines in sikuli-script/Makefile and mkdir ../release
cp target/sikuli-script.bat target/ $(RELEASE_TMP_PATH)
cp -r $(LIB_PATH)/win32 $(LIB_PATH)/tmplib $(RELEASE_TMP_PATH)

Done! you will find a zip file '' in release directory.

>cd sikuli-ide
>make install-jar
>make major-release

Done! you will find a zip file '' in release directory. This one is like we download from Sikuli website for linux version.