Video Tutorials

Advanced UI Automation

Input actions are the ones where you or the robot take some direct actions on the application or web page you’re automating: clicking, typing, keyboard shortcuts, and so on.

Output actions are those that help you extract information out of an app, and into Studio for further processing. 

Input methods:

  1. Default - it uses the mouse and keyboard drivers to simulate a human operating them. It works every time, but with a few downsides, like speed and the requirement to have the application active. So it’s a good idea to start with this method to make sure everything works, and then change it to one of the other 2 methods.
  2. Simulate Type/Click - it is the fastest and works in the background, but it does not support keyboard shortcuts. Uses the technology of the target app to simulate the type or click. 
  3. Window Messages - converts all text to lowercase, it cannot empty a field before writing to it and it is not particularly fast. It sends a specific message directly to the target application to perform the specified action. 

Output methods:

  1. Fulltext is the default method, and it’s the best one for most situations. It’s fast, accurate, and works in background.
  2. The Native method has the advantage of being able to extract word data, like the screen coordinates of each word or character.
  3. OCR - As you may know, in general, OCR technology is not 100% accurate, but it does work as a last resort in situation where none of the other direct methods give us the intended results.