ProbeScript Grammar (EBNF)
This page is the normative grammar of ProbeScript. It was derived from the parser source
(internal/parser/lexer.go, token.go, parser.go, ast.go), not from the other docs, and it is
kept honest by a conformance test (internal/parser/grammar_test.go) that:
- parses every code block on this page that is tagged
probe, - parses at least one example for every statement production defined below,
- fails when a keyword, compound keyword or filler word is added to or removed from
token.gowithout this page being updated.
The Syntax and Dictionary pages are the friendly reference; where they disagree with the parser, see Where the docs and the parser disagree.
Notation
Section titled “Notation”The grammar uses ISO/IEC 14977 style EBNF.
| Symbol | Meaning |
|---|---|
name = ... ; |
production; every production ends with ; |
a , b |
concatenation (a followed by b) |
a | b |
alternation |
[ a ] |
optional |
{ a } |
zero or more repetitions |
( a ) |
grouping |
"text" |
terminal; a keyword or word, matched case-insensitively (except where noted) |
'"' |
a terminal containing a double quote |
a - b |
a except anything that matches b |
? ... ? |
special sequence: an informal description of a character or token class |
(* ... *) |
comment |
Two levels are described. Section 1 turns characters into tokens. Sections 2 onward are written over
tokens. The token classes are STRING, INT, FLOAT, ORDINAL, ID, TAG, COLON, PUNCT, WORD,
NEWLINE, INDENT and DEDENT. A quoted terminal in a syntactic production denotes the keyword token
with that spelling, or - for words that are not in the keyword table, such as "native" or "visible" -
a WORD token with that spelling. A terminal with spaces, such as "go back", is a
compound keyword and is a single token.
The parser is a hand-written recursive-descent parser that dispatches on the first token of a line. Where two alternatives overlap, the alternative listed first wins (ordered choice); the dispatch table gives the exact order.
1. Lexical grammar
Section titled “1. Lexical grammar”Characters, lines and comments
Section titled “Characters, lines and comments”letter = ? any Unicode letter ? ;digit = ? any Unicode decimal digit ? ;TAB = ? U+0009 ? ;LF = ? U+000A ? ;CR = ? U+000D ? ;line-break = CR , LF | LF | CR ;not-eol = ? any character except CR and LF ? ;blank = " " | TAB ;
source = { source-line } ;source-line = indent , [ comment | token-run ] , ( line-break | ? end of input ? ) ;indent = { blank } ; (* width = number of spaces + 4 per TAB *)comment = "#" , { not-eol } ; (* a line whose first non-blank character is "#" is always a comment *)token-run = { blank | lexeme } , [ inline-comment ] ;inline-comment = "#" , { not-eol } ; (* only where the "#" is NOT followed by a letter or "_" ; otherwise it starts an ID *)A line that is empty, blank or comment-only produces no tokens at all. Every other line produces its
tokens followed by one NEWLINE.
Lexemes
Section titled “Lexemes”lexeme = STRING | FLOAT | ORDINAL | INT | ID | TAG | COLON | PUNCT | identifier | ignored ; (* FLOAT, ORDINAL and INT all start with a digit: the longest form wins, tried in that order *)
STRING = quoted-string | l10n-string ;quoted-string = '"' , { string-char } , '"' ;l10n-string = "l10n" , blank , { blank } , quoted-string ; (* a string whose text comes from the app's ARB files *)string-char = ? any character except '"', "\" and LF ? | escape ;escape = "\" , ( '"' | "n" | "t" | ? any other character, which stands for itself ? ) ;
INT = digit , { digit } ;FLOAT = digit , { digit } , "." , digit , { digit } ;ORDINAL = digit , { digit } , ( "st" | "nd" | "rd" | "th" ) ; (* lower case, not followed by another letter *)
ID = "#" , ( letter | "_" ) , { letter | digit | "_" } ;TAG = "@" , { letter | digit | "_" } ;COLON = ":" ;PUNCT = "(" | ")" | "," | "-" ;
identifier = ( letter | "_" | "'" ) , { letter | digit | "_" | "'" | "-" , ( letter | digit ) } ; (* a hyphen is part of a word only when a letter or digit follows it: "looking-for" *)
ignored = ? any other character except "<": it is silently dropped, e.g. / . = ; { } [ ] > ! * + $ ? ; (* an unquoted "<" outside a string is a lexical error: write "<email>" with quotes *)An identifier whose lower-cased spelling is in the keyword table becomes a keyword
token; any other identifier becomes a WORD. PUNCT, TAG and the four listed words
(verify, set, location, clipboard) are also WORD-class tokens. Keywords are
case-insensitive (TAP "x" is tap "x").
WORD = identifier - keyword ;l10n "key" is a STRING wherever a string may appear (tap l10n "saveButton", see l10n "title"): the lexer
keeps the key, and the runner replaces it with the text of that key from the ARB file of the language the app currently
runs in (probe.yaml l10n.dir; see Testing localized apps). The word l10n followed by anything
other than blanks and a quote is an ordinary WORD.
Strings cannot contain a raw line break. Placeholders such as "<email>" are ordinary string content;
see Placeholders and variables.
Layout: NEWLINE, INDENT, DEDENT
Section titled “Layout: NEWLINE, INDENT, DEDENT”(* For every token-run, compare its indent width w with the top s of the indent stack (initially 0): w > s : push w and emit INDENT w < s : pop while w < top of stack, emitting one DEDENT per pop (a width that matches no enclosing level is NOT an error: it is treated as the next lower level) w = s : nothing then emit the line's tokens and one NEWLINE. At end of input emit one DEDENT per open level, then EOF. *)block = INDENT , { statement | NEWLINE } , DEDENT ;body = [ block ] ; (* a header line is followed by NEWLINE; a missing block is an empty body *)Keywords
Section titled “Keywords”keyword = "test" | "recipe" | "use" | "before" | "after" | "on" | "open" | "tap" | "type" | "see" | "assert" | "wait" | "swipe" | "scroll" | "clear" | "close" | "drag" | "pinch" | "rotate" | "toggle" | "shake" | "press" | "if" | "otherwise" | "else" | "repeat" | "retry" | "times" | "optional" | "for" | "each" | "when" | "respond" | "the" | "a" | "an" | "in" | "into" | "at" | "of" | "to" | "from" | "is" | "are" | "that" | "this" | "it" | "with" | "as" | "and" | "until" | "appears" | "disappears" | "enabled" | "disabled" | "checked" | "contains" | "exactly" | "button" | "field" | "app" | "page" | "network" | "idle" | "load" | "seconds" | "second" | "dart" | "run" | "get" | "post" | "put" | "delete" | "body" | "examples" | "matching" | "between" | "take" | "compare" | "screenshot" | "called" | "dump" | "widget" | "tree" | "save" | "device" | "logs" | "pause" | "log" | "back" | "home" | "failure" | "restart" | "allow" | "deny" | "grant" | "revoke" | "permission" | "permissions" | "all" | "kill" | "copy" | "paste" | "call" | "below" | "above" | "left" | "right" | "focused" | "link" | "animations" | "animation" | "store" | "composite" | "sync" | "biometric" | "enroll" | "deliver" | "signal" | "read" | "travel" | "over" ;
listed-word = "verify" | "set" | "location" | "language" | "clipboard" ; (* in the keyword table, but they lex as plain WORD tokens *)"else" is an alias of "otherwise", "animation" of "animations", "permissions" of "permission".
Because every keyword is reserved, a keyword cannot be used where a plain WORD is required (a recipe
parameter, a store ... as variable, a type selector); quote it or choose another word. Keywords are
fine inside recipe-call names and inside strings. A number of keywords (pinch, between,
home, back, load, widget, …) have no built-in statement of their own; a line starting with
one is a recipe call. press is the exception only for a line that is exactly press <key> with one of the eight keys (see press-key-step). Every other line that starts with press is a recipe call, as before, so a recipe named press <word> still works unless its name is exactly one of the keys.
Compound keywords
Section titled “Compound keywords”The lexer joins these word sequences, separated by one or more spaces, into a single token before keyword lookup:
compound-keyword = "don't see" | "dont see" | "go back" | "press enter" | "long press" | "double tap" | "before all tests" | "after all tests" | "before all" | "after all" | "before each test" | "after each test" | "before each" | "after each" | "on failure" | "with examples" | "clear app data" | "grant all permissions" | "revoke all permissions" | "allow permission" | "deny permission" | "set location" | "set language" | "set appearance" | "set airplane mode" | "verify external browser" | "add media" ;"dont see" (no apostrophe) is an alias of "don't see". Matching is greedy in the order above, so
before all tests wins over before all.
Filler words
Section titled “Filler words”Filler words are skipped by the forgiving parser at fixed positions (the start of a statement and
between operands). They make steps read like English: tap the "Login" button is
meant to read the same as tap "Login".
filler = "the" | "a" | "an" | "on" | "in" | "into" | "at" | "of" | "from" | "is" | "are" | "that" | "this" | "it" | "for" | "to" ;fillers = { filler } ;Fillers are only skipped where a production below writes fillers; they are not stripped
everywhere, and a few of them are given meaning by the production that follows
("in" in a positional selector, "of" in left of, "into" in read ... into).
2. File structure
Section titled “2. File structure”program = { top-item } ;top-item = use-decl | recipe-def | test-def | composite-def | hook-def | NEWLINE | skipped-token ;skipped-token = ? any other token at file level (for example a stray statement); it is ignored, not an error ? ;
text-operand = STRING | WORD | ID ; (* a "string-ish" operand: quoted, a bare word, or an #id *)Imports
Section titled “Imports”use-decl = "use" , text-operand , NEWLINE ;use "recipes/auth.probe"
test "file imports a recipe file" open the app see "Welcome"Recipes
Section titled “Recipes”recipe-def = "recipe" , text-operand , [ param-list ] , NEWLINE , body ;param-list = "(" , [ param , { [ "," ] , param } ] , ")" ;param = WORD | STRING ;recipe "sign in as" (email, password) type "<email>" into "Email" type "<password>" into "Password" tap "Continue" see "Dashboard"
test "calls a recipe with parameters" sign in as "user@test.com" with "secret123"A recipe without parameters:
recipe "dismiss onboarding" if "Skip" appears tap "Skip" wait for the page to load
test "calls a recipe without parameters" dismiss onboarding see "Home"Tests, tags and data tables
Section titled “Tests, tags and data tables”test-def = "test" , text-operand , test-head , [ examples-block ] ;test-head = TAG , { TAG } , NEWLINE , body (* tags on the header line *) | NEWLINE , INDENT , TAG , { TAG } , NEWLINE , { statement | NEWLINE } , DEDENT (* tags as the first body line *) | NEWLINE , body ;
examples-block = ( "with examples" | "examples" ) , [ COLON ] , ( "from" , fillers , [ STRING ] , NEWLINE (* external CSV file *) | NEWLINE , examples-table ) ;examples-table = INDENT , example-row , { example-row } , DEDENT ;example-row = example-cell , { example-cell } , NEWLINE ;example-cell = ? any token except NEWLINE and DEDENT; STRING cells contribute their unquoted content ? ;The first row of an inline table holds the column names; every following row is one run of the test. The examples block must directly follow the test it belongs to.
test "test with a name only" open the apptest "tags on the header line" @smoke @critical open the app see "Home"test "tags on the first body line" @regression @ui open the app see "Home"test "login validation" type "<email>" into "Email" type "<password>" into "Password" tap "Continue" see "<expected>"
with examples: email password expected "user@test.com" "pass123" "Dashboard" "" "pass123" "Email is required"test "login with CSV data" type "<email>" into "Email" tap "Sign In" see "<expected>"
with examples from "fixtures/users.csv"hook-def = hook-head , NEWLINE , body | hook-head , ignored-word , { ignored-word } , NEWLINE ; (* a header with a word after it: the parser drops the words, the hook has no body, and the indented lines after it are not its steps: a test keyword among the words makes them a test with no name, otherwise they are dropped and do not run; probe lint says which *)ignored-word = ? any token except NEWLINE and EOF, and except the word "test" right after "before all" or "after all" ? ; (* that one is an error: "before all test" is refused, see below *)hook-head = "before all tests" | "before all" | "after all tests" | "after all" | "before each test" | "after each test" | "before each" | "after each" | "on failure" | ( "before" | "after" ) , fillers ; (* bare form: a before-each / after-each hook *)before each and after each (with or without test) are the same hook; before all / after all
likewise. Before 0.16.7 the forms without test were accepted but their body was silently discarded.
A word after a hook header is accepted: the parser drops it, and probe lint warns with “unexpected word(s) after
the hook header are ignored”. The indented lines under such a header are never the hook’s: a test keyword among the words makes them a test with no name, and any other words drop them. Write the header on
its own line. The singular before all test and after all test is an error, because it used to read as a hook
with no body and its steps as a test with no name; write before all tests. A file that used the singular form
must be changed to the plural.
before all tests open the app tap "Accept Terms"
before each test see "Home"
after each test take screenshot "after_test"
after all tests log "suite done"
on failure take screenshot "failure_state" save logs dump tree
test "hooks wrap this test" tap "Settings" see "Account"The short suite forms:
before all open the app
after all log "done"
test "short suite hooks" see "Home"Composite (multi-device) tests
Section titled “Composite (multi-device) tests”composite-def = "composite" , [ "test" ] , text-operand , { TAG } , NEWLINE , [ INDENT , [ devices-block ] , { composite-item | NEWLINE } , DEDENT ] ;devices-block = "devices" , [ COLON ] , NEWLINE , [ INDENT , { device-decl | NEWLINE } , DEDENT ] ;device-decl = alias , COLON , { ? any token except NEWLINE ? } , NEWLINE ; (* the rest is the device target *)composite-item = sync-step | device-block | skipped-token ;device-block = alias , COLON , NEWLINE , [ INDENT , { statement | NEWLINE } , DEDENT ] ;alias = ? any single token directly followed by COLON ? ;sync-step = "sync" , [ text-operand ] , NEWLINE ;Steps in a composite test are only recognised inside an alias: block (and sync between blocks).
Anything else at composite level is skipped.
composite test "alice messages bob" @smoke devices: A: iPhone 15 Simulator B: Pixel 9 Emulator A: open the app tap "Sign in as Alice" B: open the app tap "Sign in as Bob" sync "both signed in" A: tap "New message" type "hello bob" into "Message" tap "Send" B: wait until "hello bob" appears3. Statements
Section titled “3. Statements”statement = fillers , ( system-dialog-step | line-step ) , NEWLINE | fillers , block-step ;
line-step = tap-webview-step | tap-native-step | tap-step | type-webview-step | type-native-step | type-step | long-press-step | double-tap-step | clear-step | toggle-step | drag-step | swipe-step | scroll-step | see-webview-step | see-native-step | see-step | dont-see-step | assert-defects-step | wait-step | open-app-step | open-link-step | close-step | restart-step | kill-step | clear-data-step | go-back-step | press-enter-step | press-key-step | shake-step | pause-step | log-step | rotate-step | permission-step | grant-revoke-step | copy-step | paste-step | set-location-step | set-language-step | set-appearance-step | set-airplane-step | verify-browser-step | add-media-step | take-screenshot-step | compare-screenshot-step | dump-tree-step | save-logs-step | store-webview-step | store-step | read-ai-step | deliver-signal-step | biometric-step | enroll-biometric-step | http-call-step | wait-response-step | see-response-step | store-response-step | see-requests-step | clear-requests-step | start-measuring-step | stop-measuring-step | see-metric-step | recipe-call-step ;
timed-step = line-step , "within" , ( INT | FLOAT ) , ( "seconds" | "second" | "s" | "ms" | "milliseconds" ) ; (* a line-step except recipe-call-step, with its own time budget *)
block-step = if-step | repeat-step | retry-step | dart-step | mock-step | travel-step ;
wait-step = wait-webview-step | wait-until-step | wait-any-step | wait-idle-step | wait-animations-step | wait-network-step | wait-duration-step | wait-id-step | wait-page-step ;
any-token = ? any token except NEWLINE and end of input ? ;rest-of-line = { any-token } ;Statement dispatch
Section titled “Statement dispatch”The parser reads the first token of a statement (after leading fillers) and chooses as follows. The first matching row wins.
| First token / condition | Production |
|---|---|
line contains system dialog, system field or sign in sandbox tester (outside strings) |
system-dialog-step |
open followed by app or link |
open-app-step, open-link-step |
open followed by anything else |
recipe-call-step (a recipe whose name starts with “open”) |
tap, type, see, don't see or wait until whose operand is followed by in webview |
tap-webview-step, type-webview-step, see-webview-step, wait-webview-step (checked before the rows below) |
tap followed by the word native |
tap-native-step; otherwise tap-step |
type followed by the word native |
type-native-step; otherwise type-step |
a line ending in within N seconds (or s, ms, milliseconds) |
the step before it, as timed-step (not for recipe calls or block steps) |
see / don't see / dont see followed by the word native |
see-native-step; otherwise see-step / dont-see-step |
assert |
assert-defects-step |
wait |
wait-step (see below for its order) |
swipe, scroll, long press, double tap, drag, toggle |
the step of the same name |
clear followed by a bare WORD |
recipe-call-step (a recipe whose name starts with “clear”) |
clear otherwise |
clear-step |
clear app data |
clear-data-step |
close, restart, kill, go back, press enter, shake, pause, log, rotate |
the step of the same name |
allow, deny, allow permission, deny permission |
permission-step |
grant, revoke, grant all permissions, revoke all permissions |
grant-revoke-step |
copy, paste, set location, set language, set appearance, set airplane mode, verify external browser, add media |
the step of the same name |
take, compare, dump, save |
take-screenshot-step, compare-screenshot-step, dump-tree-step, save-logs-step |
store followed by text and a css:"..." (or a quoted text) and in webview |
store-webview-step (checked before store-step) |
store, read, deliver |
store-step, read-ai-step, deliver-signal-step |
biometric, enroll |
biometric-step, enroll-biometric-step |
call |
http-call-step |
wait followed by response and a response reference |
wait-response-step (before the other wait forms) |
see followed by response and a response reference; see exactly N request(s) / see no request(s) and a reference |
see-response-step, see-requests-step (before the other see forms) |
store followed by response and a response reference |
store-response-step |
clear recorded request(s) |
clear-requests-step |
start measuring / stop measuring |
start-measuring-step / stop-measuring-step |
see followed by a performance phrase (memory below, cpu below, slow frames below, …) and a number |
see-metric-step (before the other see forms) |
if, repeat, retry, run, when, travel |
if-step, repeat-step, retry-step, dart-step, mock-step, travel-step |
| an empty line (NEWLINE) | nothing (ignored) |
press <key>: press directly followed by one of the eight keys and then the end of the line |
press-key-step (press enter is above); any other line that starts with press is a recipe call, as before |
anything else, including pinch, sync outside composite tests, unknown words |
recipe-call-step |
Inside wait-step the order is: until (then any, then network, idle, otherwise a text target),
idle, animations, network, page, a number, an #id, and finally the catch-all wait-page-step.
Selectors
Section titled “Selectors”selector = fillers , ( ordinal-selector | id-selector | text-selector | type-selector ) ;target-selector = id-selector | text-selector ; (* the only forms accepted by type, swipe, scroll and "until" *)
id-selector = ID ;text-selector = STRING , [ positional-suffix | relational-suffix ] ;positional-suffix = "in" , fillers , STRING ; (* "Price" in "Product Card" *)relational-suffix = ( "below" | "above" | ( "left" | "right" ) , [ "of" ] ) , fillers , [ STRING ] ;ordinal-selector = ORDINAL , fillers , [ STRING | ID | WORD ] , [ "in" , fillers , [ STRING ] ] ;type-selector = WORD ; (* widget type name: ElevatedButton *)text-selector: matches a widget whose text contains the string.id-selector: matchesKey('name')orSemantics(identifier: 'name').positional-suffix: the text must be inside the container widget.relational-suffix: the text is below / above / left of / right of the anchor.ordinal-selector: the n-th match; it composes with a text, an#idor a bare type word. A trailingin "Container"is parsed but ignored by the parser (the container is dropped).type-selector: a bare widget-type word (tap ElevatedButton). It is not written with angle brackets.
test "selector forms" tap "Login" tap #login_button tap ElevatedButton tap 2nd "Add" tap 1st #card tap 3rd item tap "Edit" in "Settings" tap "Submit" below "Email" tap "Submit" above "Footer" tap "Cancel" left of "Submit" tap "Help" right of "Submit"Modifiers
Section titled “Modifiers”if-visible = "if" , ( "visible" | "present" ) ; (* exactly lower case; skips the step when the target is absent *)optional-mod = "optional" ; (* always attempts the step; a failure becomes a warning *)The modifiers are written at the end of a step, if visible first, then optional. Which steps
accept them is listed on each production.
Tap and gestures
Section titled “Tap and gestures”tap-step = "tap" , selector , [ if-visible ] , [ optional-mod ] ;tap-native-step = "tap" , "native" , fillers , text-operand ; (* Android native UI *)long-press-step = "long press" , selector , [ if-visible ] , [ optional-mod ] ;double-tap-step = "double tap" , selector , [ if-visible ] , [ optional-mod ] ;clear-step = "clear" , selector , [ if-visible ] , [ optional-mod ] ;toggle-step = "toggle" , selector ;drag-step = "drag" , selector , fillers , selector ;swipe-step = "swipe" , fillers , [ direction ] , fillers , [ target-selector , [ swipe-start ] ] , [ swipe-distance ] ;swipe-start = "from" , INT , "percent" ; (* where the swipe starts, counted from the edge it starts at, a whole number from 1 to 100; only after a target-selector, and before swipe-distance; without it the swipe starts at the centre *)swipe-distance = "over" , INT , "percent" ; (* a whole number from 1 to 100, ending the line; without it the swipe travels 50 percent *)scroll-step = "scroll" , fillers , [ direction ] , fillers , [ target-selector ] , [ "until" , fillers , target-selector , fillers , [ "appears" | "visible" | "present" ] ] ;direction = "up" | "down" | "left" | "right" ; (* default: down *)drag separates its two selectors with the filler "to". scroll ... until keeps scrolling until
the target is on screen and is an error without a target.
test "tap family" tap "Submit" tap "Rate this app" optional tap "Aceptar" if visible tap "Cerrar" if visible optional tap native "Choose from Gallery" long press "Item" long press #row if visible double tap "Image" optional clear "Search" clear #email if visible clear 2nd "Field" toggle "Dark Mode" toggle #unit_toggle drag "Item A" to "Item B" drag #source to #targettest "swipe and scroll" swipe left swipe up on "Card" swipe down swipe right on #pager swipe down over 30 percent scroll down scroll scroll up on "ListView" scroll down until "Rate this app" appears scroll until #share_button is visible scroll down on "Feed" until "Footer" appearsTyping
Section titled “Typing”type-step = "type" , text-operand , fillers , [ target-selector ] , { "field" | "button" } , [ if-visible ] , [ optional-mod ] ;type-native-step = "type" , "native" , fillers , text-operand , fillers , text-operand ;test "typing" type "hello@world.com" into "Email" type "secret123" into the "Password" field type "42" into #age type "x" into "Field" if visible type "x" into "Field" optional type "no target" type native "wifi" into "Search settings"Assertions
Section titled “Assertions”see-step = "see" , ( see-tail , [ "with" , "ai" ] | see-any-tail ) , [ optional-mod ] ;dont-see-step = ( "don't see" | "dont see" ) , ( see-tail | see-any-tail ) , [ optional-mod ] ;see-any-tail = fillers , "any" , fillers , [ "of" ] , alt-list ; (* passes when any alternative is on screen; don't see: when none is *)see-native-step = ( "see" | "don't see" | "dont see" ) , "native" , fillers , text-operand ;
see-tail = fillers , [ "exactly" , [ INT ] , fillers ] , selector , { "button" | "field" | filler } , [ state-check ] , [ "matching" , text-operand ] ;state-check = "enabled" | "disabled" | "checked" | "focused" | "filled" | "empty" | "contains" , text-operand ;- At most one
state-checkis allowed, optionally followed by onematchingpattern (a regular expression). with aiis only valid onsee;don't see ... with aiis a parse error. Quote the sentence as the selector text.with aiis written in lower case and needs anai:block inprobe.yaml.- A text selector matches substrings. For an exact count or zero-state check, anchor the pattern:
see "0 ml" matching "^0 ml$".probe lintwarns, without failing, about a count or zero-state check that has no anchoredmatchingpattern.
test "assertions" see "Dashboard" see the "Submit" button see "Submit" is enabled see "Submit" is disabled see "Terms" is checked see "Email" is focused see #password_field is filled see #password_field is empty see "Price" contains "$9.99" see "Email" matching ".*@.*" see "Price" contains "$" matching "[0-9]+" see "0 ml" matching "^0 ml$" see exactly 3 "Item" see any of "Create Your Account", "Confirm Your Details" don't see any of "Error", "Failed" see exactly 2 2nd "Row" see #welcome_banner see 2nd "Item" see "Price" in "Product Card" see "Submit" below "Email" see "Maybe there" optionaltest "negative assertions" don't see "Error" dont see #spinner don't see exactly 3 "Item" don't see "Promo" optional see native "IMG_0001.jpg" don't see native "Error"WebView steps
Section titled “WebView steps”webview-operand = STRING | "css" , ":" , STRING ; (* a quoted string is the visible text of an element; css:"..." is a CSS selector *)webview-target = "in" , "webview" , [ STRING ] ; (* ends the step: no modifier may follow; STRING names the page *)tap-webview-step = "tap" , fillers , webview-operand , webview-target ;type-webview-step = "type" , STRING , fillers , webview-operand , webview-target ;see-webview-step = ( "see" | "don't see" | "dont see" ) , fillers , webview-operand , webview-target ;wait-webview-step = "wait" , fillers , "until" , fillers , webview-operand , "appears" , webview-target ;store-webview-step = "store" , fillers , "text" , fillers , [ "of" ] , fillers , "css" , ":" , STRING , "in" , "webview" , [ STRING ] , "as" , fillers , ( WORD | STRING ) ; (* reads the visible text of one element into a variable; a quoted text instead of css:"..." is an error, because the stored value would be the operand itself *)These steps reach content inside an app’s WebView (Android). The kind of the operand comes from the
syntax: a quoted string is text, css:"..." is a CSS selector, so a text such as #1 Item or
.5 off is text. in webview is recognised only when the bare word webview follows in; with a
quoted container (tap "Item" in "List") the line is an ordinary tap-step. The optional quoted page
name is a part of the page’s title or URL, for an app with more than one page. Nothing may follow it on
the line except a trailing within N seconds, which wraps the step like any other timed step: if visible,
optional, a second string or any other word are errors there, not ignored. The words in, webview and
css are matched without regard to case (In Webview, CSS:"x"), and spaces around the colon of
css : "x" are allowed. store text of css:"#total" in webview as total reads the visible text of one element into a variable. The operand is a CSS selector only (a quoted text would only give back itself, and is an error that says so); the optional page name comes after in webview as in the other forms, and as <name> ends the step: nothing may follow it, not even if visible or optional, though a trailing within N seconds is accepted as on the other steps. On Android the runner runs it: the text is cleaned of hidden characters (control, format, bidi and zero-width characters, separators, invalid UTF-8), cut at 1000 characters (hidden characters include variation selectors, Hangul fillers and the braille blank), and stored as a variable whose value came from a page: later steps may type it, look for it and use it in selectors, but the steps that send a string out of the device’s UI (open link, add media, set language, type native, screenshot names, call, mock, start measuring, system dialogs) refuse it. A name the parser or the runner manages (clipboard, response...) cannot be used. If the variable’s name has a secret word (token, password, pin, …), the whole stored text is also masked wherever it could be printed, when it has 3 or more characters; a part of it shown elsewhere is not masked on its own. A quoted string follows the lexer’s rules for every STRING: \" is a quote, \\ a
backslash, \n and \t a new line and a tab, and any other \x is the character x (so \: is :; a CSS
id such as #a\:b is written "#a\\:b"); a raw new line inside a string is a parse error. A text that
starts with css: stays text when it is quoted (tap "css:x" in webview). On Android the runner runs
these steps in the app’s own WebView (the app must have WebView debugging on); on iOS they fail with a
message that says so, and in a cloud run they are skipped like the other device-only steps.
test "webview steps" tap "Contact support" in webview tap css:"#submit" in webview tap "Pay" in webview "checkout" type "hello" into css:"#message" in webview type "42" into "Quantity" in webview "order form" see "Getting Started" in webview don't see css:".error" in webview wait until "Message sent" appears in webview wait until css:"#result" appears in webview "help" store text of css:"#total" in webview as total store text of css:"#total" in webview "checkout" as "order total"AI-assisted steps
Section titled “AI-assisted steps”assert-defects-step = "assert" , fillers , "no" , fillers , [ "visual" , fillers ] , "defects" , "with" , "ai" ; (* "no", "visual", "defects", "ai": lower case *)read-ai-step = "read" , fillers , text-operand , fillers , "with" , "ai" , "into" , fillers , ( WORD | STRING ) ;see "..." with ai is described under Assertions. All three steps need ai: configured in
probe.yaml. read ... with ai into <var> stores the extracted text in a variable that later steps read as
"<var>".
test "ai steps" see "the checkout total looks correct" with ai see "the banner is readable" with ai optional assert no visual defects with ai assert no defects with ai read "the 6-digit OTP code" with ai into otp type "<otp>" into "Code"wait-duration-step = "wait" , fillers , ( INT | FLOAT ) , fillers , [ "seconds" | "second" ] ;wait-until-step = "wait" , fillers , "until" , fillers , text-operand , fillers , [ "appears" , fillers , [ "matching" , fillers , STRING ] | "disappears" ] ; (* default: appears; matching = regex the text must match *)wait-any-step = "wait" , fillers , "until" , fillers , "any" , fillers , alt-list , fillers , [ "appears" ] ;alt-list = STRING , { alt-sep } , STRING , { { alt-sep } , STRING } ;alt-sep = filler | "," | "or" ;wait-idle-step = "wait" , fillers , ( "until" , fillers , "idle" | "idle" , rest-of-line ) ;wait-animations-step = "wait" , fillers , ( "animations" | "animation" ) , rest-of-line ;wait-network-step = "wait" , fillers , ( "until" , fillers , "network" , fillers | "network" , rest-of-line ) ;wait-id-step = "wait" , fillers , ID , fillers , [ "appears" | "disappears" ] ;wait-page-step = "wait" , fillers , [ "page" ] , rest-of-line ; (* "page ..." and the catch-all *)wait-any-stepneeds at least two quoted alternatives, separated by commas,oror just spaces.wait-page-stepis the catch-all: anywait ...line that no earlier form matched (wait for the page to load,wait for the app to be idle,wait for "Foo") waits for the page to settle.wait until networkis the only until-form for the network; a trailingis idleleaves the wordidlebehind as a stray recipe call. Writewait for network idle.- Numbers need no unit:
wait 2equalswait 2 seconds.
test "waits" wait 5 seconds wait 1 second wait 1.5 seconds wait 2 wait until "Dashboard" appears wait until "0 ml" appears matching "^0 ml$" wait until "Loading" disappears wait until Dashboard wait until #spinner disappears wait until #home_title appears wait #toast disappears wait until any of "Got it", "Login", "Home" appears wait until any of "A" or "B" or "C" wait for idle wait until idle wait for animations to end wait for network idle wait until network wait for the page to load wait for the app to be idleSystem dialogs
Section titled “System dialogs”OS-level dialogs live outside the Flutter widget tree. The parser recognises these statements by
scanning the whole line (outside strings) for system dialog, system field or
sign in sandbox tester; the plain words around the quoted strings are ignored. The first word picks
the operation, the quoted strings are taken in order.
system-dialog-step = system-tap-step | system-type-step | system-see-step | system-wait-step | system-dismiss-step | system-sandbox-step ;system-tap-step = "tap" , STRING , fillers , "system" , "dialog" , [ STRING ] , [ optional-mod ] ;system-type-step = "type" , STRING , fillers , "system" , "field" , STRING , [ fillers , "system" , "dialog" , STRING ] , [ optional-mod ] ;system-see-step = ( "see" | "don't see" | "dont see" ) , "system" , "dialog" , [ STRING ] , [ optional-mod ] ;system-wait-step = "wait" , fillers , "system" , "dialog" , [ STRING ] , [ "appears" | "disappears" ] , [ optional-mod ] ;system-dismiss-step = "dismiss" , "system" , "dialog" , [ STRING ] , [ optional-mod ] ;system-sandbox-step = "sign" , "in" , "sandbox" , "tester" , [ optional-mod ] ;The optional trailing string is a dialog-title filter. A value typed into a system field may be an
environment variable ("$NAME" or "${NAME}") and is always masked in output.
test "system dialogs" tap "Allow" in system dialog tap "OK" in system dialog "Apple Account" type "$PROBE_SANDBOX_PASSWORD" into system field "Password" see system dialog "Sign in to Apple Account" don't see system dialog "Notifications" wait for system dialog "Notifications" appears wait for system dialog "Notifications" disappears dismiss system dialog sign in sandbox tester tap "Allow" in system dialog optionalApp lifecycle, device actions and utilities
Section titled “App lifecycle, device actions and utilities”open-app-step = "open" , fillers , "app" ;open-link-step = "open" , fillers , "link" , fillers , text-operand , [ fillers , "app" ] ;close-step = "close" , fillers , [ "app" | WORD | STRING ] ;restart-step = "restart" , fillers , [ "app" ] ;kill-step = "kill" , fillers , [ "app" ] ;clear-data-step = "clear app data" ;go-back-step = "go back" ; (* takes no arguments: words after it on the same line are a separate step *)press-key-step = "press" , key-name ; (* a key the agent presses, ending the line: the list below; no fillers, so "press the tab" is a recipe call *)key-name = "tab" | "escape" | "backspace" | "space" | "up" | "down" | "left" | "right" ; (* the eight keys; a line that is press and another word is a recipe call *)press-enter-step = "press enter" ; (* the keyboard action key on the focused text field *)shake-step = "shake" ;pause-step = "pause" ;log-step = "log" , [ STRING ] ;rotate-step = "rotate" , fillers , [ WORD | STRING ] ; (* default: portrait; landscape | portrait *)
permission-step = ( "allow permission" | "deny permission" | "allow" | "deny" ) , fillers , [ "permission" | "permissions" ] , fillers , STRING , fillers , [ "permission" | "permissions" ] ;grant-revoke-step = ( "grant all permissions" | "revoke all permissions" | "grant" | "revoke" ) , fillers , [ "all" ] , fillers , [ "permission" | "permissions" ] ;
copy-step = "copy" , fillers , [ STRING ] , rest-of-line ; (* "to clipboard" is swallowed *)paste-step = "paste" , rest-of-line ; (* "from clipboard" is swallowed *)set-location-step = "set location" , fillers , coordinate ;set-language-step = "set language" , fillers , STRING ; (* "de", "pt-BR", "system" *)set-appearance-step = "set appearance" , fillers , ( "light" | "dark" | "auto" | STRING ) ; (* the STRING must be one of those three; "auto" is refused on iOS when the step runs *)set-airplane-step = "set airplane mode" , [ "to" ] , ( "on" | "off" | STRING ) ; (* the STRING must be "on" or "off"; a state is not restored at the end of the test *)verify-browser-step = "verify external browser" , rest-of-line ; (* "opened" is swallowed *)add-media-step = "add media" , fillers , text-operand ;
coordinate = [ "-" ] , number , ( "," , [ "-" ] , number | number ) ;number = INT | FLOAT ;open-link-stepends with an optionalin the app(alsointo the app,in app): the URL is routed through the OS intent / URL handler instead of the external browser.open "x"(anything afteropenthat is notapporlink) is a recipe call.closetakes the wordkeyboard, the wordapp, or a quoted name.rotatelower-cases its argument;landscapeandportraitare the ones the runtime understands.coordinateislatitude, longitude. The comma may be omitted only when the longitude is positive (37 -122is rejected,37, -122is fine). Values are validated at run time, not by the parser.
test "app lifecycle" open the app open app restart the app restart kill the app kill clear app data close the app close keyboard close "Dialog" close go back press enter shake pause log "checkpoint reached" log rotate landscape rotate portrait rotatetest "deep links and browser" open link "https://example.com" verify external browser opened open link "myapp://profile/42" in the app open link "myapp://profile/43" into the app open link "myapp://profile/44" in apptest "permissions" allow permission "notifications" deny permission "camera" allow "microphone" grant all permissions revoke all permissionstest "clipboard" copy "user@example.com" to clipboard paste from clipboard type "<clipboard>" into "Email"test "location and media" set location 37.7749, -122.4194 set language "de" tap l10n "saveButton" set location -33.8688, 151.2093 add media "fixtures/photo.jpg"Screenshots, diagnostics and variables
Section titled “Screenshots, diagnostics and variables”take-screenshot-step = "take" , fillers , [ "screenshot" ] , [ "called" , text-operand | STRING ] ;compare-screenshot-step = "compare" , fillers , [ "screenshot" ] , fillers , ( "called" , text-operand | STRING ) , [ "of" , selector ] ;dump-tree-step = "dump" , fillers , [ "widget" ] , [ "tree" ] ;save-logs-step = "save" , fillers , [ "device" ] , [ "logs" ] ;store-step = "store" , fillers , text-operand , fillers , [ "as" ] , fillers , [ WORD | STRING ] ;deliver-signal-step = "deliver" , fillers , "signal" , fillers , text-operand , fillers , [ STRING ] ;compare screenshot requires a name. of <selector> scopes it to one widget. deliver signal
defaults its value to "true".
test "screenshots and diagnostics" take screenshot "checkout_page" take screenshot called "named" take screenshot compare screenshot "baseline" compare screenshot "price_tag" of "Price Label" compare screenshot called "hero" dump tree dump the widget tree save logs save device logstest "variables and signals" store "Alice" as username type "<username>" into "Name" deliver signal "payment_ready" deliver signal "payment_ready" "true"Biometrics
Section titled “Biometrics”biometric-step = "biometric" , fillers , [ "no" ] , fillers , [ "match" ] ;enroll-biometric-step = "enroll" , fillers , [ "biometric" ] ;test "biometrics" enroll biometric biometric match biometric no matchHTTP calls
Section titled “HTTP calls”http-call-step = "call" , fillers , [ "get" | "post" | "put" | "delete" ] , fillers , [ STRING ] , fillers , [ "with" , fillers , [ "body" ] , fillers , [ STRING ] ] ; (* method default: GET *)Calls run on the CLI, not on the device. The response is available as "<response.status>" and
"<response.body>".
test "http calls" call GET "https://api.example.com/health" call POST "https://api.example.com/seed" with body "{\"env\":\"test\"}" call PUT "https://api.example.com/users/1" with body "{\"name\":\"updated\"}" call DELETE "https://api.example.com/sessions" see "<response.status>"4. Block statements
Section titled “4. Block statements”A block statement is a header line followed by an indented block (body).
Conditionals
Section titled “Conditionals”if-step = "if" , fillers , ( text-operand , fillers , [ "appears" ] | "response" , response-ref , response-check ) , NEWLINE , body , [ ( "otherwise" | "else" ) , NEWLINE , body ] ;The condition is a text (or #id) that may be on screen, or the newest recorded backend response matching a
response reference (if response "/api/me" json "data.plan" equals "pro"; false when there is none). otherwise / else must follow the if
body directly, at the same indentation as the if.
test "conditionals" if "Accept Cookies" appears tap "Accept Cookies" if "Welcome Back" appears tap "Continue" otherwise tap "Sign In" if #promo_banner appears tap "Close" else log "no promo" if "Outer" appears if "Inner" appears tap "Inner"Loops and retries
Section titled “Loops and retries”repeat-step = "repeat" , [ INT ] , fillers , [ "times" ] , NEWLINE , body ; (* count default: 1 *)retry-step = "retry" , [ INT ] , fillers , [ "times" ] , NEWLINE , body ; (* attempts default: 1 *)repeat always runs every iteration. retry re-runs the whole block on failure, up to N attempts, and
stops at the first success.
test "loops and retries" repeat 3 times swipe left wait 1 second repeat 2 tap "Next" retry 3 times tap "Submit" see "Success" retry tap "Refresh"Dart escape hatch
Section titled “Dart escape hatch”dart-step = "run" , fillers , [ "dart" ] , [ COLON ] , NEWLINE , [ INDENT , { dart-line | NEWLINE } , DEDENT ] ;dart-line = { ? any token except NEWLINE and DEDENT ? } , NEWLINE ;The block must start with run (a bare dart: is not a statement). Each line is rebuilt from its tokens
joined by single spaces, so the lexer’s ignored characters (=, ;, ., braces, …)
are not part of the captured code.
test "dart block" open the app run dart: print("hello") see "Dashboard"HTTP mocking
Section titled “HTTP mocking”mock-step = "when" , { ? any token except get, post, put, delete, STRING, ID and NEWLINE ? } , [ "get" | "post" | "put" | "delete" ] , [ STRING | WORD ] , NEWLINE , [ INDENT , [ respond-clause ] , NEWLINE , DEDENT ] ;respond-clause = "respond" , fillers , [ "with" ] , [ INT | "network" , fillers , [ "failure" ] ] , fillers , [ "and" ] , [ "body" ] , [ STRING ] , [ "after" , INT , [ "seconds" | "second" | "ms" ] ] ;The words between when and the method (the app calls) are ignored. The method (get, post, put,
delete, or the plain words patch, head, options) defaults to GET, the status to 200. The indented body
is exactly one respond line. after N seconds delays the answer; respond with network failure drops the
connection instead of answering. The path is matched like a response reference. Mocks are
real: the app’s dart:io HTTP client gets the status, body and delay (agent 0.22+), they last for one test and
are re-applied if the app restarts.
test "http mocking" when the app calls POST "/api/auth/login" respond with 503 and body "{ \"error\": \"Service Unavailable\" }" when the app calls GET "/api/users" respond with 200 and body "[]" when the app calls DELETE "/api/session" respond with 204 when the app calls GET "/api/orders" respond with 200 and body "[]" after 3 seconds when the app calls patch "/api/profile" respond with network failure tap "Sign In"Time budget
Section titled “Time budget”within gives one step its own time budget: it becomes the step’s timeout and the window in which a missing target is
retried (like --implicit-wait for that step only). It is a plain word recognised only at the end of a line.
test "time budgets" tap "Export" within 5 seconds wait until "Report ready" appears within 90 seconds wait for response "/api/export" status 200 within 120 seconds see "Done" within 500 msPerformance
Section titled “Performance”start-measuring-step = "start" , "measuring" , [ STRING ] ;stop-measuring-step = "stop" , "measuring" ;see-metric-step = "see" , metric-phrase , number , [ unit ] ;metric-phrase = "memory" , "below" (* peak memory of the app, MB *) | "memory" , "growth" , "below" (* memory at the end minus at the start, MB *) | "cpu" , "below" (* average CPU, percent of one core *) | "cpu" , "peak" , "below" (* busiest second *) | "slow" , "frames" , "below" (* percent of frames over 16.7 ms *) | "frame" , "time" , "below" (* 95th percentile frame time, ms *) | "slowest" , "frame" , "below" (* slowest frame, ms *) | "data" , "transferred" , "below" ; (* request + response bytes, KB *)unit = "MB" | "KB" | "GB" | "ms" | "percent" | "%" ;start measuring opens a window that records, until stop measuring, the CPU of the app (read by the CLI from the device),
its memory and frame timings (read by the agent) and the HTTP traffic. A see ... metric step checks the window that is open
(the numbers so far) or else the last one that was closed. All limits are upper bounds. start, stop, measuring, growth,
peak, slow, frames, slowest, transferred and the other words are plain words recognised only in these positions.
test "checkout stays light" open the app start measuring "checkout" tap "Pay" wait for response POST "/api/pay" status 200 within 10 seconds stop measuring see memory below 300 MB see memory growth below 20 MB see cpu below 60 percent see slow frames below 5 percent see data transferred below 500 KB see response POST "/api/pay" below 2000 msBackend responses
Section titled “Backend responses”The agent records the app’s dart:io HTTP traffic (the http package, dio’s default adapter and most Dart clients; not
WebViews, dart:html or native SDK calls) so a test can react to what the backend returned.
response-ref = [ http-method ] , fillers , STRING ; (* GET "/api/orders", "/api/me", "/api/orders/*" *)http-method = "get" | "post" | "put" | "delete" | "patch" | "head" | "options" ;response-check = "status" , INT | "below" , number , [ "ms" | "milliseconds" | "s" | "seconds" ] (* the response took less *) | "contains" , STRING | "json" , STRING , ( "equals" , ( STRING | number ) | "exists" ) ;
wait-response-step = "wait" , fillers , "response" , response-ref , [ "status" , INT ] ;see-response-step = "see" , fillers , "response" , response-ref , response-check ;store-response-step = "store" , fillers , "response" , response-ref , "json" , STRING , fillers , "as" , fillers , ( WORD | STRING ) ;see-requests-step = "see" , ( "exactly" , INT | "no" ) , fillers , ( "request" | "requests" ) , response-ref ;clear-requests-step = "clear" , "recorded" , ( "request" | "requests" ) ;The STRING of a reference is a path with * wildcards that must match the whole path ("/api/orders" does not match
/api/orders/42; use "/api/orders/*"), or a full URL when it contains ://; the query is ignored unless the pattern
contains ?. The method is optional. response, request, requests, status, json, equals, exists, recorded and
no are plain words recognised only in these positions.
wait for response waits for a matching exchange that no earlier wait for response has taken (so
tap then wait for response works even when the answer came quickly); see response, store response and
if response look at the newest matching exchange. A JSON path is dotted with optional [n] indexes
(data.plan, items[0].id); equals compares the value as text (42, true, null). store response puts the value in a
variable usable as <name> afterwards. The log and the mocks start empty in every test.
test "premium users see the premium badge" open the app wait for response GET "/api/me" status 200 store response "/api/me" json "data.plan" as plan see exactly 1 request GET "/api/me" if response "/api/me" json "data.plan" equals "pro" see "Premium" otherwise see "Upgrade" clear recorded requestsGPS routes
Section titled “GPS routes”travel-step = "travel" , fillers , NEWLINE , [ INDENT , { waypoint-line | NEWLINE } , DEDENT ] , [ "over" , [ number ] , fillers , [ "seconds" | "second" ] , NEWLINE ] ;waypoint-line = coordinate , NEWLINE ;The optional over N seconds line is a sibling of travel to, at the same indentation, not part
of the waypoint block.
test "gps route" travel to 37.7749, -122.4194 37.7849, -122.4094 37.7949, -122.3994 over 10 seconds see "Arrived"5. Recipe calls
Section titled “5. Recipe calls”recipe-call-step = call-token , { call-token } ;call-token = STRING | INT | ? any other token except NEWLINE and DEDENT ? ;Any line that is not one of the statements above is a recipe call. The parser records the words
lower-cased and joined by spaces; every quoted string becomes the placeholder <arg> in the name and an
argument in order. A bare number is kept both ways: as part of the name (step 2 of onboarding) and as
an argument (increment counter "Likes" 3); the runtime tries the name first.
recipe "sign in as" (email, password) type "<email>" into "Email" type "<password>" into "Password"
recipe "dismiss onboarding" tap "Skip"
recipe "increment counter" (label, amount) tap "<label>" log "<amount>"
recipe "step 2 of onboarding" tap "Next"
recipe "clear search" clear "Search"
recipe "open most recent post" tap 1st "Post"
test "recipe call forms" dismiss onboarding sign in as "a@b.co" with "pw" sign in as "a@b.co" and "pw" increment counter "Likes" 3 step 2 of onboarding clear search open most recent post the dismiss onboarding6. Lexical conventions in practice
Section titled “6. Lexical conventions in practice”# A comment line: ignored, may start with anything, even a lettertest "comments and strings" # inline comment: "#" followed by a space tap "Submit" # trailing comment tap #submit_button # "#id" (a letter after #) is a selector, not a comment type "line one\nline two" into "Notes" type "tab\there" into "Notes" type "say \"hi\"" into "Notes" tap "has # hash inside a string" wait 1.5 seconds tap 3rd "Item"test "case-insensitive keywords" TAP "Login" Wait Until "Dashboard" Appears Don't See "Error" See Exactly 2 "Item"test "tabs count as four spaces" tap "Tab indented" if "Dialog" appears tap "OK"recipe "looking-for item" tap "Search"
test "hyphens and apostrophes in words" looking-for item don't see "Error"7. Placeholders and variables
Section titled “7. Placeholders and variables”Variables are written inside strings as "<name>"; the parser never looks inside a string, the runtime
substitutes them when the step runs.
placeholder = "<" , placeholder-name , ">" ; (* only inside the content of a STRING *)placeholder-name = variable | "clipboard" | "response.status" | "response.body" | random ;variable = identifier ; (* recipe parameter, example column, store target, read-into target *)random = "random.email" | "random.name" | "random.phone" | "random.uuid" | "random.number" , [ "(" , INT , "," , INT , ")" ] | "random.text" , [ "(" , INT , ")" ] ;| Source of a variable | How it is bound |
|---|---|
| recipe parameter | by position from the quoted arguments of the call |
with examples column |
one value per row, by column header |
store "v" as name, read ... into name |
assigned when the step runs |
paste from clipboard |
"<clipboard>" |
call ... |
"<response.status>", "<response.body>" |
"<random.*>" |
generated fresh each time the string is resolved |
An unquoted <name> is a lexical error, because it would otherwise be typed or matched as literal text.
recipe "fill profile" (name, city) type "<name>" into "Name" type "<city>" into "City"
test "placeholders" fill profile "Alice" with "Lisbon" type "<random.email>" into "Email" type "<random.name>" into "Full Name" type "<random.phone>" into "Phone" type "<random.uuid>" into "Reference" type "<random.number(1,100)>" into "Age" type "<random.text(8)>" into "Code" store "Alice" as username see "<username>"Filler words around operands are skipped:
test "filler words" the tap "Login" tap the "Login" type "x" into the "Email" field see the "Welcome" button scroll down on the "Feed" wait for the page to load8. Semantic notes that EBNF cannot express
Section titled “8. Semantic notes that EBNF cannot express”Environment variables. At run time ${NAME} inside any quoted text (a STRING, including selectors) is
replaced by the environment variable NAME; the step line prints the template and the value is scrubbed from
errors. The parser does not see this: a ${...} is ordinary string content to the grammar.
Bare type and clear act on the focused field. type "x" without into, and clear without a
selector, target the text field that has focus (an error when none has).
Loose text matching. With --match-loose (or defaults.match: loose) text selectors compare after folding case,
accents and other diacritics, typographic apostrophes and dashes, full-width forms, invisible characters and whitespace
(the same folding on both sides; ids are never folded). It is a run setting, not syntax.
Implicit wait. With --implicit-wait <d> (or defaults.implicit_wait) a tap, type, long press, double tap,
clear, drag or plain see whose target is not on screen is retried for up to <d> before it fails. It is a run
setting, not syntax; steps with if visible / optional, don't see and the wait steps are never retried.
Text selectors match substrings. A STRING selector matches every widget whose text contains it
(see "0 ml" is satisfied by “250 ml”). matching "<regex>" then requires that at least one of the matched
widgets has text matching the regular expression, so see "0 ml" matching "^0 ml$" is the exact form.
Statement boundaries. A statement ends at the end of its line. A built-in step consumes only what its
production names; any tokens left on the line are parsed as a separate statement, which is usually a
recipe call that fails at run time with “unknown recipe call”. This is how tap the "Login" button
(trailing button), wait 1 minute (trailing minute) and wait until "x" is gone (trailing gone)
go wrong. Lines never continue onto the next line.
Ordered choice and lookahead. The grammar is ambiguous as written; the parser resolves it by the
dispatch order above, by peeking at the rest of the line (system dialogs, open, clear), and by the
lexer’s compound-keyword lookahead (before all tests is one token, before all another).
Case. Keywords and contextual words (native, dismiss, system, dialog, devices, any, or,
match) match case-insensitively. The words visible, present, ai, no, visual and defects
must be lower case. Recipe names are stored exactly as written in recipe "Name", while a call is
lower-cased, so give recipes lower-case names.
Filler stripping. At parse time fillers are skipped only where the grammar says fillers, and at the
start of every statement. Inside a recipe call they are kept as words. At run time the recipe lookup
additionally ignores the words and, with, the, then and the <arg> markers, in both the call and
the definition name, so sign in as "a" with "b" finds recipe "sign in as". The other connecting words
(into, to, …) are not ignored by the lookup.
Recipe resolution. Recipes come from the file itself and from used files and the project’s
recipes_folder. A call is resolved by exact name, then with <arg> and the words above stripped, then
with the same stripping applied to definition names; a call containing a bare number is tried with the
number as part of the name first and as an argument second. No match is a run-time error. Arguments are
bound to parameter names by position into a flat variable map; there is no per-recipe scope.
Swipe direction. The direction is optional: a bare swipe runs a down swipe. In its place the parser takes up, down, left or right (in any case), or else a quoted name or #id that starts the target, over that starts the distance, or the end of the line. Any other token there is a parse error that lists the four: a bare word (swipe sideways, swipe Menu), a number (swipe 5) or a keyword (swipe tap), and a word followed by a target too (swipe sideways on "Menu"). Before, those ran a down swipe and the token was read as the next step. An element name goes in quotes: swipe up on "Menu". The EBNF’s direction rule lists these four words, and the parser now enforces that in this place. A word after the direction is not checked: swipe up sideways still reads sideways as the next step, and a quoted swipe "up" is read as the up direction, not as an element named up.
Swipe distance. The range (a whole number from 1 to 100) and the line end of over N percent are checked by the parser, which the EBNF cannot express; anything else after over is a parse error. The agent travels the distance from the element the swipe starts on, or from the screen, and runs 50 percent when the step leaves it out. The swipe starts at the centre of that element or screen, so above 50 percent the finger ends outside it: 50 percent ends at the edge and 100 percent ends one half-size past it. The travelled distance has not been measured on a device yet.
Swipe start. from N percent (a whole number from 1 to 100) sets where a swipe starts, counted from the edge it starts at: the bottom for swipe up, the top for swipe down, the left for swipe right and the right for swipe left. It applies only to a swipe on a target written in quotes (swipe right on "Menu" from 2 percent): a bare swipe with from, or a name without quotes before it, is a parse error. The parser checks the range and that from comes before over and ends the line after over N percent, which the EBNF cannot express. Without it the swipe starts at the centre of the target, as before. An agent that does not report the swipe_from capability in its handshake refuses the step, so an older probe_agent never swipes from the centre by mistake. The start is computed in the target’s own (local) coordinates, so a target under a scale transform starts at the wrong point: under a 2x scale, from 50 percent lands at x = 50 instead of 100. Rotated targets are untested, the same as for over N percent.
Silent leniency. The parser prefers to accept and default over reporting an error. These are accepted
without complaint and yield empty values: use, recipe, test or composite without a name; if
without a condition; a missing selector (tap); store without a name; log without a message; a
with examples from without a file name; a recipe parameter list that is never closed (it swallows the
rest of the file); unexpected tokens at file level and inside composite bodies;
Inconsistent indentation does not raise an error
either: an over-indented line becomes a recipe call named <indent> ... and a stray dedent can end a
block early.
Hooks. Hooks are file-scoped. before all and after all run once; before each / after each
run around every test; on failure runs after a failed test.
Examples. with examples repeats the preceding test once per row; "<column>" placeholders in the
body are replaced by that row’s cells. Row width is not validated against the header.
Platform notes. Native UI steps (tap native, type native, see native) and the WebView steps (in webview) are Android only.
Location, travel, media, biometric and permission steps apply to simulators and emulators.
Where the docs and the parser disagree
Section titled “Where the docs and the parser disagree”These are behaviours of the parser at the time of writing, found by cross-checking the Syntax, Dictionary, Recipes, Hooks and Data-driven pages against the source. The grammar above follows the parser.
| Documented | What the parser does | Use instead |
|---|---|---|
type <email> into "Email", see <expected> (unquoted placeholders in Recipes and Data-driven pages) |
lexical error: “unquoted placeholder” | type "<email>" into "Email" |
a recipe named log in as, called as log in as "u" with "p" (earlier versions of the Recipes, Hooks and tutorial pages and of the probe init scaffold) |
log is a keyword: parsed as log plus a call named as <arg> with <arg>; probe lint warns on the declaration |
name recipes so the first word is not a keyword (sign in as) |
tap the "Login" button “is equivalent to” tap "Login" (Dictionary, Filler Words) |
trailing button / field is only consumed by type, see and don't see; after tap it becomes a stray recipe call |
tap "Login" |
see 3 "Item" (Syntax, Assertions) |
no count is parsed; an empty assertion plus a recipe call 3 <arg> |
see exactly 3 "Item" |
tap <ElevatedButton> (Syntax, Selectors table) |
lexical error (unquoted <) |
tap ElevatedButton |
dart: on its own line (Syntax, Dart Escape Hatch) |
not a statement; becomes junk recipe calls | run dart: |
state suffixes “compose”: see "F" is enabled contains "y" matching "z" (Dictionary) |
one state check, then optional matching; the second state word starts a stray recipe call |
see "F" is enabled, or see "F" contains "y" matching "z" |
wait until network is idle, wait for the app to be idle (parser comments) |
the first leaves a stray idle recipe call; the second is the page-load catch-all |
wait for network idle, wait for idle |
filler words list lacks to (Dictionary) |
to is a filler (needed by drag A to B, travel to) |
n/a |
| recipe args “matched using connecting words like and, with, or into” (Recipes) | the run-time lookup ignores and, with, the, then only |
use and / with |
Press and Pinch DSL steps (Annotations) |
press <key> is a step for tab, escape, backspace, space, up, down, left and right on a line of its own (a recipe named exactly one of those keys is shadowed); other press lines are recipe calls; pinch is a reserved word with no statement |
go back |
a Dart block “executes arbitrary Dart” (Syntax) |
code is re-assembled from tokens; characters such as =, ;, . and braces never reach the agent |
keep Dart snippets to token-safe code |
ordinal ... in "Container" (Annotations: Ordinal(2, 'Item', container: 'List')) |
the container is parsed but dropped | use a positional selector |
rotate landscape / rotate portrait (Dictionary) |
any word or string is accepted; unknown values are only rejected at run time | landscape or portrait |
Keyword reference
Section titled “Keyword reference”All reserved words are in the keyword production; the compound forms are in
compound-keyword. The words below are not reserved, but the parser gives them
a meaning in one position: native, visible, present, ai, any, or, no, match, visual,
defects, devices, system, dialog, dismiss, sign, sandbox, tester, up, down.