nolindnaidoo/pixelactions

pixelactions

The executor half of the pixelcoords loop: consume human-verified coordinates, perform the interaction, confirm it landed.

pixelcoords freezes your screen, lets you mark labeled regions, and writes pixel-exact coordinates with crops, drift re-location, and point verification. pixelactions reads that session and acts on it — referencing regions by label, never by raw coordinate, so a run survives the UI moving.

find  →  act  →  assert

Status

Early, and macOS only. The loop works end to end: resolve a label to its click point, re-locate it against a fresh capture, act, and confirm. Windows and X11 are next; nothing here is published yet.

Three ways to drive it

One binary, three surfaces, ranked. Most people want the first. Here is the same task in each — fill a field and confirm the result.

1. Command line

pixelactions run --session ~/captures/checkout \
  click:email type:"a@b.com" key:enter verify:success --yes

Nothing to install, nothing to keep in sync. Verbs chain in one invocation, which also means one relocation pass for the whole sequence.

2. A flow file

session = "~/captures/checkout"

[[step]]
action = "click"
target = "email"

[[step]]
action = "type"
text = "a@b.com"

[[step]]
action = "key"

[[step]] action = "verify" target = "success"

pixelactions plan --flow checkout.toml # every coordinate, acts on nothing pixelactions run --flow checkout.toml --yes

Same verbs as the command line. Reviewable in a diff — a pull request
shows *click submit*, not arithmetic.

### 3. The line protocol

ui.send(do="click", target="email") ui.send(do="type", text=row["email"]) ui.send(do="key", chord="enter") if ui.send(do="verify", target="success")["outcome"] != "verified": failures.append(row)