QA With flyto: Browser Automation for ouou

Why flyto

First, the plain definition. flyto is not something we built just for ouou QA. It is an existing tool called flyto-core that you can install and run on your own machine.

For this work, the useful mental model is: two CLIs, two jobs.

1. The standalone CLI runner executes YAML workflows from the terminal, for example uv run flyto run workflows/check-publish-sync-gate.yaml.
2. The MCP server exposes the same browser modules to an AI coding agent, so the agent can inspect the live app, click around, and draft workflow steps interactively.

We use both. The CLI runner is for repeatable regression checks. The MCP mode is for exploration, debugging, and authoring those checks faster.

ouou is an offline-first PWA. The interesting behavior — card drafts, tag sync, share-target routing — lives in IndexedDB and React state, not in server responses. Standard API-level tests do not reach it. We needed something that drives a real browser.

We are experimenting with flyto-core. Workflows are plain YAML files, the browser is Playwright under the hood, and the module library covers the full loop: launch → navigate → interact → assert → screenshot → close. No custom test runner, no extra build step.

flyto vs Playwright

The honest comparison: flyto uses Playwright under the hood. The difference is not in what the browser can do — it is in the cost of authoring and the feedback loop when writing tests with an AI agent.

Authoring speed with an AI agent. A Playwright TypeScript test requires the agent to write a full file: imports, test function scaffolding, async/await, expect() calls, error handling. flyto workflows are YAML — each step is a module call with named params. The agent produces a working step in one shot because there is nothing to scaffold. In practice, a five-step workflow takes one agent turn; the equivalent Playwright test takes three to four because each attempt requires a write-run-fix cycle.

The MCP exploration loop. Before writing any assertion, you need to know what selectors exist on the page. With Playwright you write a test, run it, read the failure, fix the selector, repeat. With flyto via MCP, the agent calls browser.evaluate against the live app and reads the real DOM in a single tool call — no test file, no run, no error log. The discovery phase that takes ten minutes in Playwright takes thirty seconds with flyto MCP.

Composable setup without fixtures. Playwright has a fixture system for shared setup. It works, but it requires TypeScript configuration and lives outside the test file. flyto uses flow.invoke — a sub-workflow is just another YAML file, called with params. The same bootstrap workflow runs standalone, in CI, and as a step inside a larger workflow. There is nothing to configure.

Where Playwright is stronger. Playwright TypeScript has a type system — misnamed selectors and wrong params fail at compile time, not at runtime. It has a rich ecosystem: built-in visual diff, network interception, HAR recording, trace viewer. For a large test suite maintained by a team of engineers it scales better. flyto workflows are harder to refactor at scale and have no type safety.

Our conclusion. flyto is the right tool for AI-assisted authoring of end-to-end regression workflows against an offline-first app. For a traditional engineering team writing and maintaining a large browser test suite, Playwright TypeScript is the stronger long-term choice. The two are not mutually exclusive — flyto for AI-authored regression probes, Playwright for structured test suites.

Example Workflows

Workflow	What it verifies
bootstrap-login	Logs in with QA credentials; idempotent — skips if session exists
bootstrap-create-deck	Creates “QA Bootstrap Deck” via the home-screen UI; idempotent
check-add-card-filter-tag-sync	Persisted filter tags appear on /add and resync after leaving and returning
check-add-card-add-another	”Add Another” clears the draft but reapplies the filter-selected tags
check-add-card-share-target	/?view=add shortcut routing injects share-target fields correctly
check-publish-sync-gate	Publish button stays disabled until cloud sync is enabled and the deck is synced
check-home-published-decks-section	A deck moves from Your decks to Published decks after publish, then back after unpublish

The check-* workflows call the bootstrap workflows as their first steps via flow.invoke, so they self-prepare a known state without duplicating setup code.

Running a Workflow

Start the dev server on an allowed port — flyto’s SSRF guard blocks 5173 by default:

npm run dev -- --port 8080

Export the local mode flag once in your terminal session, then run any workflow:

export FLYTO_VSCODE_LOCAL_MODE=true
uv run flyto run workflows/check-add-card-filter-tag-sync.yaml --param url=http://localhost:8080

Workflows that require auth take email and password params:

uv run flyto run workflows/check-publish-sync-gate.yaml \
  --param url=http://localhost:8080 \
  --param email=$QA_EMAIL \
  --param password=$QA_PASSWORD

A passing run prints each step’s status and timing, then exits 0. Failed assertions print the actual versus expected values and exit non-zero.

Using flyto via MCP

The second flyto CLI is its MCP (Model Context Protocol) server. When it is installed, an AI coding agent — we use OpenCode — can call flyto modules directly as tools during a conversation, without writing a workflow file or dropping to the terminal.

What the MCP mode is good for

The MCP mode is an exploration and authoring tool, not a test runner. A typical session looks like this:

1. Discover selectors. Before writing a workflow you need to know what data-testid attributes actually exist on the page. Instead of guessing, the agent launches a browser, navigates to the relevant view, and runs browser.evaluate to read the real DOM. Confirmed in seconds, no guessing.

2. Verify a hypothesis. During a bug investigation — say, checking whether a sync warning renders correctly after a code change — the agent can open the page, set localStorage to a specific state, click through the flow, and read back element properties. Immediate, grounded feedback without a full workflow run.

3. Draft workflow steps interactively. Because MCP calls are one step at a time, the agent can try a click, see what is on screen, adjust, and try again. Once the steps work, they get written into the YAML workflow.

The key difference: persistent profile vs ephemeral context

This is the most important thing to understand when using both modes together.

When you run uv run flyto run workflows/my-workflow.yaml from the terminal, flyto uses a persistent Chromium profile stored at ~/.flyto/chrome-profile. Cookies, localStorage, and IndexedDB survive across runs. A login session established in one run is still there in the next. That is what makes idempotent bootstrap workflows possible.

When an agent calls flyto modules via MCP, it gets a separate ephemeral browser context. It does not share the persistent profile. That context is gone when the MCP session ends.

	CLI (uv run flyto run)	MCP (agent tool calls)
Browser profile	Persistent (~/.flyto/chrome-profile)	Ephemeral (fresh each session)
Auth session	Survives across runs	Gone when the session ends
IndexedDB / localStorage	Persistent	Ephemeral
Best for	Repeatable regression tests	Interactive exploration and authoring

This caught us when building the publish-sync-gate workflow. The agent logged into the app via MCP, confirmed the Publish menu item appeared, then ran the workflow via CLI — and the workflow saw no session, because the persistent profile had not been touched. The login the agent did in MCP was in a completely separate context.

The fix was to add a bootstrap-login.yaml sub-workflow that the CLI workflow invokes itself, so the persistent profile gets the session before the test steps run.

Rule of thumb: use MCP to explore and discover; use the CLI to run and verify. Only the persistent profile state is what your workflow sees when it runs.

Setting it up

Install flyto-core as an MCP server in your editor. In OpenCode, add it to opencode.json:

{
  "mcp": {
    "flyto": {
      "type": "local",
      "command": "uv",
      "args": ["run", "--directory", "qa-flyto", "flyto", "mcp"],
      "env": {
        "FLYTO_VSCODE_LOCAL_MODE": "true"
      }
    }
  }
}

The agent then has access to all flyto modules — browser.launch, browser.goto, browser.evaluate, and the rest — as first-class tools it can call during any conversation.

Lessons Learned

Sessions persist; design for it

flyto runs Chromium with a persistent profile at ~/.flyto/chrome-profile. IndexedDB survives across runs. That is a feature — a deck created in one run is visible in the next — but it means runs are not hermetic by default.

We handle this with idempotent setup workflows. The bootstrap checks whether the deck already exists before trying to create it:

- id: check_existing_deck
  module: browser.evaluate
  params:
    script: |
      return document.querySelector('[data-testid^="deck-card-"]')
        ? 'exists'
        : 'missing';

- id: click_new_deck
  when: '${check_existing_deck.result} == missing'
  module: browser.click
  params:
    click_method: selector
    selector: '[data-testid="create-deck-button"]'

When the deck is already there, the creation steps are skipped. The when: condition uses ${step.field} syntax (single braces) — not the {{...}} interpolation used in params. Mixing them up is a silent failure.

flow.invoke serialises sub-workflows

Two browsers on the same persistent profile contend for the Chromium singleton lock. flyto silently falls back to a throwaway context when this happens — your IndexedDB writes from the sub-workflow disappear.

The fix: invoke the bootstrap before your own browser.launch. The sub-workflow runs in its own browser, closes it, and only then does the caller open a browser. State written to IndexedDB by the sub-workflow is on disk and visible to the next browser.

steps:
  - id: bootstrap
    module: flow.invoke
    params:
      workflow_source: workflows/bootstrap-create-deck.yaml
      workflow_params:
        url: '{{url}}'

  - id: launch    # opens AFTER bootstrap has closed its browser
    module: browser.launch
    params:
      headless: true

Use data-testid selectors — never depend on implementation details

Tests must depend on the contract (what the UI promises), not the implementation (how it’s built). Selectors tied to CSS classes, aria labels, button text, or DOM structure break when unrelated changes happen. data-testid attributes are the explicit test contract and survive refactors.

# good — stable
selector: '[data-testid="publish-deck-submit"]'

# fragile — breaks on copy changes
selector: 'button:has-text("Publish")'

# fragile — breaks on accessibility or style changes
selector: 'button[aria-label="Deck options"]'

If the element you need has no data-testid, add one to the component first. A missing testid is a gap in the test contract, not a reason to use a weaker selector.

Navigate naturally — let the app set the URL

Click real UI elements to reach a view rather than constructing URLs by hand. This mirrors how a user actually navigates and avoids brittle assumptions about route shape (query params, path segments) that can change independently of the UI behavior.

# good — click the deck card; the app navigates to /manage?deck=<id>
- id: click_deck_card
  module: browser.click
  params:
    click_method: selector
    selector: '[data-testid^="deck-card-"]'

# bad — hardcodes an ID that must be read separately
- id: goto_manage
  module: browser.goto
  params:
    url: '{{url}}/manage?deck={{some_id_you_read_from_idb}}'

If you need to reload the same view (for example, to pick up a patched localStorage value), capture the URL after the natural navigation and reuse it:

- id: capture_url
  module: browser.evaluate
  params:
    script: return window.location.href;

- id: reload
  module: browser.goto
  params:
    url: '{{capture_url.result}}'

Use browser.evaluate for extraction; browser.type for input

browser.evaluate returns a value at {{step_id.result}}. It is the most reliable extraction path, because you control exactly what gets returned:

- id: get_label
  module: browser.evaluate
  params:
    script: |
      const span = document.querySelector('[data-testid="nav-review"] span');
      return span ? span.textContent.trim() : '';

For input, browser.type handles open-animation timing automatically — it waits for the target element to be visible, stable, and enabled before typing. The native value-setter approach (Object.getOwnPropertyDescriptor inside browser.evaluate) bypasses React’s controlled-input state, leaving required fields empty in React even though the DOM shows a value.

Assert on rendered UI, not the URL bar

ouou’s share-target handler renders /add but calls replaceState() to rewrite the URL to /. Asserting window.location.pathname === '/add' fails even when the Add Card view is showing. We assert on a rendered element instead:

- id: assert_add_view
  module: browser.evaluate
  params:
    script: |
      return Boolean(document.querySelector('[data-testid="card-front-input"]'));

Dump first, assert second

Never guess a selector. Write a dump step first:

- id: get_nav_html
  module: browser.evaluate
  params:
    script: |
      const el = document.querySelector('nav');
      return el ? el.outerHTML : 'NOT FOUND: ' + document.body.innerHTML.substring(0, 2000);

- id: save
  module: file.write
  params:
    path: output/nav-dump.html
    content: "{{get_nav_html.result}}"

Open the output file, read the real data-testid attributes and element structure, then write the assertion. The dump-nav-html.yaml and debug-extract.yaml workflows in the repo are ready-made examples of this pattern.

Double-quote YAML strings containing \n

Single-quoted YAML scalars treat \n as a literal backslash-n. When asserting against a multi-line value — for example, the share-target body text with a blank-line separator — use double quotes:

# wrong — expects the four characters  \  n  \  n
expected: 'Workflow Body\n\nSource: https://example.com'

# correct
expected: "Workflow Body\n\nSource: https://example.com"

Step output is not in the trace JSON

The output/workflow_*.json files record step status and timing, not the values returned by steps. To inspect a value, write it to a file:

- id: debug_value
  module: file.write
  params:
    path: output/debug.txt
    content: "{{some_step.result}}"

UPD — 2026-06-13

The experiment is working.

Today one of these workflows found a real bug in the Add Another flow on the Add Card screen. The app was calling the async add handler without awaiting it, so the draft clear could race behind the submit. In the real browser that meant the card could be saved while the front and back inputs still showed the old values for a moment instead of resetting cleanly.

Our existing integration tests did not catch that because they covered the pieces separately, not the full wiring. One test verified that AddCardForm calls onAddCardAndStay. Another verified tag preselection behavior. But the bug lived one level higher, in the app-level async path where handleAddCardAndStay waits for storage writes and then clears the shared draft state. The flyto workflow exercised the real browser, the real form submission path, and the real draft state together, so it caught the gap immediately.