End-to-End Testing an Astro Site Against the Real Production Build

Kevin Dench

Sr. Software Engineer

Jul 3, 2026 Updated Aug 17, 2026 13 min read

TL;DR: my end-to-end tests don’t run against the Astro dev server. They run against the actual production build, served by astro preview, because the dev server and the build are different programs that happen to render similar pages. Getting there involved four gotchas—WebKit’s system dependencies, a CI runner that chokes above one worker, third-party scripts polluting my results, and pagination markup that my tests shamed me into fixing. This post walks through all of it, with the config straight from my site.

If you only take one thing away: test the artifact you deploy, not the tooling you develop with.

Why the Production Build, Not the Dev Server

A passing test against the dev server proves your dev server works, and very little about the build. Your users don’t load the dev server. They load static HTML, minified JavaScript, and optimized images that came out of astro build—a pipeline the dev server mostly skips.

Tasting the sauce straight from the pot tells you something about dinner, but not what the dish is like by the time it’s plated, garnished, carried across the room, and sitting in front of a guest. Plating changes things—sauces break. The dev server is the pot; the production build is the plate. I want my tests eating off the plate.

And this isn’t hypothetical caution. For example, several bugs on this site have existed only in the build:

Unhydrated islands. An interactive component that worked in dev but shipped as inert HTML because hydration directives only fully matter at build time.
Image pipeline output. Astro’s image optimization produces different files (and different URLs) in the build than the dev server serves.
Dev-only routes. A couple of routes exist only in development and get stripped from the production build on purpose. A dev-server test would happily verify a page that shouldn’t exist.
JSON-LD escaping. Structured data that serialized fine in dev and broke once the build’s escaping rules got involved.

None of those are exotic. They’re the ordinary difference between a dev server doing on-demand transforms with hot-module-replacement machinery bolted on, and a build emitting final, static output.

The mechanics are pleasantly boring. Astro’s preview command serves the dist/ folder—the exact output of the build—on a local port. Playwright’s webServer option starts that server before the suite and tears it down after. Let’s look at the config from my site:

const PORT = Number(process.env.PLAYWRIGHT_PORT ?? 4321);
const isCI = !!process.env.CI;

export default defineConfig({
  webServer: {
    command: `pnpm --filter @kdd/website preview --port ${PORT}`,
    url: `http://localhost:${PORT}`,
    reuseExistingServer: !isCI,
    timeout: 120_000,
  },
  // ...
});

One trick you might miss in there: the command only says preview, but a prepreview hook in package.json runs pnpm build (which is astro check && astro build) first. npm-style pre-hooks run automatically before their matching script, so “build the site, then serve the build” collapses into one command that Playwright owns. The timeout: 120_000 gives the build two minutes to finish before Playwright gives up waiting—a plain preview server needs nowhere near that, but a build plus a type-check does.

reuseExistingServer: !isCI is the local-comfort setting. On my machine, if I already have a preview server running, Playwright uses it instead of rebuilding each run. In continuous integration (CI), the setting flips and each run builds fresh, because a CI run that reuses stale output isn’t testing the commit that triggered it.

Gotcha 1: WebKit Is an Environment Problem

I develop on a Linux distro Playwright doesn’t officially support. Playwright’s WebKit build—the engine behind Safari—wants a pile of system libraries that Fedora, Arch, and some Docker images don’t have in the expected shapes. playwright install --with-deps, for example, works great on Ubuntu and Debian and politely falls apart on distros it doesn’t know.

My first instinct was to fight the dependency list. My second, better instinct was to stop treating this as a code problem. Cross-browser testing is partly an environment problem, and the fix is to give WebKit the environment it wants instead of contorting mine.

So WebKit is opt-in. The default browser matrix is chromium, firefox, and mobile-chrome (a Pixel 7 profile)—engines that install cleanly everywhere I run tests. Setting PLAYWRIGHT_WEBKIT=1 adds desktop WebKit and mobile-safari (an iPhone 13 profile) to the matrix. And the policy is written in a comment at the top of the config: a missing WebKit must never break the acceptance gate. WebKit coverage is a bonus lane, not part of the gate itself.

When I want that lane, I run it in Microsoft’s official Playwright container, which ships the system libraries WebKit needs:

# On the host first: build + serve the preview.
pnpm --filter @kdd/website preview --host &

docker run --rm --network=host --user "$(id -u):$(id -g)" \
  -v "$PWD":/work -w /work/apps/website \
  -e PLAYWRIGHT_WEBKIT=1 -e PLAYWRIGHT_SKIP_WEBSERVER=1 \
  -e PLAYWRIGHT_BASE_URL=http://localhost:4321 \
  mcr.microsoft.com/playwright:v1.60.0-jammy \
  node_modules/.bin/playwright test --project=webkit --project=mobile-safari

Most of those flags are scar tissue, not decoration. --network=host lets the container reach the preview server running on my host—pnpm isn’t on the image’s PATH, so the server starts outside the container, and PLAYWRIGHT_SKIP_WEBSERVER=1 plus PLAYWRIGHT_BASE_URL tell Playwright to skip its own webServer and point at mine. And --user "$(id -u):$(id -g)" stops the container from writing a root-owned test-results/ folder that will break the next host run with permission errors. I learned each of these the slow way.

One caveat: Playwright’s WebKit is a cross-platform build of the engine that Playwright maintains, not literal Safari. It catches a lot of the engine-specific rendering and JavaScript differences, but “passes in Playwright WebKit” and “works on your aunt’s iPhone” are correlated, not identical.

Gotcha 2: One Preview Server, One Worker (in CI)

Playwright parallelizes tests across workers, and the obvious move is to crank workers up for speed. On my workstation, the defaults are fine—the full suite finishes in about 47 seconds. On my shared CI runner, however, two workers made the run slower and flakier: navigations timing out, pages bailing to chrome-error screens, perfectly healthy tests failing.

The picture that made it click for me is a coffee shop adding cash registers. Six registers take six orders at once—but there’s still one barista and one espresso machine. All you’ve built is a longer line in a different place, plus six customers who each think their drink got forgotten. The registers are Playwright workers; the barista is the CI runner’s CPU.

And my CI barista is having a rough decade. The runner is a modest shared box with no GPU, so each browser renders through SwiftShader—software rendering on the CPU, which means the CPU is simultaneously running the preview server, executing test code, and rasterizing pixels for multiple full browsers. The same suite that takes 47 seconds locally takes many minutes there. Two workers didn’t split that work; they saturated the machine until browsers started giving up mid-navigation.

The fix is deflating but correct: cap workers at 1 in CI, and stretch the timeouts to match reality—120 seconds per test and 30 seconds per assertion in CI, versus 30 and 10 locally. Locally, parallelism stays at Playwright’s default, because saturation is a property of that runner, not of the suite. It stung to type workers: 1. It stung less than re-running flaky pipelines all afternoon.

The general lesson: match concurrency to the slowest shared resource, not to your core count. Playwright’s CI docs say roughly the same thing, more diplomatically.

Gotcha 3: Seal the Room—No Third Parties in Tests

This site loads a few third parties in production: Plausible for analytics, Formbricks for feedback surveys, GlitchTip (via the Sentry software development kit, or SDK) for error reporting, and Google Fonts. All four are reasonable in production and noise in a test.

My first version of this fixture just blocked the analytics domains. The version I run now is stricter: it aborts every cross-origin request. If a request isn’t going to my preview server, it doesn’t leave the building.

// tests/e2e/fixtures/helpers.ts (the idea, condensed)
await page.route('**/*', (route) => {
  const url = new URL(route.request().url());
  if (url.origin !== new URL(baseURL).origin) {
    return route.abort('aborted');
  }
  return route.continue();
});

If you’re testing your speakers, you close the windows first. Traffic noise outside isn’t wrong—it’s just not what you’re measuring. A test run that depends on Plausible’s content delivery network being fast today is measuring the internet’s mood, not my site. Sealing the room makes runs faster, and it also makes them deterministic, which matters more: the same build produces the same result run after run, and when it doesn’t, the cause is in my code or my runner, not somewhere out on the internet. (A DNS-level ad blocker on my LAN already swallows some telemetry hosts, which is how I first noticed my tests behaved differently at home than in CI. Making the tests hermetic—sealed off from the outside world, in testing jargon—made both environments agree.)

However, aborting requests adds noise too, so the fixture comes with two companions. First, a console-error allowlist scoped to the SDKs I deliberately strangled—for example, /plausible/i and /glitchtip|sentry/i—because an SDK that can’t phone home complains, and that complaint is expected. Second, the fixture ignores the browser’s own spelling of a benign abort: net::ERR_ABORTED in Chromium, NS_BINDING_ABORTED in Firefox. Same event, two dialects.

The rest of the fixture stays strict. The health fixture fails the test on any uncaught page error, any console error not on the allowlist, or any same-origin request that fails. If my site 404s one of its assets, I want a red build, loudly.

Now the payoff, the bug that justified the approach. My shared CI component had a default of BASE_URL=http://localhost:4173, and that environment variable leaked into the Astro build as import.meta.env.BASE_URL. Result: the built site rendered its internal links as absolute URLs pointing at an origin that didn’t match the preview server. The hermetic fixture saw “cross-origin”, aborted the navigations, and the suite stalled. Annoying? Extremely. But notice what happened: the bug lived in the build—a CI environment variable leaking into astro build, a path the dev server on my laptop never takes—and the strict fixture turned a subtle routing-and-search-ranking landmine into a suite that refused to pass. The test setup caught exactly the class of bug it was designed for. I’d have preferred it catch someone else’s bug first, but you take the wins you get.

Gotcha 4: Query Like a User, Especially Pagination

I’ve written before about testing React like a user, and the philosophy carries straight over to Playwright: find elements by role and accessible name, the way assistive technology does, not by CSS classes that the next refactor can rename out from under you. Playwright’s locator docs push the same direction.

Locating by role is the difference between giving directions by street names and giving them by “turn left at the blue car.” The blue car works today. The blue car is someone’s commute. Class-name selectors are the parked car; roles and accessible names are the street signs.

Pagination is where role-based locators earned their keep on my site (the pagination itself has its own origin story). My blog list pages have Previous/Next controls and numbered page links, and the naive locator—“find the link that says Next”—has a hole: on the last page, Next isn’t a link at all. It’s a disabled <span>, because a disabled control shouldn’t be focusable or announce itself as navigation. So we locate the pager as a nav landmark and assert on the accessible names inside it:

// condensed from the shared pagination helper in tests/e2e/fixtures/helpers.ts
test('posts pagination never breaks [MNB:pagination]', async ({ page }) => {
  const pager = page.locator('nav', { has: page.locator('.pagination-item') });

  // Page 1: "Previous" is a disabled <span>, so it must NOT exist as a link.
  await expect(pager.getByRole('link', { name: 'Previous' })).toHaveCount(0);
  await expect(pager.getByLabel('Current page, page 1')).toBeVisible();

  await pager.getByLabel('Go to page 2').click();
  await expect(pager.getByLabel('Current page, page 2')).toBeVisible();
  await expect(pager.getByRole('link', { name: 'Previous' })).toBeVisible();
});

(That [MNB:pagination] tag in the test name is suite shorthand for must-never-break—a grep-able label on the contracts that gate each deploy. Astro pagination broke on me once, so it earned the label.)

Writing that test forced the markup to earn it. The numbered links needed proper labels—"Go to page 2", "Current page, page 2"—and Previous/Next needed names a screen reader would read aloud. Before the test existed, some of that was missing. Which is the second benefit of role-based locators, and I think the more important one: a control that’s hard to locate by role is a control that’s hard to use with a screen reader. The test checks the pagination, and it also audits accessibility on each commit.

The suite’s conventions doc boils it down to four rules: semantic locators, web-first assertions (for example, expect(...).toBeVisible() retries; a manual check doesn’t), no hard waits, and never assert on text managed in the CMS (content management system)—because content editors shouldn’t be able to break the build by rewording a heading.

Running It

Locally, the suite is one command from the monorepo root:

pnpm website test:e2e

You’ll find the tests in apps/website/tests/e2e/, with shared fixtures in tests/e2e/fixtures/. The specs split into two kinds: persona journeys—recruiter, hiring manager, potential client, and the developer who wandered in from a search result—that each walk the site the way that person would, plus cross-cutting specs for route contracts, accessibility, and performance. The persona framing keeps me focused on why a page exists, not just whether it renders.

In CI, the job comes from one of my shared GitLab pipeline components and runs the same command I run locally: pnpm website test:e2e, self-contained, no prebuilt artifact passed between jobs. The webServer config builds fresh inside the job, so the pipeline tests the same path a deploy takes. One CI-specific tweak: video: 'off'. Failure videos on the slow runner blew past the artifact size limit, the upload failed with a 413, and, worse, the traces got dropped along with the videos. I traded pretty videos for reliable traces, and traces are what you’ll use to debug a failure anyway.

And the standing rule, borrowed from the testing teams I’ve trusted: flaky tests get fixed or deleted, never skipped-and-forgotten. A suite you don’t trust is a suite you stop reading, and an unread suite is decoration.

Conclusion

The recap, in one breath: run Playwright against astro preview so you’re testing the build users get; treat WebKit as an environment problem and give it a container (as an opt-in lane that never blocks the gate); cap workers to what the shared runner can actually chew; seal off cross-origin requests so runs are deterministic; and locate elements by role and accessible name so your tests double as accessibility checks.

The throughline is smaller than any of the individual gotchas. End-to-end confidence comes from testing the deployed artifact in a real browser—and most of the actual work is hunting down the places where the setup lies to you. The dev server lies about the build, the default worker count lies about the runner, third parties lie about your latency, and brittle selectors lie about your markup. Remove the lies one at a time, and what’s left is a suite that means something when it’s green.