The checkout smoke test to run before Black Friday, and the date you stop changing things

Every November the same thing happens in some store. An app gets updated, a section gets a new setting, someone pastes a tracking snippet into the theme, and on Thursday night the Add to cart button opens a drawer that never fills. Nobody notices until the conversion rate drops on the busiest morning of the year, because nobody walked the path to checkout after the change.

This guide is about two cheap things that prevent most of that: one automated test that walks the path a shopper takes to checkout, and one date after which the store stops changing. Neither is clever. Both have to exist before the traffic does. Black Friday 2026 is Friday 27 November, so at the time of writing there are five weeks left, which is enough.

If you are looking for the full manual pass for a launch, with quantities, discount combinations, redirects and empty states, that is a different job and it is covered in a pre-launch QA pass for Shopify that finds real bugs. This article is the part you run again and again, without a person.

What a smoke test is, and why one beats fifty

A smoke test answers one question: is the main path open? The name comes from hardware, where you power the board on and see whether it smokes before testing anything finer. For a store the main path is product page, add to cart, cart, checkout. If any of those four steps breaks, nothing else you test matters that day.

The temptation is to write a suite: every template, every filter, every popup, every language. Big suites have a predictable life. They are written in a burst, they break on the first copy change, someone marks the noisy ones as skipped, and by the second month nobody runs them because a red result no longer means anything. A single test that only fails when a shopper could not reach checkout keeps its meaning. When it is red, someone gets up.

So the rules for this test are strict. It touches one product that is always in stock. It uses selectors that describe what a shopper sees, not how the theme is built, so a CSS refactor does not break it. And it stops at checkout. On a live store it never types a card number, because the only ways to finish a test order there either create a real charge or block real customers, as explained below.

The test, line by line

This uses Playwright Test. It assumes a project with @playwright/test installed and a tests/smoke folder. Every value that changes between stores comes from an environment variable, so the same file runs against your live store, a preview theme or a development store.

// tests/smoke/checkout-path.spec.ts
import { test, expect } from '@playwright/test';

const PRODUCT_PATH = process.env.SMOKE_PRODUCT_PATH || '/products/your-best-seller';
const PREVIEW_THEME_ID = process.env.PREVIEW_THEME_ID || '';
const STOREFRONT_PASSWORD = process.env.STOREFRONT_PASSWORD || '';

// The same storefront URL, rendered by an unpublished theme when an ID is set.
function onTheme(path: string): string {
  if (!PREVIEW_THEME_ID) return path;
  return path + (path.includes('?') ? '&' : '?') + 'preview_theme_id=' + PREVIEW_THEME_ID;
}

test('a shopper can get from a product page to checkout', async ({ page }) => {
  if (STOREFRONT_PASSWORD) {
    // The same form the password page submits. The cookie lands in this browser context.
    await page.request.post('/password', {
      form: { form_type: 'storefront_password', utf8: '✓', password: STOREFRONT_PASSWORD },
    });
  }

  await page.goto(onTheme(PRODUCT_PATH));
  await expect(page).not.toHaveURL(/\/password/);

  const heading = page.getByRole('heading', { level: 1 }).first();
  await expect(heading).toBeVisible();
  const title = (await heading.innerText()).trim();

  const added = page.waitForResponse(
    (res) => res.url().includes('/cart/add') && res.request().method() === 'POST',
  );
  await page.getByRole('button', { name: /add to cart/i }).click();
  expect((await added).ok()).toBe(true);

  await page.goto(onTheme('/cart'));
  await expect(page.getByRole('link', { name: title }).first()).toBeVisible();

  await page.getByRole('button', { name: /check ?out/i }).click();
  await expect(page).not.toHaveURL(/\/cart\/?(\?|$)/);
  await expect(page.getByText(title).first()).toBeVisible();
  await expect(page.getByText('Total', { exact: true }).first()).toBeVisible();
  // Stop here. Nothing past this point is typed, so no order is created.
});

What each part does:

• The three constants. SMOKE_PRODUCT_PATH is the product the test buys into the cart. Pick one that is always in stock, has a single variant or a preselected one, and is not part of a Black Friday price change, so the test does not fail because the merchandising team did its job. PREVIEW_THEME_ID and STOREFRONT_PASSWORD are empty unless you need them.
• onTheme(). Adds preview_theme_id to storefront URLs. That parameter is how Shopify renders a page with a theme that is not published: the JSON output of shopify theme push returns a preview_url of the form https://your-store.myshopify.com/?preview_theme_id=<id>. The test adds it to the product page and the cart, the two pages a theme renders on this path. Checkout is rendered by Shopify, not by the theme.
• The password step. Only runs when a password is set. Shopify's storefront_password form posts to /password with a hidden form_type field. page.request shares cookies with the browser context, per the API testing guide, so the session cookie the store sets is already there when the page opens. Posting the form directly avoids depending on how each theme lays out its password page, which differs a lot between themes.
• not.toHaveURL(/\/password/). If the password was wrong, the store sends you back to the password page and the test fails here with a clear message, instead of failing three steps later on a missing button.
• The heading. getByRole('heading', { level: 1 }) reads the product title the way a screen reader would. Playwright's locator guide recommends role locators first, because they follow what the user perceives. The title is stored so later steps can look for that exact product.
• waitForResponse before the click. The promise is created before clicking so the response cannot be missed. It matches any POST to /cart/add, which covers themes that call /cart/add.js and themes that post to /cart/add. Checking ok() catches the 422 that the Ajax Cart API returns for a sold-out variant.
• The cart. Going to /cart directly is deliberate. Whether a theme opens a drawer, a notification or a new page is a design choice; the cart page is the one place every theme has. The assertion is that a link with the product's name is visible.
• The checkout button. /check ?out/i matches "Checkout" and "Check out". If your storefront is in another language, change the pattern to your button's text, for example /pagar|finalizar/i.
• The three final assertions. The URL has left the cart, the product title appears in the order summary, and the word Total is on the page. Playwright's web-first assertions retry until they pass or time out, so there is no fixed wait. The test does not assert the checkout URL's shape, because Shopify does not promise it.

The configuration file sets the base URL from an environment variable, so page.goto('/cart') resolves against whichever store you point it at, as described in Playwright's test configuration:

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

if (!process.env.STORE_URL) throw new Error('Set STORE_URL, e.g. https://your-store.myshopify.com');

export default defineConfig({
  testDir: 'tests/smoke',
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 1 : 0,
  reporter: process.env.CI ? [['github'], ['html', { open: 'never' }]] : 'list',
  expect: { timeout: 15_000 },
  use: {
    baseURL: process.env.STORE_URL,
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
  },
  projects: [{ name: 'mobile-chrome', use: { ...devices['Pixel 7'] } }],
});

Three choices worth explaining. The project emulates a phone, because a checkout path that only works on desktop is the failure nobody sees from the office; check the device split in your own analytics and add a desktop project if desktop carries real revenue too. The expectation timeout is longer than the default five seconds because checkout loads more than a theme page does. And one retry in CI absorbs a network hiccup without hiding a real break: a test that fails twice in a row is a real failure.

Run it locally with STORE_URL=https://your-store.myshopify.com npx playwright test. On Windows PowerShell, set the variable with $env:STORE_URL="…" first. Two things commonly make it fail on the first run without anything being broken: a newsletter or cookie popup that covers the button, and a product with a variant that has to be chosen. For the popup, Playwright has page.addLocatorHandler(), which closes an overlay whenever it appears. For the variant, pick a different product.

Going further, only on a development store

The test stops at checkout because finishing an order on a live store has a cost either way. Shopify's own test order documentation gives the options: activate the test payment gateway or your provider's test mode, or pay with a real card and then cancel and refund. The first one is explicit about its side effect: customers can't place live orders while your payment providers are in test mode. The second can leave processor fees behind. Neither belongs on an automated schedule in November.

A development store is where the test can go all the way. Dev stores can't process real transactions, they accept unlimited test orders through the test payment gateway, and they are always password protected, which is why the password step exists. There, you can extend the test past the last assertion: fill contact and address fields with getByLabel(), use an email ending in @example.com as Shopify recommends, and pay with the test gateway's card numbers: 1 for an approved transaction, 2 for a declined one, 3 for a gateway failure. The declined case is worth a test of its own, because the message a shopper reads after a failed payment is rarely checked. If the dev store has a fulfillment app, turn it off first, as the processing guide advises.

Keep the two versions in separate files. The live one should never contain a line that could pay.

Before you publish a theme, and after you install an app

The two moments in November when the path breaks are a theme publish and an app change, and the test covers both.

Before publishing a theme. Push the new version as an unpublished theme with shopify theme push --unpublished --json, take the ID from the preview_url, and run the test against it with PREVIEW_THEME_ID set. The live store keeps selling on the old theme while the new one is walked from product to checkout. If you prefer the admin, the theme library's Preview action does the same; note that a shareable preview link expires two days after it is created, so do not hand one out on the first day of the freeze and expect it to work on the fourth. Shopify's help also warns that a preview_theme_id left on live URLs forces the preview bar for every visitor, which is one more reason to keep it inside the test and out of links you publish.

After installing or updating an app. Apps that add app embeds, scripts or checkout extensions change the live path at the moment you save, with no preview step. Run the test against the live store immediately after the change, and again after the app's own setup wizard has finished, because some apps only inject their code at the end of onboarding. If it goes red, disable the app embed or the extension first, and investigate after the path is open again.

Running it every hour with GitHub Actions

The workflow below follows Playwright's CI guide, with one change: instead of running on every push, it runs on a schedule and on demand. A storefront breaks because the store changed, not because the repository did, so the trigger that matters is the clock.

# .github/workflows/checkout-smoke.yml
name: Checkout smoke test
on:
  schedule:
    - cron: '17 * * * *'   # every hour at minute 17, UTC
  workflow_dispatch:
    inputs:
      preview_theme_id:
        description: 'Unpublished theme ID to test (empty = live theme)'
        required: false
        default: ''
jobs:
  smoke:
    timeout-minutes: 15
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - name: Install dependencies
        run: npm ci
      - name: Install Chromium
        run: npx playwright install --with-deps chromium
      - name: Run smoke test
        run: npx playwright test
        env:
          STORE_URL: ${{ secrets.STORE_URL }}
          STOREFRONT_PASSWORD: ${{ secrets.STOREFRONT_PASSWORD }}
          SMOKE_PRODUCT_PATH: ${{ vars.SMOKE_PRODUCT_PATH }}
          PREVIEW_THEME_ID: ${{ inputs.preview_theme_id }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

The store URL and the storefront password go in repository secrets, as GitHub describes in using secrets in GitHub Actions, and they reach the test as environment variables, never as text in the file. The storefront password is the visitor password from Online Store, Preferences, not an admin login; Shopify's dev store documentation says explicitly not to reuse your admin password there. The product path is not secret, so it is a plain repository variable. Only Chromium is installed because the config only uses a Chromium device, which keeps each run short.

Details from GitHub's trigger reference that matter here: scheduled runs use UTC by default, run on the default branch, and can be delayed when GitHub Actions is under heavy load, which on the last Friday of November is plausible. That is why minute 17 instead of minute 0, and why the urgent-fix process below runs the test by hand with workflow_dispatch rather than waiting for the schedule. In a public repository, schedules are disabled after 60 days without activity, so check the workflow is still enabled in the first week of November.

Two practical notes. Every run is a real visit with a real add to cart, so it shows up in your storefront analytics; once an hour is a rounding error, once every five minutes is not. And if the store sits behind a bot-protection layer, GitHub's runners may be challenged; allow them, or run the same command from a machine you control. Make sure a failed run notifies a person, not only a channel nobody reads during a sale.

The freeze date

A smoke test tells you the path is open now. A freeze makes it likely to stay open. The idea is simple: from a set date until the sale is over, nothing changes on the storefront or checkout unless it meets a written definition of urgent.

Our recommendation, which you should adapt to your calendar: the last large change ships at BF−14, Friday 13 November, so it gets a full week of real traffic before the freeze. The freeze starts at BF−7, Friday 20 November at the end of the working day, and ends on Tuesday 1 December, the day after Cyber Monday. Write the date, the time and the time zone in the team calendar and in the contract with every agency and freelancer who has access to the store.

What is frozen: theme code and theme settings on the live theme; installing, removing or reconfiguring apps that touch the storefront or checkout; checkout extensions and Functions deploys; tracking scripts and custom pixels; navigation and URL changes. What is not frozen: product content, images and stock; collections and merchandising order; the Black Friday discounts themselves, provided they were created and tested before the freeze and are only switched on by schedule.

What counts as urgent: something that stops a shopper from buying, or makes them pay the wrong amount. A broken add to cart, a checkout that will not load, a wrong price, a discount that does not apply or applies twice, wrong taxes or shipping rates, a security or legal problem. A broken purchase-tracking setup only qualifies if paid campaigns are spending against it that week and the fix is configuration, not code. A misaligned banner, a typo outside prices and legal text, a request for a new feature, and "a small app we want to try" are not urgent. They go on a list for 2 December.

The process for each urgent fix:

1. Report it with the input that causes it. Product, variant, quantity, discount code, device. "The cart looks wrong" is not a report.
2. Approval. Two people, named before the freeze starts: one owner on the business side who can say "yes, this costs us money", and one developer who can say what the fix touches. Both with a phone number that works on Friday morning.
3. Fix on a copy. Make the change on a duplicate of the live theme, or in a branch pushed as an unpublished theme, never in the live theme's editor.
4. Smoke test before. Run the workflow by hand with the preview theme's ID. If it fails, the fix does not ship.
5. Publish in the quietest hour. Your analytics show which hour of the day has the fewest sessions. Never during a campaign email send.
6. Smoke test after. Run it again against the live store, without an ID. Then do the one manual check that reproduces the original report.
7. Rollback if either fails. Republish the backup theme you kept untouched in the library. For an app, disable its embed or extension. For a discount, deactivate it. Decide the rollback before you publish, not after.
8. Log it. What changed, who approved it, when it was published, and both smoke test results. On 2 December this log is the review.

The rollback depends on having the backup theme in the library. According to Shopify's help on duplicating themes, a store holds 20 themes, and you have to delete one before duplicating if you are at the limit. Check the count before the freeze, not during an incident.

The checklist for the week before Black Friday

• The freeze date, time and time zone are in the shared calendar, and every agency and freelancer with store access has confirmed it in writing.
• The two approvers are named, with phone numbers, and both know the definition of urgent.
• The live theme has a dated, untouched duplicate in the theme library, and there is room for one more theme.
• The smoke test is green on the live store, scheduled hourly, and a failed run notifies a person.
• The smoke test product is in stock with enough inventory, and is not part of a price change.
• Payment test mode is off on the live store, and the test payment gateway is not active.
• Every Black Friday discount has been tested once in a cart, on a phone, with the combinations it will meet. The full method is in the pre-launch QA article.
• Add to cart responds quickly on a mid-range phone. If it does not, this guide on a slow Add to cart button shows how to find the script responsible, and that fix has to land before BF−14, not during the freeze.
• There is a list of every app that touches the storefront or checkout, with who owns it and how to disable it in one step.
• You know the quietest hour of the day for publishing an urgent fix.
• The list for 2 December exists, so non-urgent requests have somewhere to go.

None of this is much work. The test is one file, the workflow is another, and the freeze is a date and a paragraph that everyone agrees to. If you want help setting it up on your store, it is part of our Shopify theme development work: the test in your repository, the workflow running, and the freeze written down before November gets busy.