This page has moved

Testing has its own top-level section now: Testing.

The new section has dedicated pages for philosophy, test suites, running tests, writing tests, live verification, and bug tracking. Read those for the canonical guide.

This page is kept as a legacy summary for now and will be removed in a future docs cleanup.

Testing & QA Frameworks

Every test framework KA26 uses, what each one tests, when to use which, and how they fit into CI/CD. Read this when:

Adding a test for a new feature
Diagnosing a flaky test
Onboarding a new contributor
Wondering "should I write a unit test, integration test, or smoke test?"

TL;DR — three test classes, three different jobs

Class	Tool	Speed	Catches	When to use
File-shape regression	Vitest (node env)	~1s for 1900+ tests	"Did someone delete X?", "Does this constant equal Y?"	Most bugs we ship-and-revert. Fast guard against specific patterns.
Component integration	Vitest (jsdom env) + @testing-library/react	~1s per test	UX behavior — clicks, navigation calls, conditional rendering	New UI components with user input. The class file-shape can't catch.
E2E production smoke	Vitest hourly cron	5-15s per test	"Is production responding correctly?" — real HTTP calls, real responses	Critical funnels (signup, login, browse). Hourly safety net.

Plus visual regression via Playwright (currently used for snapshots only).

1. Vitest (primary test runner)

What it is

Modern unit + integration test runner. Compatible with Jest API but faster (uses Vite internally for instant transforms).

Why we picked it

Faster than Jest (most of our 1957 tests run in under 1 second)
ESM-first (Next.js + Prisma play well)
Native TypeScript support without ts-jest config
Same API as Jest so anyone with Jest experience can contribute

Test environments

We run BOTH node and jsdom environments under one config:

Default: node — fast, no DOM overhead. Used for backend file-shape tests and pure-logic tests.
Per-file // @vitest-environment jsdom — used by every file under tests/components/. Renders React components in a simulated DOM.

Config: vitest.config.ts at repo root.

npm scripts

npm test              # Runs the full suite EXCLUDING e2e smoke (default for CI)
npm run test:smoke    # Runs ONLY the e2e smoke tests (hourly cron uses this)
npm run test:all      # Everything including smoke
npm run test:full     # vitest + Playwright visual regression
npm run test:visual   # Playwright only

Where tests live

tests/
├── components/                            # Component integration (jsdom)
│   ├── SellerNotificationToast.test.tsx
│   ├── SellerPushSubscriber.test.tsx
│   └── ...
├── e2e-smoke.test.ts                      # 84 page-existence checks against production
├── e2e-signup-smoke.test.ts               # Real signup against production
├── e2e-critical-funnels-smoke.test.ts     # Login, categories, stores, products, UPI
├── notification-system-contract.test.ts   # 59 file-shape tests for notifications
├── auth-password-validation-regression.test.ts  # validatePassword bug-class guard
├── api-shape-contracts.test.ts            # API↔consumer contract drift detection
├── consumer-e2e.test.ts                   # 121 file-shape tests on consumer flows
├── public-discussion.test.ts              # File-shape tests for the public-discussion feature
└── ...                                    # 30+ more test files
mobile/tests/                              # Mobile-specific Vitest tests
└── mobile-code-integrity.test.ts          # 192 file-shape tests for mobile screens

2. File-shape regression tests (the workhorse)

What they do

Read source files via fs.readFileSync and assert patterns: "this function is imported", "this constant has this value", "this string appears in this file".

Why we use them so heavily

Fast: 1957 tests in ~1s. CI feedback in seconds, not minutes.
Stable: no jsdom flakiness, no async timing issues, no test environment setup
Targeted: each test guards against a SPECIFIC bug we've actually shipped before

What they catch well

"Someone deleted pushToSeller() from store-orders POST" → contract test fails
"Someone changed the password constant from 8 chars to 6" → consumer-e2e test fails
"Someone removed the notifType field from a push payload" → notification contract test fails
"Someone added a notification type but forgot to add a tap handler in mobile" → contract test fails

What they DON'T catch

"The dropdown is rendering at the wrong screen position"
"The button has the right text but no onClick handler"
"The form accepts input but the submit button is disabled"
"The URL is correct but doesn't actually load a page" (we added route-existence tests post-bug)

Example

// tests/notification-system-contract.test.ts
it("store-orders POST: sellerNotification.create() AND pushToSeller() both present", () => {
  const src = read("src/app/api/store-orders/route.ts");
  expect(src).toContain("pushToSeller");
  expect(src).toContain("sellerNotification.create");
  expect(src).toMatch(/pushToSeller\(\s*store\.sellerId/);
  expect(src).toContain('notifType: "new_order"');
});

Pitfalls (and how we fixed them)

False positive on comments: regex matched MAX_THREADS_PER_REQUEST = 3 inside a comment, falsely "passing" when constant was actually 200. Fix: match const X = N not just X = N.
False positive on broken paths: extracted ${storeId} was truncated to /seller/stores, which exists, falsely "passing". Fix: substitute ${var} → [id] placeholder, then check Next.js route shape.
Stale tests after refactors: test asserted data?.path but new code uses data.notifType. Fix when caught: update the assertion in the same commit as the refactor.

Verifying a contract test actually catches its bug

The discipline: when you write a contract test, reintroduce the bug and confirm the test fails. Restore. This was done for:

notification-system-contract.test.ts Section 9 (route-existence) — verified end-to-end on 2026-04-18
auth-password-validation-regression.test.ts — failed when the validatePassword-as-error pattern was reintroduced

3. Component integration tests (jsdom + React Testing Library)

What they do

Render the actual React component in jsdom, simulate real user interactions (click, type, focus), and assert on the resulting DOM + side effects (router calls, fetch calls).

Why we added them (2026-04-18)

File-shape tests missed multiple UX bugs that only manifest at render time:

6-day "[object Object]" registration outage
Seller panel routing to /seller/stores/{id}/orders (404 trap)
Notification panel sliding from bottom instead of top
Notification rows displaying but having no onClick handler
SellerPushSubscriber showing "Enable" banner when permission was already granted

Stack

@testing-library/react ^16 — render React in jsdom, query the DOM by accessibility role/label
@testing-library/user-event ^14 — realistic click/type/keyboard simulation
@testing-library/jest-dom ^6 — matchers like toBeInTheDocument, toHaveClass, toBeVisible
jsdom ^29 — DOM implementation for Node

Setup file: `tests/setup.dom.ts`

Provides for jsdom tests:

jest-dom matchers
Mocked Next.js navigation hooks (useRouter, useSearchParams, usePathname) with spy-able routerPushMock exported for assertions
Mocked Notification global, navigator.serviceWorker, EventSource (jsdom omissions)
Mocked matchMedia + scrollIntoView
In-memory localStorage polyfill (jsdom v29's shim is non-functional)

Example

// tests/components/SellerNotificationToast.test.tsx
// @vitest-environment jsdom
import { render, screen, fireEvent } from "@testing-library/react";
import { routerPushMock } from "../setup.dom";

it("clicking new_order toast routes to /seller/orders?highlight={orderId}", async () => {
  render(<SellerNotificationToast enabled={true} />);
  await simulateNewNotification({ id: 103, type: "new_order", data: { orderId: 27, storeId: 6 } });
  const toast = await screen.findByText("New Order!");
  fireEvent.click(toast.closest("[class*='cursor-pointer']") || toast);
  expect(routerPushMock).toHaveBeenCalledWith("/seller/orders?highlight=27");
  expect(routerPushMock).not.toHaveBeenCalledWith(expect.stringMatching(/\/seller\/stores\/\d+\/orders/));
});

Current coverage (as of 2026-04-18)

tests/components/SellerNotificationToast.test.tsx (9 tests) — toast positioning, click navigation, dismiss behavior
tests/components/SellerPushSubscriber.test.tsx (10 tests) — banner rendering per permission state, auto-subscribe behavior, localStorage persistence

What to test next (post-launch)

Consumer signup form (would have caught the "[object Object]" bug end-to-end)
Order placement flow (cart → checkout → confirmation)
Reel like / comment / share interactions
Address autocomplete dropdown
Public discussion thread join/leave

4. Production smoke tests (hourly cron)

What they do

Make REAL HTTP calls to ka26.shop every hour. Verify:

Pages return 200 (not 500)
API endpoints return the expected shape
Critical funnels work end-to-end (signup, login, etc.)

Why we added them (2026-04-18)

The 6-day "[object Object]" outage proved file-shape tests can pass while production is broken. We needed at least ONE test per critical funnel that exercises the real production stack. Catches "the API is silently broken" within ~1 hour instead of weeks.

Files

tests/e2e-smoke.test.ts (84 tests) — page-existence + status code checks
tests/e2e-signup-smoke.test.ts (1 test) — real consumer registration with throwaway email
tests/e2e-critical-funnels-smoke.test.ts (6 tests) — login, categories, stores, products, UPI config, register-then-login chain

How they're triggered

GitHub Actions cron .github/workflows/health-check.yml runs at :17 past every hour
Targets https://ka26.shop by default; can be overridden with SMOKE_BASE_URL
On failure: emails siddugkattimani@gmail.com via Gmail SMTP (uses SMTP_USER + SMTP_PASS + ALERT_TO GitHub secrets)

Test data hygiene

Test users created with email pattern health-cron-{ts}@ka26-test.invalid
The .invalid TLD is reserved (RFC 2606) — guaranteed never to deliver mail anywhere
Cleanup: DELETE FROM consumers WHERE email LIKE '%@ka26-test.invalid' periodically

Rate limiting tolerance

The signup route has IP-based rate limiting. Local re-runs hit 429s. The smoke test gracefully tolerates 429 — only fails if the response shape is wrong (e.g., [object Object]).

5. Mobile tests (Vitest)

What they do

Same Vitest, but in mobile/ directory with its own vitest.config.ts. File-shape tests on the React Native screens.

Files

mobile/tests/mobile-code-integrity.test.ts (192 tests) — covers all mobile screens, asserts critical patterns (auth tokens, API endpoint shapes, push registration calls, etc.)

Why no jsdom for mobile?

React Native components don't render in jsdom. Real RN test rendering needs react-native-testing-library + a React Native Vitest environment, which is more setup. Deferred until post-launch.

How they're triggered

Separate CI workflow: .github/workflows/mobile-tests.yml
Triggered on every push to main

6. Visual regression (Playwright)

What it is

Playwright takes screenshots of pages and compares them against committed baselines. Catches "did this layout shift in an unexpected way?"

Status

✅ Installed (@playwright/test ^1.59.1)
⚠️ Used minimally — only a few visual snapshot specs in tests/visual/
Not actively guarded in CI (deferred — pre-launch focus is functional correctness, not pixel-perfect)

Why not heavy E2E with Playwright?

Playwright E2E (vs. visual snapshots) would launch a real browser per test, navigate through the app, and assert behavior. It's the gold standard but slow (~10s per test) and high-maintenance. Component integration tests (jsdom) cover ~80% of what Playwright would, much faster. Playwright reserved for visual regression.

CI/CD pipeline (where tests run)

On every push to main (`.github/workflows/deploy.yml`)

Install dependencies
npm test — runs all 1957 tests EXCLUDING e2e smoke (~5s)
If any fail → block deploy
If pass → gcloud builds submit → gcloud run deploy

Hourly (`.github/workflows/health-check.yml`)

Install dependencies
Hit /api/health?key=... against production
npm run test:smoke — runs all 3 smoke files against ka26.shop
SSL cert expiry check (warn at 30 days, fail at 7 days)
On any failure → email alert via Gmail SMTP

On every push to mobile (`.github/workflows/mobile-tests.yml`)

Install mobile deps
Run mobile/tests/ Vitest suite
Block merge if anything fails

Test counts (as of 2026-04-18)

Source	Tests	Run time
`tests/*/.test.ts(x)` (excluding smoke)	1957	~1s
`tests/components/**` (jsdom subset)	19	~1s
`tests/e2e-*-smoke.test.ts` (hourly only)	91	~15s
`mobile/tests/**`	192	~1s
Total active	2240	—

Adding a new test — decision tree

Is this guarding against a bug class (missing function call, wrong constant, drift between files)?
  → File-shape regression test in tests/

Is this asserting how a UI component behaves (click, render, navigate)?
  → Component integration test in tests/components/ with @vitest-environment jsdom

Is this verifying a critical user flow works end-to-end against production?
  → E2E smoke test in tests/e2e-*-smoke.test.ts; wire into hourly cron

Is this a UI snapshot for visual regression?
  → Playwright spec in tests/visual/ (sparingly)

Is this a mobile-only screen check?
  → mobile/tests/mobile-code-integrity.test.ts

Anti-patterns to avoid (from `bugs_fixed.md`)

❌ Writing a contract test without verifying it catches the bug: re-introduce the bug, run the test, confirm it fails, restore.
❌ Asserting on regex strings that match comments: use const X = N patterns, not bare X = N.
❌ Sleeping in tests instead of await waitFor: flaky and slow.
❌ Asserting against rate-limit responses without tolerance: smoke tests must accept 429 gracefully.
❌ File-shape tests for UX behavior: use jsdom for that. Don't try to grep for "does this button work".
❌ Skipping the verification step after writing a test: always check it would fail with the bug, pass with the fix.

Pre-launch QA checklist — the manual QA pass
Monitoring & Observability — how production failures get caught
Deploy — CI/CD pipeline details

TL;DR — three test classes, three different jobs​

1. Vitest (primary test runner)​

What it is​

Why we picked it​

Test environments​

npm scripts​

Where tests live​

2. File-shape regression tests (the workhorse)​

What they do​

Why we use them so heavily​

What they catch well​

What they DON'T catch​

Example​

Pitfalls (and how we fixed them)​

Verifying a contract test actually catches its bug​

3. Component integration tests (jsdom + React Testing Library)​

What they do​

Why we added them (2026-04-18)​

Stack​

Setup file: tests/setup.dom.ts​

Example​

Current coverage (as of 2026-04-18)​

What to test next (post-launch)​

4. Production smoke tests (hourly cron)​

What they do​

Why we added them (2026-04-18)​

Files​

How they're triggered​

Test data hygiene​

Rate limiting tolerance​

5. Mobile tests (Vitest)​

What they do​

Files​

Why no jsdom for mobile?​

How they're triggered​

6. Visual regression (Playwright)​

What it is​

Status​

Why not heavy E2E with Playwright?​

CI/CD pipeline (where tests run)​

On every push to main (.github/workflows/deploy.yml)​

Hourly (.github/workflows/health-check.yml)​

On every push to mobile (.github/workflows/mobile-tests.yml)​

Test counts (as of 2026-04-18)​

Adding a new test — decision tree​

Anti-patterns to avoid (from bugs_fixed.md)​

Related docs​

TL;DR — three test classes, three different jobs

1. Vitest (primary test runner)

What it is

Why we picked it

Test environments

npm scripts

Where tests live

2. File-shape regression tests (the workhorse)

What they do

Why we use them so heavily

What they catch well

What they DON'T catch

Example

Pitfalls (and how we fixed them)

Verifying a contract test actually catches its bug

3. Component integration tests (jsdom + React Testing Library)

What they do

Why we added them (2026-04-18)

Stack

Setup file: `tests/setup.dom.ts`

Example

Current coverage (as of 2026-04-18)

What to test next (post-launch)

4. Production smoke tests (hourly cron)

What they do

Why we added them (2026-04-18)

Files

How they're triggered

Test data hygiene

Rate limiting tolerance

5. Mobile tests (Vitest)

What they do

Files

Why no jsdom for mobile?

How they're triggered

6. Visual regression (Playwright)

What it is

Status

Why not heavy E2E with Playwright?

CI/CD pipeline (where tests run)

On every push to main (`.github/workflows/deploy.yml`)

Hourly (`.github/workflows/health-check.yml`)

On every push to mobile (`.github/workflows/mobile-tests.yml`)

Test counts (as of 2026-04-18)

Adding a new test — decision tree

Anti-patterns to avoid (from `bugs_fixed.md`)

Related docs