Skip to content

Clothing AI Try-On — Internal Code Documentation

The GenAI Try-On product is served by two repositories:

  • wearfits-genai-api — the Cloudflare Workers API backend, shared with the Shoe 3D Generator
  • wearfits-genai-app — the React 19 frontend and Cloudflare Worker proxy

This document covers both layers. For the complete backend architecture (layering, data model, resilience patterns), refer also to the Shoe 3D Generator code docs, which describes the shared infrastructure in full.

API Backend

Shared Architecture

The backend follows a layered architecture: Entry (src/index.ts) → App (src/app.ts) → Middleware → Routes → Controllers → Services → Providers → Storage. Long-running AI tasks are handled asynchronously via a Cloudflare Queue consumer and processor.

For the full architecture diagram, directory structure, data model, and resilience patterns, see the Shoe 3D Generator code docs.

Digital Twin Flow

A digital twin is a reusable avatar generated from a user selfie combined with either body measurements or a full-body photo. Once created, it is cached for 30 days and reused across fitting sessions.

sequenceDiagram
    participant User
    participant Worker as API Worker
    participant Queue as Job Queue
    participant Modal as Modal Backend
    participant R2 as Storage (R2)

    User->>Worker: POST /api/v1/digital-twin (faceImage + measurements or bodyPhoto)
    Worker->>Queue: Submit Job
    Worker-->>User: 202 Accepted (Job ID)

    Queue->>Worker: Consume Job
    alt Body Measurements Mode
        Worker->>Modal: Body Mask Lookup (BodyM nearest-neighbour)
    end
    Worker->>Modal: SAM3D Body Extraction
    Modal-->>Worker: MHR Mesh
    Worker->>R2: Cache Mesh (30 days, keyed by measurements hash)
    Worker->>Modal: Render Silhouette
    Worker->>R2: Store Result
    Worker->>User: Webhook (job.completed, silhouetteUrl)

Twin creation modes:

Mode API payload When used
Size mode { faceImage, gender, clothingSize: { height, size }, poseId } Default; user selects gender, height (140–210 cm), size (XS–3XL)
Upload mode { faceImage, bodyPhotoUrl, poseId } User uploads a full-body photograph

Body Measurements Mode uses the BodyM dataset. The worker calls v1-body-mask-from-size (Modal) to find the nearest matching body mask via nearest-neighbour lookup on height, chest, waist, hip, and inseam fields (at least 2 required). The resulting mask is passed to SAM3D for mesh extraction, and the final mesh is cached by a hash of the provided measurements to skip SAM3D on repeat queries.

Virtual Fitting Flow

Virtual fitting applies garment images to an existing digital twin using neural pose transfer and rendering.

sequenceDiagram
    participant User
    participant Worker as API Worker
    participant Queue as Job Queue
    participant Modal as Modal Backend
    participant R2 as Storage (R2)

    User->>Worker: POST /api/v1/virtual-fitting (digitalTwinId + garmentIds)
    Worker->>Queue: Submit Job
    Worker-->>User: 202 Accepted (Job ID)

    Queue->>Worker: Consume Job
    Worker->>Modal: Pose Transfer (garment + twin mesh)
    Modal-->>Worker: Composed Image
    Worker->>R2: Store Result
    Worker->>User: Webhook (job.completed, resultUrl)

Each virtual fitting request accepts up to two image URLs per garment (e.g. front and back packshots). The result is a rendered image of the digital twin wearing the specified garments.

Frontend Application

The frontend is split into a React 19 SPA and a Cloudflare Worker that acts as a secure proxy between the browser and the WEARFITS Core API.

graph TD
    subgraph "Frontend Browser"
        UI[React Components]
        Hooks[Custom Hooks / Logic]
        Context[Context API / Global State]
        Service[API Client / Services]
    end

    subgraph "Serverless Middleware Cloudflare"
        Worker[Cloudflare Worker Proxy]
        WAF[Cloudflare Firewall / Turnstile]
    end

    subgraph "AI Infrastructure External"
        API[WEARFITS Core API]
        DB[(Session Database)]
    end

    UI <--> Hooks
    Hooks <--> Context
    Context <--> Service
    Service -- "HTTPS + Turnstile" --> WAF
    WAF <--> Worker
    Worker -- "Secure Proxy" --> API
    Worker -- "Query" --> DB

Directory Structure

src/
├── components/
│   ├── common/          # Reusable primitives: buttons, modals, spinners
│   ├── twin/            # Face capture and body profile forms
│   ├── fitting/         # Garment browser and result visualisation
│   ├── shoes/           # Dedicated 3D shoe try-on flow
│   └── layout/          # App shell and full-screen containers
├── context/             # Global state (ConfigContext, TwinContext, FittingContext)
├── hooks/               # Business logic hooks
├── services/            # API client, Turnstile, cookie/localStorage persistence
├── embed/               # Iframe integration script and postMessage bridge
└── styles/              # Shared CSS variables and globals
worker/                  # Cloudflare Worker source (API proxy)
e2e/                     # Playwright end-to-end test suites

Context Modules

The app uses a Reducer-in-Context pattern to avoid prop drilling while enforcing strict state transitions.

Context Responsibility
ConfigContext Application settings: locale, theme, platform
TwinContext Digital twin state, IDs, and silhouette URLs
FittingContext Current garment selection and fitting results

Custom Hooks

Hook Responsibility
usePolling Generic hook for async job status polling
useDigitalTwin Twin creation request and polling orchestration
useVirtualFitting Fitting request and polling orchestration
useCamera MediaStream management and face detection

Key Flows

Digital Twin Creation (Frontend)

sequenceDiagram
    participant U as User
    participant R as React App
    participant W as Worker Proxy
    participant B as Backend API

    U->>R: Captures Selfie & Enters Measurements
    R->>R: Process Images (resize / optimise)
    R->>W: POST /api/v1/digital-twin (with Turnstile Token)
    W->>W: Verify Turnstile & Inject API Key
    W->>B: POST /api/v1/digital-twin
    B-->>W: 202 Accepted (Job ID)
    W-->>R: 202 Accepted (Job ID)
    R->>W: GET /api/v1/jobs/:id (polling)
    W->>B: GET /api/v1/jobs/:id
    B-->>W: 200 OK (status: completed, result: {silhouetteUrl})
    W-->>R: 200 OK (silhouetteUrl)
    R->>U: Display Twin Silhouette

Virtual Fitting (Frontend)

sequenceDiagram
    participant R as React App
    participant W as Worker Proxy
    participant B as Backend API

    R->>W: POST /api/v1/virtual-fitting (twinId, garmentIds)
    W-->>B: Proxy with Auth
    B-->>W: Job ID
    W-->>R: Job ID
    loop Polling
        R->>W: GET /api/v1/jobs/:id
        W->>B: Proxy
        B-->>W: Status (pending / processing)
        W-->>R: Status
    end
    B-->>R: Status: Completed (Result Image URL)

State Management

The TwinContext state shape:

interface TwinState {
  digitalTwinId: string | null;
  status: 'idle' | 'creating' | 'completed' | 'failed';
  silhouetteUrl: string | null;
  error: Error | null;
}

Multi-Pose Support

Each pose (e.g. default, walking_pose) requires its own digital twin. Twins are persisted in cookies keyed by poseId:

{
  "twins": {
    "default": { "digitalTwinId": "...", "createdAt": "..." },
    "walking_pose": { "digitalTwinId": "...", "createdAt": "..." }
  }
}

When the user selects a new pose, a twin is created on demand if one is not already cached. Cached twins are validated on application load; invalid entries are removed.

Security

Worker Proxy Pattern

The Cloudflare Worker sits between the browser and the WEARFITS Core API. Its responsibilities are:

  1. API key injection — the WEARFITS_API_KEY secret is stored in the Worker environment and injected into outbound requests; it is never sent to or stored in the browser
  2. Turnstile verification — the worker verifies bot-protection tokens by calling https://challenges.cloudflare.com/turnstile/v0/siteverify using TURNSTILE_SECRET_KEY before forwarding any request
  3. CORS handling — origin validation is enforced at the Worker level
  4. Log sanitisation — request and response data is sanitised before any logging to prevent credential or PII leakage

Rate Limiting (Frontend)

The React app enforces client-side rate limiting (10 requests/hr, 20 requests/day) using dual storage — cookies and localStorage — for redundancy against partial storage failures.

Testing

Unit and Integration (Vitest)

  • Components are tested with @testing-library/react
  • API calls are mocked using MSW (Mock Service Worker)
  • Run with npm run test (watch mode) or npm run test:run (single pass)

End-to-End (Playwright)

  • Full user journeys are covered in e2e/
  • Camera is mocked with --use-fake-ui-for-media-stream to enable headless execution
  • Run with npm run test:e2e

Tech Stack

Component Technology
Frontend framework React 19
Build tool Vite
Styling Vanilla CSS, CSS Modules, CSS Variables
Worker proxy Cloudflare Workers (JavaScript)
Security Cloudflare Turnstile
Unit testing Vitest + @testing-library/react + MSW
E2E testing Playwright

Known Technical Debt

  • TypeScript migration — the frontend is currently JavaScript-only. Migrating to TypeScript is recommended for improved type safety in context reducers.
  • Polling interval — the polling interval is fixed. Implementing exponential backoff would reduce load on the Worker proxy.
  • Offline resilience — failed jobs are not persisted; they must be restarted if the page is refreshed before completion.