Winner · GenAI · WebXR · Agents · 2026

SnapFit

Snap a photo of any furniture and get an interactive 3D assembly guide — AI segments the parts, plans the steps, and coaches you through building it in MR.

Team: Qinchuan Zhang · Solo
Type: Hackathon · GenAI × XR
Award: Winner · Worlds in Action Hack [02-LA]
Stack: TRIPO · DeepSeek · R3F · WebXR · ElevenLabs
Date: Jul 2026 · Worlds in Action Hack, LA

Overview

Furniture manuals are a famous UX failure, and long-tail sellers can't afford 3D teams to fix them. SnapFit is a zero-modeler pipeline: a seller uploads one product photo, and the system generates a part-segmented 3D model with semantic part names and a step-by-step assembly plan. Buyers open it on desktop, phone, or a headset and physically drag parts into place with a voice-guided AI coach. Built solo at Worlds in Action Hack [02-LA].

The core insight

A model generated from a photo of assembled furniture has every part already sitting in its assembled position — so the correct placement needs no human annotation. Each part's snap target is simply its original transform: exploded views translate parts out along their centroids, and snapping lerps them back when distance and rotation fall under a threshold. The whole interaction layer is deterministic code that works for any part list, whether segmentation returns five pieces or fourteen.

Agents that can't hallucinate geometry

LLMs can't read GLB files, so the compile-time agent gets a translated view: per-part geometry summaries plus color-coded renders for multimodal grounding, with TRIPO's Smart Segmentation v2 providing the parts and GPT-Image-2 cleaning up product photos. DeepSeek then plans assembly as a dependency DAG rather than a linear list — four table legs don't care about order. Every output passes deterministic validators (each part used exactly once, DAG acyclic, targets already installed) that feed concrete errors back for retry: hallucination control by architecture, not prompt engineering.

The runtime coach

At runtime a lightweight coach agent tracks spatial state — the current step, snapped parts, what's in your hand — and only calls the LLM on events: a wrong part picked up, a step timing out, a voice question. It answers in one sentence through ElevenLabs voice while highlighting the part it means, so speech and spatial action come from the same call. The React Three Fiber + WebXR runtime deploys the same experience to browser, phone, and headset from one codebase.