Multiplayer VR · Unity 6 · 2026

Call Me Merlin

Networked VR spell-casting with bare hands — draw shapes in the air, cast synchronized magic.

Team: Qinchuan Zhang · Weibo Xu
Stack: Unity 6 · Photon Fusion 2 · XR Hands
Date: May 2026

Overview

Hand tracking ships on millions of headsets, yet multiplayer VR still defaults to controllers. Call Me Merlin closes that gap: a peer-to-peer VR battle where players cast spells by drawing magic-circle shapes in the air with bare hands — and every trajectory, circle, and spell effect stays synchronized across clients in real time.

Gesture recognition

Index-fingertip trajectories are projected to a head-relative 2D plane, Gaussian-filtered to remove hand tremor, sparsified, and segmented by directional deviation — then classified against 17 gesture templates in the spirit of the $1/Protractor recognizers. Shapes map to elements: circles to fire, lines to wind, triangles to water, squares to earth.

Two-phase interaction

Casting is split into two phases to prevent accidental triggers: draw a shape to load an element (a world-anchored magic circle confirms it), then strike a hand pose to fire — gun-point for projectile, open palm for blast, double thumbs-up for a sustained beam, double fist for shield. Four mechanics × four elements yield sixteen visual variants.

Networked architecture

A strict four-layer pipeline (perception → routing → trigger → effect) keeps the system extensible — adding a spell never touches perception or effect code. Photon Fusion 2 in shared mode synchronizes trajectories via RPCs and gesture state via [Networked] properties, while VFX spawn as network objects so visuals and spatial audio stay consistent on every client without extra round-trips.