Inspiration
The idea came from Iron Man's mid-air gesture interface, in which a person moves 3D models by hand. I wanted to find out whether I could recreate it with hardware everyone already has.
What it does
The dashboard shows a wireframe sphere with 8 sector panels orbiting it and 4 global panels above it, covering 64 real US tickers grouped by GICS sector. You point with your index finger to move a cursor, hover on a panel for a second to open it, pinch and drag to rotate the sphere, and open your hand to close the view. The cursor snaps to nearby panels, so you do not need to point precisely.
When the frame rate stays low for a few seconds, the app switches to Performance Mode. Performance Mode lowers the pixel ratio and bloom, which raises the frame rate by about 2× on integrated graphics.
How it was built
I wrote a specification document first, then refined the app step by step with Claude Code. The app uses Three.js for the scene, MediaPipe hand landmark detection in WebAssembly on the CPU, and vanilla JavaScript built with Vite. The app includes the MediaPipe bundle, so the demo runs without a CDN.
Challenges
I developed on a laptop without a GPU, so the app lagged constantly during testing. Mid-air controls with unfamiliar gestures took many rounds of experimentation before they felt natural.