Building PROTO RECON — From Vague Idea to a Browser-Based Tactical HUD

Introduction

This post walks through how PROTO RECON — an experimental app that combines phone sensors with in-browser ML — evolved from pre-coding requirements to its current implementation. I've verified the UI behavior, but I'm still at the stage of reading the source code for the first time. Here I organize the original requirements and an AI-generated source map as a foundation for deeper dives in future posts.

• Live demo (experimental): Proto Recon (Cloudflare Workers)
• Repository: github.com/katsutoshi0katsutoshi/proto_recon
• Sandbox for isolated experiments: CodePen / @wlzpaovm-the-bashful

Note: This is still in testing. Performance tuning for heavy workloads hasn't been done yet — use at your own risk.

Where It Started

It began with a vague question: Can we build a digital user experience on top of the real world? The seed of the idea was whether today's phone sensors could overlay something like the robot-view or cockpit-view perspectives from old sci-fi and games onto live scenery.

Before writing code, I summarized the requirements, turned functional specs into prose through dialogue with AI, and then started development.

Technology Choices (Before Implementation)

The noise layer was added later. When I had Cursor implement it, three.js was skipped in favor of Canvas 2D and CSS for CRT-style effects. Writing directly without extra modules kept things lighter — a decision that has paid off on real devices.

Gyro, compass, altitude (on supported devices), GPS, and similar sensors were also in scope, wired to the HUD and minimap.

Requirements at the Time (Summary)

The planning document — built through repeated conversations with AI — was titled Tactical Recon & Guidance Terminal "PROJECT: LOCK-ON (working title)". At that point, sensor accuracy led us to assume Phase 2 would require a Unity (native) rewrite. After coding with Cursor, however, things moved better than expected even without three.js, and the outlook shifted to Phase 1 might be achievable on the web alone.

Core Concept

• Visuals: Monochrome green military-terminal UI, scan lines, noise
• AI: Real-time object detection for targeting and lock-on
• Immersion: Gyro, altitude, BEEP sounds — instrument and audio feedback
• Future: VR-goggle HUD, gaze control (concept stage)

Plan vs. Current State

The full planning document is collapsed below due to length.

How Development Proceeded

Coding was delegated to Cursor, adding features one prompt at a time until the current state. For prompts, I load the requirements document into Gemini and ask what kind of prompt would work best. I've checked JS behavior through the UI, but reading the source comes next. The file list and dependency diagram below were AI-generated and cleaned up for this article.

Source Layout

HTML (root level)

src/ — runtime files

Entry, config, UI

Video & AI inference

Distance & AR

Navigation, map, sensors

Feedback & monitoring

src/ — tests only (not used on screen)

Dependencies (overview)

flowchart TB
  index[index.html] --> main[main.js]
  main --> config[appConfig / applyDevConfig / uiLexicon]
  main --> cam[camera + HUD]
  main --> det[personDetection]
  main --> face[facePrivacy]
  det --> tf[tfBackend]
  face --> tf
  main --> map[miniMap]
  map --> leaf[miniMapLeaflet]
  main --> nav[compass / motionHud / motionStabilityLink]
  main --> range[objectRange]
  range --> xr[webxrHitTestRange]
  main --> health[liveHealth]
  main --> sfx[audioFx]

main.js ties almost everything together; appConfig.js settings and uiLexicon.js copy drive behavior and display across modules.

Closing — What to Read Next

Future posts will open the source and go deep feature by feature. The study roadmap below is the planned reading order (01 is done with this article's file map).

The next post will start with roadmap #02 (appConfig.js).