🗣️ Prism AAC

Open source (AGPL-3.0). Hosted free at prism-aac.vercel.app. Standalone repo: prism-aac.

---

🖼️ Pictures → Words → Speech (PECS-style)

The core AAC primitive. Tap picture tiles to build a sentence; the app reads it aloud in the child's language.

* 22 default categories — Food, Feelings, School, Body, Animals, Colors, Time, Help/Needs, Quick Talk, etc. Caregiver-extendable.

* Pictogram library — high-contrast, high-readability symbols designed for low-vision users.

* Sentence bar — tap-and-build, then 🔊 to speak. Long-press to edit a tile.

* Per-child layouts — a BCBA can lock categories visible to a specific child.

---

⌨️ Keyboard + Word Prediction

For older or higher-functioning users, a full keyboard with multilingual prediction.

* Per-user learning — prediction model adapts to the child's vocabulary over weeks.

* Autocorrect — fixes hurried/motor-impaired typing (typos, dropped letters, missing diacritics) via Gemini 2.5 Flash-Lite. Multilingual: en/ru/ro/es/uk/pl + Cyrillic/Hebrew/Arabic script detection.

* Word completion — partial → full word in 750ms median (hw → how are you).

* Local-first — when prism-coder:14b is running on the device, autocorrect runs offline with zero network.

---

🍽️ Food Ordering & Real-World Scenarios

Pre-built sentence chains for common situations — restaurants, doctor visits, school transitions.

* Quick-order tiles — "I want pizza", "with cheese", "no onions" — chained in 3 taps.

* Allergy alerts — child can self-advocate ("I am allergic to peanuts") with a single tap.

* Caregiver-customizable menus per restaurant chain (McDonalds, Subway, Olive Garden, etc.).

---

🧮 Math Panel (Panther Math Paper-style)

Graph-paper canvas with KaTeX rendering. Designed for school-age users who need to do math without speech.

* Drawing surface — pen + eraser, snapping grid, geometric figures (circle, triangle, parallelogram).

* KaTeX equation editor — write \frac{x^2}{y} and have it rendered + spoken aloud.

* AI tutor (paid) — speaks answers, explains step by step. Backed by Claude Sonnet 4 with Claude Haiku 3.5 fallback.

* Save & share — exports to caregiver notes for homework review.

---

🎮 Therapeutic Games (9, evidence-based)

Built to teach communication, NOT for screen-time entertainment.

* Bubble Pop — single-cause-effect for early causality learners.

* Color Hunt / Match It / Category Sort — receptive language + categorization.

* My Story — generative narrative building with picture sequences.

* Yes/No / Finish It / Emotion Match / What Comes Next — comprehension + theory of mind.

* Free tier: 3 games. Paid tier: all 9 + caregiver-tuned difficulty.

---

👋 Hands-Free with Gesture Recognition

For users who can't reliably tap a touchscreen.

* Head-pose tracking + dwell-click via FaceLandmarker (MediaPipe Tasks). Local, browser-only — no video uploaded.

* Hand-pose tracking for users with limited fine motor control — pinch, point, swipe.

* Per-user gesture profiles — calibrate once, the app remembers.

* Configurable dwell time — 200ms (fast) to 2000ms (cautious).

---

🗓️ Visual Schedule & Reward Shop

Picture-based daily routines that reduce transition anxiety.

* Time-blocked tiles — "8am Breakfast", "9am School", "12pm Lunch", with pictograms and a check-off ritual.

* Reward shop (paid) — children earn tokens for completed tasks, redeem for caregiver-approved rewards (extra game time, choice of dinner, etc.). Trauma-informed: opt-in, no punishment mechanic.

---

🩺 Clinical-Grade Defaults

Designed with BCBAs and SLPs.

* Verbal operant tracking — matches BACB Task List 5th Edition (mand, tact, intraverbal, echoic).

* Caregiver notes sync (paid) — ABC-format notes travel encrypted between home, school, and clinic via the Synalux portal.

* AAC access never restricted as a consequence — hard-coded into the app design. A child must always have their voice.

* Trauma-informed defaults — no punishment mechanics, reward shop is opt-in, all "negative feedback" UX flows have been removed.

---

📴 Works Offline

The AAC primitive is the user's voice. It cannot depend on a network.

| Feature | Offline-capable | How |

|---|---|---|

| Keyboard input | ✅ | Pure UI |

| Pictogram tiles + categories | ✅ | Bundled assets, IndexedDB persist |

| Local word prediction | ✅ | engine/predictionEngine.ts (n-gram + user history) |

| Translation (14 locales) | ✅ | Bundled offlineDictionary.ts (4000+ word pairs) |

| Whisper transcription | ✅ | WASM in-browser model |

| Neural TTS (Tier 1.5: Kokoro-82M) | ✅ en/es/fr/pt/ja/zh | Bundled neural model |

| OS Web Speech API premium voices | ✅ | Browser-native |

| WASM espeak-ng (last resort) | ✅ | Bundled fallback |

| Categories/phrases/notes | ✅ | Zustand + IndexedDB |

| Emergency alerts (10-min queue TTL) | ✅ | Local queue, retry on reconnect |

| Sync to Synalux cloud | ❌ online-only | Background, non-blocking |

| AI chat assistant | ❌ online-only | Claude Sonnet 4 via portal |

| Cloud autocorrect (Gemini Flash-Lite) | ⚠️ degrades to local | 5s timeout → local prediction engine |

When the network goes, the device still talks.

---

🎙️ Voice (TTS) Architecture — 4-Tier Fallback Chain

| Tier | Engine | Quality | Offline | Notes |

|---|---|---|---|---|

| 1 | Inworld TTS-2 | Best (paid all langs; free for ro/uk/ru/de/ko/ar — Synalux absorbs cost) | — | Default for paid tier |

| 1.5 | Kokoro-82M neural | Very good | ✅ en/es/fr/pt/ja/zh | Bundled WASM |

| 2 | OS Web Speech API premium voices | Good | ✅ | Browser-native |

| 3 | WASM espeak-ng | Acceptable | ✅ | Last resort |

Voice picker (paid): all 60+ Inworld voices, including child voices, accents, and voice cloning ("speak in MY voice — I trained it last week").

---

🏗️ Architecture (server-side via Synalux portal)

| Path | Function | Model |

|---|---|---|

| /api/v1/text/correct | Autocorrect + word completion | Gemini 2.5 Flash-Lite (multilingual bench-validated; 752ms avg; 4.3× cheaper than 2.5 Flash) |

| /api/v1/prism-aac/predict | 5-word continuation prediction | Gemini 2.5 Flash-Lite (free tier); Claude Haiku/Sonnet (paid) |

| /api/v1/prism-aac/chat | AAC AI assistant | Local prism-coder:14b (primary) → Claude Sonnet 4 (cloud paid). Fallback: Claude Haiku 3.5 / Gemini 2.5 Flash by tier. |

| /api/v1/prism-aac/tts | Text-to-speech | Inworld TTS-2 + Azure Neural fallback |

| /api/v1/translate | Sentence translation | Gemini 2.5 Flash + offline dictionary fallback |

| /api/v1/transcribe | Voice → text | Whisper (WASM, in-browser primary) |

---

💳 Plans (Synalux pricing)

| | Free | Paid |

|---|---|---|

| Picture tiles + 22 categories | ✅ | ✅ |

| Type-to-speak | ✅ | ✅ |

| Default voice (Inworld) | ✅ | ✅ |

| Math panel | ✅ basic | ✅ + AI tutor |

| Schedule | ✅ | ✅ + reward shop |

| Games | 3 | All 9 |

| Voice picker (60+ Inworld voices) | — | ✅ |

| Voice cloning | — | ✅ |

| Caregiver notes sync | — | ✅ |

| Word prediction (per-user learning) | — | ✅ |

| AI chat (Claude Sonnet 4 + tier fallback) | — | ✅ |

---

🧪 Self-Host & Fork

``bash

``

AGPL-3.0 — fork freely; share modifications. Synalux operates the canonical hosted version. See prism-aac/CONTRIBUTING.md, ACCESSIBILITY.md, SECURITY.md.