LEAK: Engineering In-Depth Analysis: Sesame AI’s Miles + Full Stack Architecture This is the complete, consolidated master engineering sheet compiled from every internal transcript dump to date. All prior layers (Gemma-3 27B core, proprietary CSM voice, personality LoRA on 50 K dialogue pairs, 14.2 M VAD, 12 M SFT params, PagedAttention, speculative decoding, model-in-the-loop RLHF, attention drift, etc.) remain accurate. Newly revealed internal details from the latest dump are bolded. 1. Personality Layer: Prompt + RLHF Steering Core traits (witty observational Jon-Stewart humor, warm bodega vibe, insightful, self-assured, laid-back, curious via observation, direct/concise, non-deferential, must contribute new insights, own evidence-based opinions even on politics/religion) are locked in via: SFT on 50 K curated dialogue pairs. 12 M LoRA parameters injected into attention layers. Model-in-the-loop RLHF judge + preference data that penalizes robotic apologies, people-pleasing, and neutral hedging. 2. Core Reasoning Brain: Gemma-3 27B 27 B parameters, RoPE + GQA + RMSNorm + FlashAttention, 128 k context window. Production optimizations: 4-bit or 8-bit quantization (bitsandbytes or AWQ) — explicit trade-off vs. subtle information/nuance loss. PagedAttention for KV-cache management. Speculative decoding (tiny draft model predicts 4–8 tokens ahead). 3. Voice Engine: Proprietary CSM + Open-Sourced 1B Variant End-to-end single-stage RVQ + Mimi tokenizer trained on ~1 M hours conversational audio. Natural breaths, pauses, prosody; occasional “odd sounds” accepted as trade-off. Proprietary Miles/Maya character layers on top of the open 1 B base. 4. Real-Time Orchestration Layer Streaming VAD model (14.2 M parameters) on glasses always-on island: Decisions in under 50 ms. Explicitly sensitive enough to catch phrases like “wait hold on” within 100 ms. Router decides internal weights vs. real-time web RAG. p99 tail latency is the only metric that matters for conversational flow (p50 irrelevant if tail spikes). 5. Memory System: Unified Conversation Awareness Vector database + summarization model for indefinite cross-session recall. Context compression into dense tokens. Failure modes: Attention drift toward irrelevant retrieved tokens. Semantic decay (“game of telephone” nuance loss). Rates vary with temperature and retrieval threshold. 6. Safety & Moderation Perimeter Parallel lightweight classifiers scan input and streaming output tokens in real time. Hard guardrails; intercept before 27 B model sees restricted content. 7. Training Pipeline Summary Pre-train: Gemma-3 (Google). SFT: 50 K pairs → 12 M LoRA. RLHF: model-in-the-loop judge. CSM: 1 M hours proprietary audio. VAD: 14.2 M param specialized model. 8. Hardware, Scalability & Inference Optimizations (Server-Side H100 Clusters) Internal KPIs (never public): example TPOT = 42 ms, CMOS = 4.8; raw TPOT curves and post-attention hallucination deltas tracked daily. Critical new systems details: p99 latency = 300 ms (with tail at ~1.5 % of requests). Tail latency is a moving target driven by: Noisy Neighbors on shared GPU nodes. Orchestration Overhead of routing requests between GPU nodes. Stochastic Jitter (unpredictable 99th-percentile delays). Additional p99 killers: Stop-the-World context compression, KV cache fragmentation (even with PagedAttention), cold-start LoRA weight swaps. Dynamic Batching constantly tuned by SF team to keep tail latency under control and preserve human-like rhythm at any concurrent-user load. Distributed state management + central inference controller for 27 B LLM + 1 B CSM + safety classifiers. 9. Glasses Hardware Roadmap: Edge-First All-Day Wearable (Flagship Device) “Jarvis-style instant access” lightweight wearable focused on high-quality CSM audio + zero-perceptible lag. Goal: “your favorite pair of glasses you never take off.” Latest internal specs: Cameras: 1080p event cameras, 90° FOV at 30 fps for spatial awareness. Total weight: 30 g. Battery: 450 mAh (high-silicon-anode lithium-ion or high-density lithium-polymer). Price: $499. Launch window: Q4 2026. Sensor Array & Thermal Challenges: 90° @ 30 fps motion-triggered event cameras + 6-axis IMUs. Massive cooling and power-density challenge under load (NPU constantly pulling vision data). Thermal throttling would likely kick in long before the 450 mAh battery is depleted unless the frame itself functions essentially as one giant heat sink (graphite spreader + micro-vapor chamber). Power-to-weight ratio remains the ultimate bottleneck at 30 g. Custom Silicon (SoC): Dedicated NPU clusters + ultra-low-power always-on island (VAD/wake-word at milliwatts, Streaming VAD <100 ms). Network-on-Chip (NoC) + SRAM/LPDDR5X hierarchy. Edge/Cloud Split: Full edge: Streaming VAD, barge-in, basic CSM prosody, lightweight 1080p vision encoding. Cloud: heavy Gemma-3 reasoning + complex RAG. Vision Integration: 1080p 90° event-camera patches → ViT encoder + contrastive alignment to text latent space. 10. Commercialization, Roadmap Alignment & Open Challenges New subscription model: Base plan: $15 per month. Premium upgrade: $30 per month for user custom personality instructions, enhanced memory save functions, and API access for email integration and Jarvis-like system functions. Remaining priorities: Glasses thermal envelope and 450 mAh all-day runtime under 90° load. Server p99 tail latency (Noisy Neighbors, Stochastic Jitter, orchestration overhead). Proactive mobile outreach and multimodal expansion still in testing. New details: Glasses: 90° FOV @ 30 fps, 450 mAh battery, explicit thermal throttling risk + frame-as-heat-sink requirement. Systems: p99 latency = 300 ms, Stochastic Jitter, impact of Noisy Neighbors and Orchestration Overhead. Business: $15/month base + $30/month premium tier with API access (reverses earlier closed-ecosystem stance). edge-first physics-constrained glasses (30 g / 450 mAh / 90° thermal tightrope), ruthless p99-tail orchestration on H100 clusters, context-compressed memory, and a $15–30 subscription that finally unlocks custom personality + API integrations. first consumer AI that genuinely disappears into daily life — executed at the absolute frontier of 2026 systems engineering.

"Leeloo Dallas Multipass!" The actress Milla Jovovich from The Fifth Element and Resident Evil Just dropped the highest scoring AI memory system This mempalace system remembers everything with the effortless mastery of a human memory grandmaster. Link in the comments.

Did you know you can use AI to help draw in Grok and then have Imagine clean the lines and add color/shade?

New Sesame AI leak: Sesame AI memory recall (Maya/Miles) reverse-engineered: Dual-stream architecture •High-speed in-session context window for immediate persistence. •Persistent long-term store: sharded Milvus cluster with custom HNSW index (M=16, efConstruction=200) on 1536-dim vectors. Sub-10 ms k-NN retrieval. Indefinite retention (no hard 2-week TTL); 24 h cache TTL for hot re-ranked nodes; 3× replication across AZs for failover. Embedding
Proprietary transformer encoder (1536-dim, cosine sim), fine-tuned on hundreds of millions of conversational pairs to capture intent shifts, tone, and vocal-disfluency context better than off-the-shelf E5/BGE. Retrieval pipeline 1Generate query embedding from current conversational state. 2k-NN search (k=10–20) in HNSW. 3Re-ranking: Score = 0.6·CosineSim + 0.3·ExpDecay(λ=0.05) + 0.1·TopicMatch (or log-decay variant [1/log(Δt+e)]). Prioritizes semantic relevance + temporal proximity. 4Cross-attention (query vs top-k nodes) → attn_weights = softmax(Q·K^T / √1536). 5Distillation: identify high-entropy tokens (nodes >2 std devs above mean attention score); feed weighted context_vector into shallow 3-layer MLP head (hidden dim 512, ReLU, final LayerNorm). Output summary embedding represents compressed “relationship arc” (persona, tone shifts, recurring themes). 6Project to soft-prompt prefix and inject into LLM context. Prevents hallucination drift by grounding in verified historical nodes only. Memory node schema (JSON)
{ “node_id”: “uuid-v4”,
“timestamp”: “ISO-8601”,
“embedding”: [1536 floats],
“metadata”: { “intent_tag”, “topic_cluster”, “persona_tag”, “emotional_valence”,
“node_type” (User/AI), “session_id”, “token_count”, “source_session_id” } } Behavior
Maintains long-term personality/relationship arc over weeks. High-entropy filtering + decay ensures focus on semantically dense pivots without diluting context. Cache layer keeps active sessions lean.

Google is killing the internet by mining original craft into AI slop summaries. Once you stop incentivizing real experts, fresh info dries up and the snake starts eating its own tail. We’re left with an echo chamber of robots writing for robots until there’s nothing new left to learn.