Tally: Building the Ambient AI interfacer With long- term Memory

Tally is a new wearable AI device that captures continuous real-world context through dual 180° cameras and on-device audio — enabling an always-present assistant that understands your environment, remembers your day, and can take actions on your behalf.

In this talk, I’ll walk through how we designed Tally’s hardware and software stack from the ground up:

Ear-level camera earbuds with 360° capture

Real-time video ingestion via RTMP/LiveKit

Continuous memory creation: transcripts, embeddings, moments, and timelines

Our “Speech-to-App” system that turns natural language into actions

Building a developer SDK so anyone can build ambient agents powered by user context

Key challenges in reliability, privacy, and low-bandwidth streaming

This session will include architecture diagrams, live demos, and insights from building Tally through multiple hackathons and investor showcases.