MULTIMODAL AGENTS · research
EchoMind
research runtime for combining desktop context and persistent memory.
What I built
an experimental agent runtime that combines audio, camera, and desktop events with a persistent memory layer. current work focuses on stream ingestion, latency, consent boundaries, and deciding which events are useful enough to retain.
the architecture
Normalize three event streams into one timeline.
Audio, camera, and desktop events are encoded independently and added to a shared timeline. The research question is whether a small retained context can improve an agent without continuously replaying every raw event.
- Audio via WebRTC: speech, tone, ambient sound classification
- Vision via OpenCV: optional face and gaze signals
- OS telemetry via syscall hooks: app focus, typing cadence, idle patterns
memory
Retain selected events across sessions.
The runtime experiments with a sliding-window memory buffer and explicit retention scores. It is a software approximation for selecting context, not a claim that the system remembers like a person.
- Sliding-window episodic buffer with salience weighting
- SECT Model: Salience-Emotion-Context-Time memory architecture
- Cross-session persistence with no full context recomputation
Artifacts and limits
Project images and measurements are shown as project-produced material, not independent validation.
This page intentionally avoids architecture and performance claims that cannot be inspected from a repository, demo, or report.
research
Last content review: July 2026. Scope is limited to the material linked on this page.
Return to projects