Building an Agentic AI That Remembers

Decision Making Vectors by Vecteezy

Why context permanence may be the next frontier of large language models.

The AI journey continues.

Over the weekend, I decided to build my own Agentic AI.

Like many, I’ve been frustrated by ChatGPT’s statelessness. Forgetting context may be a design choice, but for me it was a blocker. I wanted an AI that could log memories, keep track of what I’d told it, and follow hard rules without drifting. I didn’t want to reintroduce myself or restate needs every session.

So, I built one.

Building the Foundation

I used LLaMA 3 locally with Ollama and Streamlit, then deployed via Docker on DigitalOcean so I could access it from my phone. I wired in ChromaDB and sentence-transformers to retrieve past interactions and feed relevant context into each new conversation.

The goal wasn’t to build AGI. It was to create an agent that actually remembers — an AI that can respect rules, constraints, and identity, and surface insights I haven’t received from ChatGPT.

One side observation: I believe one of the biggest frontiers for LLMs isn’t just reasoning, but context permanence. Reasoning that includes an embedded identity and accumulated history is where things get truly interesting. I like that with my setup, I can choose exactly what history persists (unlimited, except by storage cost).

What’s Next: The Insight Agent

Now that the foundation is in place, I’m already building the next layer: an Insight Agent powered by the Model Context Protocol (MCP).

This won’t be just another analytics bot. The goal is an assistant that:

  • Actively helps reason through complexity
  • Flags patterns and anomalies
  • Highlights drift
  • Offers strategic recommendations in plain language

Imagine it saying:

“Throughput dropped 22% over the last two sprints. Code review latency is spiking. You may want to reassess staffing or reset sprint expectations.”

This agent will connect into Jira, GitLab, SonarQube, and even reflections and retro logs. It will surface what really moves the needle — helping raise the bar on engineering efficiency, code quality, and throughput.

Closing Thought

This exploration has reminded me that AI is no longer just about conversation — it’s about memory, reasoning, and agency. Context permanence may be the overlooked key to the next wave of breakthroughs.

I’m excited to keep pushing forward. For me, it’s AI all the way.

Tags

, , , ,

Jen Anderson is an engineering leader, AI strategist, and writer passionate about building high-performing teams and exploring the future of technology. With experience leading transformations across industries—from scaling engineering organizations to pioneering agentic AI—Jen brings both technical depth and human-centered leadership to every project.

Through writing, speaking, and projects like Ask-Jentic, Jen shares insights at the intersection of technology, leadership, and innovation, helping others rethink how we build, lead, and work in the age of AI.