Building session memory for an AI tutor: what we got wrong first

The most common question we got in the early days of MyPyMentor was some variation of: 'Does Py actually remember me?' As in: if I come back after a week away, will Py know where I was? What I was struggling with? What we covered last time?

The answer is yes. But getting there required us to throw out our first three attempts and build something we didn't expect.

The naive approach (attempt 1)

Our first implementation was simple: pass the last 20 messages from the current session as context to the AI. It worked, sort of. Py could reference recent things accurately. But there were problems.

Twenty messages is a lot of tokens. For users on the free plan hitting the daily limit, a significant fraction of their available context was being consumed by the history. The responses got slower. And the AI sometimes got confused by outdated details buried in the history — a misconception the user had had at message 5 but corrected by message 15 would occasionally surface again at message 22.

The second attempt

We tried just passing a raw dump of the previous session: 'here's what you talked about last time'. The problem was that raw dumps aren't useful. They don't distinguish between what was important and what wasn't. Py would give equal weight to 'the user asked what a variable is' and 'the user has been stuck on recursion for two sessions and seems to find it genuinely difficult'. Those are not equally important.

What we actually built

The current system generates a structured summary after every session that reaches a certain length. The summary extracts five specific things:

1. Concepts covered in this session — what topics came up
2. Concepts that seemed clear — where the learner demonstrated understanding
3. Concepts that seemed difficult — where confusion appeared, re-explanations were needed, or the learner said they didn't understand
4. The learner's emotional state at the end of the session — engaged, frustrated, tired, confident
5. Recommended starting point for next session — what should come next given what happened

That summary — not the raw chat — is what gets passed to the next session. It's compact, structured, and focused on what actually matters for teaching.

What we learned from this

The biggest insight was that memory in a tutoring context is not the same as memory in a general conversation. A general AI assistant benefits from remembering everything said. A tutor benefits from remembering the right things — specifically, what the learner knows, what they don't, and what emotional state they're in.

Designing the summary schema was the hardest part. Every field we added created a decision about what counts as 'a concept that seemed difficult'. We got this wrong multiple times before we got it right.

The current system isn't perfect. Summaries occasionally miss nuance. But when it works — when Py opens a new session with 'last time you were working through recursion and it seemed like the base case was the sticking point, do you want to try a different example?' — that's the experience we were trying to build.