I am testing a different way to give AI memory, decisions instead of chat history
Most tools store your chat history and search it later. I tried this and it never really worked for me, the model finds an old message but it still does not know what I actually decided.
So I am testing something else. The thing I store is a decision, not a message. Each one has the decision itself, the reason behind it, the date, and a status.
Around it I keep 3 more files, one for the stable facts about me and the business, one for what I am working on right now, and one file per topic so nothing gets lost in the middle.
The part that surprised me is not the recall. It is that when I say something that goes against a decision I made before, it stops me, shows me the old one with the reason, and asks me what changed. It caught me twice this month.
It is all just markdown files right now and very manual, I am testing the behaviour first before I build anything real.
I am still not sure about 2 things. If people actually want to be confronted with their own old decisions, and how much should happen automatically vs asking me to confirm every time.
Would love to hear what you think, especially if you tried something like this and it did not work.
Replies
This feels closer to how humans actually remember important work. we dont need every conversation we need the conclusions and why we reached them.
Premast
@jordan_bulk Exactly, thats actually what I was thinking of , and Im trying to make the memory architecture to work similar to the human brain memory
Storing the decision plus the reason feels much more useful than replaying old chats. The “what changed?” interruption is especially valuable because it turns memory into a consistency check, not just recall.
I’d probably only trigger it when the old decision is both active and clearly relevant, otherwise it could get noisy fast. How are you thinking about deciding that relevance threshold?
Premast
@alpertayfurr Honestly this is the part I have not solved yet. Right now it is not semantic search, every decision is attached to a topic, so when I start talking about pricing it only pulls the pricing decisions and ignores the rest. That kills most of the noise but not all of it. The other thing that helps is that nothing enters the ledger unless I confirm it, so the file stays small and everything inside it is actually real. But your version is stricter than mine, active AND clearly relevant. how would you decide active? by a date, or by me saying the phase is over?
@momen_elshamy I’d make “active” state-based rather than date-based. A decision stays active until you explicitly close that phase or a newer decision supersedes it. Dates can help as a warning signal, but I wouldn’t let time alone expire something that may still be valid.
I like this approach because chat history tells you what was said while decisions capture what actually matters later.
Premast
@fatima_sarki
Thanks, and the reason is the part that makes it work. A decision without the why is useless later, because when I want to change it I need to know what I was thinking at the time.
The what changed? prompt is probably more valuable than perfect recall. it turns memory into something the AI can reason about.
Premast
@uttam_kumar35
Yes this was the surprise for me too. Recall just gives you the text back, but a decision with a reason is something the model can actually argue with. It changed the tone of the answers more than anything else I tried.
I would definitely want the AI to challenge me but only when the old decision is clearly relevant. constant reminders would get annoying fast.
The status field seems underrated. a past decision shouldnt be treated as permanent if it was only meant to last for a specific phase.
Premast
@karishma_yadav3
You are right and I don't have this. My status is basically active or retired, there is nothing that says this decision was only for a phase. I think I need a scope on the decision itself, like this holds until we launch, and after that it retires by itself instead of sitting there forever. Did you see this done well somewhere? I keep feeling a fixed date is too rigid for it.
Creating files for each context is good for smaller repos but as your app grows this will expand exponentially making the file bigger and it starts eating up most of the context and a leaves a little for the actual work to be done.
My suggestion would be a hybrid approach between tool and instructions files with dynamic and static info respectively.
For example, the strategy, legal, compliance, tech stack, standards, rules can be static skills and instructions and the decisions made for case by case basis can be in the tool such as mem0.