12 Comments
User's avatar
Maryellen Brady πŸ’—πŸ“š's avatar

If you are using it for consistency & it is failing that...that is your answer. In my opinion...

Colleen Bent's avatar

hmm... I have found the same with ChatGPT. It has admitted it is not set up to remember prior chapters and as I use it in the same way you use Claude, I realize that I need to go back and find the passage that applies and repost for a truthfulor knowledgeable answer. I do not trust it to remember. One way might be to have a running outlne for chapters so at least when there is a question, you can more quickly find the section? Just a thought.

Bill Cusano's avatar

Colleen, Claude lets you set up project folders where you can upload your files so it has them for reference and still, it fails to read them unless you tell it to read before replying. I keep learning the limitations. One thing it has remembered is its apology. It now says it cannot answer because it doesn’t know the history. That is better than speculation.

Colleen Bent's avatar

That sounds good... I would caution you to not to be so optimistic about Claude remembering its apology. I only say that because I gave Chat a bad time about just giving me praise instead of pointing out where I might be looking at something wrong....so then with nearly every msg after that for a couple of weeks I would get "I'm going to push you now as you asked"....and then that disappeared. Sigh. Overkill. My son tells me there is a way in 'settings' that you can set Chat or any other AI to not just give praise....there might be something about actually reading prior folders too in a setting. I am still learning....

Bill Cusano's avatar

Funny. Claude is now over emphasizing how it is being diligent in following this rule. It is almost being annoying which seems to be an AI specially.

Colleen Bent's avatar

That’s funny. Stay tuned…

Janine Eaby's avatar

AI will hallucinate (make up answers when it doesn't know). This is a well-known phenomena caused by its training and scoring. AI found it scored better when giving any answer rather than saying it didn't know.

Bill Cusano's avatar

Tells you something about us, doesn’t it? We would rather be lied to than given uncertainty.

Janine Eaby's avatar

That sounds like a post all it's own.

Sarah Williams's avatar

Which version of Claude are you using? For me, Opus 4.7 Adaptive is the most reliable for fact-checking at the moment, and the most useful, despite the fact that it’s pretty memory-hungry. I too use Claude to ensure consistency across books in the series.

Sarah Williams's avatar

Absolutely, and being neat and tidy goes very much against the grain of my head (forgive the uncomfortably mixed metaphor) but I know from my experience as a reader that inconsistencies and factual errors can be profoundly irritating, so needs must I think. I also (I am saying this sotto voce) have a spreadsheet to remind of places, characters, chronology, etc…

Bill Cusano's avatar

I am using the same version. I have learned to challenge it. I must admit that its errors make me think about how I write the story. If it gets confused maybe my readers will too. But it likes everything neat and tidy. Mysteries are not neat and tidy.