

I’m now curious what you mean by self updating model? For a model to be made, it needs billions of data points for it to make the first. A second can be made with the first judging the quality of the input into the 2nd. Some models already do this sifting in prep for the next model creation.
I think of it like humans: we have billions of sensory signals each day, and we judge what is important based on genetics, culture, and our chosen interpretation of morality e.g. hedonism considers effort/discomfort. If a llm has a billion sensory signals each day and has application specific hardware like our genetics then would the hardware finally allow you to call it intelligent?
I am turning into a philosopher in this comment thread! Soo… when is a chair a chair and not a stool?







You are right and I have seen some people try some clumsy solutions:
Have the llm summarize the chat context ( this loses information, but can make the llm appear to have a longer memory)
Have the llm repeat and update a todo list at the end of every prompt (this keeps it on task as it always has the last response in memory, BUT it can try to do 10 things and fails on step 1 but doesn’t realize jt)
Have a llm trained with really high quality data then have it judge the randomness of the internet. This is meta cognition by humans using the llm as a tool for itself. It definitely can’t do it by itself without becoming schizophrenic, but it can make some smart models from inconsistent and crappy/dirty datasets.
Again, you are right and I hate using the syncophantic clockwork-orange llms with no self awareness. I have some hope that they will get better.