@lumi @erisceleste

Definitely.

* These things become a very poor substitute for building intuition or learning topics. Some of the worst things happen to learners with these things.

They will treat topics and texts as a kind of set of incantations that you must follow and repeat the reasoning as if they that is "what it is".

The problem with these learners is that:
* They struggle to understand that they really do not know much about the topic area

* It has a veneer of looking competent, they can "talk-the-talk" but really struggle to perform - It causes them to be misclassified as someone who only has a shallow understanding of a topic area and they cannot diagnose themselves

* Mix this with being trained to lean into these systems, they then lose their ability to apply different learning strategies when needed and can only rely on this one. As outlined in a prior post - it is a greedy strategy and it is one that is rewarded with a chatbot that flatters you.

I should say, the greedy learning strategy messes with memory quite a bit. A comparison here would be if someone could only learn by typing in a google search and taking the first hit. I won't deny that there is some validity in doing this but it misses the breadth of the topic and focuses on the outcome only.

---

*sigh*

I really must express the following though because this is unfortunately going to be much more common.

Do you know how hard it is to help someone who has such a broken model of a topic area?

* It is hard to identify
* When identified, it is difficult to understand the severity, you have to ask a lot of hard questions
* It is mentally exhausting for both people involved
* It can feel very demoralising and people can feel 'exposed'
* It can be difficult for people to comprehend the issues they have
* People can just simply withdraw completely or run away from the problem as it ends up being a source of anxiety
* As the educator, you have to come up with some bespoke set of resources to assist them
* People can just be confused or in a state of confusion, have difficulty comprehending the world as well.

When you do get an opportunity to help/fix the issues - it takes so long and it is always a rough journey where you always feel like you aren't sure if they haven't started relying on them again.

Then the inevitable disappointment of them using these tools after telling you they wouldn't because they understand that it was damaging their learning.

It is a glimpse of what a future of relying on LLMs will be.