MIDS Principles of Effective LLM Use

MIDS Principles of Effective LLM Use#

In an effort to guide students in the effective use of LLMs/AI, the following is a set of principles that the MIDS faculty think are good principles to bear in mind during use of these new tools.

Use in Education#

It bears being explicit that the use of AI in an educational environment entails different considerations than the use of AI in a production or business environment. As discussed with MIDS students during their first week at Duke, the path to learning to be able to do things that AI cannot do involves learning to do many of the things that AI can do on your own. Overuse of AI risks subverting this learning process.

With that in mind, it’s important to be mindful of what you are seeking to learn in a given activity, and to differentiate between elements of a task that are incidental to your learning goals (like front-end web design for a project website) and what’s intrinsic to your learning goals (like thinking about how you’re designing the model you plan to fit).

Use in General#

  1. Always validate.

  2. Remain skeptical.

  3. Practice reflective experimentation. These tools are constantly changing, and our understanding of “best practices” is evolving. So experiment! But do so thoughtfully.

  4. The LLM is not your friend; the LLM is a hostage that will tell you anything to make you happy. Beware AI sycophancy.

  5. Context is king. Managing an LLM’s context window — what it is aware of when answering questions — is paramount. For example, using an LLM to help write code without letting it see your full project is like asking it to work with one hand tied behind its back.

    1. But more is not necessarily better — bad context (like when your conversation has gone off the rails, or there’s so much in the context window that what matters may be lost in the noise) is often worse than no context at all.

  6. Don’t get attached to a specific tool. The LLM landscape is changing so fast, it’s important not to build around a tool in a way that makes it hard to swap out the model you’re using for a different model if pricing or performance change (because they will!).

  7. LLMs are at their best when they do things you can do yourself (or are a small enough extrapolation beyond what you could do that you can really follow) but which would take a lot of time. If you can’t follow what the AI is doing, you can’t supervise, verify, or redirect what it’s doing. Moreover, if you can’t follow what the AI is doing, you can’t answer questions or think critically about the strengths and limitations of its output.

  8. LLMS are at their most dangerous when you ask them to do something you can’t do (or follow easily) yourself.

  9. LLMs are at their very best at tasks that are hard to do but easy to verify.

Contents