Link, Description, & Synopsis
Link: How I think about LLM prompt engineering
Description:
François Chollet presents a novel way to understand LLM prompting by drawing parallels with Word2Vec and vector programs.
Synopsis:
The article explores how to:
- Understand LLMs through the lens of vector spaces
- View prompts as program queries in a continuous space
- Compare modern LLMs with Word2Vec principles
- Approach prompt engineering as a program search
Context
The article parallels modern LLMs and Word2Vec’s 2013 discovery of emergent vector arithmetic.
Despite their apparent differences, you can think of both systems organizing tokens in vector spaces where correlation becomes proximity.
This perspective suggests prompting is more about searching a space of vector programs than communicating with an intelligence.
Personal Notes
Viewing LLMs as vector program databases rather than intelligent entities provides a powerful mental model for systematic improvement.
Just as Word2Vec revealed emergent properties in simple vector spaces, understanding LLMs as continuous program spaces could lead to more principled approaches to prompt engineering.
Looking Forward: LLM Development
The field will likely evolve to include:
- More systematic prompt engineering approaches
- Better vector space visualization tools
- Improved program space navigation
- Enhanced optimization techniques
- More principled testing methods
Conclusion
This perspective on LLMs as vector program databases could fundamentally change how we approach prompt engineering, moving from intuitive experimentation to systematic exploration of program space.