VLM Pictionary - with human in the loop

Play pictionary with Vision-Language-Models by drawing on an ipycanvas.

Inspired by Paul Calcraft's tweet and Simon Willison's post.

Installation

VLM Pictionary comes as a Jupyter Notebook and a set of requirements you can install using pip.

git clone https://github.com/haesleinhuepf/vlm-pictionary
cd vlm-pictionary
pip install -r requirements.txt

You also need to configure API keys to access the VLM service providers:

• OPENAI_API_KEY: OpenAI (gpt-4o)
• ANTHROPIC_API_KEY Anthropic (claude)
• GOOGLE_API_KEY Google AI (gemini)

Using it with the free Github Marketplace Models Make sure to generate a developer key / personal access token on Github and set it as an environment variable. You can generate the token via the Github website under user settings and afterwards set it like this for your current session:

bash:

export GITHUB_TOKEN= "your-github-token-goes-here"

powershell:

$Env:GITHUB_TOKEN= "your-github-token-goes-here"

Windows command prompt:

set GITHUB_TOKEN= your-github-token-goes-here

When using it more often, you can also think about storing it permanently.

Starting Pictionary

You can then execute it like this:

jupyter lab pictionary.ipynb