Skip to content

Sync Words Challenge, a fun benchmark for LLMs!

License

Notifications You must be signed in to change notification settings

Finebouche/words_synch_challenge

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

58 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WORDS SYNCH CHALLENGE

Sync Words, a fun benchmark for LLMs!

license last-commit repo-top-language repo-language-count


🔗 Table of Contents


📍 Overview

The wordssynchchallenge project addresses the challenge of synchronizing word choices between AI models and users in a game-like setting. It offers seamless AI integration, multilingual support, and engaging user interactions. Ideal for AI enthusiasts and developers, it enhances understanding of AI model interactions and natural language processing through interactive gameplay.


Subproject Summary
⚙️ Website
  • Utilizes a client-server model with a web application front-end and a backend for game logic and data management.
  • Integrates OpenAI and Hugging Face APIs for model interactions, enhancing game interactions
  • Database integration via SQLite managed by Sequelize ORM.
🔌 Data Analysis
  • Dataset creation with OpenAI API.
  • Visualisation through OpenAI embeding and pyplot

📚 Publications

2025 (Upcoming)

  • Word Synchronization Challenge: A Benchmark for Word Association Responses for Large Language Models
    Tanguy Cazalets, Joni Dambre
    Accepted for presentation at HCII 2025 Conference
    To be published in Springer’s HCII 2025 Proceedings.
    This paper introduces the Word Synchronization Challenge, a novel benchmark to evaluate word association responses in Large Language Models (LLMs). It discusses implementation insights, highlights human-LLM interaction mechanisms, and addresses ethical considerations, such as participant consent and data privacy, in studies involving human subjects.

🎗 License

This project is protected under the MIT License License.


🙌 Acknowledgments

  • Thanks to Qiao Qiao Ren and Tony Belpaeme for their help on the project.
  • This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860949

About

Sync Words Challenge, a fun benchmark for LLMs!

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published