Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(kore): ts step interpreter #195

Merged
merged 2 commits into from
Apr 23, 2022

Conversation

paradite
Copy link
Contributor

@paradite paradite commented Apr 21, 2022

This PR implements the new step interpreter for TS, which allows full interactivity during the game at each step.

This should allow for easier training in TS.

Major changes:

  • added miner bot (MinerBot.ts, miner.py) and do nothing bot (DoNothingBot.ts) in TS.
  • added a new mode step in the interpreter, alongside the old run mode. The way to initialize the board is a hack, but it works for now.
  • change KoreIO to async as training in tf.js involves async function calls.
  • fixed a few bugs in the kore/Board.ts, kore/Direction.ts and kore/Point.ts which is throwing errors when I tried to use board.next() to simulate full games. Might need to add some tests for these fixes?
  • export tick function in TS bots to allow them to be imported inside the interpreter directly, so the training code and the agent code can run within the same process.
  • added episodeSteps in kore/Configuration.ts.
  • made currentPlayerId mutable so that we can rotate currentPlayerId in interpreter to allow agents to tick interactively on the same board.

Out-standing issues:

  • The agent behaviour is not exactly the same in TS step interpreter compared to the python one. I suspect there are a few more differences between how board.next() is implemented in python vs TS. Need more time to investigate this.

Sample runs:

miner vs do_nothing

node --require ts-node/register interpreter.ts run 20 miner do_nothing
> agent miner wins: 20/20

node --require ts-node/register interpreter.ts step 20 miner do_nothing
> agent miner wins: 19/20

I have observed that miner performs consistently slightly worse in step interpreter compared to python interpreter. I will investigate this further.

./main.py (bot) vs miner

node --require ts-node/register interpreter.ts run 20 ./main.py miner
> agent main wins: 00/20

node --require ts-node/register interpreter.ts step 20 bot miner
> agent bot wins: 00/20

I have not testing training with this new interpreter but this change is already very big and it is working fine on its own, so I think we can merge this first.

@bovard please kindly help review.

@paradite paradite changed the title Feat/ts step interpreter feat(kore): ts step interpreter Apr 21, 2022
Copy link
Member

@bovard bovard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bovard bovard merged commit c4ddc3e into Kaggle:master Apr 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants