Move magics to jupyter-ai-magics package

[dlqqq]

Description

Fixes #47

Other improvements:

• No more need to call jlpm setup:dev after creating a new Hatch environment
• Adds two new developer scripts:
  • jlpm dev-install: install all Jupyter AI packages, replaces scripts/install.sh
  • jlpm dev-uninstall: uninstall all Jupyter AI packages
  • both forward --scope argument to Lerna, so this can be used to selectively test packages in your Hatch environment

Verification

This process uses Jupyter Releaser to build wheels and install them locally. This offers the highest confidence that the packages will behave as expected when released onto PyPi.

Build the wheels (uses same process that Jupyter Releaser uses):

pip install jupyter_releaser
jupyter releaser prep-git --branch move-magics
jupyter releaser bump-version --version-spec 0.5.0
jupyter releaser build-npm
jupyter releaser build-python

Verify that the wheels are built in .jupyter_releaser_checkout/dist:

 % ls -l .jupyter_releaser_checkout/dist
total 3600
-rw------- 1 dlq amazon 1781132 Apr 10 21:01 jupyter_ai-0.5.0-py3-none-any.whl
-rw------- 1 dlq amazon 1775061 Apr 10 21:01 jupyter_ai-0.5.0.tar.gz
-rw------- 1 dlq amazon   14801 Apr 10 21:03 jupyter_ai_chatgpt-0.5.0-py3-none-any.whl
-rw------- 1 dlq amazon   14176 Apr 10 21:03 jupyter_ai_chatgpt-0.5.0.tar.gz
-rw-r--r-- 1 dlq amazon    3843 Apr 10 21:00 jupyter-ai-chatgpt-0.5.0.tgz
-rw-r--r-- 1 dlq amazon   21895 Apr 10 21:00 jupyter-ai-core-0.5.0.tgz
-rw------- 1 dlq amazon   16496 Apr 10 21:02 jupyter_ai_dalle-0.5.0-py3-none-any.whl
-rw------- 1 dlq amazon   15796 Apr 10 21:02 jupyter_ai_dalle-0.5.0.tar.gz
-rw-r--r-- 1 dlq amazon    4144 Apr 10 21:00 jupyter-ai-dalle-0.5.0.tgz
-rw------- 1 dlq amazon    9031 Apr 10 21:00 jupyter_ai_magics-0.5.0-py3-none-any.whl
-rw------- 1 dlq amazon    7340 Apr 10 21:00 jupyter_ai_magics-0.5.0.tar.gz

Then, install the wheels locally and perform the same tests outlined above:

# always create a new conda environment
conda create -n <env-name>
conda activate <env-name>
conda install pip
pip install jupyterlab

# install jupyter_ai_magics
pip install ".jupyter_releaser_checkout/dist/jupyter_ai_magics-0.5.0-py3-none-any.whl[all]"

# verify that extension can be loaded from `%load_ext jupyter_ai_magics`

# install jupyter_ai
pip install .jupyter_releaser_checkout/dist/jupyter_ai-0.5.0-py3-none-any.whl

# verify that extension can be loaded from both `%load_ext jupyter_ai_magics` or `%load_ext jupyter_ai`

Notes

The CI check-release step will fail because jupyter_ai_magics isn't published to PyPi yet. We need to verify it fails for the same reason:

ERROR: Could not find a version that satisfies the requirement jupyter-ai-magics (from jupyter-ai) (from versions: none)
ERROR: No matching distribution found for jupyter-ai-magics
Traceback (most recent call last):
  File "/home/dlq/anaconda3/envs/jai/bin/jupyter-releaser", line 8, in <module>
    sys.exit(main())
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/cli.py", line 122, in invoke
    super().invoke(ctx)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/cli.py", line 470, in check_python
    python.check_dist(
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/python.py", line 75, in check_dist
    util.run(f"{bin_path}/pip install -q {dist_file}")
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/util.py", line 91, in run
    raise e
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/util.py", line 83, in run
    process = tee(cmd, **kwargs)
  File "/home/dlq/anaconda3/envs/jai/lib/python3.10/site-packages/jupyter_releaser/tee.py", line 159, in run
    raise subprocess.CalledProcessError(
subprocess.CalledProcessError: Command '/tmp/tmp_o8m8bh_/bin/pip install -q dist/jupyter_ai-0.5.0-py3-none-any.whl' returned non-zero exit status 1.

[JasonWeill]

Verified locally using Node 18.13.0 — thanks @dlqqq for your help!

[dlqqq]

Yes, for some reason the labextension build fails for the latest Node 18.15.0. Full log:

> @jupyter-ai/core:"build:prod"

@jupyter-ai/core: $ jlpm clean && jlpm build:lib && jlpm build:labextension
@jupyter-ai/core: $ jlpm clean:lib
@jupyter-ai/core: $ rimraf lib tsconfig.tsbuildinfo
@jupyter-ai/core: $ tsc
@jupyter-ai/core: $ jupyter labextension build .
@jupyter-ai/core: Building extension in .
@jupyter-ai/core: /Users/jweill/git/jupyter-ai/.jupyter_releaser_checkout/node_modules/webpack/lib/RuntimeTemplate.js:64
@jupyter-ai/core: 		trimmed.match(/^[_\p{L}][_0-9\p{L}]*$/iu) ||
@jupyter-ai/core: 		              ^^^^^^^^^^^^^^^^^^^^^^^^^^
@jupyter-ai/core: SyntaxError: Invalid regular expression: /^[_\p{L}][_0-9\p{L}]*$/: Invalid property name in character class
@jupyter-ai/core:     at internalCompileFunction (node:internal/vm:73:18)
@jupyter-ai/core:     at wrapSafe (node:internal/modules/cjs/loader:1176:20)
@jupyter-ai/core:     at Module._compile (node:internal/modules/cjs/loader:1218:27)
@jupyter-ai/core:     at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
@jupyter-ai/core:     at Module.load (node:internal/modules/cjs/loader:1117:32)
@jupyter-ai/core:     at Module._load (node:internal/modules/cjs/loader:958:12)
@jupyter-ai/core:     at Module.require (node:internal/modules/cjs/loader:1141:19)
@jupyter-ai/core:     at require (node:internal/modules/cjs/helpers:110:18)
@jupyter-ai/core:     at Object.<anonymous> (/Users/jweill/git/jupyter-ai/.jupyter_releaser_checkout/node_modules/webpack/lib/Compilation.js:53:25)
@jupyter-ai/core:     at Module._compile (node:internal/modules/cjs/loader:1254:14)
@jupyter-ai/core: Node.js v18.15.0
@jupyter-ai/core: An error occurred.
@jupyter-ai/core: subprocess.CalledProcessError: Command '['node', '/Users/jweill/git/jupyter-ai/.jupyter_releaser_checkout/node_modules/@jupyterlab/builder/lib/build-labextension.js', '--core-path', '/opt/miniconda3/envs/jai-build/lib/python3.11/site-packages/jupyterlab/staging', '/Users/jweill/git/jupyter-ai/.jupyter_releaser_checkout/packages/jupyter-ai']' returned non-zero exit status 1.
@jupyter-ai/core: See the log file for details:  /var/folders/md/x69whq_524q4kb6r0gnq3df40000gr/T/jupyterlab-debug-0m5n2kun.log
@jupyter-ai/core: error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
@jupyter-ai/core: error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

[dlqqq]

Uploaded [email protected] to PyPI. Kicking CI.

[dlqqq]

Check release workflow is passing, CI is green. Failing workflow is identical to the passing one, except that it wasn't re-run by closing & re-opening the PR. Merging. 🎉