Skip to content

Commit

Permalink
Merge pull request #53 from shinybrar/main
Browse files Browse the repository at this point in the history
Support for Private Container Registries
  • Loading branch information
tjzegmott authored Oct 21, 2024
2 parents 175ffce + d2ff5ac commit a8fc5b1
Show file tree
Hide file tree
Showing 27 changed files with 2,600 additions and 2,054 deletions.
69 changes: 33 additions & 36 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,61 +12,58 @@ jobs:
steps:
-
name: Setup code repository
uses: actions/checkout@v2
with:
fetch-depth: 1
-
name: Setup Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
uses: actions/[email protected]
-
name: Load cached pre-commit repos
id: cached-pre-commit-repos
uses: actions/cache@v2
name: Setup Python
uses: actions/[email protected]
with:
path: |
~/.cache/pre-commit
key: poetry-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
python-version: '3.9'
-
name: Perform pre-commit checks
run: |
pip install pre-commit
pre-commit run -a
name: Run pre-commit
uses: pre-commit/[email protected]

tests:
needs: pre-commit-checks
strategy:
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']
os: [ubuntu-latest, windows-latest, macos-latest]
os: [ubuntu-latest]
runs-on: ${{ matrix.os }}
steps:
-
name: Setup code repository
uses: actions/checkout@v2
uses: checkout@v4.2.1
with:
fetch-depth: 1
-
name: Setup uv
uses: astral-sh/setup-uv@v3
-
name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
run: |
uv python install ${{ matrix.python-version }}
uv sync --all-extras --dev
-
name: Install Poetry
name: Run tests
env:
CADC_USERNAME: ${{ secrets.CADC_USERNAME }}
CADC_PASSWORD: ${{ secrets.CADC_PASSWORD }}
run: |
pip install poetry
uv run cadc-get-cert --user ${{ env.CADC_USERNAME }} --password ${{ env.CADC_PASSWORD }} --days-valid 1
uv run pytest tests --cov --cov-report=xml --junitxml=junit.xml -o junit_family=legacy
-
name: Load cached Poetry virtual environment
uses: actions/cache@v2
name: Upload coverage to Codecov
env:
CODECOV_TOKEN: ${{ env.CODECOV_TOKEN }}
uses: codecov/codecov-action@v4
with:
path: ~/.cache/pypoetry/virtualenvs
key: poetry-${{ runner.os }}-${{ matrix.python-version }}-${{ hashFiles('**/poetry.lock') }}
-
name: Install dependencies
run: |
poetry install --with dev
fail_ci_if_error: true # Fail the CI if an error occurs during the upload
token: ${{ env.CODECOV_TOKEN }}
flags: ${{ matrix.python-version }}
verbose: true # optional (default = false)
-
name: Run tests
run: |
poetry run pytest tests/test_threading.py
name: Upload test results to Codecov
if: ${{ !cancelled() }}
uses: codecov/test-results-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,11 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v4.2.1

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand All @@ -51,7 +51,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
Expand All @@ -65,4 +65,4 @@ jobs:
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
uses: github/codeql-action/analyze@v2
13 changes: 6 additions & 7 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
name: docs
name: Deploy Docs

on:
push:
branches:
Expand All @@ -10,9 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.8
- run: pip install --upgrade poetry
- run: poetry install --with=dev --sync
- run: poetry run mkdocs gh-deploy --force
- uses: astral-sh/setup-uv@v3
- run: |
uv python install
uv run mkdocs gh-deploy --force
73 changes: 73 additions & 0 deletions .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# This workflow uses actions that are not certified by GitHub. They are provided
# by a third-party and are governed by separate terms of service, privacy
# policy, and support documentation.

name: Scorecard supply-chain security
on:
# For Branch-Protection check. Only the default branch is supported. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
branch_protection_rule:
# To guarantee Maintained check is occasionally updated. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
schedule:
- cron: '34 6 * * 3'
push:
branches: [ "main" ]

# Declare default permissions as read only.
permissions: read-all

jobs:
analysis:
name: Scorecard analysis
runs-on: ubuntu-latest
permissions:
# Needed to upload the results to code-scanning dashboard.
security-events: write
# Needed to publish results and get a badge (see publish_results below).
id-token: write
# Uncomment the permissions below if installing in a private repository.
# contents: read
# actions: read

steps:
- name: "Checkout code"
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
with:
persist-credentials: false

- name: "Run analysis"
uses: ossf/scorecard-action@0864cf19026789058feabb7e87baa5f140aac736 # v2.3.1
with:
results_file: results.sarif
results_format: sarif
# (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
# - you want to enable the Branch-Protection check on a *public* repository, or
# - you are installing Scorecard on a *private* repository
# To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.
# repo_token: ${{ secrets.SCORECARD_TOKEN }}

# Public repositories:
# - Publish results to OpenSSF REST API for easy access by consumers
# - Allows the repository to include the Scorecard badge.
# - See https://github.com/ossf/scorecard-action#publishing-results.
# For private repositories:
# - `publish_results` will always be set to `false`, regardless
# of the value entered here.
publish_results: true

# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@97a0fba1372883ab732affbe8f94b823f91727db # v3.pre.node20
with:
name: SARIF file
path: results.sarif
retention-days: 5

# Upload the results to GitHub's code scanning dashboard (optional).
# Commenting out will disable upload of results to your repo's Code Scanning dashboard
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: results.sarif
16 changes: 8 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
repos:

- repo: https://github.com/pycqa/isort
rev: 5.12.0
rev: 5.13.2
hooks:
- id: isort
args: [ --multi-line=3, --trailing-comma, --force-grid-wrap=0, --use-parentheses, --line-width=88]

- repo: https://github.com/pycqa/flake8
rev: 6.1.0
rev: 7.1.1
hooks:
- id: flake8
args: [--max-line-length=89, --max-complexity=15]
Expand All @@ -22,18 +22,18 @@ repos:


- repo: https://github.com/asottile/pyupgrade
rev: v3.15.0
rev: v3.18.0
hooks:
- id: pyupgrade
args: [--py36-plus]

- repo: https://github.com/psf/black
rev: 23.11.0
rev: 24.10.0
hooks:
- id: black

- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.7.0
rev: v1.12.0
hooks:
- id: mypy
additional_dependencies: ["types-toml", "types-setuptools", "types-requests", "types-attrs"]
Expand All @@ -46,12 +46,12 @@ repos:
args: [--convention=google, --add-ignore=D104]

- repo: https://github.com/asottile/blacken-docs
rev: 1.16.0
rev: 1.19.0
hooks:
- id: blacken-docs

- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.5.0
rev: v5.0.0
hooks:
- id: trailing-whitespace
exclude: ^docs/*
Expand All @@ -70,7 +70,7 @@ repos:
- id: detect-private-key

- repo: https://github.com/PyCQA/bandit
rev: '1.7.5'
rev: '1.7.10'
hooks:
- id: bandit
args: ["--skip=B101"]
Expand Down
7 changes: 6 additions & 1 deletion docs/client.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# Skaha Client

The `skaha.client` module provides a client for the Skaha server. The client is based of the
`Requests` library and provides a simple interface to the Skaha server. The client configures the
authorization headers for user authentication and container registry access.

::: skaha.client.SkahaClient
handler: python
Expand All @@ -10,4 +15,4 @@
rendering:
show_root_heading: true
show_source: true
heading_level: 2
heading_level: 2
23 changes: 21 additions & 2 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,17 @@

## Session API

The bread and butter of Skaha is the Session API. This API allows you to create, destroy, and get information about your sessions on the Skaha platform.
The bread and butter of Skaha is the Session API. This API allows you to create, destroy, and get information about your sessions on the CANFAR Science Platform.

### Creating a Session

```python title="Create a session"
from skaha.session import Session

session = Session()
```

```python title="Spawn a session"
session_id = session.create(
name="test",
image="images.canfar.net/chimefrb/testing:keep",
Expand All @@ -30,6 +33,21 @@ print(session_id)
["mrjdtbn9", "ov6doae7", "ayv4553m"]
```

!!! note "Container Replicas"
When spawning sessions with the Skaha API, it adds two additional environment variables to each container:
- `REPLICA_COUNT`: An integer representing the total number of replicas spawned, e.g. 2.
- `REPLICA_ID`: An integer representing the ID of the replica, e.g. 0, 1, 2.

!!! note "Container Registry Access"
If you are using a private container image from the CANFAR Harbor Registry, you need to provide your harbor `username` and the `CLI Secret` through a `ContainerRegistry` object.
```python
from skaha.models import ContainerRegistry
from skaha.session import Session

registry = ContainerRegistry(username="username", password="sUp3rS3cr3t")
session = Session(registry=registry)
```

### Getting Session Information

```python title="Get session information"
Expand Down Expand Up @@ -90,7 +108,8 @@ session.destroy(session_id)

## Image API

The Image API allows you to get information about the images available on the Skaha platform. Nominally,
The Image API allows you to get information about the **publicly available** images on the CANFAR Science Platform through
the CANFAR Harbor Registry.

### Getting Image Information

Expand Down
Loading

0 comments on commit a8fc5b1

Please sign in to comment.