A work in progress to build out solutions in Rust for MLOPs. This repo is more of a cookbook style. For a more gentle step by step guide to MLOps with Rust, please see my lecture notes as a Rust MDBook here.
- Do an inline python example
- Train a model in PyTorch with CPU: https://github.com/LaurentMazare/tch-rs
- Train a model in PyTorch with GPU: https://github.com/LaurentMazare/tch-rs
- Serve out ONNX with a Rust web framework like Actix
- ONNX Command-Line Tool
- Simple async network example: (network discovery or chat system)
- Rust SQLite Example
- Rust AWS Lambda
- Simple Rust GUI
- Rust Whisper Tool with C++ Bindings
- Fast Keyword Extraction (NLP)
- Emit Random Mediterranean Meals via CLI
- Web Assembly Rust
- Building a database in Rust
- Building a search engine in Rust
- Building a web server in Rust
- Building a batch processing systems in Rust
- Build a command-line chat system
- Build a locate clone
- Build a load-testing tool
One of the key goals of this project is to determine workflows that do not involve the #jcpennys (Jupyter, Conda, Pandas, Numpy, Sklearn) stack for #mlops. In particular I am not a fan of the conda installation tool (it is superfluous as I demonstrate in the Python MLOps Template) vs containerized workflows that use the Python Standard Library (Docker + pip + virtualenv) and this is a good excuse to find other solutions outside of that stack. For example:
- Why not also find a more performant Data Frame library, faster speed, etc.
- Why not have a compiler?
- Why not have a simple packaging solution?
- Why not have a very fast computational speed?
- Why not be able to write both for the Linux Kernel and general purpose scripting?
- Why not see if there is a better solution than Python (which is essentially two languages scientific python and regular Python)?
- Python is one of the least green languages in terms of energy efficiency, but Rust is one of the best.
What could #mlops and #datascience look like in 2023 without #jupyternotebook and "God Tools" as the center of the universe? It could be the command line. In the beginning, it was the command line, and it may be the best solution for this domain.
"What would the engineer say after you had explained your problem and enumerated all the dissatisfactions in your life? He would probably tell you that life is a very hard and complicated thing; that no interface can change that; that anyone who believes otherwise is a sucker; and that if you don't like having choices made for you, you should start making your own." -Neal Stephensen
- StackOverflow https://survey.stackoverflow.co/2022/#section-most-loved-dreaded-and-wanted-programming-scripting-and-markup-languages[states that #rust is on 7th year as the most loved language 87% of developers want to continue developing](https://survey.stackoverflow.co/2022/#section-most-loved-dreaded-and-wanted-programming-scripting-and-markup-languages) in and ties with Python as the most wanted technology. Clearly there is traction.
- According to http://www.modulecounts.com/[Modulecounts] it looks like an exponential growth curve to Rust.
This repository is a GitHub Template and you can use it to create a new repository that uses GitHub Codespaces. It is pre-configured with Rust, Cargo and other useful extensions like GitHub Copilot.
There are a few options:
- You can follow the Official Install Guide for Rust
- Create a repo with this template
Once you install you should check to see things work:
rustc --version
Other option is to run make rust-version
which checks both the cargo and rust version.
To run everything locally do: make all
and this will format/lint/test all projects in this repository.
You can see there several tools which help you get things done in Rust:
rust-version:
@echo "Rust command-line utility versions:"
rustc --version #rust compiler
cargo --version #rust package manager
rustfmt --version #rust code formatter
rustup --version #rust toolchain manager
clippy-driver --version #rust linter
This is an intentionally simple full end-to-end hello world example. I used some excellent ideas from @kyclark, author of the command-line-rust book from O'Reilly here. You can recreate on your own following these steps
Create a project directory
cargo new hello
This creates a structure you can see with tree hello
hello/
├── Cargo.toml
└── src
└── main.rs
1 directory, 2 files
The Cargo.toml
file is where the project is configured, i.e. if you needed to add a dependency.
The source code file has the following content in main.rs
. It looks a lot like Python or any other modern language and this function prints a message.
fn main() {
println!("Hello, world MLOPs!");
}
To run the project you cd into hello and run cargo run
i.e. cd hello && cargo run
. The output looks like the following:
@noahgift âžś /workspaces/rust-mlops-template/hello (main âś—) $ cargo run
Compiling hello v0.1.0 (/workspaces/rust-mlops-template/hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.36s
Running `target/debug/hello`
Hello, world MLOPs!
To run without all of the noise: cargo run --quiet
.
To run the binary created ./target/debug/hello
GitHub Actions uses a Makefile
to simplify automation
name: Rust CI/CD Pipeline
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
env:
CARGO_TERM_COLOR: always
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: actions-rs/toolchain@v1
with:
toolchain: stable
profile: minimal
components: clippy, rustfmt
override: true
- name: update linux
run: sudo apt update
- name: update Rust
run: make install
- name: Check Rust versions
run: make rust-version
- name: Format
run: make format
- name: Lint
run: make lint
- name: Test
run: make test
To run everything locally do: make all
.
Change into MarcoPolo
directory and run cargo run -- play --name Marco
and you should see the following output:
Polo
I have written command-line deduplication tools in many languages so this is what I choose to build a substantial example. The general approach I use is as follows:
- Walk the filesystem and create a checksum for each file
- If the checksum matches an existing checksum, then mark it as a duplicate file
Getting Started
- Create new project:
crate new dedupe
- Check latest clap version: https://crates.io/crates/clap and put this version in the
Cargo.toml
The file should look similar to this.
[package]
name = "dedupe"
version = "0.1.0"
edition = "2021"
[dependencies]
clap = "4.0.32"
[dev-dependencies]
assert_cmd = "2"
- Next up make a test directory:
mkdir tests
that is parallel tosrc
and put acli.rs
inside - touch a
lib.rs
file and use this for the logic then runcargo run
- Inside this project I also created a
Makefile
to easily do everything at once:
format:
cargo fmt --quiet
lint:
cargo clippy --quiet
test:
cargo test --quiet
run:
cargo run --quiet
all: format lint test run
Now as I build code, I can simply do: make all
and get a high quality build.
Next, let's create some test files:
echo "foo" > /tmp/one.txt
echo "foo" > /tmp/two.txt
echo "bar" > /tmp/three.txt
The final version works: cargo run -- --path /tmp
@noahgift âžś /workspaces/rust-mlops-template/dedupe (main âś—) $ cargo run -- --path /tmp
Finished dev [unoptimized + debuginfo] target(s) in 0.03s
Running `target/debug/dedupe --path /tmp`
Searching path: "/tmp"
Found 5 files
Found 1 duplicates
Duplicate files: ["/tmp/two.txt", "/tmp/one.txt"]
Next things to complete for dedupe (in another repo):
- Switch to subcommands and create a
search
anddedupe
subcommand - Add better testing with sample test files
- Figure out how to release packages for multiple OS versions in GitHub
- Query Hugging Face dataset cli
- Summarize News CLI
- Microservice Web Framework, trying actix to start, that has a calculator API
- Microservice Web Framework deploys pre-trained model
- Descriptive Statistics on a well known dataset using https://www.pola.rs/[Polars] inside a CLI
- Train a model with PyTorch (probably via bindings to Rust)
- Refer to Actix getting started guide
cargo new calc && cd calc
- add dependency to
Cargo.toml
[package]
name = "calc"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
actix-web = "4"
- create a
src/lib.rs
and place inside
//calculator functions
//Add two numbers
pub fn add(a: i32, b: i32) -> i32 {
a + b
}
//Subtract two numbers
pub fn subtract(a: i32, b: i32) -> i32 {
a - b
}
//Multiply two numbers
pub fn multiply(a: i32, b: i32) -> i32 {
a * b
}
//Divide two numbers
pub fn divide(a: i32, b: i32) -> i32 {
a / b
}
In the main.rs
put the following:
//Calculator Microservice
use actix_web::{get, web, App, HttpResponse, HttpServer, Responder};
#[get("/")]
async fn index() -> impl Responder {
HttpResponse::Ok().body("This is a calculator microservice")
}
//library add route using lib.rs
#[get("/add/{a}/{b}")]
async fn add(info: web::Path<(i32, i32)>) -> impl Responder {
let result = calc::add(info.0, info.1);
HttpResponse::Ok().body(result.to_string())
}
//library subtract route using lib.rs
#[get("/subtract/{a}/{b}")]
async fn subtract(info: web::Path<(i32, i32)>) -> impl Responder {
let result = calc::subtract(info.0, info.1);
HttpResponse::Ok().body(result.to_string())
}
//library multiply route using lib.rs
#[get("/multiply/{a}/{b}")]
async fn multiply(info: web::Path<(i32, i32)>) -> impl Responder {
let result = calc::multiply(info.0, info.1);
HttpResponse::Ok().body(result.to_string())
}
//library divide route using lib.rs
#[get("/divide/{a}/{b}")]
async fn divide(info: web::Path<(i32, i32)>) -> impl Responder {
let result = calc::divide(info.0, info.1);
HttpResponse::Ok().body(result.to_string())
}
//run it
#[actix_web::main]
async fn main() -> std::io::Result<()> {
HttpServer::new(|| {
App::new()
.service(index)
.service(add)
.service(subtract)
.service(multiply)
.service(divide)
})
.bind(("127.0.0.1", 8080))?
.run()
.await
}
Next, use a Makefile
to ensure a simple workflow
format:
cargo fmt --quiet
lint:
cargo clippy --quiet
test:
cargo test --quiet
run:
cargo run
all: format lint test run
Run make all
then test out the route by adding two numbers at /add/2/2
- Uses rust-bert crate
- Create new project
cargo new hfdemo
and cd into it:cd hfdemo
- Create a new library file:
touch src/lib.rs
- Add packages to
Cargo.toml
[package]
name = "hfdemo"
version = "0.1.0"
edition = "2021"
[dependencies]
rust-bert = "0.19.0"
clap = {version="4.0.32", features=["derive"]}
wikipedia = "0.3.4"
The library code is in lib.rs
and the subcommands
from clap
live in main.rs
. Here is the tool in action:
@noahgift âžś /workspaces/rust-mlops-template/hfdemo (main âś—) $ cargo run sumwiki --page argentina
Finished dev [unoptimized + debuginfo] target(s) in 4.59s
Running `target/debug/hfdemo sumwiki --page argentina`
Argentina is a country in the southern half of South America. It covers an area of 2,780,400 km2 (1,073,500 sq mi), making it the second-largest country in South America after Brazil. It is also the fourth-largest nation in the Americas and the eighth-largest in the world.
cd into hfqa
and run cargo run
```bash
cargo run --quiet -- answer --question "What is the best book from 1880 to read?" --context "The Adventures of Huckleberry Finn was released in 1880"
Answer: The Adventures of Huckleberry Finn
@noahgift âžś /workspaces/rust-mlops-template/sqlite-hf (main âś—) $ cargo run --quiet -- classify
Classify lyrics.txt
rock: 0.06948944181203842
pop: 0.27735018730163574
hip hop: 0.034089818596839905
country: 0.7835917472839355
latin: 0.6906086802482605
Print the lyrics:
cargo run --quiet -- lyrics | less | head
Lyrics lyrics.txt
Uh-uh-uh-uh, uh-uh
Ella despidiĂł a su amor
El partiĂł en un barco en el muelle de San Blas
El jurĂł que volverĂa
Y empapada en llanto, ella jurĂł que esperarĂa
Miles de lunas pasaron
Y siempre ella estaba en el muelle, esperando
Muchas tardes se anidaron
Se anidaron en su pelo y en sus labios
Full working example here: https://github.com/nogibjj/rust-pytorch-gpu-template/tree/main/translate
Goal: Translate a spanish song to english
cargo new translate
and cd into it fully working GPU Hugging Face Translation CLI in Rust
run it: time cargo run -- translate --path lyrics.txt
/*A library that uses Hugging Face to Translate Text
*/
use rust_bert::pipelines::translation::{Language, TranslationModelBuilder};
use std::fs::File;
use std::io::Read;
//build a function that reads a file and returns a string
pub fn read_file(path: String) -> anyhow::Result<String> {
let mut file = File::open(path)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
Ok(contents)
}
//build a function that reads a file and returns an array of the lines of the file
pub fn read_file_array(path: String) -> anyhow::Result<Vec<String>> {
let mut file = File::open(path)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
let array = contents.lines().map(|s| s.to_string()).collect();
Ok(array)
}
//build a function that reads a file and translates it
pub fn translate_file(path: String) -> anyhow::Result<()> {
let model = TranslationModelBuilder::new()
.with_source_languages(vec![Language::Spanish])
.with_target_languages(vec![Language::English])
.create_model()?;
let text = read_file_array(path)?;
//pass in the text to the model
let output = model.translate(&text, None, Language::English)?;
for sentence in output {
println!("{}", sentence);
}
Ok(())
}
-
cd into
polarsdf
and runcargo run
cargo run -- sort --rows 10
You can see an example of how Polars can be used to sort a dataframe in a Rust cli program.
One of the outstanding features of Rust is safe, yet easy paralielism. This project demos parallelism by benchmarking a checksum of several files.
We can see how trivial it is to speed up a program with threads:
Here is the function for the serial version:
// Create a checksum of each file and store in a HashMap if the checksum already exists, add the file to the vector of files with that checksum
pub fn checksum(files: Vec<String>) -> Result<HashMap<String, Vec<String>>, Box<dyn Error>> {
let mut checksums = HashMap::new();
for file in files {
let checksum = md5::compute(std::fs::read(&file)?);
let checksum = format!("{:x}", checksum);
checksums
.entry(checksum)
.or_insert_with(Vec::new)
.push(file);
}
Ok(checksums)
}
cargo --quiet run -- serial
âžś parallel git:(main) âś— time cargo --quiet run -- serial
Serial version of the program
d41d8cd98f00b204e9800998ecf8427e:
src/data/subdir/not_utils_four-score.m4a
src/data/not_utils_four-score.m4a
b39d1840d7beacfece35d9b45652eee1:
src/data/utils_four-score3.m4a
src/data/utils_four-score2.m4a
src/data/subdir/utils_four-score3.m4a
src/data/subdir/utils_four-score2.m4a
src/data/subdir/utils_four-score5.m4a
src/data/subdir/utils_four-score4.m4a
src/data/subdir/utils_four-score.m4a
src/data/utils_four-score5.m4a
src/data/utils_four-score4.m4a
src/data/utils_four-score.m4a
cargo --quiet run -- serial 0.57s user 0.02s system 81% cpu 0.729 total
vs threads
time cargo --quiet run -- parallel
Parallel version of the program
d41d8cd98f00b204e9800998ecf8427e:
src/data/subdir/not_utils_four-score.m4a
src/data/not_utils_four-score.m4a
b39d1840d7beacfece35d9b45652eee1:
src/data/utils_four-score5.m4a
src/data/subdir/utils_four-score3.m4a
src/data/utils_four-score3.m4a
src/data/utils_four-score.m4a
src/data/subdir/utils_four-score.m4a
src/data/subdir/utils_four-score2.m4a
src/data/utils_four-score4.m4a
src/data/utils_four-score2.m4a
src/data/subdir/utils_four-score4.m4a
src/data/subdir/utils_four-score5.m4a
cargo --quiet run -- parallel 0.65s user 0.04s system 262% cpu 0.262 total
Ok, so let's look at the code:
// Parallel version of checksum using rayon with a mutex to ensure
//that the HashMap is not accessed by multiple threads at the same time
pub fn checksum_par(files: Vec<String>) -> Result<HashMap<String, Vec<String>>, Box<dyn Error>> {
let checksums = std::sync::Mutex::new(HashMap::new());
files.par_iter().for_each(|file| {
let checksum = md5::compute(std::fs::read(file).unwrap());
let checksum = format!("{:x}", checksum);
checksums
.lock()
.unwrap()
.entry(checksum)
.or_insert_with(Vec::new)
.push(file.to_string());
});
Ok(checksums.into_inner().unwrap())
}
The main takeaway is that we use a mutex to ensure that the HashMap is not accessed by multiple threads at the same time. This is a very common pattern in Rust.
cd into clilog
and type: cargo run -- --level TRACE
//function returns a random fruit and logs it to the console
pub fn random_fruit() -> String {
//randomly select a fruit
let fruit = FRUITS[rand::thread_rng().gen_range(0..5)];
//log the fruit
log::info!("fruit-info: {}", fruit);
log::trace!("fruit-trace: {}", fruit);
log::warn!("fruit-warn: {}", fruit);
fruit.to_string()
}
Running an optimized version was able to sum all the objects in my AWS Account about 1 second: ./target/release/awsmetas3 account-size
cd into rust-aws-lambda
- Rust AWS Lambda docs
- Install AWS VS Code plugin and configure it to use your AWS account.
- See GitHub repo here: https://github.com/awslabs/aws-lambda-rust-runtime#deployment
To deploy: make deploy
which runs: cargo lambda build --release
- Test inside of AWS Lambda console
- Test locally with:
cargo lambda invoke --remote \
--data-ascii '{"command": "hi"}' \
--output-format json \
rust-aws-lambda
Result:
cargo lambda invoke --remote \
--data-ascii '{"command": "hi"}' \
--output-format json \
rust-aws-lambda
{
"msg": "Command hi executed.",
"req_id": "1f70aff9-dc65-47be-977b-4b81bf83e7a7"
}
Example lives here: https://github.com/noahgift/rust-mlops-template/tree/main/rrgame
- Client server echo working
cargo run -- client --message "hi"
cargo run -- server
A bigger example lives here: https://github.com/noahgift/rust-multiplayer-roulette-game
FROM rust:latest as builder
ENV APP containerized_marco_polo_cli
WORKDIR /usr/src/$APP
COPY . .
RUN cargo install --path .
FROM debian:buster-slim
RUN apt-get update && rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/cargo/bin/$APP /usr/local/bin/$APP
ENTRYPOINT [ "/usr/local/bin/containerized_marco_polo_cli" ]
cd into: pytorch-rust-docker
Here is the Dockerfile
FROM rust:latest as builder
ENV APP pytorch-rust-docker
WORKDIR /usr/src/$APP
COPY . .
RUN apt-get update && rm -rf /var/lib/apt/lists/*
RUN cargo install --path .
RUN cargo build -j 6
docker build -t pytorch-rust-docker .
docker run -it pytorch-rust-docker
- Next inside the container run:
cargo run -- resnet18.ot Walking_tiger_female.jpg
/*Rust Tensorflow Hello World */
extern crate tensorflow;
use tensorflow::Tensor;
fn main() {
let mut x = Tensor::new(&[1]);
x[0] = 2i32;
//print the value of x
println!("{:?}", x[0]);
//print the shape of x
println!("{:?}", x.shape());
//create a multidimensional tensor
let mut y = Tensor::new(&[2, 2]);
y[0] = 1i32;
y[1] = 2i32;
y[2] = 3i32;
y[3] = 4i32;
//print the value of y
println!("{:?}", y[0]);
//print the shape of y
println!("{:?}", y.shape());
}
Pre-trained model: cd into pytorch-rust-example
then run: cargo run -- resnet18.ot Walking_tiger_female.jpg
Using included model in binary. See this issue about including PyTorch with binary
Status: Works, but binary cannot pickup PyTorch, so still investigating solutions.
@noahgift âžś /workspaces/rust-mlops-template/pytorch-binary-cli (main âś—) $ cargo run -- predict --image Walking_tiger_female.jpg
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Running `target/debug/pytorch-binary-cli predict --image Walking_tiger_female.jpg`
Current working directory: /workspaces/rust-mlops-template/pytorch-binary-cli
Model path: ../model/resnet18.ot
Model size: 46831783
tiger, Panthera tigris 90.42%
tiger cat 9.19%
zebra 0.21%
jaguar, panther, Panthera onca, Felis onca 0.07%
tabby, tabby cat 0.03%
Cd into hello-wasm-bindgen
and run make install
the make serve
You should see something like this:
/* hello world Rust webassembly*/
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
extern "C" {
fn alert(s: &str);
}
//export the function to javascript
#[wasm_bindgen]
pub fn marco_polo(s: &str) {
//if the string is "Marco" return "Polo"
if s == "Marco" {
alert("Polo");
}
//if the string is anything else return "Not Marco"
else {
alert("Not Marco");
}
}
cd into linfa-kmeans
and run cargo run -- cluster
@noahgift âžś /workspaces/rust-mlops-template/regression-cli (main âś—) $ cargo run -- train --ratio .9
Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running `target/debug/regression-cli train --ratio .9`
Training ratio: 0.9
intercept: 152.1586901763224
params: [0, -0, 503.58067499818077, 167.75801599203626, -0, -0, -121.6828192430516, 0, 427.9593531331433, 6.412796328606638]
z score: Ok([0.0, -0.0, 6.5939908998261245, 2.2719123245079786, -0.0, -0.0, -0.5183690897253823, 0.0, 2.2777581181031765, 0.0858408096568952], shape=[10], strides=[1], layout=CFcf (0xf), const ndim=1)
predicted variance: -0.014761955865436382
- Based on this https://github.com/ggerganov/whisper.cpp[CPP version]
- Rust bindings here: https://github.com/tazz4843/whisper-rs
- Example repo here: https://github.com/nogibjj/rust-pytorch-gpu-template/blob/main/README.md#pytorch-rust-gpu-example
- Example repo here: https://github.com/nogibjj/rust-pytorch-gpu-template/blob/main/README.md#mnist-convolutional-neural-network
Stress Test CLI for both CPU and GPU PyTorch using Clap
cargo new stress
cd intostress
- To test CPU for PyTorch do:
cargo run -- cpu
- To test GPU for PyTorch do:
cargo run -- gpu
- To monitor CPU/Memory run
htop
- To monitor GPU run
nvidia-smi -l 1
- To use threaded GPU load test use:
cargo run -- tgpu
Full working example here: https://github.com/nogibjj/rust-pytorch-gpu-template/tree/main/stress
You can create it this repo for more info: https://github.com/nogibjj/rust-pytorch-gpu-template#stable-diffusion-demo
- clone this repo: https://github.com/LaurentMazare/diffusers-rs
- Follow these setup instructions: https://github.com/LaurentMazare/diffusers-rs#clip-encoding-weights
After all the weights are downloaded run:
cargo run --example stable-diffusion --features clap -- --prompt "A very rusty robot holding a fire torch to notebooks"
Stable Diffusion 2.1 Pegging GPU
Rusty Robot Torching Notebooks
cd into rust-ideas
cargo run -- --help
cargo run -- popular --number 4
cargo run -- random
@noahgift âžś /workspaces/rust-mlops-template/rust-ideas (main âś—) $ cargo run -- random
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Running `target/debug/rust-ideas random`
Random crate: "libc"
cd into OnnxDemo
and run make install
then cargo run -- infer
which invokes a squeezenet model.
Verified this works and is able to invoke runtime in a portable binary: https://github.com/sonos/tract/tree/main/examples/pytorch-resnet
Full working example link: https://github.com/nogibjj/assimilate-openai/tree/main/openai
-
install Rust via Rustup:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Use Rust API for OpenAI (3rd party): https://github.com/deontologician/openai-api-rust -
Create new project:
cargo new openai
and cd into it -
make format
thenmake lint
thencargo run
Working Example:
(.venv) @noahgift âžś /workspaces/assimilate-openai/openai (main) $ cargo run -- complete -t "The rain in spain"
Finished dev [unoptimized + debuginfo] target(s) in 0.14s
Running `target/debug/openai complete -t 'The rain in spain'`
Completing: The rain in spain
Loves gets you nowhere
The rain in spain
lib.rs
/*This uses Open AI to Complete Sentences */
//accets the prompt and returns the completion
pub async fn complete_prompt(prompt: &str) -> Result<String, Box<dyn std::error::Error>> {
let api_token = std::env::var("OPENAI_API_KEY")?;
let client = openai_api::Client::new(&api_token);
let prompt = String::from(prompt);
let result = client.complete_prompt(prompt.as_str()).await?;
Ok(result.choices[0].text.clone())
}
- go into
dscli
- Figure the way to make Polars work with
linfa
- How can I make a kmeans cluster using Polars
- cd into
webdocker
- build and run container (can do via
Makefile
) or
docker build -t fruit .
docker run -it --rm -p 8080:8080 fruit
- push to ECR
- Tell AWS App Runner to autodeploy
- Pyo3 Try the getting started guide:
# (replace string_sum with the desired package name)
$ mkdir string_sum
$ cd string_sum
$ python -m venv .env
$ source .env/bin/activate
$ pip install maturin
- Run
maturin init
and then runmaturin develop
ormake develop
python
- Run the following python code
import string_sum
string_sum.sum_as_string(5, 20)
The output should look like this: '25'
Follow guide here: https://pyo3.rs/v0.18.0/
- install
sudo apt-get install python3-dev
cargo new pyrust
andcd pyrust
- tweak
Cargo.toml
and addpyo3
- add source code to
main.rs
make run
Hello vscode, I'm Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110]
Q: Does the target binary have Python included?
A: Maybe. It does appear to be able to run Python if you go to the target
/workspaces/rust-mlops-template/pyrust/target/debug/pyrust
Follow up question, can I bring this binary to a "blank" codespace with no Python and what happens!
Goal: Build a high-performance Rust module and then wrap in a Python command-line tool
cargo new tyrscontainer
and cd intotyrscontainer
- copy a
Makefile
andDockerfile
fromwebdocker
Note that the rust build system container which is ~1GB is NOT in the final container image which is only 98MB.
FROM rust:latest as builder
ENV APP tyrscontainer
WORKDIR /usr/src/$APP
COPY . .
RUN cargo install --path .
FROM debian:buster-slim
RUN apt-get update && rm -rf /var/lib/apt/lists/*
COPY --from=builder /usr/local/cargo/bin/$APP /usr/local/bin/$APP
#export this actix web service to port 8080 and 0.0.0.0
EXPOSE 8080
CMD ["tyrscontainer"]
The final container is very small i.e. 94MB
strings latest 974d998c9c63 9 seconds ago 94.8MB
The end result is that you can easily test this web service and push to a cloud vendor like AWS and AWS App Runner.
Code here: https://github.com/nogibjj/assimilate-openai/tree/main/rust-curl-openai
(.venv) @noahgift âžś /workspaces/assimilate-openai/rust-curl-openai (main) $ cargo run
Compiling reqwest v0.11.14
Compiling rust-curl-openai v0.1.0 (/workspaces/assimilate-openai/rust-curl-openai)
Finished dev [unoptimized + debuginfo] target(s) in 4.78s
Running `target/debug/rust-curl-openai`
{"id":"cmpl-6rDd8mzOtMx7kKobqV0isiC7TkqU4","object":"text_completion","created":1678141798,"model":"text-davinci-003","choices":[{"text":"\n\nJupiter is the fifth planet from the Sun and the biggest one in our Solar System. It is very bright and can be seen in the night sky. It is named after the Roman god Jupiter. It is usually the third brightest thing you can see in the night sky after the Moon and Venus.","index":0,"logprobs":null,"finish_reason":"stop"}],"usage":{"prompt_tokens":151,"completion_tokens":62,"total_tokens":213}}
- Verified GCP Cloud Run works, code here: https://github.com/nogibjj/rust-mlops-template/tree/main/webdocker
First we need to compile: cargo install evcxr_jupyter
Next, lets do this: evcxr_jupyter --install
tldr; it does work! but you must do the following: jupyter notebook --generate-config
and then edit cross origin.
to run plotting tutorial do the following:
git clone https://github.com/38/plotters-doc-data
- Next session try to build a GPT 2 CLI from ORT: https://github.com/pykeio/ort
This build system is a bit unique because it recursives many Rust repos and tests them all!
- Comprehensive Rust Course Google
- Rust Async Book
- 52 Weeks of Rust
- Command-Line Rust Book
- Command-Line Rust Book Source Code
- awesome rust
- Microsoft Learn Rust
- Rust Machine Learning Book
- CLI Tools
- Sustainability with Rust
- Rust AWS Lambda
- EFS + Lambda (Great Use Case for Rust
- Rust uses less than half memory of Python AWS Lambda and 1.5 less duration in CPU
- Polars. You can see an example here.
One goal is to reduce using Notebooks in favor of lightweight markdown tools (i.e. the goal is MLOps vs interactive notebooks)
- Pyo3 rust binding for Python
- Inline Python in Rust
- Red Hat: Speed up Python using Rust
- Ruff 12-120X faster linting of Python code
- Convert Python model to Rust TensorFlow
- Hugging Face home grown inference in Rust
- onnxruntime for Rust
- Build standalone executables with Rust and ONNX
- Hugging Face discussion on Rust
- ort - ONNX Runtime Rust bindings
- Python vs Rust https://able.bio/haixuanTao/deep-learning-in-rust-with-gpu--26c53a7f
- Rust is 150x (15,000%) faster, and uses about the same amount of memory compared with Python.
- Rust 26X faster than Python sklearn
- 150+ million users with Rust MLOPs
- all language benchmarks
- polars benchmark vs pandas and dask
- Python vs Rust in AWS Lambda for a Data Guy
- Rust vs Python in AWS Lambda
- End of programming? I believe instead of end of coding it means level up