Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams

🐢 Open-Source Evaluation & Testing for AI & LLM systems

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.

Deliver safe & effective language models

[ICML 2024] TrustLLM: Trustworthiness in Large Language Models

The open-sourced Python toolbox for backdoor attacks and defenses.

Moonshot - A simple and modular tool to evaluate and red-team any LLM application.

🚀 A fast safe reinforcement learning library in PyTorch

[NeurIPS-2023] Annual Conference on Neural Information Processing Systems

A comprehensive toolbox for model inversion attacks and defenses, which is easy to get started.

[NeurIPS 2024] "Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration"

Code of the paper: A Recipe for Watermarking Diffusion Models

AI Verify

Neural Network Verification Software Tool

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

Official code repo for the O'Reilly Book - Machine Learning for High-Risk Applications

[USENIX Security 2025] PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models

A toolkit for tools and techniques related to the privacy and compliance of AI models.

The official implementation for ICLR23 paper "GNNSafe: Energy-based Out-of-Distribution Detection for Graph Neural Networks"

A project to add scalable state-of-the-art out-of-distribution detection (open set recognition) support by changing two lines of code! Perform efficient inferences (i.e., do not increase inference time) and detection without classification accuracy drop, hyperparameter tuning, or collecting additional data.