openai-unreal

OpenAI API wrapper and Blueprint Library for Unreal Engine.

Inspired by OpenAI API wrapper for Unity created by srcnalt: https://github.com/srcnalt/OpenAI-Unity

Tested in UE 5.0 and 5.1. launcher version

It covers almost all OpenAI available as of Jan 24th, 2023 - https://beta.openai.com/docs/api-reference/

• Completion
• Edits
• Files
• Fine-tune
• Image
• Moderations

Getting started

Quick setup for this plugin:

• Create a UE project
• Grab the latest .zip file in the release page: https://github.com/rezonated/openai-unreal/releases
• In your root project folder, create a folder called "Plugins"
• Extract/copy the content of the .zip file to the newly created Plugins folder.
• Open your UE project and enable the plugin under "Open AI" category.

• Restart your editor.
• Go to project settings, scroll ahead to Plugins > Open AI - Unreal category.
• Fill out your API key and Organization ID that you can obtain by registering through OpenAI's website and generate API key.
• Paste your API key and Organization ID to respective fields

• Done!

Example usages

These are some of the examples for each API usage:

Completion API - https://beta.openai.com/docs/api-reference/completions/create

Create a completion for a prompt of: "Give me a random name and greet me as a stranger."

Edits API - https://beta.openai.com/docs/api-reference/edits/create

Create an edit for an input of "What day of the wek is it?", and the instruction to "Fix typo."

Embeddings API - https://beta.openai.com/docs/api-reference/embeddings/create

Create an embedding vector for an input of "The food delicious and the waiter..."

Moderations API - https://beta.openai.com/docs/api-reference/moderations/create

Classifies if the input violates the OpenAI's content violation policy.

Files and Fine-tune API - https://beta.openai.com/docs/api-reference/files, https://beta.openai.com/docs/api-reference/fine-tunes

Uploads a .jsonl file containing training examples that will be used for creating fine-tune purposes.

Images API - https://beta.openai.com/docs/api-reference/images

You can choose to receive the response in either URL or Base64 JSON format.

• Create an image edit using base image and a mask image with a prompt of "Add bird", and then convert it into a Texture2D.

• Create an image variation using base image.

Example maps

This plugin ships with two example maps of implementations that make use of completion and images API. It can be found under plugin content folder.

Make sure you enable the view of the plugin content.

Test files

Test files can be found under {Your project directory}/Plugins/UnrealOpenAI/Content/Test/

It consists of example files for testing:

• pool_empty.png -> for base image
• pool_mask.png -> for image mask
• training.jsonl -> containing training data for fine-tuning

Tests

This plugin uses Unreal's functional testing for automated tests. The tests are separated into its own module called "UnrealOpenAITests"

To enable the module:

• Open UnrealOpenAI.uplugin file inside root directory of the plugin using any text editor you like
• Paste the following inside "Modules" entry:

,
  {
   "Name": "UnrealOpenAITests",
   "Type": "Runtime",
   "LoadingPhase": "Default",
   "AdditionalDependencies": [
  "UnrealOpenAI"
   ]
  }

• Open your solution and recompile the project. You might need to regenerate your project files.
• Open the editor and head to Test folder inside this plugin's content folder, and open TestMap map.

• Open the Session Frontend window in your editor by going through Tools > Session Frontend
• Go to Automation tab and search for "OpenAI", there will be 7 available functional tests to run through.

Contributing

Contributions are welcome! Submit your issues, pull requests or whatever~

Versioning

This plugin uses zerover versioning scheme, read more at: https://0ver.org/

Special thanks

• srcnalt for inspiration
• benui and benui discord server
• Unreal Slacker discord server
• MF Habibie
• F. Magdalena