Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for audio in chat completions in openai_dart #577

Merged
merged 1 commit into from
Oct 21, 2024

Conversation

davidmigloz
Copy link
Owner

@davidmigloz davidmigloz commented Oct 21, 2024

In addition to generating text and images, some OpenAI models enable you to generate a spoken audio response to a prompt:

final res = await client.createChatCompletion(
  request: CreateChatCompletionRequest(
    model: ChatCompletionModel.model(
      ChatCompletionModels.gpt4oAudioPreview,
    ),
    modalities: [
      ChatCompletionModality.text,
      ChatCompletionModality.audio,
    ],
    audio: ChatCompletionAudioOptions(
      voice: ChatCompletionAudioVoice.alloy,
      format: ChatCompletionAudioFormat.wav,
    ),
    messages: [
      ChatCompletionMessage.user(
        content: ChatCompletionUserMessageContent.string(
          'Is a golden retriever a good family dog?',
        ),
      ),
    ],
  ), 
);
final choice = res.choices.first;
final audio = choice.message.audio;
print(audio?.id);
print(audio?.expiresAt);
print(audio?.transcript);
print(audio?.data);

And to use audio inputs to prompt the model:

final res = await client.createChatCompletion(
  request: CreateChatCompletionRequest(
    model: ChatCompletionModel.model(
      ChatCompletionModels.gpt4oAudioPreview,
    ),
    modalities: [
      ChatCompletionModality.text,
      ChatCompletionModality.audio,
    ],
    audio: ChatCompletionAudioOptions(
      voice: ChatCompletionAudioVoice.alloy,
      format: ChatCompletionAudioFormat.wav,
    ),
    messages: [
      ChatCompletionMessage.user(
        content: ChatCompletionUserMessageContent.parts([
          ChatCompletionMessageContentPart.text(
            text: 'Do what the recording says',
          ),
          ChatCompletionMessageContentPart.audio(
            inputAudio: ChatCompletionMessageInputAudio(
              data: 'UklGRoYZAQBXQVZFZm10I...//X//v8FAOj/GAD+/7z/',
              format: ChatCompletionMessageInputAudioFormat.wav,
            ),
          ),
        ]),
      ),
    ],
  );
);
final choice = res.choices.first;
final audio = choice.message.audio;
print(audio?.id);
print(audio?.expiresAt);
print(audio?.transcript);
print(audio?.data);

Docs:

@davidmigloz davidmigloz self-assigned this Oct 21, 2024
@davidmigloz davidmigloz added t:enhancement New feature or request p:openai_dart openai_dart package. labels Oct 21, 2024
@davidmigloz davidmigloz added this to the v0.8.0 milestone Oct 21, 2024
@davidmigloz davidmigloz merged commit 0fb058c into main Oct 21, 2024
3 checks passed
@davidmigloz davidmigloz deleted the audio branch October 21, 2024 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p:openai_dart openai_dart package. t:enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant