feat(llms)!: Move all model config options to OpenAIOptions

[davidmigloz]

Note: default completions model has been updated from text-davinci-003 to gpt-3.5-turbo-instruct. As the former will shut down on January 4th, 2024.

Until now, OpenAI completions wrapper used to accept the completions request parameters (e.g. temperature) in the constructor as parameters. Like:

final llm = OpenAI(apiKey: openaiApiKey, temperature: 0.9);

This was not ideal, as it was not possible to change the parameters after the object was created. So, for example, if you wanted to change the temperature, you had to create a new instance.

Now, all the completions request parameters are part of the OpenAIOptions class.

When you create an instance of OpenAI, you can define the defaultOptions that will be used for all the completions requests.

If you want to change the parameters for a specific request, you can pass the desired OpenAIOptions when making the request. The options passed in the request will override the defaultOptions.

Migration

Move all the request parameters from the constructor to the defaultOptions parameter.

Before:

final llm = OpenAI(
  apiKey: openaiApiKey,
  temperature: 0.9,
  maxTokens: 100,
);

Now:

final llm = OpenAI(
  apiKey: openaiApiKey,
  defaultOptions: const OpenAIOptions(
    temperature: 0.9,
    maxTokens: 100,
  ),
);

Example of using different options for different requests

final prompt = PromptValue.string('How are you?');
final llm = OpenAI(
  apiKey: openaiApiKey,
  defaultOptions: const OpenAIOptions(
    temperature: 0,
    maxTokens: 50,
  ),
);

final res1 = await llm.invoke(prompt);

final res2 = await llm.invoke(
  prompt,
  options: const OpenAIOptions(seed: 9999),
);

You can also change the options in a Runnable pipeline using the bind method. In this case, we are using two totally different models for each question:

final llm = OpenAI(apiKey: openaiApiKey,);
const outputParser = StringOutputParser();
final prompt1 = PromptTemplate.fromTemplate('How are you {name}?');
final prompt2 = PromptTemplate.fromTemplate('How old are you {name}?');
final chain = Runnable.fromMap({
  'q1': prompt1 | llm.bind(const OpenAIOptions(model: 'gpt-3.5-turbo-instruct')) | outputParser,
  'q2': prompt2| llm.bind(const OpenAIOptions(model: 'text-davinci-003')) | outputParser,
});
final res = await chain.invoke({'name': 'David'});