v1.1.5

New Features:

• Audio Responses: Agents can now deliver audio responses (both with streaming and non-streaming).
  • The audio is in the agent.run_response.response_audio.
  • This only works with OpenAIChat with the gpt-4o-audio-preview model. See their docs for more on how it works.
  • ‍See the audio_conversation_agent cookbook to test it out on the Agent Playground.

• Image understanding support for Together.ai and XAi: You can now give images to agents using models from XAi and Together.ai.

Improvements:

• Automated Tests: Added integration tests for all models. Most of these will be run on each pull request, with a suite of integration tests run before a new release is published.
• Grounding and Search with Gemini: Grounding and Search can be used to improve the accuracy and recency of responses from the Gemini models.

Bug Fixes:

• Structured output updates: Fixed various cases where native structured output was not used on models.
• Ollama tool parsing: Fixed cases for Ollama with tools with optional parameters.
• Gemini Memory Summariser: Fixed cases where Gemini models were used as the memory summariser.
• Gemini auto tool calling: Enabled automatic tool calling when tools are provided, aligning behavior with other models.
• FixedSizeChunking issue with overlap: Fixed issue where chunking would fail if overlap was set.
• Claude tools with multiple types: Fixed an issue where Claude tools would break when handling a union of types in parameters.
• JSON response parsing: Fixed cases where JSON model responses returned quoted strings within dictionary values.

‍