Python: Introducing Realtime Clients for OpenAI and Azure OpenAI

[eavanvalkenburg]

Motivation and Context

Implements the OpenAI Realtime API with Semantic Kernel

Description

Adds a RealtimeClientBase
Adds RealtimeEvents
Adds OpenAI Realtime clients

Closes: #10108

TODO:

• tests

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[markwallace-microsoft]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel/connectors/ai
chat_completion_client_base.py	124	2	98%	395, 405
function_calling_utils.py	48	6	88%	161–173
realtime_client_base.py	33	8	76%	104, 111–112, 116, 132–135
semantic_kernel/connectors/ai/open_ai/services
azure_config_base.py	56	2	96%	95, 97
azure_realtime.py	41	13	68%	38–40, 99–116, 168
open_ai_realtime.py	41	14	66%	37–39, 86–100, 130, 151
semantic_kernel/connectors/ai/open_ai/services/realtime
open_ai_realtime_base.py	180	135	25%	83–123, 144–193, 207–247, 255–428, 437–441, 447
open_ai_realtime_webrtc.py	123	85	31%	60–62, 65–73, 83–128, 133–140, 143–173, 181–190, 194–213
open_ai_realtime_websocket.py	55	26	53%	46–61, 64–70, 80–83, 88–91
utils.py	40	32	20%	38–44, 54, 69–126
semantic_kernel/connectors/ai/open_ai/settings
azure_open_ai_settings.py	26	4	85%	119–122
semantic_kernel/contents
audio_content.py	21	1	95%	88
binary_content.py	117	15	87%	99, 120, 126–128, 134, 147, 152–153, 194–199
function_call_content.py	107	3	97%	197, 225–226
TOTAL	19438	2555	87%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
3200	5 💤	0 ❌	0 🔥	1m 46s ⏱️

[Copilot]

Copilot reviewed 37 out of 37 changed files in this pull request and generated 1 comment.