How to get OpenAI realtime API's text return from Agora RTC client side?

`Hello guys I have a question about intergrating Agora with Openai’s realtime api. My question is: How to get the text returned by openai realtime api from the client?

I’m following this page: https://docs.agora.io/en/open-ai-integration/get-started/quickstart?platform=python. I’ve set my own backend on AWS EC2, and the Voice part of OpenAI realtime api works good. Now I also want to allow the Client (Android and Ios) could access the text return by OpenAI.

I’ve noticed that is the official backend implementation: https://github.com/AgoraIO/openai-realtime-python/blob/main/realtime_agent/agent.py, -> _process_model_messages -> ResponseAudioTranscriptDelta, already provides the text return:

  case ResponseAudioTranscriptDelta():
                    # logger.info(f"Received text message {message=}")
                    asyncio.create_task(self.channel.chat.send_message(
                        ChatMessage(
                            message=to_json(message), msg_id=message.item_id
                        )
                    ))

                case ResponseAudioTranscriptDone():
                    logger.info(f"Text message done: {message=}")
                    asyncio.create_task(self.channel.chat.send_message(
                        ChatMessage(
                            message=to_json(message), msg_id=message.item_id
                        )
                    ))

So my final questions is basiclly how to get the data sent by “self.channel.chat.send_message” on IOS/Android/Web

Thanks for you help!

I’ve tried to use
1.signaling sdk
https://www.agora.io/en/products/signaling/

2.chat sdk
https://www.agora.io/en/products/chat/

But since I’m new to agora, I have not got things done so far.
`

You need to sign in to view this answers

Related Post