Receive Real-Time Video

Receive a real-time video stream from a bot's meeting.

📘

To start receive real time video streams, you need to include your websocket URL in create_bot.real_time_media.websocket_video_destination_url.

📘

This endpoint is optimized for those doing real time AI video analysis. We output 2 frames per second as analysis typically doesn't require all 30 frames per second.

If you are looking to receive real time video for human consumption instead, we advise you use create_bot.real_time_media.rtmp_destination_url, which will give you back normal 30 frames per second video.

If 2 frames per second works for your use case, keep reading for information on the protocol.

The first message on websocket connection will be:
{"protocol_version": 1, "bot_id": "YOUR-BOT-ID-HERE"}
The following websocket messages will be in binary format as follows:

• First 32 bits are a little-endian unsigned integer representing the "participant_id". This participant ID will be equivalent to the one used in the separate audio streams.
• Second 32 bits are a little-endian unsigned integer representing the millisecond timestamp of this frame. The timestamp is relative to the start of the video (not a unix timestamp).
• The remaining data in the websocket packet is the PNG encoded frame.

The following is sample code to decode these messages:

Python

import asyncio
import websockets


async def echo(websocket):
    async for message in websocket:
        if isinstance(message, str):
            print(message)
        else:
            stream_id = int.from_bytes(message[0:4], byteorder='little')
            timestamp = int.from_bytes(message[4:8], byteorder='little')
            with open(f'output/{stream_id}-{timestamp}.png', 'wb') as f:
                f.write(message[8:])
                print("wrote message")


async def main():
    async with websockets.serve(echo, "0.0.0.0", 8765):
        await asyncio.Future()

asyncio.run(main())