Rime Text-to-Speech

Overview

Rime AI provides two TTS service implementations: RimeTTSService (WebSocket-based) with word-level timing and interruption support, and RimeHttpTTSService (HTTP-based) for simpler use cases. RimeTTSService is recommended for real-time interactive applications.

Rime TTS API Reference

Pipecat’s API methods for Rime TTS integration

Example Implementation

Complete example with word timestamps

Rime Documentation

Official Rime WebSocket and HTTP API documentation

Voice Models

Explore available voice models and features

Installation

To use Rime services, install the required dependencies:

uv add "pipecat-ai[rime]"

Prerequisites

Rime Account Setup

Before using Rime TTS services, you need:

Rime Account: Sign up at Rime AI
API Key: Generate an API key from your account dashboard
Voice Selection: Choose from available voice models

Required Environment Variables

RIME_API_KEY: Your Rime API key for authentication

Configuration

RimeTTSService

str

required

Rime API key for authentication.

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(voice=...) instead.

str

default:"wss://users-ws.rime.ai/ws3"

Rime WebSocket API endpoint.

str

default:"coda"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(model=...) instead.

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

TextAggregationMode

default:"TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries, producing more natural speech. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

bool

default:"None"

Deprecated in v0.0.104. Use text_aggregation_mode instead.

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeTTSService.Settings(...) instead.

RimeTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeHttpTTSService

str

required

Rime API key for authentication.

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(voice=...) instead.

aiohttp.ClientSession

required

An aiohttp session for HTTP requests.

str

default:"coda"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(model=...) instead.

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeHttpTTSService.Settings(...) instead.

RimeHttpTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeTTSService Settings below.

RimeNonJsonTTSService

A deprecated non-JSON WebSocket service that uses plain-text messages and raw audio responses. Use RimeTTSService for current integrations.

str

required

Rime API key for authentication.

str

required

deprecated

ID of the voice to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(voice=...) instead.

str

default:"wss://users.rime.ai/ws"

Rime WebSocket API endpoint.

str

default:"coda"

deprecated

Model ID to use for synthesis. Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(model=...) instead.

str

default:"pcm"

Audio output format.

int

default:"None"

Output audio sample rate in Hz. When None, uses the pipeline’s configured sample rate.

TextAggregationMode

default:"TextAggregationMode.SENTENCE"

Controls how incoming text is aggregated before synthesis. SENTENCE (default) buffers text until sentence boundaries. TOKEN streams tokens directly for lower latency. Import from pipecat.services.tts_service.

bool

default:"None"

deprecated

Deprecated in v0.0.104. Use text_aggregation_mode instead.

InputParams

default:"None"

deprecated

Deprecated in v0.0.105. Use settings=RimeNonJsonTTSService.Settings(...) instead.

RimeNonJsonTTSService.Settings

default:"None"

Runtime-configurable settings. See RimeNonJsonTTSService Settings below.

RimeTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`None`	Model identifier. (Inherited.)
`voice`	`str`	`None`	Voice identifier. (Inherited.)
`language`	`Language \| str`	`None`	Language for synthesis. (Inherited.)
`segment`	`str`	`NOT_GIVEN`	Segment type for synthesis.
`speedAlpha`	`float`	`NOT_GIVEN`	Speed alpha parameter.
`reduceLatency`	`bool`	`NOT_GIVEN`	Whether to reduce latency.
`pauseBetweenBrackets`	`bool`	`NOT_GIVEN`	Pause between brackets.
`phonemizeBetweenBrackets`	`bool`	`NOT_GIVEN`	Phonemize between brackets.
`noTextNormalization`	`bool`	`NOT_GIVEN`	Disable text normalization.
`saveOovs`	`bool`	`NOT_GIVEN`	Save out-of-vocabulary words.
`inlineSpeedAlpha`	`str`	`NOT_GIVEN`	Inline speed alpha.
`repetition_penalty`	`float`	`NOT_GIVEN`	Token repetition penalty (coda only, 1.0-2.0).
`temperature`	`float`	`NOT_GIVEN`	Sampling temperature (coda only, 0.0-1.0).
`top_p`	`float`	`NOT_GIVEN`	Cumulative probability threshold (coda only, 0.0-1.0).
`timeScaleFactor`	`float`	`NOT_GIVEN`	Audio playback speed factor (coda only). Values above 1.0 slow down the audio; values below 1.0 speed it up.

RimeNonJsonTTSService Settings

Runtime-configurable settings passed via the settings constructor argument using RimeNonJsonTTSService.Settings(...). These can be updated mid-conversation with TTSUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
`model`	`str`	`None`	Model identifier. (Inherited.)
`voice`	`str`	`None`	Voice identifier. (Inherited.)
`language`	`Language \| str`	`None`	Language for synthesis. (Inherited.)
`segment`	`str`	`NOT_GIVEN`	Segment type for synthesis.
`repetition_penalty`	`float`	`NOT_GIVEN`	Token repetition penalty.
`temperature`	`float`	`NOT_GIVEN`	Sampling temperature.
`top_p`	`float`	`NOT_GIVEN`	Cumulative probability threshold.

Usage

Basic Setup (WebSocket)

from pipecat.services.rime.tts import RimeTTSService

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        model="coda",
        voice="luna",
    ),
)

With Customization (WebSocket)

from pipecat.services.rime.tts import RimeTTSService
from pipecat.transcriptions.language import Language

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        voice="cove",
        model="mistv2",
        language=Language.ES,
        speedAlpha=1.2,
        reduceLatency=True,
    ),
)

HTTP Service

import aiohttp
from pipecat.services.rime.tts import RimeHttpTTSService

async with aiohttp.ClientSession() as session:
    tts = RimeHttpTTSService(
        api_key=os.getenv("RIME_API_KEY"),
        settings=RimeHttpTTSService.Settings(
            model="coda",
            voice="luna",
        ),
        aiohttp_session=session,
    )

Non-JSON WebSocket

from pipecat.services.rime.tts import RimeNonJsonTTSService

tts = RimeNonJsonTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeNonJsonTTSService.Settings(
        model="coda",
        voice="luna",
    ),
)

Customizing Speech

RimeTTSService provides a set of helper methods for implementing Rime-specific customizations, meant to be used as part of text transformers. These include methods for spelling out text, adjusting speech rate, and modifying pitch. See the Text Transformers for TTS section in the Text-to-Speech guide for usage examples.

RimeTTSService.SPELL(text: str) -> str

Static method that implements Rime’s spell function to spell out text character by character. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert Rime's spell tags around the provided text.
async def spell_out_text(text: str, type: str) -> str:
    return RimeTTSService.SPELL(text)

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("phone_number", spell_out_text),
    ],
)

RimeTTSService.PAUSE_TAG(seconds: float) -> str

Static method that implements Rime’s custom pause functionality to generate a properly formatted pause tag you can insert into the text. Call this method on the class, not on an instance.

# Text transformers for TTS
# This will insert a one second pause after questions.
async def pause_after_questions(text: str, type: str) -> str:
    if text.endswith("?"):
        return f"{text}{RimeTTSService.PAUSE_TAG(1.0)}"
    return text

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    text_transforms=[
        ("sentence", pause_after_questions), # Only apply to sentence aggregations
    ],
)

PRONOUNCE(text: str, word: str, phoneme: str) -> str

Instance method that implements Rime’s custom pronunciations feature. It replaces word in text with the phoneme tag. Unlike SPELL and PAUSE_TAG, call it on the service instance, not on the class. Its three arguments do not match the (text, aggregation_type) transform shape, so wrap it in a function.

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
    settings=RimeTTSService.Settings(
        # Keep bracket phonemization on for the whole session
        phonemizeBetweenBrackets=True,
    ),
)

# Text transformers for TTS
# This puts a phoneme in place of the word "potato" to define how it
# should be pronounced.
async def maybe_say_potato_all_fancylike(text: str, type: str) -> str:
    if using_fancy_voice:
        return tts.PRONOUNCE(text, "potato", "potato")
    else:
        return tts.PRONOUNCE(text, "potato", "poteto")

tts.add_text_transformer(maybe_say_potato_all_fancylike, "*")  # Apply to all text

PRONOUNCE turns on bracket phonemization for the next message only. If any message may contain a phoneme, set phonemizeBetweenBrackets=True in Settings (as above) so it stays on for the whole session.

PRONOUNCE handles one word at a time. For hundreds of overrides, use the replace_text transform. See Bulk pronunciation overrides.

INLINE_SPEED(text: str, speed: float) -> str

Instance method that implements Rime’s inline speed adjustment feature. It wraps text in [] and adds speed to the inlineSpeedAlpha request field. Like PRONOUNCE, call it on the service instance, not on the class.

tts = RimeTTSService(
    api_key=os.getenv("RIME_API_KEY"),
)

# Text transformers for TTS
# This will make the word slow always be spoken more slowly.
async def slow_down_slow_words(text: str, type: str) -> str:
    return text.replace(
        "slow",
        tts.INLINE_SPEED("slow", speed=0.5)
    )

tts.add_text_transformer(slow_down_slow_words, "*")  # Apply to all text

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

Word-level timestamps: RimeTTSService provides word-level timing information, enabling synchronized text highlighting.
WebSocket vs HTTP: The WebSocket service supports word-level timestamps, interruption handling, and maintains context across messages within a turn. The HTTP service is simpler but lacks these features.
Non-JSON WebSocket: RimeNonJsonTTSService is the deprecated plain-text and raw-audio WebSocket implementation. It does not support word-level timestamps.

Event Handlers

Rime WebSocket TTS services support the standard service connection events:

Event	Description
`on_connected`	Connected to Rime WebSocket
`on_disconnected`	Disconnected from Rime WebSocket
`on_connection_error`	WebSocket connection error occurred

@tts.event_handler("on_connected")
async def on_connected(service):
    print("Connected to Rime")

Get Started

Pipecat Server

Client SDKs

Pipecat Flows

Pipecat Cloud

CLI

Pipecat Context Hub

Rime Text-to-Speech

Overview

Rime TTS API Reference

Example Implementation

Rime Documentation

Voice Models

Installation

Prerequisites

Rime Account Setup

Required Environment Variables

Configuration

RimeTTSService

RimeHttpTTSService

RimeNonJsonTTSService

RimeTTSService Settings

RimeNonJsonTTSService Settings

Usage

Basic Setup (WebSocket)

With Customization (WebSocket)

HTTP Service

Non-JSON WebSocket

Customizing Speech

RimeTTSService.SPELL(text: str) -> str

RimeTTSService.PAUSE_TAG(seconds: float) -> str

PRONOUNCE(text: str, word: str, phoneme: str) -> str

INLINE_SPEED(text: str, speed: float) -> str

Notes

Event Handlers

​Overview

Rime TTS API Reference

Example Implementation

Rime Documentation

Voice Models

​Installation

​Prerequisites

​Rime Account Setup

​Required Environment Variables

​Configuration

​RimeTTSService

​RimeHttpTTSService

​RimeNonJsonTTSService

​RimeTTSService Settings

​RimeNonJsonTTSService Settings

​Usage

​Basic Setup (WebSocket)

​With Customization (WebSocket)

​HTTP Service

​Non-JSON WebSocket

​Customizing Speech

​RimeTTSService.SPELL(text: str) -> str

​RimeTTSService.PAUSE_TAG(seconds: float) -> str

​PRONOUNCE(text: str, word: str, phoneme: str) -> str

​INLINE_SPEED(text: str, speed: float) -> str

​Notes

​Event Handlers

Overview

Installation

Prerequisites

Rime Account Setup

Required Environment Variables

Configuration

RimeTTSService

RimeHttpTTSService

RimeNonJsonTTSService

RimeTTSService Settings

RimeNonJsonTTSService Settings

Usage

Basic Setup (WebSocket)

With Customization (WebSocket)

HTTP Service

Non-JSON WebSocket

Customizing Speech

RimeTTSService.SPELL(text: str) -> str

RimeTTSService.PAUSE_TAG(seconds: float) -> str

PRONOUNCE(text: str, word: str, phoneme: str) -> str

INLINE_SPEED(text: str, speed: float) -> str

Notes

Event Handlers