AI Assistants Overview

What is an AI Assistant?

An AI Assistant is a configured voice bot that handles real phone calls — listening, understanding, speaking, and acting in real time. Use it for inbound (support, routing, scheduling) and outbound (qualification, follow-up, booking).

Why use an AI Assistant?

• 24/7 coverage with consistent quality
• Scale instantly for spikes and campaigns
• Save time/cost on repetitive calls; route high-intent calls to humans

General Settings

Define the assistant’s identity, channel, numbers, and voice — the core configuration that powers behavior.

Quick Start Guide

1. Choose call direction: Inbound or Outbound.
2. Set assistant name: Any descriptive label.
3. Configure phone number: Assign a dedicated number for this assistant.
4. Select voice & language: Pick a voice/accent and the primary language.
5. Adjust advanced settings: Voice engine, protection, VAD, transcription, etc.

Engine Type (Voice Processing Mode)

• Pipeline Mode: STT → LLM → TTS. Maximum control over voice selection and tuning.
• Speech-to-Speech Mode: Direct speech generation without intermediate text. Most natural flow.
• DualPlex Mode (Recommended): Fast multimodal processing + premium ElevenLabs/Vogent output. Best default.

Language Configuration

• Primary Language: The language your assistant starts with.
• Secondary Languages (up to 2): Optional; can switch mid-call based on caller speech.

AI Voice Selection

Option 1 — Voice Library: Choose from existing voices tuned for specific languages/accents. Pick gender (male/female) and tone (formal/casual).

Option 2 — Custom Voice: Upload a 1–5 minute MP3/WAV recording (min 1 minute) of a single speaker with consistent tone, low noise. Processing takes only a few minutes.

Call Protection Settings

• Noise Cancellation: Filters caller background noise for clearer speech.
• End Call on Voicemail: Detect voicemail; hang up to conserve minutes.
• Call Recording: Enable/disable recordings based on your compliance needs.
• Max Initial Silence: End the call if no caller response within N seconds.

Synthesizer Settings (Pipeline & DualPlex)

• Voice Stability (0.0–1.0): Default 0.7. Lower = more expressive but less predictable; higher = steadier, less emotional.
• Voice Similarity (0.0–1.0): Default 0.5. Higher = closer to original (may include artifacts); lower = cleaner but less exact.
• Speech Speed (0.7–1.2): Default 1.0. Slower for complex topics/older demographics; faster for short, time-sensitive calls.

Transcriber Settings (Pipeline only)

• Azure: Highest accuracy; slightly higher latency. Best when fidelity matters most.
• Gladia: Good accuracy; fast latency; strong multilingual support.
• Deepgram: Fastest; solid for English and major languages.

Voice Activity Detection (VAD)

• Endpoint Sensitivity: 0.0–5.0s (default 0.5). Lower = quicker replies but may cut callers; higher = waits longer.
• Interruption Sensitivity: How easily the assistant stops when interrupted.
• Minimum Interrupt Words: Require N caller words before interrupting to avoid false triggers.

Knowledge Base

Attach PDFs, text, or website URLs (single or multiple) for the assistant to reference.

• Function-Call Mode (recommended): Searches KB only when needed via a tool call. Efficient and accurate.
• Prompt Injection Mode: Always retrieves after every utterance. Simpler but potentially slower without filler audio.

Custom Mid-Call Tools

Create your own API tools the assistant can invoke mid-conversation (check inventory, verify customer data, fetch quotes, etc.). Assign tools per assistant.

Default Tools

• Call Transfer: Warm transfer to human agents/queues.
• DTMF Input: Collect keypad input securely.
• Appointment Scheduling: Cal.com / GoHighLevel and other integrations.

System Prompt Best Practices

• Role & Identity: “You are a professional sales assistant for XYZ…”
• Conversation Style: Friendly, concise, clear escalation rules.
• Key Information: Products, pricing, promos, policies.
• Behavioral Guidelines: Verify identity, handle objections, when to transfer, etc.
• Response Framework: “When asked about X, do Y…” Include worked examples.
• Be Specific & Structured: Bullets > paragraphs; test and iterate like training a rep.

Call Variables

Define interactive variables directly in the prompt. Reference with {variable_name}. Use them to guide logic, collect data, and personalize responses.

Variable Configuration

After the call, the AI extracts variables from the full transcript based on your schema.

• Fields: name (3–16 chars, lowercase, alphanumeric/underscores), type (string, number, true_false), description.
• Defaults: status (boolean — objective achieved) and summary (string).
• Extraction Process: Reviews transcript → considers system prompt & objectives → finds data per variable description → validates → outputs.
• Best Practices: lowercase identifiers; clear descriptions; align with system prompt goals.

Webhook Configuration

• Enable: Turn on when your endpoint is ready.
• Webhook URL: HTTPS endpoint to receive data.
• Test: Built-in “Make Test Request.”
• Delivery: Send on completed calls only (default) or all calls.
• Include Recording: Optional recording URL in payload.

Testing Process

1. Configure variables and webhook URL.
2. Save settings.
3. Click Make Test Request.
4. Verify your endpoint receives data.
5. Confirm payload fields/format.

Testing Methods

Web “Speak With Your Assistant”: Browser-based, real-time voice; tests tools & knowledge; fastest iteration.

Phone Call Testing: Place/receive real calls through assigned numbers for realistic latency and routing.

Widget Setup

• Easy Integration: Add to any website.
• Features: Voice chat with text fallback; customizable UI; mobile responsive.
• Behavior: Uses the same knowledge, tools, languages, and post-call variables as phone calls; webhooks fire for web sessions too.

Purpose & When to Use

Custom mid-call tools let your assistant call your APIs during the conversation to check orders, verify identity, fetch quotes, schedule, or push data. Use them when the AI must fetch or write real-time data that isn’t in the knowledge base.

• Examples: check_order_status, verify_customer, get_inventory, create_ticket, fetch_quote, reserve_time_slot.
• Invocation: The assistant decides to call a tool based on your prompt rules (“Use check_order_status when the caller provides an order number”).
• Results: Tool responses are summarized back to the caller and can update variables/state mid-call.

Setup (Step-by-Step)

1. Name: lowercase_with_underscores (e.g., check_order_status).
2. Description: When/why to use this tool (clear triggers + expected outcome).
3. Endpoint (URL): e.g., https://api.yourcompany.com/orders/status
4. Method: GET / POST / PUT / PATCH / DELETE.
5. Timeout (s): How long to wait before the assistant moves on (e.g., 8-15s).
6. Headers: e.g., Content-Type: application/json, Authorization: Bearer <token>.
7. Parameters: Define inputs the assistant must collect (name, type, description, required).
8. Response Mapping: Tell the assistant what fields to read from the JSON response (e.g., status, eta).
9. On Error / Fallback: Friendly message + what to try next (retry, escalate, or continue).

Parameters & Validation

• string: names, addresses, references.
  Example: order_number (“10-digit order number from the customer”).
• number: amounts, quantities, numeric IDs.
  Example: quantity (e.g., 3).
• true_false: boolean flags (yes/no).
  Example: express_shipping (true/false).

Validation in descriptions: “date in DD/MM/YYYY”, “phone without spaces”, “email address format”. The assistant will confirm/correct with the caller before calling the tool.

Dynamic Endpoints

You can insert variables collected during the call into your endpoint or query/body.

The assistant substitutes {order_id} and {customer_email} with confirmed values before sending.

Security, Timeouts & Reliability

• Auth: Bearer tokens or API keys in headers. Rotate regularly; never put secrets in prompts.
• Allowlist: Optionally allow RepForce IPs only on your API gateway.
• Timeouts: Choose a sane limit (8–15s). On timeout, assistant apologizes and offers to retry or escalate.
• Retries: Keep 1–2 retries max with backoff; avoid loops.
• Logging: Log request ID, timestamp, input params (no PII when possible), HTTP status, and response time.
• Rate Limits: If 429, assistant can wait briefly and retry once or route to a human.

Response Contract

Return consistent JSON so the assistant can reliably summarize and act.

Errors:

In your prompt, teach the assistant how to phrase results (“Your order A123… ships in 2 days”), and how to handle errors (“I couldn’t find that order — can I double-check the number?”).

Example Tool Definition (JSON)

Quick cURL Test

Confirm the JSON matches your contract before enabling the tool for callers.

Default Tools (Built-In)

• Call Transfer: Warm transfer to human agents/queues.
• DTMF Input: Collect keypad digits securely (PINs, extensions).
• Appointment Scheduling: Cal.com / GoHighLevel and other integrations.

Prompt Rules (Make It Reliable)

• Specify exact triggers: “If the caller states or confirms an order number, call check_order_status.”
• Require the assistant to confirm parameters back (“Let me confirm your order number is A123… correct?”).
• Teach error phrasing and fallbacks (retry once → escalate if still failing).
• After a tool call, summarize succinctly and recommend next steps.

Testing Checklist

1. Create the tool and set headers, timeout, and params.
2. Test with cURL or Postman (happy path + errors + timeouts).
3. In the UI, enable the tool and add clear prompt rules.
4. Use Speak With Your Assistant to verify live behavior.
5. Check logs for request/response timing, status codes, and mapping.
6. Simulate edge cases (missing params, 404, 429, 500) and confirm graceful handling.