AI Virtual Agent (Voicebot - LLM) Integration (Teggy Voicebot Integration)

Tegsoft Contact Center solution is now offering LLM integrations to be used as an AI Virtual Agent. This article is for developers and AI software vendors to develop virtual agent integrations to integrate their solutions with the Tegsoft Contact Center.

Most AI Virtual Agent vendors are expecting input in text format and generating output in text format. Being a bridge between the communications world and the software world is the mission of Tegsoft company. To have efficient and robust solutions Tegsoft is handling speech-to-text and text-to-speech and transferring text (generated from customer speech) to LLM  and announce / voice (generated from LLM text) to customers.

Simply integration has been developed and published under the terms listed below,

• The virtual agent is nothing different than a human agent in terms of communication processing.

• The virtual agent is always available on the defined communication channel (like voice queues, webchat skills, or voice campaigns).

• The virtual agents are configured and assigned to skills just like human agents

• The virtual agents communicate with only one customer at a time like human agents. If 10 virtual agents are needed 10 virtual agents need to be configured and activated.

• Communications with virtual agents are reported and recorded like a human agent.

• All the communication is processed under one unique ID as a session ID.

Configuration

Voicebot Scenario Configuration

Under the “Management / Teggy Management” application a new Teggy scenario needs to be defined as Voicebot. This named scenario will be used for related agents.

• Name: Please type a name for the scenario. That name will be selected under the agent configuration

• Assistant Type: Customer Assistant (Voicebot) needs to be selected for voicebot.

• Config Type: “designer” value can be selected to manage the JSON file locally or external URL can be defined for fetching config from external source.

Voicebot Scenario (in JSON Format)

The scenario has 5 sections,

1. config: is a reserved section. Can be empty or be removed.

2. greetings: this is an optional section. If you want to start the conversation with a set of greeting messages you can use this JSONArray

   1. type: “TTS”, and “announce” values can be used.

   2. message: to play after generating voice prompt from TTS engine.

   3. announceId: is the pre-recorded announce to be played as greeting

3. STT: Section is the section that holds the configuration parameters of the speech-to-text engine. (Only Google STT is supported at the moment, additional engines will be implemented soon)

   1. provider: “google” value is the only supported engine now.

   2. URL: Provider endpoint URL needs to be defined here.

   3. apiKey: is the service account API access key

   4. languageCode: is in the default language in BCP-47 format please check the link for the options.

   5. maxSilence: There are two types of STT, the  “continuous speech recognition” option or “record and recognize” when using the “record and recognize” technique this is the amount of silence (in seconds) to allow before returning despite the lack of dtmf digits or reaching timeout.

   6. timeout: Max duration of recognition in milliseconds. Less than 10 seconds (10000) is recommended.

   7. beep: false will do the silent recognition, true will play a beep sound before recognition

   8. sampleRateHertz: The value must be 8000. It is not configurable at the moment and needs to be fixed value as 8000.

   9. headers: The value must be "headers": {"Content-Type": "application/json; charset=utf-8"}. The feature is not configurable at the moment. This option stays for future use. At the moment needs to be fixed value.

Example file,

{
  "config": {
  },
  "greetings": [
    {
      "type": "TTS",
      "message": "Welcome!"
    },
    {
      "type": "announce",
      "announceId": "d33e88fe-3fcf-4bc8-a315-f349ab6b3a69"
    }
  ],
  "STT": {
    "provider": "google",
    "URL": "https://speech.googleapis.com/v1/speech:recognize",
    "apiKey": "AIza1HyuuHowYuareaaGGttt1122AAa1Za",
    "languageCode": "en-US",
    "maxSilence": 2,
    "timeout": 5000,
    "beep": false,
    "sampleRateHertz": 8000,
    "headers": {
      "Content-Type": "application/json; charset=utf-8"
    }
  },
  "TTS": {
    "provider": "tegsoft",
    "providerId": "6d427f34-0c15-49d8-8d4d-49909c6de282"
  },
  "LLM": {
    "provider": "tegsoft",
    "URL": "https://arge23.tegsoftcloud.com/Tobe/view/LLM?service=processText",
    "methodType": "POST",
    "parameters": {
      "param": "value"
    },
    "headers": {
      "Content-Type": "application/json; charset=utf-8"
    }
  } 
}

Agent Configuration

Virtual Voice agents need to be configured like human agents. The following steps will activate a Teggy Virtual Agent Voicebot,

• The extension needs to be defined

• A user needs to be defined

• User and extension mapping needs to be completed

• User primary role needs to be “Agent” and user needs to be defined as an agent

• On the agent settings agent type needs to be Teggy and related scenario needs to be selected

• Once above settings are done virtual agent is ready to be assigned to related skills (Voice Queues, Webchat skills, or Campaigns)

Communication Protocol

When there is an active agent on the queue or campaign you should see presence of the agent on the dashboard. For inbound when there is an incoming call to the queue virtual agent will ring just like human agent and will answer the call and start processing. The same process applies for the campaign as well. When customer connects the protocol flow will take place.

1. Customer requests will be processed like a regular queue. This allow LLM providers to run different skills, language options or different scenarios for the virtual boat provider.

2. The speech recording of the customer is transferred to STT provider to receive text message

3. The text message is transferred to the LLM provider to receive next action set.

4. If there are any message to read in the action set of the LLM provider those messages are converted to audio announcements by the help of TTS provider

5. Speech goes back and forth, the actions (playing announcements, TTS responses, or transfers) are executed and two way communication takes place.

LLM Request

• status: Reserved.

• agent: All the agent details including state and busy contact details

• sessionid: Unique channel id

• message: Message to process (message in text format converted via STT)

• history: Conversation history

Example,

{
  "status": "busy",
  "agent": {},
  "sessionid": "1231231.123",
  "begindate": "2021-11-08 22:25:02.555",
  "dataVersion": "1.1",
  "contactName": "John Doe",
  "teggyConfigDetails": {},
  "skillName": "Sales",
  "skill": "3997e3c3-9e23-447d-92a2-3bd7cd594a68",
  "contid": "0221f910-9595-4538-954c-4bcd80ad6b4c",
  "message": "NEW_MESSAGE",
  "history": [
  {
      "processDate": 1747046560230,
      "type": "changeLanguage",
      "message": "Language changed to tr-TR",
      "party": "system"
    },
    {
      "processDate": 1747046560230,
      "type": "sayTTS",
      "message": "hoşgeldiniz",
      "party": "agent"
    },
    {
      "processDate": 1747046561250,
      "type": "playAnnounce",
      "message": "d33e88fe-3fcf-4bc8-a315-f349ab6b3a69",
      "party": "agent"
    },
    {
      "processDate": 1747046574214,
      "type": "speechToText",
      "message": "eştir",
      "party": "customer"
    },
    {
      "processDate": 1747046574249,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046585364,
      "type": "speechToText",
      "message": "dil değiştir dil değiştir",
      "party": "customer"
    },
    {
      "processDate": 1747046585394,
      "type": "changeLanguage",
      "message": "Language changed to en-US",
      "party": "agent"
    },
    {
      "processDate": 1747046589062,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046597945,
      "type": "speechToText",
      "message": "set variable",
      "party": "customer"
    },
    {
      "processDate": 1747046597987,
      "type": "setVariable",
      "message": "Variables are assigned to the channel.",
      "party": "agent"
    },
    {
      "processDate": 1747046604181,
      "type": "speechToText",
      "message": "set headers",
      "party": "customer"
    },
    {
      "processDate": 1747046604215,
      "type": "setHeader",
      "message": "Headers are assigned to the channel.",
      "party": "agent"
    },
    {
      "processDate": 1747046607878,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046616535,
      "type": "speechToText",
      "message": "play audio",
      "party": "customer"
    },
    {
      "processDate": 1747046616561,
      "type": "playAnnounce",
      "message": "d33e88fe-3fcf-4bc8-a315-f349ab6b3a69",
      "party": "agent"
    },
    {
      "processDate": 1747046630221,
      "type": "speechToText",
      "message": "read message read message",
      "party": "customer"
    },
    {
      "processDate": 1747046630246,
      "type": "sayTTS",
      "message": "Bu bir test mesajıdır.",
      "party": "agent"
    },
    {
      "processDate": 1747046635537,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046644935,
      "type": "speechToText",
      "message": "transaction",
      "party": "customer"
    },
    {
      "processDate": 1747046644948,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046654081,
      "type": "speechToText",
      "message": "transfer to secure",
      "party": "customer"
    },
    {
      "processDate": 1747046654105,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046664613,
      "type": "speechToText",
      "message": "transfer to secure",
      "party": "customer"
    },
    {
      "processDate": 1747046664640,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046675271,
      "type": "speechToText",
      "message": "cancer to sit",
      "party": "customer"
    },
    {
      "processDate": 1747046675296,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046682096,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046691230,
      "type": "speechToText",
      "message": "we look transfer",
      "party": "customer"
    },
    {
      "processDate": 1747046691260,
      "type": "sayTTS",
      "message": "Talebinizi anlayamadım. Lütfen tekrar edin.",
      "party": "agent"
    },
    {
      "processDate": 1747046700407,
      "type": "speechToText",
      "message": "change skill",
      "party": "customer"
    },
    {
      "processDate": 1747046708960,
      "type": "transfer",
      "message": "Call transfered to 7000",
      "party": "agent"
    }
  ]
}

Expected LLM Response

• nextActions: Array of actions

• action: Type of the expected action. Result actions (like queue, hangup, listen, etc.) need to be last action in the array.

Example,

{
    "nextActions": [
        {
            "action": "TTS",
            "message" : "{{MESSAGE}}",
            "providerId": "{{PROVIDERID(Optional}}"
        },
        {
            "action": "announce",
            "announceId": "{{ANNOUNCEID}}"
        },
        {
            "action": "changelanguage",
            "languageCode": "{{LANGUAGECODE(BCP-47)}}"
        },
        {
            "action": "setvariable",
            "variables": {
                "var1": "value1",
                "var2": "value2"
            }
        },
        {
            "action": "setheader",
            "headers": {
                "name1": "value1",
                "name2": "value2"
            }
        },
        {
            "action": "queue",
            "skill": "{{SKILL}}"
        },
        {
            "action": "hangup"
        },
        {
            "action": "listen"
        }
    ]
}