# Overview
**Sunbird AI Assistant** enables building Chat Bot based solutions powered by GenAI, that can provide easy access to timely, relevant and contextual information and suggestions to user queries. As part of Sunbird, it is a Digital Public Good (DPG) designed based on micro-services architecture and open for anyone to use.
The AI Assistant provides ability to understand and respond to queries in natural language (can support multiple languages). It analyses the overall context instead of just searching for exact keywords, leading to more precise and contextual responses.
The AI Assistant differs from generic AI chat bots like ChatGPT by its ability to train the bots on specific set of contents based on which the responses are provided. This would make the responses trustworthy and a lot more relevant to the use cases being implemented.
As a DPG, it enables the following:
1. By being platform independent, DPG based solutions bring in the value of **eliminating vendor or product lock-in**. This is very important for long term sustainability of the solution.
2. Provides more **choices for implementation** and hence can enable **optimized solutions** for a given scenario.
3. Through open specifications and standards, it enables **interoperability**. This helps in easy stitching together of multiple solutions creating **multiplier value**.
4. Open license and an open ecosystem around it, enables easy **evolvability**. The DPG can adapt and evolve over time, allowing for **continuous updates**. It would invite others to join and build solutions, thereby **enhancing its value proposition** and ensuring **ongoing relevance** and usefulness.
5. Apart from software, it enables creating a **common knowledge base** as part of the DPG in terms of guidelines, best practices and processes in rolling out solutions.
6. The open source software also **reduces initial time and cost** of creating solutions.
A national scale adoption of Sunbird AI Assistant is [e-Jaadui Pitara](/functional-overview/use-cases/e-jaadui-pitara) by NCERT.
# Functional Overview
Following sections provide the functional overview of Sunbird AI Assistant:
1. [The Problem](/functional-overview/the-problem)
2. [The Solution](/functional-overview/the-solution)
3. [Use cases](/functional-overview/use-cases)
4. [Capabilities](/functional-overview/capabilities)
# The Problem
# The Solution
# Use Cases
Following are a set of use cases that AI Assistant can enable:
* A farmer getting information about multiple govt. schemes and programs
* A parent getting assistance on handling their child with special needs
* A teacher getting assistance on using a new teaching technique in her class
* A social worker getting suggestions on how she can help a victim of specific type of domestic violence
* A healthcare professional getting information about preventing a new virus
* A citizen getting help about available Citizen Services
A Smart Assistant solution to Teachers, Parents and Caregivers in the space of early education has been rolled out as [e-Jaadui Pitara](/functional-overview/use-cases/e-jaadui-pitara) by NCERT.
# e-Jaadui Pitara
e-Jaadui Pitara is a transformative initiative of Ministry of Education, Government of India, rolled out by NCERT for Foundational Stage learning.
Following are the objectives of e-Jaadui Pitara:
1. Accelerate and amplify the awareness, reach, and impact of the transformative idea of Jaadui Pitara as a symbol of NCF-FS
* Increase awareness of Jaadui Pitara among teachers, parents and communities.
* Democratise access to content to overcome the limitations of the physical Jaadui Pitara.
2. Digital technologies complement the physical Jaadui Pitara.
* Leverage multiple channels- e.g. computers, smartphones, feature phones, television and radio.
* Leverage the potential of generative AI for relevant content creation and for language translation.
* Leverage existing digital infrastructure to enable and empower the local ecosystem to participate and create content.
e-Jaadui Pitara consists of the following applications:
1. Mobile App
2. AI Bots - accessible through Mobile App, Telegram and WhatsApp
3. IVRS
4. Data and Dashboards
5. Website
{% embed url="" %}
# Capabilities
## Smart Assistant Bot(s)
* Different bots for different purposes, powered by GenAI, can be created.
* The bots can
* Answer user queries by retrieving information from given set of contents. The answers can provide information as well as suggest solutions to the queries.
* Remember the message history (up to a configurable number of messages) and can respond based on the context of the conversation.
* Create stories for specific targeted users (like Indian children up to age 8) with a given context (like situation, character names etc.)
* The bots can be trained on a set of contents - makes it trustworthy.
* The bots can be accessed through multiple channels like WhatsApp, Telegram, Mobile/Web App etc.
* Bots can support multi-lingual and voice based communication by plugging in a third-party language service that provides translation, text to speech, speech to text capabilities for the required languages.
**Note** that following are **NOT** currently supported by Smart Assistant Bots
1. Answer for every question is always generated afresh. There is no caching of the question and answers. Hence, even if the same question is asked more than once, each time GenAI service is used to generate answer and hence may get different answers for the same question.
2. The AI Assistant responses can only provide required information and suggestions based on the trained content. It cannot strike a general conversation with the user.
3. User persona can only be broadly defined, such as - a student, a parent, a farmer etc. It is not be possible to be more specific like - users between 13 to 18 years old, users between 18-30 years old etc. Responses can also be not personalized for each user.
4. Considering the nature of GenAI, smart assistant bots are primarily targeted for adults only. Any use of the bots by children has to be strictly under an adult guidance.
## Reference Mobile and Web (PWA) apps
* Access and play a curated set of content (videos, documents)
* Create own playlists of content for easy access
* Access the Smart Assistant Bots
## Dashboard
Can create dashboard that shows usage metrics - such as total messages for different bots, number of plays etc.
**Note**: Smart Assistant Bots or Apps do not require any user login. They do not store any personally identifiable information.
# Technical Overview
Sunbird AI Assistant can be used to create solutions that help citizens access just-in-time, context-relevant information from trusted sources.
To get the functional overview and possible use cases, refer to [Functional Overview](/functional-overview).
The Sunbird AI Assistant primarily consists of an AI-driven backend bot service and services to integrate with Telegram and WhatsApp that can be used to provide front-end access to the bot service.
This section provides a technical overview of Sunbird AI Assistant. It provides the details of the technology stack, high-level components, and APIs. It also provides links to the source code.
# Architecture
Below is a high-level diagram of the various components powering the AI Assistant.
## Components
#### sakhi-api-service
This is the backend service that takes a question and generates a relevant answer using the RAG mechanism. The answers are based on the information in a set of ingested documents. The service has to be configured with the specific marqo indexes to retrieve the sections of relevant documents and the GenAI prompts to generate a response in the required form.
Code:
{% embed url="" %}
#### Ingest-Documents Script
Document ingestion into the Marqo vector database is done through the below standalone python script.
{% embed url="" %}
Details on how to run is available under “To ingest data to marqo” section in the below readme file:
{% embed url="" %}
#### sakhi-telegram-unified-service
This is a webhook service that registers with Telegram Service to oer the bot flow in the Telegram client. This doesn’t have any APIs exposed to the consumers. The calls to this service are delegated through the Telegram Service.
Code:
{% embed url="" %}
#### whatsapp-bot
This is a webhook service that registers with WhatsApp Gupshup Service to oer the bot flow in the WhatsApp client. This doesn’t have any APIs exposed to the consumers. The calls to this service are delegated through the WhatsApp Gupshup Service.
Code:
{% embed url="" %}
# Technology Stack
Below is the technology stack used for the implementation of the AI Assistant. AI Assistant primarily uses the RAG mechanism, powered by Marqo (for retrieval) and GenAI services to generate answers for the user's questions.
**Front End**
* Telegram (a Telegram Bot)
* WhatsApp (a WhatsApp business number)
**Backend**
* Python - Programming Language
* FastAPI - Web framework for building APIs with Python
* Marqo - Marqo is an end-to-end vector search engine. Vector generation, storage
and retrieval are handled out of the box through API.
* Cloud Storage Services (Supported are OCI Object Storage & AWS Buckets)
* Frameworks
* LangChain - Framework to simplify the creation of applications using large language models.
* LlamaIndex - Flexible data framework for connecting custom data sources to large language models.
* For Telegram:
* Telegram Service Wrapper - Webhook to receive the request from the user
through a Telegram Bot and to respond with answers. The webhook is
registered with the Telegram Service Provider.
* Redis - In-memory data store for holding users language and bot selection
* For WhatsApp GupShup provider:
* WhatsApp Service Wrapper - Webhook to receive the request from the user and to respond with answers. The webhook is registered with the WhatsApp Service Provider, which in this implementation uses Gupshup.
* PostgreSQL - Used for storing the user's session, language selected and bot selection for WhatsApp.
### External Services
* GenAI Services - LLM for,
* Text Generation
* Supported ones are,
* OpenAI (GPT4) with Moderation
* Azure OpenAI (GPT4) with Moderation
* Language Services for,
* Language Translation
* Speech to Text
* Text to Speech
* Supported ones are,
* Bhashini Dhruva
* EkStep Dhruva
* Google Language Services
* Telemetry - Log events are getting captured for
* api\_access
* api\_call
### Other Tools / Services
* Docker
* Cloud Infrastructure - Supported ones are,
* OCI
* AWS
### Dependencies
List of external libraries used:
# Get Started with AI Assistant
### Quick-links to start with AI Assistant development
[**Key Steps to role out an AI Assistant Solution**](/get-started-with-ai-assistant/key-steps-to-role-out-an-ai-assistant-solution)
#### [Pre-requisites](/get-started-with-ai-assistant/pre-requisites)
#### [Installation](/get-started-with-ai-assistant/installation)
#### [Data Ingestion Process](/get-started-with-ai-assistant/data-ingestion-process)
#### [Configuration](/get-started-with-ai-assistant/configuration)
#### [APIs](/get-started-with-ai-assistant/apis)
# Key Steps to role out an AI Assistant Solution
Implementing a successful AI assistant solution requires careful planning and execution. Follow the below steps:
1. Identify the cloud service provider in which the AI Assistant Services needs to be installed.
2. Identify the Language AI Services and GenAI services to be used and procure licenses if any.
3. Identify the Sources of Information to be ingested and get access to it.
* Types of documents to be ingested (documents, videos, and audio)
* Optimizing or fine-tuning the ingestion process to work with the supplied information.
* Different contexts/personas that will be using the data and the number of indexes to be created.
4. Fine-tune the retrieval configurations as appropriate
5. Optimize and fine-tune the GenAI prompts based on the use case
* Prompt configuration for different contexts/personas
6. If the client interface includes Telegram, creation of the bot and configuring it
7. If the client interface includes Mobile App, developing and configuring the mobile app as appropriate.
8. If the client interface includes WhatsApp,
* Identify the service provider and develop service as per the provider
* Sunbird AI Assistant comes with a default interface for Gupshup. If you have an existing service, then it can be used and configured to the account
# Pre-requisites
### Pre-requisites
Before running installation, ensure the following:
1. Have a cloud account ready with administrative access and have the secret access key and access key ID handy.
2. Administrative credentials details will be required and to be provided while running the installation script.
3. In case the bot service needs to be enabled through Telegram, you will need to create a telegram bot and secure public domain to configure webhook URL and add the details for the bot in the configuration (global-values.yaml) file.
4. Refer to configuration (global-values.yaml) file and update each environment variable with the required values.
5. Register the domain, add the TXT records in DNS mappings and generate the certificates, private key, domain and add that in the configuration (global-values.yaml) file.
6. Please note, global-cloud-values.yaml will be auto-generated during installation with terraform in the path ./helmcharts. if you have any file name as global-cloud-values.yml file in the same path please remove it.
# Installation
## Installer
Here is the link for the [installer](https://github.com/Sunbird-AIAssistant/ai-bot-installer).
Once the pre-requisites are satisfied, run the installation script (install\_on\_aws.sh).
{% hint style="info" %}
Check for your cloud specific installation script. The above one is for the AWS which is the default one.
{% endhint %}
## Load Balancer Configuration
After completing the provisioning process, log in to your cloud account. Then, map the load balancer DNS to your domain by updating the CNAME records in the DNS settings.
## WebHook for Telegram Client
To set the BOT webhook, use the below curl:
```sh
curl --location 'https://api.telegram.org/bot{BOT_TOKEN_HERE}/setWebhook'
--header 'Content-Type: application/json'
--data '{"url": "{DOMAIN_URL_HERE}/api/webhook/telegram"}'
```
{% hint style="info" %}
Replace the `{BOT_TOKEN_HERE}` and `{DOMAIN_URL_HERE}/api/webhook/telegram` with the actual values.
{% endhint %}
## Accessing the Telegram Wrapper Service
To access the telegram-unified service externally, hit the URL: `https:///api/webhook/telegram`
### Manual Installation Steps:
1. [Sakhi API Service](https://github.com/Sunbird-AIAssistant/sakhi-api-service?tab=readme-ov-file#getting-started)
2. [Telegram Service ](https://github.com/Sunbird-AIAssistant/sakhi-telegram-unified-service?tab=readme-ov-file#sakhi-telegram-unified-bot-service)
3. [WhatsApp Service](https://github.com/Sunbird-AIAssistant/whatsapp-bot?tab=readme-ov-file#getting-started)
# Data Ingestion Process
After completing the installation, follow these steps to index all contents related to a specific use case:
### Release 3.0.0
1. Install Python on the machine where the files need to be ingested.
2. Clone Git Repo from [https://github.com/Sunbird-AIAssistant/sakhi-api-service](https://www.google.com/url?q=https://github.com/Sunbird-AIAssistant/sakhi-api-service\&sa=D\&source=editors\&ust=1712142234872409\&usg=AOvVaw23d0wT4lcVdCV9qqQUVOq8).
3. Go to the root directory and update the `.env` file with the necessary [vector store configuration](/components/sakhi-api-service/environment-variables) values.
4. Run the following:
{% code overflow="wrap" %}
```python
Step 1: pip install -r requirements-dev.txt
Step 2: python3 index_documents.py --folder_path= --fresh_index --chunk_size=1024 --chunk_overlap=100
# --fresh_index: Create a new index from scratch.
# --chunk_size: Divide the documents into chunks of 1024 characters. Default: 1024
# --chunk_overlap: Overlap each chunk by 100 characters for context. Default: 100
```
{% endcode %}
### Before Release 3.0.0
1. Install Python on the machine where the files need to be ingested.
2. Place the files to be indexed in a folder on the machine.
3. Download index\_documents.py and requirements-dev.txt file from [https://github.com/Sunbird-AIAssistant/sakhi-api-service](https://www.google.com/url?q=https://github.com/Sunbird-AIAssistant/sakhi-api-service\&sa=D\&source=editors\&ust=1712142234872409\&usg=AOvVaw23d0wT4lcVdCV9qqQUVOq8)
4. Run the following:
**Notes**:
1. Please run the commands via screen background, as it will take a couple of hours to run
2. “--fresh\_index” is to be used when you run the indexing for the first time or delete the existing index and freshly index it. If you want to append new files to the existing index, run it without --fresh\_index
3. For running without --fresh\_index, ensure your new files are kept in a new folder and the --folder\_path is pointed to only the new files.
# Configuration
### Configurations Configuring the Services
#### Functional configuration
#### sakhi-api-service
Sakhi Service enables configuration such as defining the indexes from where the documents are to be retrieved for a given context (parent, teacher, etc.) from Marqo, defining the bot behaviour through prompt, etc.
A detailed set of configurations is captured here:
{% embed url="" %}
**Configuration file:**
{% embed url="" %}
#### **sakhi-telegram-unified-service**
Sakhi Telegram Service can be configured with the set of welcome messages to be shown to the user on Telegram, languages supported, set of bots supported (one bot for each context/persona) etc.
**Configuration file:**
{% embed url="" %}
{% hint style="info" %}
**Note**: As currently these functional configuration files are inside the image that is created, the changes either need to be added when the image is built or needs to be overridden when the container is created.
{% endhint %}
### System configurations
The System configuration is specified in the respective repository as given below.
{% embed url="" %}
**sakhi-api-service**\
Refer to point#5 of the instructions given
{% embed url="" %}
**sakhi-telegram-unified-service**\
Refer to point#4 of the instructions given
{% embed url="" %}
**whatsapp-bot**
{% embed url="" %}
# APIs
## sakhi-api-service
It has the below APIs:
### v1/query
API is used for getting answers for user questions.
API details are available in this link
{% embed url="" %}
### **v1/chat**
API is used in engaging conversation with the user, users chat history is preserved.
API details are available in this link
{% embed url="" %}
# Bot Creation 101
Here is the list of steps to install and configure the services:
1. [Install the services](/get-started-with-ai-assistant/installation)
2. [Ingest Data and create an Index](/get-started-with-ai-assistant/data-ingestion-process)
3. [Configure a new bot in the Sakhi API service](https://drive.google.com/file/d/1OvNvmJawDpgSW3Xkunqrkc6fY53ZIlXR/view)
4. Integrate the Sakhi API service with a front-end app:
1. [WhatsApp](/faqs#how-to-configure-a-new-bot-in-whatsapp)
2. [Telegram](/faqs#how-to-configure-a-new-bot-in-telegram)
# Components
# Sakhi API Service
# Environment Variables
Sakhi API service supports different environment variables to configure your instance. You can specify the following variables in the `.env` file inside the `root` folder. Refer to the [**.env.example**](https://github.com/Sunbird-AIAssistant/sakhi-api-service) file.
| Variable | Description | Type | Default |
| ------------------------ | ------------------------------------------------------ | ------------------------------------- | ------------ |
| SERVICE\_ENVIRONMENT | Name of the application instance | String | `dev` |
| LOG\_LEVEL | It's used to define the level of logs to be captured. | Enum String: `error`, `info`, `debug` | `info` |
| CONFIG\_INI\_PATH | Path of config.ini file | String | `config.ini` |
| REDIS\_HOST | Redis host URL | String | `localhost` |
| REDIS\_PORT | Redis port number | Number | `6379` |
| REDIS\_DB | Redis index | Number | `0` |
| TELEMETRY\_ENDPOINT\_URL | Telemetry service host URL | String | - |
| TELEMETRY\_LOG\_ENABLED | Used to enable/disable telemetry logging. | Enum String: `true`, `false` | `true` |
### LLM
A chat model is a language model that uses chat messages as inputs and returns chat messages as outputs.\
Sakhi API service currently supports 3 LLM types:
1. [OpenAI](#openai)
2. [Azure OpenAI](#azure-openai)
3. [Ollama](#ollama)
These LLMs can be configured with the following env variables:
#### OpenAI
```markdown
LLM_TYPE=openai
OPENAI_API_KEY=
GPT_MODEL=
```
#### **Azure OpenAI**
```markdown
LLM_TYPE=azure
AZURE_OPENAI_ENDPOINT=
AZURE_OPENAI_API_KEY=
OPENAI_API_VERSION=
AZURE_MODEL=
```
#### **Ollama**
```markdown
LLM_TYPE=ollama
OLLAMA_API_ENDPOINT=
LLM_MODEL=
```
{% hint style="info" %}
If no env variables are specified, the service will throw the runtime exception.
{% endhint %}
For more information, Please refer to [LLM](/components/sakhi-api-service/pluggability-of-llm-chat-model) page.
### Storage
Storage is used for storing audio files when the user wants output as audio. Users should specify a configuration option, like BUCKET\_TYPE, to choose between different cloud provider services such as AWS S3, OCI, or GCP.
Here's the list of storage available to use in the Sakhi API Service:
1. [OCI](#oci-oracle)
2. [AWS](#aws-amazon)
3. [GCP](#gcp-google)
#### OCI (Oracle)
```markdown
BUCKET_TYPE=oci
BUCKET_ENDPOINT_URL=
BUCKET_REGION_NAME=
BUCKET_NAME=
BUCKET_SECRET_ACCESS_KEY=
BUCKET_ACCESS_KEY_ID=
```
#### **AWS (Amazon)**
```markdown
BUCKET_TYPE=aws
BUCKET_REGION_NAME=
BUCKET_NAME=
BUCKET_SECRET_ACCESS_KEY=
BUCKET_ACCESS_KEY_ID=
```
#### **GCP (Google)**
```markdown
BUCKET_TYPE=gcp
BUCKET_NAME=
GCP_CONFIG_PATH=
```
{% hint style="info" %}
If no env variables are specified, the service will throw the runtime exception.
{% endhint %}
### Translation
Sakhi API service currently supports 3 translation services and can be configured with the following env variables:
1. [Bhashini Dhruva](#bhashini-dhruva)
2. [Ekstep Dhruva](#ekstep-dhruva)
3. [Google ](#google)
#### **Bhashini Dhruva**
```markdown
TRANSLATION_TYPE=bhashini
BHASHINI_ENDPOINT_URL=
BHASHINI_API_KEY=
```
#### **Ekstep Dhruva**
```markdown
TRANSLATION_TYPE=dhruva
BHASHINI_ENDPOINT_URL=
BHASHINI_API_KEY=
```
#### **Google**
```markdown
TRANSLATION_TYPE=google
GCP_CONFIG_PATH=
```
{% hint style="info" %}
If no env variables are specified, the service will throw the runtime exception.
{% endhint %}
### Vector Store
A vector store or vector database refers to a type of database system that specializes in storing and retrieving high-dimensional numerical vectors. These stores are designed for efficient management and indexing of vectors, enabling fast similarity searches.
Here is the list of vector stores/databases available to use in the Sakhi API Service:
1. [Marqo](#marqo)
#### **Marqo**
```shellscript
VECTOR_STORE_TYPE=marqo
VECTOR_STORE_ENDPOINT=http://localhost:8882
EMBEDDING_MODEL=flax-sentence-embeddings/all_datasets_v4_mpnet-base
VECTOR_COLLECTION_NAME=
```
{% hint style="info" %}
If no env variables are specified, the service will throw the runtime exception.
{% endhint %}
For more information, Please refer to [Vector Store](/components/sakhi-api-service/pluggability-of-vector-store) page.
# Pluggability of LLM Chat Model
In this guide, we will learn how to plugin a new chat model client. The plugin needs to extend/adhere to `BaseChatModel` of Langchain framework. Hence, you can refer to the [langchain's LLM chat model integration page](https://python.langchain.com/v0.1/docs/integrations/chat/).
If you don't find your LLM chat model client on the above link, please refer to[ this link](https://python.langchain.com/v0.1/docs/modules/model_io/chat/custom_chat_model/) to create your custom LLM chat client by extending `BaseChatModel`
A chat model is a language model that uses chat [messages](https://python.langchain.com/v0.1/docs/modules/model_io/chat/custom_chat_model/#messages) as inputs and returns chat messages as outputs.
You can seamlessly leverage it within existing Sakhi API workflows with minimal code changes.
### Interface
The `BaseChatClient` class has the below method to implement and returns an instance of [`BaseChatModel`](https://python.langchain.com/v0.1/docs/modules/model_io/chat/custom_chat_model/#base-chat-model) representing the specific Chat model client.
| Method/Property | Description | Required/Optional |
| --------------- | ------------------------------------------------------------------------------------------------------------------ | ----------------- |
| get\_client | Takes LLM model name as string and required additional arguments, and returns an instance of specific Chat model. | Required |
### Implementation
Let's say we create a `YourChatClient` class inheriting from `BaseChatClient` using the chat model client provided by Langchain (in the link mentioned above). You must implement `get_client` method to instantiate `YourChatClient` with the specified model and additional arguments.
Add the necessary environment variables to the `.env` file. `LLM_TYPE` is to be defined mandatorily. LLM chat model specific environment variables (Ex: API\_KEY) are also to be mentioned.
{% code title="env" %}
```
LLM_TYPE=
```
{% endcode %}
Now go to the `llm`folder and create a file named: `your_chat_model_name.py`
{% code title="your\_chat\_model\_name.py" %}
```python
import os
from typing import Any
from langchain.chat_models import
from llm.base import BaseChatClient
class YourChatClient(BaseChatClient):
"""
This class provides a chat interface for interacting with .
"""
def get_client(self, model:str, **kwargs: Any) -> :
"""
This method creates and returns a instance.
Args:
model (str, optional): chat model to use.
**kwargs: Additional arguments to be passed to the constructor.
Returns:
An instance of the class.
"""
return LangchainChatModelClient(model=model, **kwargs)
```
{% endcode %}
{% hint style="info" %}
**``** should be one of the LLM chat model clients mentioned [here](https://python.langchain.com/v0.1/docs/integrations/chat/).
{% endhint %}
Go to the LLM folder and update `__init__.py` with the module lookup entry for `YourChatClient`.
{% code title="**init**.py" %}
```python
_module_lookup = {
...,
"YourChatClient": "llm."
}
```
{% endcode %}
Modify `env_manager.py` to import `YourChatClient` and add a mapping for `YourChatClient` in the `self.indexes` dictionary.
{% code title="env\_manager.py" %}
```python
from llm import (
...,
YourChatClient
)
```
{% endcode %}
This setup ensures that `YourChatClient` can be instantiated based on specific environment variables, effectively integrating it into the environment management system. The `self.indexes` the dictionary now includes a mapping where `` corresponds to the `YourChatClient` class, and uses `"LLM_TYPE"` as the environment key.
### Example usage
```python
from env_manager import llm_class
```
{% code overflow="wrap" %}
```python
llm = llm_class.get_client(model=)
result = llm.invoke("Tell me a game using two sticks")
print(result.content)
#The game you can play using two sticks is called Gilli Danda. This game originated in India and requires two sticks. The smaller stick should be an oval-shaped wooden piece known as Gilli and the longer stick is known as danda. The player needs to use the danda to hit the Gilli at the raised end, which then flips.
```
{% endcode %}
For more examples, please refer to [Langchain](https://python.langchain.com/v0.1/docs/integrations/chat/google_generative_ai/#streaming-and-batching).
# Pluggability of Cloud Storage
In this guide, we will learn how to plugin a new cloud storage wrapper by extending the `BaseStorageClass` interface.
You can seamlessly leverage it within existing Sakhi API workflows with minimal code changes.
### Integration
Cloud storage is used for storing the response generated from LLM which is then converted to **audio format** using a translation util. Link of the audio response files will be used by clients connecting to sakhi-api-service for playing the response.
### BaseStorageClass
This `BaseStorageClass` class has below methods which are to be implemented mandatorily.
| Method/Property | Description | Required/Optional |
| --------------------- | ------------------------------------------------------------------ | ----------------- |
| \_\_init\_\_ | Used to instantiate the storage account client | Required |
| upload\_to\_storage | Takes name of the file to be uploaded | Required |
| generate\_public\_url | Takes name of the file for which the public url is to be generated | Required |
### Implementation
Let's say we create a `YourStorageClass` class inheriting from `BaseStorageClass.` The class needs to implement the following methods \_\_init\_\_, upload\_to\_storage and generate\_public\_url.
Add the necessary environment variables to `.env` file. `BUCKET_TYPE` is to be defined mandatorily.
```
BUCKET_TYPE=your_cloud
```
Now go to the 'storage' folder and create your plugin: `your_cloud.py`
{% code title="your\_cloud.py" %}
```python
from storage.base import BaseStorageClass
class YourBucketClass(BaseStorageClass):
def __init__(self):
your_client = # instantiate your client
super().__init__(your_client)
def upload_to_storage(self, file_name: str, object_name: Optional[str] = None) -> bool:
# code to upload file to your cloud storage
return True
def generate_public_url(self, object_name: str) -> Union[tuple[str, None], tuple[None, str]]:
try:
public_url = # you code to fetch public url of the uploaded file
return public_url, None
except Exception as e:
logger.error(f"Exception Preparing public URL: {e}", exc_info=True)
return None, "Error while generating public URL"
```
{% endcode %}
Go to the 'storage' folder and update `__init__.py` with the module lookup entry for "YourBucketClass" and also for TYPE\_CHECKING.
```python
if TYPE_CHECKING:
...
from storage.your_cloud import (
YourBucketClass
)
_module_lookup = {
...,
"YourBucketClass": "storage.your_cloud"
}
```
Modify `env_manager.py` to import `YourStorageClass` and add a mapping for `YourStorageClass` in the `self.indexes` dictionary.
```python
from storage import (
...,
YourBucketClass
)
```
This setup ensures that `YourBucketClass` can be instantiated based on specific environment variables, effectively integrating it into the environment management system. The `self.indexes` the dictionary now includes a mapping where `"your_cloud"` corresponds to the `"YourBucketClass"` class and uses `"BUCKET_TYPE"` as the environment key.
# Pluggability of Transaltion service
In this guide, we will learn how to plugin a new translation service by extending the `BaseTranslationClass` interface.
You can seamlessly leverage it within existing Sakhi API workflows with minimal code changes.
### Integration
Translation service is used for supporting multiple languages in Sakhi-api-service.
### BaseTranslationClass
This `BaseTranslationClass` class has below methods which are to be implemented mandatorily.
| Method/Property | Description | Required/Optional |
| ---------------- | ------------------------------------------------------------ | ----------------- |
| \_\_init\_\_ | Method to initialise or define necessary variables | Optional |
| translate\_text | This method translates a text string to another language. | Required |
| text\_to\_speech | This method converts text to speech in a specified language. | Required |
| speech\_to\_text | This method converts speech from an audio file to text. | Required |
### Implementation
Let's say we create a `YourTranslationClass` class inheriting from `BaseTranslationClass.` The class needs to implement the following methods translate\_text, text\_to\_speech and speech\_to\_text.
Add the necessary environment variables to `.env` file. `TRANSLATION_TYPE` is to be defined mandatorily.
```
TRANSLATION_TYPE=your_translation_service
```
Now go to the 'translation' folder and create your plugin '`your_translation_service.py`'. Provide the implementation for the abstract methods.
{% code title="your\_translation\_service.py" %}
```python
from translation.base import BaseTranslationClass
class YourTranslationClass(BaseTranslationClass):
def translate_text(self, text: str, source: str, destination: str) -> str:
# code to translate text
return translated_text
def text_to_speech(self, language: str, text: str) -> Any:
return base_64_decoded_audio
def speech_to_text(self, audio_file: Any, input_language: str) -> str:
return transcripted_text
```
{% endcode %}
Go to the 'translation' folder and update `__init__.py` with the module lookup entry for "YourTranslationClass" and also for TYPE\_CHECKING.
```python
if TYPE_CHECKING:
...
from translation.your_translation_service import (
YourTranslationClass
)
_module_lookup = {
...,
"YourTranslationClass": "translation.your_translation_service"
}
```
Modify `env_manager.py` to import `YourTranslationClass` and add a mapping for `YourTranslationClass` in the `self.indexes` dictionary.
```python
from translation import (
...,
YourTranslationClass
)
```
This setup ensures that YourTranslationClass can be instantiated based on specific environment variables, effectively integrating it into the environment management system. The `self.indexes` the dictionary now includes a mapping where `"your_translation_service"` corresponds to the `"YourTranslationClass"` class and uses `"TRANSLATION_TYPE"` as the environment key.
# Pluggability of Vector Store
This document outlines the steps for creating a custom vector store class that inherits from the provided `BaseVectorStore` interface.
As we know, many LLM applications leverage vector stores to efficiently retrieve relevant information for generating responses. A vector store acts as a specialized database designed to store and retrieve high-dimensional vector representations of data, such as documents. These vectors capture the semantic meaning and relationships between concepts in the data.
### Understanding the Base Interface:
To create your own vector store, you need to extend the `BaseVectorStore` class and implement the following methods:
Method/Property
Description
Required/Optional
get_client
An abstract method that subclasses must implement to retrieve the client object used to interact with the specific vector store backend (e.g., Pinecone, Faiss).
Required
chunk_list
Helper function that splits a document list into batches of a specified size.
Optional
add_documents
An abstract method for adding documents to the vector store. Subclasses implement their specific logic for document insertion.
Required
similarity_search_with_score
An abstract method for performing similarity search on the vector store. Subclasses implement their specific logic for retrieving similar documents and scores based on a query string.
Required
### Implementation
Let's implement a custom vector store `YourVectorStoreClass` inheriting from `BaseVectorStore` that adds documents and returns relevant documents whose text contains the semantic meaning in the user query.
Create a new file named `.py` in the `vectorstores` folder to define the `YourVectorStoreClass` .
Here's a brief overview of how you'll implement the `YourVectorStoreClass`:
```python
from vectorstores.base import BaseVectorStore
class YourVectorStoreClass(BaseVectorStore):
def get_client(self):
# Implement logic to retrieve the client for your vector store backend
pass
def chunk_list(self, document_list, chunk_size):
# Optional: Implement logic to chunk documents into batches
pass
def add_documents(self, documents, fresh_collection: bool = False):
# Implement logic to generate embedding, add documents to your vector store and return document IDs
pass
def similarity_search_with_score(self, query, collection_name: str, k: int = 20):
#Args:
#query: The query string to search for.
#collection_name: Name of the collection within the vector store to search in.
#k: The maximum number of documents to fetch from the vector store (default: 20).
# Implement logic to perform a similarity search and return results with scores
pass
```
In this example, we implement the `YourVectorStoreClass`, which provides concrete implementations for the abstract methods defined in `BaseVectorStore`.
* **get\_client**: Retrieves the specific client used to interact with the vector store backend.
* **chunk\_list**: Optionally, splits documents into chunks for more manageable processing.
* **add\_documents**: Adds documents to the vector store.
* **similarity\_search\_with\_score**: Performs a similarity search and returns documents along with their similarity scores based on the query.
Go to the `vectorstores` folder and update `__init__.py` with the module lookup entry for `YourVectorStoreClass`.
{% code title="**init**.py" %}
```python
_module_lookup = {
...,
"YourVectorStoreClass": "llm."
}
```
{% endcode %}
Modify `env_manager.py` to import `YourVectorStoreClass` and add a mapping in the `self.indexes` dictionary.
{% code title="env\_manager.py" %}
```python
from vectorstores import (
...,
YourVectorStoreClass
)
```
{% endcode %}
This setup ensures that `YourVectorStoreClass` can be instantiated based on specific environment variables, effectively integrating it into the environment management system. The `self.indexes` the dictionary now includes a mapping where **`customVector`** corresponds to the `YourVectorStoreClass`, and uses `VECTOR_STORE_TYPE` as the environment key.
### Configuration
Configure your environment variables in the `.env` file for connecting to the vector store.
{% code title=".env" %}
```
VECTOR_STORE_TYPE=
VECTOR_STORE_ENDPOINT=
EMBEDDING_MODEL=
VECTOR_COLLECTION_NAME=
```
{% endcode %}
### Example Usage
Here is an example of how to add and query documents using the `vectorstore_class`.
#### Adding/Appending Documents
#### Querying Documents
```python
query = "Test one"
collection_name = "test"
documents = vectorstore_class.similarity_search_with_score(query, collection_name, k=20)
print(documents)
# Expected output:
# [
# (Document(page_content="Test one", metadata={}), 0.95),
# (Document(page_content="Test two", metadata={}), 0.65),
# (Document(page_content="Test three", metadata={}), 0.61)
# ]
```
By following this structure, you can efficiently interact with your custom vector store, adding and querying documents as needed.
# Release Notes
# Release Convention
The release packages of Sunbird AI Assistant follow the Semantic Versioning convention.
The release packages are available through the [GitHub packages](https://github.com/orgs/Sunbird-AIAssistant/packages) under the respective repositories.
# 3.0.0 (Latest)
| Release Version | Date |
| --------------- | ----------- |
| 3.0.0 | 15-May-2024 |
### Overview
#### Sunbird AI Assistant Packages: Release 3.0.0
We are excited to announce the release of **Sunbird AI Assistant** packages, version 3.0.0. Users are encouraged to upgrade to the latest version to benefit from enhanced features, improved stability, and comprehensive support.
### New Features
It contains the following features
* **sakhi-api-service**
* Generalisation of code for integration with various LLMs. Adaptors are available for integrating with OpenAI (GPT4) , Azure OpenAI (GPT4) and Ollama
* Generalisation of code for integration with various Vector stores/databases. Adaptor is available for integrating with Marqo.
* Changes to environment variables
Old Variable
New Variable
Comments
MARQO_URL
deprecated with addition of below variables related to VectorDB generalisation.
VECTOR_STORE_TYPE
Vector database type. Example: 'marqo'
VECTOR_STORE_ENDPOINT
Vector DB connection URL
EMBEDDING_MODEL
Sentence Transformer model name. Please note that only sentence transformer models are supported as of now. Example: 'flax-sentence-embeddings/all_datasets_v4_mpnet-base'
VECTOR_COLLECTION_NAME
Index/Collection name to which data will be stored to
OPENAI_TYPE
LLM_TYPE
Name changed for generalising
{% hint style="info" %}
While indexing documents to VectorDB, previous command `python3 index_documents.py --marqo_url= --index_name= --folder_path= --fresh_index` has been now updated to `python3 index_documents.py --folder_path= --fresh_index`
{% endhint %}
### Release Tags: (GitHub Packages)
#### [sakhi-api-service](https://github.com/Sunbird-AIAssistant/sakhi-api-service/pkgs/container/sakhi-api-service/215892532?tag=3.0.0)
####
### **Bug Fixes**
### **Open/Known Bugs**
### **Breaking Changes**
* Renaming / Refactoring the environment variables
### **Installation**
* Installation steps are same as before, one thing to ensure before installation is to check the environment variables as it has undergone refactoring as part of this release.
### **Build Tags**
[**sakhi-api-service**](https://github.com/Sunbird-AIAssistant/sakhi-api-service/releases/tag/release-3.0.0)
# 2.0.0
| Release Version | Date |
| --------------- | ---- |
| 2.0.0 | |
### Overview
#### Sunbird AI Assistant Packages: Release 2.0.0
We are excited to announce the release of **Sunbird AI Assistant** packages, version 2.0.0. Users are encouraged to upgrade to the latest version to benefit from enhanced features, improved stability, and comprehensive support.
### New Features
It contains the following services
* **sakhi-api-service**
* Enabling the chat APIs for conversational history
* Generalising the language services to use either Dhruva Bhashini, Dhruva EkStep or Google Services
* Generalising the AI Services to use either OpenAI (GPT4) or Azure OpenAI (GPT4)
* Renaming audience\_type to context as part of generalisation
* Build fixes in the Dockerfile
* Changes to environment variables
Old Variable
New Variable
Comments
N/A
TRANSLATION_TYPE
New variable
N/A
BUCKET_TYPE
New variable
N/A
OPENAI_TYPE
New variable
N/A
REDIS_HOST
New variable
N/A
REDIS_DB
New variable
N/A
REDIS_PORT
New variable
N/A
TOP_DOCS_TO_FETCH
New variable
N/A
CONFIG_INI_PATH
New variable
OCI_ENDPOINT_URL
BUCKET_ENDPOINT_URL
Name changed for generalising
OCI_REGION_NAME
BUCKET_REGION_NAME
Name changed for generalising
OCI_BUCKET_NAME
BUCKET_NAME
Name changed for generalising
OCI_SECRET_ACCESS_KEY
BUCKET_SECRET_ACCESS_KEY
Name changed for generalising
OCI_ACCESS_KEY_ID
BUCKET_ACCESS_KEY_ID
Name changed for generalising
gpt_model
GPT_MODEL
Name changed for generalising
* **sakhi-telegram-unified-service**
* Enabling the chat feature as default option for Bot Interactions, that retains the conversation history
* Renaming audienceType to context as part of generalisation
* **whatsapp-bot**
* Enabling the chat feature as default option for Bot Interactions, that retains the conversation history
* Renaming audienceType to context as part of generalisation
### Release Tags: (GitHub Packages)
#### [sakhi-api-service](https://github.com/Sunbird-AIAssistant/sakhi-api-service/pkgs/container/sakhi-api-service/207278223?tag=2.0.0)
#### [sakhi-telegram-unified-service](https://github.com/Sunbird-AIAssistant/sakhi-telegram-unified-service/pkgs/container/sakhi-telegram-unified-service/207260776?tag=2.0.0)
#### [whatsapp-bot](https://github.com/Sunbird-AIAssistant/whatsapp-bot/pkgs/container/whatsapp-bot/207267846?tag=2.0.0)
### **Bug Fixes**
### **Open/Known Bugs**
### **Breaking Changes**
* Renaming / Refactoring the environment variables
* Renaming audienceType to context as part of generalisation
### **Installation**
* Installation steps are same as before, one thing to ensure before installation is to check the environment variables as it has undergone refactoring as part of this release.
### **Build Tags**
[**sakhi-api-service**](https://github.com/Sunbird-AIAssistant/sakhi-api-service/releases/tag/release-2.0.0)
[**sakhi-telegram-unified-service**](https://github.com/Sunbird-AIAssistant/sakhi-telegram-unified-service/releases/tag/release-2.0.0)
[**whatsapp-bot**](https://github.com/Sunbird-AIAssistant/whatsapp-bot/releases/tag/release-2.0.0)
# 1.0.0
| Release Version | Date |
| --------------- | ---- |
| 1.0.0 | |
### Overview
#### Sunbird AI Assistant Packages: Release 1.0.0
We are pleased to announce the official release of **Sunbird AI Assistant** packages, version 1.0.0. This version marks significant development efforts aimed at delivering an optimized and robust AI assistant experience. Users are encouraged to upgrade to the latest version to benefit from enhanced features, improved stability, and comprehensive support.
### New Features
It contains the following services:
* **sakhi-api-service**
* The base version of API Service for the Q\&A capability
* Document Ingestion process for indexing the documents
* **sakhi-telegram-unified-service**
* The base version of Telegram wrapper for the interaction with the Bots
* **whatsapp-bot**
* The base version of WhatsApp wrapper for the interaction with the Bots
### Release Tags: (GitHub Packages)
#### [sakhi-api-service](https://github.com/Sunbird-AIAssistant/sakhi-api-service/packages)
#### [sakhi-telegram-unified-service](https://github.com/Sunbird-AIAssistant/sakhi-telegram-unified-service/packages)
#### [whatsapp-bot](https://github.com/Sunbird-AIAssistant/whatsapp-bot/packages)
### **Bug Fixes**
### **Open/Known Bugs**
### **Installation**
### **Build Tags**
# Roadmap
Coming Soon...
# Contribution Guide
### Welcome Contributors
We're thrilled you're interested in contributing to the Sunbird AI Assistant Building Block! As an open-source project in a fast-paced field, we welcome all contributions, from new features and improved infrastructure to better documentation and bug fixes.
### Getting Started
#### New to the Project?
No worries! Get acquainted with our project by exploring the [documentation](https://ai-assistant.sunbird.org/).
#### Ways to Contribute
There are many ways to get involved and make a difference. Here are some common areas where people contribute:
* **Improve the Documentation:** Help us make the docs, including this one, even better! Your edits, suggestions, and additions are invaluable.
* **Code, Fix Bugs, and Improve Infrastructure:** We welcome your coding expertise! Help us write new features, fix bugs, or improve our infrastructure.
* **Integrate with Favourite Vendors or Tools:** Do you have a favourite tool or vendor you'd love to see integrated with Sunbird? Please share your knowledge and help us make it happen!
* **Answer Usage Questions and Discuss Issues:** Help us build a strong community by answering user questions, discussing issues, and providing valuable insights.
{% hint style="info" %}
It's essential to discuss significant contributions to our discussion forum before proceeding to ensure your work aligns with the project's goals and direction.
{% endhint %}
#### Discussions Forums
We have a [Discussions](https://github.com/orgs/Sunbird-AIAssistant/discussions/categories/q-a) page where users can ask usage questions, discuss design decisions, and propose new features.
If you can help answer questions, please do so! This will allow the maintainers to focus more on development and bug fixing.
### Core Contributions
We primarily accept contributions through pull requests. Here's what you need to know:
* **Fork the Repository**: Create a fork of our repository on GitHub.
* **Clone your Fork**: Clone your forked repository to your local machine.
* **Make Changes**: Make your changes and write clear commit messages.
* **Test your Changes**: Ensure your changes function properly and adhere to our coding standards.
* **Push to your Fork**: Push your changes to your forked repository.
* **Open a Pull Request**: Submit a pull request to our main repository.
{% hint style="info" %}
Important: By issuing a Pull Request, you agree to allow the project owners to license your work under the terms of the License.
{% endhint %}
#### Pull Request Checkpoints:
* **Testing**: All pull requests should include thorough testing to guarantee functionality and avoid breaking existing features.
* **Code Style**: Adhere to coding style guidelines to promote readability and maintainability for everyone.
* **Clear Description**: Provide a clear and concise description of your changes in the pull request. This will help reviewers understand your contribution.
# FAQs
Do you have a query, suggestion or bug you would like to discuss? Discuss in our [forum](https://github.com/orgs/Sunbird-AIAssistant/discussions).
What is Sunbird AI Assistant?
Sunbird AI Assistant is a building block of Sunbird that provides capabilities to build AI bot-related solutions. It is a DPG, a Digital Public Good, which is designed based on micro-services architecture and is open source.
What types of content can be ingested or indexed?
The following content format types are supported:
* Documents - PDF, DOCX, PPT, CSV, EPUB, TXT
* Videos - mp4
* Audios - mp3
* Images - JPEG, JPG, PNG
Note that all the contents should be in English language only.
What is the level of configurable customisation available?
Following are the different configurations supported:
1. GenAI service - Open AI, Azure Open AI
2. Language service - Bhashini, Google
3. Supported languages - This can be English and the set of Indian languages supported by Bhashini
4. Response types - Can be text, audio or both
5. Gen AI GPT Model value - Can be 3.5 or 4
6. Enable or disable telemetry events logging to Sunbird Telemetry service
How to add new documents to the index for AI bots to use?
Please check the data ingestion document: [Data Ingestion Process](/get-started-with-ai-assistant/data-ingestion-process)
How to change the Welcome message or other static messages of a bot (in Mobile App, Telegram and WhatsApp)?
The configuration that applies to the static messages of Telegram and WhatsApp is explained in the document “[Sunbird AI Assistant - Technical Overview](https://www.google.com/url?q=https://docs.google.com/document/d/1dEIAOHu2chVVZYDmqaOSXxHTLcQLmxEKFKcPnETz3pg/edit?usp%3Dsharing\&sa=D\&source=editors\&ust=1712137078204887\&usg=AOvVaw0Nf-dbDI8Zs4rPa7nEoA7y)” under the section “[Configurations](https://www.google.com/url?q=https://docs.google.com/document/d/1dEIAOHu2chVVZYDmqaOSXxHTLcQLmxEKFKcPnETz3pg/edit%23heading%3Dh.5l6owyvsr9io\&sa=D\&source=editors\&ust=1712137078205181\&usg=AOvVaw35F02p6IIj33E-8J58ci1Z)”.
If the page numbers of the sources given in the bot replies are not matching with the page numbers in the documents, what could be the issue?
This typically is seen when there are pages in the document ingested that don't carry a page number. In these cases, you can cross-check the content against the page number and remove these pages before ingestion so that the ingestion script can correctly tag the page number against the content.
If the bot replies “I am not currently trained with relevant documents to provide a specific answer for your question” - what could be the possible reasons? How to validate it?
This is an expected behavior when the Bot is not able to find a relevant answer to the user’s question. In such cases, it politely declines to answer by providing the message. You can also cross-check the content by using text searches to see if you are able to find any close matches. Additionally, you can cross-check the logs by engaging the operations team to check what information is retrieved from Marqo and check the relevance of the content.
How can I support more languages in the bots?
The configuration that enables the languages for Telegram and WhatsApp “request.supported\_lang\_codes” is explained in the document “[Sunbird AI Assistant - Technical Overview](https://www.google.com/url?q=https://docs.google.com/document/d/1dEIAOHu2chVVZYDmqaOSXxHTLcQLmxEKFKcPnETz3pg/edit?usp%3Dsharing\&sa=D\&source=editors\&ust=1712137078205985\&usg=AOvVaw2DUsMM44nlCjiH6mMATCaF)” under the section “[Configurations](https://www.google.com/url?q=https://docs.google.com/document/d/1dEIAOHu2chVVZYDmqaOSXxHTLcQLmxEKFKcPnETz3pg/edit%23heading%3Dh.5l6owyvsr9io\&sa=D\&source=editors\&ust=1712137078206215\&usg=AOvVaw2wAF0zfmW7Vi_H4FGGtgcR)”. This section covers the configuration that helps with language enablement in all the services.
If I am not getting correct answers in an Indian language, what could be the problem?
If it is an audio query that you are doing, try repeating it again. Maybe because a certain word is not correctly pronounced or maybe due to some external noise, it is not able to detect the question well.\
For audio or text, you can also check to make sure that it is the language you have selected in which you have sent the question. For example, if you selected a language as Kannada and if your question is in Hindi, the system will not detect the question. You can recheck the language that you have previously selected or select the language again and submit your question.\
The other possibility is that there are no relevant answers to the question you have asked.
What is the difference between the AI Assistant and a normal search engine?
Unlike a traditional search engine, an AI Assistant can accept input in any language, understand context, and generate answers by collating information from different parts of the content. The AI Assistant allows users to input queries in their own language or vocabulary and provides answers based on context rather than just matching specific words.
What is the infrastructure required for running Sunbird AI Assistant?
The infrastructure depends on the scale. Following is the suggested cloud infrastructure to run Sunbird AI Assistant in production with a minimum load of 1000 users per day:
**Cloud Infra requirements:**
1. Number of VMs - 2
2. Size of VMs (size of memory, No of CPU cores and type of processor) - 4 CPU Core, 16 GB RAM, 60 GB Disk
Apart from cloud infrastructure following are required:
1. Access to Language Service like Bhashini
2. Access to GenAI service and Open AI or Azure Open AI is the currently supported GenAI
Refer to [Sunbird AI Assistant - Technical Overview](https://www.google.com/url?q=https://drive.google.com/drive/u/1/folders/1sBwJxUk2lwh-mz-qxkCoTX6U-uhcbD3N\&sa=D\&source=editors\&ust=1712137078214420\&usg=AOvVaw1Aaq11bypqULuO0uusJ_lW) document for the detailed architecture
| Infrastructure | Required specifications |
| --------------------------------------------------------------------------- | --------------------------------- |
| Number of VMs | 2 |
| Size of VMs (size of memory, No of CPU cores and type of processor) | 4 CPU Core, 16 GB RAM, 60 GB Disk |
| OS type \~ linux or windows or paid linux | Ubuntu 22.04 LTS |
| No. of clusters | 1 |
| No of worker nodes | 2 |
| Size of worker node (size of memory, No of CPU cores and type of processor) | 4 CPU Core, 16 GB RAM, 60 GB Disk |
| Storage types \~ frequent access, infrequent access and archival | Object storage buckets - 5 GB |
| Replication criteria across different availability zones | |
| Open AI subscription | 1 |
| Bhashini/Google translate subscription | 1 |
| Domain name and SSL certs | 1 |
| Database managed service(s) | Managed Postgres service |
What will be the cost for running Sunbird AI Assistant?
The cloud infrastructure would cost approximately Rs. 60K per month for a user base of 10,000 users per day, and open AI would cost approximately Rs. 4 per message.
If you access the bot's front end through WhatsApp, there would be an additional cost. It is approximately Rs. 0.3 per user per day.
For a large-scale deployment, Bhashini language services might also have some costs, which the Bhashini team must obtain.
How long will it take to setup Sunbird AI Assistant and run?
A minimal testing instance can be set up in half a day using one-click installer (available for AWS). It would take around two to three days if you install it from scratch.
What is the scale of Sunbird AI Assistant?
It can horizontally scale to any scale. It is only limited by the OpenAI token limit. OpenAI has a limit on the number of tokens that it can service within a given time.
How fast will the responses to messages be?
This depends on the latency of OpenAI and Bhashini services. It can take from 20 seconds to a minute to get the response.
Has Sunbird AI Assistant been adopted in any live use case?
Yes, it is adopted as part of eJaaduiPitara recently launched by the Ministry of Education. Leveraging the potential of Sunbird AI Assistant, three Bots are developed as part of My Jaadui Pitara.
1. **Story Generation Bot**: This generates stories as per the context given by the user.This is not a replacement for the traditional storytelling forms and skills, but it helps the user by creating contextual stories in the local language. It also helps with generating interesting questions related to the story that can be asked to children. This virtual assistant is trained on a collection of traditional Indian stories mentioned in the NCF, like Panchatantra, Hitopadesh and Jataka katha to start with.
2. **Parent Assistance Bot**: This enables Parents to understand and relate to the definitions, key concepts and principles of foundation stage learning, using examples and illustrations in their simple and contextual language. Additionally, it offers guidance on day-to-day activities such as suggesting content, teaching activities, and connecting day-to-day activities to learning outcomes. The Bot is trained to answer questions based on the information from a predefined set of policy documents, training manuals, guidebooks, etc- such as Unmukh, Jaadui Pitara Manual, NCF, etc.
3. **Teacher Assistance Bot**: This enables Teachers to ask questions on multiple topics related to foundational stage learning. It also offers guidance on day-to-day activities such as selecting content, teaching methods, assessments, connecting assessments to learning outcomes, and managing the classroom. The Bot is trained to answer questions based on the set of documents trained for the Parent bot, plus an additional set of documents that are only relevant to Teachers, such as NISHTHA course materials.
Following are a few work-in-progress use cases that are being worked on for adoption:
1. An assistant for children on Child Abuse
2. An assistant for citizens with information on government administrative processes
How to configure a new bot context in sakhi-api-service?
To configure a new bot context, you will have to add below configurations in 'config.ini' of **sakhi-api-service**. Let us look at it with an example where our context is "*test\_bot*" and the corresponding vector collection/index it should refer to is "*test\_documents\_collection*".
1. Add context to index mapping in the below configuration to specify the vector db collection/index from which documents for the bot is to be fetched from.\
2. Configure the bot context in 'supported\_context' for performing bot context validation. Incoming requests to API will be validated with the configured bot context.
3. In order to handle bot specific queries from the user, configure 'enable\_bot\_intent' to 'true' and add 'bot\_prompt' which will contain information of your bot that you want to reply with.
3. To generate answers for user queries, configure the 'activity\_prompt' which will have the steps to perform by LLM using the passed documents from vector db.
How to configure a new bot in Telegram?
### Single Bot configuration
If you are using single bot under your Telegram bot channel, configure the 'context' value under the 'default' section for the bot in 'config.ini' of 'sakhi-telegram-unified-service'
Default context information, 'default\_context\_selection', is to be configured in all language files (ex: '**en.json**') under '**languages**' folder.
### Multiple Bots Configuration
Steps to configure multiple bots under your Telegram bot channel:
1. Go to '**BotFather'** in your Telegram application where you have created the channel
2. select '/mybots' and select the channel under which you wants to create bots.
3. Click on 'Edit Bot' button and the click on 'Edit Commands' button.
4. Add the list of commands to the channel. Example:\
`select_language - user language selection`\
`select_context - user bot context selection`
For both of the scenarios, necessary context information is to be configured in corresponding language file (ex: '**en.json**') under '**languages**' folder.
How to configure a new bot in WhatsApp?
This guide describes how to configure a new bot for the WhatsApp application. We'll use the example of a "farmer\_bot" to illustrate the process.
**User Interface:**
Modify the initial screen where users choose bots. Include a new quick reply button labeled "Farmer Assistant" alongside the existing options (Story Sakhi, Parent Tara, Teacher Tara).
* Update the `bot_selection` section in your language file (e.g., `en.json`) located under `assets/language`.
* Add a new option object within the `options` array for "Farmer Assistant."
* Set the `title` to "Farmer Assistant" and the `postbackText` to "bot\_\_bot\_4" (assuming this is the new identifier for the farmer bot).
Here's an example of the updated JSON snippet:
{% code title="en.json" overflow="wrap" %}
```json
{
"bot_selection": {
"message": {
"type": "quick_reply",
"msgid": "bot",
"content": {
"type": "text",
"header": "e-Jaadui Pitara",
"text": "... (Welcome message content) ...\n\nPlease select below option to proceed",
"caption": "Please select below option to proceed",
"options": [
{
"type": "text",
"id": "bot",
"title": "Story Sakhi",
"postbackText": "bot__bot_1"
},
{
"type": "text",
"id": "bot",
"title": "Parent Tara",
"postbackText": "bot__bot_2"
},
{
"type": "text",
"id": "bot",
"title": "Teacher Tara",
"postbackText": "bot__bot_3"
},
{
"type": "text",
"id": "bot",
"title": "Farmer Assistant",
"postbackText": "bot__bot_4" // New postback text for Farmer Assistant
}
]
}
}
},
....
}
```
{% endcode %}
#### Individual Bot Configurations
Each bot (Story Sakhi, Parent Tara, Teacher Tara) has its own welcome message defining its capabilities and providing examples of prompts users can provide.
Let's configure the welcome message for Farmer Assistant as below:
{% code title="en.json" overflow="wrap" %}
```json
{
"bot_selection": {
....
"bot_1": { ... (Existing Story Sakhi configuration) ... },
"bot_2": { ... (Existing Parent Tara configuration) ... },
"bot_3": { ... (Existing Teacher Tara configuration) ... },
"bot_4": { // New welcome message section for Farmer Assistant
"Welcome": {
"message": {
"type": "text",
"text": "Welcome to Farmer Assistant! I can help you with various farming related tasks. Here are some examples:\n- Ask me about weather forecasts for the next week.\n- What are some good pest control methods for tomatoes?\n- Tell me about crop rotation techniques for corn.\n\nFeel free to ask me anything related to farming."
}
},
}
}
```
{% endcode %}
**Note:** Remember to repeat this configuration process for all supported languages within their respective language files under the `assets/language` folder.
# Knowledge Base
# Best Practices
1. Providing a reference to sources in responses increases trustworthiness.
2. Work on the documents chunking strategy like keeping chunk size of 512/768/1024 with text overlap as 150/200 and verify the retrieval results efficiency for minimum of 30 queries.
3. Ensure your prompt has instructions to avoid responding to queries involving politics, political personalities, religion, caste, skin colour or any other sensitive topics. Perform a round of testing of the bot involving these corner cases.
4. Put a disclaimer (in the bot welcome message or any other suitable place) stating that bot responses are from GenAI and may not be 100% accurate.
5. Try different embedding models/sentence transformers to verify/improve indexing+retrieval efficiency. You can also refer to
6. Good to know: ,
# Indexing CSV Data
Page discusses about different approaches one can take for indexing csv data based on their use-case.
### Solution 1: Summarized Text in Vector Database
Process:
1. Generate a summary of each row in the CSV using an LLM.
2. Store the summarized text in a vector database.
3. Pass the user's query to the vector database to fetch the related information.
Pros:
* Semantic Search: Can handle natural language queries effectively, understanding the context and intent beyond keyword matching.
* Flexibility: Can deal with diverse and complex queries without requiring a predefined schema.
* Ease of Use: Users can ask questions in natural language, which is intuitive and user-friendly.
Cons:
* Resource Intensive: Generating summaries and maintaining a vector database can be resource-intensive in terms of computational power and storage.
* Performance: Vector similarity searches can be slower for very large datasets compared to traditional database queries.
* Dependency on Summarization Quality: The quality of search results depends heavily on the accuracy and completeness of the summaries generated by the LLM.
#### Use Cases:
#### Customer Support: For handling a wide range of queries from customers and retrieving relevant support articles or FAQs.
Sample CSV:
ticket\_id,issue,description
1,Login Issue,"User unable to log into account despite correct credentials."
2,Payment Failure,"Payment failed during checkout, transaction not completed."
3,Account Hacked,"User's account has been accessed by unauthorized persons."
4,Product Not Received,"User has not received the product even after the delivery date."
5,Refund Request,"User requested a refund for the returned product but has not received it."
Example User Queries:
1. "I can't log into my account even though my password is correct."
2. "My payment didn't go through during checkout."
3. "I think my account has been hacked."
#### Document Management: In legal, academic, or corporate environments where documents need to be retrieved based on complex queries.
Sample CSV:
document\_id,title,summary
1,Contract Agreement,"A contract agreement between two parties outlining terms and conditions."
2,Research Paper,"A detailed study on the effects of climate change on marine life."
3,Legal Case,"A legal case involving patent infringement and intellectual property rights."
4,Policy Document,"A document detailing the company's new HR policies and procedures."
5,Technical Manual,"A manual providing technical specifications and usage instructions for a new software."
Example User Queries:
1. "Show me the document related to patent infringement."
2. "I need the research paper on climate change effects."
3. "Can you find the contract agreement document?"
#### Content Recommendation: For recommending products, articles, or media based on natural language descriptions of user preferences or needs.
Sample CSV:
content\_id,title,description
1,The Great Gatsby,"A classic novel set in the Jazz Age that explores themes of wealth and excess."
2,Inception,"A science fiction film about a thief who enters people's dreams to steal secrets."
3,Introduction to Python,"A beginner's guide to programming in Python."
4,Healthy Eating,"An article about the benefits of a balanced diet and healthy eating habits."
5,Yoga for Beginners,"A guide to starting yoga practice, including basic poses and tips."
Example User Queries:
1. "Recommend me a classic novel to read."
2. "I'm looking for a movie about dreams and reality."
3. "Do you have any articles on healthy eating?"
### Solution 2: SQL/NoSQL Database with LLM for Query Interpretation
Process:
1. Store the CSV data in a SQL/NoSQL database.
2. Use an LLM trained with few-shot examples to interpret the user's query and generate a corresponding database query.
3. Fetch records from the SQL/NoSQL database based on the interpreted query.
Pros:
* Performance: SQL/NoSQL databases are optimized for fast querying of structured data, making this solution more efficient for large datasets.
* Existing Infrastructure: Leverages existing database infrastructure, which many organizations already have in place.
* Structured Queries: Can provide more accurate results when the data is well-structured and indexed.
Cons:
* Complex Query Mapping: Translating natural language queries into accurate database queries can be challenging and may require extensive training and fine-tuning of the LLM.
* Less Flexible: May not handle very complex or abstract queries as well as a vector-based approach.
* Schema Dependency: Changes to the database schema may necessitate updates to the LLM's query interpretation logic.
Use Cases:
* Business Intelligence: Generating reports and insights from large datasets based on structured queries.
Sample CSV:
report\_id,department,metrics,period
1,Sales,"Total sales, revenue, profit margins",Q1 2023
2,HR,"Employee retention rate, new hires, turnover rate",Q1 2023
3,Marketing,"Ad spend, ROI, lead generation",Q1 2023
4,Finance,"Expenses, net income, cash flow",Q1 2023
5,IT,"System uptime, incident response time, new projects",Q1 2023
Example User Queries:
1. "Show me the sales report for Q1 2023."
2. "I need the HR metrics for the first quarter of 2023."
3. "What are the marketing KPIs for Q1 2023?"
* Inventory Management: Managing and querying inventory data where queries are typically structured around specific fields like product ID, location, etc.
Sample CSV:
product\_id,product\_name,quantity,location
1,Widget A,100,Warehouse 1
2,Gadget B,50,Warehouse 2
3,Device C,75,Warehouse 1
4,Tool D,20,Warehouse 3
5,Item E,10,Warehouse 2
Example User Queries:
1. "How many Widget A are in stock?"
2. "Give me the inventory details for Warehouse 2."
3. "What is the quantity of Device C?"
* Healthcare Records: Retrieving patient records or treatment histories based on specific attributes like patient ID, date, or diagnosis.
Sample CSV:
patient\_id,name,diagnosis,treatment,date
1,John Doe,Diabetes,Insulin Therapy,2023-06-01
2,Jane Smith,Hypertension,Medication,2023-06-02
3,Alan Brown,Asthma,Inhaler,2023-06-03
4,Emily Davis,Flu,Rest and Fluids,2023-06-04
5,Michael Johnson,Fracture,Cast and Physical Therapy,2023-06-05
Example User Queries:
1. "Fetch the medical record for John Doe."
2. "Show me all patients diagnosed with diabetes."
3. "Get the treatment details for the patient with a fracture."
#### Conclusion
Solution 1 is ideal for scenarios where the nature of the queries is complex, varied, and context-rich. It is well-suited for environments where users benefit from the flexibility and semantic understanding of natural language processing, such as customer support, document management, and content recommendation.
Solution 2 is better suited for environments where queries can be well-defined and mapped to a structured database schema. It is more efficient for large datasets and situations where performance and accuracy are critical, such as business intelligence, inventory management, and healthcare records.
#### Choosing the Right Solution
Solution 1 (Vector Database) is recommended if:
* You need to handle a wide variety of complex and semantically rich queries.
* User queries are often in natural language and require a high degree of contextual understanding.
* You have the computational resources to support vector database operations.
Solution 2 (SQL/NoSQL Database with LLM Query Interpretation) is recommended if:
* Queries are typically structured and can be mapped to specific database fields.
* You need high performance and efficiency for large datasets.
* You want to leverage existing database infrastructure and ensure accuracy through structured queries.
Ultimately, the choice depends on the specific requirements of your use case, including the nature of the queries, the structure of the data, and the performance considerations.
# Contact us
You can have discussions with the Sunbird AI Assistant community at [Sunbird-AIAssistant](< https://github.com/orgs/Sunbird-AIAssistant/discussions>) discussion forum.
3. To generate answers for user queries, configure the 'activity\_prompt' which will have the steps to perform by LLM using the passed documents from vector db.
