Technology Stack
Below is the technology stack used for the implementation of the AI Assistant. AI Assistant primarily uses the RAG mechanism, powered by Marqo (for retrieval) and GenAI services to generate answers for the user's questions.
Front End
Telegram (a Telegram Bot)
WhatsApp (a WhatsApp business number)
Backend
Python - Programming Language
FastAPI - Web framework for building APIs with Python
Marqo - Marqo is an end-to-end vector search engine. Vector generation, storage
and retrieval are handled out of the box through API.
Cloud Storage Services (Supported are OCI Object Storage & AWS Buckets)
Frameworks
LangChain - Framework to simplify the creation of applications using large language models.
LlamaIndex - Flexible data framework for connecting custom data sources to large language models.
For Telegram:
Telegram Service Wrapper - Webhook to receive the request from the user
through a Telegram Bot and to respond with answers. The webhook is
registered with the Telegram Service Provider.
Redis - In-memory data store for holding users language and bot selection
For WhatsApp GupShup provider:
WhatsApp Service Wrapper - Webhook to receive the request from the user and to respond with answers. The webhook is registered with the WhatsApp Service Provider, which in this implementation uses Gupshup.
PostgreSQL - Used for storing the user's session, language selected and bot selection for WhatsApp.
External Services
GenAI Services - LLM for,
Text Generation
Supported ones are,
OpenAI (GPT4) with Moderation
Azure OpenAI (GPT4) with Moderation
Language Services for,
Language Translation
Speech to Text
Text to Speech
Supported ones are,
Bhashini Dhruva
EkStep Dhruva
Google Language Services
Telemetry - Log events are getting captured for
api_access
api_call
Other Tools / Services
Docker
Cloud Infrastructure - Supported ones are,
OCI
AWS
Dependencies
List of external libraries used: https://github.com/Sunbird-AIAssistant/sakhi-api-service/blob/main/requirements-prod.txt
Last updated