Implementing Custom AI Assistants in Nepal: A Guide to Enterprise RAG

The rise of large language models (LLMs) like GPT-4, Claude 3.5, and Gemini 1.5 has triggered a global wave of AI adoption. However, for most enterprises in Nepal—ranging from commercial banks to telecom providers and e-commerce platforms—generic AI models fail to address their most critical needs. Out-of-the-box LLMs are prone to hallucinations, lack context about proprietary datasets (e.g., custom API documents, internal HR manuals, or local tax regulations), and pose severe data security risks if company records are processed through public API endpoints.

To bridge this gap, forward-thinking organizations are implementing Retrieval-Augmented Generation (RAG). RAG is a software architecture that optimizes the output of an LLM by referencing an authoritative, external knowledge base before generating a response.

In this comprehensive guide, we will walk you through the architecture, benefits, security aspects, and step-by-step implementation of custom RAG systems tailored to the Nepalese business landscape.

Why Out-of-the-Box AI Falls Short for Local Enterprises

Nepalese enterprises face three unique challenges when deploying public LLMs:

1. Context Lack and Localization: Foundation models are trained on global web scrapes. They lack fine-grained, updated knowledge about Nepalese commercial laws, central bank (Nepal Rastra Bank) directives, or specific local company guidelines.
2. Hallucinations: Asking a model about a specific corporate policy or pricing structure often leads to fabricated facts. In enterprise settings, an incorrect response to a customer or employee can have legal and financial consequences.
3. Data Residency and Sovereignty: Sending confidential customer records, financial statements, or proprietary code to foreign servers violates compliance guidelines and internal security protocols.

Under the Hood: The RAG Architecture

RAG splits the AI question-answering task into two main phases: Ingestion and Retrieval & Generation.

1. Ingestion Phase

Before the AI can answer any question, your business documents must be converted into a machine-readable format.

• Document Extraction: PDFs, Word documents, Markdown files, or database entries are extracted.
• Chunking: Text is split into smaller, overlapping chunks (typically 500 to 1000 characters). This preserves local context and prevents model context-window overflows.
• Embedding Generation: Each chunk is processed through an embedding model (e.g., text-embedding-3-small or all-MiniLM-L6-v2) to produce a vector—a list of floating-point numbers representing the semantic meaning of the text.
• Vector Database Storage: The vectors and their corresponding text metadata are stored in a specialized database (such as pgvector, Pinecone, or Qdrant).

2. Retrieval and Generation Phase

When a user asks a question, the system dynamically retrieves the relevant content:

1. Query Embedding: The user's query is converted into a vector using the same embedding model.
2. Vector Similarity Search: The vector database performs a cosine similarity search to find the chunks of text that are semantically closest to the user's query.
3. Context Augmentation: The system builds a prompt containing both the user's original query and the retrieved text chunks as context.
4. LLM Generation: The prompt is passed to the LLM (e.g., Gemini 2.5 Flash), which synthesizes a natural-language response based strictly on the provided context.

Step-by-Step Implementation Guide

Let’s look at a concrete TypeScript implementation utilizing Node.js, Express, and pgvector.

Step 1: Initialize Database Connection

We will use PostgreSQL with the pgvector extension, which is highly suited for enterprise setups as it allows storing vectors directly alongside structured application data.

import { Pool } from 'pg';

const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
});

// Verify pgvector extension is enabled
await pool.query('CREATE EXTENSION IF NOT EXISTS vector;');

Step 2: Set Up Document Embeddings

Here is how to extract and store document chunks with OpenAI's embedding API:

import axios from 'axios';

async function getEmbedding(text: string): Promise<number[]> {
  const response = await axios.post(
    'https://api.openai.com/v1/embeddings',
    {
      input: text,
      model: 'text-embedding-3-small',
    },
    {
      headers: { Authorization: `Bearer \${process.env.OPENAI_API_KEY}` },
    }
  );
  return response.data.data[0].embedding;
}

Step 3: Vector Similarity Search

When a query is received, we fetch the top 3 semantically relevant documents using PostgreSQL's cosine distance operator (<=>):

async function searchKnowledgeBase(userQuery: string): Promise<string[]> {
  const queryVector = await getEmbedding(userQuery);
  const vectorStr = `[\${queryVector.join(',')}]`;

  const result = await pool.query(
    \`SELECT content FROM document_chunks 
     ORDER BY embedding <=> \$1::vector 
     LIMIT 3;\`,
    [vectorStr]
  );

  return result.rows.map(row => row.content);
}

Step 4: Construct augmented prompt for the LLM

Finally, we send the prompt to Gemini/OpenAI:

async function generateResponse(userQuery: string): Promise<string> {
  const contextChunks = await searchKnowledgeBase(userQuery);
  const contextText = contextChunks.join('\\n\\n');

  const systemInstruction = \`
    You are an AI assistant. Answer the user's question using ONLY the provided Context below.
    If the context does not contain the answer, say: "I cannot find the answer in the provided documents."
    Do not make up facts or extrapolate beyond the text.

    Context:
    \\\${contextText}
  \`;

  // Code here calling LLM SDK passing systemInstruction and userQuery...
}

Security, Compliance, and Local Constraints

When deploying RAG in Nepal, enterprises must consider the following guardrails:

1. Access Control (RBAC): Ensure the retrieval system does not fetch documents the user has no authorization to view. For instance, a regular employee's query should never retrieve financial projections or board minutes.
2. Local Hosting Options: If your organization has strict compliance requirements (e.g., banks subject to NRB IT guidelines), you can run open-source models (such as Llama-3 or Mistral) on-premise using platforms like Ollama or vLLM, paired with self-hosted PostgreSQL.
3. Data Sanitization: Before storing or processing texts, strip out Personally Identifiable Information (PII) like mobile numbers, bank account numbers, or citizenship card IDs.

Conclusion

Implementing Retrieval-Augmented Generation is the most practical way for Nepalese enterprises to deploy reliable, context-aware AI. By feeding domain-specific knowledge to LLMs, you avoid hallucinations, maintain security compliance, and deliver actionable responses to your users.

At RightKod, we design and build secure, bespoke RAG architectures integrated with your existing corporate databases and workflows. Get in touch with our team at contact@rightkod.com to schedule an infrastructure assessment.