Cognitive Search

Cognitive search can be enabled with multiple types of Knowledge:

1. Web based: This enables to directly scrap the content from your Sitemap’s/URL's
2. Document based: This enables you to read content from your documents and generate answers.
3. API based: This enables you to get the knowledge base content via an API call.

Web based: This enables to directly scrap the content from your Sitemap’s/URL's. Refer below steps to set up a web-based scraper.

1. Name your scrapper
2. Toggle between “Website URL” or “Sitemap URL”. If you are using Sitemap based, toggle to respective side.
3. Enter the website URL that needs to be scrapped.
4. Add specific tags.
5. Set the request delay. Request delay is the time the scrapper should wait to load the content when a URL is hit.
6. Set a frequency for scrapping your website.
7. A sample of Sitemap format is attached. You can click to download the same.(Click here)

Document based: This enables you to read content from your documents and generate answers. Refer below steps to set up a document-based scraper.

1. Upload a document that needs to be scrapped.
2. Set the tags for your document.
3. Click on the save button.

API Based: This enables you to get the knowledge base content via an API call. Refer below steps to set up an API-based scraper.

1. Name your scrapper
2. Select the snippet which calls the API. The response should be in a specific JSON format.
3. Select the appropriate tags.
4. Set an appropriate frequency.

Once the documents are uploaded, they will appear on your main screen as below.

Configuring the Settings

Once the documents are processed, click on the settings options to configure the right settings for your use case.

• Select the right embedding model
• Select the right LLM model (If GenAI toggle is enabled in those cases this model needs to be configured)
• Set the right confidence as per your use case.
• Click on the Advance settings to set below options
  • Rephraser: When a query needs to be rephrased based on the previous context, you can enable this option.
  • References: When enabled it will show you the reference URL of the KB from where the answer is generated.
  • Response evaluator: This functionality is used to verify the answer generated and decide if the answer generated is correct or not.
  • Remove stop words: Disables stop words to generate accurate answers.
  • Analyze documents: When turned on, it uses LLM Models to process the document.
  • Select the right number of chunks: Chunks refer to how many chunks/parts of KB needs to be passed to get the right answers.
  • Select the prompt: prompts can be Concise, Descriptive or Voice specific.

Reindexing the knowledge base

Once the right settings are configured, it’s time to reindex the documents. Reindexing is nothing but processing/training the document. You can enable the GenAI toggle to enable GenAI to your AI agent.