Search Table Block

Learning Objectives

• Understand the configuration and usage of the Search Collection Table Block.
• Learn how to perform vector and hybrid searches.
• Master the use of filters and Jinja templating for dynamic queries.

1. Add Search Table Block:

• On the workflow canvas, click "+" and add Search Table block
  • For a basic workflow this is usually placed after the input block and before your LLM block.

2. Configuration:

• Collection: Select the collection to query from a dropdown. This is a required field.
• Table: Choose the table within the selected collection.
• Search Term: The term or phrase to search for. Supports Jinja templating for dynamic query construction.
  • For example: To search using the user's input, use {{inputs.message}} as the search term.

3. Advanced Configuration and Usage:

• Minimum Similarity: A float value between 0-1 (default 0.35) that sets the threshold for filtering search results based on vector distance. Lower values mean stricter matches. Use a lower value for specific matches like product names.
• Hybrid Search: A boolean (default false) that toggles between key-word search and a semantic-search. Enable this for results that consider both semantic meaning and exact keyword matches.
• Alpha: Weighting factor for hybrid search. Must be between 0 and 1. A value closer to 0 will favor key word lookup while a value closer to 1 will favor semantic searches.

3. Filters:

• Filters are specified as JSON arrays, e.g., ["attribute", "operator", "value"].
• Operators Include:

• An example filter: ["cm9h4csj40ttk08s6yrgawpsf", "Eq", "{{inputs.metadata.model}}"]
  • Lets break this down:
    • The first attribute cm9h4csj40ttk08s6yrgswpr corresponds with a column in a Scout table. You can find your column ID by clicking into the header column and selecting "Copy ID"
    • Next is the operator, in the example above "Eq" is used to check if two values are equal.
    • Lastly, the value here represents an input value, in this particular workflow we are searching through documentation based on a "Model #", your value should cite the input block, check the second input titled "metadata" and then finally pass in the output of that block which will return a "model #".

4. Outputs:

• The block outputs a list of results from the queried collection table based on the specified search criteria. On the workflow console, once the run has completed you can inspect the Search Table block and see all the results it gathered along with their similarity scores.

Tips and Best Practices:

• Use Jinja Templating: Dynamically construct queries and filters based on user input or workflow variables.
• Fine-Tune Parameters: Experiment with Vector Minimum Distance, Alpha, for desired precision and recall.
• Apply Filters Thoughtfully: Enhance search efficiency by narrowing results to the most relevant subset.
• Leverage Hybrid Search: Use for complex queries requiring a balance of semantic understanding and keyword-based ranking.
• Limit Results: Set a reasonable limit to prevent overwhelming outputs and ensure relevance.

Micro-Challenge:

• Using the basic RAG workflow template linked below, populate a collection with a sample CSV file (e.g., a list of products, documents, or support tickets). Then:
  • Use the Search Collection Table block to search your data.
  • Apply a filter to narrow results based on a column value (e.g., filter by product category, model number, or document type).
  • Experiment with vector minimum distance, hybrid search, and alpha to observe how your results change.
• Goal: Return only the most relevant subset of results for a specific input, using filters and search tuning effectively.
  • Basic RAG workflow Template