Federated Search is a technology that enables you to perform a search across multiple content repositories simultaneously — from a single search interface — and which then aggregates the results in that origin interface.
You can use federated search in Vanilla to enhance your Vanilla-based search functionality by incorporating one or multiple external search engines into your users' community searches. These search engines can be either public (e.g., The New York Times) or private (e.g., a company-specific Zendesk Guide knowledge base) as you'll see in the list of Search Connectors in Vanilla, below.
When one or more of these content repositories ("repos") is incorporated into your Vanilla search, the results can display alongside your community search results.
In the context of your Vanilla community, federated search is facilitated by one or more Search Connectors. You can configure any of our Search Connectors to a search API and map its results so that when users initiate a search in your Vanilla community, they'll get results from those other content repositories.
⚠️ IMPORTANT: The ability of a Vanilla Search Connector to work might be limited by the API restrictions of your third-party repo.
📝 NOTE: This article provides a high-level look at the various configuration fields that you'll have to complete in order to configure a connection to a generic search API. The values for those fields are dependent on the repo that you are connecting to. Further, not all search APIs will be compatible with Vanilla's federated search service. It depends on what data is returned on search queries and how it is packaged.
Federated search via Search Connectors in Vanilla
As Vanilla continues to develop its various search functionality options (adding to the likes of External Search and Member Search), you can capitalize by incorporating external repos that provide even more content options to your community users — content that is relevant and easy to access.
Vanilla currently offers the following out-of-the-box Search Connectors that you can configure in order to connect to your own existing account with those applications.
You can also add and set up custom Search Connectors, such as we have documented for:
✔️ TIP: If you have a suggestion for a Search Connector, submit it as a product idea in the Success Community, https://success.vanillaforums.com/categories/product-ideas.
Pre-requisites
In order to use the federated search functionality with any Vanilla search connector, you must have a/an:
- JSON array of results
- number of the total results
- Next Page
- Previous Page
- Title
- Excerpt
- URL
In addition to these, it's beneficial to have a:
- Creation date
- Last update date
Manage existing Search Connectors
This section describes how to:
- enable a Search Connector and
- access the configuration options in order to make changes to a Search Connector.
Enable a Search Connector
To enable an existing Search Connector:
- Access the Dashboard and navigate to Settings > Addons > Federated Search.
- On the Search Connectors page, slide the toggle of a Search Connector to the right to enable it. (In the image below, the Zendesk Guide connector is enabled but the Salesforce Article Search connector is not.)
Modify a Search Connector
To modify a (out-of-the-box or custom) Search Connector:
- Access the Dashboard and navigate to Settings > Addons > Federated Search.
- On the Search Connectors page, click the pencil icon of a Search Connector to access its configuration panel.
Add a custom Search Connector
In addition to the out-of-the-box Search Connectors, you can create your own custom connections to your preferred content repos. You'll have to provide information for multiple fields across several configuration panels.
⚠️ Before you begin…
- Before starting to create a custom Search Connector, review this section to see what types of information are required. Some fields are required and, if they're not provided with a valid value, the connector will not work.
- The following sections offer a first-look at the configuration panels that you'll encounter when creating (and modifying) a custom Search Connector.
- These images are from an existing, example custom Search Connector and the values are unique to the connector and the Vanilla account.
- Some values have been blurred out for security reasons.
Steps
To create a custom Search Connector:
- Access the Dashboard and navigate to Settings > Addons > Federated Search.
- Click the Add button, located in the upper right.
Custom search connector - Basic information
Provide basic, high-level information about the repo you're connecting to.
- ID - The main identifier of this connector. It is a machine-readable index to the connection, and it displays in the search page URL. It can be any string of letters, numerals, and "special" characters.
⚠️ IMPORTANT: The ID cannot be changed after the Search Connector has been set up, so be sure it is recognizable. You can, however, delete it and create a new connector with a different ID. - Label - The name that displays on this connector's dedicated tab on the search page. The connector's search results display on the tab.
- Endpoint - The full and valid URL of your connected application. Do not include variables in this URL.
Custom search connector - API information
Provide API information about the repo you're connecting to.
📝 NOTE: Some of the required parameters might be unique to your API. Be sure to configure your custom search connector with the appropriate parameters so that the connector can correctly format search queries for your community.
The required parameters can usually be found in the API documentation.
- Query - The indicator in any query that tells your Search Provider/API what term to search for. The letter q ("query") is frequently used for this purpose.
- Optional Static Parameters - These might be required by your Search Provider. Or you might want to pass parameters that will broaden or refine your search results. You can specify key-value pairs (e.g.,
role=fan) and multiple key-value pairs (e.g., role=fan&rate=general).- These parameters are static (i.e., they do not change) and they will be added to every query you send. You cannot add, for example, the date or the user who is making the request.
Optional Static Headers - These headers are static (i.e., they do not change). Your Search Provider might require that certain headers be passed in the HTTP header either as a requirement of the search API or for logging purposes. Input headers as a well-formed JSON object, such as:
{
"x-search-from" : "community"
}
If either the Text Custom API button or the Next button is disabled, your JSON could be correctly formatted.
- Do not add Authentication Headers in this field; use the dedicated Optional Authentication Headers field.
- Authentication - Slide the toggle to right in order to add authentication headers to this connector.
- Optional Authentication Headers - If your Search Provider has permission-based results, you might have to pass an Access Token or a Basic Auth Token to access it. Toggle this on and add the Header in as a well-formed JSON object. For example:
{
"Authorization": "Bearer WvJbscJv8K863EopyRYeDmsGc0dsR1dE11SJF23lzf7v2McgEDgdyRzS5amepmkO"
}
- Allowed Roles - Choose which community Roles have access to this connector. To grant access to everyone, leave this blank.
- Test search term - This should be a term that will return some results. The test will be important to map the results to your Vanilla Search page. Click Test Custom API to see the results.
- Test your custom API - Click to test your API. The results display below.
- If testing returns an error, review the query, parameters, and headers you have configured above.
- If testing returns no results, change the Test search term to something that is more likely to be found on your repo.
Click Next specify where the search results are.
Map the results
Here you will see an example of a search result based on a Test search term of "hockey" displayed as a JSON object called data:
{ "searchTerm": "hockey",
"results": {
"count":25,
"next":2,
"previous":null,
"results": [
{
"id": "2pr2eb8jwqd4o",
"article": {
"id": "1nzi2m71amaob",
"title": "Ice hockey", ...
Click the dropdown and choose which part of the data object contains the array of results. In this case, it is data.results.results.
Custom search connector - Pagination
Provide information about how the search connector should fetch new results.
- What kind of pagination does your API use? - Pagination is how a Search Provider divides your search results into individual pages. Your Search Provider is already set up to serve results using either Cursor or Offset; choose one from the dropdown.
✔️ TIP: To learn about these options, refer to Cursor vs. Offset pagination, below. - Current page number - Click the dropdown and choose the value that represents the current page.
- Total Results - Click the dropdown and choose the value that represents the total number of results.
- Number of results per page - Specify the variable name that your Search Provider looks for to determine how many results per page it is to return.
- Offset (Offset type pagination) - Specify how to offset (shift) the number of search results. This is usually specified in the URL parameters.
- Next Page and Previous Page (Cursor type pagination) - Click the dropdown and choose which values represent the Next and Previous pages.
- Test your pagination configuration - Click the Next button to verify your pagination configuration.
Custom search connector - Search results
Provide information on where to find values within the result object.
- Title - Click the dropdown and choose a value to display as the title of the result on the Vanilla Search page.
- URL - Click the dropdown and choose the value that represents the URL of the result that will display on the Vanilla Search page.
- Highlight/Excerpt - Click the dropdown and choose the value that represents the excerpt of text that will display on the Vanilla Search page.
- Date Updated - Click the dropdown and choose the value to display as the "last-update" date for the result on the Vanilla Search page.
- Creation Date - Click the dropdown and choose the value to display as the "content-creation" date for the result on the Vanilla Search page.
- Remove HTML from title and highlight/excerpt - Check this box to remove formatting that could affect how the results display on the Vanilla Search page.
Custom search connector - finishing up
Click Next to go to the final step of the set-up process.
Verify that your mapping choices reflect the search results. This is roughly how it will appear on the Vanilla search page.
If all is well, test different search terms and check the results on the Vanilla Search page.
Cursor vs. Offset pagination
Cursor
Cursor is a method that a Search Provider will use to keep track of which search results have been served and which results are to be served next. It is a form of internal bookmarking. The Search Provider will return a token (a random string) along with the search results. It will usually be called a "cursor," but it could be called something else, as in the following example, where it is called "currentPage":
"results": {
"entriesPerPage":25,
"currentPage": "275f3aaa-6fc1",
When Vanilla makes a request to the Search Provider, we send back the "cursor" (or, in this case, "currentPage") in the request so that the Search Provider knows to get the next set of results.
Offset
Offset pagination is when a Search Provider requires:
- an offset (the number of results already served) and
- the limit (the index of the last result in the requested search).
So, if Vanilla is displaying 50 results per page, then for the first request, we will send a request like:
'&offset=0&limit=50'
…and then on each page we will have a link with:
'&offset=50&limit=100'
