Backend search — how to embed AI search into your webapp

Apr 16th 2024Engineering

Here at Algolia, we’re big proponents of focusing on the big picture. Most of our customers aren’t just developers or engineers, but architects — you’re building something you believe in, and you shouldn’t be bothered with the implementation details. In a usual project, features like search are important but ultimately not the entire point of your site or app, so we give you the tools to essentially drag and drop search into your site and move on. It’s our way of empowering you to focus on the big picture, what makes your app stand out among the crowd.

But what if the search experience is the core feature of your site? It’s all the more reason to go with the world’s leading search provider, but maybe that hands-off approach isn’t right for you. If you’re finding yourself in this situation, this article is for you. Today, we’re going to dive into backend search, the less advertised but more powerful implementation method that gives you deeper control over how search in your app works.

A hypothetical example

This is FishingBuddy. It’s a platform for web and mobile that does what it says on the tin: it’s a companion app for those who enjoy fishing.

Let’s put ourselves in the mindset of a first-time site visitor exploring whether FishingBuddy is worth their time. Here’s what the first part of their homepage looks like:

The FishingBuddy homepage, based on the AllTrails homepage (which actually does something similar with Algolia)

Quick, what’s your first impression about what FishingBuddy actually does?

The immediate answer is obvious: they let you search through a huge base of fishing locations by where you are and what you want to catch. It’s the search that defines FishingBuddy. Of course, FishingBuddy could be a lot more than just a glorified database, but the focus, what they highlight, what sets them apart… it’s all search.

Clearly, they’d want more control than the average Algolia customer. While most of our users come to Algolia because we’ve already done so much of the hard work for them — a completely understandable and productive choice — FishingBuddy would come to us to build their app’s whole premise on extending our expertise with theirs, necessitating that they take a slightly different implementation route than most.

Enter backend search. By connecting to Algolia on the backend, FishingBuddy gets full control over the search process. Why is that helpful here?

1. First of all, from the placeholder in the input, it’s not hard to imagine that they’re probably querying multiple indexes. After all, it’d be tedious to list every single type of fish that could appear in every single body of water on that body’s record in the index, so they probably have different indexes for bodies of water, regions of the world, and types of fish. But they’ve merged the search for these indexes into one interface so the user never has to think about that technical implementation detail. Sending the search query first to their own backend lets them process it first, deciding which of the indexes it should run the search on (if not all of them) and then send the user to the right page preloaded with the search results.
2. This step also makes it easier to modify results with data that’s not stored in Algolia. Normally, we’d recommend putting everything that might end up in a search result into your optimized Algolia index so that you don’t have to make extra time-consuming calls to your database, but in some cases (like this one) that’s simply not feasible. What happens if FishingBuddy wants to exclude results for docks that aren’t open right now, or results for bodies of water that aren’t suitable for fishing right now for some reason? It’d be impractical to constantly update the Algolia index with that regularly-changing data. If they ran their search completely on the frontend, they’d have to send not only the HTTP request to Algolia’s servers with the search query, but then also a second consecutive HTTP request to the backend requesting database data. By running the search from the backend, they can combine those two HTTP requests into one, speeding up the search load time. It’s still ideal to never have to access the database at all if that’s possible, but this compromise actually makes what would have been an unideal situation almost indistinguishable from the recommended implementation method.
3. They also get to decide whether to build their own UI… we’ll pick back up on this in a bit.

In this niche, backend search can be very useful. So let’s get technical — how can you actually implement it? Well, we have an excellent guide in the docs, but I’ll walk you through the two major options here: Backend InstantSearch or the API route.

A compromise solution: Backend InstantSearch

Back to the bit I said we’d return to: do you want to build your own UI? While that can sometimes be useful, it’s a bit much for the average app with backend search. Is it possible to get the flexibility that comes with backend search, but without the huge responsibility of designing and implementing an unproven UI?

Here’s where one of my favorite pieces of Algolia tech comes into play. It’s called Backend InstantSearch, and there’s a whole guide in the docs about it. The gist is that your site’s backend serves as a sort of middleware between the InstantSearch frontend and the Algolia servers. This way, you’ll still get to do all your custom backend processing, but you’ll also get the benefit of all the work Algolia’s engineers have put into researching and developing a world-class search experience that makes your customers want to convert.

Here’s the general process:

1. Install and instantiate the Algolia library that’s appropriate to the backend you’re using. I like JavaScript, so I typically create server functions in Node.JS. That makes the JavaScript library my friend here. Generally, it’s better to use a library than the REST API directly because (a) it gives you some cool additional features, and (b) it handles the boilerplate stuff that you’d really never have any reason to customize, like the retry strategy.

   const algoliasearch = require('algoliasearch');
   const algoliaClient = algoliasearch('YourApplicationID', 'YourSearchOnlyAPIKey');
   

2. Set up a function on your backend that takes in some input, runs your pre-search processing, feeds the data into Algolia’s API, and returns the result. That function can then be included in an endpoint.

   const search = async query => {
   	// some logic modifying the variable query
   	
   	const results = (await algoliaClient.search(query));
   	
   	// some logic modifying the variable results
   	
   	return results;
   };
   
   // your chosen framework's endpoint, example in express.js
   app.post('/search', async (request, response) => {
     res.status(200).send(
   	  await search(request.body)
     );
   });
   

3. Typically, on the frontend, you’d do something like this to use InstantSearch:

   const search = instantsearch({
   	indexName: 'my-index',
   	searchClient: algoliasearch('YourApplicationID', 'YourSearchOnlyAPIKey')
   });
   
   search.start();
   

   All we’re going to differently is define a custom search client. A search client is really just an object that implements some search methods — Algolia usually abstracts the API requests into those functions so you don’t have to do it yourself, but we’ll build a simple barebones version to redirect those requests to your backend instead of Algolia’s servers:

   const search = instantsearch({
   	indexName: 'my-index',
   	searchClient: {
   		search: query => fetch(
   			'/search', 
   			{
   				method: 'post',
   				body: query
   			}
   		)
   			.then(res => res.json()); // this returns a promise
   	}
   });
   
   search.start();
   

And you’re done! Now you’re set up to use InstantSearch like any other project.

Skipping InstantSearch altogether

Of course, you don’t have to use InstantSearch. It’s just easier in most scenarios. But for our fictional company FishingBuddy above, search is their whole thing — it’s what people visit their site for. Hiring a specialist designer and building their very own search UI might be their best option from a branding perspective. Take a look at that homepage again, and you’ll see why they’d want control:

The FishingBuddy homepage, based on the AllTrails homepage (which actually does something similar with Algolia)

They probably could accomplish what they want with InstantSearch, but they might not be as responsive when their designers want to change things or run A/B tests, so it’s a reasonable choice. However they chose to implement it, it looks good! It’ll definitely accomplish the goal: to draw new users toward their primary feature (search) immediately when they get to the page.

In this case, it’d be straightforward for the developers to set this up without InstantSearch. It’s the same backend code as before:

const search = async query => {
	// some logic modifying the variable query
	
	const results = (await algoliaClient.search(query));
	
	// some logic modifying the variable results
	
	return results;
};

// your chosen framework's endpoint, example in express.js
app.post('/search', async (request, response) => {
  res.status(200).send(
	  await search(request.body)
  );
});

Nothing much going on here. Just a simple function that runs the search (and whatever custom logic we attach to it) and an endpoint that runs our search function with whatever data comes in through the HTTP request. It’s about as simple as it gets for a project of this complexity.

On the frontend though, we’ll just skip InstantSearch and interface directly with the backend. We’ll just pull that search function we created before out of the search client and run it standalone:

const search = query => fetch(
	'/search', 
	{
		method: 'post',
		body: query
	}
)
	.then(res => res.json()); // this returns a promise

Then, you can use that function whenever you’d like to actually run the search, like whenever the content of the <input> has changed (though you might want to consider a small delay so you’re not running unnecessary searches). While these minor details are all very well thought-out in InstantSearch, you’re going to have to implement them yourself if you take this route, so sit down and really think through the entire user flow before you try to code this UI.

Are you looking to implement backend search, but you’re still trying to figure out where to start? We’ve got you covered. Our team of engineers loves to answer questions and help folks learn more about all the cool stuff Algolia can do. Just click the chat button at the bottom right of your screen to get in contact 🙂