A couple of weeks, Google CSE blog announced the release of a new feature and a new kind of Custom Search Engine (CSE) that you could create on the fly. This post dig a bit deeper and describe the underlying infrastructure that powers this. With our new Linked CSEs, we are exposing the API to create and control CSEs.
Until now, you created a CSE either by using the wizard or by writing an XML file and uploading it to Google (via the “Advanced” tab on the control panel). To change any aspect of the CSE, you had to either use the control panel or upload the new XML specification. This imposed several limitations:
- Creating and maintaining a CSE was a manual process.
- It was difficult to create a large number of CSEs.
- It was difficult to use other data sources such as iCal, RSS, Google Base, etc. to programmatically create CSEs.
The search box code for these CSEs (found on the “Code” tab in the control panel) includes a “cx” parameter with every search request (for example, <input type=”hidden” name=”cx” value=”005946352831473999820:qs1idu8ptku” />), which specifies an internal identifier for the CSE.
How does this work? With Linked CSEs, you designate a CSE specification URL with each search request (as a hidden form field in your search box HTML code). Google retrieves the CSE specification from the URL when your user searches in the CSE. We cache and refresh the results so that only the first search to your CSE incurs any delay. The flexibility to specify how your search engine should behave, just when your user is doing the query, using whatever data sources you want, opens up many possibilities:
- You can use our makecse tool to generate CSEs from different sources of links:
- You can combine multiple sources of links using our makeannotations tool and the <Include> tag. For example, its easy to create a search engine from the links on the front pages of techmeme, slashdot and digg.
- You can write your own tools to produce <Annotations /> XML from other data sources such as Google Calendar or iCal feeds, Google Base or any other structured source of information.
- You can automatically generate any number of CSEs, each possibly tuned to a particular user. For example, we’ve created a sample that builds a CSE from a user’s digg.com friend network and submissions using the Digg API. Try it out and view the source. This makes use of two simple python CGI scripts:
- diggannos.py generates <Annotations> from the specified user’s submitted stories
- diggcse.py generates <GoogleCustomizations> from the specified user’s friend network. For each friend, it generates an <Include> element pointing to the appropriate diggannos.py URL
You can test any Custom Search Engine XML by going to http://www.google.com/coop/cse/cref and entering the URL. Putting a search box on your site is as easy as copying a small bit of HTML code and modifying the “cref” parameter.
Linked CSEs are a very big step for Google Custom Search. We hope you will find them as cool as we do.
Source:→ Google Custom Search Blog
Google, Custom Search Engine, CSEs, API, On The Fily, Featured News, CSE News