The Search and Retrieve API provides a means for looking up content from an application based on search criteria including related content and returning the relevant fields for a specific use case. The application can be an iOS or Android mobile app or it could be a web application or a web site requesting the content. The Search and Retrieve API can be used to search for any content, but it is especially useful for headless content because it allows for referenced content to be included and it also offers field filtering. It is very fast thanks to it use of the Apache Solr search engine and because it retrieves content via the XperienCentral cache. The search queries are concise and written either in JSON which is usually used when querying from code or in YAML which is easier to read and write when developing or testing queries.

In This Topic

Basic usage

To send an HTTP POST request to https://yoursite.com/web/searchretrieve/v1/yaml, the search query would be:

search:
  text: The text to search for in your content

This open query will return JSON containing the contents of all content items matching the text in all languages. For example:

{
  "results" : [ {
    "_searchRetrieve" : {
      "contentItemID" : "M123",
      "language" : "nl_NL",
      "success" : true
    },
    ... JSON contents of the first result ...
  }, {
    "_searchRetrieve" : {
      "contentItemID" : "P12345",
      "language" : "en_US",
      "success" : true
    },
    ... JSON contents of the first result ...
  } ]
}

In order to use this functionality, you need XperienCentral versions R32 and higher or versions R27 and higher if you have the  Headless Add-on upgraded to version 2.2.5 or higher. The setting search_retrieve_enabled in the headless_search_retrieve section on the General tab of the Setup Tool must be enabled. A free tool for sending these types of requests is Postman.

More Sample Queries

To retrieve results 11 to 20 of content items in English with the keywords, tags, labels and terms) "news" and "sports", the query is:

search:
  keywordsAnd:
  - news
  - sports
  languages:
  - en_US
  from: 11
  to: 20

Find the Dutch content item versions for the content items with ID M1 and P26111

search:
  ids: 
  - M1
  - P26111
  languages:
  - nl

Releases and work in progress

The first version of the Search and Retrieve API is released with XperienCentral R32 on May 10, 2021. It contains the functionalities described here and is intended as a release to be used during development but it is discouraged to use it in production. In the first few weeks after that release, major functionalities will be added to make it production ready:

• Rate limitations to protect the server from Denial of Service attacks ("DDoS") and other security enhancements.
• Inclusion of referenced items in the response.
• Field filtering to only keep relevant fields in the response.
• Support for more kinds of search criteria, including custom fields.

If you have access to our Jira issue tracking system, see this issue and its sub-tasks for all technical details.

Notice that the Headless add-on which contains this functionality can be upgraded without upgrading XperienCentral. The Search and Retrieve API works on XperienCentral R27 or higher, but if using a version older than R32 the Headless add-on must be upgraded to version 2.2.5.

Developers guide

You have already seen a few examples on how to use the API. In this section you can read the details and possibilities the API offers.

Search Request

URL

A search query is made using an HTTP POST request to an URL like this:

https://yoursite.com/web/searchretrieve/v1/yaml

, where https could also be http for your site and /web could be another context path. For a search query in JSON format use an URL with json at the end:

https://yoursite.com/web/searchretrieve/v1/json

(As mentioned Postman is a free tool able of POSTing.)

Search Query

YAML crash course

The examples here are written as YAML, but also JSON is supported. YAML is quicker and clearer to read and write (when you know it) and therefore useful when trying out search queries by hand. JSON is the more logical choice when serializing an object for the search query in your code (or when you are simply more comfortable with it).

This YAML:

search:
  ids: 
  - M123
  - P12345
  languages:
  - nl

...translates to this JSON:

{
  "search": {
    "ids": [
      "M123",
      "P12345"
    ],
    "languages": [
      "nl"
    ]
  }
}

Compare the two for a few seconds... (1... 2... 3...) The content of an object (map) is on a separate level in the YAML with its own indent level and list items are denoted by hyphens ("-"). No brackets, commas and quotes needed. That's about what you need to know about YAML to read the examples. (On json2yaml.com you can easily convert between the two formats.)

Query syntax

See the examples above and Search Parameters for a list of all valid parameters and a detailed description on each one of them.

IDs and ID based lookup

If the ID of a content item is already known from a previous search request it can easily be retrieved using the ids parameter as in the example above.

There are different kinds of IDs used in XperienCentral for different purposes. Content Item IDs are used to request specific content items via this API, often in combination with a specific language. That will return the current (=active) version in that language. Version IDs are never used in the API. A Content Item can be either a Page or a Media Item. Since the numerical ID used for pages and media items might overlap this internal ID is prefixed by either "P" for Page or "M" for Media Item in this API.

Languages and "Display-on" pages

The results can be limited to one or a number of explicit languages using the languages parameter. See the Search Parameters below for details on the format.

A Media Item is rendered via its "display-on" page. The language of the Media Item decides the language version of the display-on page. Therefore ensure that display-on pages are available in all different languages.

Response

Successful responses

A successful response has a results field containing a list of results. See examples above. A successful result always starts with the following field:

    "_searchRetrieve" : {
      "contentItemID" : "M123",
      "language" : "nl_NL",
      "success" : true
    }

As you can see it contains the Content Item ID, language and a field saying it is okay. The JSON that comes thereafter is decided by the JSON presentation of the Content Item.

Response item errors

Notice: Possibly subject to change in the next version

If there is an error for a specific item, the result for that item will look like this:

    "_searchRetrieve" : {
      "error" : "A message describing the error",
      "success" : false
    }

This rarely happens, since these items are usually not even returned by the search. Therefore this situation may be fully removed so that items with errors will never be returned at all.

Failed search request

A search might fail, for example because of a malformed search query. In that case the response will contain an error message. Since an internal parse error is passed on it might look complicated like this:

{
  "error" : "Error parsing search query: Unrecognized field \"idsxxx\" (class nl.gx.product.wmaheadlesssearchretrieveapi.api.data.SearchDefinition), not marked as ignorable (9 known properties: \"keywordCategories\", \"keywordsNot\", \"from\", \"text\", \"keywordsOr\", \"keywordsAnd\", \"ids\", \"languages\", \"to\"])\n at [Source: java.io.StringReader@4e4183f0; line: 3, column: 4] (through reference chain: nl.gx.product.wmaheadlesssearchretrieveapi.api.data.SearchAndRetrieveQuery[\"search\"]->nl.gx.product.wmaheadlesssearchretrieveapi.api.data.SearchDefinition[\"idsxxx\"])"
}

Still, if you look carefully it can tell you that a field name idsxxx was used in place of one of the valid ones listed, including ids.

Settings

There is only one setting for the API in the Setup Tool ("/web/setup"), but it is an important one, since its checkbox must be checked to enable access to the API. The setting is called:

search_retrieve_enabled

Search parameters

The list below are the currently supported parameters for querying the Search & Retrieve API. This list will be expanded in future releases.

search: 
  ids:
  - P26111
  - M1

{
"search": {
  "ids": [
    "P26111",
    "M1"
 ]}
}

search: 
  languages:
  - en_US
  - nl

{
"search": {
  "languages": [
    "en_US",
    "nl"
  ]}
}

search:
  text: lorem

{
"search": {
  "text": "lorem"
  }
}

search:  
  keywordsAnd:
  - term 1
  - term 2

{
"search": {
  "keywordsAnd": [
    "term 1",
    "term 2"
  ]}
}

search:  
  keywordsOr:
  - term 1
  - term 2

{
"search": {
  "keywordsOr": [
    "term 1",
    "term 2"
  ]}
}

search:
  keywordsNot:
  - term 1
  - term 2

{
"search": {
  "keywordsNot": [
    "term 1",
    "term 2"  
  ]}
}

search:
  keywordCategories:
  - term category 1
  - term category 2

{
"search": {
  "keywordCategories": [
    "term category 1",
    "term category 2"
  ]}
}

search:
  from: 3

{
"search": {
  "from": 3
  }
}

search:
  to: 3

{
"search": {
  "to": 5
  }
}