Troubleshooting Elasticsearch and Quepid

Connecting Elasticsearch to Quepid

The traditional way to connect Quepid to Elasticsearch is to configure Elasticsearch to accept requests from the browser, and is documented below. However if your Elasticsearch cluster is set up to only accept API requests, then you can deploy the Quepid ES Proxy alongside Quepid to tunnel the requests to your Elasticsearch Cluster.

Elasticsearch Accepting Browser Requests

You will need to configure Elasticsearch to accept requests from the browser using CORS. To enable CORS, add the following to elasticsearch's config file. Usually, this file is located near the elasticsearch executable at config/elasticsearch.yml.

http.cors:
  enabled: true
  allow-origin: /https?:\/\/localhost(:[0-9]+)?/

Elasticsearch w/ Basic auth

If you are using Basic Authentication, ie a URL in the format of [http|https]://[username:password@][host][:port]/[collectionName]/_search then you will need to add:

http.cors:
  enabled: true
  allow-origin: /https?:\/\/localhost(:3000)?/
  allow-credentials: true
  allow-headers: "X-Requested-With, Content-Type, Content-Length, if-modified-since, Authorization"

The last two lines allow for the authentication to happen via headers, which is the way Elasticsearch likes it. Notice the allow-origin is set up to only allow access from localhost port 3000. If you don't care, just use "*".

Setup SSH Port Forwarding to a Locked Down Cluster

Faced with a locked down Elasticsearch cluster? For example, using AWS Elasticsearch that has a private subnet? One option is to use a local ssh port forward into the cluster that forwards requests to your local http://localhost:9200/<index>/_search, and use that URL in Quepid.

ssh -i ~/.ssh/<gateway-machine-security-file>.pem -L9200:<es-cluster-ip>:443 ec2-user@<gateway-machine>

The -L parameter is critical, the -i is machine dependent.

This approach is replaced by using the Quepid ES Proxy

Demo Elasticsearch Indexes with the TMDB dataset

We now have a setup of Elasticsearch deployed with the TMDB dataset.

• http://quepid-elasticsearch.dev.o19s.com:9206/tmdb/_search is ES7

We have deployed a search template that you can play with:

curl -X POST "http://quepid-elasticsearch.dev.o19s.com:9206/_render/template?pretty" -H 'Content-Type: application/json' -d'
{
  "id": "tmdb-title-search-template",
  "params": {
    "search_query": "star",
    "from": 0,
    "size": 1
  }
}
'

Using the template via Quepid:

{
  "id": "tmdb-title-search-template",
  "params": {
    "search_query": "#$query##",
    "from": 0,
    "size": 2
  }
}

Using the template via CURL:

curl -X GET "http://quepid-elasticsearch.dev.o19s.com:9206/tmdb/_search/template?pretty" -H 'Content-Type: application/json' -d'
{
  "id": "tmdb-title-search-template",
  "params": {
    "search_query": "star",
    "from": 0,
    "size": 2
  }
}
'