Elasticsearch (Search service)
Back to home
On this page
Elasticsearch is a distributed RESTful search engine built for the cloud.
See the Elasticsearch documentation for more information.
Use a framework
If you use one of the following frameworks, follow its guide:
Supported versions
From version 7.11 onward:
Premium Service
Elasticsearch isnโt included in any Platform.sh plan. You need to add it separately at an additional cost. To add Elasticsearch, contact Sales.
The following premium versions are supported:
Grid | Dedicated Gen 3 | Dedicated Gen 2 |
---|---|---|
|
None available |
|
You can select the major and minor version.
Patch versions are applied periodically for bug fixes and the like. When you deploy your app, you always get the latest available patches.
Deprecated versions
The following versions are still available in your projects for free, but they’re at their end of life and are no longer receiving security updates from upstream.
Grid | Dedicated Gen 3 | Dedicated Gen 2 |
---|---|---|
|
|
|
To ensure your project remains stable in the future, switch to a premium version.
Alternatively, you can switch to one of the latest, free versions of OpenSearch. To do so, follow the same procedure as for upgrading.
Relationship reference
Example information available through the PLATFORM_RELATIONSHIPS
environment variable
or by running platform relationships
.
Note that the information about the relationship can change when an app is redeployed or restarted or the relationship is changed. So your apps should only rely on the PLATFORM_RELATIONSHIPS
environment variable directly rather than hard coding any values.
{
"username": null,
"scheme": "http",
"service": "elasticsearch",
"fragment": null,
"ip": "123.456.78.90",
"hostname": "azertyuiopqsdfghjklm.elasticsearch.service._.eu-1.platformsh.site",
"port": 9200,
"cluster": "azertyuiopqsdf-main-7rqtwti",
"host": "essearch.internal",
"rel": "elasticsearch",
"path": null,
"query": [],
"password": "ChangeMe",
"type": "elasticsearch:8.5",
"public": false,
"host_mapped": false
}
For premium versions, the service type is elasticsearch-enterprise
.
Usage example
1. Configure the service
To define the service, use the elasticsearch
type:
# The name of the service container. Must be unique within a project.
<SERVICE_NAME>:
type: elasticsearch:<VERSION>
disk: 256
If youโre using a premium version, use the elasticsearch-enterprise
type instead.
Note that changing the name of the service replaces it with a brand new service and all existing data is lost. Back up your data before changing the service.
2. Add the relationship
To define the relationship, use the following configuration:
# Relationships enable access from this app to a given service.
# The example below shows simplified configuration leveraging a default service
# (identified from the relationship name) and a default endpoint.
# See the Application reference for all options for defining relationships and endpoints.
relationships:
<SERVICE_NAME>:
You can define <SERVICE_NAME>
as you like, so long as it’s unique between all defined services
and matches in both the application and services configuration.
The example above leverages default endpoint configuration for relationships. That is, it uses default endpoints behind-the-scenes, providing a relationship (the network address a service is accessible from) that is identical to the name of that service.
Depending on your needs, instead of default endpoint configuration, you can use explicit endpoint configuration.
With the above definition, the application container now has access to the service via the relationship <RELATIONSHIP_NAME>
and its corresponding PLATFORM_RELATIONSHIPS
environment variable.
Example configuration
Service definition
# The name of the service container. Must be unique within a project.
elasticsearch:
type: elasticsearch:8.5
disk: 256
App configuration
# Relationships enable access from this app to a given service.
# The example below shows simplified configuration leveraging a default service
# (identified from the relationship name) and a default endpoint.
# See the Application reference for all options for defining relationships and endpoints.
relationships:
elasticsearch:
If you’re using a premium version,
use the elasticsearch-enterprise
type in the service definition.
Use in app
To use the configured service in your app, add a configuration file similar to the following to your project.
Note that configuration for premium versions may differ slightly.
package sh.platform.languages.sample;
import org.elasticsearch.action.admin.indices.refresh.RefreshRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshResponse;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import sh.platform.config.Config;
import sh.platform.config.Elasticsearch;
import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.Supplier;
import static java.util.concurrent.ThreadLocalRandom.current;
public class ElasticsearchSample implements Supplier<String> {
@Override
public String get() {
StringBuilder logger = new StringBuilder();
// Create a new config object to ease reading the Platform.sh environment variables.
// You can alternatively use getenv() yourself.
Config config = new Config();
Elasticsearch elasticsearch = config.getCredential("elasticsearch", Elasticsearch::new);
// Create an Elasticsearch client object.
RestHighLevelClient client = elasticsearch.get();
try {
String index = "animals";
String type = "mammals";
// Index a few document.
final List<String> animals = Arrays.asList("dog", "cat", "monkey", "horse");
for (String animal : animals) {
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("name", animal);
jsonMap.put("age", current().nextInt(1, 10));
jsonMap.put("is_cute", current().nextBoolean());
IndexRequest indexRequest = new IndexRequest(index, type)
.id(animal).source(jsonMap);
client.index(indexRequest, RequestOptions.DEFAULT);
}
RefreshRequest refresh = new RefreshRequest(index);
// Force just-added items to be indexed
RefreshResponse refreshResponse = client.indices().refresh(refresh, RequestOptions.DEFAULT);
// Search for documents.
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(QueryBuilders.termQuery("name", "dog"));
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices(index);
searchRequest.source(sourceBuilder);
SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);
logger.append("<p>");
for (SearchHit hit : search.getHits()) {
String id = hit.getId();
final Map<String, Object> source = hit.getSourceAsMap();
logger.append(String.format("Result: id: <code>%s</code> source: <code>%s</code>", id, source)).append('\n');
}
logger.append("</p>");
// Delete documents.
for (String animal : animals) {
client.delete(new DeleteRequest(index, type, animal), RequestOptions.DEFAULT);
}
} catch (IOException exp) {
throw new RuntimeException("An error when execute Elasticsearch: " + exp.getMessage());
}
return logger.toString();
}
}
const elasticsearch = require("elasticsearch");
const config = require("platformsh-config").config();
exports.usageExample = async function () {
const credentials = config.credentials("elasticsearch");
const client = new elasticsearch.Client({
host: `${credentials.host}:${credentials.port}`,
});
const index = "my_index";
const type = "People";
// Index a few document.
const names = ["Ada Lovelace", "Alonzo Church", "Barbara Liskov"];
const message = {
refresh: "wait_for",
body: names.flatMap((name) => [
{ index: { _index: index, _type: type } },
{ name },
]),
};
await client.bulk(message);
// Search for documents.
const response = await client.search({
index,
q: "name:Barbara Liskov",
});
const outputRows = response.hits.hits
.map(
({ _id: id, _source: { name } }) =>
`<tr><td>${id}</td><td>${name}</td></tr>\n`
)
.join("\n");
// Clean up after ourselves.
await Promise.allSettled(
response.hits.hits.map(({ _id: id }) =>
client.delete({
index: index,
type: type,
id,
})
)
);
return `
<table>
<thead>
<tr>
<th>ID</th><th>Name</th>
</tr>
</thhead>
<tbody>
${outputRows}
</tbody>
</table>
`;
};
<?php
declare(strict_types=1);
use Elasticsearch\ClientBuilder;
use Platformsh\ConfigReader\Config;
// Create a new config object to ease reading the Platform.sh environment variables.
// You can alternatively use getenv() yourself.
$config = new Config();
// Get the credentials to connect to the Elasticsearch service.
$credentials = $config->credentials('elasticsearch');
try {
// The Elasticsearch library lets you connect to multiple hosts.
// On Platform.sh Standard there is only a single host so just
// register that.
$hosts = [
[
'scheme' => $credentials['scheme'],
'host' => $credentials['host'],
'port' => $credentials['port'],
]
];
// Create an Elasticsearch client object.
$builder = ClientBuilder::create();
$builder->setHosts($hosts);
$client = $builder->build();
$index = 'my_index';
$type = 'People';
// Index a few document.
$params = [
'index' => $index,
'type' => $type,
];
$names = ['Ada Lovelace', 'Alonzo Church', 'Barbara Liskov'];
foreach ($names as $name) {
$params['body']['name'] = $name;
$client->index($params);
}
// Force just-added items to be indexed.
$client->indices()->refresh(array('index' => $index));
// Search for documents.
$result = $client->search([
'index' => $index,
'type' => $type,
'body' => [
'query' => [
'match' => [
'name' => 'Barbara Liskov',
],
],
],
]);
if (isset($result['hits']['hits'])) {
print <<<TABLE
<table>
<thead>
<tr><th>ID</th><th>Name</th></tr>
</thead>
<tbody>
TABLE;
foreach ($result['hits']['hits'] as $record) {
printf("<tr><td>%s</td><td>%s</td></tr>\n", $record['_id'], $record['_source']['name']);
}
print "</tbody>\n</table>\n";
}
// Delete documents.
$params = [
'index' => $index,
'type' => $type,
];
$ids = array_map(function($row) {
return $row['_id'];
}, $result['hits']['hits']);
foreach ($ids as $id) {
$params['id'] = $id;
$client->delete($params);
}
} catch (Exception $e) {
print $e->getMessage();
}
import elasticsearch
from platformshconfig import Config
def usage_example():
# Create a new Config object to ease reading the Platform.sh environment variables.
# You can alternatively use os.environ yourself.
config = Config()
# Get the credentials to connect to the Elasticsearch service.
credentials = config.credentials('elasticsearch')
try:
# The Elasticsearch library lets you connect to multiple hosts.
# On Platform.sh Standard there is only a single host so just register that.
hosts = {
"scheme": credentials['scheme'],
"host": credentials['host'],
"port": credentials['port']
}
# Create an Elasticsearch client object.
client = elasticsearch.Elasticsearch([hosts])
# Index a few documents
es_index = 'my_index'
es_type = 'People'
params = {
"index": es_index,
"type": es_type,
"body": {"name": ''}
}
names = ['Ada Lovelace', 'Alonzo Church', 'Barbara Liskov']
ids = {}
for name in names:
params['body']['name'] = name
ids[name] = client.index(index=params["index"], doc_type=params["type"], body=params['body'])
# Force just-added items to be indexed.
client.indices.refresh(index=es_index)
# Search for documents.
result = client.search(index=es_index, body={
'query': {
'match': {
'name': 'Barbara Liskov'
}
}
})
table = '''<table>
<thead>
<tr><th>ID</th><th>Name</th></tr>
</thead>
<tbody>'''
if result['hits']['hits']:
for record in result['hits']['hits']:
table += '''<tr><td>{0}</td><td>{1}</td><tr>\n'''.format(record['_id'], record['_source']['name'])
table += '''</tbody>\n</table>\n'''
# Delete documents.
params = {
"index": es_index,
"type": es_type,
}
for name in names:
client.delete(index=params['index'], doc_type=params['type'], id=ids[name]['_id'])
return table
except Exception as e:
return e
Note
When you create an index on Elasticsearch,
don’t specify the number_of_shards
or number_of_replicas
settings in your Elasticsearch API call.
These values are set automatically based on available resources.
Authentication
By default, Elasticsearch has no authentication. No username or password is required to connect to it.
Starting with Elasticsearch 7.2 you may optionally enable HTTP Basic authentication.
To do so, include the following in your .platform/services.yaml
configuration:
# The name of the service container. Must be unique within a project.
elasticsearch:
type: elasticsearch:8.5
disk: 2048
configuration:
authentication:
enabled: true
If you’re using a premium version,
use the elasticsearch-enterprise
type.
That enables mandatory HTTP Basic auth on all requests.
The credentials are available in any relationships that point at that service,
in the username
and password
properties.
Note that the information about the relationship can change when an app is redeployed or restarted or the relationship is changed. So your apps should only rely on the PLATFORM_RELATIONSHIPS
environment variable directly rather than hard coding any values.
This functionality is generally not required if Elasticsearch isn’t exposed on its own public HTTP route.
However, certain applications may require it, or it allows you to safely expose Elasticsearch directly to the web.
To do so, add a route to .platform/routes.yaml
that has elasticsearch:elasticsearch
as its upstream
(where elasticsearch
is whatever you named the service).
For example:
"https://es.{default}/":
type: upstream
upstream: "elasticsearch:elasticsearch"
Plugins
Elasticsearch offers a number of plugins.
To enable them, list them under the configuration.plugins
key in your .platform/services.yaml
file, like so:
# The name of the service container. Must be unique within a project.
elasticsearch:
type: elasticsearch:8.5
disk: 1024
configuration:
plugins:
- analysis-icu
If you’re using a premium version,
use the elasticsearch-enterprise
type.
In this example you’d have the ICU analysis plugin and Python script support plugin.
If there is a publicly available plugin you need that isn’t listed here, contact support.
Available plugins
This is the complete list of official Elasticsearch plugins that can be enabled:
Plugin | Description | 2.4 | 5.x | 6.x | 7.x | 8.x |
---|---|---|---|---|---|---|
analysis-icu |
Support ICU Unicode text analysis | * | * | * | * | * |
analysis-nori |
Integrates Lucene Nori analysis module into Elasticsearch | * | * | * | ||
analysis-kuromoji |
Japanese language support | * | * | * | * | * |
analysis-smartcn |
Smart Chinese Analysis Plugins | * | * | * | * | * |
analysis-stempel |
Stempel Polish Analysis Plugin | * | * | * | * | * |
analysis-phonetic |
Phonetic analysis | * | * | * | * | * |
analysis-ukrainian |
Ukrainian language support | * | * | * | * | |
cloud-aws |
AWS Cloud plugin, allows storing indices on AWS S3 | * | ||||
delete-by-query |
Support for deleting documents matching a given query | * | ||||
discovery-multicast |
Ability to form a cluster using TCP/IP multicast messages | * | ||||
ingest-attachment |
Extract file attachments in common formats (such as PPT, XLS, and PDF) | * | * | * | * | |
ingest-user-agent |
Extracts details from the user agent string a browser sends with its web requests | * | * | |||
lang-javascript |
JavaScript language plugin, allows the use of JavaScript in Elasticsearch scripts | * | ||||
lang-python |
Python language plugin, allows the use of Python in Elasticsearch scripts | * | * | |||
mapper-annotated-text |
Adds support for text fields with markup used to inject annotation tokens into the index | * | * | * | ||
mapper-attachments |
Mapper attachments plugin for indexing common file types | * | * | |||
mapper-murmur3 |
Murmur3 mapper plugin for computing hashes at index-time | * | * | * | * | * |
mapper-size |
Size mapper plugin, enables the _size meta field |
* | * | * | * | * |
repository-s3 |
Support for using S3 as a repository for Snapshot/Restore | * | * | * | * | |
transport-nio |
Support for NIO transport | * | * |
Plugin removal
Removing plugins previously added in your .platform/services.yaml
file doesn’t automatically uninstall them from your Elasticsearch instances.
This is deliberate, as removing a plugin may result in data loss or corruption of existing data that relied on that plugin.
Removing a plugin usually requires reindexing.
To permanently remove a previously enabled plugin, upgrade the service to create a new instance of Elasticsearch and migrate to it. In most cases it isn’t necessary as an unused plugin has no appreciable impact on the server.
Upgrading
The Elasticsearch data format sometimes changes between versions in incompatible ways. Elasticsearch doesn’t include a data upgrade mechanism as it’s expected that all indexes can be regenerated from stable data if needed. To upgrade (or downgrade) Elasticsearch, use a new service from scratch.
There are two ways to do so.
Destructive
In your .platform/services.yaml
file, change the version and name of your Elasticsearch service.
Be sure to also update the reference to the now changed service name in its corresponding application’s relationship
block.
When you push that to Platform.sh, the old service is deleted and a new one with the new name is created with no data. You can then have your application reindex data as appropriate.
This approach has the downsides of temporarily having an empty Elasticsearch instance, which your application may or may not handle gracefully, and needing to rebuild your index afterward. Depending on the size of your data that could take a while.
Transitional
With a transitional approach, you temporarily have two Elasticsearch services.
Add a second Elasticsearch service with the new version, a new name, and give it a new relationship in .platform.app.yaml
.
You can optionally run in that configuration for a while to allow your application to populate indexes in the new service as well.
Once you’re ready to switch over, remove the old Elasticsearch service and relationship. You may optionally have the new Elasticsearch service use the old relationship name if that’s easier for your app to handle. Your application is now using the new Elasticsearch service.
This approach has the benefit of never being without a working Elasticsearch instance. On the downside, it requires two running Elasticsearch servers temporarily, each of which consumes resources and needs adequate disk space. Depending on the size of your data, that may be a lot of disk space.