pyelasticsearch converts transparently between Python datastructures and JSON.
- In request bodies, all the standard conversions are made: strings, numeric types, nulls, etc.
- We convert
dateinstances to the format ES understands:
dateobjects are taken to represent midnight on their day.
- Python sets are converted to ES lists.
You can customize JSON conversion by setting the
json_encoder attribute on an
Connection pooling saves setting up a whole new TCP connection for each ES request, dropping latency by an order of magnitude. The ElasticSearch object is thread-safe; to take best advantage of connection pooling, create one instance, and share it among all threads. At most, the object will hold a number of connections to each node equal to the number of threads.
Load-balancing and Failover¶
ElasticSearch object can take a list of node URLs
on construction. This lets us balance load and maintain availability when
nodes go down: pyelasticsearch will randomly choose a server URL for each
request. If a node fails to respond before a timeout period elapses, it is
assumed down and not tried again for awhile. Meanwhile, pyelasticsearch will
retry the request on a different node if
max_retries was set to something
greater than zero at construction. If all nodes are marked as down,
pyelasticsearch will loosen its standards and try sending requests to them,
marking them alive if they respond.
All methods that correspond to ES calls take an arbitrary set of kwargs that
can be used to pass query string parameters directly to ES. Certain kwargs
(called out by the
@es_kwargs decorator) are explicitly recognized as being
claimed by ES and will never be trod upon by future versions of
pyelasticsearch. To avoid conflicts, kwargs not yet so recognized should have
“es_” prepended by the caller. pyelasticsearch will strip off the “es_” and
pass the rest along to ES unscathed. Ideally, we’ll then add explicit
recognition of those args in a future release.
These “pass-through” kwargs are converted to text as follows:
- Passed unmolested
- Ints, longs, and floats
- Converted to strings via
- Lists and tuples
- Joined with commas, e.g.
- Datetimes and dates
- Datetimes are converted to ISO strings, like
2001-12-25T13:04:56, dates convert to midnight:
Anything else raises a TypeError.