Chewy

Chewy is an ODM and wrapper for the official Elasticsearch client.

Why Chewy?

Multi-model indices.
Index classes are independent from ORM/ODM models. Now, implementing e.g. cross-model autocomplete is much easier. You can just define the index and work with it in an object-oriented style. You can define several types for index - one per indexed model.
Every index is observable by all the related models.
Most of the indexed models are related to other and sometimes it is necessary to denormalize this related data and put at the same object. For example, you need to index an array of tags together with an article. Chewy allows you to specify an updateable index for every model separately - so corresponding articles will be reindexed on any tag update.
Bulk import everywhere.
Chewy utilizes the bulk ES API for full reindexing or index updates. It also uses atomic updates. All the changed objects are collected inside the atomic block and the index is updated once at the end with all the collected objects. See Chewy.strategy(:atomic) for more details.
Powerful querying DSL.
Chewy has an ActiveRecord-style query DSL. It is chainable, mergeable and lazy, so you can produce queries in the most efficient way. It also has object-oriented query and filter builders.
Support for ActiveRecord, Mongoid and Sequel.

Installation

Add this line to your application's Gemfile:

gem 'chewy'

And then execute:

$ bundle

Or install it yourself as:

$ gem install chewy

Usage

Client settings

There are two ways to configure the Chewy client: the Chewy.settings hash and chewy.yml

You can create this file manually or run rails g chewy:install.

# config/initializers/chewy.rbChewy.settings={host: 'localhost:9250'}# do not use environments

# config/chewy.yml# separate environment configstest:
host: 'localhost:9250'prefix: 'test'development:
host: 'localhost:9200'

The resulting config merges both hashes. Client options are passed as is to Elasticsearch::Transport::Client except for the :prefix, which is used internally by Chewy to create prefixed index names:

Chewy.settings={prefix: 'test'}UsersIndex.index_name# => 'test_users'

The logger may be set explicitly:

Chewy.logger=Logger.new

See config.rb for more details.

Index definition

Create /app/chewy/users_index.rb

classUsersIndex < Chewy::Indexend

Add one or more types mapping

classUsersIndex < Chewy::Indexdefine_typeUser.active# or just model instead_of scope: define_type Userend

Newly-defined index type class is accessible via UsersIndex.user or UsersIndex::User

Add some type mappings

classUsersIndex < Chewy::Indexdefine_typeUser.active.includes(:country,:badges,:projects)dofield:first_name,:last_name# multiple fields without additional optionsfield:email,analyzer: 'email'# Elasticsearch-related optionsfield:country,value: ->(user){user.country.name}# custom value procfield:badges,value: ->(user){user.badges.map(&:name)}# passing array values to indexfield:projectsdo# the same block syntax for multi_field, if `:type` is specifiedfield:titlefield:description# default data type is `string`# additional top-level objects passed to value proc:field:categories,value: ->(project,user){project.categories.map(&:name)ifuser.active?}endfield:rating,type: 'integer'# custom data typefield:created,type: 'date',include_in_all: false,value: ->{created_at}# value proc for source object contextendend

See here for mapping definitions.

Add some index- and type-related settings. Analyzer repositories might be used as well. See Chewy::Index.settings docs for details:

classUsersIndex < Chewy::Indexsettingsanalysis: {analyzer: {email: {tokenizer: 'keyword',filter: ['lowercase']}}}define_typeUser.active.includes(:country,:badges,:projects)dorootdate_detection: falsedotemplate'about_translations.*',type: 'string',analyzer: 'standard'field:first_name,:last_namefield:email,analyzer: 'email'field:country,value: ->(user){user.country.name}field:badges,value: ->(user){user.badges.map(&:name)}field:projectsdofield:titlefield:descriptionendfield:about_translations,type: 'object'# pass object type explicitly if necessaryfield:rating,type: 'integer'field:created,type: 'date',include_in_all: false,value: ->{created_at}endendend

See index settings here. See root object settings here.

See mapping.rb for more details.

Add model-observing code

classUser < ActiveRecord::Baseupdate_index('users#user'){self}# specifying index, type and back-reference# for updating after user save or destroyendclassCountry < ActiveRecord::Basehas_many:usersupdate_index('users#user'){users}# return single object or collectionendclassProject < ActiveRecord::Baseupdate_index('users#user'){userifuser.active?}# you can return even `nil` from the back-referenceendclassBadge < ActiveRecord::Basehas_and_belongs_to_many:usersupdate_index('users'){users}# if index has only one type# there is no need to specify updated typeendclassBook < ActiveRecord::Baseupdate_index(->(book){"books#book_#{book.language}"}){self}# dynamic index and type with proc.# For book with language == "en"# this code will generate `books#book_en`end

Also, you can use the second argument for method name passing:

update_index('users#user',:self)update_index('users#user',:users)

In the case of a belongs_to association you may need to update both associated objects, previous and current:

classCity < ActiveRecord::Basebelongs_to:countryupdate_index('cities#city'){self}update_index'countries#country'do# For the latest active_record changed values are# already in `previous_changes` hash,# but for mongoid you have to use `changes` hashprevious_changes['country_id'] || countryendend

You can observe Sequel models in the same way as ActiveRecord:

classUser < Sequel::Modelupdate_index('users#user'){self}end

However, to make it work, you must load the chewy plugin into Sequel model:

Sequel::Model.plugin:chewy_observe# for all models, or...User.plugin:chewy_observe# just for User

Multi (nested) and object field types

To define an objects field you can simply nest fields in the DSL:

field:projectsdofield:titlefield:descriptionend

This will automatically set the type or root field to object. You may also specify type: 'objects' explicitly.

To define a multi field you have to specify any type except for object or nested in the root field:

field:full_name,type: 'string',value: ->{full_name.strip}dofield:ordered,analyzer: `ordered`field:untouched,index: 'not_analyzed'end

The value: option for internal fields would no longer be effective.

Crutches™ technology

Assume you are defining your index like this (product has_many categories through product_categories):

classProductsIndex < Chewy::Indexdefine_typeProduct.includes(:categories)dofield:namefield:category_names,value: ->(product){product.categories.map(&:name)}# or shorter just -> { categories.map(&:name) }endend

Then the Chewy reindexing flow would look like the following pseudo-code (even in Mongoid):

Product.includes(:categories).find_in_batches(1000)do |batch|
bulk_body=batch.mapdo |object|
{name: object.name,category_names: object.categories.map(&:name)}.to_jsonend# here we are sending every batch of data to ESChewy.client.bulkbulk_bodyend

But in Rails 4.1 and 4.2 you may face a problem with slow associations (take a look at rails/rails#19423). Also, there might be really complicated cases when associations are not applicable.

Then you can replace Rails associations with Chewy Crutches™ technology:

classProductsIndex < Chewy::Indexdefine_typeProduct.includes(:categories)docrutch:categoriesdo |collection| # collection here is a current batch of products# data is fetched with a lightweight query without objects initializationdata=ProductCategory.joins(:category).where(product_id: collection.map(&:id)).pluck(:product_id,'categories.name')# then we have to convert fetched data to appropriate format# this will return our data in structure like:# {123 => ['sweets', 'juices'], 456 => ['meat']}data.each.with_object({}){ |(id,name),result| (result[id] ||= []).push(name)}endfield:name# simply use crutch-fetched data as a value:field:category_names,value: ->(product,crutches){crutches.categories[product.id]}endend

An example flow would look like this:

Product.includes(:categories).find_in_batches(1000)do |batch|
crutches[:categories]=ProductCategory.joins(:category).where(product_id: batch.map(&:id)).pluck(:product_id,'categories.name').each.with_object({}){ |(id,name),result| (result[id] ||= []).push(name)}bulk_body=batch.mapdo |object|
{name: object.name,category_names: crutches[:categories][object.id]}.to_jsonendChewy.client.bulkbulk_bodyend

So Chewy Crutches™ technology is able to increase your indexing performance in some cases up to a hundredfold or even more depending on your associations complexity.

Types access

You can access index-defined types with the following API:

UsersIndex::User# => UsersIndex::UserUsersIndex.type_hash['user']# => UsersIndex::UserUsersIndex.user# => UsersIndex::UserUsersIndex.types# => [UsersIndex::User]UsersIndex.type_names# => ['user']

Index manipulation

UsersIndex.delete# destroy index if it existsUsersIndex.delete!UsersIndex.createUsersIndex.create!# use bang or non-bang methodsUsersIndex.purgeUsersIndex.purge!# deletes then creates indexUsersIndex::User.import# import with 0 arguments process all the data specified in type definition# literally, User.active.includes(:country, :badges, :projects).find_in_batchesUsersIndex::User.importUser.where('rating > 100')# or import specified users scopeUsersIndex::User.importUser.where('rating > 100').to_a# or import specified users arrayUsersIndex::User.import[1,2,42]# pass even ids for import, it will be handled in the most effective wayUsersIndex.import# import every defined typeUsersIndex.importuser: User.where('rating > 100')# import only active users to `user` type.# Other index types, if exists, will be imported with default scope from the type definition.UsersIndex.reset!# purges index and imports default data for all types

If the passed user is #destroyed?, or satisfies a delete_if type option, or the specified id does not exist in the database, import will perform delete from index action for this object.

define_typeUser,delete_if: :deleted_atdefine_typeUser,delete_if: ->{deleted_at}define_typeUser,delete_if: ->(user){user.deleted_at}

See actions.rb for more details.

Index update strategies

Assume you've got the following code:

classCity < ActiveRecord::Baseupdate_index'cities#city',:selfendclassCitiesIndex < Chewy::Indexdefine_typeCitydofield:nameendend

If you do something like City.first.save! you'll get an UndefinedUpdateStrategy exception instead of the object saving and index updating. This exception forces you to choose an appropriate update strategy for the current context.

If you want to return to the pre-0.7.0 behavior - just set Chewy.root_strategy = :bypass.

`:atomic`

The main strategy here is :atomic. Assume you have to update a lot of records in the db.

Chewy.strategy(:atomic)doCity.popular.map(&:do_some_update_action!)end

Using this strategy delays the index update request until the end of the block. Updated records are aggregated and the index update happens with the bulk API. So this strategy is highly optimized.

`:resque`

This does the same thing as :atomic, but asynchronously using resque. The default queue name is chewy. Patch Chewy::Strategy::Resque::Worker for index updates improving.

Chewy.strategy(:resque)doCity.popular.map(&:do_some_update_action!)end

`:sidekiq`

This does the same thing as :atomic, but asynchronously using sidekiq. Patch Chewy::Strategy::Sidekiq::Worker for index updates improving.

Chewy.strategy(:sidekiq)doCity.popular.map(&:do_some_update_action!)end

`:urgent`

The following strategy is convenient if you are going to update documents in your index one by one.

Chewy.strategy(:urgent)doCity.popular.map(&:do_some_update_action!)end

This code would perform City.popular.count requests for ES documents update.

It is convenient for use in e.g. the Rails console with non-block notation:

> Chewy.strategy(:urgent)
> City.popular.map(&:do_some_update_action!)

`:bypass`

The bypass strategy simply silences index updates.

Nesting

Strategies are designed to allow nesting, so it is possible to redefine it for nested contexts.

Chewy.strategy(:atomic)docity1.do_update!Chewy.strategy(:urgent)docity2.do_update!city3.do_update!# there will be 2 update index requests for city2 and city3endcity4..do_update!# city1 and city4 will be grouped in one index update requestend

Non-block notation

It is possible to nest strategies without blocks:

Chewy.strategy(:urgent)city1.do_update!# index updatedChewy.strategy(:bypass)city2.do_update!# update bypassedChewy.strategy.popcity3.do_update!# index updated again

Designing your own strategies

See strategy/base.rb for more details. See strategy/atomic.rb for an example.

Rails application strategies integration

There are a couple of predefined strategies for your Rails application. Initially, the Rails console uses the :urgent strategy by default, except in the sandbox case. When you are running sandbox it switches to the :bypass strategy to avoid polluting the index.

Migrations are wrapped with the :bypass strategy. Because the main behavior implies that indices are reset after migration, there is no need for extra index updates. Also indexing might be broken during migrations because of the outdated schema.

Controller actions are wrapped with the :atomic strategy with middleware just to reduce the number of index update requests inside actions.

It is also a good idea to set up the :bypass strategy inside your test suite and import objects manually only when needed, and use Chewy.massacre when needed to flush test ES indices before every example. This will allow you to minimize unnecessary ES requests and reduce overhead.

RSpec.configuredo |config|
config.before(:suite)doChewy.strategy(:bypass)endend

Index querying

scope=UsersIndex.query(term: {name: 'foo'}).filter(range: {rating: {gte: 100}}).order(created: :desc).limit(20).offset(100)scope.to_a# => will produce array of UserIndex::User or other types instancesscope.map{ |user| user.email}scope.total_count# => will return total objects countscope.per(10).page(3)# supports kaminari paginationscope.explain.map{ |user| user._explanation}scope.only(:id,:email)# returns ids and emails onlyscope.merge(other_scope)# queries could be merged

Also, queries can be performed on a type individually:

UsersIndex::User.filter(term: {name: 'foo'})# will return UserIndex::User collection only

If you are performing more than one filter or query in the chain, all the filters and queries will be concatenated in the way specified by filter_mode and query_mode respectively.

The default filter_mode is :and and the default query_mode is bool.

Available filter modes are: :and, :or, :must, :should and any minimum_should_match-acceptable value

Available query modes are: :must, :should, :dis_max, any minimum_should_match-acceptable value or float value for dis_max query with tie_breaker specified.

UsersIndex::User.filter{name == 'Fred'}.filter{age < 42}# will be wrapped with `and` filterUsersIndex::User.filter{name == 'Fred'}.filter{age < 42}.filter_mode(:should)# will be wrapped with bool `should` filterUsersIndex::User.filter{name == 'Fred'}.filter{age < 42}.filter_mode('75%')# will be wrapped with bool `should` filter with `minimum_should_match: '75%'`

See query.rb for more details.

Additional query action.

You may also perform additional actions on the query scope, such as deleting of all the scope documents:

UsersIndex.delete_allUsersIndex::User.delete_allUsersIndex.filter{age < 42}.delete_allUsersIndex::User.filter{age < 42}.delete_all

Filters query DSL

There is a test version of the filter-creating DSL:

UsersIndex.filter{name == 'Fred'}# will produce `term` filter.UsersIndex.filter{age <= 42}# will produce `range` filter.

The basis of the DSL is the expression. There are 2 types of expressions:

Simple function

UsersIndex.filter{s('doc["num"] > 1')}# script expressionUsersIndex.filter{q(query_string: {query: 'lazy fox'})}# query expression

Field-dependent composite expression Consists of the field name (with or without dot notation), a value, and an action operator between them. The field name might take additional options for passing to the resulting expression.

UsersIndex.filter{name == 'Name'}# simple field term filterUsersIndex.filter{name(:bool) == ['Name1','Name2']}# terms query with `execution: :bool` option passedUsersIndex.filter{answers.title =~ /regexp/}# regexp filter for `answers.title` field

You can combine expressions as you wish with the help of combination operators.

UsersIndex.filter{(name == 'Name') & (email == 'Email')}# combination produces `and` filterUsersIndex.filter{must(should(name =~ 'Fr').should_not(name == 'Fred') & (age == 42),email =~ /gmail\.com/) | ((roles.admin == true) & name?)}# many of the combination possibilities

There is also a special syntax for cache enabling:

UsersIndex.filter{ ~name == 'Name'}# you can apply tilde to the field nameUsersIndex.filter{ ~(name == 'Name')}# or to the whole expression# if you are applying cache to the one part of range filter# the whole filter will be cached:UsersIndex.filter{ ~(age > 42) & (age <= 50)}# You can pass cache options as a field option also.UsersIndex.filter{name(cache: true) == 'Name'}UsersIndex.filter{name(cache: false) == 'Name'}# With regexp filter you can pass _cache_keyUsersIndex.filter{name(cache: 'name_regexp') =~ /Name/}# Or notUsersIndex.filter{name(cache: true) =~ /Name/}

Compliance cheatsheet for filters and DSL expressions:

Term filter

{"term": {"name": "Fred"}}
{"not": {"term": {"name": "Johny"}}}

UsersIndex.filter{name == 'Fred'}UsersIndex.filter{name != 'Johny'}

Terms filter

{"terms": {"name": ["Fred", "Johny"]}}
{"not": {"terms": {"name": ["Fred", "Johny"]}}}
{"terms": {"name": ["Fred", "Johny"], "execution": "or"}}
{"terms": {"name": ["Fred", "Johny"], "execution": "and"}}
{"terms": {"name": ["Fred", "Johny"], "execution": "bool"}}
{"terms": {"name": ["Fred", "Johny"], "execution": "fielddata"}}

UsersIndex.filter{name == ['Fred','Johny']}UsersIndex.filter{name != ['Fred','Johny']}UsersIndex.filter{name(:|) == ['Fred','Johny']}UsersIndex.filter{name(:or) == ['Fred','Johny']}UsersIndex.filter{name(execution: :or) == ['Fred','Johny']}UsersIndex.filter{name(:&) == ['Fred','Johny']}UsersIndex.filter{name(:and) == ['Fred','Johny']}UsersIndex.filter{name(execution: :and) == ['Fred','Johny']}UsersIndex.filter{name(:b) == ['Fred','Johny']}UsersIndex.filter{name(:bool) == ['Fred','Johny']}UsersIndex.filter{name(execution: :bool) == ['Fred','Johny']}UsersIndex.filter{name(:f) == ['Fred','Johny']}UsersIndex.filter{name(:fielddata) == ['Fred','Johny']}UsersIndex.filter{name(execution: :fielddata) == ['Fred','Johny']}

Regexp filter (== and =~ are equivalent)

{"regexp": {"name.first": "s.*y"}}
{"not": {"regexp": {"name.first": "s.*y"}}}
{"regexp": {"name.first": {"value": "s.*y", "flags": "ANYSTRING|INTERSECTION"}}}

UsersIndex.filter{name.first == /s.*y/}UsersIndex.filter{name.first =~ /s.*y/}UsersIndex.filter{name.first != /s.*y/}UsersIndex.filter{name.first !~ /s.*y/}UsersIndex.filter{name.first(:anystring,:intersection) == /s.*y/}UsersIndex.filter{name.first(flags: [:anystring,:intersection]) == /s.*y/}

Prefix filter

{"prefix": {"name": "Fre"}}
{"not": {"prefix": {"name": "Joh"}}}

UsersIndex.filter{name =~ re' }UsersIndex.filter{ name !~ 'Joh'}

Exists filter

{"exists": {"field": "name"}}

UsersIndex.filter{name?}UsersIndex.filter{ !!name}UsersIndex.filter{ !!name?}UsersIndex.filter{name != nil}UsersIndex.filter{ !(name == nil)}

Missing filter

{"missing": {"field": "name", "existence": true, "null_value": false}}
{"missing": {"field": "name", "existence": true, "null_value": true}}
{"missing": {"field": "name", "existence": false, "null_value": true}}

UsersIndex.filter{ !name}UsersIndex.filter{ !name?}UsersIndex.filter{name == nil}

Range

{"range": {"age": {"gt": 42}}}
{"range": {"age": {"gte": 42}}}
{"range": {"age": {"lt": 42}}}
{"range": {"age": {"lte": 42}}}
{"range": {"age": {"gt": 40, "lt": 50}}}
{"range": {"age": {"gte": 40, "lte": 50}}}
{"range": {"age": {"gt": 40, "lte": 50}}}
{"range": {"age": {"gte": 40, "lt": 50}}}

UsersIndex.filter{age > 42}UsersIndex.filter{age >= 42}UsersIndex.filter{age < 42}UsersIndex.filter{age <= 42}UsersIndex.filter{age == (40..50)}UsersIndex.filter{(age > 40) & (age < 50)}UsersIndex.filter{age == [40..50]}UsersIndex.filter{(age >= 40) & (age <= 50)}UsersIndex.filter{(age > 40) & (age <= 50)}UsersIndex.filter{(age >= 40) & (age < 50)}

Bool filter

{"bool": {
"must": [{"term": {"name": "Name"}}],
"should": [{"term": {"age": 42}}, {"term": {"age": 45}}]
}}

UsersIndex.filter{must(name == 'Name').should(age == 42,age == 45)}

And filter

{"and": [{"term": {"name": "Name"}}, {"range": {"age": {"lt": 42}}}]}

UsersIndex.filter{(name == 'Name') & (age < 42)}

Or filter

{"or": [{"term": {"name": "Name"}}, {"range": {"age": {"lt": 42}}}]}

UsersIndex.filter{(name == 'Name') | (age < 42)}

{"not": {"term": {"name": "Name"}}}
{"not": {"range": {"age": {"lt": 42}}}}

UsersIndex.filter{ !(name == 'Name')}# or UsersIndex.filter{ name != 'Name' }UsersIndex.filter{ !(age < 42)}

Match all filter

{"match_all": {}}

UsersIndex.filter{match_all}

Has child filter

{"has_child": {"type": "blog_tag", "query": {"term": {"tag": "something"}}}
{"has_child": {"type": "comment", "filter": {"term": {"user": "john"}}}

UsersIndex.filter{has_child(:blog_tag).query(term: {tag: 'something'})}UsersIndex.filter{has_child(:comment).filter{user == 'john'}}

Has parent filter

{"has_parent": {"type": "blog", "query": {"term": {"tag": "something"}}}}
{"has_parent": {"type": "blog", "filter": {"term": {"text": "bonsai three"}}}}

UsersIndex.filter{has_parent(:blog).query(term: {tag: 'something'})}UsersIndex.filter{has_parent(:blog).filter{text == 'bonsai three'}}

See filters.rb for more details.

Faceting

Facets are an optional sidechannel you can request from Elasticsearch describing certain fields of the resulting collection. The most common use for facets is to allow the user to continue filtering specifically within the subset, as opposed to the global index.

For instance, let's request the country field as a facet along with our users collection. We can do this with the #facets method like so:

UsersIndex.filter{[...]}.facets({countries: {terms: {field: 'country'}}})

Let's look at what we asked from Elasticsearch. The facets setter method accepts a hash. You can choose custom/semantic key names for this hash for your own convenience (in this case I used the plural version of the actual field), in our case countries. The following nested hash tells ES to grab and aggregate values (terms) from the country field on our indexed records.

The response will include the :facets sidechannel:

< { ... ,"facets":{"countries":{"_type":"terms","missing":?,"total":?,"other":?,"terms":[{"term":"USA","count":?},{"term":"Brazil","count":?}, ...}}

Script fields

Script fields allow you to execute Elasticsearch's scripting languages such as groovy and javascript. More about supported languages and what scripting is here. This feature allows you to calculate the distance between geo points, for example. This is how to use the DSL:

UsersIndex.script_fields(distance: {params: {lat: 37.569976,lon: -122.351591},script: "doc['coordinates'].distanceInMiles(lat, lon)"})

Here, coordinates is a field with type geo_point. There will be a distance field for the index's model in the search result.

Script scoring

Script scoring is used to score the search results. All scores are added to the search request and combined according to boost mode and score mode. This can be useful if, for example, a score function is computationally expensive and it is sufficient to compute the score on a filtered set of documents. For example, you might want to multiply the score by another numeric field in the doc:

UsersIndex.script_score("_score * doc['my_numeric_field'].value")

Boost Factor

Boost factors are a way to add a boost to a query where documents match the filter. If you have some users who are experts and some who are regular users, you might want to give the experts a higher score and boost to the top of the search results. You can accomplish this by using the #boost_factor method and adding a boost score of 5 for an expert user:

UsersIndex.boost_factor(5,filter: {term: {type: 'Expert'}})

Objects loading

It is possible to load source objects from the database for every search result:

scope=UsersIndex.filter(range: {rating: {gte: 100}})scope.load# => scope is marked to return User instances arrayscope.load.query(...)# => since objects are loaded lazily you can complete scopescope.load(user: {scope: ->{includes(:country)}})# you can also pass loading scopes for each# possibly returned typescope.load(user: {scope: User.includes(:country)})# the second scope passing way.scope.load(scope: ->{includes(:country)})# and more common scope applied to every loaded object type.scope.only(:id).load# it is optimal to request ids only if you are not planning to use type objects

The preload method takes the same options as load and ORM/ODM objects will be loaded, but the scope will still return an array of Chewy wrappers. To access real objects use the _object wrapper method:

UsersIndex.filter(range: {rating: {gte: 100}}).preload(...).query(...).map(&:_object)

See loading.rb for more details.

`ActiveSupport::Notifications` support

Chewy has notifying the following events:

`search_query.chewy` payload

payload[:index]: requested index class
payload[:request]: request hash

`import_objects.chewy` payload

payload[:type]: currently imported type
payload[:import]: imports stats, total imported and deleted objects count:
```
{index: 30,delete: 5}
```

payload[:errors]: might not exists. Contains grouped errors with objects ids list:

{index: {'error 1 text'=>['1','2','3'],'error 2 text'=>['4']},delete: {'delete error text'=>['10','12']}}

NewRelic integration

To integrate with NewRelic you may use the following example source (config/initializers/chewy.rb):

ActiveSupport::Notifications.subscribe('import_objects.chewy')do |name,start,finish,id,payload|
metric_name="Database/ElasticSearch/import"duration=(finish - start).to_flogged="#{payload[:type]}#{payload[:import].to_a.map{ |i| i.join(':')}.join(', ')}"self.class.trace_execution_scoped([metric_name])doNewRelic::Agent.instance.transaction_sampler.notice_sql(logged,nil,duration)NewRelic::Agent.instance.sql_sampler.notice_sql(logged,metric_name,nil,duration)NewRelic::Agent.record_metric(metric_name,duration)endendActiveSupport::Notifications.subscribe('search_query.chewy')do |name,start,finish,id,payload|
metric_name="Database/ElasticSearch/search"duration=(finish - start).to_flogged="#{payload[:type].presence || payload[:index]}#{payload[:request]}"self.class.trace_execution_scoped([metric_name])doNewRelic::Agent.instance.transaction_sampler.notice_sql(logged,nil,duration)NewRelic::Agent.instance.sql_sampler.notice_sql(logged,metric_name,nil,duration)NewRelic::Agent.record_metric(metric_name,duration)endend

Rake tasks

Inside the Rails application, some index-maintaining rake tasks are defined.

rake chewy:reset # resets all the existing indices, declared in app/chewy
rake chewy:reset[users] # resets UsersIndex only
rake chewy:update # updates all the existing indices, declared in app/chewy
rake chewy:update[users] # updates UsersIndex only

rake chewy:reset performs zero-downtime reindexing as described here. So basically rake task creates a new index with uniq suffix and then simply aliases it to the common index name. The previous index is deleted afterwards (see Chewy::Index.reset! for more details).

Rspec integration

Just add require 'chewy/rspec' to your spec_helper.rb and you will get additional features: See update_index.rb for more details.

If you use DatabaseCleaner in your tests with the transaction strategy, you may run into the problem that ActiveRecord's models are not indexed automatically on save despite the fact that you set the callbacks to do this with the update_index method. The issue arises because chewy indexes data on after_commit run as default, but all after_commit callbacks are not run with the DatabaseCleaner's' transaction strategy. You can solve this issue by changing the Chewy.use_after_commit_callbacks option. Just add the following initializer in your Rails application:

#config/initializers/chewy.rbChewy.use_after_commit_callbacks= !Rails.env.test?

TODO a.k.a coming soon:

Typecasting support
Advanced (simplified) query DSL: UsersIndex.query { email == 'my@gmail.com' } will produce term query
update_all support
Maybe, closer ORM/ODM integration, creating index classes implicitly

Contributing

Fork it (http://github.com/toptal/chewy/fork)
Create your feature branch (git checkout -b my-new-feature)
Implement your changes, cover it with specs and make sure old specs are passing
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

Use the following Rake tasks to control the Elasticsearch cluster while developing.

rake elasticsearch:start # start Elasticsearch cluster on 9250 port for tests
rake elasticsearch:stop # stop Elasticsearch

Name		Name	Last commit message	Last commit date
gemfiles		gemfiles
lib		lib
spec		spec
.gitignore		.gitignore
.rspec		.rspec
.travis.yml		.travis.yml
Appraisals		Appraisals
CHANGELOG.md		CHANGELOG.md
Gemfile		Gemfile
Guardfile		Guardfile
LICENSE.txt		LICENSE.txt
README.md		README.md
Rakefile		Rakefile
chewy.gemspec		chewy.gemspec
filters		filters

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Chewy

Why Chewy?

Installation

Usage

Client settings

Index definition

Multi (nested) and object field types

Crutches™ technology

Types access

Index manipulation

Index update strategies

:atomic

:resque

:sidekiq

:urgent

:bypass

Nesting

Non-block notation

Designing your own strategies

Rails application strategies integration

Index querying

Additional query action.

Filters query DSL

Faceting

Script fields

Script scoring

Boost Factor

Objects loading

ActiveSupport::Notifications support

search_query.chewy payload

import_objects.chewy payload

NewRelic integration

Rake tasks

Rspec integration

TODO a.k.a coming soon:

Contributing

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages