作者:來自 Elastic?Fernando Briano
將 Ruby 代碼庫從 OpenSearch 客戶端遷移到 Elasticsearch 客戶端的指南。
OpenSearch Ruby 客戶端是從 7.x 版 Elasticsearch Ruby 客戶端分叉而來的,因此代碼庫相對相似。這意味著當將 Ruby 代碼庫從 OpenSearch 遷移到 Elasticsearch 時,來自相應客戶端庫的代碼看起來會非常熟悉。在這篇博文中,我將展示一個使用 OpenSearch 的示例 Ruby 應用程序以及將此代碼遷移到 Elasticsearch 的步驟。
這兩個客戶端都是根據流行的 Apache 許可證 2.0 發布的,因此它們是開源和免費軟件。Elasticsearch 的許可證最近進行了更新,Elasticsearch 和 Kibana 的核心自 8.16 版起根據 OSI 批準的開源許可證 AGPL 發布。
版本
遷移時需要考慮的一個問題是要使用哪個版本的 Elasticsearch。我們建議使用最新的穩定版本,在撰寫本文時為 8.17.0。Elasticsearch Ruby 客戶端次要版本遵循 Elasticsearch 次要版本。因此,對于 Elasticsearch 8.17.x,你可以使用 Ruby gem 的 8.17.x 版本。
OpenSearch 是從 Elasticsearch 7.10.2 分叉而來的。因此,API 可能已更改,并且可以使用不同的功能。但這超出了本文的范圍,我只會在示例應用程序中研究最常見的操作。
對于 Ruby on Rails,你可以使用官方 Elasticsearch 客戶端或 Rails 集成庫。我們建議分別遷移到 Elasticsearch 和客戶端的最新穩定版本。elasticsearch-rails gem 版本 8.0.0 支持 Rails 6.1、7.0 和 7.1 以及 Elasticsearch 8.x。
代碼
對于此示例,我按照以下步驟從 tarball 安裝 OpenSearch。下載并解壓 tarball 后,我需要設置一個初始管理員密碼,稍后我將使用該密碼來實例化客戶端。
我創建了一個包含 Gemfile 的目錄,如下所示:
source 'https://rubygems.org'gem 'opensearch-ruby'
運行 bundle install 后,我的項目就安裝了 gem。這安裝了 opensearch-ruby 版本 3.4.0,我運行的 OpenSearch 版本是 2.18.0。我在同一目錄中的 example_code.rb 文件中編寫了代碼。此文件中的初始代碼是 OpenSearch 客戶端的實例化:
require 'opensearch'client = OpenSearch::Client.new(host: 'https://localhost:9200',user: 'admin',password: ENV['OPENSEARCH_INITIAL_ADMIN_PASSWORD'],transport_options: { ssl: { verify: false } }
)
傳輸選項 ssl: { verify: false} 參數按照用戶指南傳遞,以便于測試。在生產中,應根據 OpenSearch 的部署進行設置。
自 OpenSearch 2.12.0 版起,運行安裝腳本時必須將 OPENSEARCH_INITIAL_ADMIN_PASSWORD 環境變量設置為強密碼。按照從 tarball 安裝 OpenSearch 的步驟,我在控制臺中導出了該變量,現在它可用于我的 Ruby 腳本。
確保客戶端連接到 OpenSearch 的簡單 API 是使用 cluster.health API:
puts 'HEALTH:'
pp client.cluster.health
確實有效:
$ be ruby example_code.rb
HEALTH:
{"cluster_name"=>"opensearch",
"status"=>"yellow","timed_out"=>false,"number_of_nodes"=>1,"number_of_data_nodes"=>1,
我測試了 Elasticsearch Ruby 客戶端文檔中的一些常見示例,它們按預期工作:
index = 'books'
puts 'Creating index'
response = client.indices.create(index: index)
puts response
# Creating index
# {"acknowledged"=>true, "shards_acknowledged"=>true, "index"=>"books"}puts 'Indexing a document'
document = { title: 'The Time Machine', author: 'H. G. Wells', year: 1895 }
response = client.index(index: index, body: document, refresh: true)
puts response
# Indexing document
# {"_index"=>"books", "_id"=>"esalT5MB4vnuJz5TtqOc", "_version"=>1, "result"=>"created", "forced_refresh"=>true, "_shards"=>{"total"=>2, "successful"=>1, "failed"=>0}, "_seq_no"=>0, "_primary_term"=>1}id = response['_id']
puts 'Getting document'
response = client.get(index: index, id: id)
puts response
# Getting document
# {"_index"=>"books", "_id"=>"esalT5MB4vnuJz5TtqOc", "_version"=>1, "_seq_no"=>0, "_primary_term"=>1, "found"=>true, "_source"=>{"title"= >"The Time Machine", "author"=>"H. G. Wells", "year"=>1895}}puts "Does an index exist?"
puts client.indices.exists(index: 'imaginary_index')
# Does an index exist?
# falseputs 'Processing Bulk request'
body = [{ index: { _index: 'books', data: { name: 'Leviathan Wakes', author: 'James S.A. Corey', release_date: '2011-06-02', page_count: 561 } } },{ index: { _index: 'books', data: { name: 'Hyperion', author: 'Dan Simmons', release_date: '1989-05-26', page_count: 482 } } },{ index: { _index: 'books', data: { name: 'Dune', author: 'Frank Herbert', release_date: '1965-06-01', page_count: 604 } } },{ index: { _index: 'books', data: { name: 'Dune Messiah', author: 'Frank Herbert', release_date: '1969-10-15', page_count: 331 } } },{ index: { _index: 'books', data: { name: 'Children of Dune', author: 'Frank Herbert', release_date: '1976-04-21', page_count: 408 } } },{ index: { _index: 'books', data: { name: 'God Emperor of Dune', author: 'Frank Herbert', release_date: '1981-05-28', page_count: 454 } } },{ index: { _index: 'books', data: { name: 'Consider Phlebas', author: 'Iain M. Banks', release_date: '1987-04-23', page_count: 471 } } },{ index: { _index: 'books', data: { name: 'Pandora\'s Star', author: 'Peter F. Hamilton', release_date: '2004-03-02', page_count: 768 } } },{ index: { _index: 'books', data: { name: 'Revelation Space', author: 'Alastair Reynolds', release_date: '2000-03-15', page_count: 585 } } },{ index: { _index: 'books', data: { name: 'A Fire Upon the Deep', author: 'Vernor Vinge', release_date: '1992-06-01', page_count: 613 } } },{ index: { _index: 'books', data: { name: 'Ender\'s Game', author: 'Orson Scott Card', release_date: '1985-06-01', page_count: 324 } } },{ index: { _index: 'books', data: { name: '1984', author: 'George Orwell', release_date: '1985-06-01', page_count: 328 } } },{ index: { _index: 'books', data: { name: 'Fahrenheit 451', author: 'Ray Bradbury', release_date: '1953-10-15', page_count: 227 } } },{ index: { _index: 'books', data: { name: 'Brave New World', author: 'Aldous Huxley', release_date: '1932-06-01', page_count: 268 } } },{ index: { _index: 'books', data: { name: 'Foundation', author: 'Isaac Asimov', release_date: '1951-06-01', page_count: 224 } } },{ index: { _index: 'books', data: { name: 'The Giver', author: 'Lois Lowry', release_date: '1993-04-26', page_count: 208 } } },{ index: { _index: 'books', data: { name: 'Slaughterhouse-Five', author: 'Kurt Vonnegut', release_date: '1969-06-01', page_count: 275 } } },{ index: { _index: 'books', data: { name: 'The Hitchhiker\'s Guide to the Galaxy', author: 'Douglas Adams', release_date: '1979-10-12', page_count: 180 } } },{ index: { _index: 'books', data: { name: 'Snow Crash', author: 'Neal Stephenson', release_date: '1992-06-01', page_count: 470 } } },{ index: { _index: 'books', data: { name: 'Neuromancer', author: 'William Gibson', release_date: '1984-07-01', page_count: 271 } } },{ index: { _index: 'books', data: { name: 'The Handmaid\'s Tale', author: 'Margaret Atwood', release_date: '1985-06-01', page_count: 311 } } },{ index: { _index: 'books', data: { name: 'Starship Troopers', author: 'Robert A. Heinlein', release_date: '1959-12-01', page_count: 335 } } },{ index: { _index: 'books', data: { name: 'The Left Hand of Darkness', author: 'Ursula K. Le Guin', release_date: '1969-06-01', page_count: 304 } } },{ index: { _index: 'books', data: { name: 'The Moon is a Harsh Mistress', author: 'Robert A. Heinlein', release_date: '1966-04-01', page_count: 288 } } }
]
puts client.bulk(body: body, refresh: true)
# Processing Bulk request
# {"took"=>38, "errors"=>false, "items"=>[{"index"=>{"_index"=>"books", "_id"=>" ...query = { query: { multi_match: { query: 'dune', fields: ['name'] } } }
puts 'Search results'
response = client.search(index: index, body: query)
puts response
# Search results
# {"_index"=>"books", "_id"=>"oEawT5MBOXHuGXdEu5Wu", "_score"=>2.2886353, "_source"=>{"name"=>"Dune", "author"=>"Frank Herbert", "release_date"=>"1965-06-01", "page_count"=>604}}
# {"_index"=>"books", "_id"=>"oUawT5MBOXHuGXdEu5Wu", "_score"=>1.8893257, "_source"=>{"name"=>"Dune Messiah", "author"=>"Frank Herbert", "release_date"=>"1969-10-15", "page_count"=>331}}
# {"_index"=>"books", "_id"=>"okawT5MBOXHuGXdEu5Wu", "_score"=>1.6086557, "_source"=>{"name"=>"Children of Dune", "author"=>"Frank Herbert", "release_date"=>"1976-04-21", "page_count"=>408}}
# {"_index"=>"books", "_id"=>"o0awT5MBOXHuGXdEu5Wu", "_score"=>1.40059, "_source"=>{"name"=>"God Emperor of Dune", "author"=>"Frank Herbert", "release_date"=>"1981-05-28", "page_count"=>454}}puts 'Updating document'
document = { title: 'Walkaway', author: 'Cory Doctorow', release_date: '2017' }
response = client.index(index: index, body: document, refresh: true)
id = response['_id']
response = client.update(index: index, id: id, body: { doc: { release_date: '2017-04-26' } })
puts response
# Updating document
# {"_index"=>"books", "_id"=>"degnZJMBIGr4X0Yim55L", "_version"=>2, "result"=>"updated", "_shards"=>{"total"=>2, "successful"=>1, "failed"=>0}, "_seq_no"=>26, "_primary_term"=>1}puts 'Retrieveing multiple documents'
response = client.search(index: index, body: { query: { match_all: {} }, size: 3, stored_fields: '_id' })
ids = response['hits']['hits']
ids.map { |a| a.delete('_score') }
response = client.mget(body: { docs: [{ _index: index, _id: ids }] })
puts response
# Retrieveing multiple documents
# {"docs"=>[{"_index"=>"books", "_id"=>"qeg2ZJMBIGr4X0YiiqD2", "_version"=>1, "_seq_no"=>0, "_primary_term"=>1, "found"=>true, "_source"=>{"title"=>"The Time Machine", "author"=>"H. G. Wells", "year"=>1895}}, {"_index"=>"books", "_id"=>"q-g2ZJMBIGr4X0Yii6Ah", "_version"=>1, "_seq_no"=>1, "_primary_term"=>1, "found"=>true, "_source"=>{"name"=>"Leviathan Wakes", "author"=>"James S.A. Corey", "release_date"=>"2011-06-02", "page_count"=>561}}, {"_index"=>"books", "_id"=>"rOg2ZJMBIGr4X0Yii6Ah", "_version"=>1, "_seq_no"=>2, "_primary_term"=>1, "found"=>true, "_source"=>{"name"=>"Hyperion", "author"=>"Dan Simmons", "release_date"=>"1989-05-26", "page_count"=>482}}]}puts "Count #{client.count(index: index)['count']}"
puts 'Deleting by query'
response = client.delete_by_query(index: index, body: { query: { match: { author: 'Robert A. Heinlein' } } }, refresh: true)
puts response
puts "Count #{client.count(index: index)['count']}"
# Count 26
# Deleting by query
# {"took"=>16, "timed_out"=>false, "total"=>2, "deleted"=>2, "batches"=>1, "version_conflicts"=>0, "noops"=>0, "retries"=>{"bulk"=>0, "search"=>0}, "throttled_millis"=>0, "requests_per_second"=>-1.0, "throttled_until_millis"=>0, "failures"=>[]}
# Count 24puts 'Deleting document'
response = client.delete(index: index, id: id)
puts response
# Deleting document
# {"_index"=>"books", "_id"=>"nEawT5MBOXHuGXdEu5WA", "_version"=>2, "result"=>"deleted", "_shards"=>{"total"=>2, "successful"=>1, "failed"=>0}, "_seq_no"=>25, "_primary_term"=>1}puts 'Deleting index'
response = client.indices.delete(index: index)
puts response
# Deleting index
# {"acknowledged"=>true}
遷移到 Elasticsearch
第一步是在 Gemfile 中添加 elasticsearch-ruby。運行 bundle install 后,將安裝 Elasticsearch Ruby 客戶端 gem。如果你想在完全遷移之前測試你的代碼,你可以先將 opensearch-ruby gem 保留在那里。
下一個重要步驟是客戶端實例化。這將取決于你如何運行 Elasticsearch。為了保持這些示例的類似方法,我按照下載 Elasticsearch 并在本地運行它中的步驟進行操作。
運行 bin/elasticsearch 時,Elasticsearch 將啟動并自動配置安全功能。請確保復制 elastic 用戶的密碼(但你可以通過運行 bin/elasticsearch-reset-password -u elastic 來重置它)。如果你按照此示例操作,請確保在啟動 Elasticsearch 之前停止 OpenSearch,因為它們在同一個端口上運行。
在 example_code.rb 的開頭,我注釋掉了 OpenSearch 客戶端實例并添加了 Elasticsearch 客戶端的實例:
# require 'opensearch'# client = OpenSearch::Client.new(
# host: 'https://localhost:9200',
# user: 'admin',
# password: ENV['OPENSEARCH_INITIAL_ADMIN_PASSWORD']
# transport_options: { ssl: { verify: false } }
# )require 'elasticsearch'client = Elasticsearch::Client.new(host: 'https://localhost:9200',user: ENV['ELASTICSEARCH_USER'],password: ENV['ELASTICSEARCH_PASSWORD'],transport_options: { ssl: { verify: false } }
)
如你所見,此測試場景中的代碼幾乎相同。它會根據 Elasticsearch 的部署以及你決定如何連接和驗證它而有所不同。這里與 OpenSearch 中的安全性相同,不驗證 SSL 的選項僅用于測試目的,不應在生產中使用。
設置客戶端后,我使用以下命令再次運行代碼:bundle exec ruby?? example_code.rb。一切正常!
調試
根據你的應用程序使用的 API,如果 OpenSearch 的 API 不同,則在針對 Elasticsearch 運行代碼時可能會收到錯誤。REST API 文檔是有關如何使用 API 的詳細信息的重要參考。請務必檢查你正在使用的 Elasticsearch 版本的文檔。你還可以參考 Elasticsearch::API 參考。
你可能遇到的一些 Elasticsearch 錯誤可能是:
- ArgumentError: Required argument '<ARGUMENT>' missing - 這是一個客戶端錯誤,當請求缺少必需參數時會引發此錯誤。
- Elastic::Transport::Transport::Errors::BadRequest: [400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/example/_doc] contains unrecognized parameter: [test]"}]... 此錯誤來自 Elasticsearch,這意味著客戶端代碼正在使用 Elasticsearch 無法識別的參數。
Elasticsearch 客戶端將通過服務器發送的詳細錯誤消息引發 Elasticsearch 錯誤。因此,即使對于不支持的參數或端點,錯誤也應該會告知你有什么不同。
結論
正如我們通過此示例代碼所演示的那樣,從 Ruby 的角度來看,將 Ruby 應用程序從 OpenSearch 遷移到 Elasticsearch 并不太復雜。你需要了解搜索引擎之間的版本控制和任何潛在的不同 API。但對于最常見的操作,遷移客戶端時的主要變化是在實例化中。它們在這方面都很相似,但主機和憑據的定義方式因 Stack 的部署方式而異。設置客戶端并驗證它是否連接到 Elasticsearch 后,你可以用 Elasticsearch 客戶端無縫替換 OpenSearch 客戶端。
想要獲得 Elastic 認證?了解下一次 Elasticsearch 工程師培訓何時開始!
Elasticsearch 包含新功能,可幫助你為你的用例構建最佳搜索解決方案。深入了解我們的示例筆記本以了解更多信息,開始免費云試用,或立即在你的本地機器上試用 Elastic。
原文:https://www.elastic.co/search-labs/blog/ruby-opensearch-elasticsearch-migration