R&R: Problem with Solr Managed resources
We have to change the list of synonyms and empty words used by the service of R & R. We do not want to load all the documents after each addition of synonyms or empty words , so we decided to use ManagedResources Solr . But by invoking the url to add a word ('palabra') to the stopwords we found a problem. We obtain different values of the list of managed stopwords (stopwords_man_es) in two consecutive invocations:
$ curl -u "{username}":"{password}" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc9bdf1dcf_741f_4074_81d1_2bbbee938778/solr/comex-collection/schema/analysis/stopwords/stopwords_man_es"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 237 0 237 0 0 205 0 --:--:-- 0:00:01 --:--:-- 211{ "responseHeader":{ "status":0, "QTime":1}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-11-23T10:30:54.006Z", "updatedSinceInit":"2015-11-23T10:38:21.533Z", "managedList":["palabra"]}}
$ curl -u "{username}":"{password}" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc9bdf1dcf_741f_4074_81d1_2bbbee938778/solr/comex-collection/schema/analysis/stopwords/stopwords_man_es"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 176 0 176 0 0 152 0 --:--:-- 0:00:01 --:--:-- 154{ "responseHeader":{ "status":0, "QTime":2}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-11-23T10:30:00.33Z", "managedList":[]}}
Retrive and rank support solr managed resources api ?
Hi - we're currently looking into this issue and we believe we will need to open up an API that we currently block so that you can force a "reload" on your collections. See the "applying changes" section of https://cwiki.apache.org/confluence/display/solr/Managed+Resources
Until that is done you can't guarantee that all cores / replicas have the same information - but given that we currently block that call it means for the time being this can't reliably work. (FYI we're looking to allow this API call very shortly for this specific reason).
As for what you're seeing with two successive calls giving different results - can you add more details around how you got into this state? Do you have something scripted to create a collection and quickly add a synonym perhaps?
Thanks,
-Scott
Thanks for the answer! I create a collection and i add stopwords.... but no 'quickly'. No script, manual invocation using curl only for testing reasons.
Steps after load configuration (with managedstopword list in queryanalizer) ...
1) create collection:
default@S4RCD11 ~ $ curl -X POST -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/admin/collections" -d "action=CREATE&name=comex-collection&collection.configName=caixabank-config"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 556 0 482 100 74 135 20 0:00:03 0:00:03 --:--:-- 136
0231001718comex-collection_shard1_replica202131comex-collection_shard1_replica1
2) List managed stopword list "spanish" (no words, correct):
default@S4RCD11 ~ $ curl -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 179 0 179 0 0 143 0 --:--:-- 0:00:01 --:--:-- 145{ "responseHeader":{ "status":0, "QTime":144}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-12-14T10:06:11.087Z", "managedList":[]}}
3) Add word "palabra" to managed stopword list "spanish" (status 0 - no error, correct):
default@S4RCD11 ~ $ curl -X POST -H "Content-Type: application/json" -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish" --data-binary '["palabra"]'
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 66 0 55 100 11 51 10 0:00:01 0:00:01 --:--:-- 51{ "responseHeader":{ "status":0, "QTime":9}}
4) List managed stopword list "spanish" (no words, INCORRECT):
default@S4RCD11 ~ $ curl -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 179 0 179 0 0 147 0 --:--:-- 0:00:01 --:--:-- 151{ "responseHeader":{ "status":0, "QTime":116}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-12-14T10:06:11.112Z", "managedList":[]}}
5) List managed stopword list "spanish" RETRY (no words, INCORRECT):
default@S4RCD11 ~ $ curl -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 177 0 177 0 0 162 0 --:--:-- 0:00:01 --:--:-- 164{ "responseHeader":{ "status":0, "QTime":1}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-12-14T10:06:11.112Z", "managedList":[]}}
6) List managed stopword list "spanish" RETRY (1 word, CORRECT):
default@S4RCD11 ~ $ curl -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 237 0 237 0 0 220 0 --:--:-- 0:00:01 --:--:-- 223{ "responseHeader":{ "status":0, "QTime":1}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-12-14T10:06:11.087Z", "updatedSinceInit":"2015-12-14T10:07:30.833Z", "managedList":["palabra"]}}
7) List managed stopword list "spanish" RETRY (no words, INCORRECT):
default@S4RCD11 ~ $ curl -u "e7f5aa8d-bbcb-4ac5-8bc4-ea883e7713f0":"ljEjZJH63AHy" "https://gateway.watsonplatform.net/retrieve-and-rank/api/v1/solr_clusters/sc48f94378_fd0a_4606_911e_8d71b56e98f9/solr/comex-collection/schema/analysis/stopwords/spanish"
Response:
% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 177 0 177 0 0 166 0 --:--:-- 0:00:01 --:--:-- 169{ "responseHeader":{ "status":0, "QTime":1}, "wordSet":{ "initArgs":{"ignoreCase":false}, "initializedOn":"2015-12-14T10:06:11.112Z", "managedList":[]}}