Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Searching for something with ( breaks it #12

Open
ghost opened this issue Aug 7, 2019 · 7 comments
Open

Searching for something with ( breaks it #12

ghost opened this issue Aug 7, 2019 · 7 comments

Comments

@ghost
Copy link

ghost commented Aug 7, 2019

Open Network Traffic Debugger

Open https://standard.open-contracting.org/latest/en/search/?q=json(&check_keywords=yes&area=default#

Watch the request to the server 500
@ghost
Copy link
Author

ghost commented Aug 7, 2019

Standard Search: Searching for something with ( breaks it

@kindly
Copy link
Contributor

kindly commented Oct 8, 2019

We currently use text query in elasticsearch

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html

This parses the search term and errors if there are problems with it. A single open ( will cause an issue as elastic-search can not parse this.

We should probably just catch the error and supply feedback to the user as to why the search failed.

@ghost
Copy link
Author

ghost commented Oct 17, 2019

@kindly Can you check that the simple search feature in elastic does actually support multi-term search? Ta

@robredpath
Copy link
Member

We should probably just catch the error and supply feedback to the user as to why the search failed.

I'm not sure. If I'm searching docs, I don't think I expect to be able to use any sort of advanced search operators

@jpmckinney
Copy link
Contributor

Copying comment by @robredpath in Trello:

Great! So we think there's a choice to be made here between:

  • Using a simple search feature in elastic that treats special chars as normal characters that can be searched on, or
  • Keeping the current featureful search that we use, and exposing UI to allow users to understand the advanced search operators. This UI would require translation, of course.

@kindly is going to check that the simple search feature in elastic does actually support multi-term search

@jpmckinney
Copy link
Contributor

FYI, simple_query_string is fine and has the desired behavior: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html

@jpmckinney
Copy link
Contributor

Noting that queries with / also break, which are quite common given the character appears in JSON paths.

$ zgrep -Eo 'query_string":{"query":"[^"]+"' /var/log/elasticsearch/*
elasticsearch-2018-08-14-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-14-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-14-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-14-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-14-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/procurementMethod"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/moo"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/moo"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/moo"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/moo"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"tender/moo"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-08-15-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-10-24-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-10-24-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-10-24-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-10-24-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-10-24-1.log.gz:query_string":{"query":"\"
elasticsearch-2018-12-06-1.log.gz:query_string":{"query":"contract/suppliers"
elasticsearch-2018-12-06-1.log.gz:query_string":{"query":"contract/suppliers"
elasticsearch-2018-12-06-1.log.gz:query_string":{"query":"contract/suppliers"
elasticsearch-2018-12-06-1.log.gz:query_string":{"query":"contract/suppliers"
elasticsearch-2018-12-06-1.log.gz:query_string":{"query":"contract/suppliers"
elasticsearch-2018-12-17-1.log.gz:query_string":{"query":"AAAA-MM-DDThh:mm:ssZ"
elasticsearch-2018-12-17-1.log.gz:query_string":{"query":"AAAA-MM-DDThh:mm:ssZ"
elasticsearch-2018-12-17-1.log.gz:query_string":{"query":"AAAA-MM-DDThh:mm:ssZ"
elasticsearch-2018-12-17-1.log.gz:query_string":{"query":"AAAA-MM-DDThh:mm:ssZ"
elasticsearch-2018-12-17-1.log.gz:query_string":{"query":"AAAA-MM-DDThh:mm:ssZ"
elasticsearch-2019-02-06-1.log.gz:query_string":{"query":"application/pdf"
elasticsearch-2019-02-06-1.log.gz:query_string":{"query":"application/pdf"
elasticsearch-2019-02-06-1.log.gz:query_string":{"query":"application/pdf"
elasticsearch-2019-02-06-1.log.gz:query_string":{"query":"application/pdf"
elasticsearch-2019-02-06-1.log.gz:query_string":{"query":"application/pdf"
elasticsearch-2019-02-19-1.log.gz:query_string":{"query":"https://openopps.com/tenders/architectural-services-for-the-construction-of-disc-iii-new-offices-and-laboratories-at-harwell-campus-didcot/ocds-0c46vo-0001-d0f20c14-afc2-4a87-a649-9e4dcdac7ec6?format=json"
elasticsearch-2019-02-19-1.log.gz:query_string":{"query":"https://openopps.com/tenders/architectural-services-for-the-construction-of-disc-iii-new-offices-and-laboratories-at-harwell-campus-didcot/ocds-0c46vo-0001-d0f20c14-afc2-4a87-a649-9e4dcdac7ec6?format=json"
elasticsearch-2019-02-19-1.log.gz:query_string":{"query":"https://openopps.com/tenders/architectural-services-for-the-construction-of-disc-iii-new-offices-and-laboratories-at-harwell-campus-didcot/ocds-0c46vo-0001-d0f20c14-afc2-4a87-a649-9e4dcdac7ec6?format=json"
elasticsearch-2019-02-19-1.log.gz:query_string":{"query":"https://openopps.com/tenders/architectural-services-for-the-construction-of-disc-iii-new-offices-and-laboratories-at-harwell-campus-didcot/ocds-0c46vo-0001-d0f20c14-afc2-4a87-a649-9e4dcdac7ec6?format=json"
elasticsearch-2019-02-19-1.log.gz:query_string":{"query":"https://openopps.com/tenders/architectural-services-for-the-construction-of-disc-iii-new-offices-and-laboratories-at-harwell-campus-didcot/ocds-0c46vo-0001-d0f20c14-afc2-4a87-a649-9e4dcdac7ec6?format=json"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-02-1.log.gz:query_string":{"query":"tender/item"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TEST L)IVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TEST L)IVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TEST L)IVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TEST L)IVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TEST L)IVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TES (TIVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TES (TIVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TES (TIVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TES (TIVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"TES (TIVE"
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-08-07-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-09-05-1.log.gz:query_string":{"query":"json("
elasticsearch-2019-10-30-1.log.gz:query_string":{"query":"validator}"
elasticsearch-2019-10-30-1.log.gz:query_string":{"query":"validator}"
elasticsearch-2019-10-30-1.log.gz:query_string":{"query":"validator}"
elasticsearch-2019-10-30-1.log.gz:query_string":{"query":"validator}"
elasticsearch-2019-10-30-1.log.gz:query_string":{"query":"validator}"
elasticsearch-2019-11-15-1.log.gz:query_string":{"query":"UN/CEFACT"
elasticsearch-2019-11-15-1.log.gz:query_string":{"query":"UN/CEFACT"
elasticsearch-2019-11-15-1.log.gz:query_string":{"query":"UN/CEFACT"
elasticsearch-2019-11-15-1.log.gz:query_string":{"query":"UN/CEFACT"
elasticsearch-2019-11-15-1.log.gz:query_string":{"query":"UN/CEFACT"
elasticsearch-2019-11-29-1.log.gz:query_string":{"query":"UN/LOCODE"
elasticsearch-2019-11-29-1.log.gz:query_string":{"query":"UN/LOCODE"
elasticsearch-2019-11-29-1.log.gz:query_string":{"query":"UN/LOCODE"
elasticsearch-2019-11-29-1.log.gz:query_string":{"query":"UN/LOCODE"
elasticsearch-2019-11-29-1.log.gz:query_string":{"query":"UN/LOCODE"
elasticsearch-2019-12-17-1.log.gz:query_string":{"query":"tender/mainProcurementCategory"
elasticsearch-2019-12-17-1.log.gz:query_string":{"query":"tender/mainProcurementCategory"
elasticsearch-2019-12-17-1.log.gz:query_string":{"query":"tender/mainProcurementCategory"
elasticsearch-2019-12-17-1.log.gz:query_string":{"query":"tender/mainProcurementCategory"
elasticsearch-2019-12-17-1.log.gz:query_string":{"query":"tender/mainProcurementCategory"
elasticsearch-2019-12-27-1.log.gz:query_string":{"query":"[SCHEMA VERSION"
elasticsearch-2019-12-27-1.log.gz:query_string":{"query":"[SCHEMA VERSION"
elasticsearch-2019-12-27-1.log.gz:query_string":{"query":"[SCHEMA VERSION"
elasticsearch-2019-12-27-1.log.gz:query_string":{"query":"[SCHEMA VERSION"
elasticsearch-2019-12-27-1.log.gz:query_string":{"query":"[SCHEMA VERSION"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-02-28-1.log.gz:query_string":{"query":"contratacion cerrada}"
elasticsearch-2020-03-09-1.log.gz:query_string":{"query":"/VZRequest/VZ/CisloEvidencni"
elasticsearch-2020-03-09-1.log.gz:query_string":{"query":"/VZRequest/VZ/CisloEvidencni"
elasticsearch-2020-03-09-1.log.gz:query_string":{"query":"/VZRequest/VZ/CisloEvidencni"
elasticsearch-2020-03-09-1.log.gz:query_string":{"query":"/VZRequest/VZ/CisloEvidencni"
elasticsearch-2020-03-09-1.log.gz:query_string":{"query":"/VZRequest/VZ/CisloEvidencni"
elasticsearch-2020-04-08-1.log.gz:query_string":{"query":"procurmentOfficer\"
elasticsearch-2020-04-08-1.log.gz:query_string":{"query":"procurmentOfficer\"
elasticsearch-2020-04-08-1.log.gz:query_string":{"query":"procurmentOfficer\"
elasticsearch-2020-04-08-1.log.gz:query_string":{"query":"procurmentOfficer\"
elasticsearch-2020-04-08-1.log.gz:query_string":{"query":"procurmentOfficer\"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-06-22-1.log.gz:query_string":{"query":"contracts/items/classification/scheme"
elasticsearch-2020-07-02-1.log.gz:query_string":{"query":"OBJECT_DESCR/JUSTIFICATION"
elasticsearch-2020-07-02-1.log.gz:query_string":{"query":"OBJECT_DESCR/JUSTIFICATION"
elasticsearch-2020-07-02-1.log.gz:query_string":{"query":"OBJECT_DESCR/JUSTIFICATION"
elasticsearch-2020-07-02-1.log.gz:query_string":{"query":"OBJECT_DESCR/JUSTIFICATION"
elasticsearch-2020-07-02-1.log.gz:query_string":{"query":"OBJECT_DESCR/JUSTIFICATION"
elasticsearch-2020-07-23-1.log.gz:query_string":{"query":"use cases\\"
elasticsearch-2020-07-23-1.log.gz:query_string":{"query":"use cases\\"
elasticsearch-2020-07-23-1.log.gz:query_string":{"query":"use cases\\"
elasticsearch-2020-07-23-1.log.gz:query_string":{"query":"use cases\\"
elasticsearch-2020-07-23-1.log.gz:query_string":{"query":"use cases\\"
elasticsearch-2020-08-03-1.log.gz:query_string":{"query":"http://www.perucontrata.com.pe/"
elasticsearch-2020-08-03-1.log.gz:query_string":{"query":"http://www.perucontrata.com.pe/"
elasticsearch-2020-08-03-1.log.gz:query_string":{"query":"http://www.perucontrata.com.pe/"
elasticsearch-2020-08-03-1.log.gz:query_string":{"query":"http://www.perucontrata.com.pe/"
elasticsearch-2020-08-03-1.log.gz:query_string":{"query":"http://www.perucontrata.com.pe/"
elasticsearch-2020-08-10-1.log.gz:query_string":{"query":"tender/additionalProcurementCategories"
elasticsearch-2020-08-10-1.log.gz:query_string":{"query":"tender/additionalProcurementCategories"
elasticsearch-2020-08-10-1.log.gz:query_string":{"query":"tender/additionalProcurementCategories"
elasticsearch-2020-08-10-1.log.gz:query_string":{"query":"tender/additionalProcurementCategories"
elasticsearch-2020-08-10-1.log.gz:query_string":{"query":"tender/additionalProcurementCategories"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-05-1.log.gz:query_string":{"query":"tender/awardCriteria"
elasticsearch-2020-10-20-1.log.gz:query_string":{"query":"Buyer / Vendor Unique Hash"
elasticsearch-2020-10-20-1.log.gz:query_string":{"query":"Buyer / Vendor Unique Hash"
elasticsearch-2020-10-20-1.log.gz:query_string":{"query":"Buyer / Vendor Unique Hash"
elasticsearch-2020-10-20-1.log.gz:query_string":{"query":"Buyer / Vendor Unique Hash"
elasticsearch-2020-10-20-1.log.gz:query_string":{"query":"Buyer / Vendor Unique Hash"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jpmckinney @kindly @robredpath