Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance with efficient use of FieldMappingCache #389

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

V1NAY8
Copy link
Contributor

@V1NAY8 V1NAY8 commented Sep 16, 2021

This small change will improve the performance of FieldMappingCache

Existing drawbacks:

  • field_mapping_cache is initialized on every yield of itertuples and iterrows
  • Whenever eland dataframe is used to repr() or print() the field_mapping_cache is re initialized on every call.

Now, field_mapping_cache is initialized once per query_compiler, that too only when ETL such as to_pandas() or itertuples or iterrows or repr() or print() is done.

Also, by default scripted_fields are considered as "object". Previously we were finding source field, then if it's not source field, an exception is raised and then its data type is assigned as "object".

This also improves the readability of code.

@sethmlarson Take a look :)

@elasticmachine
Copy link

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

@sethmlarson
Copy link
Contributor

jenkins test this please

@sethmlarson
Copy link
Contributor

jenkins test this please

@V1NAY8
Copy link
Contributor Author

V1NAY8 commented Mar 31, 2022

Ok, So, the tests are good. So, a simple rebase should be fine.

@sethmlarson
Copy link
Contributor

jenkins test this please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants