Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

design: avoid creating a name:value mapping each time we evaluate expression #233

Open
zyang-tudb opened this issue Aug 4, 2022 · 3 comments
Assignees

Comments

@zyang-tudb
Copy link
Contributor

zyang-tudb commented Aug 4, 2022

Currently we have to create a name:value mapping when evaluating expression against each row. Then every time we get a certain column value we have to look up this map. This is inefficient and unnecessary:

eval(expr)(ec.withVars(df.schema.map(_._1).zip(record).toMap)) match {

@zyang-tudb
Copy link
Contributor Author

@LianxinGao @xuanying2020

@LianxinGao
Copy link
Contributor

Currently we have to create a name:value mapping when evaluating expression against each row. Then every time we get a certain column value we have to look up this map. This is inefficient and unnecessary:

eval(expr)(ec.withVars(df.schema.map(_._1).zip(record).toMap)) match {

Yes, in pr #227 , I noticed this problem. It is a key function, fix it first?

@zyang-tudb
Copy link
Contributor Author

It's not a easy problem if you think about it. We might need to revamp the way we evaluate expression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants