Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: add_operation attempts to call into removed translator_class #9158

Open
1 task done
DavidSlayback opened this issue May 8, 2024 · 3 comments
Open
1 task done
Assignees
Labels
backends Issues related to all backends bug Incorrect behavior inside of ibis compiler Issues or PRs related to compilation

Comments

@DavidSlayback
Copy link

What happened?

Previously (under Ibis 7.2.0), we had defined a SAFE_CAST operation for our usage of Ibis with bigquery with the following:

import ibis
import ibis.expr.datatypes as dt
import ibis.expr.rules as rlz
from ibis.backends.bigquery.datatypes import BigQueryType
from ibis.expr.operations.core import Value as ValueOp
from ibis.expr.types import Value as ValueType


class SafeCast(ValueOp):
    """Safe cast value or column to a specific data type."""

    arg: ValueOp
    to: dt.DataType

    shape = rlz.shape_like("arg")

    @property
    def name(self):  # noqa
        return f"SAFE_CAST({self.arg.name} as {self.to})"

    @property
    def dtype(self):  # noqa
        return self.to


def safe_cast(col, dtype):
    """Safely cast a column to a specific data type."""
    return SafeCast(col, dtype).to_expr()


@ibis.bigquery.add_operation(SafeCast)
def _safe_cast_gbq(translator, expr):
    arg, target_dtype = expr.args
    compiled_arg = translator.translate(arg)
    sql_type = BigQueryType.from_ibis(target_dtype)
    return f"SAFE_CAST({compiled_arg} as {sql_type})"


ValueType.safe_cast = safe_cast

When attempting to update to Ibis 9.0.0, this broke with the error:
AttributeError: 'BigQueryCompiler' object has no attribute 'translator_class'

Not sure if this is a bug or an intended change in the way a user can add an operation to Ibis

What version of ibis are you using?

7.2.0 originally, 9.0.0 for the error

What backend(s) are you using, if any?

BigQuery

Relevant log output

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@DavidSlayback DavidSlayback added the bug Incorrect behavior inside of ibis label May 8, 2024
@gforsyth
Copy link
Member

gforsyth commented May 8, 2024

Hey @DavidSlayback -- looks like we forgot to update how add_operation works with the new sqlglot backends. We'll get that fixed up for 9.1, and I'll see if I can get you a workaround in the meantime

@gforsyth gforsyth added the bigquery The BigQuery backend label May 8, 2024
@gforsyth gforsyth added this to the 9.1 milestone May 8, 2024
@gforsyth
Copy link
Member

gforsyth commented May 8, 2024

Hey @DavidSlayback -- I forgot, we added built-in support for try_cast (or SAFE_CAST). That doesn't answer the larger issue about adding custom ops, but for your particular use-case, you can do

some_table.my_int_col.try_cast("float64")

e.g.

[ins] In [1]: import ibis

[ins] In [2]: t = ibis.memtable({"b": [1, 2]})

[ins] In [3]: ibis.to_sql(t.b.try_cast("float64"), dialect="bigquery")
Out[3]: 
SELECT
  SAFE_CAST(`t0`.`b` AS FLOAT64) AS `TryCast_b_ Float64`
FROM `ibis_pandas_memtable_cywg4g55dbgethukk5btbdeyve` AS `t0`

@gforsyth gforsyth changed the title (BigQuery) Adding operations to bigquery backend fails in Ibis 9 bug: add_operation attempt to call into removed translator_class May 8, 2024
@gforsyth gforsyth changed the title bug: add_operation attempt to call into removed translator_class bug: add_operation attempts to call into removed translator_class May 8, 2024
@DavidSlayback
Copy link
Author

Thank you! Both for pointing to existing functionality and for considering a fix next release. Overall we get a ton of use out of this project, thanks for building it 😀

@gforsyth gforsyth added backends Issues related to all backends compiler Issues or PRs related to compilation and removed bigquery The BigQuery backend labels May 10, 2024
@gforsyth gforsyth removed this from the 9.1 milestone May 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backends Issues related to all backends bug Incorrect behavior inside of ibis compiler Issues or PRs related to compilation
Projects
Status: backlog
Development

No branches or pull requests

3 participants