Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Greater than 1024 rows of some variable types causes an encoding error #53

Open
brianstamper opened this issue Apr 12, 2022 · 1 comment

Comments

@brianstamper
Copy link

brianstamper commented Apr 12, 2022

I'm finding if a table has more than 1,024 rows that dbWriteTable has an issue with some variable types.

library(implyr)

connect_impala <- function() {
  src_impala(drv = odbc::odbc(),
             driver = 'Cloudera ODBC Driver for Impala',
             host = 'impala.xxxxxxxxxxxxx',
             port = xxxxx,
             database = 'default',
             AuthMech = 3,
             ssl = 1,
             uid = rstudioapi::askForPassword('Username'),
             pwd = rstudioapi::askForPassword('Password'))
}

cx <- connect_impala()


any_number_you_like <- 0L
set.seed(any_number_you_like)


my_test_data_1024 <- data.frame(X = sample(1024))

dbWriteTable(cx$con,
             Id(schema = 'XXXXXXX', table = 'TEST_1024'), 
             my_test_data_1024)

dbRemoveTable(cx$con, Id(schema = 'XXXXXXX', table = 'TEST_1024'))



my_test_data_1025 <- data.frame(X = sample(1025))

dbWriteTable(cx$con,
             Id(schema = 'XXXXXXX', table = 'TEST_1025'), 
             my_test_data_1025)

The first dbWriteTable is fine, but the second throws an error like this:

Error in result_insert_dataframe(rs@ptr, values, batch_rows) : 
  nanodbc/nanodbc.cpp:1617: HY000: [Cloudera][ImpalaODBC] (110) Error while executing a query in Impala: [HY000] : AnalysisException: Target table 'XXXXXXX.TEST_1025' is incompatible with source expressions.
Expression 'cast('208À€' as string)' (type: STRING) is not compatible with column 'x' (type: INT)

I've replaced the sample with versions that produce double and logical variables with similar results, but sampling from letters did not produce this error.

@brianstamper
Copy link
Author

Edited to remove tidyverse/tibble, appears to be the same with base data.frame.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant