We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hello,
Can't import tokenizer created in python and saved to JSON as recomended in method KerasTokenizer.fromJson
Please help me to resolve the issue. What could be done wrong?
Thank you,
Please describe our issue, along with:
tokenizer_json = tokenizer.to_json() with io.open('tokenizer.json', 'w', encoding='utf-8') as f: f.write(json.dumps(tokenizer_json, ensure_ascii=False))
importing to Java project by the command:
KerasTokenizer.fromJson(TOKEN_PATH);
as a result an object of tokenizer should be created.
Exception appeared
Exception in thread "main" org.nd4j.shade.jackson.databind.exc.MismatchedInputException: Cannot construct instance of java.util.HashMap (although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('{"class_name": "Tokenizer", "config": {"num_words": 10, "filters": "!"#$%&()*+,-./:;<=>?@[]^_{|}~tn", "lower": true, "split": " ", "char_level": false, "oov_token": null, "document_count": 23, "word_counts": "{"url1": 12, "url2": 12, "url3": 10, "url4": 16, "url5": 15, "url6": 16}", "word_docs": "{"url1": 12, "url2": 10, "url3": 10, "url4": 16, "url5": 14, "url6": 13}", "index_docs": "{"4": 12, "5": 10, "6": 10, "1": 16, "3": 14, "2": 13}", "index_word": "{"1": "url4", "2": "url6", "3": "url5", "4": "url1", "5": "url2", "6": "url3"}", "word_index": "{"url4": 1, "url6": 2, "url5": 3, "url1": 4, "url2": 5, "url3": 6}"}}') at [Source: (String)""{\"class_name\": \"Tokenizer\", \"config\": {\"num_words\": 10, \"filters\": \"!\"#$%&()*+,-.\/:;<=>?@[]^_{|}~tn", "lower": true, "split": " ", "char_level": false, "oov_token": null, "document_count": 23, "word_counts": "{"url1": 12, "url2": 12, "url3": 10, "url4": 16, "url5": 15, "url6": 16}", "word_docs": "{"url1": 12, "url2": 10, "url3": 10, "url4": 16, "url5": 14, "url6": 13}", "index_docs": "{"4": 12, "5": 10, "6": 10, "1": 16, "3"[truncated 246 chars]; line: 1, column: 1] at org.nd4j.shade.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63) at org.nd4j.shade.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1728) at org.nd4j.shade.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1353) at org.nd4j.shade.jackson.databind.deser.std.StdDeserializer._deserializeFromString(StdDeserializer.java:311) at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:444) at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32) at org.nd4j.shade.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) at org.nd4j.shade.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674) at org.nd4j.shade.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629) at org.nd4j.shade.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3612) at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelUtils.parseJsonString(KerasModelUtils.java:400) at org.deeplearning4j.nn.modelimport.keras.preprocessing.text.KerasTokenizer.fromJson(KerasTokenizer.java:107) at com.example.importh5.ImportKerasModel.main(ImportKerasModel.java:40)
java.util.HashMap
{|}~tn", "lower": true, "split": " ", "char_level": false, "oov_token": null, "document_count": 23, "word_counts": "{"url1": 12, "url2": 12, "url3": 10, "url4": 16, "url5": 15, "url6": 16}", "word_docs": "{"url1": 12, "url2": 10, "url3": 10, "url4": 16, "url5": 14, "url6": 13}", "index_docs": "{"4": 12, "5": 10, "6": 10, "1": 16, "3": 14, "2": 13}", "index_word": "{"1": "url4", "2": "url6", "3": "url5", "4": "url1", "5": "url2", "6": "url3"}", "word_index": "{"url4": 1, "url6": 2, "url5": 3, "url1": 4, "url2": 5, "url3": 6}"}}') at [Source: (String)""{\"class_name\": \"Tokenizer\", \"config\": {\"num_words\": 10, \"filters\": \"!\"#$%&()*+,-.\/:;<=>?@[]^_
Please indicate relevant versions, including, if relevant:
Where applicable, please also provide:
version 1.0.0-M2.1
If you'd like to help us fix the issue by contributing some code, but would like guidance or help in doing so, please mention it!
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Hello,
Can't import tokenizer created in python and saved to JSON as recomended in method KerasTokenizer.fromJson
Please help me to resolve the issue. What could be done wrong?
Thank you,
Issue Description
Please describe our issue, along with:
Tokenizer, created by
tokenizer_json = tokenizer.to_json()
with io.open('tokenizer.json', 'w', encoding='utf-8') as f:
f.write(json.dumps(tokenizer_json, ensure_ascii=False))
importing to Java project by the command:
KerasTokenizer.fromJson(TOKEN_PATH);
as a result an object of tokenizer should be created.
Exception appeared
Exception in thread "main" org.nd4j.shade.jackson.databind.exc.MismatchedInputException: Cannot construct instance of
java.util.HashMap
(although at least one Creator exists): no String-argument constructor/factory method to deserialize from String value ('{"class_name": "Tokenizer", "config": {"num_words": 10, "filters": "!"#$%&()*+,-./:;<=>?@[]^_{|}~tn", "lower": true, "split": " ", "char_level": false, "oov_token": null, "document_count": 23, "word_counts": "{"url1": 12, "url2": 12, "url3": 10, "url4": 16, "url5": 15, "url6": 16}", "word_docs": "{"url1": 12, "url2": 10, "url3": 10, "url4": 16, "url5": 14, "url6": 13}", "index_docs": "{"4": 12, "5": 10, "6": 10, "1": 16, "3": 14, "2": 13}", "index_word": "{"1": "url4", "2": "url6", "3": "url5", "4": "url1", "5": "url2", "6": "url3"}", "word_index": "{"url4": 1, "url6": 2, "url5": 3, "url1": 4, "url2": 5, "url3": 6}"}}') at [Source: (String)""{\"class_name\": \"Tokenizer\", \"config\": {\"num_words\": 10, \"filters\": \"!\"#$%&()*+,-.\/:;<=>?@[]^_
{|}~tn", "lower": true, "split": " ", "char_level": false, "oov_token": null, "document_count": 23, "word_counts": "{"url1": 12, "url2": 12, "url3": 10, "url4": 16, "url5": 15, "url6": 16}", "word_docs": "{"url1": 12, "url2": 10, "url3": 10, "url4": 16, "url5": 14, "url6": 13}", "index_docs": "{"4": 12, "5": 10, "6": 10, "1": 16, "3"[truncated 246 chars]; line: 1, column: 1]at org.nd4j.shade.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
at org.nd4j.shade.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1728)
at org.nd4j.shade.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1353)
at org.nd4j.shade.jackson.databind.deser.std.StdDeserializer._deserializeFromString(StdDeserializer.java:311)
at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:444)
at org.nd4j.shade.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32)
at org.nd4j.shade.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at org.nd4j.shade.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
at org.nd4j.shade.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629)
at org.nd4j.shade.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3612)
at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelUtils.parseJsonString(KerasModelUtils.java:400)
at org.deeplearning4j.nn.modelimport.keras.preprocessing.text.KerasTokenizer.fromJson(KerasTokenizer.java:107)
at com.example.importh5.ImportKerasModel.main(ImportKerasModel.java:40)
Version Information
Please indicate relevant versions, including, if relevant:
Additional Information
Where applicable, please also provide:
version 1.0.0-M2.1
Contributing
If you'd like to help us fix the issue by contributing some code, but would
like guidance or help in doing so, please mention it!
The text was updated successfully, but these errors were encountered: