Problems converting Doc object to/from json

[undercertainty]

I'm having problems converting Docs to and from json. (I'm also getting errors with pickling in some cases, but the json problem is more reproducible.)

The docstring for .from_json() says that it takes a dict, but when I send a dict generated by to_json(), a doesn't apply to a 'dict' object error is raised.

Perhaps I'm using the methods wrongly, but I'm not sure how.

TIA

How to reproduce the behaviour

Python 3.11.4 (v3.11.4:d2340ef257, Jun 6 2023, 19:15:51) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>> import spacy
>>> nlp = spacy.load('en_core_web_sm')
>>> d=nlp("Dave saw Mary.")
>>> d
Dave saw Mary.
>>> j=d.to_json()
>>> spacy.tokens.doc.Doc.from_json(j)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: descriptor 'from_json' for 'spacy.tokens.doc.Doc' objects doesn't apply to a 'dict' object
>>> 

Your Environment

Info about spaCy

• spaCy version: 3.7.4
• Platform: macOS-14.3.1-x86_64-i386-64bit
• Python version: 3.11.4
• Pipelines: en_core_web_sm (3.7.1)

[svlandeg]

So this error message is confusing/misleading, because there's really just one thing you'd need to change in your code for this to work, and that is to call from_json on a Doc object instead of on the class directly:

spacy.tokens.doc.Doc(nlp.vocab).from_json(j)

and then your snippet should work. Or, similarly, look at the example from the docs:

from spacy.tokens import Doc
doc = nlp("All we have to decide is what to do with the time that is given us.")
doc_json = doc.to_json()
deserialized_doc = Doc(nlp.vocab).from_json(doc_json)
assert deserialized_doc.text == doc.text == doc_json["text"]