[FEATURE REQUEST] - Add Calibre Content importer and integration

[oderyn]

Calibre does a great job of maintaining my library. RYOT does a great job of tracking the books that I want to read. Since I already use Calibre to maintain my metadata, it would be great if RYOT could just use it as a metadata source.

Calibre has an API that should make this relatively straightforward. (Famous last words, I know.)

First step is the user must have the Calibre content server enabled. There are instructions here:
https://manual.calibre-ebook.com/server.html

Once that is enabled, we would need to create the integration between RYOT and Calibre. For a great reference design, see Readarr's integration here:

Like Readarr, it would be great to be able to integrate multiple Calibre libraries. For more details on Readarr's Calibre integration see:
https://wiki.servarr.com/en/readarr/settings#root-folders

(Note: Ignore the stuff about the Root Folders. That is strictly a Readarr setting.)

Integrating Calibre into RYOT could look something like this:

And then you would use it like so:

AFAICT, the API would work something like this:

Query the Calibre content server using this format:
/ajax/search/{LIBRARY}/?num=100&offset=0&sort=title&sort_order=asc&query={QUERY}&vl=

EXAMPLE: http://localhost/ajax/search/fiction/?num=100&offset=0&sort=title&sort_order=asc&query=percy&vl=

Which will return json that looks like this:

{"total_num": 11, "sort_order": "asc", "num_books_without_search": 4212, "offset": 0, "num": 11, "sort": "title", "base_url": "/ajax/search/fiction", "query": "percy jackson", "library_id": "fiction", "book_ids": [3133, 2560, 3128, 3135, 4126, 4127, 5579, 5628, 6247, 7282, 9280], "vl": ""}

This will return a list of book IDs that you can use to get information about individual books:
/ajax/book/{BOOKID}/{LIBRARY}

EXAMPLE: http://localhost/ajax/book/1091/fiction

Which returns json that looks like this:

{
    "application_id": 1091,
    "authors": [
        "Rick Riordan"
    ],
    "title": "Percy Jackson and the Olympians 1: The Lightning Thief",
    "series_index": null,
    "comments": "PERCY JACKSON AND THE OLYMPIANS, Book 1 by Rick Riordan. The story is largely based on Greek mythology. Film adaptation of Book 1 of this New York Times bestselling series, Lightening Thief, is scheduled for release on February 12, 2010.",
    "uuid": "{UUID-DISPLAYED-SHERE}",
    "user_categories": {},
    "series": null,
    "rating": 0.0,
    "last_modified": "2023-12-22T21:31:52+00:00",
    "cover": "/get/cover/1091/fiction",
    "pubdate": "2007-05-01T06:46:47+00:00",
    "author_sort": "Riordan, Rick",
    "publisher": "",
    "author_sort_map": {
        "Rick Riordan": "Riordan, Rick"
    },
    "thumbnail": "/get/thumb/1091/fiction",
    "identifiers": {
        "isbn": "9788983781345",
        "google": "lN-sRQAACAAJ"
    },
    "languages": [
        "kor"
    ],
    "title_sort": "Percy Jackson and the Olympians 1: The Lightning Thief",
    "timestamp": "2011-11-29T20:11:03+00:00",
    "tags": [
        "Juvenile Fiction",
        "Fantasy & Magic"
    ],
    "format_metadata": {
        "mobi": {
            "path": "/books/fiction/Rick Riordan/Percy Jackson and the Olympians 1_ The Lightning Thief (1091)/Percy Jackson and the Olympians 1_ The Lig - Rick Riordan.mobi",
            "size": 441010,
            "mtime": "2022-09-12T20:41:43+00:00"
        }
    },
    "formats": [
        "mobi"
    ],
    "main_format": {
        "mobi": "/get/mobi/1091/fiction"
    },
    "other_formats": {},
    "category_urls": {
        "authors": {
            "Rick Riordan": "/ajax/books_in/617574686f7273/393633/fiction"
        },
        "series": {},
        "publisher": {},
        "languages": {},
        "tags": {
            "Juvenile Fiction": "/ajax/books_in/74616773/3736/fiction",
            "Fantasy & Magic": "/ajax/books_in/74616773/3737/fiction"
        }
    }
}

Note: I edited the JSON to remove any thing that might be a personal identifier. If there are errors in the JSON, it is likely due to my edits.

And, lastly, you can also retrieve book covers from Calibre as well:
/get/cover/{BOOKID}/{LIBRARY}

EXAMPLE: http://localhost/get/cover/1091/fiction

Which returns a cover graphic.

Unfortunately, Calibre does not have full API documentation, but you can see all of the available endpoints here:
https://github.com/kovidgoyal/calibre/blob/master/src/calibre/srv/ajax.py

And the code is actually decently commented.

Happy to help test this.

[IgnisDa]

So if I'm understanding this correctly, you're wanting to add calibre as a provider, correct?

[oderyn]

Correct. I've curated the metadata in my Calibre library, but it falls short as a media tracker. It is really a data management tool whereas RYOT seems more about tracking books that I want to read, am reading, and have read.

[IgnisDa]

I won't be adding calibre as a provider.

But adding it as an import (for books you've already completed) and integration (for books that are currently in progress) option does make sense. What do you think?

[oderyn]

Getting any kind of Calibre integration would be great.

I had envisioned using the Caliber integration as a way to tell RYOT what books I already own so that I could better organize them into collections, making it easier to browse through my library to determine what I wanted to read next -- which is why I thought Calibre being a provider made some sense.

Regardless, I imagine that I could make the integration or import work for my use case.

I appreciate everything you're doing here. This is, by far, the best of these types of apps out there.

[IgnisDa]

Adding Calibre as a provider would require me to think up as implementation of custom providers and I do not really want to implement that since I would not be using that feature myself. Adding importers/integrations is easy since I have already done it.

Thanks for the kind words, I would appreciate a donation too 😝.

[oderyn]

Happy to. :)