Proof-of-concept: custom Bukuserver bookmarklet (with site-specific logic) #798
LeXofLeviafan
started this conversation in
Show and tell
Replies: 2 comments
-
|
I hardly use buku in the browser, so will wait for users to provide some feedback on this. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
…I've added another comment for some optional modifications. I've also implemented support for another usecase: taking webpage screenshots.
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Usecase
Suppose you want to use your bookmarks as, well… bookmarks. That is, to track your reading progress (e.g. on web-fiction sites).
There's a little problem though: that convenient
@bukubookmarklet you've got… is completely useless here! Because it's almost never going to open the bookmark record that you actually need.First of all, chapters tend to have their own URLs; so the Bukuserver bookmarklet will always try to create a new bookmark record, instead of opening the one you have for the story.
Secondly, stories themselves tend to not have a fixed URL; there's typically a template (or even multiple) that the URL follows, including a fixed ID and a possibly arbitrary part which tends to be ignored entirely by the server (those tend to include a most recent title slug, for user convenience… this isn't limited to just web-fiction sites either – forum engines in general often handle threads the same way).
So, we'll need a bookmarklet that can at the very least calculate a reliable permalink of a story, both from its main page and its chapters.
…And while we're at it, it'd be nice to get some other site-custom stuff in there (such as story tags, publishing status, or even the story image – assuming we actually have a way to store it, anyway).
Setup & usage
Here's the gist containing a (proof-of-concept) implementation of a custom bookmarklet. (See also the implementation notes at the bottom.)
In order to have it running, you'll need to complete the following steps:
Install the nightly version of Buku. (Base functionality should work on v4.9 with no issue, but tags support was only added to the bookmarklet endpoint quite recently.)
Install a userscript manager in your browser; preferably Tampermonkey. (There's multiple reasons I've implemented this functionality in a userscript, the most obvious one being that there's only so much JavaScript you can actually cram into a URL.)
Add the file
Bukuserver bookmarklet plugin.user.jsfrom the gist as a userscript. (This will likely involve opening the dashboard in your userscript manager, clicking the "New Userscript" button, and replacing the default template with the contents of the file from the gist… and then saving it, of course. Alternatively, just click the "Raw" button on the gist file: it appears to trigger Tampermonkey script installation for.user.jsfiles.)You may also want to add
Bukuserver bookmarklet plugin tester.user.jsthe same way. (It's used to display the extra data collected by the other userscript in your bookmark dialog.)Finally, add/replace your Bukuserver bookmarklet with the contents of
bookmarklet.urlfrom the gist. (It's generated from the contents ofbookmarklet.js, where{{url}}was replaced withhttp://localhost:5000/bookmarklet.)After completing these steps, you can use the new bookmarklet exactly the same way you were using the old one; it will just collect bookmark data better (on the supported websites)!
Screenshots (usage example)
Each of the following examples will include:
Bukuserver bookmarklet plugin.user.js)Bukuserver bookmarklet plugin tester.user.js)note: the screenshot contains a dialog window, vertically resized to showcase rendered content
Note that the
_url:parameter (visible on each screenshot) displays the original URL of the page that the bookmarklet was invoked from.The two userscripts are entirely independent of each other, so generating custom URLs does not require you to install the 2nd userscript, and testing them out does not require you to install the 1st one.
creating a bookmark (from the story page)
(the image at the bottom is incomplete because it didn't fit in the screenshot 😅)
creating a bookmark (from the chapter page)
editing an existing bookmark (from the chapter page)
(the bookmark was created from the story page – note the tags and description… also note that the matching
id=parameter will likely differ if you're going to try it on your DB)Q&A
So what is this thing anyway? And how does it work?
It's a userscript (custom script that is injected by a browser extension that runs on arbitrary websites), invoked by the bookmarklet to enhance your generated URL. And also another userscript that renders extra parameters in your dialog.
Both are proof-of-concept (i.e. created not as a final product but to show off what can be achieved potentially; thus limited in scope – in this case to a handful of websites, which are hardcoded into the script, and of course you can't really save data which was not supported by already existing Buku versions).
As for how this works; the userscript (run at the time of page load) creates a function, which the bookmarklet attempts to invoke; the function returns a new, modified URL that gets invoked instead of the URL generated by the bookmarklet. The new URL contains updated
url=parameter (the story permalink), and depending on the data available on the current page it also includes story metadata (e.g. tags) or the current reading progress (e.g. chapter number). Part of this data is not supported by Buku, but it can still be accessed by JavaScript – hence the 2nd userscript, which displays this data in the dialog.The data generation logic was custom-made for each supported website on the individual basis.
(For more information check out the implementation notes.)
What websites is this supported on?
I implemented this logic for the following web-fiction sites:
fiction.live,forums.spacebattles.com,www.royalroad.com,www.scribblehub.com,www.fictionpress.com,www.fanfiction.net,archiveofourown.org.And before you ask: the reason why my 1st userscript specifies a blanket "anywhere" as its URL (instead of listing each these websites) is simply because that's what the "full implementation" would be doing (since it would not be hardcoded to support these sites only).
So… what happens on the other websites? Or if I disable the userscript altogether?
If the new bookmarklet fails to invoke userscript function for any reason (including it not existing or throwing an error), it simply falls back to the original behaviour of the old bookmarklet.
(If you use the bookmarklet on an unsupported URL – i.e. not a story URL, – that's likely going to happen there as well.)
Alright, these new fields are nice I guess (especially the image); but they aren't saved so what's the point?
Well… First of all, since this is a demo, it can (and does) showcase what can (and hopefully will) be supported eventually; such as including custom fields (e.g. images) in the bookmark data. (This feature has been requested more than once as of late, and I've been meaning to draft support for such a feature to be included in future Buku versions.)
As for now, I guess you can copy whatever data you want into the description field; that's probably the best you can get until Buku supports extra fields anyway 😅
I clicked the bookmarklet and the browser showed me some weird confirmation page with a countdown! WTF?!
So… while most sites don't care that much, some outright refuse to display their images elsewhere. Thus, in order to support these images, I had to convert them into dataurls instead (which requires downloading them first)… but downloading images is not the same as showing them, so these sites can block that as well!
Because of that, my script is requesting the userscript manager (i.e. the browser extension) to use its special privileges to download it anyway; so it's making sure the user actually wants it to happen.
(TL;DR click "Allow for domain" to enable downloading images on this site.)
The bookmarklet dialog doesn't open on some sites anymore! What's going on?
You must be using Firefox, yes? If you pay close attention, you'll notice that instead of a dialog, there's a small toolbar popping up at the top of the page, saying something about "preventing a pop-up window". It's actually behaviour enabled by a browser setting ("Block pop-up windows"), and clicking "Options" > "Allow pop-ups for …" on that toolbar allows you to disable this setting on the current site.
(As for "why this happens only on some sites"… Oddly enough, it doesn't happen when triggered directly by a user action – like a click; but if it's delayed by a background action, such as a download/network request, then Firefox no longer treats it as something caused by a user action.)
…Alright, I've checked it out; now what?
🤔
First of all, the idea is to show off what could be potentially supported for any website (via per-site settings stored in an extended Buku DB, and fetched via Bukuserver API)… But implementing such general support comes with certain challenges; and while I have a general idea of what can be done about it, actually doing so would take certain time and effort.
In the meantime, you still have a userscript that actually does work as described, if only on a limited set of websites; so you can use it already. And feel free to extend it, if you have the basic know-how; implementing the minimum functionality (permalink generation) on any particular website comes down to adding a line like this into the
CUSTOMdictionary:(check out already existing entries for more specific examples… and, again, make sure to read implementation notes)
Beta Was this translation helpful? Give feedback.
All reactions