Remove movies already in the list, or use as feedback

[AndrewHartAR]

Great idea and easy to use.

I noticed that it gave me recommendations for movies I've already rated, and also some which I've rated quite poorly, came up as recommendations. Perhaps there could be a feedback loop on this. Perhaps it could have its own feedback loop, something like:

"then, respond to these suggestions. call out reasons that movies may not match this pattern. call out movies which are already listed. then, generate a new list."

[harrisonpim]

Hi @AndrewHartAR, thanks for the feedback! I'm glad you've enjoyed playing with it 😁

TBH, I'm not surprised that some recommendations were poor, or that they overlapped with films which you've already rated - the system doesn't have much context to work with, and I think LLMs are actually a pretty poor choice for this type of problem! This project was really just a test of whether the idea was feasible at all!

A lot of my time is taken up with other projects right now, so I'm unlikely to work on this issue myself in the near future. That said, I think your suggestions are great, and I'd welcome PRs on any of the parts!

I think a sensible first step would be to fetch a complete list of films from the target user's profile, rather than the 50 most recent ratings which are provided in the RSS feed (see current implementation). That way, we could confidently filter the recommendations for films which the user hasn't seen.

Do you think you could take on that job?

[blakegearin]

I think a sensible first step would be to fetch a complete list of films from the target user's profile, rather than the 50 most recent ratings which are provided in the RSS feed

@harrisonpim Wouldn't that require Letterboxd API access? Looks like that has to be requested: https://letterboxd.com/api-beta/

[harrisonpim]

Hey @blakegearin, thanks for dropping in here!

You're right, the letterboxd API is probably the best way of getting hold of that data. In the past I've reached out to letterboxd with a request for API access, but I didn't hear back from them.

Since I last reached out, their statement on that page has also been updated to say

At this time we are not granting access for data-analysis or recommendation projects, or for LLM or GPT-related use.

which I think covers this project!

Assuming we can't get access to the API directly, and given that this is a non-commercial and very low-traffic personal project, I think a little bit of scraping via the letterboxd front-end wouldn't be raise too many eyebrows...

What do you think?

[blakegearin]

Tapping into the native "Export your data" functionality might be less fraught. It can be found in the settings but also directly accessed with this URL: https://letterboxd.com/data/export/

Theoretically could do something like...

• Add a button to open the URL

  • If not signed in.... a login page will load

  • When signed in, the ZIP download will begin or ask where to save (depending on browser settings)

• Add a file upload for the user to submit the ZIP

• On the back-end... unzip, process in the watched.csv and ratings.csv (requires some deduplication)

• Optionally process in watchlist.csv to exclude and/or help inform the recommendation (could have a section like top 3 recommended from your watchlist)

This would be a more advanced version, and requires slightly more from the user. However, it's really not too difficult to get the ZIP, and CSV parsing is usually dead simple.

[harrisonpim]

That's a really smart suggestion!

My only concern is that adding the functionality to unzip and parse the files might take us over the response timeout limits on vercel, where this project is currently deployed. It's a snag I've hit before, particularly in projects like this where we're already relying on a slow third-party API for the generated responses...

Having said that:

• I think the same flaw exists in the scraping approach, so we'd have to deal with that issue either way
• A bit of googling while writing this comment has revealed that vercel have added the option to increase function response times up to 60s since I last checked the documentation! Happy days 🥳

I'm happy to move forward on that basis.

How much direct involvement would you like in the implementation here? Is this work something you'd like to take on yourself?

[blakegearin]

Sorry for the delay. I don't have prior experience with Next.js and minimal experience with TypeScript, so it would take me a bit to get my bearings and be able to contribute.

[harrisonpim]

No problem at all 😁 I've also been AFK for the last week or two, so haven't been waiting on a response

If you're not up for learning a bunch of new frameworks (which would be totally understandable!), I'm more than happy to run with your suggestions and implement them myself 👍

That said, if you are looking for an excuse to pick up some next.js/typescript, I'd also be fine with going slow, working together, etc so that you have a gentle intro to them. There's no rush to add this feature immediately. Totally happy either way, just let me know!