Add perms to assets in GCS

[mattkrick]

AC

• Accessing GCS bucket assets should require a presigned key or something similar
• user avatar should be open to everyone
• org avatar open to everyone
• assets should be open to the Organization? (this'll be kinda tricky since we store assets on the User, maybe we move it to /Organization?) or we could make a query that finds if 2 users are in the same org. tbd
• SAML config locked to the org
• ?? audit the rest of anything else we have in there

https://parabol.slack.com/archives/C836NA350/p1738279429094589?thread_ts=1738256782.141939&cid=C836NA350

[mattkrick]

did some more research. from slack:

options for security are:

• presigning asset URLs in advance. This is harder on the application because e.g. we’ll need to traverse every tiptap JSON document, extract the URLs and then replace them with presigned URLs.
• put GCS behind Cloud CDN and issue a signed cookie to each client so they can access the assets we want them to. This means issuing new cookies when they join/leave orgs. this is harder on infra as well as the app 😅 .
• prefix all asset URLs with our own image proxy endpoint. create an image proxy endpoint that takes a request. if it’s public, it redirects to that URL. if it’s private, it returns a presigned URL. <-- this is how notion does it. Also fun fact avatars in notion are public 🙃

The thing that I don't get is why notion went with the 3rd option. It requires an extra service + an extra round trip for each asset! Essentially it's like cloud CDN, but they control the CDN so they don't need to mange 3rd party cookies. Asking GPT why, it suggested short TTLs so folks can't hotlink the asset. we could do that too. DDoS prevention, but again I don't see that as valid because notion still 302s to the presigned URL so someone could still hit that URL until it expires.

Other reasons notion may take this approach:

• custom caching. look for hot images & cache them at the edge. i mean, i guess? s3 requests aren't that expensive
• on-the-fly image alteration. this makes sense. mobile devices can requires a scaled down version of the image. IMO storage is cheaper than CPU cycles so I'm not sure why I wouldn't just store a few sized down versions & serve up the smallest one I can?
• analytics. notion sees all the image requests going through the proxy. fair, but since notion loads all images and not just those above the fold, we could get the same analytics by looking at the document we're sending to the client & assuming each image in the document is going to get fetched.

I don't see a killer reason for the added complexity (i.e. an image proxy server).
Option 1 seems like the best for us.

• We store the tiptap doc as JSONB so the PG driver parses it for us when pulling from the DB
• if the file storage manager is GCS, we assume the bucket is secured & perform the next step
• we traverse it looking for URLs. Each URL we find we replace with a presigned URL lasting 5 mins.
  • in the future, the request could include the device type so we send down smaller images for mobile clients
  • if traversing gets expensive, we can cache all the asset URLs inside the doc & let the client rehydrate accordingly. or cache the location of all the URLs so we don't have to do an exhaustive search
• we serialize & send the doc down to the client

[rafaelromcar-parabol]

Just one note on the file stores: whatever we implement, we should do it for both AWS and GCP.

I've been reading on signed cookies over GCP Cloud CDN, and I actually think it's the best approach for us, that would allow us to better structure our bucket as well. Our use case fits exactly on their recommended use cases for cookies.

For example, say we decide images are available only for organization members only. We should then upload images to a folder in that org, instead of on the folder of the user, and we could just generate the cookie (and refresh it when necessary) for users that are part of that org. That would simplify a lot the access management and reduce/eliminate the need of signing hundreds or thousands of urls multiple times. It's just one cookie to sign per user each X time vs multiple urls to sign per user each X time. We would also use prefixes to control the access. In this case, urls would still be our urls (action-files.parabol.co/whatever), which is also nice, but would only work if the user cookie is valid. And we could have short living cookies if we wish. I really think this is the correct approach for us, we just need to structure our bucket a little bit (most of the work is already done!). We could make images available only if you are part of the org, part of the team, part of the meeting even! and in the future we could do it by activity/pages or whatever we wish.

We also need to test if we can implement this starting with public buckets, I'm quite sure the answer is yes and the transition would be quite easy.

Could you hold this work off until I'm back? I really really would like to explore this design with you, once I'm back in March 5th.

[mattkrick]

those are good points! I've got 2 tensions with using signed cookies:

• Pages permissions will follow an RBAC model, i.e. i can give access to me, 1 other person in my org, my entire org, 1 person outside my org, or a team. cookies require using a path-based permissions model, so there's no way to make it as fine-grained as we need... at least as far as I can tell! Is there a strategy you could suggest?
• I suggested cookies for our gov friends, unfortunately they use S3 but not cloudfront, so it was a non-starter, which is why we went the presigned URL route when we built this functionality a few months ago

[rafaelromcar-parabol]

The first point is interesting and for sure harder in terms of engineering, which is great. I think that could be solved with a correct path structure in our buckets (like a foldr per page with its documents inside of it) + cookies with small TTLs (say minutes our 1h max) to whoever have access when it requires access. That said, I haven't spent enough time thinking and reading about this for now, and sadly I wouldn't be able to do it today.

unfortunately they use S3 but not cloudfront

As in, they cannot use CloudFront or they are just not using it at the moment? If it's a possibility, we shouldn't get it off the table IMHO. @dbumblis-parabol do you know/can ask if there is any change they might create a CloudFront distribution in front of an AWS S3 bucket for security reasons?

[mattkrick]

like a foldr per page with its documents inside of it

that implies:

• images can't be shared across documents
• cookies go invalid when permissions change! that means if i'm offline & you update permissions, when I go online the server would have to validate my cookie by performing an O(n) operation on every document my cookie has access to. In notion i have access to 1000 documents. That's a HEAVY operation to perform on connection! Unless there's another strategy you're considering?

[rafaelromcar-parabol]

No, you are right 😸 as I said, I haven't spent the time thinking this thoroughly. I would like to spend at least a day going through this.

Do you know how Confluence does this?

[Dschoordsch]

I think that's why notion went with the 3rd approach. This way you can decouple the image permissions from the rest of the application logic and distribute the load better.

[mattkrick]

Do you know how Confluence does this?

don't know, don't care 🤣 Atlassian engineering is bottom of the barrel.

if we had the time & resources, i'd say we go for the notion approach because then we could handle permissions in a whole separate microservice, but that would also mean more resources, both human & compute. That said, handling them in a gql resolve function that reduces a roundtrip? Pretty darn good IMO...