Custom rate limit for database collection access per user #1309
Replies: 2 comments 1 reply
-
I really like the idea. Appwrite already uses a built-in abuse mechanism that is very flexible and easy to apply for internal resources like the create account API endpoint. Today we highly avoid abuse check on read operation to make sure read per seconds rate is as fast as possible. I can definitely see us implement this feature per resource container (database collections & storage buckets) to control write rate. With some extra testing we might also allow this feature on read operations, but it would definitely come at the price of performance for a collection where this is enabled. |
Beta Was this translation helpful? Give feedback.
-
I really like the proposal, this would be a really useful feature. I am wondering how much of these issues could be solved by putting your Appwrite instance behind something like a Cloudflare CDN which has built-in protections against this sort of attack (correct me if I'm wrong). Should rate limits be enforced at the CDN level or at the Appwrite collection level? |
Beta Was this translation helpful? Give feedback.
-
Hi, what do you think about adding custom rate limits to database collections?
Current Problem
I created a for loop that writes 10.000 entires to a database, with a single field:
The Appwrite instance went offline for about 3 minutes to handle all those requests.
(The server, where my Appwrite instance is running on, has 16GB of RAM and 4 Cores)
All those 10k entries are now in my database.
Current Solution
Currently, we could call a Function, which does the rate limiting. The problem with that is, that the current implementation of Functions, spawns a process every time the function is called, which uses a lot of resources and takes its time, as discussed in #1261. Handling rate limits via Functions would definitely also bring the server down, on such an amount of requests.
The bad part is, that nothing is preventing a harmful user to do exactly what I did.
Possible Solution
So let's say that we have a collection named
posts
, and we'd like the user to create 10 posts per hour and modify their own posts (1 modification per 5 seconds to prevent spamming), but view 10.000 posts per day (to prevent scraping).This could be achived by a rule like:
allow [read/create/update/delete/*] every N [seconds/minutes/hours/days] per [user/team/member/ip]
This would add very easy spam/scraping protection and admins could configure the limits in the UI very quickly.
I think this feature would be highly appreciated, as it's simple to use and powerful enough to prevent most bots/scrapers/spammers without the need of any code / function.
What would this fix?
This would enhance the security of Appwrite dramatically, and provide custom API abuse prevention mechanics that are very easy to configure.
Personal Note
I am very new to Appwrite (I switched from Firebase) and I find it hard to prevent users from abusing the API to get every public post in a couple of seconds/minutes. I didn't spend much time with Functions for now, but I think it would take a lot of time to introduce rate limits for every Database collection per user. If there is a trick or something, you're free to tell me ^^. This is why that feature idea came up. Creating rate limits for different collections could potentially save every spam attack a user could try. Not to leave out scraping, as currently, if a user has read permissions to a collection and all it's documents, that user could easily scrape every posts on the platform within seconds or minutes. I also think that adding such a rule/setting is very valuable, as it's also hard in other services to implement such rate limits. In Firebase for example, we need to read a timestamp of the last action a user did to figure out if he might be spamming. To get this timestamp, a
read
operation is needed. So even if the spammer doesn't get results anymore, theread
operation count will go up and increase the price for the project. Introducing those custom rate limits, would be a major security enhancement. Also rate limiting is the only reason that I would use Functions for now because my App doesn't need custom server side logic, except for abuse prevention, which could easily be replaced by a custom rate limit rule. I do think that rate limiting users on Database operations is a powerful feature and would enhance Appwrite to a next level of security.Let me know what you think :)
Beta Was this translation helpful? Give feedback.
All reactions