Anime Characters Database |
☰ ACDB Log In |
SMASH or PASS |
The visual search engine for animated characters. We index characters by eye color, hair color, hair length, age, gender, and animal ears.
Characters | 142393 | 152 this month |
Titles | 13863 | 7 this month |
Members | 79527 | 154 this month |
Quotes | 335745 | 321 this month |
Voice Actors | 10354 | 15 this month |
One down side of adding additional translations to the site is: web spiders, bots, and crawlers end up having index entire copies of the site. This has been pushing daily page requests near 3,000,000. while Google Analytics reported requests have remained around 100,000.
This works out to for every 1 request by a member, there are 30 requests by bots.
Our web server Nginx allows for rate limiting either at an IP level or a server level.
ACDB makes a lot of use of redirects so I can't really rate limit people to 1r/s without
1) Breaking the lounge
2) Making redirects take 2 seconds for the page to respond
I also can't give members more than 3r/s without risking race conditions.
This left me wanting a solution where I could have more generous rate limits for members and stricter rate limits for everyone else.
Unfortunately Nginx has some serious hatred problems for simple conditionals. They live under a mantra of "IfIsEvil".
Luckily, the world of programing is one where if you have a will, there is always a way.
Pushing aside useless AI summaries, I eventually landed on https://nginx.nginx.narkive.com/tRRv3Xv8/limit-req-inside-a-if-block-not-allowed
While this page's solution made use of a user agent check; that conditional could easily be replaced with a cookie test. A few tweaks later, the setup is live on ACDB. Members will enjoy a fast and responsive site as it should be while the bots will be limited to an amount of requests that shouldn't gum up the site again
for a while.
--
Rei
Kawaii or NOT and Smash or Pass are quite popular! Or test your knowledge witch Series Quizzes.
Latest Family Tree Relations
Quick Links for members.
02:09 am
|
Anonymous
Who is this?
|
What is her height?
- Anonymous
Rendered in 81.9 ms. R-27-W-1-M-3177.8 KB