• News
• New Article
• List Articles
• About

Finally, found a solution to intermittent server performance issues.

At all times, we have a lot of bots accessing ACDB. As far as Google Analytics is to be trusted, of our 500,000 - 1,000,000 daily page views, roughly 10% are from human sources. That other 90% is a legion of bots set upon us.

Our web server works a little weirdly in that for each connection, it allocates roughly 256 file descriptors. So it was giving me a useless error stating "out of fds" as it crashed. I set up trips and traps to monitor at a kernel level how many fds were actually in use as tracked by the OS, and it never went much beyond 3000. So how in the world could more than 5000 have been exceeded by the web server? It's that fractional reserve banking like counting, it is.

Even then, all I learned was another config value, one that turned a hard crash ( non-recoverable ) into a soft crash ( recoverable ).

Still, performance was shit. And it made no sense to be soft crashing over and over and over again.

So I turned my attention to the firewall. It's quite amazing how even after all these years, you can't have a firewall rule with two limit conditions.

You can
A) limit the connections per IP
B) limit the connections per rule
but not both.

I can not say a max of X IP's get Y connections each to a max of Z connections. Nope.

Made no diff anyways, more connections STILL came in than the server could handle. There was no 1-1 ratio here.

Thinking my only resource would be to hack the code myself to give me more info, I came across a better solution using mod_status. At one glance I understood the problem, the solution, and the action to take.

We're now severing 30 requests per second with 80% slack capacity.

Bring on the traffic !

Made some improvements to our Master Assignments Editor.

There was a bit of an issue when dealing with primary assignments.

They are now highlighted in red, and rather than giving an error when clicked, they will direct you to a page where they can be properly edited.

Every character in the DB needs a primary assignment, one title I can use as a short cut to fetch publisher related data from. Ideally it should be wherever they first appeared in a franchise.

No character should ever be assigned directly to a franchise, as that would provide us the least amount of useful information about them.



Disqus

What else ... well, we're going to try out Disqus yet again. I have it on character pages as the new comment system. To prevent it from lagging pages, you'll need to click Load Comments to use it.

It is hot this week! Going to be 30C and higher here in Winnipeg. Heat and humidity make it extremely hard to focus and concentrate - I may not be doing much programming for the days ahead.

There's a lot I have planned.

Now that our core content has been simplified, I'd like to return to translating common labels. With a focus on Japanese. アニキャラベー is coming back to life.

I'm turning my attention to favorites, as you may see the above linking to your favorites. I'll try to borrow the layout from Steam, and make it super easy to browse, comment and rate your favorite characters. As well as link up the organize favorites tool. Still waiting any feedback on how to improve that.

I'm indexing SAO this week.

I've got big big plans ahead for the quotes. Eiki and I have indexed more than 80,000 lines now. I'll hit 100,000 this year.

Then I'd like people to start favoriting quotes.

From there, select an additional image that best matches the quote.

From there, let anyone vote which quote + image combination they like best.

From there, build the largest collection of anime meme's!!!!

If you're interesting in being part of this project, please contact me. Tools for indexing episodes have matured to a point now that you could frustration free index even from a series with hundreds of characters.

I'd also like to make it a little bit easier to interact with your watching list. Rating, tracking, indexing character appearances, episode blogging .... lots of old systems just collecting dust that could be used again.

1 2 3 4 5 102