Sadly our current Stats implementation, which is tasked with being a software level DoS Protection, is itself causing minor DoS issues.
Every page requires access to the stats server, and despite a fairly well optimized MySQL implementation ... it's just not fast enough.
Luckily, I can do better. I'm coding a complete replacement in C. Writing my own database back-end, drafting my own API, implementing all the functions, and writing my own DBA Protocols.
Everything has been moving forward well...
A) Wrote a server in C that binds to a UNIX Domain SOCKET and listens for connections.
B) Write a client in PHP that connects to the database, sends a command, reads a reply.
C) Coded the server to read and parse commands, call handler functions.
D) Coded a handler for inserting new records.
E) Drafted the database as a structure, wrote a linked list storage engine
F) Wrote a primary key lookup handler
G) Retrieved records by primary key
Still a hell of a lot more to do.
I'll be reading about "man 3 dbopen" and "man 3 btree" tomorrow.
Creating working binary tree indexes will be something to explore.
We have very expensive CPU: Intel(R) Xeon(R) CPU X5670 @ 2.93GHz (2926.02-MHz K8-class CPU)
So it's possible it could linearly scan 1,000,000 records many, many, many times per second. But proper testing is necessary before I take that on faith alone. haha