• News
• New
• Index
• About

All the features needed to replace the MySQL version have been coded, and uploaded to the main server now.

I'll be keeping an eye on the performance today.

Since it's optimized for O(n) it should slow down as more and more records are added ...

Should that become a big issue, I have a fairly simple way to optimize Updates for O(c). ;-)

Rebuilding the index at midnight is another operation I'll be curious about...

Will it lock for a fraction of a second?
For a second?
For seconds?

Another tough day writing this stats server.

Progress has been made on the core 5 handlers.

void handle_lookup( char * buf, int cfd );
void handle_record( char * buf, int cfd );
void handle_count( char * buf, int cfd );
void handle_update( char * buf, int cfd );
void handle_insert( char * buf, int cfd );

Each of those expands into multiple functions and already hundreds of lines of code.


mysql> select count(*) as n from ustats;
+------+
| n |
+------+
| 2747 |
+------+
1 row in set (0.00 sec)


Hm, interesting. It's not 1,000,000+ records as I feared. Reminds me, I need a means to delete old records. Like our current stats implementation, will keep track of your page views until it is 24 hours old. At which point, it is scheduled for deletion at midnight when our garbage handler runs. However, if like me you use the site day after day, same browser... er, apparently that doesn't matter. I just delete everything at midnight? lol Ah to have a simple implementation that makes sense... oh well, still time to sort all this out.

I'm gradually rewriting the PHP Include file that this aims to replace. Every function using MySQL API's will use Goral Software's Database API's. hehe

With a little luck, might be able to do some live testing this weekend.

Hm... although, I might want to run this through a Memory Leak detector before going live. Tho with 10GB of RAM, it's not like the server would instantly go down. haha. but nonetheless, it'd be a good idea

Sadly our current Stats implementation, which is tasked with being a software level DoS Protection, is itself causing minor DoS issues.

Every page requires access to the stats server, and despite a fairly well optimized MySQL implementation ... it's just not fast enough.

Luckily, I can do better. I'm coding a complete replacement in C. Writing my own database back-end, drafting my own API, implementing all the functions, and writing my own DBA Protocols.

Everything has been moving forward well...

A) Wrote a server in C that binds to a UNIX Domain SOCKET and listens for connections.
B) Write a client in PHP that connects to the database, sends a command, reads a reply.
C) Coded the server to read and parse commands, call handler functions.
D) Coded a handler for inserting new records.
E) Drafted the database as a structure, wrote a linked list storage engine
F) Wrote a primary key lookup handler
G) Retrieved records by primary key

Still a hell of a lot more to do.

I'll be reading about "man 3 dbopen" and "man 3 btree" tomorrow.

Creating working binary tree indexes will be something to explore.

We have very expensive CPU: Intel(R) Xeon(R) CPU X5670 @ 2.93GHz (2926.02-MHz K8-class CPU)

So it's possible it could linearly scan 1,000,000 records many, many, many times per second. But proper testing is necessary before I take that on faith alone. haha