∞Anime ∞Female ∞Male ∞Quote
101476 Characters
Login | Register

You need more training.

- Tenshi


Stats Patches Jul 16, 2011 | Rei
Fixed a major error in how it handled deletes.

If the Parent Node in the linked list was removed, the static reference to it was not updated. Which would cause some massive errors~ seg faults, bus errors when other function calls expecting the parent node to not be null were called.

It was nice watching debug skew in my terminal, but funny thing is, displaying that information is more computationally expensive than running the rest of the program! lol Understanding MySQL, you'd know it logs data to binary logs, replay logs, so yet another optimization on our part not wasting cycles on those operations.

Made the debug skew use a Debug Flag that can be set with my PHP Admin Interface.

Also wrote a quick stats function that prints to stdout a few counters ( connections, max id, num records ).

And rewrote the SELECT * WHERE id function to print the record to stdout.

Kinda was suprized that "SELECT * WHERE id" was made redundant. Being able to move client code into the server is.... beautifully efficient.

Previously, on MySQL, we'd look up your record ID using keys. We'd then fetch your record or create it. Then update your record if needed. 3 operations. I moved all of that into one operation, the server now handles the other two on it's own.

The code and server looks and feels pretty solid so far.

It'll need a full day of testing, plus I have yet to run the "Delete all records older than 24 hours" function ...

I'll know in a week ( unless something breaks before then ) if this project is ready to be marked complete.

Using auto increment ID's in a binary tree proved not possible... however, looking at these CRC32's I might have an option with them!

Oh! Almost forgot another awesome optimization.

That User's Online thing at the bottom?

Calculated it meant O(n) operations every single page view. Since we're doing about 10+ pages a second, traversing that list several times a second, just seemed a stupid waste. I don't know how good MySQL's query cache is, but it's got nothing on this. I cache the time the query was made, hold it for 3 seconds, and send the cached count. Thus, we traverse the list only once every 3.9999 seconds. Practically a 4000% increase in speed. Muahahah well i do suck at math. lol
Stats Server live beta Jul 16, 2011 | Rei
All the features needed to replace the MySQL version have been coded, and uploaded to the main server now.

I'll be keeping an eye on the performance today.

Since it's optimized for O(n) it should slow down as more and more records are added ...

Should that become a big issue, I have a fairly simple way to optimize Updates for O(c). ;-)

Rebuilding the index at midnight is another operation I'll be curious about...

Will it lock for a fraction of a second?
For a second?
For seconds?
Stats Server Day 3 Jul 15, 2011 | Rei
Another tough day writing this stats server.

Progress has been made on the core 5 handlers.

void handle_lookup( char * buf, int cfd );
void handle_record( char * buf, int cfd );
void handle_count( char * buf, int cfd );
void handle_update( char * buf, int cfd );
void handle_insert( char * buf, int cfd );

Each of those expands into multiple functions and already hundreds of lines of code.

mysql> select count(*) as n from ustats;
| n |
| 2747 |
1 row in set (0.00 sec)

Hm, interesting. It's not 1,000,000+ records as I feared. Reminds me, I need a means to delete old records. Like our current stats implementation, will keep track of your page views until it is 24 hours old. At which point, it is scheduled for deletion at midnight when our garbage handler runs. However, if like me you use the site day after day, same browser... er, apparently that doesn't matter. I just delete everything at midnight? lol Ah to have a simple implementation that makes sense... oh well, still time to sort all this out.

I'm gradually rewriting the PHP Include file that this aims to replace. Every function using MySQL API's will use Goral Software's Database API's. hehe

With a little luck, might be able to do some live testing this weekend.

Hm... although, I might want to run this through a Memory Leak detector before going live. Tho with 10GB of RAM, it's not like the server would instantly go down. haha. but nonetheless, it'd be a good idea

Please know that cookies are required to operate and enhance our services as well as for advertising purposes. We value your privacy. If you are not comfortable with us using this information, please review your settings before continuing your visit. [close]

Who's This Character Board | New Thread

08:29 am
Who is this ?
08:20 am
Who is this ?
05:54 am
Who is this ?

New Thread

All images are copyright of their respective owners.
Users Online 117 | Rendered in 16.8 ms. R-21-W-1-M-2087.85 KB Modified: Mon, 11 Nov 2019 14:39:59 -0500 | [Options] Copyright © 2007-2019 Goral Software