[20 gh] NMCBit pool 3% fee prop 6.6% PPS

[DavinciJ15]

Pool luck has gone sour over last 36 hours however the over all luck is very good as our total shares are 2.6 million and our total blocks found is 29 so we should have found 27.65 blocks thus we are up by 4.9%!!  Yeah!

All good boys!

Major down time coming soon
I'll be moving the system to amazon EC2 copying the db and wallet up to the server will take time.  This should cut our down time and improve access to the servers.

[1714036478]

1714036478

[1714036478]

1714036478

[DutchBrat]

Damn we hit an unlucky patch.... hashing has gone down and we are at 200K+ shares....

Talk about dumb luck....

[DavinciJ15]

Yeah and my AC broke and I cant get it fixed for few days.
When it rains...

[DavinciJ15]

Finally finished the move to Amazon servers I want to move on to Pay-Per-Share coding but the database is so in efficient and slow that I had to move from micro server to large and it's still SLOW!

I must fix this bug first.

The cool thing is I have the 2 boxes at Amazon!  One is the MySQL server and the other is namecoin, pushpool and web server.   The link between the MySQL and the web server is secure via ssl but pushpool is not so do not use the same password for workers as you do for your log-in.

YES, YES I do not have SSL enabled on nmcbit.com YET but it's good practice for when I do.

[Beta-coiner1]

Who created it?

Ah well I will take it add it to my web site.

Finally found the thread where I got the program from,It was made from Demonblack and the you can find it originally here.

[DutchBrat]

Great to see you moved to Amazon !

About the pay-per-share, will that be choice for the miners, or will that be the only way to participate ?

Brat

[DavinciJ15]

Great to see you moved to Amazon !

About the pay-per-share, will that be choice for the miners, or will that be the only way to participate ?

Brat

Miner choice.

[DutchBrat]

Davinci,

I appreciate u trying to get a pool ready for namecoin mining, but it seems you have to get merged mining working before the next difficulty change (somewhere in november).

If BTCguild and Deepbit will support it, they will dominate the namecoin space, with their hashing power. Difficulty will be at the forecasted low levels for only 1 or 2 days.

With namecoin mining only you won't be able to keep up.

Have you thought about that ?

Brat

[DavinciJ15]

Davinci,

I appreciate u trying to get a pool ready for namecoin mining, but it seems you have to get merged mining working before the next difficulty change (somewhere in november).

If BTCguild and Deepbit will support it, they will dominate the namecoin space, with their hashing power. Difficulty will be at the forecasted low levels for only 1 or 2 days.

With namecoin mining only you won't be able to keep up.

Have you thought about that ?

Brat

Yes I have and I will be providing merge mining 1 month before it's actually available but taking huge loses, effectively buying the miners.  But there are MANY hurdles, currently my script is WAY to slow as it is.  Second the free version of MySQL can not handle the kind of load Deepbit has in shares.  3rd this means I must scale back on the features I would have wanted to add.  I will sacrifice the namecoin client and maybe the affiliate program.

The code is a mess it does to many slow queries on data that can be static!  Like the previous round, all it's data is collected from the shares the don't change!!!  CRAZY.

Then I have to change the UI so it's neater.

Not sure if I can make Sept 5th.

[DutchBrat]

Sounds great

I'm curious how you will provide merged mining before the offical launch. Would be awesome, but I understood there will be a change to namecoind to provide compatibility with the already mined coins

About Sep 5th, no worries, it's an experimental pool, I'll stick it out 'till u get the job done

Keep up the good work

Thx

Brat

[phorensic]

Yes I have and I will be providing merge mining 1 month before it's actually available but taking huge loses, effectively buying the miners.  But there are MANY hurdles, currently my script is WAY to slow as it is.  Second the free version of MySQL can not handle the kind of load Deepbit has in shares.  3rd this means I must scale back on the features I would have wanted to add.  I will sacrifice the namecoin client and maybe the affiliate program.

The code is a mess it does to many slow queries on data that can be static!  Like the previous round, all it's data is collected from the shares the don't change!!!  CRAZY.

Once our pool got above about a couple million shares I had to do major database tweaking.  The 3 things that helped the most were COUNT(*) instead of mysql_num_rows, indexing the columns in a table that get the most action based on the statements after a SELECT - WHERE xxx, and then trying to fit the whole database in RAM with huge buffer pools.  mySQL locks a table from read and write if you try to add an index, but there is a tool to do it without any disruption in service called oak online alter table.  I did a million other tweaks to my ext4 mount options, php code and to mySQL variables that all add up too, but that would take a novel to explain.

[DavinciJ15]

Yes I have and I will be providing merge mining 1 month before it's actually available but taking huge loses, effectively buying the miners.  But there are MANY hurdles, currently my script is WAY to slow as it is.  Second the free version of MySQL can not handle the kind of load Deepbit has in shares.  3rd this means I must scale back on the features I would have wanted to add.  I will sacrifice the namecoin client and maybe the affiliate program.

The code is a mess it does to many slow queries on data that can be static!  Like the previous round, all it's data is collected from the shares the don't change!!!  CRAZY.

Once our pool got above about a couple million shares I had to do major database tweaking.  The 3 things that helped the most were COUNT(*) instead of mysql_num_rows, indexing the columns in a table that get the most action based on the statements after a SELECT - WHERE xxx, and then trying to fit the whole database in RAM with huge buffer pools.  mySQL locks a table from read and write if you try to add an index, but there is a tool to do it without any disruption in service called oak online alter table.  I did a million other tweaks to my ext4 mount options, php code and to mySQL variables that all add up too, but that would take a novel to explain.

Thanks man you are a life saver I was wondering about taking the system down to do some alters.   I'll check out oak as for buffer pools how do I do that?

You put the db on an ext4 drive?  Why what tweaks can you do to a filesystem?  Give me a few more hints and I'll send you a bitcoin.

[DavinciJ15]

Sounds great

I'm curious how you will provide merged mining before the offical launch. Would be awesome, but I understood there will be a change to namecoind to provide compatibility with the already mined coins

About Sep 5th, no worries, it's an experimental pool, I'll stick it out 'till u get the job done

Keep up the good work

Thx

Brat

By paying people bitcoins and namecoins for mining namecoins only I will come out of pocket for the bitcoins until the merge mining starts this way I have the edge!

[lxFlasHxl]

hmmm its alright

[DutchBrat]

I have a lot of connection problems tonight

At the moment, 5 out 9 miners are hashing away at their backup pool

Any idea what might cause this ? For a couple of hours all went smooth, but last 2 - 3 hours I have seen them go on and off NMCBIT

Brat

[phorensic]

Yes I have and I will be providing merge mining 1 month before it's actually available but taking huge loses, effectively buying the miners.  But there are MANY hurdles, currently my script is WAY to slow as it is.  Second the free version of MySQL can not handle the kind of load Deepbit has in shares.  3rd this means I must scale back on the features I would have wanted to add.  I will sacrifice the namecoin client and maybe the affiliate program.

The code is a mess it does to many slow queries on data that can be static!  Like the previous round, all it's data is collected from the shares the don't change!!!  CRAZY.

Once our pool got above about a couple million shares I had to do major database tweaking.  The 3 things that helped the most were COUNT(*) instead of mysql_num_rows, indexing the columns in a table that get the most action based on the statements after a SELECT - WHERE xxx, and then trying to fit the whole database in RAM with huge buffer pools.  mySQL locks a table from read and write if you try to add an index, but there is a tool to do it without any disruption in service called oak online alter table.  I did a million other tweaks to my ext4 mount options, php code and to mySQL variables that all add up too, but that would take a novel to explain.

Thanks man you are a life saver I was wondering about taking the system down to do some alters.   I'll check out oak as for buffer pools how do I do that?

You put the db on an ext4 drive?  Why what tweaks can you do to a filesystem?  Give me a few more hints and I'll send you a bitcoin.

This is one of the guides I used for tweaking mySQL - http://www.mysqlperformanceblog.com/2006/09/29/what-to-tune-in-mysql-server-after-installation/.  I'll save you the hours of tuning options and restarting mysql and tell you that innodb_buffer_pool_size is the only variable that made a difference and right now we have it set to 75% of our server's RAM.  This assumes you have your main shares table as innodb, which most of us do.

I didn't choose ext4 for the database, and I understand there are some other filesystems that may be better for raw mysql performance.  I think XFS is arguably better.  Anyway, changed mount options in fstab to "commit=60" and "noatime" to reduce the overhead on write operations.  This frees up the drive for more iops for mysql if it has to go to disk.  This reduced my await times by a lot.

[DavinciJ15]

I have a lot of connection problems tonight

At the moment, 5 out 9 miners are hashing away at their backup pool

Any idea what might cause this ? For a couple of hours all went smooth, but last 2 - 3 hours I have seen them go on and off NMCBIT

Brat

Updating the database doing large queries trying to put together static data, also adding stored procedures to lighten the load this caused a lot of short busts of down time.

Going to bed now so it should be good.

[DavinciJ15]

This is one of the guides I used for tweaking mySQL - http://www.mysqlperformanceblog.com/2006/09/29/what-to-tune-in-mysql-server-after-installation/.  I'll save you the hours of tuning options and restarting mysql and tell you that innodb_buffer_pool_size is the only variable that made a difference and right now we have it set to 75% of our server's RAM.  This assumes you have your main shares table as innodb, which most of us do.

I didn't choose ext4 for the database, and I understand there are some other filesystems that may be better for raw mysql performance.  I think XFS is arguably better.  Anyway, changed mount options in fstab to "commit=60" and "noatime" to reduce the overhead on write operations.  This frees up the drive for more iops for mysql if it has to go to disk.  This reduced my await times by a lot.

Cool thanks send me an address to send you 2 BTCs.

Cheers.

[zx9r]

Sounds great

I'm curious how you will provide merged mining before the offical launch. Would be awesome, but I understood there will be a change to namecoind to provide compatibility with the already mined coins

About Sep 5th, no worries, it's an experimental pool, I'll stick it out 'till u get the job done

Keep up the good work

Thx

Brat

By paying people bitcoins and namecoins for mining namecoins only I will come out of pocket for the bitcoins until the merge mining starts this way I have the edge!

Have you thought about the huge number of bitcoins that you could have to pay ?

A simple number: 500 Ghash * 1 month = 8.5k BTC

You must be very rich if you are thinking in doing that 

[DavinciJ15]

Have you thought about the huge number of bitcoins that you could have to pay ?

A simple number: 500 Ghash * 1 month = 8.5k BTC

You must be very rich if you are thinking in doing that 

The problem is not paying it the problem is creating a pool that can support that kind of hash rate.  As far as I can see there is no fracking way mysql can support that kind of insert per second without being clustered.  What I am looking for next is a way to do select statements from a stored procedure to a remote sql database  (MS SQL Server does it).  This way I can create a MySQL server that has only flat data such as stats and many pushpool DBs with worker and share tables.

I will probably spend less than that but as I will only do it on a PPS and take a bigger % off the top than normal just so the miner is more profitable in my pool.  The people who have been with me since the beginning will be getting more than those that are band wagoners because they risked more and I respect that since they have no idea if I'm going to work my ass off to get this working.  Loyalty goes a long way with me.

Cheers