Bitcoin Forum

and to top it off I made two of them.

I decided to go with GPU rather than with the non existent ASIC.  I see it as deciding to buy a v-8 sports car for $100k rather than wait for the $1k atomic powered flying sports car to be finished (like with BFL).

I work with both clusters and co-location, so I got a good deal on a massive amount of hardware and I got a good deal on coloco (air kept at 66F all day plus a staff).

My rigs are a cluster of 4 nodes each and one embedded *BSD licensed controller of my own design.  Each one has 8 8 core cpus, 512GB of RAM and 12 GPU cards with logging and storage done in an encrypted VIA Mini ITX controlling a storage array (so an error won't crash the system by filling up system disks).  The other rig is exactly the same, located in a different location with the same colo company and has hot failover capability with the other machine using something Linux HA heartbeat.  So if one rig fails it takes over the tasks of the other rig as its virtual instances switch tasks (why I needed so much RAM).

I really think I spent too much for this ($60k for both.  if I had not got the deal I did I would have paid about $90k) and tonight or tomorrow I should be able to start mining once it is set up at my colo.

The amount I spent is making me have doubts about bitcoin to be honest but I am here and I am obviously jumping all in.  So I figured I would introduce myself.

Hi!

hi
how much hashing speed it s having?

Hey man, that is an amazing setup.  I'm trying to get into mining myself but don't have the funds at the scale you do!  Can we talk offline, perhaps?  I'd love to pick your brain about GPU mining setups!

For the first commenter:  I'll let you know tomorrow.

For Ionux.  Maybe we should see how I do first before you start getting information from me.  I might suck :)

But seriously, Go with ATI / Radeon.  Its gotta be as open as possible.   What I am doing now is this:

Trying to tweak my system on a theoretical computing level.  That is why I have the BSD licensed controller, so I can protect my work.  The USA is lawsuit happy and if this works like I want it to work I want to make sure that someone does not file a suit for violating the GPL or something.

Before GPU mining I would advise you make a cluster.  If you can find some old VIA mini ITX boards you can have a low power 3 node cluster than can fit in an old case.  Those boards usually have one PCI slot to experiment with an old GPU.  You wont get a mining rig that will give you plenty o' bitcoin but you will learn how to make a cluster, learn Linux and learn how to tweak a GPU.  By doing this you will learn something that could even become a profession just in case the whole bitcoin thing turns out to be a fraud someday.

What I have to do with my clusters now is see if I can arrange for cluster leasing or something in case the whole bitcoin thing is nonsense and I end up getting stuck with $60k worth of future landfill.

You know that all the new bitcoind roads are designed for the atomic powered flying sports car and that your $100k v-8 sports car will have to remain parked in your garage?


Why a cluster? Mining bitcoins is a parallel task: you don't need any failover or load balancing tech.


High-end CPUs and lots of RAM? For mining? Seriously if you want to throw money away you might as well give it to me: I could invest it in hardware that would produce some income and not generate electric bills like there's no tomorrow...


I use USB keys for each of my "nodes". On some of the first USB keys filesystems were remounted read-only on medium errors: the mining process was undisturbed...


You think so? You have a grand total of 12*4*2 = 96 GPU. If these are 7970 you may have a total of 96 * 700MH/s (overclocked) = 67.2GH/s for $60k.

The fact that you have 12 GPUs/node will certainly be a problem (there's a limitation of 8 GPUs/server in the AMD drivers).

So you will probably be restricted to 64 GPUs -> 44.8GH/s (still for $60k). Being in a datacenter, power won't come cheap: even if you manage to put these 44.8GH/s online you may not even make a dime mining Bitcoins (and unless the BTC price rise to absurd levels you certainly will begin to lose money in one or two month).

For a point of reference I spent ~$7k for 12GH/s one year ago: your hardware is 3x less dollar efficient than mine one year later. Mining is about optimizing the cost of hardware and the power usage, your setup is simply over-engineered.

If you can post some pictures..those are some big money machines. :o

This is a gigantic waste of money. An Avalon mining rig would use less power and cost 10% of your rig, for the same amount of hashpower. Your setup is depreciating so fast it is more efficient to avoid the sunk cost fallacy and sell the entire thing now. All it will accomplish if it runs is contributing to global warming.

First: Ouch!

Second: I suggest you compare mining litecoins when the btc difficulty starts to skyrocket soon.

Very bad timing on the purchase but best of luck..hopefully you can at least break even on it

Just to clarify I have 12 GPU cards.    I do things for failover and redundancy because that is how I set up everything that handles potential income.  It might be excessive but it is just a habit.

As far as ASIC goes,  How many people do you know personally with ASIC rigs? 

As far as my huge amount of memory, I will need that to run numerous instances.  One thing I am just checking out is the possibility of adding nodes even if I could virtualize an ASIC chip.  They have such low power useage and would take up low CPU and perhaps GPU cycles that if I can virtualize and ASIC instance , then maybe this rig would not be a waste of money.

My one test was a dismal failure at about 26600 Mh but I was testing on cracking phpBB3 hashes when I got that speed, so I will see for sure in a day or two.

Most likely this will be a bad investment though.  I can face that but I am going to look for ways to "hedge" a potential loss.

No pictures yet.  On the off chance this does work, I really don't want copycats with my rig chugging away at what could be my bitcoins.  If it works I might be open to "leasing" as part of my hedge plan.  Being that if it works, I lease instances as a hedge against BTC for dollars.

Very interesting choice on BSD for the OS as opposed to win/linux.  How's the driver support?

I'm guessing copycats are the absolute least of your worries. I hope you have a use for this hardware beyond mining.

The controller is *BSD.  The rest is CentOS.

Heh. You are probably correct.  But it is unique on the inside but really, really boring on the outside (quad 6U cases being driven to two racks somewhere now). 

Well, according to the Bitcoinx mining profitability calculator, that hashrate would gross $4403.21/month.  Granted, you did spend a lot for your rig, but I wouldn't call that a dismal failure!

Pics or it didn't happen.

Failover means that when , lets say, The State of Missouri is destroyed in an act of clumsiness typical of that state and my rig is destroyed with it, my other rig either surrenders half of its tasks and takes on the tasks of the dead Missouri rig or it takes over in full by creating more instances to make up for those lost.  This is why I need huge logging space because the failover rig will have to collect data from the failed rig.  The colos provide 24/7 AC and backup power so I figured I might as well take advantage of them. 

Redundant means they are the same and have the ability to failover.

It'd be nice if lack of pics could get me a refund.

But here we are.  Things happened.

+1

We want pics! Anyway, what an ugly choose! 512GB of ram, you are trolling us, correct?


Yes... you must be really trolling.

GG :)

Do you think I could use my cluster for trolling and make some trollcoin with it?

I think what he was asking is how you are making this work for bitcoin.
Failing over the running applications does not help you because there is no "progress" in bitcoin mining.

Failover and cluster are two concepts that doesn't really apply to mining.

Certainly  http://temporary-playshadowrun-com-forums.1045820.n5.nabble.com/file/n5706802/troll.png

http://ohgovernment.com/wp-content/uploads/2013/01/coin.png


Seriously now, some pics would be nice. They worth more than 1k words.

Ok.  Hmmm...  More to test when it gets set up then.

Crap.

I think I may have been a tad done in by hubris in thinking clusters are better at everything.

Does google face recognition work for pictures of your rig?

What GPUS are you going to be using?

Unless I'm mistaken phpBB3 uses md5 hashes (~4x quicker to compute than sha256d) which fits with TCollar saying there's only 12 GPU total (~8GH/s with 7970 for sha256d))...

I hope it's all a big joke...

Can you promise to ship in the next .. oh 2 months?

If so, where is the pre-order page?  I'm looking for a 2014 delivery.  ;D

Epic fail


Doesn't matter, what matters is the Bitcoin difficulty and the speed at which it rises. If you think it rises because of new GPU miners, please share whatever you are smoking...


Virtualize an ASIC chip? You really don't know what you are talking about.


I suspect LSD now.

I am starting to wish it was.  But I did take into account the speed difference.  We will see in my test tomorrow.  I might only lose half at this point if I part everything out on e-bay.

stuffthatdidnthappen.txt

Why would anyone buy a $90K of hardware without researching first?

Well I watched $70K disappear because I didn't set stops on some short options. I was near death at the time, so my only excuse is that it wasn't me who made that decision.

I'm going to lay this out pure and simple.

You made a huge financial miscalculation out of sheer ignorance.  My opinion is that you would be wise to cut your losses and sell the thing immediately.  If ASICs reach consumers at anywhere near the scale that a company like BFL intends to do within the next few months, you will likely never earn more than a maximum of $10,000-$30,000 (if you're lucky) before the difficulty gets so high that all of your useless extras start costing you more in electrical costs than what you'll earn generating BTC.

...

Wait a second...

You only have 12 GPUs?  HOLY DEAR GOD SELL!!!

This rig is called a sunk cost (https://en.wikipedia.org/wiki/Sunk_costs). Believe it or not, selling the thing now is the financially optimal choice.

You should start mining Litecoin NOW, and make bank.

edit: http://dustcoin.com/mining

For much less than you paid, you could have instead won the bid for this

http://www.ebay.com/itm/Avalon-ASIC-Bitcoin-Miner-System-Rig-65-Gh-s-Guaranteed-Batch-2-PRE-ORDER-/171013152886?pt=Desktop_PCs&hash=item27d12da076

Last one on ebay sold for $20,600. As suggested, best way to try to make profit with what you have is to mine Litecoin.

also, do not immediately convert mined LTC to BTC...hold until LTC reaches $1

There is no data to collect, No reason to fail over. Your rigs are running calculations and reporting the results to the pool operator. If you want to solo mine, you may want redundancy in your main wallet but not for the whole rig. JUST RUN BOTH RIGS. No need for virtualization either. Like the guy up there said, you can run the os on thumbdrives with write errors. Hopefully those 96gpu are 6990's

I can't see how photographs would help someone build a competing rig.

This guy is either a troll or batshit insane. He has no clue what he's talking about and he's completely lying about everything. Op please setup an appt with your psychologist. Everyone needs to stop interacting with him.

And what happens when the ASICs hit the scene. You think you can redeem your investment?

He's a pathological liar

He may be testing how gullible the Bitcoiners are:

https://bitcointalk.org/index.php?topic=158111.0

Rid yourself of that beast!

somebody checked if is it true this http://dustcoin.com/mining ?

really mining ltc is more profitable than mining btc with gpus ?

I don't like you either

Sell it all as soon as possible to recover as much as possible money; you have no idea whatsoever how to build a mining rig that will not be a liability (not to mention that the days of GPU mining are nearing the end). Doesn't matter which coin you will want to mine it's not worth with what you've built.

So...? Were you trolling us or did you really buy 60K worth of equipment without properly researching Bitcoin? You didn't even know how mining worked? Don't tell me you thought it would be a good idea to mine on all those CPU cores too?

Because spending 60K on hardware instead of Bitcoin means you expected to make back 750BTC with ^{(12 GPUs X 4 systems X 600MH/s (assuming they are all 7970s)=)} 28.8GH/s

There are people that don't see reality like we do. He believes he bought all this stuff. He's lying. He needs to see his psychiatrist immediately. Or he could be a rich kid that is in a manic episode.

I had my doubts before it started running But it is doing OK.  Not ASIC level but much better than projected (59GH per rig ) So yes, massive expense.  Butterfly Labs is a much better deal and all those people who have those much, much cheaper rigs than me sure are laughing at me now. (how many people is that?  two maybe?).

I know clusters.  I'm 48 years old and have been doing clusters since 2000.  I know financial data modeling and I know how to make this stuff sing and it has.  Not at ASIC level but much better than I thought it would.  Still, my loss or my profit, but I got my hardware delivered.

I am sure BFL will deliver their product right away, and be just as reputable as bASIC was, then I will be sorry.

You don't have any hardware. You are out of touch with reality.

As a basketball machine elf I beg to differ.

What universe are you on?

I don't really understand your justification for the price.
Your mining wont nearly pay itself as fast as the ASIC stuff. 60k of ASIC would have gotten you so much more power.

I stuck with what I know and that is crazy ass hardware.  I know 60k of ASIC would get me more power.  Man, 60K of ASIC would get me 3TH which would then make me buy a very large boat and sail to Richard Branson's island and kick him in the testicles.

But ASICs are not being delivered.  Avalons...yes, kinda.  But there is something in the "too good to be true" in the typical ASIC promises that seem to be filled with slowness at best and felons at worst that made me decide that if I was going to gamble I was going to do it on something I knew I could get my hands on.

How many ASIC rigs are out there right now do you think?

The number suggest it is the Godot of hardware and we can always replace lost money but we can never replace lost time.

I think they will ship.
If BFL ships everything, non-ASIC gear will just be a waste of electricity.

If it's not Avalon or BFL that ends up shipping, someone else will do it. Technology is here and market is here, there has to be people making the gear availible.

You are right in that if it ships I will be stuck with rigs that will not be totally worthless, but I will lose money on them.  I had an offer to lease them out for animation work but that won't pay for them.

You are also right about your prediction that someone will ship, but chip makers are a conservative lot and making and shipping chips that have the promise of having productivity halved in the future is something that makes them skittish.  Also combined with the possibility of a bitcoin crash (always possible) that too is likely to keep the chip production down.

So I jumped in while others were waiting.  I might lose a chunk of cash.  I might mine a ton of bitcoins and have the bitcoin crash on me.  ASIC rigs could end up fitting on wrists and generate "free money" for the wearers.  I don't know.  In fact, the economic modeling I do in my cluster really cannot tell me what will happen either since it has only been around since 2009 and really only been a currency for less time than that.  So my economic models say 0.00, which means I built a cluster that says I should never have built a cluster, but I still think it is a better bet than ASIC.

So you have decided to keep everything? And bitpop has decided to keep trolling because he is bored?

I would like to see how this pans out but purchasing 750BTC with that money now would have been a better investment. My calculations and predictions tell me that I should expect less than 2BTC per week at 60GH/s by the end of this year.

Just to make sure, $60K is something you are willing to lose? Are you in that kind of financial position?

if you're in a position to hemmorhage cash like that, you should point the wound here: 1KvB9Nv9xvkNg1LPNkWL6WLHJJQhtopYJu

unlike your hardware, i'll actually appreciate it! :D

bitpop has reason to be suspicious.  I just show up and say "here is what I did!" so I got no problem with that.  It makes sense actually, but I gotta tell someone, somewhere.

I lost more on housing in 2008 but not that much.  The trick is that when you lose you try to take that lesson and make sure you lose better each and every time (I think Samuel Beckett said something like that). So yes, it was either this or buy a midlife crisis sports car.  Buying a midlife crisis sports car is an awful choice because middle aged men look ridiculous in them with wrinkly faces and bald spot flowing in the wind.

I've just decided to focus on what I know and if I fail I will just say "I was wrong" because it won't be the first time and I hope it won't be the last either because the only time we stop failing is when we are dead.

How do you know my hardware doesn't love me?

I think what you did was a calculated risk that was well reasoned. Mine the crap out of BTC before the ASICS arrive in force.
Screw BFL, I wouldn't worry about them. They haven't even demonstrated a working prototype.
The Avalon guys, on the other hand, could really start to make things difficult by mid summer to next fall once batch 3 is shipped....I say once your net profit starts dropping like a rock, switch over and start mining litecoin instead.
I think litecoin and bitcoin are kind of symbiotic...lots of GPU miners who get chased out of bitcoin by the ASICs are gonna have two choices: mine litecoin without worry of ASIC competition and still be able to trade it for USD or BTC, or sell their stuff. The Litecoin network is going to offer resource redundancy (it 'stores' yesteryear's computing power and equipment), kind of how the used car market still provides a useful transportation for people. 
Litecoin will always play second fiddle to bitcoin, but it has demonstrated staying power and offers an alternative to Bitcoin that has enough advantages to make it a viable alternative (and some might argue, complementary) currency.

I think you should make changes to the hardware to correct the wrong assumptions you made, in order to divert capital to other projects or recover it.

Take the important bits (the GPU cards) out of all the expensive hardware and repurpose that hardware right where it is and sell it or lease it to someone that really does need what it offers.

Then put the GPU cards into cheaper hardware and put them to work, before ASIC's spoil the party. They don't need to be in geographically diverse locations. Stick them all in a location with the cheapest possible power costs, set them running and recover some funds.

You could start selling hot air.

/me Grabbed some popcorn and hit the "watch" button.

I am so for Pics or it Didn't happen on this one. I don't understand why you would spend that much. If I had 10k right now id be buying BTC and flipping having and hoarding as many coins as I possibly could. I am doing that with less as we speak!! Good Luck though, I would love to see pictures. No one is copying your Rig, TRUST ME!! 8)

I'm mining right now.  The second rig will be up soon.  The other hardware has actually suited my needs in spite of being overkill because I have used the CPUs and RAM to do some modeling on pools and their mining success and times when it is most likely to happen.  I am at 59KH/s now and I hope to double that by today or tomorrow. But it is still a costly prospect for me but I am up and running unlike most of the ASIC folks.

I never took pics while I was building it.  I have actually never taken a picture of a computer in my life except for at work when something went wrong and needed a photo back up for a meeting.

But, I am going to ask the folks down at the colo to take a pic for me, if you do not mind seeing racked computers next to other racked computers.

To any noob who might be mislead the OP is full of crap.  The numbers simply make no sense.  First of all the fact that he didn't realize that redundancy doesn't matter, nor does high end CPU, nor does massive amounts of ram or storage should be a clue.

Mining is perfectly parallel.  There is no need for inter hasher communication just some very very basic control and management.  A low end sempron is more than enough to run OS & cgminer.

Still that would just mean the (imaginary) rig is just a bad investment.  The OP stated 12 GPU and 59 GH/s.  There is no GPU which can produce ~5GH/s.   A 5970 heavily overclocked can get about 850 MH/s but it isn't going to be stable, cool, or efficient and getting more than 800 MH/s out of 7970 is unlikely.  More realistic 24/7 efficiently clocked GPU are going to get much less. OP didn't build anything but a tall tale.

or TL/DR version:

Not to mention that there isn't any OS that supports more than 8 GPU's native (as i recall!)

You are right about the redundancy part not being useful.  I made what I knew and failed at that part.

I have two 8 core cpus per board at 4 per cluster.  CentOS and many other versions of Linux support 8 CPUs just fine.  Mine is actually small compared to many others I have worked with.

Now that is comedy gold right there.

Yep, turns out that would be impossible to very hard.  So I have stopped looking into that possibility.

But still, the promise of my virtual ASIC chip, as improbable as it is, is still about the same delivery rate as the real ASIC chip makers are doing.  THAT is comedy gold there, except that real people are getting really ripped off.  I just made a foolish purchase thinking that my knowledge of another field would make me do good in this one and I am not doing too bad. I am not going to clear my initial investment if I keep going as I am going, but others fell for a "Straw to Gold" scheme and while calling me foolish (which I admit has truth to it) they are singing the praises of non-existent hardware that they paid money for.

So, turns out, this is just another BFL bashing thread.  Very clever and entertaining though.  You get the Pinkie Pie award for this week!

Good. Because you obviously don't know what an ASIC is. You know what virtualizing an ASIC means? It means buying an FPGA.

If you go back you will see people telling me how foolish I am to not get an ASIC rig. My point is they barely exist.  They are like trying to find sasquatch on a day where it is too hot to wear an ape suit.

So, yah, a bit of bashing but I had to put up with GPU and cluster bashing so I am sure ASIC and BFL can take my criticisms just fine.

Meanwhile I will just row across the lake while listening to taunts from people who bought a speedboat they don't have yet.

ASIC or not you didn't build ANYTHING.  It isn't that you super rig is pointless or not economical .... it doesn't exist.   You claim to get 59 GH/s out of 12 GPU.  That is a 100% lie.   You made up a stupid lie, got caught, now own up to it.

Hmmm. BTW, what do you think this is?

https://i.imgur.com/oEIjyj8.jpg

Besides boxes of fans and blue lights of course

You didn't took into consideration the intel eptacores and the various TB of ram. That has to speedup the rig somehow. Not sure of how!

Did I misread? I thought he said that that rate was between 4 systems. 4x 12GPUs = 48GPUs. My estimates before he posted his rates were based on them all being 7970s. His new rate is possible with them all being this: http://www.newegg.ca/Product/Product.aspx?Item=N82E16814131483&Tpk=7990&IsVirtualParent=1

Or similar by another brand. Considering the official 7990 has not been released and the limited quantities of custom designs from other vendors, it is incredible that he was able to get 48 of them.

Also insane that they aren't overheating all to crap, never mind even fitting due to the coolers. What's your solution for that?

It looks like a shelf of ASIC rigs,  Probably from Avalon.

Easy to buy those Avalons, aren't they?  ASIC chip production is just increasing exponentially with the demand huh? 

cooling solution is a colocation facility. 

66F to 68F 24/7.

DROOOOOOOOOL! let me play!

What you're paying for the colo?



You better put those "beasts" at work and minimize the losses, here http://www.openstack.org/software/, or just mine some litecoins at the same time http://litecoin.org

If I were in your shoes, I would be more than pissed with the whole thing.

About as easy to buy as an Absurdocluster of OverpriChips and a bunch of FAIL safes. But way cheaper.

Got a bargain on the colo (managed too) since my employer is a good customer of theirs I got setup for a BBQ and beer for the staff who did it next time I am around and I pay about $200.00 per month for both locations.

As far as being pissed goes, It may be a money pit but it is working better than I though.  If the price drops below $20 then I will be pulling my hair out (or what hair I have) .

Why?  If the exchange rate falls to $20 (or difficulty increases by a factor of 10) why not just imagine up a higher hashrate and make even more "profit"?  I think you could install a couple of quantum doublers and vortex fans and get those "rigs" up to 500 GH/s each pretty easy.

I hate you.


But I love you.