DIY miner software (miner from scratch)

[blasthash]

Hey guys,

I was wondering if you could give me some ideas on how to build a miner from scratch. This is a half-serious/half-fun project. What I want to build is a Java miner that's capable of handling the whole gamut of mining in a platform-independent design that'll run without the dependencies (libcurl, etc.) that cgminer requires (as I've been fighting cgminer to get it to install for months with no avail )

Once I know how the mining process is achieved, I can proceed to lay it into code, and I know how to handle well in code, but how is the process acheived? For lack of a better world, because I'm still relatively new to cryptos, what is exactly going on during the mining process?

Obviously, if I'm able to pull it off, I'll make it available to all of you for testing and use and what not.,

Thanks, guys.

[1715109200]

1715109200

[1715109200]

1715109200

[dancrn]

i wrote a CPU and OpenCL miner from scratch, in C, with no libraries.

it's not as fun as you might think. http is easy to hack together in any language (although the edge cases are.. bleh). once you get your head around the endian changes and stuff, it's really simple.

cgminer is a monster, so i don't blame you.

thanks,

[blasthash]

i wrote a CPU and OpenCL miner from scratch, in C, with no libraries.

it's not as fun as you might think. http is easy to hack together in any language (although the edge cases are.. bleh). once you get your head around the endian changes and stuff, it's really simple.

cgminer is a monster, so i don't blame you.

thanks,

Makes sense.

So how do the fundamental blocks of a miner work? Does it go something like this:

1. Miner makes a HTTP request to the pool address using username (worker included) and worker pass
2. HTTP connect responds with block

The parts I'm lost on are what the actual miner does to the block, and how it transmits the data back.

Ideally I'll be using an OO language like Java and I might release an Obj-C version on Mac so that I can use the high-level classes that are designed to make things like HTTP access and GUIs much nicer and much simpler.

[dancrn]

if you're trying to make it portable, you'll wanna stay away from obj-c, unless you want gnustep as a requirement.. java has enough to make it cross platform.

a java miner has already been done - diablominer. you should take a look at that.

my miner uses the getwork protocol, most pools prefer stratum. it allows the mining client to generate more work for its miners per request, and is just generally better. you should read the protocol description, it's fairly exhaustive..!

thanks,

[blasthash]

if you're trying to make it portable, you'll wanna stay away from obj-c, unless you want gnustep as a requirement.. java has enough to make it cross platform.

a java miner has already been done - diablominer. you should take a look at that.

my miner uses the getwork protocol, most pools prefer stratum. it allows the mining client to generate more work for its miners per request, and is just generally better. you should read the protocol description, it's fairly exhaustive..!

thanks,

Will do, thanks for the info.

I found some more info on the mining process, and many suggest something of the sort:

1: JSON request the block
2: Hash the function per SHA-256.
3: JSON the result (?) back to server

Does this sound correct? Also, what is being transmitted back?

I'll most likely write the main release of the app in Java, and I was saying I might also end up writing a Obj-C variety for the Mac specifically.

[-ck]

Let me get this straight, you can't compile cgminer but you want to write your own mining software?

[blasthash]

Let me get this straight, you can't compile cgminer but you want to write your own mining software?

Let me lay a few things out.

It's not that I can't compile it because it's too conceptually difficult. It's because as a Windows/Mac developer I am very much unaccustomed to the Linux dependencies that it requires, libcurl etc. When I was trying to install/comp on my Ubuntu machine, it was giving me all sorts of problems such as 'cannot find libcurl' when I had installed it the step before. Confusing problems. It wasn't that I couldn't compile it, it was that I could never get to the point where the system had the necessary tools. A combination of my machine being a piss-ass (this isn't the first time it's done things like this) and me getting too frustrated over time led to me to stop attempting it.

If you're going to crucify me for this, don't bother.

Second off, I still believe in the idea of 'education by doing'. I don't care if this miner I write isn't optimized for half a damn, it'll be my own and I will have learned something about the bitcoin/litecoin process while doing it. I'm by no means a programming guru; I haven't written my own compiler yet if that's the benchmark for skill we're going by, but I have done quite a bit and my fair share of work in at least 4 different languages.

Half of the reason being, I haven't worked with server-client communications (i.e. JSONs) and I'd like a reason to get familiar with them before I actually have a legitimate requirement to be. Also; I'm a much better hardware engineer than I am software, and I can take what I get from the process and apply it here.

[-ck]

Ah apologies then, for I didn't know you were speaking about compiling it on windows. I agree, compiling cgminer for windows is bullshit as a result of it being primarily developed as a unix program.

[blasthash]

Ah apologies then, for I didn't know you were speaking about compiling it on windows. I agree, compiling cgminer for windows is bullshit as a result of it being primarily developed as a unix program.

One of the reasons I'm asking is to actually build a miner; the other part is so I know how the process works so I can write the processing assembly for the project I linked to. That project ideally would use a main computer with the program to put values into memory, where the processing array fulfills the task and pushes data back.

The idea is that the computer with the 'miner' interface would only be a server-side link and control configuration options - the actual 'processing' would be done by the Cell array. If you'd like to help me develop that concept, I'd be more than happy to share credit and designs.

So can you provide me with any insight as to how any of the following works: (?)

1. My understanding of the process is the the client-side makes a HTTP/JSON request to the pool server with the authentication details and receives the block to be hashed. It then runs that through a hashing function in C and returns the result. Is this correct?
2. How does the processing of Scrypt vary? My intention is to develop the array primarily for LTC mining.
3. Do you have any information on the various protocols? (stratum, long polling, etc.)?

My apologies for a bit of a defensive tone on the last post.

[-ck]

https://en.bitcoin.it/wiki/Block_hashing_algorithm
https://en.bitcoin.it/wiki/Getwork
https://bitcointalk.org/index.php?topic=108533.0
http://mining.bitcoin.cz/stratum-mining/

[blasthash]

Now, here's the multi-million dollar question:

How are these done on a multi-core basis? Is each core working toward its own independent block, or are the cores synergistically working toward a single block as a whole? Through the use of vector pipelining I can, in assembly, theoretically configure each core of the 8 vector cores on the Cell to run 64 block operations as vectors. But in a multi-core approach is a miner putting different blocks to the cores or merely giving different elements of the work to each core?

[Schleicher]

Since the nonce is a 32bit value you can theoretically use 4 billion cores parallel.

[blasthash]

https://dl.dropboxusercontent.com/u/34410241/Bitcoinmining.jpg
Since the nonce is a 32bit value you can theoretically use 4 billion cores parallel.

Wow, that flow chart puts it surprisingly well.

Now, for Scrypt, is the process similar? (That is, it hashes the block per Scrypt algorithm twice)?

[Schleicher]

Now, for Scrypt, is the process similar? (That is, it hashes the block per Scrypt algorithm twice)?

Unfortunately I don't know much about the way the Scrypt coins work.
I think they probably only used Scrypt instead of SHA256 with everything else the same.

[blasthash]

Now, for Scrypt, is the process similar? (That is, it hashes the block per Scrypt algorithm twice)?

Unfortunately I don't know much about the way the Scrypt coins work.
I think they probably only used Scrypt instead of SHA256 with everything else the same.

Okay, that's alright.

What is the target value from the flowchart?

[Schleicher]

What is the target value from the flowchart?

You get that from the pool.
A target of
00000000 ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
 means difficulty 1, lower target means more difficult.