[50 BTC total bounty] for Groupcoin development and help

[markm]

EDIT: oops, I had generator_0.csv instead of receiver_0.csv !

Trying again...

Still got errors, plus it made few more receiver-#.csv files with bitcoin addresses instead of groupcoin addresses in them

Code:

 Traceback (most recent call last):
  File "receiver.py", line 76, in <module>
    main()
  File "receiver.py", line 73, in main
    writeOutput(sys.argv)
  File "receiver.py", line 67, in writeOutput
    text = getOutput(directoryName, fileName, int(heightString), float(shareString), int(stepString))
  File "receiver.py", line 44, in getOutput
    modulo = remainder % len(coinLists)
ZeroDivisionError: integer division or modulo by zero
Traceback (most recent call last):
  File "receiver.py", line 76, in <module>
    main()
  File "receiver.py", line 73, in main
    writeOutput(sys.argv)
  File "receiver.py", line 67, in writeOutput
    text = getOutput(directoryName, fileName, int(heightString), float(shareString), int(stepString))
  File "receiver.py", line 44, in getOutput
    modulo = remainder % len(coinLists)
ZeroDivisionError: integer division or modulo by zero
Traceback (most recent call last):
  File "receiver.py", line 76, in <module>
    main()
  File "receiver.py", line 73, in main
    writeOutput(sys.argv)
  File "receiver.py", line 67, in writeOutput
    text = getOutput(directoryName, fileName, int(heightString), float(shareString), int(stepString))
  File "receiver.py", line 44, in getOutput
    modulo = remainder % len(coinLists)
ZeroDivisionError: integer division or modulo by zero
Traceback (most recent call last):
  File "receiver.py", line 76, in <module>
    main()
  File "receiver.py", line 73, in main
    writeOutput(sys.argv)
  File "receiver.py", line 67, in writeOutput
    text = getOutput(directoryName, fileName, int(heightString), float(shareString), int(stepString))
  File "receiver.py", line 44, in getOutput
    modulo = remainder % len(coinLists)
ZeroDivisionError: integer division or modulo by zero

-MarkM-

[1715062722]

1715062722

[1715062722]

1715062722

[1715062722]

1715062722

[1715062722]

1715062722

[1715062722]

1715062722

[1715062722]

1715062722

[Unthinkingbit]

By the way, just in case you didn't see the last edit:

EDIT: I just noticed that the block number is 5108, I thought it wouldn't be more than 2000 for a week.  So for testing, change the step to 100,000.  To do that replace:

"receiver.py "

with:

"receiver.py -step 100000 -height "

[Unthinkingbit]

EDIT: oops, I had generator_0.csv instead of receiver_0.csv !

Trying again...

Still got errors, plus it made few more receiver-#.csv files with bitcoin addresses instead of groupcoin addresses in them

Did you see the edit and try replacing:

"receiver.py "

with:

"receiver.py -step 100000 -height "

[markm]

Yes, that gives memory errors:

Code:

ThreadBitcoinMiner exiting, 3 threads remaining
Running BitcoinMiner with 2 transactions in block
Running BitcoinMiner with 2 transactions in block
Running BitcoinMiner with 2 transactions in block
received: verack (0 bytes)
received: getaddr (0 bytes)
received: getblocks (805 bytes)
getblocks -1 to 00000000000000000000 limit 500
sending: addr (6451 bytes) 
No valid UPnP IGDs found
07/14/11 04:04 hashmeter   3 CPUs   1373 khash/s
keypool reserve 1687
BitcoinMiner:
proof-of-work found  
  hash: 0000000010f5a34386a950f513f882e267a7b209a06d12b3c9a5c62b12230956  
target: 0000000014c1a700000000000000000000000000000000000000000000000000
CBlock(hash=0000000010f5a34386a9, ver=1, hashPrevBlock=00000000124e9fdb6c96, hashMerkleRoot=3e5d46afd4, nTime=1310616451, nBits=1c14c1a7, nNonce=3761439628, vtx=2)
  CTransaction(hash=2346014636, ver=1, vin.size=1, vout.size=1, nLockTime=0)
    CTxIn(COutPoint(0000000000, -1), coinbase 04a7c1141c0135)
    CTxOut(nValue=50.00000000, scriptPubKey=049bbea080bd5b23ae401f6d1763ba)
  CTransaction(hash=6d3e0083a2, ver=1, vin.size=1, vout.size=2, nLockTime=0)
    CTxIn(COutPoint(394bf27d17, 0), scriptSig=304502200e7ebd73e2375a9d)
    CTxOut(nValue=25.26880000, scriptPubKey=OP_DUP OP_HASH160 ce4a95fb61d2)
    CTxOut(nValue=0.02000000, scriptPubKey=OP_DUP OP_HASH160 d4d5b4cae13b)
  vMerkleTree: 2346014636 6d3e0083a2 3e5d46afd4 
07/14/11 04:07 generated 50.00
keypool keep 1687
WalletUpdateSpent found spent coin 25.2888bc 394bf27d17ed6799ef16f5f853ad79e3673997b4ae53a13812dc1d07402acdf8
AddToWallet 2346014636  new
AddToWallet 6d3e0083a2  new
SetBestChain: new best=0000000010f5a34386a9  height=5111  work=92566489266000
ProcessBlock: ACCEPTED
sending: inv (37 bytes) 
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f701409a010 ***
received: getdata (37 bytes)
received getdata for: block 0000000010f5a34386a9
sending: block (473 bytes) 
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f7014097010 ***
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f701409c010 ***
keypool reserve 1799
07/14/11 04:07:40 Flushing wallet.dat
^C

Notice also that for some reason one of my four CPU's threads was stopped right at the start.

(Maybe though that could be caused by the miner connecting, maybe it drops one of its CPU mining threads in favour of feeding getwork to the miner? Hmm not sure.)

On the bright side though, the other machine did receive the 0.02

EDIT: the string you are returning from your getPythonResult routine is allocated inside that routine, thus is a local variable of that routine. It would need to be copied into some larger-scope allocated memory to survive beyond the invocation of the routine. That might be the memory problem. Do std::string objects auto-allocate space for themselves or do you have to do that explicitly like in C ?

-MarkM-

[Unthinkingbit]

Yes, that gives memory errors:
..

On the bright side though, the other machine did receive the 0.02

Since there were no python errors posted and the other machine received coins, that probably means receiver.py is working correctly.

Code:

ThreadBitcoinMiner exiting, 3 threads remaining
..

Memory errors and a thread dying indicate it could be a popen thread problem.  Popen is apparently not thread safe:
http://lists.uclibc.org/pipermail/uclibc-cvs/2011-May/029585.html

So please try running the miner with only one thread.  If that fixes the error, then unfortunately, the only certain way I can see to fix the problem is to port all of receiver.py and pluribusunum to C++ in devcoin.  Otherwise, maybe there is some thread safe way of calling python code from a multithreaded C++ application.

[markm]

Threads might turn out to be a problem, but might it simply be that the memory you allocated for the result of your getPythonResult routine gets de-allocated when that routine ends, thus leaving the char* pointers referring to it pointing at de-allocated memory?

In other words did you see my edit above:

EDIT: the string you are returning from your getPythonResult routine is allocated inside that routine, thus is a local variable of that routine. It would need to be copied into some larger-scope allocated memory to survive beyond the invocation of the routine. That might be the memory problem. Do std::string objects auto-allocate space for themselves or do you have to do that explicitly like in C ?

EDIT AGAIN: I don't understand C++ scope, I tried moving the returned string's allocation to just above the function so it would exist outside the function, but then inside the function claimed it was not in scope! So I guess I have no idea how to use variables that exist outside a function from inside the function. Weird as surely lots of functions are using variables declared outside them, I just haven't figured out how they decide which ones they can use and which ones they will decide are "out of scope"...

-MarkM-

[Unthinkingbit]

EDIT: the string you are returning from your getPythonResult routine is allocated inside that routine, thus is a local variable of that routine. It would need to be copied into some larger-scope allocated memory to survive beyond the invocation of the routine. That might be the memory problem. Do std::string objects auto-allocate space for themselves or do you have to do that explicitly like in C ?

Well, it's worth a try.  Try using the code below.

Code:

    int pathLength = 8192;
    char path[pathLength];
    std::ostringstream heightStream;
    heightStream << pindexPrev->nHeight+1;
    std::string heightString = heightStream.str();
    getPythonResult(std::string("receiver.py -step 100000 -height ") + heightString, path, pathLength)
    std::string payTo = std::string(path);

Code:

void getPythonResult(const std::string& argumentString, char* path, int pathLength)
{
    FILE *filePointer;

    filePointer = popen((std::string("python ") + argumentString).c_str(), "r");

    while (fgets(path, pathLength, filePointer) != NULL)

    int status = pclose(filePointer);
}

EDIT AGAIN: I don't understand C++ scope..

I don't understand it either, I just keep playing around with a trivial test case till it works.

EDIT2:  I just updated the code. deleting the return variable.  Path is only passed through now, it is converted to a std:string outside.

[markm]

I moved the declarations of the crucial items to the very top of the file, just inside the critical block identifier:

//
// Global state
//

CCriticalSection cs_main;

    const int pyResultLength = 8192;
    char pyResult[pyResultLength];




and I changed the routine a little to accomodate that:

std::string getPythonResult(const std::string& argumentString)
{
    FILE *filePointer;

    filePointer = popen((std::string("python ") + argumentString).c_str(), "r");

    while (fgets(pyResult, pyResultLength, filePointer) != NULL)

    int status = pclose(filePointer);
    return std::string(pyResult);
}

This time all four threads ran. But I still got memory errors. I am not confident though that I am not still confusing pointers and arrays. I got compile errors trying to use &pyResult but am not sure just plain old pyResult is actually the same thing as &pyResult. Been a long time since I used C a lot. Gotta think hard on this.

Could be you are right though about it being a thread problem.

-MarkM-

[markm]

I tried it your way too. Still get memory corruption. Notice it happens 4 times. Maybe you are correct about it being the thread-safe problem.

Code:

Running BitcoinMiner with 2 transactions in block
Running BitcoinMiner with 2 transactions in block
Running BitcoinMiner with 2 transactions in block
Running BitcoinMiner with 2 transactions in block
07/14/11 05:15 hashmeter   4 CPUs   1343 khash/s
BitcoinMiner:
proof-of-work found  
  hash: 000000000d2655bd92fe0c45a1aa5176324ab7a07ae7b1fa60ac324e4a96a02b  
target: 0000000014c1a700000000000000000000000000000000000000000000000000
CBlock(hash=000000000d2655bd92fe, ver=1, hashPrevBlock=00000000140c7f022ff7, hashMerkleRoot=bcda71509c, nTime=1310620588, nBits=1c14c1a7, nNonce=411743883, vtx=2)
  CTransaction(hash=54eba8ca6e, ver=1, vin.size=1, vout.size=1, nLockTime=0)
    CTxIn(COutPoint(0000000000, -1), coinbase 04a7c1141c0111)
    CTxOut(nValue=50.00000000, scriptPubKey=04ef6abceb89cf2c0bb444cb55157d)
  CTransaction(hash=b80248d783, ver=1, vin.size=1, vout.size=2, nLockTime=0)
    CTxIn(COutPoint(2b12e8a5e1, 0), scriptSig=3046022100fb67b8e49b74b5)
    CTxOut(nValue=25.23880000, scriptPubKey=OP_DUP OP_HASH160 e148bc3e1d31)
    CTxOut(nValue=0.02000000, scriptPubKey=OP_DUP OP_HASH160 59504bfa9821)
  vMerkleTree: 54eba8ca6e b80248d783 bcda71509c 
07/14/11 05:16 generated 50.00
keypool keep 1679
WalletUpdateSpent found spent coin 25.2688bc 2b12e8a5e165923c8ad80946f2538f940ae1a95339bb94296ee972e31e96476d
AddToWallet 54eba8ca6e  new
AddToWallet b80248d783  new
SetBestChain: new best=000000000d2655bd92fe  height=5112  work=92619461300770
ProcessBlock: ACCEPTED
sending: inv (37 bytes) 
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f5d0c56e010 ***
received: getdata (37 bytes)
received getdata for: block 000000000d2655bd92fe
sending: block (474 bytes) 
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f5d0c56b010 ***
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f5d0c571010 ***
*** glibc detected *** /home/bitcoin/bitcoin/groupcoind-PRIVNET: malloc(): memory corruption: 0x00007f5d0c569010 ***
keypool added key 1901, size=101
keypool reserve 1801
^C

I don't know if I can run only one thread on demand other than by running it on a single-CPU machine, both of which are 32 bit Fedora 14 not the 64 bit Fedora 15 I have been running these tests on so far. I will have to move the code to the 32-bit compiling-machine and try.

If the thing is not thread safe then maybe we can just do some kind of semaphore around it, that example you linked to seemed to be using some kind of bracketing of the endangered area with some kind of signal to keep the threads from trying to all use it at the same time by the look of it.

Using C++ instead of python is probably the least-hackish way to go though of course.

-MarkM-

[Unthinkingbit]

Using C++ instead of python is probably the least-hackish way to go though of course.

Eventually, we'd have to move to C++ for speed in any case.  I'm starting to port over receiver.py, but not all pluribusunum at first.  So at the beginning it would just read from a file on disk, later it would get internet files and so on.

It will take at least a day to get something useful.  In the meantime, as I make a function, I might post it, like the function below:

Code:

// Get the entire text of a file.
std::string getFileText(const std::string& fileName)
{
	std::ifstream fileStream(fileName.c_str());
	if (!fileStream.is_open())
	{
		return std::string();
	}
	std::string fileText;
	fileStream.seekg(0, std::ios::end);
	fileText.reserve(fileStream.tellg());
	fileStream.seekg(0, std::ios::beg);
	fileText.assign((std::istreambuf_iterator<char>(fileStream)), std::istreambuf_iterator<char>());
	return fileText;
}

[Unthinkingbit]

The basic file reading address code is now at:
https://raw.github.com/Unthinkingbit/charity/master/receiver.h

Please download it into the folder which includes the CreateBlock file.  Then, add the following include at the top of the CreateBlock file:

#include "receiver.h"

then in the CreateBlock function, replace:

std::string payTo = "2hix2ZoA175cC1aF6fFwnyt3XuwqQMKPg9M";

with:

std::string payTo = getCoinAddressString(std::string("receiver.csv"), (int)pindexPrev->nHeight+1);


If all works, the miner should send to the receivers in your receiver_0.csv file.

[markm]

That seems to be working.

-MarkM-

[smoothie]

Approximate release date?

[smoothie]

Can someone post up a brief overview/summary of where the groupcoin project is currently and approx. when it will be available for use?

Thanks

[markm]

Currently we have I think about two to four people with clients online listening on port 51333 so that other clients have someone to connect to.

We have people lurking on the #groupcoin channel in Freenode IRC.

We have tested some of the feaures DEVcoin is to have; DEVcoin will be starting from scratch once we have deeloed and tested the code for the desired features.

I have now set the four byte hello sequences for GRouPcoin that wll prevent it fro mtalking to bitcoin, testnet, namecoin etc if it happens to stumble across such a client, best come to the IRC channel for co-ordination in switching to that system all at once to minimise time of not being able to connect.

I have been testing new code privately without affecting the real GRouPcoin blockchain, which should not need any "restart" unless for some reason we are unable to develop DEVcoin (on which development seems to be proceeding okay).

See you at #groupcoin ?

-MarkM-

[cunicula]

I have a fairly complex proposal for a new currency which addresses numerous flaws in bitcoin.
Is there an alternate venue to discuss crpytocurrencies which is not forum.bitcoin.org?

The forum is generally hostile to posts that point out problems with bitcoin and propose possible solutions.
A new discussion platform would be a good first step to generating a sustained flow of innovative ideas.
The new platform doesn't need to tied to any particular project, but it should not be focused on new projects rather than bitcoin.

Is there anything like this around yet?

In the meantime, I will work on a blog to introduce my own proposal in moderate detail (not done yet).

[markm]

Most variants are simply different, they exist not because bitcoin has flaws but because bitcoin works just fine thus they hope that variants will also work. Is it really necessary to take the position that bitcoin is flaws in order to "justify" a variant?

-MarkM-

[cunicula]

Most variants are simply different, they exist not because bitcoin has flaws but because bitcoin works just fine thus they hope that variants will also work. Is it really necessary to take the position that bitcoin is flaws in order to "justify" a variant?

-MarkM-

Yes, in my opinion. But we don't have to agree on that.

[patvarilly]

I have a fairly complex proposal for a new currency which addresses numerous flaws in bitcoin.
Is there an alternate venue to discuss crpytocurrencies which is not forum.bitcoin.org?

The forum is generally hostile to posts that point out problems with bitcoin and propose possible solutions.
A new discussion platform would be a good first step to generating a sustained flow of innovative ideas.
The new platform doesn't need to tied to any particular project, but it should not be focused on new projects rather than bitcoin.

Is there anything like this around yet?

In the meantime, I will work on a blog to introduce my own proposal in moderate detail (not done yet).

This sounds like a workable site to host such a thing: http://www.freeforums.org/.

[caston]

So can someone please write a quick guide on how to start mining on the groupcoin existing network or its testnet?

I have tried to work this out by going back and reading posts but I am a bit confused. I am using a Linux coin harddrive install and have compiled multicoin.