OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.1

[-ck]

Auspicious occasion, solo mining confirmed working (no it's not my block).

https://blockchain.info/tx/f335e79ce0a8130efca4dcf0efc6e05f9e616e63a89ed8c2747ddd8d5de09d9e

The coinbase gives away that it was mined with cgminer, see decoded

He was rewarded 25 BTC = $14 000 USD

Just a little bit of money

Well he was mining with almost 200TH (aimed at just one bitcoind!) confirming nicely the scaleability of cgminer's solo mining. And here's another just for good measure:
https://blockchain.info/tx/1f59b91615ea0f9e6c633f7c426daa367cde119b03a36ad235def2236b9c0f7d

There are a number of small improvements/fixes going into the solo mining code as a consequence of this testing which should be wrapped up into a minor bugfix release soon. There is one showstopper that makes solo mining not work properly on big endian machines (like antminers) which warrants this release. The other limitation is that bitcoind does not like a lot of persistent connections at the same time so I have reworked the code to drop connections as much as possible allowing multiple cgminer instances to connect to the one bitcoind.

[techman05]

CKOlvias its wierd that at least for the moment under both conditions they were all detected on this turn. Whatever the deal they are almost imediatly in both situations being piped to the end of the list

I got 1 anu listed device showing its only had 2 sharse to work on and zero hardware errors and one with 146 with 2 errors that just zombied with the overclock its showed about 7 minutes in the zombie. They're both mostly doing the normal blinking but no continous solid green light. When I see a light go on I cant corespond it to any errors on the screen.

In verbatim I think when one of them goes solid for a moment its getting discarded work. The zombie one is still in a work state(not solid green) but doing nothing.

All I know for certain is I got nothing from them when I place them in my aitech 3.0 hub with 5 anus and a drill bit thumb and I've got about the same on my roswill 2.0 device. Each have 10 ports and 4 amps to spare.

[os2sam]

There are a number of small improvements/fixes going into the solo mining code as a consequence of this testing which should be wrapped up into a minor bugfix release soon.

If there is going to be an update soon could I ask for

1.  Bitcoin Address displayed to verify which address the block reward would go to?  I saw that you said you would add this.

2.  Ability to sign the block?

I'm pretty happy with my testing so far.  But the memory utilization is still climbing, 408MB so far.
Thanks,
Sam

[os2sam]

The other limitation is that bitcoind does not like a lot of persistent connections at the same time so I have reworked the code to drop connections as much as possible allowing multiple cgminer instances to connect to the one bitcoind.

Would this include the connections to the Bitcoin network?  My Bitcoin client currently has 70 connections.  Should I be restricting the max connections in the client itself?

Sam

[os2sam]

I do have two different machines I'm mining with and 4.2 seems to be working pretty good on one.  But the other machine if when I try to run any 4.x version a bunch of AMU's get turned off and mining is very erratic.  It's a low end P3 with 512MB RAM but it works fine with CGMiner versions 3.11 and earlier.  Currently running 3.9 with 31 BE's.

[kano]

Edit: there's a later 4.3.2a update here: https://bitcointalk.org/index.php?topic=28402.msg6533537#msg6533537

Here's another new AntS1 binary.

An endian issue was found in the solo code for the ant (no issue for desktop computers), so this update includes the fix for that.
I've removed both old AntS1 binaries

This uses the same patch as before for the luci display.
My source for this version is here:
https://github.com/kanoi/cgminer/tree/ants1-4.2.0-00567a4

The binary and README are in my cgminer-binaries git here:
https://github.com/kanoi/cgminer-binaries/tree/master/AntS1  <-- Follow this link to get the new binary

Read the README on the screen there for how to replace the /usr/bin/cgminer binary in your AntS1 with the new one.

This binary includes all cgminer changes up to 4.2.0 and forward after that up to:
https://github.com/ckolivas/cgminer/commit/cec48cee5f1759e5996de5339537fab4aeff416c

The previous report of having trouble with running the binary turned out to be caused by incorrectly specifying cgminer options.

[canford]

Thank you kano for the S1 binary, running successfully on two S1s and using your luci pool password hack to pass temp params to cgminer.  Thank you artpego for the last share time fix, also working on both ants.  Thank you ckolivas for cgminer 4.2.0 working on win7 with 12 BE's and the Avalon 4.2.0 image running on a Mini.  The Mini is running at 59 gh/s clocked at 400, turns out miner 15 has failed, but 1-14 and 16 are working.  It might be a loose connection, but I'm not sure it's worth attempting to disassemble the Mini to check.   

[artpego]

Glad to help, was an easy tweak to grep out "lastsharetime" recursively down the lua/luci dir chain and work back from there. Like I said, I am new to all this (mining) and have 3 Antminer S1's running kano's great stuff (thanks mucho kano, i will be sending some coin your way), running real stable at low temps 200GH/s with nice low temps, but I am also a hacker at heart and these little rigs are proving fun for the interface side and cgminer side. This little test seems to be earning it's keep so-far (initial hardware costs and recurring electricity cost vs the payout I'm seeing)... but we'll see at the next difficulty increase... which is all too quickly approaching ... and the erratic bit-coin pricing. Anyway, long term investment, we'll see how it works out.

My "day job" these past 12 years has been writing enterprise software (shop floor controls, low level machine control for component placement, health system logs to pull/burn onto ROMS/B2B,ERP, web reporting & eCommerce) all day long for a PCBA fabrication facility, systems integration factory, engineering and program management site, our eCommerce group, and an a couple of RMA facilities (1 man "manager of 1", waaaaayyyyyy many hats)..... with a wife who designs the boards in the engineering wing of the same company (won't mention any names, but we make just about some percentage everyone's retail stuff they sell like servers, PC's, phones, etc... , they just have us design it and then re-brand or we build on their specs as a CM).... if anyone ever wants to design and build a "better ant", maybe we can figure something out and come up with a team... pretty sure I can do NPI's and get PCB's (since we own a PCB fab as well) without any hassles Weekdays suck for comms, but nights and weekends are usually free to mess around in my hardware lab all day.

[-ck]

New version: 4.2.1, 24th March 2014

Bugfixes.


Human readable changelog:

- Bitcoind did not like lots of persistent connections at once meaning it would fall over if more than one cgminer instance was trying to mine at the same time from the one bitcoind instance. Cgminer now opens and closes the connection every time it talks to bitcoind allowing any number of cgminer instances to mine solo from the one instance. Confirmed working with 200TH of miners aimed at the one bitcoind...
- Big endian hosts (like the antminer S1, avalon) did not work with solo mining.
- Solo mining setups will not mine unless a btc address is specified now, and the address is displayed on startup if it exists.
- Solo mining disconnections to bitcoind are handled better, not spawning more polling threads every failure.
- Low level optimisations for solo mining
- AntminerS1 fixes to decrease CPU usage and actually honour overheat conditions.
- Network diff when submitting a block is shown correctly on screen when >2billion.
- Build fixes for avalon2
- miner.php improvements


Full changelog:

- Fix various ava2 build issues generically
- Minimise the amount of heap memory allocations/frees when submitting gbt
shares.
- Make varint in gbt submission a stack object.
- Fix big endian problems with gbt submissions.
- Fix 32bit overflow on relative diff shown.
- ants1 - stop results read hard looping
- ants1 - slow down mining if overheat occurs
- miner.php allow gen before (bgen) and after (gen) grouping
- Change default solo mining to failing when no btc address is specified.
- Use upgrade cglock variants in get_gbt_curl
- Provide a cg_uilock to unlock the intermediate variant of cglocks.
- Use the one curl instance for all gbt solo operations, protecting its use with
a bool set under gbt lock.
- Only start block detection with gbt solo if setup succeeded
- One less block detection message
- Toss out the curl handle after each solo poll
- Don't reuse any curl handles for solo mining and break out of the lp thread if
the pool is removed.
- Make sure to only start the lognpoll thread once on gbt solo.
- Don't keep RPC connections open for solo mining since bitcoind doesn't like
having many persistent connections.
- GBT solo pools should be considered localgen pools.
- miner.php - speed up formatting and allow calc on gen fields
- Always show the address we're solo mining to to avoid confusion for when no
address is set.

[loshia]

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code



static inline void _cg_wlock(cglock_t *lock, const char *file, const char *func, const int line)
{
   _mutex_lock(&lock->mutex, file, func, line);
   _wr_lock(&lock->rwlock, file, func, line);
}

static inline void _cg_rlock(cglock_t *lock, const char *file, const char *func, const int line)
{
   _mutex_lock(&lock->mutex, file, func, line);
   _rd_lock(&lock->rwlock, file, func, line);
   _mutex_unlock_noyield(&lock->mutex, file, func, line);
}

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Please take a look at the flowing peace of code

stale_work(struct work *work, bool share)   
.....
   cg_rlock(&pool->data_lock);
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_runlock(&pool->data_lock)

It may turn that while we are performing strcmp
static bool parse_notify(struct pool *pool, json_t *val)
.......

   cg_wlock(&pool->data_lock);
   free(pool->swork.job_id);
   pool->swork.job_id = job_id;

frees pool->swork.job_id

Resulting in flowing line in BLOCKED gets (9) id=248038834 by cgminer.c stale_work()....
I have changed stale_work
      cg_rlock(&pool->data_lock);
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_runlock(&pool->data_lock);
to
    cg_wlock(&pool->data_lock);
   
      if (strcmp(work->job_id, pool->swork.job_id))
         same_job = false;
      cg_wunlock(&pool->data_lock);
I have moved

gen_stratum_work.....

work->job_id = strdup(pool->swork.job_id); under
cg_wlock(&pool->data_lock);
Second:

   cg_wlock(&control_lock);
   local_work++;
   work->id = total_work++;
   cg_wunlock(&control_lock);

I do think that everywhere total_work++; and local_work++ shod be changed under &control_lock. This is not happening currently. I am digging to find all places to do it

[-ck]

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code


[snip]

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Actually the unfortunate thing is that cglocks are unique upgradeable read write locks, and it is normal for a write lock to be able to grab the mutex component of the cglock while a read variant is holding the read lock - but it will not be able to grab the write lock. The same rules don't quite apply as per regular locks.

Yes they are an implementation of my own, originally developed for the linux kernel scheduler code I maintain. See:
http://ck-hack.blogspot.com/2012/06/upgradeable-rwlocks-and-bfs.html

[loshia]

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code


[snip]

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Actually the unfortunate thing is that cglocks are unique upgradeable read write locks, and it is normal for a write lock to be able to grab the mutex component of the cglock while a read variant is holding the read lock - but it will not be able to grab the write lock. The same rules don't quite apply as per regular locks.

Yes they are an implementation of my own, originally developed for the linux kernel scheduler code I maintain. See:
http://ck-hack.blogspot.com/2012/06/upgradeable-rwlocks-and-bfs.html

So what will happen when  if (strcmp(work->job_id, pool->swork.job_id)) and free(pool->swork.job_id); are executed simultaneously?

[-ck]

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code


[snip]

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Actually the unfortunate thing is that cglocks are unique upgradeable read write locks, and it is normal for a write lock to be able to grab the mutex component of the cglock while a read variant is holding the read lock - but it will not be able to grab the write lock. The same rules don't quite apply as per regular locks.

Yes they are an implementation of my own, originally developed for the linux kernel scheduler code I maintain. See:
http://ck-hack.blogspot.com/2012/06/upgradeable-rwlocks-and-bfs.html

So what will happen when  if (strcmp(work->job_id, pool->swork.job_id)) and free(pool->swork.job_id); are executed simultaneously?

That can't happen.

[loshia]

Hello Con,

I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!

Here are my findings:

First Issue
From the following code


[snip]

It seems that when a thread acquires a read lock (_cg_rlock) another thread can acquire a write lock (_cg_wlock)

Actually the unfortunate thing is that cglocks are unique upgradeable read write locks, and it is normal for a write lock to be able to grab the mutex component of the cglock while a read variant is holding the read lock - but it will not be able to grab the write lock. The same rules don't quite apply as per regular locks.

Yes they are an implementation of my own, originally developed for the linux kernel scheduler code I maintain. See:
http://ck-hack.blogspot.com/2012/06/upgradeable-rwlocks-and-bfs.html

So what will happen when  if (strcmp(work->job_id, pool->swork.job_id)) and free(pool->swork.job_id); are executed simultaneously?

That can't happen.

Oki You are the boss..
Thank you for your comments.

Best

[-ck]

That can't happen.

Oki You are the boss..
Thank you for your comments.

Heh no problem Appreciate extra eyes on the code, always.

BTW if it wasn't obvious: The cg write lock variant would have grabbed the mutex successfully but be unable to grab the write lock while the other thread holds the read lock.

[kano]

...
I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!
...

You're welcome

Yes Con designed all of the cglock code and wrote almost all of it.
I've had a few deadlocks I've coded in the past, so I wrote the lock tracking to be able to find them easily when I cause them.

[loshia]

...
I have played with #define LOCK_TRACKING 1 - GREAT WORK THANK YOU!
...

You're welcome

Yes Con designed all of the cglock code and wrote almost all of it.
I've had a few deadlocks I've coded in the past, so I wrote the lock tracking to be able to find them easily when I cause them.

Nice hack for lockstats to make them usable or at least for me api.c

#if LOCK_TRACKING

FILE * pFile;

#define LOCK_FMT_FFL " - called from %s %s():%d"

#define LOCKMSG(fmt, ...)   fprintf(pFile, "APILOCK: " fmt "\n", ##__VA_ARGS__)
#define LOCKMSGMORE(fmt, ...)   fprintf(pFile, "          " fmt "\n", ##__VA_ARGS__)
#define LOCKMSGFFL(fmt, ...) fprintf(pFile, "APILOCK: " fmt LOCK_FMT_FFL "\n", ##__VA_ARGS__, file, func, linenum)
#define LOCKMSGFLUSH() fflush(pFile)

then

void show_locks()
{
   pFile = fopen ("/tmp/cglocks","w");
..........


   fclose (pFile);

}

So no need to stare over console and catch the lines moving

PS:

Guy's
What about
   cg_wlock(&control_lock);
   .....local_work++;
   .....total_work++;
   cg_wunlock(&control_lock);
I think there is a need to lock them everywhere or ?

Thanks

[storm2k5]

Hashfast per die clock rate setting:

I have noticed that in driver-hashfast.c, there seems to be adaptive clock rate setting per die:

Code:

	applog(LOG_INFO, "%s %d: Die temp below range %.1f, increasing die %d clock to %d",
hashfast->drv->name, hashfast->device_id, info->die_data[die].temp, die, hdd->hash_clock);
hfa_send_frame(hashfast, HF_USB_CMD(OP_WORK_RESTART), hdata, (uint8_t *)&diebit, 4);

Is there a way to set per die clock from command line or conf file? Reason I'm asking is one of my dies seems to work fine up to 150MHz then it completely stops working and I would like to clock dies 0-2 with normal clock and underclock die 3.

Edit: By the looks of your comments and the code it seems that per die clock setting will be possible for FW 0.5 and up. Do you already got FW 0.5 from hashfast by any chance?

[-ck]

PS:

Guy's
What about
   cg_wlock(&control_lock);
   .....local_work++;
   .....total_work++;
   cg_wunlock(&control_lock);
I think there is a need to lock them everywhere or ?

Thanks

Mostly harmless, but probably wouldn't hurt to protect them with the write lock for when mining at multiple different types of pools at once (eg GBT + stratum), yes, thanks.

[-ck]

Hashfast per die clock rate setting:

I have noticed that in driver-hashfast.c, there seems to be adaptive clock rate setting per die:

Code:

	applog(LOG_INFO, "%s %d: Die temp below range %.1f, increasing die %d clock to %d",
hashfast->drv->name, hashfast->device_id, info->die_data[die].temp, die, hdd->hash_clock);
hfa_send_frame(hashfast, HF_USB_CMD(OP_WORK_RESTART), hdata, (uint8_t *)&diebit, 4);

Is there a way to set per die clock from command line or conf file? Reason I'm asking is one of my dies seems to work fine up to 150MHz then it completely stops working and I would like to clock dies 0-2 with normal clock and underclock die 3.

Edit: By the looks of your comments and the code it seems that per die clock setting will be possible for FW 0.5 and up. Do you already got FW 0.5 from hashfast by any chance?

Older firmware 0.3+ can do per die clockspeeds as well but when there is a large difference in the clocks it causes huge dips and peaks in the metering out of work which can present temperature and other issues. It is not possible to set clock speed per die on the command line yet but it is possible to add that feature.