Bitcoin Forum
June 22, 2024, 03:57:22 AM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1] 2 »
1  Alternate cryptocurrencies / Mining (Altcoins) / The most profitable altcoin to mine on CPU on: January 15, 2018, 08:18:03 PM
What is the most profitable altcoin to mine on CPU considering:
1. Current difficulty
2. Price
3. Comunity and R&D team
4. Potential for growth in the future
?
2  Economy / Exchanges / Re: Kraken online again but no withdrawals on: January 15, 2018, 10:27:52 AM
What are other good alternatives with fiat to crypto?
3  Economy / Exchanges / Re: Kraken online again but no withdrawals on: January 14, 2018, 10:36:17 PM
Still, Kraken was partially unaccessible today 520 error.
4  Bitcoin / Project Development / Re: Who is writing robots for day trading here on: January 14, 2018, 10:34:32 AM
Work on tradebots for a few years now and it's still a lot of fun! 

If you have some coding skills, grab one of the libs and code away!

 Cool

Sorry, I am new here, where is the lab?
5  Alternate cryptocurrencies / Mining (Altcoins) / What is the most profitable altcoin to mine on CPU only? on: January 13, 2018, 03:34:44 PM
What is the most profitable altcoin to mine on CPU considering current difficulty, price and potential price growth in the future?
6  Economy / Exchanges / Re: Kraken online again but no withdrawals on: January 13, 2018, 01:11:40 PM
I can't wait withdrawals enabled... To escape this crapy 520error market ASAP.
7  Alternate cryptocurrencies / Marketplace (Altcoins) / Re: Kraken Down? on: January 13, 2018, 12:54:38 PM
The way they've made system upgrade is completely unprofessional. It's about money... huge money and
they just deploy and test the new system in production!!! Not a staging environment, not a sandbox, but production.
OK, even if they've found some critical issue, they should've rolled back immediately to the stable working version.
NO, they kept fixing the issue for 50 hours making all people panicking like hell.

Their messages were like "Thank you for your continued patience while we work through these unexpected and delicate issues."
When I read 'delicate' I think that my money are gone... This is ridiculous.. And the best one: "we will push off a bit to get some rest and be able to better monitor systems and react to problems following launch. Unfortunately, this means several more hours of delay. "
8  Alternate cryptocurrencies / Service Announcements (Altcoins) / Re: 🔥🔥🔥🔥🔥[GUNBOT] Automatic Poloniex Profit Generator🔥🔥🔥🔥🔥 on: January 12, 2018, 07:42:09 PM
Is it possible to have a trial license for back tests only to simulate trading?
There are no trial licences available.
Ok, where can I watch live trading? Or where can I run back tests of your bot as the black box like on this platform https://cryptotrader.org/backtests ?
9  Alternate cryptocurrencies / Marketplace (Altcoins) / How do markets generate private key for all coins wallets? on: January 12, 2018, 07:17:48 PM
I wonder what is the random algorithm do they use to generate PK and get wallet address out of it. Do they use dictionaries with salts of some
other advanced random seed?
10  Alternate cryptocurrencies / Service Announcements (Altcoins) / Re: 🔥🔥🔥🔥🔥[GUNBOT] Automatic Poloniex Profit Generator🔥🔥🔥🔥🔥 on: January 12, 2018, 07:10:17 PM
Is it possible to have a trial license for back tests only to simulate trading?
I would like to consider buys a full license, but need to play with it first to see it  worth money or not
11  Bitcoin / Development & Technical Discussion / Re: Block #488466 BTC on: January 12, 2018, 06:06:39 PM
I'm trying to parse blockchain using parser https://github.com/znort987/blockparser but it throws an error (segfault) on block #488466.

Did anyone have the same error and how to overcome it?

P.S Ran with "./parser allBalances" param on Ubuntu 14 64Gb RAM, 32 cores

yes, that is old peice of software., it doesnt work anymore. read this  https://github.com/znort987/blockparser/issues

and this

https://github.com/znort987/blockparser/issues/65

install bitcoin-core-0.11.2 and do rescan on whole blockahain and then try to run

./parser allBalances

and see if it works., theriotically it should from above references.



That sw is like what 4+ years old, and it will just bog down around 400k blocks, and certainly there would be a core crash, they even caveat to tell you to have 32gb of RAM on your system, as all their work is done in malloc()

The problem here is this sw is 4+ years old, and if the github is even a month since the last update, chances are 99% it aint' going to work, its the nature of BTC sw, ... because core is always changing the block-chain format, and nobody is going to maintain the shit for long, cuz you would go nutz

...

Said it before and will say again, if you want to parse bTC you must write you  own parser, its not hard, only ten lines of python to parse, what most ppl here are after, the problem, is that its clear that most here aren't programmers, and that is where the rubber meets the road,

Just like this post about a seg-fault, a programmer, would have just just used GDB and resolved the problem, but not here no way, there are no programmers.

there are lots of programmers here.,

but why reinvent the wheel ? there are already libraries/apps on internet to do the job., besides learning constantly changing bitcoin-core is not easy task.



I am also interested in this topic. Can you please list some of the most reliable and maintained tools for parsing the blockchain for different purposes?

Thanks
12  Bitcoin / Development & Technical Discussion / Re: How to generate list of all non-zero balances of Bitcoin blockchain? on: January 12, 2018, 05:51:20 PM
Hi all,

I have tried a couple of clients and libs but none of them allows smoothly generate balances all non-zero balances from the blockchain.
Some do fail on the latest block which is probably related to some format change, and some just throw 'segmentation fault' error and do not provide any additional data.
For instance, blcokparser tool only allows me to get all the transactions till block 488466. Does anyone know solution working out of the box?

Regards


Very good question, I think the first step for all bitcoin 'hackers' is having the database's

Snort is a good start, its bundled with brainflayer, but as you already known starting from genesis block 1 to N is easy, around 400k most parsers break

Lots of ppl wrote code in 2013, so my first 'advice' is work backwards, anybody can start at zero and bomb at +400,000 all the SHIT on github can do that, so ...

1.) get your full node running with TXINDEX set to 1, this gets you all the transactions decoded into script

2.) run your parser backwards from 530,000 ( where ever now ) and decrement, ... do the hard first, it only get easier

3.) addresses are the easy part, but you really don't want them, because the good stuff is the hashed public keys and r-values that can actually tell you stuff

4.) python is probably best cuz its easy to hack, and if your going to parse the entire btc blockchain from 5xxxxxx to 1, your going to be doing a lot of hacking

5.) don't bother with these databases, they all bomb out on the 200GB of required data, have many databases, I mainly just have .txt and csv, and then use bloom-filters as my datbase front-end that way all my querys have zero latency, filling the bloom with data is one time, and acquiring the data is what takes time, but once you have the data, its zero time making a decision

6.) sadly its probably going to be RPC with python all the way down hill and then work in JSON, the script is a pain in the ass as its not documented except in the C++ bitcoin core, and they break it on every version, thus if your running RPC at least you have a chance of reading all the data down to one

7. ) u say u want addresses, but once you have them you'll find them more than useless, most addresses hold no bitcoins, have never been used,

8.) U may want to look at snort and brainflayer they include the 'pristine' and all account with balance address list up to about 2015, that's a good place to start so you have something to look at now, pristine means blocks mined from day one, but never spent

9.) again addresses in themselves are rather worthless, what I find useful is to keep a running system that keeps a BLOOM-FILTER fed with all addresses that have a balance, once the balance goes to zero then that bloom-filter for that address is set to zero, thus I have an instant way to know for any address 1 to N ( base 10 ), I can tell u if that address has a balance today, this is useful, because in actuality your running on the memory pool looking at addresses and want to know how to update your bloom-filter for addresses

10.) most useful is deriving private-key/pairs & public-key, say in order of 10 million, and then watch your address bloom-filter to see if an address is used, then you know whether to run more software and go down that rabbit hole

I think I attempt to say more than is needed, just parsing is just in PYTHON

import pycoin/bitcoin ( your favorite shit coin lib on github that none work very well, and all were written 5+ years ago )

rpc=openrpc on port ( 127.0.0.1:8545 )

blkn=rpc.getblocknum('latest')

for blk blkn to 1: # yes  we start with the most recent blocks and work back to dinosaur age
  tx = rpc.gettransaction list from block
  for all txid in tx ...
    for all script in tx
       if tx['value'] > 100M Satoshi THEN # U said high value right?
            print address in script ( or write to file txt/csv )

Easy peezy, whats to say? nothing to it, ... the problem is it takes  a long time, I mean 'getting your list is easy'

Sure you can read the raw blk000n.dat files N to 1 and read the raw script and get the data, but you will find a mess of spaghetti code that makes you insane, its not 'python like' to fuss with hexadecimal, and its no fun in C/C++ given the fact that the entire bitcoin legacy code from day one is a KLUDGE, hack, mess and sucks real bad

Well I make it sound sort of easy, addresses and value come in JSON, but when you want R/S & Public Keys, and hashed keys then you must decode the asm/hex script, which means you must read the C++ source cuz its the only place that documents the script 03/02/01/N/R,...

Besides Adresses in themselves are useless, most addresses you gather have no meaning, here I will give you some numbers

I parse the block chain from N to 1  ( 500k) blocks, about 4,000 transactions per block, and about 2-3 addresses per tx, so that 20 million addresses, of those maybe 5-10% are interesting and less than 0.1% have a balance ( remember ppl are told not to use same addresss twice )

**

I have said it before and I will say it again NONE of the CODE on github works, its all shit, and its all not maintained, as the kids like BUTERIN seem to get bored 1-2 years after they write their library and get 'famous', and then move on and never look back, and some of these libs are just out&out read-only non-maintainble, I have spents MONTHS trying to get ABE ( and many other of these so called bitcoin pasers databases ) to work just to realize that its hard-coded to a specific bTc fork, which means that is worthless if you want to parse 1-N

I wish to say there was a fast way to parse, and even ABE sure it works great 1 to 410,000, and then it stops and never works again, cuz the script is so convoluted, and the code is so tightly wound on early BTC data-structures that its all hopeless, SEGWIT and all these new TX scripts seem to have broken all the shit libs

I find that most of the time i have to parse my own script, that's why I have one parser for each task, and don't bother with one fits all, cuz its too much hacking, ( but its easy as shown above the writer you own parser to gather all high-value address on btc is only 10 lines of python )

One parser to get ALL the addresses, another to get all public-key and their hashes, and another to to get all the R&S values for EcDSA hacking, ... and other databases for other good stuff like dates and time for unusual transactions and all these different databases go into different bloom-filters and then in productions all my main code just uses the bloom filters,

I might add the bloom filters should be updated every 10 minutes, but actually earlier, cuz the early bird is always two steps ahead of consensus Smiley



Hats off to you for your comprehensive answer and huge respect! It's very interesting to read as I have the same frustration
with bitcoin blocks and their protocols which have evolved over time without having proper tooling around it, but only some
enthusiasts who wrote it for fun or for tips.

I have seen at least 4 different block version and not sure if they are backward compatible. For instance with block headers only I ran into
the error with very lightweight parser (https://github.com/tenthirtyone/blocktools) in blk00976.dat
########## Block Header ##########
Version:            536870914
Previous Hash    000000000000000000520200e29fade6f8b03e9eeb57eefd8e7004088a055c7f
Merkle Root    f8e0556bd0ce982a118165a22e18de27ea97a7ac239a9535d203beee598b25b6
Time               2017-08-24 02:15:22
Difficulty            402731232
Nonce       4273024596
##### Tx Count: 2280

Next block throws an exception:
blocktools/blocktools.py", line 11, in uint4
    return struct.unpack('I', stream.read(4))[0]
struct.error: unpack requires a string argument of length 4

I don't want to spend much time on digging into this, cause this was just out of my curiosity and I don't want to sit and twist
bits/bites/words which have been changed many times over 9 years.

Your post helped me a lot to sort these things out, thank a lot again!!!
13  Other / Off-topic / Kraken downtime on: January 12, 2018, 02:27:56 PM
Anyone know more information then is publish here: https://status.kraken.com/incidents/nswthr1lyx72 ?

The downtime is >30h now...  Sad
14  Bitcoin / Project Development / Re: Who is writing robots for day trading here on: January 12, 2018, 01:43:04 PM
If you know typescript you may try to play with your own bot on https://cryptotrader.org/
There you can run back-tests on all markets out of the box. The only this you may want to pay is
already implemented solutions or deploying your own bot to production system
15  Alternate cryptocurrencies / Mining (Altcoins) / Re: Mining in cloud AWS/Azure/Google on: January 12, 2018, 11:56:07 AM
Is it profitable to mine some CPU optimised hashes like Monero CryptoNight algorithm on cloud machines?

Let's say Intel Xeon e5-2620 full time 32 cores would cost ~1k$/month on AWS.

Would it work or it doesn't make sense?

NO, Read the terms and conditions on these clouds, they doesn't allow Crypto Currency mining, Been there, my VM's has been banned on google Cloud.  Cheesy

Some providers are not that strict as Google and maybe it could be possible to run the instance for a while which would potentially generate some coins Smiley
16  Bitcoin / Development & Technical Discussion / Re: Block #503198 what is it ? on: January 12, 2018, 11:44:10 AM
Maybe it was 51% attack ? I do not know how but this is the place to be, big amount of miners accept this version of block #503198

It is almost impossible to make 51% after 9 years blockchain history and >500k blocks generated...
You basically need to re-compute all the hashes having higher CPU power than all miners in the world!
17  Bitcoin / Development & Technical Discussion / Re: How to generate list of all non-zero balances of Bitcoin blockchain? on: January 12, 2018, 06:57:28 AM
I have the statistics of all the addresses ever used in Bitcoin blockchain till the block 488466 where
my parser throws an exception and quits. I can upload this file somewhere and shre the link if you are interested in.
Most of the addresses are disposable and empty, probably used by mixers... it's just my guess.

Anyway, thanks for your help!

Btw, Bitcoin-qt client is using levelDB for quick access to unspent transactions.
18  Bitcoin / Development & Technical Discussion / Re: How to generate list of all non-zero balances of Bitcoin blockchain? on: January 12, 2018, 06:50:29 AM
Thanks for the hint! I will give a try and come back ones I have some results to share.
19  Alternate cryptocurrencies / Mining (Altcoins) / Mining in cloud AWS/Azure/Google on: January 11, 2018, 08:29:43 PM
Is it profitable to mine some CPU optimised hashes like Monero CryptoNight algorithm on cloud machines?

Let's say Intel Xeon e5-2620 full time 32 cores would cost ~1k$/month on AWS.

Would it work or it doesn't make sense?
20  Bitcoin / Development & Technical Discussion / Re: How to generate list of all non-zero balances of Bitcoin blockchain? on: January 11, 2018, 07:32:03 PM
Thanks for your reply! Yes, I agree that reading all the transaction and caching all the addresses is heavily memory consuming task, but I've seen in the blockparser implementation that they use Google's sparsehash library for this data structure and it's possible to fit into 64Gb of RAM if to properly release unused transactions. As I've said: it's possible to have the full list of non zero balances, but the problem is that at some point of time the format of the block or protocol has been changed and old parsers are not compatible anymore with them. So I wonder if there are any up-to-date solutions for parsing the whole blockchain in an efficient way?



Pages: [1] 2 »
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!