Bitcoin Forum
April 29, 2025, 03:55:53 PM *
News: Merit Poker is still available for a limited time
 
  Home Help Search Login Register More  
  Show Posts
Pages: [1] 2 »
1  Bitcoin / Development & Technical Discussion / Re: Regex for Testnet WIF [solved] on: July 21, 2021, 01:51:23 AM
a function to validate Testnet WIF ,
This is not how you should "validate" a WIF though. The string length and starting character are not important to the code (only for readability).

What you should do is [...]

Thanks, pooya87! I now understand the correct validation method of Private Addresses. That info is really useful but also made me aware I really only need a pattern check, as kano explained.

NotATether was nice, too. I never found that regex in Binance API docs but I did not read those docs thoroughly when I was messing with them. All the discussion about the pattern regex was interesting (I know you guys are making history here) and I was able to add the mainnet uppercase BC1 regex to my collection as that case was not covered yet.  Cheers!
2  Bitcoin / Development & Technical Discussion / Regex for Testnet WIF [solved] on: July 12, 2021, 07:07:41 PM
I am writing a script for dealing with bitcoin hashing and it would be useful to have a function to validate Testnet WIF , both uncompressed (starting with 9) and compressed (starting with c), such as:

Code:
92Pg46rUhgTT7romnV7iGW6W1gbGdeezqdbJCzShkCsYNzyyNcc
cNJFgo1driFnPcBdBX8BrJrpxchBWXwXCvNH5SoSkdcF6JXXwHMm
#https://en.bitcoin.it/wiki/List_of_address_prefixes

I have got regexes for most of other types of addresses, including regexes for mainet WIF types but they don't seem to match if I merely add [9c] to the prefix list.. I read somewhere that prefix and ending of testnet WIFs are different from those of mainet. Below is the regex for mainet WIFs I have got and which does not work.

Code:
[59KLc][1-9A-HJ-NP-Za-km-z]{50,51}
UPDATE: THIS ACTUALLY WORKS THO

If you guys know a regex that could work with testnet WIFs, please let me know. Or point to a page of testnet WIFs I can check and try something myself (could not find anything that helped me on Google yet)..

Cheers

UPDATE:
OK, I reckon that regex is working when adding the right prefixes in the character list.
[...]
3  Other / Beginners & Help / Re: Do you check the code of open source software? on: April 18, 2021, 02:10:53 PM
Checking the License or lack of that is important.
All lawyers can dig that..

It is fearful misunderstanding what `open source' means.
If you think `open source' means that you can check the source code,
there is a long way ahead..
4  Bitcoin / Development & Technical Discussion / Re: List transactions for given address with API call on: April 18, 2021, 12:04:45 PM
You can try blockchain.com:

Code:
% curl -L --compressed "https://blockchain.info/rawaddr/[ADDR_HERE]" | jq

I have got a shell script called binfo.sh which does some API calls and format the data, don't know if you want to build your own db tho.
https://github.com/mountaineerbr/markets/blob/master/binfo.sh
5  Other / Beginners & Help / Re: The bad side of close source wallets on: April 18, 2021, 10:57:20 AM
The most generic and correct term is `Free Software'. Bitcoin is free software, not open source.
Open source software has got no philosophy involved and pointing users to it
is a big problem because they will only get a fraction of what
Free Software and the Free Software Movement are all about..

Cheers!
6  Other / Beginners & Help / Re: Bitcoind using all my RAM on: April 18, 2021, 10:25:11 AM
There's a documentary ...
No, it's not normal behavior. Can you share command you use to run bitcoind?

Have you restarted the node after adding the configurations? Because you have to.

Thanks for that link reference! That is really good.
I did restart the node after changing configs.
Will reboot the server and start ONLY bitcoind to be sure that no other service is interfering with its RAM usage, just to be sure..
Will also try to decrease mempool db a little from defaults.

No, it's not normal behavior. Can you share command you use to run bitcoind?

That is a Manjaro system updated in the beginning of the month with latest version.
The config file is located in a usd hd together with blockchain data.
The command I use to start the daemon as a normal user is:
Code:
% bitcoind -conf=/media/usbhd/blockchain/bitcoin.conf

Now that I know that is not the expected RAM usage, I will try to investigate it in the following days.

Thanks for the answers. In the meantime, I will be checking this thread and will report later.
Thanks guys!


PS: I rebooted the server and mostly the excessive ram usage was gone. bitcoind does seem to build cached memory, but that is not a problem.
I started a shell daemon which monitors mempool median fees through bitcoin-cli and bitcoind used memory increased. The cause seems to be my shell daemon probably, but RAM usage does not go down when it is killed.. Will check into that... Cheers!
7  Other / Beginners & Help / Bitcoind using all my RAM on: April 18, 2021, 01:02:21 AM
I am setting a home server for multiple purposes.
However, bitcoind is using a lot of RAM.
I wish to limit resource usage as I would prefer more RAM
was available for other tasks..

I tried setting some configs as per the following post:
https://bitcoin.stackexchange.com/questions/50580/how-to-run-bitcoind-in-a-low-memory-environment

Is there anything else I can do or is that just bitcoind normal behaviour to consume as much RAM as the kernel will let it have?

Here are my configs:
Code:
datadir=/media/usbhd/blockchain
txindex=1
rpcuser=XXX
rpcpassword=YYY
dbcache=300
onlynet=onion
listen=0
listenonion=0
maxconnections=8
proxy=127.0.0.1:9050
discover=0

Code:
% free -h
               total        used        free      shared  buff/cache   available
Mem:           5,7Gi       4,1Gi       1,1Gi       6,0Mi       419Mi       1,3Gi
Swap:          3,9Gi       660Mi       3,3Gi

And here is htop output:
https://imgur.com/a/wXjtqPL

OBs: the node is fully sync'ed.
8  Bitcoin / Project Development / Re: List of all Bitcoin addresses ever used - currently available on temp location on: January 30, 2021, 04:04:12 AM
Just to let you guys know i updated my bitcoin-all-addresses list on 2021 Jan 19.
That is available in my github repo https://github.com/mountaineerbr/bitcoin-all-addresses
All addresses are uniquely printed in the order they first appeared in Blockchair output dumps.

I was able to reproduce my methodology after 6 moths from the first lists.
The methodology is described in the read me of the git repo
and some code i used here: https://github.com/mountaineerbr/bitcoin-all-addresses/blob/master/blockchair.btcoutputs.process.sh

If you export LANG=C and LC_ALL=C, that will speed up sorting and as we are dealing
with base58 and segwit base addresses, that should be OK.
9  Bitcoin / Development & Technical Discussion / Re: [MERGED] BIP-39 List of words in Portuguese accepted!! on: December 22, 2020, 06:04:20 AM
Thanks for your effort.

Maybe a quote from rudimentary UNIX philosophers:

Those who don't understand Unix are condemned to reinvent it

I reckon that refers to communal computing and its joy.
10  Bitcoin / Development & Technical Discussion / Re: btc rpc how to get the output address on: November 21, 2020, 04:35:37 PM
If you are using Linux shell and run my script (bitcoin.tx.sh), which is a bitcoin-cli wrapper:

Code:
user@archhost ~ % bitcoin.tx.sh a64be218809b61ac67ddc7f6c7f9fbebfe420cf75fe0318ebc727f060df48b37
txs: 1/1

--------
Transaction information
TxId____: a64be218809b61ac67ddc7f6c7f9fbebfe420cf75fe0318ebc727f060df48b37
Hash____: a64be218809b61ac67ddc7f6c7f9fbebfe420cf75fe0318ebc727f060df48b37
BlkHash_: 0000000049a63b4dda3a43450c19d085d6c28bfb4cbb2e0576815d7f31919c5d
Time____: 1233636459    2009-02-03T04:47:39Z
BlkTime_: 1233636459    2009-02-03T04:47:39Z
LockTime: 0
Version_: 1
Vsize___: 276
Size____: 276
Weight__: 1104
Confirma: 650779

Vins
  TxIndex: 131f68261e28a80c3300b048c4c51f3ca4745653ba7ad6b20cc9188322818f25
  Sequenc: 4294967295
  VoutNum: 1
  Addresses
  Number_: 1    Value__: 49.99
    type: pubkey
    1AbHNFdKJeVL8FRZyRZoiTzG9VCmzLrtvm


Vouts
  Number_: 0    Value__: 0.01
  Addresses
    type: pubkey
    1CbAopr7sNMKjsJa8dAS6P5sM788kmMooo

  Number_: 1    Value__: 49.98
  Addresses
    type: pubkey
    1AbHNFdKJeVL8FRZyRZoiTzG9VCmzLrtvm


VinSum__: 49.99000000
VoutSum_: 49.99000000
TxFee___: 0

>>>final transaction parsing at: /home/user/00.transactions.parsed.txt
>>>took 1 seconds  (0 minutes)

The script uses grondilu's bitcoin-bash-tools to make the address from the pub key..
Script from my githubh repo: https://github.com/mountaineerbr/scripts/blob/master/bitcoin.tx.sh


The next script wraps grondilu's bitcoin bash functions. Here I
get the ASM (assembly) code, pack, double-sha256 and then
convert it to address with byte number 00.

Code:
user@archhost /tmp % bitcoin.hx.sh 041d1ffff1175ce4628ed11b4074956a3f0facc95ab388e47b95daa02891f6e0b9642d4ae2b68c0787d2c95288ec42045a087c262d803b6fa14ecedb2a632f3df1
1CbAopr7sNMKjsJa8dAS6P5sM788kmMooo

user@archhost /tmp % bitcoin.hx.sh 04f9804cfb86fb17441a6562b07c4ee8f012bdb2da5be022032e4b87100350ccc7c0f4d47078b06c9d22b0ec10bdce4c590e0d01aed618987a6caa8c94d74ee6dc
1AbHNFdKJeVL8FRZyRZoiTzG9VCmzLrtvm
https://github.com/mountaineerbr/scripts/blob/master/bitcoin.hx.sh
11  Bitcoin / Project Development / [BASH] Script to get and process transactions by hash (bitcoin-cli wrapper) on: October 01, 2020, 01:46:52 AM
Hello guys!
It has been some months my working on some bash scripts.
These scripts warp arounf bitcoin-cli rpc calls to parse block and transaction data.
Just give it the trasaction hash or block hash/height

As i wrote some wrappers for blockchain.com APIs, I wanted to get
more or less the same information. I used functions from grondilu's
bitcoin-bash-tools..

Before trying the scripts, one needs have bitcoin-cli, openssl, jq and
bash (v4 or above) installed. Also, that is a very good idea to have bitcoin-cli
set with txindex=1 (otherwise you will need supply the block hash for the
transactions and some addresses will not be coded to base58 from their hex).

The scripts are at https://github.com/mountaineerbr/scripts

There are two scripts, one for getting block information and transaction
hashes in that block, and another to get transaction information by
transaction hash. For example:

Get info of a transaction by its hash id:
Code:
$ bitcoin.tx.sh a8bb9571a0667d63eaaaa36f9de87675f0d430e13c916248ded1d13093a77561

The output:

Code:
--------
Transaction information
TxId____: a8bb9571a0667d63eaaaa36f9de87675f0d430e13c916248ded1d13093a77561
Hash____: eac5391d9f7b411c7d6cbadb59a38bfa13399c3ecb440573f5f52b8a34001be1
BlkHash_: 0000000000000000000fb6a4d6f5dc7438f91a1bc3988c4f32b4bb8284eed0ec
Time____: 1594156611 2020-07-07T21:16:51Z
BlkTime_: 1594156611 2020-07-07T21:16:51Z
LockTime: 0
Version_: 1
Vsize___: 224
Size____: 414
Weight__: 894
Confirma: 12546

Vins
  TxIndex: 44f672f301772cf3e3fc15d424818aae8ed43468deb0cc56550dd9374578b816
  Sequenc: 4294967295
  VoutNum: 1
  Addresses
  Number_: 1 Value__: 4.68295
    type: witness_v0_scripthash
    bc1qwqdg6squsna38e46795at95yu9atm8azzmyvckulcc7kytlcckxswvvzej


Vouts
  Number_: 0 Value__: 0.26139693
  Addresses
    type: scripthash
    3KLwdgGf5QLHEW8qjxyv7wFpPMdVCDqu55

  Number_: 1 Value__: 0.19883951
  Addresses
    type: pubkeyhash
    1Lv9hU7h86REpDakYy9kpeTfT929b9twM5

  Number_: 2 Value__: 4.22231356
  Addresses
    type: witness_v0_scripthash
    bc1qwqdg6squsna38e46795at95yu9atm8azzmyvckulcc7kytlcckxswvvzej

One can parse all transactions from a block (ex: height 400000) using my two scripts, such as:
Code:
$ bitcoin.blk.sh -ii 400000 | bitcoin.tx.sh

I have been checking some blockchair parses that try to parse binary blockchain data,
which don't output as much useful information (at least for me, a regular user)
of a transaction as my script do.

The transaction script (bitcoin.tx.sh) is slow because depending on the transaction vins and vouts
it needs query previous transactions, too. This script can take from a fraction of a second
for a simple transaction up to a few seconds to parse a transaction with various in and out vectors.

Transaction from block 400000 and 668385 took approximately 12 minutes and 18 minutes to
parse, respectively with my script (intel i7).

The script (bitcoin.blk.sh) for parsing block data can also decode transaction hex of coinbase.

Decode coinbase from block height 668385:
Code:
$ bitcoin.blk.sh -y 668385
bb/BTC.TOP/

And the bitcoin.hx.sh will help you guys encoding and decoding public and private keys
(legacy addresses only), as well as decoding hex strings that you may get from the blockchain.

The scripts contain a help page, print with -h :
https://github.com/mountaineerbr/scripts

Cheers,
12  Bitcoin / Project Development / Re: Historic Bitcoin Exchange Rates (since July of 2010) on: August 27, 2020, 08:14:15 AM
Seems that api is stuck for now..
You can try these but they do not go as far back (until 2013-04-28)
and will stop working at any time:
Code:
$ curl https://www.coingecko.com/price_charts/export/bitcoin/usd.csv

Code:
$ curl "https://web-api.coinmarketcap.com/v1.1/cryptocurrency/quotes/historical?convert=USD&format=chart_crypto_details&id=1&interval=1d&time_end=$( date -ud00:00:00 +%s )&time_start=2013-04-28" | jq
13  Bitcoin / Development & Technical Discussion / Re: All used addresses on: August 24, 2020, 11:44:49 AM
i wrote my methodology for my addresses list with more details in my github repo ..
Here is how i did the sorting..

Code:
$ export TMPDIR='/large/tmp/dir'
$ export LC_ALL=C
$ nl concat.txt | sort -k2 -u | sort -n | cut -f2 > final.txt

Note that using LC_ALL=C will greatly speed up sorting!
14  Bitcoin / Project Development / Re: Historic Bitcoin Exchange Rates (since July of 2010) on: August 24, 2020, 01:38:27 AM
NastyFans gathers a daily exchange rate from multiple sources to use for statistical conversion data on charts.  This information is posted publicly so that anyone may utilize it and dates all the way back to July of 2010!  Enjoy!

https://nastyfans.org/exchangerate.csv

That is a very nice API..
Nastyfans.org is a pool? Hope they live long and keep this api up..
I love to check a snapshot-price from the terminal, coin market cap
and coingecko apis i use only go back to ~2014 max..

here is a suggestion (table formatting is optional):
Code:
$ curl https://nastyfans.org/exchangerate.csv | tr , \\t  #| column -et -NDATE,PRICE
Thanks!
15  Bitcoin / Project Development / Re: Linux command line tool for Bitcoin / crypto quotes (bash) on: August 24, 2020, 01:12:53 AM
Hi all,

I wrote a small linux command line tool as a bash scripting learning project; it uses the CoinGecko-API and is capable of displaying all their supported crypto-currencies, as well as cross-currencies (>50 at the moment, including some cryptos and commodities gold and silver).

Enjoy!


Hey mate, nice script! I was very interested when i saw the topic here..
That is a very nice script of yours indeed. I love the colours!

I wrote some scripts for similar purposes and you may be interested.
https://github.com/mountaineerbr/markets
Specially, i wrote the cgk.sh which uses CoinGecko API, too.
Syntax is something like:

Code:
$ cgk.sh 5+5 BTC CNY
804340.0000000000000000

It has many options and i hope it is robust, however if the user is not
connected to the internet or the error is from CoinGecko side, i dont care
to troubleshoot for the user too much..
16  Bitcoin / Project Development / Re: List of all Bitcoin addresses ever used on: August 23, 2020, 10:14:58 PM
I actually can Cheesy I found this regexp on Stackoverflow:
Code:
egrep --regexp="^[13][a-km-zA-HJ-NP-Z1-9]{25,34}$" filename
With some slight changes it stops matching parts of Eth-addresses:
Code:
egrep -w --regexp="[13][a-km-zA-HJ-NP-Z1-9]{25,34}" *


I have compiled these from various sources and use them to automatically set my blockchain explorer options based on user input, and also keep them at my .zshrc :
Code:
#cryptocurrency greps

#btc1 and btc2 combined
alias btcgrep="grep -Ee '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b' -e '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'"

#legacy addresses only
alias btcgrep1="grep -E '\b[13][a-km-zA-HJ-NP-Z1-9]{25,34}\b'"
#http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html

#bech32 v1 and v0 addresses
alias btcgrep2="grep -E '\bbc(0([ac-hj-np-z02-9]{39}|[ac-hj-np-z02-9]{59})|1[ac-hj-np-z02-9]{8,87})\b'"
#https://stackoverflow.com/questions/21683680/regex-to-match-bitcoin-addresses

#bech32 addresses only
alias btcgrep3="grep -E '\bbc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71}\b'"

#both legacy and bech32
alias btcgrep4="grep -E '\b([13][a-km-zA-HJ-NP-Z1-9]{25,34}|bc1[ac-hj-np-zAC-HJ-NP-Z02-9]{11,71})\b'"
#http://mokagio.github.io/tech-journal/2014/11/21/regex-bitcoin.html

#private keys
alias btcgrep5="grep -E '\b[5KL][1-9A-HJ-NP-Za-km-z]{50,51}\b'"
#word boundary: '\b'
#https://bitcoin.stackexchange.com/questions/56737/how-can-i-find-a-bitcoin-private-key-that-i-saved-in-a-text-file

#transaction hashes
alias btcgrep6="grep -E '\b[a-fA-F0-9]{64}\b'"
#https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex
#https://bitcoin.stackexchange.com/questions/70261/recognize-bitcoin-address-from-block-hash-and-transaction-hash

#block hashes
alias btcgrep7="grep -E '\b[0]{8}[a-fA-F0-9]{56}\b'"
#https://stackoverflow.com/questions/46255833/bitcoin-block-and-transaction-regex

#ethereum address hash
#test for 'plausibility'
alias ethgrep="grep -E '\b(0x)?[0-9a-fA-F]{40}\b'"
#https://ethereum.stackexchange.com/questions/1374/how-can-i-check-if-an-ethereum-address-is-valid

#ethereum transaction hash
alias ethgrep2="grep -E '\b(0x)?([A-Fa-f0-9]{64})\b'"  #parentheses are not necessary
#https://ethereum.stackexchange.com/questions/34285/what-is-the-regex-to-validate-an-ethereum-transaction-hash/34286

Flag -w is 'word bondary' and can also be set within the regex with '\b' at the ends.

Very good work on compiling those addresses, mate!
17  Bitcoin / Development & Technical Discussion / Re: All used addresses on: August 23, 2020, 10:06:59 PM
I made List of all Bitcoin addresses ever used.

one feature of my lists is i tried to keep the original order in which addresses first appeared in the blockchair dumps..
It works with awk:
Code:
awk '!a[$0]++'
But this requires far too much memory. I can use this on data per day, but not on all data.
So for now, I gave up trying to keep addresses in chronological order. I'll keep the original data in case I find a different solution (or enough RAM) later.

Hey that is very nice , bro.
You have got the means, work hard and you are very good at it.
I knew that awk one-liner you wrote, tho i tried using perl because thought that might need less ram..
The sort command can keep chronological order without using ram but it needs a large temp directory (/tmp will not work as it is limited by the system ram value).
ok, cheers!
18  Bitcoin / Development & Technical Discussion / Re: Collection of 18.509 found and used Brainwallets on: July 21, 2020, 12:59:48 AM
that is almost silly
we need to be sure the parametre space is large enough
19  Bitcoin / Development & Technical Discussion / Re: All used addresses on: July 21, 2020, 12:44:45 AM
please donate to me instead( LOL), i never received a donation lol
no i am not selling data
[it will take me some days (not many, though) to upload every thing to git as of today]

and i think this is a much powerful proof we are NOT
really interested in money, except some private companies may


only one liners,  yeah oh... go learn some more
@loyce you didnt post the resulting lists, how can anybody be sure about; 
@windows/osx ;people please remove windows or mac os
20  Bitcoin / Development & Technical Discussion / Re: All used addresses on: July 20, 2020, 07:03:27 PM
really pleased with the GNU sort (that is called the genius sort)..

this ought to be about right.. will try and upload split unique address list files to git.

one feature of my lists is i tried to keep the original order in which addresses first appeared in the blockchair dumps..

you should not do it all at once, if you have got disc space.. i have produced some intermediate  files for processing..


dump files from blockchair (4210 files):

from 20090103 to 20200718

addresses total: 1483853800

unique addresses total: 692773144

https://github.com/mountaineerbr/bitcoin-all-addresses
Pages: [1] 2 »
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!