Thanks, pooya87! I now understand the correct validation method of Private Addresses. That info is really useful but also made me aware I really only need a pattern check, as kano explained.

NotATether was nice, too. I never found that regex in Binance API docs but I did not read those docs thoroughly when I was messing with them. All the discussion about the pattern regex was interesting (I know you guys are making history here) and I was able to add the mainnet uppercase BC1 regex to my collection as that case was not covered yet.  Cheers!

I am writing a script for dealing with bitcoin hashing and it would be useful to have a function to validate Testnet WIF , both uncompressed (starting with 9) and compressed (starting with c), such as:


I have got regexes for most of other types of addresses, including regexes for mainet WIF types but they don't seem to match if I merely add [9c] to the prefix list.. I read somewhere that prefix and ending of testnet WIFs are different from those of mainet. Below is the regex for mainet WIFs I have got and which does not work.

UPDATE: THIS ACTUALLY WORKS THO

If you guys know a regex that could work with testnet WIFs, please let me know. Or point to a page of testnet WIFs I can check and try something myself (could not find anything that helped me on Google yet)..

Cheers

UPDATE:
OK, I reckon that regex is working when adding the right prefixes in the character list.
[...]

Checking the License or lack of that is important.
All lawyers can dig that..

It is fearful misunderstanding what `open source' means.
If you think `open source' means that you can check the source code,
there is a long way ahead..

You can try blockchain.com:


I have got a shell script called binfo.sh which does some API calls and format the data, don't know if you want to build your own db tho.
https://github.com/mountaineerbr/markets/blob/master/binfo.sh

The most generic and correct term is `Free Software'. Bitcoin is free software, not open source.
Open source software has got no philosophy involved and pointing users to it
is a big problem because they will only get a fraction of what
Free Software and the Free Software Movement are all about..

Cheers!

Thanks for that link reference! That is really good.
I did restart the node after changing configs.
Will reboot the server and start ONLY bitcoind to be sure that no other service is interfering with its RAM usage, just to be sure..
Will also try to decrease mempool db a little from defaults.


That is a Manjaro system updated in the beginning of the month with latest version.
The config file is located in a usd hd together with blockchain data.
The command I use to start the daemon as a normal user is:

Now that I know that is not the expected RAM usage, I will try to investigate it in the following days.

Thanks for the answers. In the meantime, I will be checking this thread and will report later.
Thanks guys!


PS: I rebooted the server and mostly the excessive ram usage was gone. bitcoind does seem to build cached memory, but that is not a problem.
I started a shell daemon which monitors mempool median fees through bitcoin-cli and bitcoind used memory increased. The cause seems to be my shell daemon probably, but RAM usage does not go down when it is killed.. Will check into that... Cheers!

I am setting a home server for multiple purposes.
However, bitcoind is using a lot of RAM.
I wish to limit resource usage as I would prefer more RAM
was available for other tasks..

I tried setting some configs as per the following post:
https://bitcoin.stackexchange.com/questions/50580/how-to-run-bitcoind-in-a-low-memory-environment

Is there anything else I can do or is that just bitcoind normal behaviour to consume as much RAM as the kernel will let it have?

Here are my configs:


And here is htop output:
https://imgur.com/a/wXjtqPL

OBs: the node is fully sync'ed.

Just to let you guys know i updated my bitcoin-all-addresses list on 2021 Jan 19.
That is available in my github repo https://github.com/mountaineerbr/bitcoin-all-addresses
All addresses are uniquely printed in the order they first appeared in Blockchair output dumps.

I was able to reproduce my methodology after 6 moths from the first lists.
The methodology is described in the read me of the git repo
and some code i used here: https://github.com/mountaineerbr/bitcoin-all-addresses/blob/master/blockchair.btcoutputs.process.sh

If you export LANG=C and LC_ALL=C, that will speed up sorting and as we are dealing
with base58 and segwit base addresses, that should be OK.

Thanks for your effort.

Maybe a quote from rudimentary UNIX philosophers:

Those who don't understand Unix are condemned to reinvent it

I reckon that refers to communal computing and its joy.

If you are using Linux shell and run my script (bitcoin.tx.sh), which is a bitcoin-cli wrapper:


The script uses grondilu's bitcoin-bash-tools to make the address from the pub key..
Script from my githubh repo: https://github.com/mountaineerbr/scripts/blob/master/bitcoin.tx.sh


The next script wraps grondilu's bitcoin bash functions. Here I
get the ASM (assembly) code, pack, double-sha256 and then
convert it to address with byte number 00.

https://github.com/mountaineerbr/scripts/blob/master/bitcoin.hx.sh

Hello guys!
It has been some months my working on some bash scripts.
These scripts warp arounf bitcoin-cli rpc calls to parse block and transaction data.
Just give it the trasaction hash or block hash/height

As i wrote some wrappers for blockchain.com APIs, I wanted to get
more or less the same information. I used functions from grondilu's
bitcoin-bash-tools..

Before trying the scripts, one needs have bitcoin-cli, openssl, jq and
bash (v4 or above) installed. Also, that is a very good idea to have bitcoin-cli
set with txindex=1 (otherwise you will need supply the block hash for the
transactions and some addresses will not be coded to base58 from their hex).

The scripts are at https://github.com/mountaineerbr/scripts

There are two scripts, one for getting block information and transaction
hashes in that block, and another to get transaction information by
transaction hash. For example:

Get info of a transaction by its hash id:

The output:


One can parse all transactions from a block (ex: height 400000) using my two scripts, such as:

I have been checking some blockchair parses that try to parse binary blockchain data,
which don't output as much useful information (at least for me, a regular user)
of a transaction as my script do.

The transaction script (bitcoin.tx.sh) is slow because depending on the transaction vins and vouts
it needs query previous transactions, too. This script can take from a fraction of a second
for a simple transaction up to a few seconds to parse a transaction with various in and out vectors.

Transaction from block 400000 and 668385 took approximately 12 minutes and 18 minutes to
parse, respectively with my script (intel i7).

The script (bitcoin.blk.sh) for parsing block data can also decode transaction hex of coinbase.

Decode coinbase from block height 668385:

And the bitcoin.hx.sh will help you guys encoding and decoding public and private keys
(legacy addresses only), as well as decoding hex strings that you may get from the blockchain.

The scripts contain a help page, print with -h :
https://github.com/mountaineerbr/scripts

Cheers,

Seems that api is stuck for now..
You can try these but they do not go as far back (until 2013-04-28)
and will stop working at any time:

i wrote my methodology for my addresses list with more details in my github repo ..
Here is how i did the sorting..


Note that using LC_ALL=C will greatly speed up sorting!

That is a very nice API..
Nastyfans.org is a pool? Hope they live long and keep this api up..
I love to check a snapshot-price from the terminal, coin market cap
and coingecko apis i use only go back to ~2014 max..

here is a suggestion (table formatting is optional):
Thanks!

Hey mate, nice script! I was very interested when i saw the topic here..
That is a very nice script of yours indeed. I love the colours!

I wrote some scripts for similar purposes and you may be interested.
https://github.com/mountaineerbr/markets
Specially, i wrote the cgk.sh which uses CoinGecko API, too.
Syntax is something like:


It has many options and i hope it is robust, however if the user is not
connected to the internet or the error is from CoinGecko side, i dont care
to troubleshoot for the user too much..

I have compiled these from various sources and use them to automatically set my blockchain explorer options based on user input, and also keep them at my .zshrc :

Flag -w is 'word bondary' and can also be set within the regex with '\b' at the ends.

Very good work on compiling those addresses, mate!

Hey that is very nice , bro.
You have got the means, work hard and you are very good at it.
I knew that awk one-liner you wrote, tho i tried using perl because thought that might need less ram..
The sort command can keep chronological order without using ram but it needs a large temp directory (/tmp will not work as it is limited by the system ram value).
ok, cheers!

that is almost silly
we need to be sure the parametre space is large enough

please donate to me instead( LOL), i never received a donation lol
no i am not selling data
[it will take me some days (not many, though) to upload every thing to git as of today]

and i think this is a much powerful proof we are NOT
really interested in money, except some private companies may


only one liners,  yeah oh... go learn some more
@loyce you didnt post the resulting lists, how can anybody be sure about; 
@windows/osx ;people please remove windows or mac os

really pleased with the GNU sort (that is called the genius sort)..

this ought to be about right.. will try and upload split unique address list files to git.

one feature of my lists is i tried to keep the original order in which addresses first appeared in the blockchair dumps..

you should not do it all at once, if you have got disc space.. i have produced some intermediate  files for processing..


dump files from blockchair (4210 files):

from 20090103 to 20200718

addresses total: 1483853800

unique addresses total: 692773144

https://github.com/mountaineerbr/bitcoin-all-addresses