faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
Hi,
I am trying to determine if I could use pruning blockchain option on my future bitcoin node or not.
I will create a multiple wallets by RPC calls and I need to be able to check the history of transactions for each wallets, even if a wallet has been created 1 years ago for which the transactions blocks are no more present on the node due to pruning.
Do you know if it is possible to continue to check transactions of a generated wallet for transactions blocks who are no more kept ?
Thanks
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17357
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 22, 2022, 01:27:50 PM |
|
You'll need to sync the wallet before your pruned node is too far behind. You can set it up to keep many blocks, say 100 GB, so you'll still be able to use it after a year. Or you'll need to load each wallet more often.
|
|
|
|
Cookdata
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 22, 2022, 01:36:14 PM |
|
If you prune a node, you just deleted old copies of your verified blockchain, they will be deleted from your computer, calling a blocks that have been deleted from your computer will be unavailable except you connect your wallet with another node as an SPV. If space is not a problem on your machine, there is no point to delete everything your node downloaded and verified. If old transactions are that precious to you, keep those file on your machine. Another suggestion is that, if you complete initial blockchain download and you have all your wallet history verify, you can prune the node to first day your wallets had it first transaction and delete the rest of the file.
|
|
|
|
DaveF
Legendary
Offline
Activity: 3598
Merit: 6573
Crypto Swap Exchange
|
If you are ever going to need any old transaction data from before the data that you have not pruned it's going to be an issue.
I have been getting grumpy about this but will say it again, if you want to run a node for business and cost the storage difference of a pruned node vs a full node makes a difference they you should not be doing it that way. There are many ways to get the data using API calls to services. That one time you need something that is not on your node for some reason the entire process comes crashing down. Use the right tools for the job.
-Dave
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 22, 2022, 03:35:32 PM |
|
Another suggestion is that, if you complete initial blockchain download and you have all your wallet history verify, you can prune the node to first day your wallets had it first transaction and delete the rest of the file.
No good way to really do this because prune works by the size of the blockchain and not an absolute block height. It is not what it is really designed to do anyways. I'm currently running a pruned wallet on one of my PCs and I've never opened the other wallets for a few months and they seem to be synchronized just fine. I'm inclined to think that they are actually initialized at the start of the synchronize and they synchronize in tandem so it isn't really an issue if you don't open any of the wallets for a long time. I could be wrong, but I know that at least the wallets are checked at start up.
|
|
|
|
Cookdata
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 22, 2022, 05:33:48 PM |
|
No good way to really do this because prune works by the size of the blockchain and not an absolute block height. It is not what it is really designed to do anyways.
I didn't put some things into consideration. At first, I thought that if the OP could view the transaction history of his addresses before pruning, he might be able to trace the first transaction of his address to the specific block where those files are saved on the local drive, and then calculate the total amount of space from that block to the current block and prune the rest of the file. For example: If OP had its first transaction in block K and was able to determine the overall size of blocks from K to the present one, he may simply prune=N where N is the total amount he wishes to keep (MB) on his computer. I'm not sure the strategy would work. The best option is to restart a full blockchain download, or never prune your node in the first place.
|
|
|
|
faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 22, 2022, 07:54:09 PM |
|
Thanks all for your relevant replies !
@DaveF : You have right, this node will allow me to create multiple wallets and send transactions from each of them. Is it possible to create a wallet now and have to send a transaction in 5 years. But I have to regulary check if on each wallet a new transaction is coming on, thats why I need a long history of data. But as you said, maybe I can use my node for wallet creation/send transaction and an external API to check if new transaction exist for wallets. I would like to save data space because I will have to sync 10+ differents blockchains...
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 08:03:28 AM Last edit: May 23, 2022, 11:22:57 AM by ranochigo |
|
@DaveF : You have right, this node will allow me to create multiple wallets and send transactions from each of them. Is it possible to create a wallet now and have to send a transaction in 5 years. But I have to regulary check if on each wallet a new transaction is coming on, thats why I need a long history of data. But as you said, maybe I can use my node for wallet creation/send transaction and an external API to check if new transaction exist for wallets. I would like to save data space because I will have to sync 10+ differents blockchains...
If storage space is really a concern for you, then wouldn't an SPV wallet be a more suitable alternative for you? Electrum would suffice, without having external API or stuff like that. The wallet will be synchronized from the point which you've closed it. It shouldn't really be a problem because I think that all of the wallets are being initialized and synchronized at the same time regardless of if you're opening it or not. CMMIW on that, but it obviously wouldn't apply if you're importing a wallet in the future.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17357
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 10:17:13 AM |
|
this node will allow me to create multiple wallets and send transactions from each of them. Is it possible to create a wallet now and have to send a transaction in 5 years. Is there a reason to use multiple wallets, instead of many different addresses in one wallet? That might be easier. Also, if you're keeping funds on an online wallet for 5 years, depending on the value you may want to look at offline cold storage instead. But I have to regulary check if on each wallet a new transaction is coming on, thats why I need a long history of data. Depending on what service you're offering (I'm thinking it's either an exchange, casino or webshop), you'll need to check each wallet very offen, so you won't need a long pruned history. I would like to save data space because I will have to sync 10+ differents blockchains... Can you share what you're planning?
|
|
|
|
faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 12:47:19 PM |
|
@DaveF : You have right, this node will allow me to create multiple wallets and send transactions from each of them. Is it possible to create a wallet now and have to send a transaction in 5 years. But I have to regulary check if on each wallet a new transaction is coming on, thats why I need a long history of data. But as you said, maybe I can use my node for wallet creation/send transaction and an external API to check if new transaction exist for wallets. I would like to save data space because I will have to sync 10+ differents blockchains...
If storage space is really a concern for you, then wouldn't an SPV wallet be a more suitable alternative for you? Electrum would suffice, without having external API or stuff like that. The wallet will be synchronized from the point which you've closed it. It shouldn't really be a problem because I think that all of the wallets are being initialized and synchronized at the same time regardless of if you're opening it or not. CMMIW on that, but it obviously wouldn't apply if you're importing a wallet in the future. Hi @ranochigo, Thanks for help. I don't know if it can be use on my case because I have to generate new wallets from my server and not on the user device. Do you think it can be possible to set my server as a client to generate multiple wallets ?
|
|
|
|
NeuroticFish
Legendary
Offline
Activity: 3794
Merit: 6495
Looking for campaign manager? Contact icopress!
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 12:54:17 PM |
|
I don't know if it can be use on my case because I have to generate new wallets from my server and not on the user device. Do you think it can be possible to set my server as a client to generate multiple wallets ?
If you want to create something like e-commerce, creating different deposit address for each of your customers can be done in any wallet. And if the number of customers (and transactions) is not too big, even a SPV wallet can handle. But if it's something more serious, the wallet software will have to be Bitcoin Core, not a SPV wallet. But: for this you need only one wallet, you don't need to create different wallet per customer, it's enough to create new address per customer, all in the same wallet. However, you better confirm your actual use case....
|
|
|
|
faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 12:54:33 PM |
|
this node will allow me to create multiple wallets and send transactions from each of them. Is it possible to create a wallet now and have to send a transaction in 5 years. Is there a reason to use multiple wallets, instead of many different addresses in one wallet? That might be easier. Also, if you're keeping funds on an online wallet for 5 years, depending on the value you may want to look at offline cold storage instead. But I have to regulary check if on each wallet a new transaction is coming on, thats why I need a long history of data. Depending on what service you're offering (I'm thinking it's either an exchange, casino or webshop), you'll need to check each wallet very offen, so you won't need a long pruned history. I would like to save data space because I will have to sync 10+ differents blockchains... Can you share what you're planning? Hi @LoyceV, Thanks for reply. I already thought about the possibility of generating only address on the same wallets but the problem will be on the sending transaction. I need to send BTC from specific address but if I use the same wallet with many address, the sending address will always be the same. You have right on this point, I will regulary check and store transactions on my database for each wallets, so I don't really need to have the whole blockchain history. That's a good point ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) I am working on a crypto exchange platform, so I will have to generate multiple wallets on differents blockchains for each of users. So we need to have something sure and secure on the long terms.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17357
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 01:21:02 PM |
|
I already thought about the possibility of generating only address on the same wallets but the problem will be on the sending transaction. I need to send BTC from specific address but if I use the same wallet with many address, the sending address will always be the same. If you enable Coin Control in Bitcoin Core, you can manually select each input. I am working on a crypto exchange platform, so I will have to generate multiple wallets on differents blockchains for each of users. For an exchange, there's no need to use the user's deposited inputs for their withdrawals. Usually people withdraw a different coin than they deposited.
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 02:24:22 PM |
|
I don't know if it can be use on my case because I have to generate new wallets from my server and not on the user device. Do you think it can be possible to set my server as a client to generate multiple wallets ?
SPV wallets would probably not be very good for any service in general, simply because they are very sensitive on both the security and the privacy aspect. I would probably recommend you to get a stronger server instead, because if you can't hold the entire blockchain with your storage, then your server has a good chance of not being strong enough. Using multiple wallets won't be viable for use-case with loads of individual wallets because that would result in a huge delay when synchronizing and loading up the various wallets while offering little to no benefits in terms of segregation. Having a database with the different addresses tagged to each user would be a far better option both in terms of organization and efficiency.
|
|
|
|
faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 23, 2022, 07:30:40 PM |
|
I already thought about the possibility of generating only address on the same wallets but the problem will be on the sending transaction. I need to send BTC from specific address but if I use the same wallet with many address, the sending address will always be the same. If you enable Coin Control in Bitcoin Core, you can manually select each input. Thanks for the tip with Coin Control, you are a BTC core genious ! ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) The only bad thing with this option is to retrieve a balance of specific address. I checked and the only possiblity to get a address balance is to parse all the transactions. So yes, anyway I will parse the transactions and store it in database, so I have the balance but if a security breach exist and someone are able to withdrawal more than he really have in his wallet, he will be able because the main wallet will have the money of everybody. It is very dangerous whereas if I create a wallet for each user, this case is not possible. What do you think ? Maybe you know some security tips to avoid it ? I am working on a crypto exchange platform, so I will have to generate multiple wallets on differents blockchains for each of users. For an exchange, there's no need to use the user's deposited inputs for their withdrawals. Usually people withdraw a different coin than they deposited. Users can use their wallets as a simple wallet, so they can receive and send crypto on it, it will be not only used to exchanges cryptos.
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 24, 2022, 03:29:46 AM |
|
The only bad thing with this option is to retrieve a balance of specific address. I checked and the only possiblity to get a address balance is to parse all the transactions. So yes, anyway I will parse the transactions and store it in database, so I have the balance but if a security breach exist and someone are able to withdrawal more than he really have in his wallet, he will be able because the main wallet will have the money of everybody. It is very dangerous whereas if I create a wallet for each user, this case is not possible. What do you think ? Maybe you know some security tips to avoid it ?
Using a wallet system for this is not a solution, because as your user base grows, you'll run out of resources to handle that many wallets in Bitcoin Core. The problem that you highlighted isn't really a concern if you know how to properly code and maintain your database. Using a wallet system to segregate your funds isn't a solution because you are operating an exchange, not a wallet. There are tons of transfers within the users themselves either through buying or selling other coins. If you were to initiate a transfer everytime the user makes a transaction, then things can get very messy quite quickly. I would encourage you to explore the possibility of building and managing your own backend which would greatly simplify your workflow. Do hire an experienced developer if you want to properly manage your site, the community here can only guide you so much.
|
|
|
|
NotATether
Legendary
Offline
Activity: 1722
Merit: 7248
In memory of o_e_l_e_o
|
The only bad thing with this option is to retrieve a balance of specific address. I checked and the only possiblity to get a address balance is to parse all the transactions. So yes, anyway I will parse the transactions and store it in database, so I have the balance but if a security breach exist and someone are able to withdrawal more than he really have in his wallet, he will be able because the main wallet will have the money of everybody. It is very dangerous whereas if I create a wallet for each user, this case is not possible. What do you think ? Maybe you know some security tips to avoid it ?
Even creating a single wallet with many addresses inside, one for each user, will pose a scalability problem for you when you get thousands of users, as the load time for opening the wallet will increase severely. You should look into creating the transactions from the private keys manually - this does mean that you'll need to generate a private key for each user (and store it inside a database), and there are some libraries that can help facilitate this (depending on the programming language you choose), but it's not a good idea to outsource this functionallity to Bitcoin Core. It wasn't designed to be an exchange backend.
|
|
|
|
LoyceV
Legendary
Offline
Activity: 3430
Merit: 17357
Thick-Skinned Gang Leader and Golden Feather 2021
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 24, 2022, 08:04:32 AM |
|
So yes, anyway I will parse the transactions and store it in database, so I have the balance but if a security breach exist and someone are able to withdrawal more than he really have in his wallet, he will be able because the main wallet will have the money of everybody. It is very dangerous whereas if I create a wallet for each user, this case is not possible. What do you think ? Maybe you know some security tips to avoid it ? I think you shouldn't build an exchange from scratch. Ten years ago "homebrew" solutions were the reason so many exchanges lost a lot of money. Nowadays, I don't think you should do this if your budget doesn't even allow to store the full blockchain on a node.
|
|
|
|
faak (OP)
Newbie
Offline
Activity: 7
Merit: 5
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 24, 2022, 09:28:02 AM |
|
Thanks all for you reply, you are very helpful ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) I have the budget to store all the blockchain but if I can avoid it, I will. Many of you are saying there is not the good option to use Bitcoin core for my use-case, thanks for the information. @NotATether you are suggesting to generate a private key for every user, that something I can do. I found library to do that, I am able to generate private key of each user and get associated btc address without having bitcoin core. So I have to find the solution to create and sign a transaction from a private key. So If I resume, the best solution to manage the creation and the managing of thousands of wallets, is not to use bitcoin core but to generate private key for each user and create raw bitcoin transaction and sign it with this private key. Everyone agree with it ?
|
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
May 24, 2022, 09:39:57 AM Last edit: May 24, 2022, 12:14:29 PM by ranochigo |
|
Many of you are saying there is not the good option to use Bitcoin core for my use-case, thanks for the information. @NotATether you are suggesting to generate a private key for every user, that something I can do. I found library to do that, I am able to generate private key of each user and get associated btc address without having bitcoin core. So I have to find the solution to create and sign a transaction from a private key.
So If I resume, the best solution to manage the creation and the managing of thousands of wallets, is not to use bitcoin core but to generate private key for each user and create raw bitcoin transaction and sign it with this private key.
Everyone agree with it ?
The problem is that you still need a source to get the transactions, as well as the data for your transactions. You cannot do this without running an instance of Bitcoin Core or another full node/client. The solution that you've proposed can work, but you have to be careful to audit and understand the code when you're using it and not just ripping the code off somewhere else. Bitcoin Core is still a better solution, because you have the entire RPC API for you to utilize and all of the functions being built into it. The threshold for Bitcoin Core to slow down is quite high, and you shouldn't really face a problem with it unless you're having an extremely big operation, for which you should invest in a better server. Again, you don't have to manage thousands of wallets. You can easily just create a single wallet with all of the addresses and tag each unique address to each user. Best practice is to generate a new address for the user after a single transaction.
|
|
|
|
|