Is it possible to generate a consensus algorithm using machine learning?

[Cryptopotato021]

All of us know that everything(Including the data we generate) on the blockchain has pattern and people who have been using bitcoin and ether complain that the network is slow. People often try to hard fork the entire network to combat this problem!.
We are also on the brink of witnessing quantum computers which would prove very detrimental for the progress of blockchain. I have two questions

1. Could we write an algorithm to generate a consensus algorithm using the data

2. Is it anyway possible to merge two or three consensus algorithms together?

[Cryptopotato021]

Most hard-fork/chain-split to solve scalability problem have certain trade-off & most of people who do it don't care or never research the impact.

Could we rejoin the chain once it splits?

Yes, but depends on the consensus algorithm to be merged. The most popular is PoW-PoS hybrid.

How do they decide on which aspect of the algorithm to retain and which to discard?

[Cryptopotato021]

What i meant was it's possible to combine multiple consensus algorithm before the cryptocurrency is released, but it's not possible to add/remove consensus method on running network (at least without hard-fork)

But Vitalik is incorporating a new consensus algorithm right? So he's actually changing the consensus algorithm on a running network right?

[HeRetiK]

What i meant was it's possible to combine multiple consensus algorithm before the cryptocurrency is released, but it's not possible to add/remove consensus method on running network (at least without hard-fork)

But Vitalik is incorporating a new consensus algorithm right? So he's actually changing the consensus algorithm on a running network right?

Using a hard fork (as mentioned in ETFbitcoin's brackets).

So it's not impossible, but very hard and risky, depending on the size and activity of the network (both from a technical and a social point of view, ie. the larger the community the harder it is to consent on upgrades via hard fork).

[Cryptopotato021]

Using a hard fork (as mentioned in ETFbitcoin's brackets).

So it's not impossible, but very hard and risky, depending on the size and activity of the network (both from a technical and a social point of view, ie. the larger the community the harder it is to consent on upgrades via hard fork).

Can anyone hard fork the network? Or you need certain requirements to do it?

[HeRetiK]

Using a hard fork (as mentioned in ETFbitcoin's brackets).

So it's not impossible, but very hard and risky, depending on the size and activity of the network (both from a technical and a social point of view, ie. the larger the community the harder it is to consent on upgrades via hard fork).

Can anyone hard fork the network? Or you need certain requirements to do it?

Anyone can. Which is why we got this clusterfuck of Bitcoin hard forks after Bitcoin Cash caused a Bitcoin hard fork hype:

https://bitcointalk.org/index.php?topic=2597083.0

Ethereum Classic also forked off Ethereum without requiring any "permissions" -- that's the beauty of an open system like cryptocurrencies.

(sidenote: Technically Ethereum forked off, with Ethereum Classic following the original blockchain while Ethereum reverted the transactions caused by the DAO attack)

Whether your hard fork would be successful is a different question however.

[Cryptopotato021]

Won't forking the chain randomly decrease the value of a particular asset? Don't you think we should have like a committee of people who gets to decide if a blockchain has to be forked or not?

[HeRetiK]

Won't forking the chain randomly decrease the value of a particular asset?

Why?

If it's hard fork no one cares about the market impact is close to nil.

If there's a market impact then people care about the hard fork, in which case we can assume that there's likely some merit to their approach.

Historically speaking hard forks caused price increases at least as often as they lead to price deprecation, if not moreso (e.g. the impending BCH and B2X hard forks lead to a run up on BTC's price, the impending BSV BCH hard fork... not so much). In conclusion the market impact is pretty much unpredictable, if there is one (which in the case of the numerous Bitcoin hard forks was relatively seldom).

Don't you think we should have like a committee of people who gets to decide if a blockchain has to be forked or not?

If a committee of people could decide on who forks off and who doesn't we'd lose a major aspect of the freedom and permissionlessness of cryptocurrencies. No one is in control, and that's the whole point of it.

In practice it's the free market as an extension of our community that decides which fork becomes successful and which doesn't. You don't need a committee for that.

Note that I'm talking about "independent" hard forks. "Sanctioned" upgrade hard forks are a wholly different thing (e.g. the one that Vitalik has in store for Ethereum). In this case you don't just "randomly hard fork" off the main chain but rather have a developer team that orderly decides on a course of action (e.g. the switch from PoW to PoS) and -- optimally -- a community that supports this decision. Or not, in which case parts of the community can fork off and see whether their approach is more viable.

[mixoftix]

Re: Is it possible to generate a consensus algorithm using machine learning?

your topic subject was interesting - refer to engaging ML in consensus process, but I found nothing about it in content. so I really need to focus to the subject and discuss it more in depth. do you know, the human brain is the best practice that ever developed about node-consensus in the universe? every decision that we make as human, is coming from a perfect consensus algorithm that we try to emulate it by Neural Networks (NN) in AI. just with Bio-NN there is always one stake-holder (the owner of the brain) that the whole process belongs (is centralized) to him, but in Artificial-NN the outcome of the whole process has its own impact on all stake-holders and this makes it complicated.

NN always provides a Black-Box processing unit, so nobody really could understand what would be the next step of one person and if you put a cryptocurrency in the hand of an Artificial-NN, you may have a blockchain that after many years, somehow may gives its vote to a traitor node and nobody could stop it!!

BUT, you could learn something really useful if you ask: "what would be the main characteristics of an advanced consensus algorithm?". so, just simply take a look at the human brain, which is drastically DECENTRALIZED and powerful by its PARALLEL-PROCESSING-UNITS that SHARE their jobs (not in the race of "who is doing it first..") including HUGE-STORAGE-SPACE and MULTI-ROUTES among nodes including HIGH-BANDWIDTH between them, with the least POWER-CONSUMPTION rates!

because of the black-box part that I have mentioned above, we can not provide a consensus algorithm that works fully by ML, but based on lessons that we could learn from human brain, now we know what kind of characteristics for a consensus algorithm we should look for.

[d5000]

because of the black-box part that I have mentioned above, we can not provide a consensus algorithm that works fully by ML,

Your perspective is interesting, and even as a non-expert I understand the problem that we maybe do not really understand what's going on inside the "black box". So machine learning may not be (in its present state) the way to "create" a totally new algorithm.

But it could be interesting not to let the NN "create" the algorithm but to use it to select an algorithm from a group of "humanly constructed" algos, creating simulations in the same way AlphaGo or AlphaZero did, only using Bitcoin as the "game" and the attacker(s) as one party and the honest participants as the other one.

The idea would be to simulate different complex attacks, and then compare the values of the different algorithms. This way, one could create inputs based on the incentive structure one wants to achieve, and play around with the values. One of the NNs, for example, could simulate to be an attacker using a "51% + short-sell" attack with different algo configurations, trying to profit from it, with several other NNs playing the roles of different market participants.

[mixoftix]

But it could be interesting not to let the NN "create" the algorithm but to use it to select an algorithm

Yes, this is interesting. in theory we could train one miner or node entity as honest and another one as traitor - then SAVE their mind (trained NN) somewhere and CLONE them as much as we need for the sample algorithm in different environmental incentives.

BUT, the applied version of including NN in crypto world could happen in client applications for FRAUD DETECTION purposes.. specially when they run in SPV mode and they are more vulnerable to attack..

[Cryptopotato021]

That's very interesting to hear. How about this?
The neural network takes decision based on weights that is given to each and every input. This is what i was thinking when i posted this. I'll just write it down in steps, Please let me know if i'm correct.
 
Step 1 : So at the beginning we set the same amount of weight to all the inputs( Taking good nodes and bad nodes as the inputs).

Step 2 : We make this model the administrator of the network which has equal amounts of good nodes and the bad nodes. and there is a pool of other nodes(good) which could be introduced into the network based on the actions in the network.

Step 3 : Now, We just let the network run. The algorithm is written in such a way that once a bad node tries to manipulate some node he's immediately replaced with a good node from the pool.

Step 4 : We could also give weights to attacks. If it's irrelevant and if it's making no much damage let's just keep the node. If it again repeats the same thing the node would be replaced.

Step 5 : For scalability we could just choose a particular set of nodes and simulate a tremendous amount of transactions and pin point to what kind of nodes are causing this and we could drastically reduce or try applying compression algorithms to that particular node.

I don't know if what i'm saying makes sense. This is what i've been trying to build from the past week

[HeRetiK]

That's very interesting to hear. How about this?
The neural network takes decision based on weights that is given to each and every input. This is what i was thinking when i posted this. I'll just write it down in steps, Please let me know if i'm correct.
 
Step 1 : So at the beginning we set the same amount of weight to all the inputs( Taking good nodes and bad nodes as the inputs).

Step 2 : We make this model the administrator of the network which has equal amounts of good nodes and the bad nodes. and there is a pool of other nodes(good) which could be introduced into the network based on the actions in the network.

Step 3 : Now, We just let the network run. The algorithm is written in such a way that once a bad node tries to manipulate some node he's immediately replaced with a good node from the pool.

Step 4 : We could also give weights to attacks. If it's irrelevant and if it's making no much damage let's just keep the node. If it again repeats the same thing the node would be replaced.

Step 5 : For scalability we could just choose a particular set of nodes and simulate a tremendous amount of transactions and pin point to what kind of nodes are causing this and we could drastically reduce or try applying compression algorithms to that particular node.

I don't know if what i'm saying makes sense. This is what i've been trying to build from the past week

That sounds like the end result would be a centralized administrator node run by an AI, rather than a decentralized consensus scheme? Or what am I missing here?

[Cryptopotato021]

That sounds like the end result would be a centralized administrator node run by an AI, rather than a decentralized consensus scheme? Or what am I missing here?

What if we divide the administrator power equally? The decision would go through if only if majority of them accept it's correct. We could also process the data in multiple containers so that not one entity has control over the process.

[HeRetiK]

That sounds like the end result would be a centralized administrator node run by an AI, rather than a decentralized consensus scheme? Or what am I missing here?

What if we divide the administrator power equally? The decision would go through if only if majority of them accept it's correct. We could also process the data in multiple containers so that not one entity has control over the process.

That sounds like we'd be back to square one though.

How to determine the majority (ie. how to prevent the majority vote from being manipulated)? How to tell "good" AI containers from "bad" AI containers?

[Cryptopotato021]

That sounds like we'd be back to square one though.

How to determine the majority (ie. how to prevent the majority vote from being manipulated)? How to tell "good" AI containers from "bad" AI containers?

The thing is each node makes a decision on it's own and it doesn't know what's happening with the other nodes. It wouldn't even know that there are other nodes. It would just send out it's decision based on the data that is fed into it so i don't see how it would turn bad!

[mixoftix]

Please let me know if i'm correct.
I don't know if what i'm saying makes sense. This is what i've been trying to build from the past week

I afraid not. as HeRetik said above, you are going to centralize everything.

When we talk about AI, this means we are talking about a Decision Support System (DSS), not Decision System. don't go back to the idea of giving the whole consensus responsibility to the AI - this is just another kind of centralization and doesn't work.

and if you are going to work with some artificially-living nodes in your laboratory for studying something about Well-Known Algorithms, after training an entity you should let them work and never stop them in run time by any means - otherwise, you just kill your study. you need to see how many times of how much traitors could break down the consensus model and what would be their achievements. if you kill them, you will never understand your answers..

==========

BUT, instead of that, you try to use AI in enhancing the consensus protocol in run time. for example:

1- WHICH nodes would be better for my FULL/SPV node to connect? parameters: ping time, up time, etc..
2- WHAT difficulty target would fit better for the next block? parameters: distribution of hash power among pools, a disaster close to one major pool, maintenance in pools, changing regulation policies regarding to cryptocurrencies in one country, etc..
3- HOW we could arrange a format in nonce values that protect the network from selfish miners? statistically analysis of nonce values in
history of blockchains.. [I use this one in introducing BOUNCE value instead of classic nonce]
4- even WHEN will be the deadline of planning a hard fork for the network? parameters: number of nodes that follow the network, number of miners, volume of trades, destination of payments, etc..

these are living-problems that you could use DSS to better identify them..

[HeRetiK]

That sounds like we'd be back to square one though.

How to determine the majority (ie. how to prevent the majority vote from being manipulated)? How to tell "good" AI containers from "bad" AI containers?

The thing is each node makes a decision on it's own and it doesn't know what's happening with the other nodes. It wouldn't even know that there are other nodes. It would just send out it's decision based on the data that is fed into it so i don't see how it would turn bad!

But in the end there still needs to be a consensus about the state of the ledger, right? What if you simply flood the network with malicious nodes? There doesn't seem to be any way to prevent sybil attacks in this scheme. If the majority of the network is malicious, there's no way for nodes to know the "correct" behaviour. And flooding a network with malicious nodes is fairly trivial if the economic cost is negligible (unlike proof of resource schemes such as PoW or PoS).


Edit:

BUT, instead of that, you try to use AI in enhancing the consensus protocol in run time. for example:

[...]

3- HOW we could arrange a format in nonce values that protect the network from selfish miners? statistically analysis of nonce values in
history of blockchains.. [I use this one in introducing BOUNCE value instead of classic nonce]

[...]

Neat examples, especially #3 makes me curious.

Could you elaborate on that? How would changing the nonce format help prevent selfish mining?

[Cryptopotato021]

Thank you for taking some time out to help me with this @mixoftix and @HeRetik. I get where i am going wrong and would go back and try finding solutions. Wouldn't give up on it though

Edit

Neat examples, especially #3 makes me curious.

Could you elaborate on that? How would changing the nonce format help prevent selfish mining?

As long as the format remains the only thing that matters is the amount of computational power (PoW) or the amount of coins (PoS) you have. But once you change the format it would become very hard for you to change the requirements to mine more.

[mixoftix]

Edit:

BUT, instead of that, you try to use AI in enhancing the consensus protocol in run time. for example:

[...]

3- HOW we could arrange a format in nonce values that protect the network from selfish miners? statistically analysis of nonce values in
history of blockchains.. [I use this one in introducing BOUNCE value instead of classic nonce]

[...]

Neat examples, especially #3 makes me curious.

Could you elaborate on that? How would changing the nonce format help prevent selfish mining?

the main idea was about finding a pattern in nonce values, then a miner could speed up his mining process. for example, what if we know 90% of nonce values in mined blocks already are EVEN and only 10% of them goes for ODD values? then I found this link that contains more detail information:

https://bitcoin.stackexchange.com/questions/24650/looking-for-nonces-of-even-numbers

I recently also checked the nonces from block 552,780 to 253,898 of Litecoin.

totally 298,883 blocks.

    number of odds = 42,963 (14.374521%)
    number of evens = 255,920 (85.625479%)
    Among the evens, the number of multiples of 256 = 225,746
        75.529890% of total

therefore after difficulty target, I decided to exclude/enforce some patterns in block header too.. for example, sometime nonce values CAN NOT be even , sometimes nonce values SHOULD be EVEN, etc. and I have called these sort of conditional-nonce-values as BOUNCE..