OP.

"I guess it will be simply impossible to process millions (billions) of transactions daily "
but that mindset is overshadowing this:
"It is almost a given that if the Bitcoin adoption should increase multifold in the coming years"

lets emphasise..
"coming years".

just 16 years ago the largest hard drive someone could have was 4gb. and the largest portable storage was only a few megabytes.
the internet was dialup.

things have changed.
in 2009 1mb blocks were acceptable.
stats done on average internet speeds (even by companies like livestreaming, videocalling services) and by those concentrating on bitcoin have shown 8mb is ok now.
and storage is well over 2 terrabyte for average home cost PC's.

meaning we can grow upto 8x now. no issue..
even core now believe 4x bloat (bloat not capacity) is safe enough within the next month..

and as you say millions/billions over YEARS. (not hours/days) is plenty of time for the hardware and bandwidth to allow more growth.

in short.
put yourself 10-16 years in the past and while on dialup with a 4gb hard drive.... ask your younger self.
"could you see in 10-16 years that just one shoot-em-up game will require the storage space of 20x the largest hard drives available, and people wont even care/think about it and just treat it as ok/normal..
that playing others while talking to others and showing each other your game play would seem normal too. but require internet speeds of over 20x the max available"

your younger self could not imagine that..
much like you cannot imagine the future in 10-16 years time

by the way, if you done some research XT was just a bait and switch for the same guys behind blockstream. R3 and blockstream have been working together for months. go checkout the names of those involved in hyperledger.

oh and before crying FUD about everything that goes against blockstream. research first.
as for segwit. you claim there is large support by mining pools..
um...
https://blockchain.info/charts/bip-9-segwit    7.1%
7.1% at time of posting.

BU. also in the 7% range.
so using your "competitor" mindset.. they are neck and neck

as for you saying there are only 4 options and 2 are dead meaning you are suggesting there are only two options.
im kind of glad you didnt mention knots and bitcoinj. im hoping its because you already know they are already under the umbrella of core.

but there are other implementations too.

but anyway you are acting like its a competition to be reigned in as king. rather than thinking of the bitcoin network remaining diverse. and not having a king

BU has not been making claims that core should fork off to an altcoin. yet core have been making claims that BU should fork off to an altcoin. funny that!.
please take off your core fanboy hat and put on a unbiased bitcoin network hat and try to research what you preach

for months now the solution has been clear.
dynamic base blocksize along with dynamic weight to allow the features some love with the capacity growth we ALL want.
which would require both a node consensus followd by a pool consensus. and dont worry. nothing happens unless consensus is reached.
and if not reached nothing happens.
again if reached there is no intentional split/altcoin rhetoric. there is just a period of orphaning by the minority that have not updated where they cannot sync. again not chain splitting. just a loss of sync-ability by a low minority. and they upgrade if they want to remain full nodes.

after that drama has surpassed.
where the default at a high consensus starts at 2mb base 4mb weight, and can grow naturally without as much orphan risk and without the sync ability drama, by using the consensus of independent nodes settling their differences to an acceptable rate, . rather than dev teams setting the rules(no longer need developers to release versions just for limit changes, meaning no upgrading of nodes just for limit changes).

EG
imagine a future where
50% of nodes have 3mb base 6mb weight
30% of nodes have 2.7mb base 5.4mb weight
10% of nodes have 2.5mb base 5mb weight
6% of nodes have 2.2mb base 4.4mb weight
4% of nodes have 2mb base 4mb weight

pools can see that 100% agree to 2mb base 4mb weight. and see 96% agree with 2.2mb base 4.4mb weight.
yes those numbers are correct. because a 3mb base agreement means anything below 3mb is acceptable. its not the opposite.
so a 0.99mb block IS.. emphasis IS happily accepted by nodes with higher setting. just like (random number 0.2mb) is acceptable to 1mb limit nodes for the last 7 years

meaning pools can while making blocks anywhere between 250bytes to 2mb(when consensus shows 100% agreement to upto 2mb),.. pools can go through their own mining flagging consensus to see if 2.2mb base should go forward which will prompt the 4% laggers to dynamically change if pools reach an agreement.

all done dynamically without developer spoonfeeding.
obviously it stays at 2mb unless pools consensus reaches agreement

basically everyone gets their cake and gets to eat it.

the ones i made red are because of cost of set up, legal frictions with regulators and low chance of income/returns unless alot is invested in time and funds to set them up right

to add, for a low upfront cost but getting the ball rolling faster and more chance of healthy headache free returns
bitcoin meetup, conferences, conventions
bitcoin consulting services EG setting businesses up to accept bitcoin
bitcoin learning centres, colleges, hackathons

the issues with the fee war are.
the obvious ofcourse:
its not a first in first confirmed.
its no longer based on age of coin metrics (of how long ago the funds were recently used) meaning a rich guy can keep spending every~10minutes uninhibited)

but here are some other failures people are not noticing
1. at say 10sat/byte.. knowing average tx is 400byte. thats 4000sat/tx MINIMUM, on a no demand day (just under 3cents)

2. sites that suggest fees are operating in 10sat/byte intervals. meaning it psychologically makes people outbid each other in amounts of 4000sats (3cents rather than 'micropennies/individual sats). meaning if the average was 0.00012. people then pay 0.00016 instead of 0.00012001. pushing the price up by near 3cents per outbid rather than 0.001cent

3. fee's are now estimated over an average. rather then being instantly reacting to demand. meaning it hold the suggested price up for longer even with empty mempools

4. with the "average" metric instead of instantly dropping to suggest that free transactions are allowed in when mempool is empty. it is still biased towards pushing for fee's

basically
economics/price wars should never be used to sort out priority/spam.. CODE and filtering out bloated transactions or repeat spends too quickly, should be used

EG same as blockreward, not mature for X blocks to spend
EG no arbitrary data/messages, just lean tx code

it seems core devs are spending coin to bloat up mempool. to try to later suggest that only core have the solution and people should download core and get pools to use core to solve the problem.

the issue however is that its not that straight forward and requires people to sway pools to flag for cores feature. and then wait a month. and then download a new core implementation AGAIN. followed by moving funds from legacy (old) keypairs, to new HD segwit compatible seeded keypairs.

thus actually causing more bottlenecking later with everyone moving funds just to utilise a core feature that only has potential boost of 1.8x capacity.

its making everyone sit down to let core run around to demonstrate there is no room for everyone to walk. purely to then get everyone to run around with half promises they can freely walk around (basically child games to make a point)

short answer nope
no financial incentive to run a node.

the fees end up with the mining pools who add the fee's onto the blockreward as a bonus for themselves(in many decades the reward deminishes and the fee's become the main income for mining pools)

nodes receive no financial incentive

however if you care about security and trust that your private keys are safe and that by being part of the network you are helping to distribute(decentralize) the network to ensure there are less chances of attack, then that is your incentive(securing bitcoin).

however if everyone was to "just run core" then core developers then control the rules(become kings). so look for alternative full nodes to add more network security by being diverse, as well as decentralized, that way no one controls the network, and thus bitcoin is more secured

and here is another reason core are happy with the fee war.. its not about spam. its about money..
finding new ways to give people headaches and then cost them more money to get around a headache.

if only devs used code to sort out spam. like limiting sigops and having a cooldown period (like the blockreward has) before spending.. instead of using "fees" as the mechanism to avoid spending or making it costly to spend.

i feel ashamed that the dev's are using economic rules instead of code rules to sway who should and shouldnt use bitcoin.
2013+ has definetly been the period bitcoin jumped into the bankers pockets

kiklo
your crapcoin zeit can do small times for a simple reason. it has no utility.

as soon as you start having to propagate/relay zeit blocks to more than a couple hops away (which adds time to when the entire network gets informed of a new block), things will slowdown and you end up with a network fighting orphans due to some nodes not getting one block solution before a competitor has another.

changing the blockreward mechanism alone as a work around to enable changing the timing also has ramifications.

but hey, go play with your altcoin and see it crash out and get high orphan counts when blocks start to fill and cant get passed around the network quick enough.
you dont see the problem because your altcoin is not decentralized and distributed enough. it also uses PoS which shows your lack of understanding PoW
 

the theory of messing with difficulty to allow blocks to average, lets say 5 minutes "may" increase transaction count. but you have to take into account the other ramifications.

1. it messes with the coin production(blockreward) timing and changes the 2140 when mining should end. to being much sooner
2. pools need time to validate solved blocks and propagate them. if difficulty changes to average 5 minutes. we will see more "empty blocks" where pools circumvent the validation/propagation in an attempt to get blocks solved faster. thus less transactions get added to a block.
3. the 10 minute average has many reasons behind it.
4. changing the 2016blocks over 2 weeks has many negative ripple effects

research 2 and 3 and 4.
learn the why's and hows and whats.

reason 1 alone is reason enough not to do it. for economic, and for reasons revolving around rules that just should not be messed with.

i know you hate the mining pools so lets word it in a way you will understand, purely based on your hate of pools..

changing the timings to 2016blocks a week(5min average). means that in 10 minutes, the pools get a 200% pay rise because the change would mean they get paid twice every 10 minutes instead of once.
plus not guaranteeing that blocks will be full in those 5minute blocks (look at reasons why "empty blocks" are happening. (hint: its not due to low mempool(demand), but pools trying to be efficient solving blocks by starting empty blocks))

also changing the speed cant be "scaled" because there is a limit to how man times you can actually mess with all the mechanisms before it causes security weaknesses, bugs and issues.

so drop the change "blockspeed" idea. the side effects have more negatives than positives.

---

as for increasing the base blocksize. yes that has more noticeable effects and can be changed again and again, thus is about "scaling"..

unlike segwit which is a one time fix. that cannot then be repeated so its not "scaling". its just a side effect one time boost.

so yes changing the base blocksize is about "scaling", unlike segwit or your buzzword "blockspeed"

no

on AVERAGE??
you can check the last 100+ blocks here and count how many are empty
blocksize on the right.. most 999kb
https://blockchain.info/blocks

atleast learn the CONTEXT of the stats you try to use.

especially if your ignoring avoiding other stats simply because out of the mountain of stats available you have chosen one. but not gone as far as explain the limitations of the one stat you chose.

by the way the stat you prefer is something YOU chose, so the onus is on you to explain it. i have just tried to be helpful to show how to atleast back up your one stat by saying you can use other stats, and if a pattern emerges then good for you. but if other stats dont measure up then its best to find MANY stats that can combine to show some correlation..

or just face up that while there are still transactions in the mempool even with the occassional empty block. you need to explain the empty block concept.

atleast try.

and dont assume something because you formed an opinion. back it up with data, stats, information, rational logic, etc

blocksize on the right.. most 999kb
https://blockchain.info/blocks

average blocksizes
https://blockchain.info/charts/avg-block-size?scale=1&timespan=60days

you get the occasion blocks which are near empty.
this is not due to low demand or anything about trends.. growth or popularity of usage, this is about the pools doing "empty block" or spv mining.
meaning they start a new block empty and while hashing away they validate the prvious block. if valid, then they start filling the block with unconfirmed transactions and keep hashing.

sometimes they are lucky to get a new solve thanks to starting before everyone else hense the block isnt full.

learn context of the stats you pull. especially if you only want to use one statistic.
if you cannot learn the context of why one stat is what it is. then its best to use many stats and combine them to find a pattern. but even then you still need to attach some context to explain your opinion of the stats your using.

bad.
the blocksize caps the amount of transactions.

although stats from 2009 to say 2015 could have shown a pattern of growth.
2015-2016 has slowed down. not due to real world adoption slowing down. but more so due to the block limit and fee war limiting bitcoins utility.

unless there was uninhibited growth of the amount of transactions. its not a good idea to use transaction numbers as a bases.

your best plan is to get MANY stats from MANY sources and put them into a chart TOGETHER to form a average display of utility.
EG coinbase customer registrations. merchant adoption. unique bitcoin address counts.

grab every stat your can find and then put them together and if they correlate or you can explain why the ones that dont correlate to atleast show a pattern. then you can use the combined stats as your "evaluation" of adoption.

again just using one stat alone is not a good idea if you cannot explain the context or not explaining why one stat is hitting a limit

banks wont jump to bitcoin.
we dont want or need banks to jump to bitcoin.

bitcoin should remain open and uninhibited from regulation and banker controls.
the only people wanting banker controls are those with dollar signs glued to their eyes that only wanna get rich quick, sitting in a basement.. rather then seeing bitcoin for what it should be.. an opposition to the banking cartel.

bitcoin 2009-2013 was great. it actually offered everything M-pesa had, but without the need of vodaphone control.
people could program their own exchanges, do local trades using paper wallets and the cost of using bitcoin was negligible.

however bitcoin 2013-2016 has changed, not offering the same freedoms as m-pesa. the tx fee alone has priced out developing countries from using bitcoin as a real day-to-day currency.
the lack of momentum of scaling has slowed down being able to use it in a short time guaranteed transfer. thus not useful for actually shopping with bitcoin.

bitcoin has now become just an investment vessel for developing countries. and is starting to do the same for developed countries too. all because coders in certain groups decided to not use real code rules to scale bitcoins utility forward. but use economics to shift demand backwards purely for a "fiat" money grab

yes it would be funny if all my fears of blockstream were actually just blockstream baiting the banks with scammy ICO's.
and then sometime see blockstream shout "thanks for all the fish, goodbye", and crumble hyperledger to the ground, to mess with banks.

but im still hazarding that blockstream are going to hedge WITH the banks and make hyperledger into something and cripple bitcoin.
but id love to see the opposite happen, where the banks lose out

i see that R3 joined hyperledger. there is no need for banks to be under the R3 group while also being part of hyperledger, when they can instead freely invest direct to hyperledger and not have the middleman(R3).. especially if R3 is making financial demands just to stay under the R3 banner.

as for why the banks are investing in the hyperledger, firstly it costs them nothing. their millions of dollars are a tax right off, while then being able to on the side still use the funds even after having it wrote off as a loss.

as for why banks are inventing the hyperledger. because unlike traditional DBMS(database management systems) and RDBMS(relational database management systems) blockchain adds extra layers, in essense an extra programmable layer to RDBMS to give them more freedoms of what they can do with their data that the old RDBMS could not.

as for why not use bitcoin.
banks want more control and choice to how their system works. afterall they are regulated to handle legal tender, so bitcoin has issues with that.

as for why we should not be asking banks to use bitcoin.
we dont wont/need banks to make bitcoin into fiat 2.0. bitcoin should remain independent and open. and not turned into the fiat controlled system that doesnt empower people openly without judgement.

scaling:
the solution is not segwit or dynamic baseblocks.
it's both
EG
(Xmb base Xmb weight dynamic), default set to 2mb base 4mb weight to allow upgrade. and then consensus (independent node users vote) adjusts in own time. without dev spoonfeeding/controlling how or when

diversity vs centralisation:
the solution is not segwit or dynamic baseblocks. its not core or BU or classic or xt or knots, etc,
it's both, its all, its everyone included
all implementations should all have code for all features and then let consensus choose what features get utilised because all implementations can handle it, all on one chain
EG
core allows dynamic baseblocksize and segwit
classic, BU, etc includes segwit and dynamic blocksize

bloat/spam:
the solution is not to use the economic banker mindset of fee wars to control "spam" as thats just a barrier of entry for developing countries
its instead
limit sigops of transactions. as there is no rational need for a single user to need say 500-80,000 sigops in one tx.
its instead
use CODE for rules, not price tags/costs
its instead
lean clean transactional data.
its instead
not allowing arbitrary data to be added that has nothing to do with transaction/validation (even twitter/sms limits how much crap people can type and that is a communication tool. so its obvious bitcoin shouldnt be used for 2.2mb of non transactional waffle in the 4mb weight)


then the choice of "brand" implementation is not about which king should control the rules, but getting back to no king controls the rules, all "brands" are on a level playing field and the choice is then about bug risk or which has the better user interface.

split or no split:
it should never be a chain split debate. it should be about one mainnet utilising consensus.
EG if gmaxwell fears that a feature wont be popular, dont tell other implementations to intentionally fork off.. but instead code it, and then let consensus prove him right or wrong by seeing if it activates or not.

technically its more.. thats why block rewards are unspendable for 100 confirms.

but the chance of a orphan block is 1-2% (atmost risk is 3blocks a day)
in a period where upgrades and consensus changes are happening where pools wont do anything to the blocks unless >95% compliance, means a <5% risk = ~6-7 blocks might orphan

so in a time when there is a bug or a time of consensual feature upgrade where risks are higher, some services may request upto 10-20 confirms, until the risk level subsides.

as for the "trust" / honourability between buyer and seller of zero confirms.. its still not good
some people say its ok to risk $1-2 without confirming.. but.

people buying something for $1-$2 may only see it as value of a loaf of bread in western countries so no big deal.
but people making $1-$2 transaction is not going to pay 7cents-14cents to make such a purchase (7% cost)
so chances are the tx wont even have a fee and it WILL get dropped by the network rather than accepted into a block.

also in the third world. that $1-$2 is a weeks wages. and the tx fee is 1-3 hours minimum wage labour. again chances are no fee added
again dont trust it, if you are transacting with someone from a developing country. treat $2 spend as if they were trying to spend $400 U.S.
you may think its just a loaf of bread. but they think of it as a weeks wage so would be happy to put the effort in to double spend and keep the weeks wage.

its not about how much you spend. its about WHO you are transacting with and the tell tell signs that the transaction may or may not get into a block EG low fee, or opted in for RBF or spot the same inputs being used elsewhere

nah.. i think this is a better realisation. and yes the labels have shifted for a good reason


we have had alot of "bug fixing".. but innovation has only just begun.
and im not talking about sidechains or LN. im talking about ONCHAIN features. like better smart contracts and better ONCHAIN capacity

well many know of the dollar.
but around the world 6.6billion do not know how many loaves of bread you can buy with $10 (only about 320mill truely know.. roughly)
but around the world 7billion do not know that its mortgages/loans that are the route of money creaction (only sat 100mill know roughly)

so even though people "know" the dollar.. they dont actually KNOW the dollar.

same goes for any currency, including bitcoin.
you may hear it exists and is used by millions but dont expect people to know how it works or its "values"