So I was wondering if any attempt has been made to profile node performance in the over all, and where bottleneck could be, and what has been attempted to improve it ?

Im not talking about transaction confirmation time, or protocol related things, but how fast node can process blocks, mostly in order to speed up synchronisation process, or node latency in general in serving the different requests.

Synchronisation time is still big problem, and from what I could see in quick tests, it doesn't seem to come mainly from raw download time, or network speed that much, so I was wondering what has already been thought about or tried to improve this.

In the few things id try it would be :

Optimising operations on hash, like at least hash comparison,  possibly with sse or avx,as there is probably a good number of those being executed for each block process.

Asynchronous i/o, using asynchronous ( interrupt based) io I guess could speed things up signifiantly, as cpu can process data while the io operation take place, from my experience with video streaming it make a good difference.

Optimising storage engine, I already saw certain coin bitmonero tweaked with different kind of storage engine, either Berkeley db, level db, or other things, or even zero cache nodb disk storage like in purenode, wonder how much of a difference it really makes.

Parallelisation/threading ,  parallelizing as many things as possible to take full advantage of multi core architecture, it cant be applied to block processing because each block validation depend on the previous one , but for processing all transactions and signatures in a block, it could make a difference if total processing time is long enough to benefit from threading.

Optimising signature check, most coin use openssl for signature, but there are other libraries like micro ecc or others , often made for embeded system, would could be potentially faster than openssl.

Block compression potentially could help too to reduce slightly download time.

Saw about thin block but from what I can understand, it wouldnt change much on synchronisation time or request processing.


I guess that's it, ill add more if more things come to mind.

Also is there any profiling data available somewhere ?

Asking for depression advise on bitcoin forum 😄

Id say the big thing to avoid dépression is accept yourself, and dont think you should be something you are not and cant achieve, or that you should be more than what you are, it's putting too much pressure on yourself and deplete your mental energy to reach goals that probably arent even really making you happy be side the image you have of it.

And also you need to find things that make you feel happy, and do them more, instead of trying to push yourself to do things that doesn't really make you happy, and doesn't make you feel good about yourself.

You can only push yourself when you have good mental stability and not plenty of different inner conflict about what you should be doing or not, and how bad you are to fail to reach those ideal. You need to find that mental stability and be at peace with yourself, be side fail ure or not meeting a certain ideal version of yourself, then you can eat any problem alive

Need patience, compassion with yourself, nobody is perfect and really you never know what you can become

Well just my 2 satoshis lol

I would believe there is still potentially repetitive data in blocks that could gain from compression, but not sure how much. But there are still adresses hashes in the script and certain thing that could gain a bit from compression, not sure the gain in transfert speed would over come the loss in compression/decompression time though.

Oki i'll check those

Found this :

"Datastream compression: Testing various compression libraries such as LZO-1x and Zlib have shown it is possible to further reduce block and transaction sizes by 20 to 30% without affecting response times which could also be applied to thinblocks"

Not so marginal

Is there any attempt in compressing block data sent over network ? Would that improve block propagation time ?

Is there a place I can find such kind of test and benchmarking ?

Is there any test using asynchronous io too to see if that can make a difference in synch speed or if it's mostly processing power on signature checking that take most of the time ?

Im working also on light weight core in C, and I agree the way bitcore is coded make innovation very slow and hard to do , and it seriously limit innovation potential, especially for non expert in c++ programming

Is there any serious profiling made with coin software to identity performance bottle neck ?

It's probably à thing im going to do in a near future, but it doesn't seem very easy to me cause it involve both i/o on local drive and network, and cpu processing, plus sending nodes dont have necessarily the same through put, and the Connectivity between two nodes is not necessarily the same either, seem hard to do real profiling with this kind of software, but im probably going to get into this soon, and seeing how simd / avx and multi threading can increase performance, but blockchain are not necessarily good target for multi thread cause each block has to be checked after the previous one, so it's more serial than parallele, but im trying to find where threading could really improve performance. Maybe on intra block level as transactions in the same block are not supposed to have dépendance on transactions in the same block, that could be good candidate for threading.

Maybe even different compiler used with different options can make performances difference too.

But as the purenode core code is very simple, it will be much simpler to do good profiling of different parts and see where performances can be improved.

For the moment it seem hard to me to identity clear bottle neck right off the bat like this, there is too much things that can affects node synchronisation speed.

I didn't do much testing with this, just tested with bitmonero cause they have different options to import/export blockchains in different format, but when I tested it didn't even seem to make that big difference to import from local file or download from network, I have optical fiber too, so I guess network speed is not the main issue, but I didn't do much heavy testing yet.

Imo if you want to have meaning result out of this kind of test, you need to do it at least 2/3 times and average the time différence to already eliminate issue coming from the network Connectivity itself and see if you get the same order of result doing it several times, because it's very possible the difference doesn't even come from the computer itself. If one is synchronising from slow node, or have to reject bad blocks or eliminate issue from network Connectivity.

Even WiFi sometime you move the devices 5cm you get x3 signal :p

Would need to make this sort of test on the two node synchronising from the same node who have stable through put and stable connection, like local wire network synchronising from same node, one after each other there you can have more meaning full result on the computer performances itself.

All together it doesn't seem trivial to do good benchmarking / profiling on synchronisation speed.

Last real innovation is multitasking and virtual memory in the 90, since then nothing really new under the sun, just rehashing and re packaging of same stuff over and over again under different brand and shape until people understand they are being sold the same thing over and over again as being innovative :p

It's always the Steve Jobs myth of the guy who Mount millions dollar company out of his basement, it surely looks like the investor holy grail :p

As rumi said, there is fake gold because there is true gold :p

Im trying to quit fud, but I still have relapses from time to times :p

One of the good way i found is playing vidéo games where you can put the voice in one language and subtitle in another, like this you can learn some vocabulary easily, with the wrote and spoken version, can repeat the sentence many times, and see how it fit within a context in the game world and it's funny and motivating

The market for fud is very large, you cant fill the whole demand for internet fud on your own, you are going to burn out

There is a coin about lepen too, but she wants to ban bitcoin, how much sense does it make ?

I'm fairly new in bitcoin community , so I can't tell how true what you say is, but it's still unfair for certain persons

In general term anyway, computer world changed a lot , especially since the 90, there has been the hype at first with website, php, then same hype/bubble with web 2.0, Android applications, all serving the same thing over and over again pretending at some innovation  to get increased worth, and the IT industry become very speculative and less and less people really want to bother with code or see it as something cool and fun like it could be before.

Personally im also an oddball, I pretty much left companies behind some years ago because I think professionnal IT is becoming whatever, especially since web 2.0 era, people putting software stack upon software stack upon software to get 40mo to have the most simple application, framework get bigger and bigger all the time and software quality is decreasing à lot.

And especially in france, the culture is not that much tech friendly or on open market, id say on the over all very conservative, and I dont see a lot of companies pulling that much great things currently.

But it's why I took lot of time to develop systems that are much smaller and easy to grasp for even noob developpers, that can still deal with internet standard, like javascript & json, rpc, dynamic typing, hierarchised data, and can replace boost and templated c++ very efficiently for lot of different purposes, in a way inspired from AS3 logic.

Actually I crossed the blockchain world accidentelly, but I was already interested in developping small optimised and portable solution to deal with distributed component, asynchronous requests, keeping things safe and private in open world, and I was turning this problem around until I stumbled on blockchain who completly solve all this in very elegant manner.

It's how I came up with making a coin and implementing the functions needed to run a blockchain as a proof of concept application for my framework. But it's already quite advanced, especially on optimising parallele systems, dealing with json dynamic data, having a very robust and safe memory system.

Need less to say I did all this for free without having clear commercial objective in mind, im still un sure how to goes with this, but ico still seem to me a good way to rise funds for projects, and still keep the possibility to maintain high quality open source software without having to deal with all the complexity of patents, NDA, proprietary software, and without having to ressort on sun day afternoon coding or part time developpers who drop a line of code when they have free time.

In last ressort, there is always this :

https://en.m.wikipedia.org/wiki/IP_over_Avian_Carriers

It's not based on blackcoin, basically you could say I took the blackcoin core as a model to implement the protocol, but im developping another core with much wider scope, but blackcoin seemed a good model as it implement 3 staking algorithm and pow, and seem robust and reliable , so I took it as a starting point to understand the protocol and how wallets and node are made.

But the purenode is completly different, built totally from scratch in C and assembler with a framework I developped originally for an operating system project, based on micro kernel architecture, so I can tell the code is really entierely from scratch

I mostly take blackcoin as reference core, in the optic that if my core can interact with it, synchronise blocks and tx, and emit new blocks that are accepted by it, it mean my implementation is kasher, and I guess the black coin core will be used at first the time I finish my core 100%.

But in my mind even dealing with coins is only a subset of the possibilities I want to exploit, like low latency private chain without block reward, or many other things related to crypto.

But still blockchain are big on coins, so good proof of concept is managing a full featured coin with pow and pos, and I took blackcoin as a model to understand the gist of it and test my implementation

Im still un sure if I should better go wild on creativity on new blockchain features based on my core, or improving compatibility with existing core and protocols to make classic coin on the alt market, the two path are interesting, but for the moment im still focused on implementing the protocols and middleware things, even if it's already quite advanced, I think the first real use as in line with multi coin wallets, decentralized exchange/swapping, and globally middleware for other coins, but I have also plenty of idea to develop on non conventional coin chains, like the raytracing and audio/vidéo streaming in which i already have some experience


The code of purenode is in the other repository

https://github.com/iadix/purenode



Hope it's not too long and answer your question

Hello, im new on the forums, and in bitcoin community, I saw the issue of ascendency, quite elaborated scam, very disapointing  ..

That said been into various computer scenes since im teen, this whole idea of people wanting to invent themselves à life on the Internet by pretending to be super good programmer, and only practicing copy paste is not news, neither vaporware projects that have been plenty in the 2000'.

it's not specific to blockchain and is inherent to highly innovative environment where investors are always scared to miss the train and get convinced they getting a ticket to the first wagon for cheap. And lot of people who are no life and bored and thinks they can get some easy recognition in the nerds world by pretending some kind of skills or projet to drag attention, the temptation is even higher with btc because there is direct money involved, beyond reputation on Internet community.

That said im not very active on forums for a while, i tend to consider it mostly waste of time, I tend to think the level on forums decreased a lot past ten years, and I dont see that much really deep technical informational discussion going on, mostly need to get through wall of trolls and parano to get one or two eventually useful answer, I prefer to keep my mind focused on the code, rather than on making reputation in a social community.

Now I know the blockchain world is very collaborative , and based also on community , and need to keep an eye on the eco system, but im not rabid forum poster either, and I can figure out the info I need much faster by grabhing code and doc rather than sorting out trolls, paranoiacs and fake on forums.

The two last time ive been active on forum it didn't really make me want to invest much time in being active on forums , and it's much faster and less stress full to figure things on my own quietly with a coffee

Things are still going to move before the start of the ico, and we are still reviewing some details, normally I removed the php notices above and changed the mail contact name, we are still working on improving mailing to get throught spam filters, and on some update on the website, thanks for your feed back

Date of birth is not mandatory, the only mandatory things are the sign up infos, aka the mail & pass word, the other infos are optional and can be left blank, but it can be used to have more infos on participants, but it's not really used for the ico for the moment

To break it down in simple points :

- It's light, 20mo of ram, 500kb of application data, including node, html5 wallet & block explorer.

- Oriented on html5 web app, and externalising functionalities that deal with private keys in the browser.

- It's modular, code can be changed and recompiled in a minute, and binary modules works on both win & linux.

- It can be configured to synchronise with different blockchains, or create new coin with customized parameters without recompilation. It can handle pos2 , pos3 and sha256d/scrypt pow for the moment, but more algorithm can be added easily.

- Include a full application framework to deal with distributed application, eg. html5 GLSL raytracing ( soon online).

Basically it's the best blockchain engine ever, that's all you need to know !

I wanted to add some informations and clarify certain things because the site is not that much informational.

Already the reason why there is no download link or source on the site, is due to the nature of the project which is more conceived as a blockchain engine rather than a monolithic core linked exclusively to a coin, and we have been testing various kind of projects with different kind of blockchains, not all of them are focused mainly on holding coin, or hold any coin of value, so to avoid confusion  on the website we prefer to focus on the ico and have simple interface rather than having different kind of wallet and test and middleware which would be confusing on what is exactly related to the ico coin.

The way i view it, it's more to be seen on the over all as an application framework to work with blockchain for different purpose, rather than being focused on handling à single coin, the property of blockchain that is very interesting to me is the principle of publicly verifiable digital ownership, and ultimatly im more interested in developping distributed application that exploit this property, rather than developping a single coin.

Now also having a well maintained coin of our own is still a must to exist in the blockchain world, im not sure which future update are to be integrated into this coin vs other blockchain, but changes should be easy to integrate into it, either it will require some kind of fork to integrate future progress on non transaction blocks and hierarchised data, or integrated on the basis of external modules if they dont change the way blocks are validated.

One of the problem I see with most core ive seen is that it's complex c++ code, very monolithic, and not developped as a library or framework that can easily be integrated in other applications, the rpc interface mitigiate this, but it still limit the possibilities to make advanced low latency application, cryptonote used an approach more in tune with having a core library that implement the base functionalities, and different blockchain based on the same core, with the node separated from the wallets, but for most coin need to deal with the whole thing with the forks and tweaks specific to a coin, and it's not that trivial to update the code due to high coupling and complex code, long compilation time, need for lot of framework to compile and modify the code etc

The system of standalone binary module is also a key feature, for example it allow to add and deploy new rpc functions very easily, just need to declare the function in the C file, recompile the rpc module (which is standalone , do not depend on anything on the system, not even base stdc header and runtime), and then the module can be copied from windows to linux and vice versa, which allow for very fast deployment of updates, and easy check of code integrity as it's the exact same binary module that will run on all systems.

Another point that is also key is related to graphic interface, most coin opted for QT , but it's still rather heavy, and not trivial to modify, and need to recompile the wallet for each platform each time a change is to be made on the interface. With the html5 approach, the wallet application is made in html5/js, so it works on all platform, can connect remote node with the graphic interface from a browser, and can be changed and customized easily, without recompiling.

The last aspect I want to speak of is related to separation of public and private data which i find rather loose with the monolithic core approach, the public data is the blockchain itself, and private data are private keys and wallet based infos, and both are stored in the node without a true system of inner accounting, and data signature also has to be done with the node itself, whereas with the html5 approach, it works more like a credit card where the browser can sign transaction on it's own, without a node being locally installed,or the node having to know about the private key at all, and private data remain out the node and remain private on a per user basis rather than a per node basis where all the keys stored in a node are accessible and useable by anyone having access to the node.

The modifications will mostly affect purenode engine and the qt wallet will disapear latter on, it's used to kick start the coin with a well tested and recognized mature core, purenode can already synchronise and check the block and tx, and can support either pow or pos or hybrid coins, with properties being customisable via a conf file.

Hello

Im not sure i understand your question very well, if you are asking for win32 support, both the qt wallet and purenode should works on 32bits windows, and I dont see good reason to stop the support for 32 bit platform in the future

After all I bothered to code 64bits and 256bits arithmetics for 32 bits architecture without openssl and C runtime, I guess the support for 32 bit should be kept in the future  

Normally purenode should works on xp too.

Hello, im iadix developper

First I want to say there is already some working source code, and we wont "run away" after the ico ends

For the moment there is a wallet based on blackcoin that works on win 32/64 and linux, and the expérimental core im working on which is modular portable and light, easy to customize, it use no big c++ framework, so it compile easily , and include already most function to synch à blockchain, and have web app like wallets and block explorer.

The definitive coin used for the ico is not compiled with the definitive parameters yet, but it will be before the start of the ico, there is already a "demo blockchain " working currently though, and we have been testing it for a while already.

We plane to be transparent on developpement , and I should be active on it in the near future. And I can answer questions if there is any

Im working on distributed raytracing based on blockchain currently too, Im not sure what to make public at which pace yet but there should be some working app based on html5 and glsl in near future.

Some html5 based on audio streaming and blockchain is also under development and already tested on small scale.

The experimental core im working on is highly customisable so it can be adapted easily to many blockchains, like the one we use currently based on blackcoin pos3, or it can create new blockchain with customized parameters easily, even without recompiling anything, in a way similar to multichain with configuration file. It supports only scrypt or sha256 Hashing algorithm for the moment, but more can be added easily, and the api is compatible with bitcore & traditional web block explorer api.