WE DID IT, 2000 transactions PER SECOND OVER THE NETWORK!!
_{proof will follow, test it for yourself from your browser}

Hi fellow HEATers, please allow me to share a short development status update.

Some interesting and major milestones have been achieved over the past week. To list a few:

Blockchain-Event-Cache

Big steps have been made towards finalizing the Blockchain-Event-Cache, the event cache gave us some real hard challenges to solve. An example being how to store to disk the tip or last N events of a never ending stream of events, without running out of disk space! And do it in such a way that no matter if the HEAT server crashes or shuts down, that at all times the persisted state of the blockchain matches exactly with all persisted events.

The significance of the event cache really can't be overstated. It's workings and design are unique and as far as I know exclusive to HEAT. The event cache should be considered as an ordered list of events where most events map to a whole or part of a blockchain transaction. The event cache has a cursor which points to the current latest event and this cursor can move up and down the list of events.

Observers of the events (microservices listening to transactions for instance) will be notified when an event first arrives. HEAT being a decentralized consensus mechanism must account for the inevitable slowness of reaching full network wide consensus simply because nodes need time to talk to each other, the network speed determines this. The event cache is your friend that watches blocks come and go, switches to better forks and reorganizations of transactions, all the while giving you a normalized view of the transaction stream as if you observed centralized service.

When building business applications on top of HEAT using the blockchain as a place to transport and store your data having this 'normal' view of the transaction stream is of vital importance for any kind of application you build. 

High-speed-RPC-channel

What was probably the coolest part of the week was to fire up the new high speed RPC channel we added to HEAT core. From unit tests we could already see that HEAT has been very fast when it comes to processing transactions, we've seen numbers of well over 2000 transactions a second when feeding transactions directly from memory into HEAT.

What hasn't been achieved yet was to process such high numbers over a network, until now that is!!. Two major obstacles of feeding lots of transactions to a HEAT server over the network are:

• Everything is encoded as JSON, parsing lots of JSON costs memory and downloading large blocks of JSON makes things slow
• Every transaction broadcasted did so over its own unique and new HTTP connection

The data encoding issue has been something we've looked at for some time now. What we wanted is basically write out raw binary data which is as dense and optimized as possible, yet still be able to handle this new format somewhat easily and preferably portable so it could be used for instance with the https://www.npmjs.com/package/heat-sdk. We would be using such for:

• Microservices user defined messaging protocols
• P2P message encoding
• To encode high speed RPC messages

We decided to go with https://avro.apache.org/ which is the data serialization stack from http://hadoop.apache.org/ [HADOOP=The Apache™ Hadoop® project develops open-source software for reliable, scalable, distributed computing.]. A major factor to choose AVRO is its support for JavaScript and the fact it supports versioning of schemas. When developing microservices those aspects are extremely important and allow service operators to easily declare their custom binary protocols which can be shared and used in a decentralized way where service users only ever need the public schema (hosted perhaps on the blockchain - since even the schemas can be efficiently encoded to binary data) in order to communicate with a microservice.
 
To solve the slowness of sending each transaction over a new connection we've added a new channel to the existing websocket handlers. You can now open a websocket connection and leave it open, not needing to connect again you can now send binary transactions over the already open connection. For now this channel supports only one operation, to broadcast a single transaction, but we expect this RPC mechanism to grow to support more methods in the future.

And the cool thing is you can use this super fast transport from everywhere where there is Websocket support, this includes everywhere the HEAT-SDK can be used (browser, mobile, nodeJS).

Playing around, we've already seen huge speeds. On my PC alone we could already reach over 2000 transactions a second for the duration of 500,000 transactions. This was done with 4 websocket connections and after pre-generating and signing the 500,000 transactions. That particular node was forging but with no connected peers, for that to be possible we need to update the P2P layer still.

Currently we are working on completing the RPC and AVRO binary data support to HEAT-SDK so that all this can be used simply by getting HEAT-SDK.
When that is ready we'll be hosting the custom HEAT server somewhere and we'll release some in-browser samples that people could use to fire transactions at the HEAT instance. Would be fun to see what numbers it can do, I know you guys like breaking things. 

Thanks for sticking around! And let's make HEAT great again.

Windows and Linux desktop client, with server..

HEAT Wallet 2.3.0

These are the desktop clients with updated server embedded.

https://github.com/Heat-Ledger-Ltd/heatwallet/releases/tag/v2.3.0

Thanks! Much appreciated.

Heatledger 2.2.0

This is a mandatory update, all nodes on the network need to run this version
or higher.

Nodes below 2.1.0 will automatically be blacklisted with this version.

UPGRADE INSTRUCTIONS [upgrade only, new installations can ignore]

This version will perform a full rescan of the blockchain upon first use, this
happens automatically.

API Documentation

Please look here for API docs https://heatwallet.com/api

List of updates/fixes/features.

- fixes bug involving the last hard fork and the way expired orders are handled

Download here:

https://github.com/Heat-Ledger-Ltd/heatledger/releases/tag/v2.2.0

We are sorry for the inconvenience but the most recent hard fork introduced a bug which is now fixed.

This means https://heatwallet.com/#/explorer is back in business and fully functional.

We will be releasing a new server, version 2.2.0 which is a mandatory update.

The release of the new server version is expected in 1 hour to be precise at 23.00 CET.

We will update you here with the download location.

Desktop versions with the new 2.2.0 version embedded will follow at a later time.

We are currently running 2.2.0 on the company servers and are mining blocks initially with a company account, we will donate the forger rewards to a goal to be decided upon with the community. When enough forgers are mining blocks again on version 2.2.0 we will stop mining blocks with the company account.

The reason we needed the hard fork so badly was because of a bug.

It turned out through a not so obvious coding error that when you'd send HEAT to yourself it would incorrectly affect your forging stake. Not your spendable balance, only the amount of stake considered by the protocol.

What went wrong was that where a transaction comes in and is first processed two objects where instantiated, one for the sender account and one for the recipient account. These objects are passed on to lower level operations where depending on the transaction types these account objects are updated and these updates are recorded to a versioning mechanism.

What happened was that when the objects where first created some initial balance info was kept in object memory and when we first saved the sender object and later the recipient object, it turned out that updates to the first where not available to the second.

Solution was to check if sender and recipient are equal and simply instantiate a single object.

The fork was kindof difficult since we could not foresee if anyone would trigger this in between us releasing the update and the fork height. To deal with this at the fork height we had to make a scan of the blockchain (which luckily can do that in about a second) to search for offending transactions and correct the account stake for those accounts.

All in all the effect has been that just some of these payments have occurred and always with low amounts, the single large transfer was reported to us and we did the fork within the week.

-------------

Apart from this work has been progressing on finalizing the microservices, lets call it version 2. The main difference being that the usage has been greatly simplified, the api as published in heat-dev-kit will remain the same for the most part but some of the more difficult aspects are handled by the heat platform now and dont need any work on the part of the microservice author.

Its hard to describe this in a few words, but I'll try anyway..

Because of the nature of a decentralized consensus network, basically being decentralized, it can happen that the order of transactions differ at different points in the network. Since microservices are invoked through transactions a microservice had to be aware of this fact. Another aspect of a blockchain is that blocks get generated and accepted at some point, while at a later moment a better block could come along which required the old one to be undone and the new one to be applied, this also had to be handled by your microservice.

These two things are handled now by a transaction cache solution which is at the heat core, this cache ensures transactions are reported to your microservice only once and in case the order of transactions changes only the transaction that has shifted position is reported to your microservice.

The cache is also persisted to disk so that when you redownload the blockchain or rescan it, your microservices are not invoked again.

Off to slush now for the second day in a snowy Helsinki.

DESKTOP WALLET LINUX + WINDOWS

HEAT WALLET 2.2.1

Wallet/desktop specific changes:

• Fixed Join Slack link
• Orderbook alignment adjusted
• Fixed history entries display bug

https://github.com/Heat-Ledger-Ltd/heatwallet/releases/tag/v2.2.1

So sorry, but experimental mac desktop is giving some issues to build. We'll update as soon as we can.

BUG FIX MANDATORY UPDATE

Heatledger 2.1.0

This is a mandatory update, all nodes on the network need to run this version
or higher.

A hard fork will happen at block 870,000 at which we will update the block
version number, after that block older versions will not accept blocks from the
new fork and vice versa.

Nodes below 2.0.0 will automatically be blacklisted with this version.

Installation

To install and run heatledger server you need Java JDK 1.8 or higher installed, note that JDK is different from standard java distributions.

On ubuntu:

sudo apt-get install default-jdk

For other platforms look here

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

UPGRADE INSTRUCTIONS [upgrade only, new installations can ignore]

This version will perform a full rescan of the blockchain upon first use, this
happens automatically.

API Documentation

Please look here for API docs https://heatwallet.com/api

List of updates/fixes/features.

• one critical and two lesser bugs are fixed with this release. details of which will be released when the fork is complete.
• the custom memory mapped storage maps for storing things like account balances or block ids are reduced in size, this saves several Gigabytes of storage needed in the blockchain folder
• in case storage maps grow too big they will now automatically be resized

https://github.com/Heat-Ledger-Ltd/heatledger/releases/tag/v2.1.0

Did you take care of installing the correct Java version?

Reward always goes to account that forges a block. By leasing balances to an account you increase its chances of forging a block.


You dont need permission to lease to an account


You should lease to your own account or in case it is not your account you should be compensated for leasing your balance.

Experimental fully-functional MAC OSX desktop wallet.


We have an experimental MAC OS build now!! Its fully functional including possibility of mining blocks.
We are in the process of obtaining official developer status and can thus not yet publish the wallet to the app store.

Please make sure that in order to run the HEAT server you need Java SDK version installed on your Mac.

To install Java SDK on your Mac look here http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

No doubt in newer versions of OSX (which I dont have currently) there will be some warnings about the wallet not being officially signed, as said thats because we are still in the process of signing up in Apple's developer program.

Please report your findings and let us know if it works for you!

Thanks



Download MAC OSX Wallet here:
http s://github.com/Heat-Ledger-Ltd/heatwallet/releases/download/v2.2.0/Heatwallet_MacOS_2.2.0.EXPERIMENTAL.zip

Update - this time with working Cut/Copy/Paste
https://github.com/Heat-Ledger-Ltd/heatwallet/releases/download/v2.2.0/Heatwallet_MacOS_2.2.0.EXPERIMENTAL.CONTEXT-MENU-FIXED.zip

heat-sdk version 0.5.0 with WebSocket/Push support

heat-sdk version 0.5.0 is ready for use in your browser and on your server.

Now with native and managed WebSocket support it becomes easy to build your real-time connected blockchain apps. In the browser and on mobile heat-sdk will fall back on native HTML5 WebSocket support available in modern browsers. When you build on the we've opted for the nodejs cross OS `ws` package.

Any way, whether you are developing for mobile or the server know that our websocket implementation ensures you are always connected and when you ever got disconnected from our or your own HEAT servers. Heat-sdk will make sure your connection is restored while transparently managing all your previous subscriptions with the server. Basically re-subscribing your application so that your application logic wont even know you got disconnected.

Samples on using heat-sdk 0.5.0 with websocket support
^{Click the button to load the sample. You can edit the code and play around in the editor on the left}



https://www.npmjs.com/package/heat-sdk

Max is 300,000 blocks. But after one block you can issue a next lease which can again be 300,000 blocks and starts the moment the first lease ends.
With heat-sdk you can create the lease transaction offline, sign it and broadcast it. (but this is through code, just a few lines of javascript, that you could run locally in your browser)
Through heat wallet you can do the same by opening the Lease Balance dialog.

We where wondering that ourselves. So we made this..

When is block 777,777 ?

Heat-sdk runs in your browser.

heat-sdk runs natively in your server environment as well as on your mobile and in your browser.

Please checkout another demo but now running in a browser.
Feel free to play around with the code, either click fork or just edit the code in the left hand window, it will immediately run in the preview window on the right.

Online Sample/Demo

Good progress has been made with heat-sdk!

We are at version 0.3.3 now and counting.
Heat-sdk is an essential building block much needed in order to built upon the amazing HEAT core.
With heat-sdk we bring the full power of the decentralized HEAT blockchain to the three most used computing platforms in the world!

Heat-sdk runs natively and can do all you would ever need when working with HEAT on:

   1. Mobile phones
   2. Web browsers
   3. Web servers (in the form of NodeJS)

While still under heavy development you can use heat-sdk today!

Check out https://www.npmjs.com/package/heat-sdk and grab a copy for yourself, or if you feel really adventurous go and try out one of our growing list of samples that run in a virtual server environment and can be edited straight from your browser.

package home | https://www.npmjs.com/package/heat-sdk
source code | https://github.com/Heat-Ledger-Ltd/heat-sdk
documentation | https://heat-ledger-ltd.github.io/heat-sdk/

Run these samples in your browser.
^{These are just some samples we created today - for further samples see our test suite https://github.com/Heat-Ledger-Ltd/heat-sdk/tree/master/test}





Heat-sdk will play a major role in the long awaited HEAT microservices architecture, so stay tuned ..  

We are working on that.
An IMac for exactly this purpose was purchased recently and we are working on expanding our target platforms.
Unfortunately until that time if you want to use HEAT on your Mac we recommend the webwallet or if you want to run a node you would have to use the 2.0.0 HEAT server.

Meanwhile over at Slack:


So that means LOTS AND LOTS of 1.1.0 nodes folks.
Remember:

When block 777,777 comes and knocks ... you won't be getting da blocks!

So make sure you update to 2.0.0 - https://github.com/Heat-Ledger-Ltd/heatledger/releases

HARDFORK AT BLOCK 777,777 (~ october 22'nd)

Heatledger 2.0.0

This is a mandatory update, all nodes on the network need to run this version
or higher.

A hard fork will happen at block 777,777 which is approximately two weeks from
the moment we do this release (somewhere around 2017-10-22).

UPGRADE FROM 1.1.0

The following applies **ONLY** if you run in 'non-light' mode. (when you've added `heat.replicatorEnabled=true` to your configuration).

While you can re-use your existing `blockchain` which can be found in the `blockchain` directory in your previous 1.1.0 installation.

You now need to DELETE THE DIRECTORY CALLED `heat_db` before starting heat 2.0.0, doing this will force HEAT to rescan the blockchain and rebuild its replicator database model.

API Documentation

Please look here for API docs https://heatwallet.com/api

Balance Leasing

By sending a LEASE transaction you transfer your forging power (so thats your
effective balance or stake) to a different account.

The amount you lease is always your full stake at any given moment, you also
provide a period expressed in blocks after which the lease ends. The period has
to be between 1,440 and 300,000 blocks. 

While your stake is leased you can still move your HEAT to another account or
receive HEAT in your account. The same rules that applied before for how to
calculate your effective balance apply now. Whenever you add HEAT to your account
those funds will add up to your leased balance after 1440 blocks. At the same
time whenever you transfer HEAT out of your account that will immediately be
deducted from your leased stake.

An account to which balances are leased can start forging as usual, when determining
the account stake during block creation and broadcasting we add up all leased
stakes to come to that account current forging power.

After sending a LEASE transaction and before that lease period is over you can
send any number of follow up LEASE transactions each one overwriting your 'next lease'
which takes effect automatically the moment your first lease ends.

Use case for balance leasing is two fold.

    1. It allows accounts with small stakes to group together to form forging
       pools. This way increasing your chance of forging blocks.
    2. It's a security measure where you lease your (large) stake to a proxy
       account. Meaning you no longer have to expose your secret phrase of
       your main account to the server you use for foring blocks.

As is the case with forging blocks without leasing your balance, we require a minimum of
1,000 HEAT in order for your stake to be counted as leased stake.
       
List of updates/fixes/features.

- fixed placeAsk, placeBid unknown asset bug
- replicator model schemas now support MYSQL
- new replicator schema version manager based on external property files, please
  see 'resources' directory for all sql schemas
- server side order book running total and sum calculations
- changed the default logger to daily rollovers, logs are archived for 30 days
  and automatically deleted
- forger rewards are traced by a new replictor
- forger rewards can be returned by the API
- order cancellations are now traced by a new replicator, now the UI can display
  order cancellation details including asset, currency, quantity and price
- blocks generated per account are now traced in a new replicator and are availble
  in the UI and from the API
- improvements/optimizations of replicator SQL and where needed combining SQL
  based solutions with custom caching solutions. this is an ongoing process,
  major updates to the way we deal with our rather unique replicator mechanism
  are to be expected
- thread safety improvements

When the hard fork happens only servers running this version or higher will be
able to connect.

HEAT SERVER 2.0.0

https://github.com/Heat-Ledger-Ltd/heatledger/releases/tag/v2.0.0

HEAT WALLET 2.2.0 [DESKTOP WITH SERVER]

https://github.com/Heat-Ledger-Ltd/heatwallet/releases/tag/v2.2.0

HEAT CLIENT 2.2.0 [DESKTOP LIGHT VERSION - NO SERVER]

https://github.com/Heat-Ledger-Ltd/heat-ui/releases/tag/v2.2.0