[ANN] SuperNET NXT asset 12071612744977229797, SUPERNET KMD assetchain in summer

[jl777]

I have some dates!

We will be doing a pre-release stress test and aiming for release on Friday. Assuming we are not finding any showstopper bugs. The release should be viewed as a beta release, so it is for a more general feedback for compatibility, usability, etc.

keep in mind this is just the first version and not all the advanced tech is made available via GUI. To access the more advanced tech, you would need to be using something not so easy, as it is via JSON API. But this is the nature of this as first there must be the advanced functions via API, then a GUI that is working but not the most easy to use, then finally a nice looking intuitive GUI that needs no documentation.

I am currently working on finalizing InstantDEX API and we are pushing for an initial GUI release in a week. This is independent from the release above, which is making easy to use multiwallet. And there are also some features available in the v0 GUI that is there, but not in the v1 series.

I hope this is not too confusing, but it is the best process that we could come up with to manage the nature of SuperNET. Some are not sure if the 60,000+ lines of code in https://github.com/jl777/libjl777/ is making any sense. What the v1 GUI is doing is making about 10,000 of these lines easily accessible. To put it in perspective, I wrote ramchains from scratch starting the new years. I ported the old MGW into realtime MGW, total line count around 10,000. So to go from concept to coded to debugged to production, in this timeframe it is a demonstration. The full SuperNET is much bigger, so I wanted an easy well defined project I can finish quickly.

I hope that this ramchains release puts to ease everybody who is wondering if jl777 can actually write code that works. It is doing a complete blockchain rescan and richlist calculation for BTCD in about 30 milliseconds. But it doesnt even have to do the recalc as it is maintaining in memory current account balances, so all that is needed is a sort, which takes around 1 millisecond.

Now how long will this all take to be finished? This is like asking when linux will be finished. So let us ask a more specific question, when will InstantDEX be finished? I hope to get this done by the end of the month so people can have a much safer place to trade than central exchanges. Next week for test releases and we can get things done faster the more people are testing. This is the standalone InstantDEX, I have no idea the time for it to be finished in the v0 or v1 series GUI.

I want to get InstantDEX prioritized due to the situation with half a dozen exchanges being attacked and not even clear how many will survive. so this will shuffle the schedule a bit, but I think we can all agree it is worth the tradeoff. So I will be concentrating on gettting InstantDEX to be usable for large scale crypto trading. This will include Tradebots, at least default ones with standard functionality. A more advanced tradebot profit sharing will require some website and GUI things.

In parallel, still waiting for some design reviews on the privacy side of the project. I have received invaluable feedback from Come-from-Beyond and we licensed some tech that will really enhance the security of the SuperNET. This will require some time to properly integrate into the codebase, I am estimating about a month and then realistically another month to properly test it. Before all that it will still be possible to use the privacy features, with the caveat that it is not fully validated yet.

So we have the teleport, telepathy and enhanced security all converging to a release in a couple months, but these features will be testable way before it is in the final form.

At this point, I will only have the PrivateBet tech left to complete, and it is possible I can squeeze that in during the privacy improvements. Now I can write a lot of code and I estimate it will end up around 80,000 to 100,000 lines by the time these things are all done. However, I am not the only one coding. There are literally a dozen projects going on and I am not aware of the detailed status on these, but to facilitate others to help with coding, I have added the ability for non-C modules to be dynamically loaded and extend the SuperNET API. Pangea is written in python, so that was the trigger for this, but while adding support for that I realized we can support virtually any other language, so as we get devs who work in a particular language we can extend the SuperNET to support running single instance invocations or even as daemons.

Unfortunately, every person that is taking the task to clearly document all this, well their head seems to explode...
And I must allocate my time to coding and helping the other projects, so I just dont have the time to be making regular updates.

That being said, if there are any specific questions, then I will do my best to answer them.

James

tl:dr releases are happening, tech is being completed, wont happen all at once but it will happen

[Cassius]

 
Thank you Mr. 777.

[HarriHirsch]

Thanks James,

great to hear about the Progress.

Btw, I love the asset dividends we all got...

[jl777]

Thanks James,

great to hear about the Progress.

Btw, I love the asset dividends we all got...

there are more in the pipeline, but it is not such a good idea to release it during bear market. So I am holding them in reserve for now

James

[raimch89]

sir,
have you information about supernet and btcd coins blocked on bter?

[jl777]

sir,
have you information about supernet and btcd coins blocked on bter?

i heard that in a week they will allow for withdraws, but this is third hand from chinese twitter site or something like that.
we do see all the SuperNET assets still in the bter account, so it is still there.
I dont know for sure but I would be surprised if the BTCD isnt still there.

only altcoins like XCP that are linked to BTC address appear to be at risk

James

[jl777]

since nobody complained about the tech details confusing them, I will do a daily progress.

   static char *allorderbooks[] = { (char *)allorderbooks_func, "allorderbooks", "V", 0 };
    static char *orderbook[] = { (char *)orderbook_func, "orderbook", "V", "baseid", "relid", "allfields", "oldest", 0 };
    static char *placebid[] = { (char *)placebid_func, "placebid", "V", "baseid", "relid", "volume", "price", 0 };
    static char *placeask[] = { (char *)placeask_func, "placeask", "V", "baseid", "relid", "volume", "price",0 };
    static char *bid[] = { (char *)bid_func, "bid", "V", "baseid", "relid", "volume", "price", "baseamount", "relamount", 0 };
    static char *ask[] = { (char *)ask_func, "ask", "V", "baseid", "relid", "volume", "price", "baseamount", "relamount", 0 };

All the above InstantDEX api calls are spot tested and released into testing.

orderbooks are created dynamically when placebid or placeask is called. this is then sent to the network as bid or ask API calls with the price/volume pair changed to base and rel satoshi amounts for the assets.

allorderbooks returns an array of, yes you guess it!, all the orderbooks

orderbook returns, yes!, an orderbook with base -> rel. So the base NXT, rel BTC orderbook is NXT/BTC
but you can ask for it with base BTC and rel NXT and it will return it with the prices as BTC/NXT

a bunch of gory details regarding asset decimal places and other mundane things, but in my tests it is propagating pretty fast to other nodes. still need to get more timing results to know if I need to optimize it more.

now that the orderbooks are back online, next up is ordermatching. This is pretty tricky as I need to make it do an atomic swap of any two assets, with NXT itself treated as a special case asset. Good thing this was done last summer, but still need to get it ported into the new codebase and get it debugged.

I remember it was quite touchy, so I will plan for just achieving this automated orderfilling triggered with a makeoffer API. it actually needs at least two more internal ones to match the state transitions.

At that point, the low level InstantDEX API would basically be done for asset<->asset and other than supporting the GUI port to use this, I dont foresee too much more at this level. However, this is only the lowest level. After this step I need to add another layer for the tradebots. But I like to just do one step at a time as often when I do that next step I can see a bit better what is best to do next

James

[Cassius]

sir,
have you information about supernet and btcd coins blocked on bter?

i heard that in a week they will allow for withdraws, but this is third hand from chinese twitter site or something like that.
we do see all the SuperNET assets still in the bter account, so it is still there.
I dont know for sure but I would be surprised if the BTCD isnt still there.

only altcoins like XCP that are linked to BTC address appear to be at risk

James

Thanks for the coding update!

Latest I've seen is after Chinese new year, which is 19th February (tomorrow).
So we'll see: who will be the first to release, SuperNET or BTER?

EDIT: depending on how you interpret it, that could mean new year itself or the new year holiday, which could last a full week, so it would be sometime after 26th.

[raimch89]

thanks james for rapid answer
don't understand anything in coding but rare to see dev giving such details.
i have faith in btcd and supernet project.

[jl777]

thanks james for rapid answer
don't understand anything in coding but rare to see dev giving such details.
i have faith in btcd and supernet project.

I used to do it all the time, but then a one guy started complaining about it. saying it was a form of spam...
then several others all agreed with the spam point of view

needless to say I didnt feel like spending time to communicate and then be insulted for it
i do have quite a lot to get done

James

[jl777]

The InstantDEX core API is shaping up pretty good:

    static char *allorderbooks[] = { (char *)allorderbooks_func, "allorderbooks", "V", 0 };
    static char *openorders[] = { (char *)openorders_func, "openorders", "V", 0 };
    static char *orderbook[] = { (char *)orderbook_func, "orderbook", "V", "baseid", "relid", "allfields", "oldest", "subscribe", 0 };
    static char *placebid[] = { (char *)placebid_func, "placebid", "V", "baseid", "relid", "volume", "price", 0 };
    static char *placeask[] = { (char *)placeask_func, "placeask", "V", "baseid", "relid", "volume", "price", 0 };
    static char *bid[] = { (char *)bid_func, "bid", "V", "baseid", "relid", "volume", "price", "baseamount", "relamount", "type", 0 };
    static char *ask[] = { (char *)ask_func, "ask", "V", "baseid", "relid", "volume", "price", "baseamount", "relamount", "type", 0 };
    static char *makeoffer[] = { (char *)makeoffer_func, "makeoffer", "V", "baseid", "relid", "baseamount", "relamount", "other", "type", 0 };

All but the last one is tested and appears to work in most cases (got stuck for some hours today to get all permutations correct). Suffice it to say it is a lot more complicated than it looks to handle all permutations of orderbooks while also accepting new orders in either polarity. Everything is looking the same, but a bit different and any mistake is getting the inverse of the price, or the volume of the other coin, so we need to really test all possibilities.

Good news is that there are only around 16 possible cases, so once they are tested it will work for all asset pairs in all polarities.

The last API call, makeoffer is how InstantDEX will allow for instant trading. But before this could work, I needed to know all the open orders a node has, so I made the openorders API call, which uncovered the polarity confusions.

Anyway, I had the makeoffer sequence done last July! so just have to port it into the current codebase and make an improvement to eliminate an obscure security hole. But we can put it into testing without that part done, so I am working toward a nofrills order matching (fill or kill) without much error handling so we can test a full cycle with the GUI.

Then as I am fixing any bugs, I will bulletproof things and add nice things like the ability to query current orderbooks remotely and create some monetization possibilities for MMatrix nodes.

It looks like I should be able to get the InstantDEX core pretty much done in the next day or two, so there will be time to start making some automated market makers and other support things. The GUI team says they are on track, so if all goes well, we will be able to test a decentralized exchange that looks and feels like a centralized one.

I know some of you are not 100% comfortable with MGW's distributed setup, especially with recent events. After I get the basic InstantDEX up and running and into testing, I will make some tradebots that will do the "deposit, trade, withdraw" that we all recommend people to do, but sometimes are forgetting to do, well this will be automated. So your funds will be in MGW just for the duration of the trade.

MGW will do this at its cost to cover txfees and the servers. I dont want there to be reasons for people to not do the "deposit, trade, withdraw" over fees. Safety first! I also have additional methods to boost security which will become a necessity when InstantDEX is reaching 100 trades per minute, as that will pretty much fill up all the NXT blocks!

I also want to make sure that the key trading pairs are liquid and have low spreads. We need a safe place to trade crypto that isnt charging big fees. InstantDEX will be this. At first not so many trading pairs will be supported, but it is structured in a way that adding new ones is pretty easy.

The NXT AE is very nice, but it only trades against NXT, it costs a txfee for each bid/ask and even for cancelling them and of course there is the occasional long time between blocks usually just as you really want to do a trade. InstantDEX solves all of these, there wont be any fees for bids and asks and trading can be against NXT or any asset. During the beta test there wont even be any trading fees other than the txfees and even when we activate the fees it will be 1/1024'th of the trade.

My goal is to make InstantDEX the lowest cost trading platform that is also the safest.

James

[ActualUpsurge]

Thanks James keep up the good work


Regards,
ActualUpsurge

[jl777]

A bug has been found with the realtime MGW. So I need to get this fixed. the bug is having many withdraws at once slows/clogs up withdraws for that coin. So wont have much time for progress on InstantDEX today as solving this wont go quickly

James

[RichardT]

Considering the size and depth of the SuperNET project, perhaps a more realistic release date is summer 2015

[jl777]

Considering the size and depth of the SuperNET project, perhaps a more realistic release date is summer 2015

There will certainly be parts of SuperNET that wont be ready until the summer and I expect that we will continue to come up with new projects, so there will be some things that wont be done until after summer too.

But this is why I wanted to get a release structure so we can release new things as they become ready.

With the recent coordinated attacks on the centralized exchanges I am prioritizing getting InstantDEX fully done ahead of the rest, as the crypto community needs a safe place to trade.

James

[jl777]

Finally was able to get back to InstantDEX. Some orderbook confusions fixed.

I want to support display of the orderbook in either direction, eg. NXT/BTC or BTC/NXT based on user preference. Sounds easy, but to do this without any redundant storage is a bit tricky and I had the wrong dataset for while, which just confused me

Now it is rewritten it is much less code and much fewer cases to deal with, so I think it will work pretty well and it is also now setup for virtual orderbooks.

One strange parsing crash, but this has probably been there for a while as not many things are using extensive floating point inputs.

Now to port the atomic swap

James

[jl777]

cryptosleuth found the crashing bug! it was an off by one in memory allocation in a totally separate module. Very nice to get some help for this as the crash was only for floating point numbers, but it turned out to be in a totally different area. just using floating point conversions moved things around in the memory so the overwrite of 0 ended up causing a crash on some computers in some cases.

I also  had to verify orderbooks for remote nodes, I think I got all the bugs fixed, still needs to be verified

James

[jl777]

The first version will require fill or kill orders, meaning an incoming order has to match exactly a bid (or ask) that was published. I changed the orderbook API to show the exact "makeoffer" API that is needed to fill a specify entry in the orderbook.

So, display orderbook, find one you like, user the premade makeoffer API and send that.

All the above works and the makeoffer in turn issues a processutx command with the appropriate raw tx bytes:

{
        "fullHash":     "fa063515a7f00288bf6e6a538165df70387c1239dfb069aa443fe8887e2c1aab",
        "requestProcessingTime":        3,
        "signatureHash":        "4a3798451193f9ae3b23dd65b87f60bf78f440a9e39d89e38011dcbb1cdd37c0",
        "transactionBytes":     "0211f3665602d002ec7f665fccae39025531b1cb3c48e584916dba00a7034edc60f9e4111f86145 d048b3c6e60cbe672000000000000000000e1f50500000000000000000000000000000000000000 00000000000000000000000000000000004d8d909417f77f8014fe107b53162e8be913c30cbc82a 74468fc35343ee87c0cbb56d34d524ce9b291d50c4ffc850ef559cbe61a6954b676aeb5fbf6616b b53100000000fd8b0500fdeb9de0ca6441be01b7faf17b381380997f27000000000000",
        "transaction":  "9800660339517622010", 
        "transactionJSON":      {
                "fullHash":     "fa063515a7f00288bf6e6a538165df70387c1239dfb069aa443fe8887e2c1aab",
                "signatureHash":        "4a3798451193f9ae3b23dd65b87f60bf78f440a9e39d89e38011dcbb1cdd37c0",
                "transaction":  "9800660339517622010",
                "amountNQT":    "0",
                "ecBlockHeight":        363517,
                "attachment":   {
                        "asset":        "11060861818140490423", 
                        "quantityQNT":  "10111",
                        "version.AssetTransfer":        1
                },
                "recipientRS":  "NXT-T4S6-E3R5-9QQT-9DQT7",
                "type": 2,
                "feeNQT":       "100000000",
                "recipient":    "8279528579993996036",
                "version":      1,
                "sender":       "12240549928875772593",
                "timestamp":    39216883,
                "ecBlockId":    "13709349563207117821",
                "height":       2147483647,
                "subtype":      1, 
                "senderPublicKey":      "ec7f665fccae39025531b1cb3c48e584916dba00a7034edc60f9e4111f86145d",
                "deadline":     720,
                "senderRS":     "NXT-7LPK-BUH3-6SCV-CDTRM",
                "signature":    "4d8d909417f77f8014fe107b53162e8be913c30cbc82a74468fc35343ee87c0cbb56d34d524ce9b 291d50c4ffc850ef559cbe61a6954b676aeb5fbf6616bb531"
        },
        "broadcasted":  false, 
        "unsignedTransactionBytes":     "0211f3665602d002ec7f665fccae39025531b1cb3c48e584916dba00a7034edc60f9e4111f86145 d048b3c6e60cbe672000000000000000000e1f50500000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000000000000000000 000000000000fd8b0500fdeb9de0ca6441be01b7faf17b381380997f27000000000000" 
}


The other node, even gets it!

PARSED OFFER.({"requestProcessingTime":0,"amountNQT":"0","verify":false,"ecBlockHeight":363517,"recipientRS":"NXT-T4S6-E3R5-9QQT-9DQT7","attachment":{"asset":"11060861818140490423","quantityQNT":"10111","version.AssetTransfer":1},"type":2,"feeNQT":"100000000","recipient":"8279528579993996036","version":1,"timestamp":39216883,"sender":"12240549928875772593","ecBlockId":"13709349563207117821","height":2147483647,"subtype":1,"senderPublicKey":"ec7f665fccae39025531b1cb3c48e584916dba00a7034edc60f9e4111f86145d","deadline":720,"senderRS":"NXT-7LPK-BUH3-6SCV-CDTRM"}) full.(fa063515a7f00288bf6e6a538165df70387c1239dfb069aa443fe8887e2c1aab) (fa063515a7f00288bf6e6a538165df70387c1239dfb069aa443fe8887e2c1aab)

But here the process stops. I need to make sure the accounts have the assets they are trading. But i need to also add verification that this is an order we want to do, and also that it is not already in progress. I also see a problem where orders dont expire in 5 minutes like they should.

So progress, but not quite there yet. Once these issues are fixed and the other node responds back, theoretically it will do a swap of the two assets.

James

[NiGuEnEd]

Any updates about BTER ?

[XCASH]

Any updates about BTER ?

This is the latest tweet. There is still no specific time given for anything.

https://twitter.com/btercom/status/568944687257530368