Qora | 100% POS | Assets | Names | Voting | Open Source

[calvinstm]

the qora wiki @ https://wiki.qora.org/ isn't working?

Qora had a forum and a wiki set up by the dev, but as far as I'm aware, they're both down at the moment..

However, most info in the Wiki can be found in the API guide.. If there is something you need to know, just post them here!

Another good source of info is Qora's source code at github.. The source is pretty organized.. Check it out if you haven't!

-----------------------------------

A little bit of intro to Qora's API --

There are two ways to access Qora's API --

1. Through the built-in debug console.
2. Through jetty's html (JSON) interface using your browser or make calls from inside your applications...

 

[1715226760]

1715226760

[1715226760]

1715226760

[1715226760]

1715226760

[wizzardTim]

I am asking because i have googled both errors (ClosedByInterruptException, ClosedChannelException) together with the term MapDB and i have found several websites
e.g.:
https://groups.google.com/forum/#!topic/mapdb/9MBNRFbX3g4
https://github.com/jankotek/MapDB/issues/352

 that claim memory mapped files (mmapFileEnable()) might solve both problems, but they only work on 64 bit VMs.
So i looked into the sourcecode relase and found:
.mmapFileEnableIfSupported()
in https://github.com/Qora/Qora/blob/master/Qora/src/database/DBSet.java

Then i looked at the decompiled v18 but didn't find it there.

mmapFileEnable() - Enables Memory Mapped Files, much faster storage option. However on 32bit JVM this mode could corrupt your DB thanks to 4GB memory addressing limit.

You genius!    I think you actually solved the mystery!

Very good job, DeMap!  

PS: I've been running Qora on 64-bit windows (64-bit JVM), and I can attest that my Qora client never once encountered any problem (No exception thrown)... I'm pretty sure it's a JVM problem.

----------------

PS 2: If this is the case, what Poloniex needs to do is to compile a new Qora wallet with memory map file always enabled, and switch to 64-bit JVM.  I think someone needs to contact Poloniex or Tristen and let them know about the solution.

Thank you DeMap!  It is very important for qora to have poloniex up and running.

I tried to give some ideas in the slackroom, regarding the ways to reproduce the scenario and help find the bug, but if this is the bug, you saved  all a lot of time for the community.

Contrats!

[rlh]

I think this is the bug.  I vaguely recall Qora mentioning that the issue (or multiple issues) were related to MapDb, and that he needed to refactor some code so that hard-kills of the app wouldn't break the db.

[delulo]

the qora wiki @ https://wiki.qora.org/ isn't working?

Qora had a forum and a wiki set up by the dev, but as far as I'm aware, they're both down at the moment..

However, most info in the Wiki can be found in the API guide.. If there is something you need to know, just post them here!

Another good source of info is Qora's source code at github.. The source is pretty organized.. Check it out if you haven't!

-----------------------------------

A little bit of intro to Qora's API --

There are two ways to access Qora's API --

1. Through the built-in debug console.
2. Through jetty's html (JSON) interface using your browser or make calls from inside your applications...

 

I'd like to know about the mechanics / incentives structures of the Qora POS. Any documentation about it?

[Vrontis]

I'd like to know about the mechanics / incentives structures of the Qora POS. Any documentation about it?

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

Source

[Vrontis]

I am asking because i have googled both errors (ClosedByInterruptException, ClosedChannelException) together with the term MapDB and i have found several websites
e.g.:
https://groups.google.com/forum/#!topic/mapdb/9MBNRFbX3g4
https://github.com/jankotek/MapDB/issues/352

 that claim memory mapped files (mmapFileEnable()) might solve both problems, but they only work on 64 bit VMs.
So i looked into the sourcecode relase and found:
.mmapFileEnableIfSupported()
in https://github.com/Qora/Qora/blob/master/Qora/src/database/DBSet.java

Then i looked at the decompiled v18 but didn't find it there.

mmapFileEnable() - Enables Memory Mapped Files, much faster storage option. However on 32bit JVM this mode could corrupt your DB thanks to 4GB memory addressing limit.

You genius!    I think you actually solved the mystery!

Very good job, DeMap!  

PS: I've been running Qora on 64-bit windows (64-bit JVM), and I can attest that my Qora client never once encountered any problem (No exception thrown)... I'm pretty sure it's a JVM problem.

----------------

PS 2: If this is the case, what Poloniex needs to do is to compile a new Qora wallet with memory map file always enabled, and switch to 64-bit JVM.  I think someone needs to contact Poloniex or Tristen and let them know about the solution.

Indeed, this seems to be a workaround that solves this problem

https://groups.google.com/d/msg/mapdb/9MBNRFbX3g4/otD5fYEQBM0J

ClosedChannelException happens if one of the threads is interrupted while doing IO. The FileChannel gets closed on interruption and all future IOs will throw CCE. There is workaround to use memory-mapped files (mmapFileEnable()), but they only work on 64bit systems.
 
So either enable mmap or investigate what is interrupting your threads.

https://github.com/jankotek/MapDB/issues/352#issuecomment-48128816

Some threads are interrupted while reading/writing from FileChannel. As result FC is closed and always throws ClosedByInterruptException. That is documented behavior of FileChannel, and there is not much I can do without sacrificing performance.

So my suggestion is to stop interrupting threads while they are reading/writing.

Other solution is to use memory mapped files (mmapFileEnable()). Perhaps they are not affected by this

https://github.com/jankotek/MapDB/issues/384#issuecomment-68485183

I would recommend to use mmap files as
alternative to prevent this.

Also make sure that no thread is being
interrupted while doing IO. That would close
FileChannel

mmapFileEnable

public DBMakerT mmapFileEnable()
Enables Memory Mapped Files, much faster storage option. However on 32bit JVM this mode could corrupt your DB thanks to 4GB memory addressing limit.

You may experience java.lang.OutOfMemoryError: Map failed exception on 32bit JVM, if you enable this mode.

mmapFileEnableIfSupported

public DBMakerT mmapFileEnableIfSupported()
Enable Memory Mapped Files only if current JVM supports it (is 64bit)

I have also asked MapDB founder about that and he told me to check MapDB Code for thread interrupts.

Also a new version of MapDB has been released

[delulo]

I'd like to know about the mechanics / incentives structures of the Qora POS. Any documentation about it?

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

Source

That is a little little
Is there not more information?

[Vrontis]

I'd like to know about the mechanics / incentives structures of the Qora POS. Any documentation about it?

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

Source

That is a little little
Is there not more information?

Below are quoted all that I have found from the dev about POS

Qora uses a new proof-of-stake algorithm making it future-proof.

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

- The proof of stake algorithm uses a combination of randomness, your generating balance and the time since the last block to determine if a new block is valid. Currently there is no protection against 51% attacks. I will create a detailed explanation of how the system works.

To generate blocks you will need to unlock your wallet. the POS relies on the fact that the client has access to a somewhat reliable time source. Basicly it uses time but does not have to rely on a centralized time server to work. Currently it still uses NTP but later on this will be changed so it works completely decentralized.

Also a short review from a core dev of NXT

Spent a little bit of my time on Qora sources during last weekend and even made Scala version of QORA's BlockGenerator class: https://gist.github.com/kushti/605dfe4f17d0dd75cedf (220 lines of code instead of original 400), also submitted the bug on the way: https://github.com/Qora/Qora/issues/10 . Haven't investigated forging algo in details, but that's definitely not TF. Read this epic topic about TF https://nxtforum.org/general/nxt-forging-algorithm-simulating-approach/

Qora is used on the Forging Simulation Tool created for the Consensus research.

[delulo]

I'd like to know about the mechanics / incentives structures of the Qora POS. Any documentation about it?

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

Source

That is a little little
Is there not more information?

Below are quoted all that I have found from the dev about POS

Qora uses a new proof-of-stake algorithm making it future-proof.

A Proof-of-Stake algorithm that is different from anything that is described here https://en.bitcoin.it/wiki/Proof_of_Stake.

The algorithm will allow accounts to start generating blocks after only 10 confirmations.
Each account will also know at exactly which point they will generate the next block.

- The proof of stake algorithm uses a combination of randomness, your generating balance and the time since the last block to determine if a new block is valid. Currently there is no protection against 51% attacks. I will create a detailed explanation of how the system works.

To generate blocks you will need to unlock your wallet. the POS relies on the fact that the client has access to a somewhat reliable time source. Basicly it uses time but does not have to rely on a centralized time server to work. Currently it still uses NTP but later on this will be changed so it works completely decentralized.

Also a short review from a core dev of NXT

Spent a little bit of my time on Qora sources during last weekend and even made Scala version of QORA's BlockGenerator class: https://gist.github.com/kushti/605dfe4f17d0dd75cedf (220 lines of code instead of original 400), also submitted the bug on the way: https://github.com/Qora/Qora/issues/10 . Haven't investigated forging algo in details, but that's definitely not TF. Read this epic topic about TF https://nxtforum.org/general/nxt-forging-algorithm-simulating-approach/

Qora is used on the Forging Simulation Tool created for the Consensus research.

Seem like you got everything out you could Thanks for the info!

[Vrontis]

Seem like you got everything out you could Thanks for the info!

I am not able to read and understand the POS from the source code.
If anyone is capable enough it would be great to have a review.

[calvinstm]

Seem like you got everything out you could Thanks for the info!

Sorry, but currently there is no documentation for Qora's forging algorithm.  However, like what Vrontis said, the best place to start is Qora's Block Generator (BlockGenerator.java).  Half way through the code, you will see how the new block is generated.    I think someone should definitely write a paper on that! 

[calvinstm]

I have also asked MapDB founder about that and he told me to check MapDB Code for thread interrupts.

Also a new version of MapDB has been released

Nice job! Thanks for doing this! 


Okay, I just wanna quickly summarize and tell you what we need to do here:

1. We need to come up with a new workaround that does NOT involve the use of Memory Mapped Files since Memory Mapped Files only works on 64-bit system.  Using MMF is not an option since we need qora to work on all systems including 32-bit.  We need to find out what's causing the thread to get interrupted before all file I/O operations are completed.

rlh's comment got me thinking, and I think that it's possible the client does not exit cleanly (FC was closed while the changes to database were not fully committed).  Since I've only run Qora on 64-bit machines, it's difficult for me to replicate Poloniex's problems and do a debug.  Someone definitely needs to follow up on this and identify which calls are causing the problems.

2.  I hope we can incorporate the following changes before we roll out the next version of Qora wallet - 1) Skerberus' Installer, 2) MMF workaround (for 32-bit system' sake), 3) the new Wallet design Pondsea was working on, 4) encoding method as suggested by deMap.

Keep up the good work, guys! 

[Tuck Fheman]

FWIW, if anyone's selling I know of a few lurkers looking to buy right now. Post here and I'm sure they'll respond.

[PondSea]

I have also asked MapDB founder about that and he told me to check MapDB Code for thread interrupts.

Also a new version of MapDB has been released

Nice job! Thanks for doing this! 


Okay, I just wanna quickly summarize and tell you what we need to do here:

1. We need to come up with a new workaround that does NOT involve the use of Memory Mapped Files since Memory Mapped Files only works on 64-bit system.  Using MMF is not an option since we need qora to work on all systems including 32-bit.  We need to find out what's causing the thread to get interrupted before all file I/O operations are completed.

rlh's comment got me thinking, and I think that it's possible the client does not exit cleanly (FC was closed while the changes to database were not fully committed).  Since I've only run Qora on 64-bit machines, it's difficult for me to replicate Poloniex's problems and do a debug.  Someone definitely needs to follow up on this and identify which calls are causing the problems.

2.  I hope we can incorporate the following changes before we roll out the next version of Qora wallet - 1) Skerberus' Installer, 2) MMF workaround (for 32-bit system' sake), 3) the new Wallet design Pondsea was working on, 4) encoding method as suggested by deMap.

Keep up the good work, guys! 

Vrontis has a copy of the wallet design

[calvinstm]

Vrontis has a copy of the wallet design

Totally awesome!  Honestly can't wait to see what you've done with Qora client. 

I, for one, am tired with the old plain wallet design.  Thanks so much for doing this for Qora! 

[calvinstm]

I'm working on adding Qora to the voting on Comkort but need to know ...

• Is there a main website any longer?
• Who should I list as the contact (will need email address)?

I'm going to use the forum thread if there isn't a main website now (the one linked is dead).

Update : I went ahead and listed this thread and used my email address for now.

Vote here (when active) : https://comkort.com/vote#QORA

Hi Tuck,

Thanks for creating the Qora poll.    Unfortunately, Qora is still not in the voting list after 3 days.     Is there any news or update? Could you please follow up on that?

It'd be nice for Qora to have two primary trading sites. Comkort seems like a really good choice.

Thanks for doing this!

[Tuck Fheman]

Hi Tuck,

Thanks for creating the Qora poll.    Unfortunately, Qora is still not in the voting list after 3 days.    Is there any news or update? Could you please follow up on that?

It'd be nice for Qora to have two primary trading sites. Comkort seems like a really good choice.

Thanks for doing this!

Yeah, not sure what else I can do.  Maybe the original question needs to be upvoted before they will acknowledge the vote portion(?) ... no clue.

You can vote up the original question here.

I'm hoping ShapeShift comes through, it actually sounds more promising than I was originally led to believe.

Update : I've never used hitBTC, but I contacted them today as well about adding Qora.

[calvinstm]

Yeah, not sure what else I can do.  Maybe the original question needs to be upvoted before they will acknowledge the vote portion(?) ... no clue.

You can vote up the original question here.

I'm hoping ShapeShift comes through, it actually sounds more promising than I was originally led to believe.

Update : I've never used hitBTC, but I contacted them today as well about adding Qora.

Thanks, Tuck.    Just registered a new account at ComKort, but for some reason, I didn't get an activation email.  I'll definitely upvote your post when the email comes through, and I'd suggest everyone does the same. Both hitBTC and Comkort look great, and hopefully we can get at least one of them to list Qora.  Just let us know where to go if you need our votes.  

[Vrontis]

BTW, in case you haven't been following our conversation -- the V18 Qora Client is totally stable on a 64-bit system (64-bit JVM) but will most likely crash on a 32-bit system due to a database issue.  It's not a bug, it's just that some codes need to be restructured/refactored so the file stream doesn't close unexpectedly while the application is still expected to do additional I/O.

We're waiting for someone to dig into the code and provide a workaround for it, but in the mean time, Qora is 64-bit only.

 

I have tested the wallet in a low end virtual machine (32 bit Linux),  loaded 100 addresses and the wallet haven't faced any issues except the time that it was needed to load which was around 5 minutes.

[calvinstm]

I have tested the wallet in a low end virtual machine (32 bit Linux),  loaded 100 addresses and the wallet haven't faced any issues except the time that it was needed to load which was around 5 minutes.

Thanks, Vrontis.    I just modified my post.  The truth is Qora client has always been very stable for me since beginning, and I've never encountered any problem others mentioned. I don't know if it has to do with the fact that I've only run Qora on 64-bit windows based system or not.

In any case, I realized I wasn't qualified to say anything since I wasn't able to replicate Polo's errors.  Apologize for the confusion and possibly misleading statement.

I think I'll shut up now and leave this matter in your capable hands.