Monero Development and Engineering

[monero-extended]

››› Development and Engineering ‹‹‹
Monero ANN • Monero Support • Monero Mining • Monero Speculation • Monero Economy
#monero • #monero-dev • #monero-otc

The usage for this thread will encompass questions and discussion on topics that are more technical than can be found in the Monero Support thread. It will focus on development discussion, engineering-related topics such as the API usage, pool developer support as well as information to be found on the upcoming Monero Wikia.

This thread is self-moderated. Off-topic questions will either be deleted or possibly re-posted in their appropriate Monero thread depending on the discretion of the OP. Rude, offensive and completely off-topic discussions are not encouraged and will have to be removed to keep a high quality thread. Aside from that posts may be deleted for whatever reason. Deletion does not necessarily mean that the post was offensive. It may also have been too long to quote (in which case either the original, or the reply may be pruned), repetition of yours or somebody else's point, or anything else.

Here are useful links (Links will be added as necessary):

Links to available software:
Github member tree: https://github.com/tewinget/bitmonero/network/members
CryptoNote Pool Source: https://github.com/zone117x/node-cryptonote-pool
Tsiv's Nvidia Miner Source: https://github.com/tsiv/ccminer-cryptonight
Jojatekok's Windows wallet source: https://github.com/Jojatekok/monero-client-net
tewinget's repo: https://github.com/tewinget/bitmonero/commits/master
Wolf0's CPU Miner source: https://github.com/wolf9466/cpuminer-multi
Monero Improvement Protocols: https://github.com/monero-developers/mips
I2PD Source: https://github.com/orignal/i2pd

Links to other information sources:
Main Website: http://www.monero.cc/
Community FAQ Thread: https://bitcointalk.org/index.php?topic=686086.msg7784707#msg7784707
Community Recognition Thread: https://bitcointalk.org/index.php?topic=694762.msg7845979
White Paper: https://cryptonote.org/whitepaper.pdf
White Paper Review: http://monero.cc/downloads/whitepaper_review.pdf


-updated by kbm

Sorry in advance for being a tyrant. I know that im deleting a bunch of well meaning posts from well meaning people but they simply aren’t topical. This isn't a thread for discussion about the virtues of or problems with moderation. I still love you guys. Please don't be mad. This is my job. We have other threads with looser standards. This thread is for development related discussion, NOTHING ELSE.

[1715499244]

1715499244

[1715499244]

1715499244

[1715499244]

1715499244

[kbm]

I will not participate in a self-moderated thread because I've already seen a desire for censorship which impact engineering considerations. My posts will continue in the unmoderated thread.

I edited the post to explain that technical considerations were impacted by the topic you wanted to censor.

You know as well as I do that this isn't really about a self-moderated thread, and I'm not going to start off with having the first thing on the thread done by deleting all the posts inside of it. I'm just not. Make of that what you will. If it (edit: aimless conversation - not you personally) gets to the point where it's driving people insane, that's a different story.

I posted on your self-moderated thread as 4 different profiles and not even once did I care if I was censored. Your keen insights won't be censored just because you're worried someone won't know you made an awesome technical point. If you can bear it, please stay around. If not and everyone else follows suit, then this will be a lonely thread where I will toss any information I can find into it about anything related to the topics.

It's a self-moderated thread, therefore by extension the discussion has simply been about self-moderation itself. I'd consider that a part of the implicit nature of the thread.

Can we move on from that or will this be an endless tantrum? You literally just said to aminorex "your IQ is higher than that" ... well ...!

Back on topic.

Current development discussion topics:

I2P C++ Integration
Code descriptions/labeling/annotations
GUI wallet: https://github.com/Neozaru/bitmonero-qt
Website progress
Pool updates

- what kinda of commits are people seeing on github?

[smooth]

Database

[kbm]

Database

That's a pretty good topic. What kind of database is being worked on? How does it differ from bitcoin?

I'm going to try to take same descriptions from the bitcoin wiki/wikipedia here:

https://en.bitcoin.it/wiki/Main_Page
http://en.wikipedia.org/wiki/Bitcoin

And try to retype some of them to describe Monero right in this thread when I get the chance, so that the information can be copied/pasted in the future. If someone working on the wikia reads this before I get a chance to contact you, please post here or pm me topics you'd like to see written up.

The FAQ that was started is good but some of it can be an entire topic for sure, and some of it can stay as a FAQ.


Other topics I can think of:

Transaction Auto Splitting
Deterministic Wallets

White Paper Citations:


1. http://bitcoin.org

2. https://en.bitcoin.it/wiki/Category:MixingServices - dead link

3. http://blog.ezyang.com/2012/07/secure-multiparty-bitcoin-anonymization/

4. https://bitcointalk.org/index.php?topic=279249.0

5. http://msrvideo.vo.msecnd.net/rmcvideos/192058/dl/192058.pdf

6. https://github.com/bitcoin/bips/blob/master/bip-0034.mediawiki#Speci%0Ccation.

7. https://github.com/bitcoin/bips/blob/master/bip-0016.mediawiki#BackwardsCompatibility

8. https://en.bitcoin.it/wiki/Mininghardwarecomparison - dead link

9. https://github.com/bitcoin/bips/blob/master/bip-0050.mediawiki

10. http://luke.dashjr.org/programs/bitcoin/%0Cles/charts/branches.html - dead link

11. https://bitcointalk.org/index.php?topic=196259.0

12. https://en.bitcoin.it/wiki/Contracts

13. https://en.bitcoin.it/wiki/Script

14. http://litecoin.org

15. Moderately hard, memory-bound functions   

16. Ad-hoc-group signatures from hi-jacked keypairs
   
17. Short linkable ring signatures revisited

18. High-speed high-security signatures
   
19. Group signatures
   
20. Exponential memory-bound functions for proof of work protocols
   
21. Proofs of partial knowledge and simplied design of witness hiding protocols
   
22. On memory-bound functions for fighting spam - another nice one
   
23. Sub-linear size traceable ring signatures without random oracles

24. Traceable ring signature
   
25. Peer review of "quantitative analysis of the full bitcoin transaction graph"
   
26. Linkable spontaneous anonymous group signature for ad hoc groups
   
27. Linkable ring signatures: Security models and new schemes (Link courtesy of binaryFate)

28. Zerocoin: Anonymous distributed e-cash from bitcoin
   
29.  Structure and anonymity of the bitcoin transaction graph
   
30. Universal electronic cash
   
31. The bitcoin transaction graph | anonymity
   
32. Stronger key derivation via sequential memory-hard functions
   
33. An analysis of anonymity in the bitcoin system
   
34. How to leak a secret
   
35. Quantitative analysis of the full bitcoin transaction graph
   
36. Analysis of hashrate-based double-spending
   
37. Rational points on certain hyperelliptic curves over finite fields
   
38. Ad hoc group signatures (Link courtesy of binaryFate)

[smooth]

Database

That's a pretty good topic. What kind of database is being worked on? How does it differ from bitcoin?

It was mentioned in the latest Missives #6: https://bitcointalk.org/index.php?topic=583449.msg7847156#msg7847156

What I gather from browsing the commits is that the work is focusing on refactoring to support a clean database interface, and then one or more databases will be supported.

[fluffypony]

Database

That's a pretty good topic. What kind of database is being worked on? How does it differ from bitcoin?

It was mentioned in the latest Missives #6: https://bitcointalk.org/index.php?topic=583449.msg7847156#msg7847156

What I gather from browsing the commits is that the work is focusing on refactoring to support a clean database interface, and then one or more databases will be supported.

Spot on - first we refactor in order to support arbitrary key-value stores, and then we test a bunch of them for our specific workloads before deciding on one. Initial candidates that we liked out the box were hamsterdb and rocksdb. There's been some contention as there is a slightly faster version of hamsterdb available for purchase (you get the source, but can't distribute it), and some people feel its very availability would compromise the integrity of Monero. On the other hand, rocksdb (a leveldb fork by Facebook) breaks leveldb's Windows compatibility.

So everything is staying as loose and generic as possible, and then we'll plug into as many embedded databases as we can get our hands on, and publish the results. I'm also terribly keen on having a full node pushing the blockchain on to Tahoe-LAFS and having a bunch of independent nodes "verify" that the correct blockchain is being pushed, partly for fun, but also because I can envision some interesting use-cases where applications can be built accessing the blockchain on Tahoe-LAFS instead of locally (lightweight clients, for instance).

[nexern]

for databases, i like to mentioned two additional candidates to test.

1. kyoto cabinet | c++ [ http://fallabs.com/kyotocabinet/ ]
2. Sophia | c [ http://sphia.org/benchmarks.html ]

kyoto has a very rich feature set and includes also kyoto tycoon (server part), mapreduce and lua scripting,
which could be very usefull for bc related services later (if any planed). i have tested and using kyoto under
heavy read/write conditions up to 0.5 tb files without any problems so far.
it's a lean, fast and very well written kv core with many usefull extensions to mimik relational workload if needed.
i didn't test sophia under real world conditions until now and sophia is more generic from it's handling but from the
performance view it looks promising. a test could make sense (just in case you are not already settled with hamsterdb).

[smooth]

for databases, i like to mentioned two additional candidates to test.

1. kyoto cabinet | c++ [ http://fallabs.com/kyotocabinet/ ]
2. Sophia | c [ http://sphia.org/benchmarks.html ]

kyoto has a very rich feature set and includes also kyoto tycoon (server part), mapreduce and lua scripting,
which could be very usefull for bc related services later (if any planed). i have tested and using kyoto under
heavy read/write conditions up to 0.5 tb files without any problems so far.
it's a lean, fast and very well written kv core with many usefull extensions to mimik relational workload if needed.
i didn't test sophia under real world conditions until now and sophia is more generic from it's handling but from the
performance view it looks promising. a test could make sense (just in case you are not already settled with hamsterdb).

What is great about the approach being taken is that databases will be pluggable and it will be easy for others in the community to test their favorite databases and report. There is no reason to think the official build might not eventually use a database identified by the community as the best one, or that there will be different builds for different applications (pool node, high volume non-mining node, end user wallet, etc.)

 

[rdnkjdi]

What is great about the approach being taken is that databases will be pluggable and it will be easy for others in the community to test their favorite databases and report. There is no reason to think the official build might not eventually use a database identified by the community as the best one, or that there will be different builds for different applications (pool node, high volume non-mining node, end user wallet, etc.)

I'll just throw my opinion in for what it's worth.  I used to work for a company that mostly used SQL Server (for end user applications).  Later on they picked up web developers who used MySQL for the websites (except for one & on that one they used SQL Server).  A little later on, they used postresql for a new project.

There was some resistance to using different backends and by the time it was all said and done having one solution seemed like it would've been a better idea.  Because the company experience was duplicated across all projects.  For something like this it seems like one database solution for all wallets would be a better idea as all troubleshooting & optimization would be focused on one solution rather than several.

I know we are talking about completely different types of databases - but I think the substance of the issue is applicable. 

My (uneducated) .02

[smooth]

What is great about the approach being taken is that databases will be pluggable and it will be easy for others in the community to test their favorite databases and report. There is no reason to think the official build might not eventually use a database identified by the community as the best one, or that there will be different builds for different applications (pool node, high volume non-mining node, end user wallet, etc.)

I'll just throw my opinion in for what it's worth.  I used to work for a company that mostly used SQL Server (for end user applications).  Later on they picked up web developers who used MySQL for the websites (except for one & on that one they used SQL Server).  A little later on, they used postresql for a new project.

There was some resistance to using different backends and by the time it was all said and done having one solution seemed like it would've been a better idea.  Because the company experience was duplicated across all projects.  For something like this it seems like one database solution for all wallets would be a better idea as all troubleshooting & optimization would be focused on one solution rather than several.

I know we are talking about completely different types of databases - but I think the substance of the issue is applicable. 

Maybe you misunderstand (I'm not sure).

We will likely ship and support just one database, or maybe a database and a flat file version.

Other people (for example a payment processing service, exchange, etc.) who want to make their own customized deployment might do something else, but we wouldn't be supporting that, they would.

[rdnkjdi]

Maybe you misunderstand (I'm not sure).

We will likely ship and support just one database, or maybe a database and a flat file version.

Other people (for example a payment processing service, exchange, etc.) who want to make their own customized deployment might do something else, but we wouldn't be supporting that, they would.

ah.  got it.  my understanding was the devteam having a payment processing download, exchange download, etc.

bad reading comprehension.

[craig-st]

I did a 'git clone' followed by 'make test-release' to run the Monero project's built-in test suite.  Two of the twelve tests fail: coretests and unit_tests.  I'm on Linux (Ubuntu).  Anybody else have this problem?  I have the same problem with both the latest release ('git checkout tags/v0.8.8.6') and the most recent updates (v0.8.8.6-46-g6e5797d).  Coretests takes a really long time before it fails.  Like half an hour.

[lijingang]

Is The APEPEPOW Project a meme coin in the Monroe community? Whether the APEPEPOW project developer is your Monero developer？
https://github.com/APEPEPOW