[ANN] FACTOM - Introducing Honesty to Record-Keeping

[dansmith]

To be a Factom Server
...
I hope this helps.

Thanks for trying but it helped only 20% Perhaps my Q number 2 was answered indirectly.
Maybe I'm not taking enough time to break down my Qs more minutely, maybe you are busy and skim over my Qs without realizing what is being asked.
It's OK though, take your time. I understand that probably much of what I'm asking is not yet set in stone.

[BrianDeery]

dansmith:


I must admit there is a lot we didn't put in the paper.  There isn't a lot on consensus or incentives.  38 pages was stretching it already.  Satoshi's was only 9.  AlanX is memorializing more of what we are designing in a forthcoming consensus paper.  Incentives have yet to be explored formally.


1: Yes, something like that.  historical crowd sales map the private key owner in Bitcoin to be the new owner in the funded system.  A lot of that heavily relies on having the same elliptic curve as Bitcoin, though.  5 years have passed, so we can build our system using updated crypto, but that would make shortcuts like this harder.  It would be nice to get some advantages by using Edwards Curves, etc.  We haven't gotten quite to that level of detail yet for that part of the system.  Also, we have very limited time to experiment with new things.

2: Yes.  The cost of doing business includes paying the BTC transaction fee.  For the Federated servers to play in the system, they need to have a small amount of hot wallet BTC to pay transaction fees.  Since there is a small (~=16) set of Federated servers who need to hold the BTC for transaction fees, users or non-authoritative full nodes do not need to hold BTC.  Every 10 minutes, a deterministic algorithm determines which one of the 16 Federated servers is responsible for anchoring the group's data.

3: There are varying levels of trust that an application can have.  Just like Bitcoin, Factom is eventually consistent.  Bitcoin gives the irreversibility metric by number of confirmations and by how many of the nodes have the TX in their mempools.  Factom will have granularity of certainty too.   

certainty level 1: A Federated server has acknowledged receipt of the data.

certainty level 2: The Federated server's peers have not alerted a problem over the Factom P2P network.

certainty level 3: The Anchor (Merkle root of all Factom data over last 10 minutes) with your data in it has a Bitcoin transaction on the Bitcoin network.

certainty level 4: Federated server peers have not created alert transactions in the form of alert BTC transactions.

certainty level 5: The Anchor has 1 Bitcoin confirmation.

certainty level 6: The Anchor is buried under under several Bitcoin blocks, without also seeing some kind of alert or canceling transactions from the Federated server peers.  Eventually, Factom will extend beyond a statute of limitations style period to prevent the Federated server majority from canceling an anchor of a misbehaving peer.


Since we use a multistep commit and reveal process, the Federated servers will be accepting payment for Entries which they do not know the contents of.  If later, they censor based on the Entry content, it is provable.  A user ( or more probably an Audit server) will publicize this censorship.  The Audit servers have an incentive to make the community dislike the current Federated servers, so the Audit servers can get enough votes to be promoted up to Federated servers, and get more Factoids.  Adam Back advocated for blind commitments similar to what we are doing in Bitcoin earlier.  https://bitcointalk.org/index.php?topic=206303.0

A lot of this is still preliminary.


4:  When the Entry Credits are used to pay for Entries, new Factoids will be credited and spread amongst the various participating Federated and Audit servers.  This is similar to transaction fees paid to Bitcoin miners.  Distribution proportionality schemes are also preliminary.


5: I think this is where a misunderstanding is occurring.  The 16 Federated servers cooperate to collectively sign the Directory Block, which has Merkle roots of all the Entry Blocks.  They need to agree as a majority to a single directory block.  If a server were to pay for Entries to the system itself, it would get back less than 1/16th of the value it put in. 




Concerning the threshold multisig:

Factom's reason for existence is to move transactions off of the blockchain.  looping half of the process back into BTC again goes against that core rationale, but I can see where you are going with it. 

Factom is a dynamic system.  With a multisig system, someone has to actually control the private keys.  That doesn't afford a really dynamic system. 

We have a type of delegated proof of stake.  This gives the active users in the system the say in who they want to package transactions.  This has some merits.  I hear the Ethereum folks are planning on now using some type of POS in their consensus.  There are even serious proposals for bringing POS elements into Bitcoin (see Proof of Activity http://www.cs.technion.ac.il/~idddo/PoAslides.pdf). 

The POS elements mean that the system authorities (federated servers and audit servers) can change in a short period of time based on the expressed wishes of the users.  There are often calls to change what mining pools are patronized by individual miners.  this will give a similar ability but enclosed in the Factom system.


If we brought in merge mining to the system, we would rely on miners twofold.  we are already using miners for the non-reversibility.  with merge mining, they would also be setting policy.  This is a secondary business for them, and may not give it the attention someone who made it their sole business.


Bitcoin multisig currently doesn't have a high enough M of N for standard scripts.  we are trying to get this project working without the help of miners (other than them not censoring our anchors, etc).  current standard transactions give a maximum of 4 of 6, which isn't enough separation of powers for my taste.  http://bitcoin.stackexchange.com/questions/23893/what-are-the-limits-of-m-and-n-in-m-of-n-multisig-addresses  Going higher would involve cooperation with miners. 


But back to the threshold multisig.  For the system to be dynamic and in control of the users, there needs to be some coupling between the stored value in the system and a say in who controls the system.  in a threshold multisig system, the coupling is permanent once a payment is made.  this encourages keeping smaller amounts in multisig as a balance, defeating the purpose of the whole system (moving transactions off the blockchain).

Threshold multisig also changes the system dynamics.  If the service gets big enough, the majority can misbehave and steal all the stored bitcoin in the system.  With the current design, if the majority misbehave, they will end up destroying the value they created in the system.  This seems like the biggest problem with a threshold multisig to me. 

Systems build on top of Factom are expecting it to last indefinitely.  Factom also gives proof of the negative, which makes switching from one system to another expensive for a distributed system.  There is some commentary on this in our AMA thread in Reddit. http://www.reddit.com/r/Bitcoin/comments/2mkyrg/we_are_the_founders_of_factom_see_our_white_paper/  Systems are looking for a canonical source to hold the entirety of their transactions.  Setting up competing threshold multisig systems would expand the search space needed to prove a negative.  Also a distributed system would need to decide which system to expand its search into.  Too wide of a search and spam becomes a problem.  Seperating Factom from bitcoin value also makes it portable.  Factom can switch the POW system which provides its non-reversibility.  If Bitcoin falls to another POW system, Factom can switch to being secured by that system, or a multitude of systems.

I haven't considered all the pros and cons of a system you are imagining.  I can imagine a system like it working technically.  I can also imagine a purely altruistic system of packaging transactions and securing them with Bitcoin.  It boils down to what system correctly aligns incentives of all the parties involved, rather than what is technically possible.  On the topic of incentives, I must admit I have a strong incentive to believe the things I have described.  Please do consider exploring further the system you are starting down. 

[AlanX]

To be a Factom Server
...
I hope this helps.

Thanks for trying but it helped only 20% Perhaps my Q number 2 was answered indirectly.
Maybe I'm not taking enough time to break down my Qs more minutely, maybe you are busy and skim over my Qs without realizing what is being asked.
It's OK though, take your time. I understand that probably much of what I'm asking is not yet set in stone.

Brian did a much better job of answering... 

[Alex_crypto]

Watch Paul Snow on Hangouts for the latest Factom updates from the Denver Bitcoin Center. Click on the pic.  

https://i.imgur.com/aVkj4fC.png

Paul Snow is a star! Thanks for the great explanation of Factom.

How is Factom not everywhere? This is such a great project that is going to change record keeping everywhere. Record keeping that is needed by a trusted source!

[dansmith]

@BrianDeery, thanks for breaking things down. I now have a better understanding of how Factom works.

Threshold multisig also changes the system dynamics.  If the service gets big enough, the majority can misbehave and steal all the stored bitcoin in the system.  With the current design, if the majority misbehave, they will end up destroying the value they created in the system.  This seems like the biggest problem with a threshold multisig to me. 

I agree that my thresh.mult. scheme creates an extra incentive for a 51% attack. I'm saying "extra", because with the current design the attacker can short his Factoid position (on the market) and then perform the attack.

[dansmith]

4:  When the Entry Credits are used to pay for Entries, new Factoids will be credited and spread amongst the various participating Federated and Audit servers.  This is similar to transaction fees paid to Bitcoin miners.  Distribution proportionality schemes are also preliminary.

Could you please elaborate on this. Because seems like what you are saying is that every single Entry into Factom will result in a transaction on the Factoid chain. This will surely bloat the Factoid chain very quickly.

[AlanX]

4:  When the Entry Credits are used to pay for Entries, new Factoids will be credited and spread amongst the various participating Federated and Audit servers.  This is similar to transaction fees paid to Bitcoin miners.  Distribution proportionality schemes are also preliminary.

Could you please elaborate on this. Because seems like what you are saying is that every single Entry into Factom will result in a transaction on the Factoid chain. This will surely bloat the Factoid chain very quickly.

When Factoids are converted into Entry Credits, they return to the protocol.  The number of Factoids currently in the protocol can be computed.  When entry credits are used to create entries, the entry credits are effectively burned, and the Factoids released to the servers.

That is all high level stuff.  The actual payment to the servers will occur on a far less frequent basis.  We are thinking every four hours.  The payment will be their block reward share, plus their share of the Factoids released in the previous four hour period. (So we collect the payments over four hours, then we wait four hours, then we release the payments.  So payments would be made every four hours, and at the time they are made, they are four hours old.)

In Bitcoin, block chain rewards are paid out after 100 confirmations, though most pools wait for 120 confirmations.  That's 7 to 10 hours.

[dansmith]

When entry credits are used to create entries, the entry credits are effectively burned, and the Factoids released to the servers.

How exactly are they released? What will the "release" transaction look like on the Factoid chain? What kind of linking will there be to show that the newly released Factoids are associated with burned entry credits?

[BrianDeery]

This will be a self policing style system, similar to how the Bitcoin miners self police amongst themselves for the block reward.  If a miner decides to award themselves more BTC than the fees that were paid in, then the majority of other miners will reject that block.  Factom will have a similar system.  The majority will reject a block which has outputs that exceed the number of signed inputs. 

As far as that level of detail for constructing proofs of transfers, we haven't gotten to that yet.  There is a lineage that can be traced, protected by signatures, so it can't be forged.

The Entry will be associated with an Entry Credit commit payment, which was signed by a key.  That key had an Entry Credit balance because of a signed Factoid transaction, which essentially converted Factoid value into Entry Credit value at some exchange rate.

The progression of value can be followed back to where the Factoid's were initially committed. Since signatures and hashes provide cryptographic assurance, then a proof can be constructed.  If we are going to record that proof, or have it be an in-memory operation like Bitcoin is yet to be determined.

[FACTOM]

Join Factom Founders Peter Kirby and possibly Paul Snow on Hangouts Wednesday December 10th at 6.30PM (PT) @Seattle Bitcoin Meetup.

You are invited to join us at the Seattle Bitcoin Meetup, if you can't make it you can tune in to the Google Hangouts at the below link.
Live stream starts at 6.30PM (PT).

https://plus.google.com/events/cgotpt5d6g5ad8k0idniuk6eo64?authkey=CIG519ynvf78cA

[FACTOM]

What is Factom?

The first explainer video now available on our YouTube channel! Click below to view.

[FACTOM]

Factom Use Case - Saving BoA $17 Billion

Click below to view on our YouTube channel.

[dansmith]

Thanks, that's an interesting video about BoA's 10 bill.docs.
I was prompted to this thought:
32-byte hashes of 10 bil. docs add up to 300GB of storage. That's for only one bank.
Eventually, only the Federated servers (FSs) will be able to store the whole DHT.
The attacker can query DHT to learn a unique subset of nodes which store all data, i.e FSs.

Then the attacker can mount a botnet DDOS attack on those FSs.
This attack will not grind Factom to a halt, it will only prevent FSs from serving legitimate customers which require the unique data stored by FSs.
Of course, having FSs running as Tor services would mitigate a DDOS attack.

[FACTOM]

Factom Presentation at the Seattle Bitcoin Meetup

Checkout the video below and more on our YouTube channel.

[BrianDeery]

DanSmith:

The magic of having the DHT, though is that subsets of the data can be shared independently.  The bank itself could spool up many copies of its data if it needed to share it in a distributed fashion.  A potential auditor knowing they might need the data in the future could collect it in real time.  If the network were attacked, the proof would still be rooted in Bitcoin.

One mitigation to your attack would be for the federated servers to share different subsets of the data on different IPs.

I do envision different availabilities for different datasets.  Popular datasets will be massively shared, and niche ones would be harder to get.  In the distant future, it may even come to a point where payment is required to download rare data.  Businesses would sell this historical data which doesn't have altruists sharing it.


What I worry more about is sibyl attacks on the DHT routing table.  If you only get lies as to where you can find your data, then you are out of luck.  I have read about ISPs sibyl attacking their customer's bittorrent connections, but those were more to force their customers to share amongst themselves, instead of over the ISP's peering connections.  There is a lot of research being done finding how to burn through DHT sibyl attacks, but it is indeed a complex problem.


Running Federated servers over tor will not likely work, mostly because of the bandwidth.  The federated servers need to have low latency to turn the data around.  It is a flood fill network, so every full node needs to see all Entries, plus 3 packets with assorted signatures covering that Entry.  The federated servers come to consensus every minute.

well... now that I think about it, only a majority is needed for consensus.  One of the federated servers could listen over clearnet, pretending to be a non-authoritative full node.  Its signed transactions could be broadcast through Tor.  only 2 sigs of hashes would go through Tor for every Entry.  If they take too long to broadcast consensus, then the rest of the network can quickly come to consensus without them.  Timing attacks from their peers are still possible to find them, though.  we do have heartbeats that are needed from every federated server to show the rest of the network they are still alive.  this may be hampered by tor's latency.  maybe the heartbeats can be sent over multiple tor routes to decrease the chance of one getting lost. 

Bitcoin bans spamming peers, but that wouldn't work over tor.  a full node deciding to listen on tor would just have to eat all the DOS traffic, since it couldn't ban an IP. 

Sharing data over Tor after the fact will work much better, since latency and bandwidth are less critical.  a tor node can insert itself into the DHT routing table, and offer up contraband data.

Lots to think about with Factom and Tor.  Bitcoin's flood fill network has some of the properties of tor, since peers don't know if a new transaction originated from that peer.  Factom's network is probably closer to the streamtorrent system.  It would be interesting to see how traceable that network is.

Apologies for the rambling response to your post. 

[BrianDeery]

Here are some thoughts that came from a different channel I thought were worth sharing:

Factom is more about record-keeping than storing data.

We are trying to give users the irreversibility of the Bitcoin blockchain without having every transaction in Bitcoin.  We are trying to make a system which is both censorship resistant as well as spam resistant.  The two are diametrically opposed.  Bitcoin itself is trying to achieve both of these as well. 

Bitcoin is a big complex system which boils down to the world agreeing on a UTXO set.  It is a big datastructure to see if money has been sent to someone else first. 

Bitcoin proves the negative (that money hasn't been spent earlier).  A BTC receiver must have the entire UTXO set to prove that negative.  (There may be more optimizations later, with patricia-merkle trees, etc, but the miners need the entire UTXO.  Also, light clients can trust miners, but I digress.)

Factom is trying to generalize the proof of the negative.

Factom is also a big complex system to let the world agree on a data set.  Factom takes disparate sets of data, and classifies them into buckets.  These buckets, which we call Chains, and have user specified IDs.  The idea is that all of the data relevant to an application is tagged and placed into the correct buckets.  For an application to prove a negative, it needs to download and verify only the data in its Chains.  If the data is not included in the Chain, it can be assumed not to exist.  This is very similar to how a bitcoin can be assumed not to have been spent if it is not included in a block.


The intent is to not store large pieces of data.  we are trying to discourage that by setting the pricing for other services to be more attractive. 

The data is shared on a Distributed Hash Table, so a user could reshare out the data which they are interested in, but not store data for other's uninteresting applications.

The storing of the bulk data in storj or maidsafe is what we had in mind too.  One of the problems with storing in storj, though is someone needs to keep paying a maintenance fee.

[delulo]

Is Factom designed to have some kind of coin distribution / fundraiser / initial mining? I could have found out I guess by reading the whitepaper but maybe you can give me a short version

Would the following summary about Factom be about correct? Factum lets you create ledgers that are periodically recorded in the Bitcoin blochchain. The developer only has to deal with the rules / API of the factom ledger(s) and not with Bitcoin itself. Also the centralized handling of the Factom ledgers makes "confirmation" fast. This might be pretty wrong since I just read the wsj article. But please enlighten me if it is.

[embicoin]

I discovered this project on storj forums and I am very interested on it, I will follow close.

[escalicha]

Getting place! Maybe I will invest! looks crazy!

[FACTOM]

Working on a project and need a system of records? Factom is here to help. Introduce us to your use case.

Factom is a simple extension of the Bitcoin blockchain that lets you build faster, cost effective applications.
Factom is a system for securing millions of realtime records in the blockchain with a single hash. This gives you the tools to build applications with all of the security of the blockchain without the speed, cost, or size limitations. The blockchain opens tremendous possibilities to bring honesty to all systems. And Factom creates tools to build on top of it.

• What is Factom? Watch our explainer video: http://youtu.be/YBXNLW-YvoA
• Read our whitepaper: https://github.com/FactomProject/FactomDocs/raw/master/Factom_Whitepaper.pdf
• Test our Factom API: https://github.com/FactomProject/FactomDocs/blob/master/FactomAPI.pdf
• Factom Use Case - Saving BoA $17 Billion: http://youtu.be/DbRVPBFiy64

Are you working on a project and need a system of records? Introduce your project to us, we are looking for use cases and integrating our Open Source code and API with as many ideas as possible.