[BTC-TC] Virtual Community Exchange [CLOSED]

[InstantBTC]

anyone else not able to login using 2FA? leaving the 2FA field blank gives me a missing 2FA error (obviously), using 2FA takes forever to load and finally redirects me to a cloudflare site telling me the requested site is offline.

anyone else not able to login using 2FA? leaving the 2FA field blank gives me a missing 2FA error (obviously), using 2FA takes forever to load and finally redirects me to a cloudflare site telling me the requested site is offline.

Even if I can log in successfully, once I get to trying to make a trade it hangs and then eventually comes back with an invalid gauth message.  pretty frustrating, been trying to swap some of my 1/100 for 1/1 all morning

I might be closer to the webserver that you, or my connection may just be faster, but I can make 2FA work if I wait for a new auth code and send it ASAP.

[carnitastaco]

anyone else not able to login using 2FA? leaving the 2FA field blank gives me a missing 2FA error (obviously), using 2FA takes forever to load and finally redirects me to a cloudflare site telling me the requested site is offline.

anyone else not able to login using 2FA? leaving the 2FA field blank gives me a missing 2FA error (obviously), using 2FA takes forever to load and finally redirects me to a cloudflare site telling me the requested site is offline.

Even if I can log in successfully, once I get to trying to make a trade it hangs and then eventually comes back with an invalid gauth message.  pretty frustrating, been trying to swap some of my 1/100 for 1/1 all morning

I might be closer to the webserver that you, or my connection may just be faster, but I can make 2FA work if I wait for a new auth code and send it ASAP.

can you actually make trades, or just log in?

[gimme_bottles]

logging in works again now (using 2FA). I haven't tested trading yet because i had no intentions to do so.

[burnside]

I'm actually leaning toward the bottleneck being something CPU related.  When the site slows down, the cpu on bitcoind pegs at 100% and disk IO does not seem to spike a whole lot.  (I've been graphing the system via collectd.)

Since bitcoind is not multithreaded, I think I'm pretty much at it's limit.

I've never seen it done before, but would running two bitcoind servers (and subsequently moving half of the accounts to the new one) mitigate the problem?

Yeah, sharding the bitcoind would work.  Even if I sharded four of them onto the same (quad-core) box it would make a huge difference.  Only thing keeping me from doing that is I'd have to do away with the free internal transfers.  Account to account transfers would have to be done over the bitcoin network.

[furuknap]

Only thing keeping me from doing that is I'd have to do away with the free internal transfers.  Account to account transfers would have to be done over the bitcoin network.

Why not do a similar thing you do with AM-PT and 'sponsor' one internal transfer per month per user and charge for the rest?

.b

[Carnth]

Yeah, sharding the bitcoind would work.  Even if I sharded four of them onto the same (quad-core) box it would make a huge difference.  Only thing keeping me from doing that is I'd have to do away with the free internal transfers.  Account to account transfers would have to be done over the bitcoin network.

Oh no. I rely on free internal transfers when paying dividends to my pass-through.
It's a feature I would really loathe to see go, but I can understand.

Perhaps if my user accounts were all on the same shard I can still get free transfers?

[creativex]

The loss of free internal transfers would be problematic for asset issuers. I have to make several of these everyday to move funds around to pay dividends.

[parseval]

Have you considered throwing a message queue, like RabbitMQ, up in front of bitcoind and then using worker processes to perform the operations?  You'd keep two sets of books for your internal balance movements, one in the database and the other in bitcoind, and the workers would synchronize the two.  Transactions involving bitcoind are sent to a queue and the workers could process the tasks at whatever priority is associated with the task.  (eg, a withdrawl or a balance check could be given higher priority than an internal move command to change balances internally between accounts --  the database would already 'know' and bitcoind would be playing catch up).

[parseval]

I found this document with options you can tune to improve the performance of leveldb, the database backend for .8 and above.  It looks like modifying these from the default values would require recompiling bitcoind..

From the bitcoin git repository:

https://github.com/bitcoin/bitcoin/blob/master/src/leveldb/doc/index.html

Edit:  Adding a link to options.h, which has parameters which affect performance:

https://github.com/bitcoin/bitcoin/blob/master/src/leveldb/include/leveldb/options.h#L71

[burnside]

Have you considered throwing a message queue, like RabbitMQ, up in front of bitcoind and then using worker processes to perform the operations?  You'd keep two sets of books for your internal balance movements, one in the database and the other in bitcoind, and the workers would synchronize the two.  Transactions involving bitcoind are sent to a queue and the workers could process the tasks at whatever priority is associated with the task.  (eg, a withdrawl or a balance check could be given higher priority than an internal move command to change balances internally between accounts --  the database would already 'know' and bitcoind would be playing catch up).

I have considered this.  I haven't completely rejected it, but there are several concurrency issues around it that could be tricky to address.

I'm slowly moving down the path away from using bitcoind for balance tracking.  I'll have to parallel the balance tracking in the db while coding it all up, then run with overlap for a while, (making sure balances stay in sync)  then finally I'll be able to pull all of the balance related stuff out of bitcoind.  At that point I can shrink the wallet by extracting all of the private keys and importing them into a new wallet.  From there on out bitcoind would be purely for deposit/withdrawal processing.

I expect this process to take a couple of weeks.

It's all manageable.  I just hate that it means it will be a while before I get to do other fun stuff.  

I found this document with options you can tune to improve the performance of leveldb, the database backend for .8 and above.  It looks like modifying these from the default values would require recompiling bitcoind..

From the bitcoin git repository:

https://github.com/bitcoin/bitcoin/blob/master/src/leveldb/doc/index.html

If I understand things correctly, the wallet itself is still BerkleyDB?

The sad thing is that if I understood C++ better this is probably easily fixable.  All you have to do is store the users balance in a single place every time their balance is altered.  You'd have to store it like 7 times, one for 0 confs, 1 conf, 2 conf, .... etc up to 6 confs, at which point you could revert to the slower method of incrementing through all the transactions.  But even storing each users balance 7 times in a single place on each transaction would be way lighter than iterating through the entire transaction stack on an account to find the balance.  At least with any kind of regular getbalance traffic.  I'd setup a bounty for it, but I want to get away from BerkeleyDB entirely anyway so that I can get to a db stack with real-time replication.

[burnside]

The email functions are back now.

I have also unlocked registration, which I'd had to block due to the fact that it sent emails. 

[parseval]

If I understand things correctly, the wallet itself is still BerkleyDB?

Ah, yep, so it is.  I missed that.  It looks like BerkleyDB needs the tuning instead of leveldb.  I found a couple of documents here on Berkley DB tuning methods, though I'm not certain how useful they'll prove. 

The sad thing is that if I understood C++ better this is probably easily fixable.  All you have to do is store the users balance in a single place every time their balance is altered.  You'd have to store it like 7 times, one for 0 confs, 1 conf, 2 conf, .... etc up to 6 confs, at which point you could revert to the slower method of incrementing through all the transactions.  But even storing each users balance 7 times in a single place on each transaction would be way lighter than iterating through the entire transaction stack on an account to find the balance.  At least with any kind of regular getbalance traffic.  I'd setup a bounty for it, but I want to get away from BerkeleyDB entirely anyway so that I can get to a db stack with real-time replication.

That would be interesting, definitely..  I wonder how some of the sites like BlockChain.info do it.

[inh]

Burnside, would it be possible to have the oauth verifier not expire when the page for a specific token is called again? I can browse to https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc with my browser, and approve it, but then it reloads the same page with the oauth verifier token appended to the URL and when I use that verifier the API tells me it's already invalid

[burnside]

Burnside, would it be possible to have the oauth verifier not expire when the page for a specific token is called again? I can browse to https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc with my browser, and approve it, but then it reloads the same page with the oauth verifier token appended to the URL and when I use that verifier the API tells me it's already invalid

I hate to say it but that confused me pretty good.    Something tells me though that if it doesn't invalidate it the first time around, it may be open to replay attacks?

[inh]

Burnside, would it be possible to have the oauth verifier not expire when the page for a specific token is called again? I can browse to https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc with my browser, and approve it, but then it reloads the same page with the oauth verifier token appended to the URL and when I use that verifier the API tells me it's already invalid

I hate to say it but that confused me pretty good.    Something tells me though that if it doesn't invalidate it the first time around, it may be open to replay attacks?

I don't think so. Each time you visit the URL, for example

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc

and approve the access request, it returns the verification code in the form of a URL like

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifiestringhere>

 Each time I grant access, a NEW verifier string is generated, and the old one is invalidated. The problem is that once I approve access, the page redirects to

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifier here>

and it prompts for access again. If I just ignore the request and paste the verifier string from the URL in to my program, it comes back with this error:

Code:

OAuth server status 401, oauth_problem=verifier_invalid

Hope that helps clears things up. What I think is happening is whenever the page is called, the program invalidates any credentials used for the token in the request URL, so when the page refreshes, BAM, the verifier string in the URL is already invalid.

[lritter]

Hey guys, got a primer on how the options on BTCT work? I think I understand the calls but I'm not so sure about the puts -- when a put has 2 btc strike and 1 btc premium it is shown as 3 btc cost -- but wouldn't the issuer of the option pass _me_ those 2 btc strike if i make use of it? Who gets the premium then? Him, right, so I would effectively get 1 btc from that operation - or am I misunderstanding something here?

[burnside]

Hey guys, got a primer on how the options on BTCT work? I think I understand the calls but I'm not so sure about the puts -- when a put has 2 btc strike and 1 btc premium it is shown as 3 btc cost -- but wouldn't the issuer of the option pass _me_ those 2 btc strike if i make use of it? Who gets the premium then? Him, right, so I would effectively get 1 btc from that operation - or am I misunderstanding something here?

I think someone mentioned before that it probably should be displayed the other way around.

Strike - Premium = Price, instead of what it is right now with Strike + Premium = Price

There's a few places I need to fix that.

[burnside]

Burnside, would it be possible to have the oauth verifier not expire when the page for a specific token is called again? I can browse to https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc with my browser, and approve it, but then it reloads the same page with the oauth verifier token appended to the URL and when I use that verifier the API tells me it's already invalid

I hate to say it but that confused me pretty good.    Something tells me though that if it doesn't invalidate it the first time around, it may be open to replay attacks?

I don't think so. Each time you visit the URL, for example

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc

and approve the access request, it returns the verification code in the form of a URL like

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifiestringhere>

 Each time I grant access, a NEW verifier string is generated, and the old one is invalidated. The problem is that once I approve access, the page redirects to

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifier here>

and it prompts for access again. If I just ignore the request and paste the verifier string from the URL in to my program, it comes back with this error:

Code:

OAuth server status 401, oauth_problem=verifier_invalid

Hope that helps clears things up. What I think is happening is whenever the page is called, the program invalidates any credentials used for the token in the request URL, so when the page refreshes, BAM, the verifier string in the URL is already invalid.

When you grant access, it should redirect back to your callback page, correct?  Is this the oob situation we were discussing earlier?  No callback causes it to call itself again?  With the DDoS issues lately I haven't had any time to fix the oob functionality.  It's high on the list though.

[parseval]

When you grant access, it should redirect back to your callback page, correct?  Is this the oob situation we were discussing earlier?  No callback causes it to call itself again?  With the DDoS issues lately I haven't had any time to fix the oob functionality.  It's high on the list though.

I think oob is working okay.  At least, I've been able to register and retrieve working credentials with it.

[inh]

Burnside, would it be possible to have the oauth verifier not expire when the page for a specific token is called again? I can browse to https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc with my browser, and approve it, but then it reloads the same page with the oauth verifier token appended to the URL and when I use that verifier the API tells me it's already invalid

I hate to say it but that confused me pretty good.    Something tells me though that if it doesn't invalidate it the first time around, it may be open to replay attacks?

I don't think so. Each time you visit the URL, for example

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc

and approve the access request, it returns the verification code in the form of a URL like

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifiestringhere>

 Each time I grant access, a NEW verifier string is generated, and the old one is invalidated. The problem is that once I approve access, the page redirects to

Code:

https://btct.co/authorize?oauth_token=c6da005d45642ba73589e37444304412020f36bc&oauth_verifier=<verifier here>

and it prompts for access again. If I just ignore the request and paste the verifier string from the URL in to my program, it comes back with this error:

Code:

OAuth server status 401, oauth_problem=verifier_invalid

Hope that helps clears things up. What I think is happening is whenever the page is called, the program invalidates any credentials used for the token in the request URL, so when the page refreshes, BAM, the verifier string in the URL is already invalid.

When you grant access, it should redirect back to your callback page, correct?  Is this the oob situation we were discussing earlier?  No callback causes it to call itself again?  With the DDoS issues lately I haven't had any time to fix the oob functionality.  It's high on the list though.

Correct. When I pass 'oob' for the callback, the webpage reloads itself as the callback URL.

When you grant access, it should redirect back to your callback page, correct?  Is this the oob situation we were discussing earlier?  No callback causes it to call itself again?  With the DDoS issues lately I haven't had any time to fix the oob functionality.  It's high on the list though.

I think oob is working okay.  At least, I've been able to register and retrieve working credentials with it.

Really? I haven't had any luck at all. I can fetch the URL to approve application access to my account, and when I approve it the page reloads again, this time with the verification string in the URL. When I take that string and try to use it in my program, the website API says that it's invalid. I'll see if I can find some other oauth code to work with and test it out, the stuff I'm using has worked fine on other sites but maybe something is off with the way it works with btct.co.