mtgox API

[grondilu]

MtGox API is weird.

They say we should authenticate by sending or login and password via POST method, but they give an example where they are in URL:

https://mtgox.com/code/getFunds.php?name=blah&pass=blah

I don't get it.  It data is in URL, isn't that GET method?

[mndrix]

I'm guessing they show URL parameters as shorthand notation, since POST was specified elsewhere in the documentation.

[purplezky]

It should be a post.
I used to have a script mtgox.sh for that:
curl -d `cat credentials` https://mtgox.com/code/${1}.php

where the file credentials would contain:
name=username&pass=password

so you could do:
mtgox.sh getFunds

But lately, my script stopped working with:
curl: (35) Unknown SSL protocol error in connection to mtgox.com:443

I tried looking into it, but it didn't seem to work,
no matter if i change protocol version --sslv2 or use the --no-sessionid option,
which people suggested when this error occurs.

I guess mtgox SSL requirements changed somehow...

[grondilu]

Indeed, I could connect with a command like:

$ read -s passwd ; wget -q --no-check-certificate --no-proxy -O - "https://www.mtgox.com/code/getFunds.php" --post-data="name=grondilu&pass=$passwd"

And I had my JSON string:

{"usds":0,"btcs":0}

This is pretty cool.

[grondilu]

It should be a post.
I used to have a script mtgox.sh for that:
curl -d `cat credentials` https://mtgox.com/code/${1}.php

where the file credentials would contain:
name=username&pass=password

so you could do:
mtgox.sh getFunds

But lately, my script stopped working with:
curl: (35) Unknown SSL protocol error in connection to mtgox.com:443

I tried looking into it, but it didn't seem to work,
no matter if i change protocol version --sslv2 or use the --no-sessionid option,
which people suggested when this error occurs.

I guess mtgox SSL requirements changed somehow...

HAve you tried with www.mtgox.com instead of just mtgox.com?  I think there is an issue about that.

[purplezky]

www.mtgox.com gives the same error 

Code:

* About to connect() to www.mtgox.com port 443 (#0)
*   Trying 69.64.54.59... connected
* Connected to www.mtgox.com (69.64.54.59) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* Unknown SSL protocol error in connection to www.mtgox.com:443
* Closing connection #0
curl: (35) Unknown SSL protocol error in connection to www.mtgox.com:443

[purplezky]

It seems like strace is saying the filehandle #3 which is the socket connecting to www.mtgox.com is temporarily unavailable... 

Code:

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.3.5.9")}, 16) = 0
gettimeofday({1302256486, 388590}, NULL) = 0
poll([{fd=3, events=POLLOUT}], 1, 0)    = 1 ([{fd=3, revents=POLLOUT}])
send(3, "\325\327\1\0\0\1\0\0\0\0\0\0\3www\5mtgox\3com\0\0\1\0\1", 31, MSG_NOSIGNAL) = 31
poll([{fd=3, events=POLLIN|POLLOUT}], 1, 5000) = 1 ([{fd=3, revents=POLLOUT}])
send(3, "\331\231\1\0\0\1\0\0\0\0\0\0\3www\5mtgox\3com\0\0\34\0\1", 31, MSG_NOSIGNAL) = 31
gettimeofday({1302256486, 389446}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 4999)  = 1 ([{fd=3, revents=POLLIN}])
ioctl(3, FIONREAD, [31])                = 0
recvfrom(3, "\331\231\200\204\0\1\0\0\0\0\0\0\3www\5mtgox\3com\0\0\34\0\1", 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.3.5.9")}, [16]) = 31
gettimeofday({1302256486, 390649}, NULL) = 0
poll([{fd=3, events=POLLIN}], 1, 4997)  = 1 ([{fd=3, revents=POLLIN}])
ioctl(3, FIONREAD, [47])                = 0
recvfrom(3, "\325\327\201\200\0\1\0\1\0\0\0\0\3www\5mtgox\3com\0\0\1\0\1\300"..., 2017, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.3.5.9")}, [16]) = 47
close(3)                                = 0
time(NULL)                              = 1302256486
alarm(0)                                = 300
rt_sigaction(SIGALRM, {SIG_DFL, [], 0}, NULL, 8) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 314203809}) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 314352260}) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
setsockopt(3, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
connect(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("69.64.54.59")}, 16) = -1 EINPROGRESS (Operation now in progress)
clock_gettime(CLOCK_MONOTONIC, {1803000, 315242561}) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 315361942}) = 0
poll([{fd=3, events=POLLOUT|POLLWRNORM}], 1, 1000) = 1 ([{fd=3, revents=POLLOUT|POLLWRNORM}])
clock_gettime(CLOCK_MONOTONIC, {1803000, 423547755}) = 0
getsockopt(3, SOL_SOCKET, SO_ERROR, [0], [4]) = 0
getpeername(3, {sa_family=AF_INET, sin_port=htons(443), sin_addr=inet_addr("69.64.54.59")}, [16]) = 0
getsockname(3, {sa_family=AF_INET, sin_port=htons(51327), sin_addr=inet_addr("10.3.5.152")}, [16]) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 423965800}) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 424148309}) = 0
clock_gettime(CLOCK_MONOTONIC, {1803000, 424284143}) = 0
stat64("/dev/urandom", {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
open("/dev/urandom", O_RDONLY)          = 4
read(4, "\205h\fg\212\306\271t\301Y[\25\20\".\201=\17\232\230s\315\3217]\272\340\7\266\207\265*"..., 1024) = 1024
close(4)                                = 0
open("/dev/urandom", O_RDONLY|O_NOCTTY|O_NONBLOCK) = 4
fstat64(4, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
poll([{fd=4, events=POLLIN}], 1, 10)    = 1 ([{fd=4, revents=POLLIN}])
read(4, "\272\353\325\367M\n\316\360\23}\204\321U\275\223\264\332P6Q7\27\300\247\20\321v\313P\360\344\361", 32) = 32
close(4)                                = 0
getuid32()                              = 0
time(NULL)                              = 1302256486
clock_gettime(CLOCK_MONOTONIC, {1803000, 427442199}) = 0
time(NULL)                              = 1302256486
brk(0x8b1c000)                          = 0x8b1c000
time(NULL)                              = 1302256486
write(3, "\26\3\1\0\340\1\0\0\334\3\1M\236\333f\372|\273Lw\373\7.\232&\1$\354<\303\273\241"..., 229) = 229
read(3, 0x8af36d8, 7)                   = -1 EAGAIN (Resource temporarily unavailable)

[grondilu]

Well, if it doesn't work with curl, at least it does with wget.  See above.

However, you might need to use a SSL-compiled version.  See docs.

[purplezky]

using wget gives the same SSL problem.
The strange thing is that connecting to the api worked fine last month.
Now when i connect from a server in the USA or from Europe, they both give the same error.
That's why i suspect that either curl and wget both got updated SSL libraries with an backwards incompatible SSL handshake (highly unlikely),
or something changed in the implementation of the SSL handshake at mtgox.com (more plausible).

Code:

Setting --check-certificate (checkcertificate) to 0
Setting --proxy (useproxy) to 0
Setting --output-document (outputdocument) to -
Setting --post-data (postdata) to name=username&pass=password
DEBUG output created by Wget 1.12 on linux-gnu.

--2011-04-08 13:05:40--  https://mtgox.com/code/getFunds.php
Resolving mtgox.com... 69.64.54.59
Caching mtgox.com => 69.64.54.59
Connecting to mtgox.com|69.64.54.59|:443... connected.
Created socket 3.
Releasing 0x09a22868 (new refcount 1).
Initiating SSL handshake.
SSL handshake failed.
Closed fd 3
Unable to establish SSL connection.

[grondilu]

hum... it worked for me.

What versions of wget and libssl do you use?

Mines are:

libssl0.9.8:
  Installed: 0.9.8o-7
  Candidate: 0.9.8o-7
  Version table:
 *** 0.9.8o-7 0
        500 http://ftp.fr.debian.org/debian/ sid/main i386 Packages
        100 /var/lib/dpkg/status
     0.9.8o-5 0
        500 http://ftp.fr.debian.org/debian/ testing/main i386 Packages
wget:
  Installed: 1.12-3
  Candidate: 1.12-3
  Version table:
 *** 1.12-3 0
        500 http://ftp.fr.debian.org/debian/ sid/main i386 Packages
        100 /var/lib/dpkg/status
     1.12-2.1 0
        500 http://ftp.fr.debian.org/debian/ testing/main i386 Packages

[The Madhatter]

If it works with wget and not with curl that's because the CA mtgox used isn't in the certificate bundle that's included with curl. The CA bundle included with curl is notorious for being out of date.

If it doesn't work at all, my best guess is a misconfigured load balancer or poisoned DNS.

[grondilu]

If it works with wget and not with curl that's because the CA mtgox used isn't in the certificate bundle that's included with curl. The CA bundle included with curl is notorious for being out of date.

If it doesn't work at all, my best guess is a misconfigured load balancer or poisoned DNS.

BTW, I haven't seen any equivalent of the "--no-check-certificate" option in curl's manual page.

[The Madhatter]

BTW, I haven't seen any equivalent of the "--no-check-certificate" option in curl's manual page.

"curl -k"

[purplezky]

the gentoo packages with given USE flags, is what i used to produce the above errors

Code:

[ebuild] dev-libs/openssl-1.0.0d  USE="zlib -bindist -gmp -kerberos -rfc3779 -sse2 -test"
[ebuild] net-misc/wget-1.12-r3  USE="debug ipv6 nls ssl -idn -ntlm -static"

I now tried to compile curl with gnutls to see if it makes a difference

Code:

[ebuild  N    ] dev-libs/libtasn1-2.9-r1  USE="-doc"
[ebuild  N    ] net-libs/gnutls-2.10.5  USE="cxx nls zlib -bindist -doc -examples -guile -lzo -test"
[ebuild   R   ] net-misc/curl-7.21.4  USE="gnutls* ipv6 ssl static-libs* -ares -idn -kerberos -ldap -libssh2 -nss -test -threads"

I tried the -k option to not check certificates since curl-7.21.4 ships without any certificate bundle on gentoo.

Code:

curl -v -k -d `cat credentials` https://www.mtgox.com/code/getFunds.php
* About to connect() to www.mtgox.com port 443 (#0)
*   Trying 69.64.54.59... connected
* Connected to www.mtgox.com (69.64.54.59) port 443 (#0)
* found 142 certificates in /etc/ssl/certs/ca-certificates.crt
* gnutls_handshake() failed: A TLS packet with unexpected length was received.
* Closing connection #0
curl: (35) gnutls_handshake() failed: A TLS packet with unexpected length was received.

same handshake which fails, but a more verbose error message.

[purplezky]

ah i finally found out what was going wrong:
the server has disabled SSLv2,
and if you don't specify to use SSLv3 with curl,
it looks like it's doing a weird SSL handshake.

Code:

curl --sslv3 -d `cat credentials` https://www.mtgox.com/code/getFunds.php
{"usds":0.22,"btcs":39.74}

Thanks for all the advice.

[MagicalTux]

Hi,

Sorry for not seeing this thread sooner, I see there's a bug here (when ServerName matches the host name, TLS protocol doesn't work anymore), I fixed it for now by not providing a hostname in the vhost (and I'll try to upgrade openssl to see if it fixes the issue)


Mark

[error]

Hi,

Sorry for not seeing this thread sooner, I see there's a bug here (when ServerName matches the host name, TLS protocol doesn't work anymore), I fixed it for now by not providing a hostname in the vhost (and I'll try to upgrade openssl to see if it fixes the issue)

This needs openssl 0.9.8j or later. On both ends.

[MagicalTux]

Hi,

Sorry for not seeing this thread sooner, I see there's a bug here (when ServerName matches the host name, TLS protocol doesn't work anymore), I fixed it for now by not providing a hostname in the vhost (and I'll try to upgrade openssl to see if it fixes the issue)

This needs openssl 0.9.8j or later. On both ends.

Running openssl 1.0.0d here~

[error]

Hi,

Sorry for not seeing this thread sooner, I see there's a bug here (when ServerName matches the host name, TLS protocol doesn't work anymore), I fixed it for now by not providing a hostname in the vhost (and I'll try to upgrade openssl to see if it fixes the issue)

This needs openssl 0.9.8j or later. On both ends.

Running openssl 1.0.0d here~

Well the bug probably isn't in OpenSSL then.

[grondilu]

If I want to do this more properly, i.e. using the MtGox certificate, anyone knows how I could do?

I've seen on the web that I can retrieve the certificate with:

openssl s_client -connect www.mtgox.com:443 -showcerts

but I see several certificates (I think it's a chain or something).

Any idea?