Bitcoin Forum
May 04, 2024, 10:58:38 PM *
News: Latest Bitcoin Core release: 27.0 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 3 [All]
  Print  
Author Topic: Ripple API  (Read 3744 times)
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 04, 2013, 08:48:28 PM
 #1

Ok, so I have been hard at understanding the whole ripple concept and it has spawned some neat ideas. As a coder I tend to try and understand frameworks in as much of a hands on approach as possible, so I thought I'd ask those who are in the know how to go about this.

The history and all transactions are public, so I could parse my way through said history to get to the information I need, but is there a simpler way? I want to know, for a specific currency and issuer, who is holding IOUs and how much each wallet has, say I want to know every wallet that has Bitstamp BTCs and how many of them they have.
The forum was founded in 2009 by Satoshi and Sirius. It replaced a SourceForge forum.
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction.
1714863518
Hero Member
*
Offline Offline

Posts: 1714863518

View Profile Personal Message (Offline)

Ignore
1714863518
Reply with quote  #2

1714863518
Report to moderator
1714863518
Hero Member
*
Offline Offline

Posts: 1714863518

View Profile Personal Message (Offline)

Ignore
1714863518
Reply with quote  #2

1714863518
Report to moderator
moocowpong1
Member
**
Offline Offline

Activity: 79
Merit: 10


View Profile
March 04, 2013, 08:56:40 PM
 #2

I think rippled does a lot of what you want, but you can't play with it yet. There's a lot information on the wiki about that sort of thing, although I haven't taken the time to look at it closely.

If you want an example ledger to see how the information is stored, JoelKatz posted one here: https://bitcointalk.org/index.php?topic=148278.msg1573763#msg1573763
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 04, 2013, 10:37:38 PM
 #3

Thank you for the pointer. Turns out this initial request I made was right in front of my face and I just failed to see it; https://ripple.com/wiki/RPC_API#account_lines

I still have to filter the currencies and there's no batching which may make things hard with huge ledgers, but it is there and is real easy.
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 02:47:41 PM
 #4

In trying to jump in to find out why I'm having issues and which paths are being involved, I tried calling the ripple_path_find RPC command, according to the instructions on the wiki https://ripple.com/wiki/RPC_API#ripple_path_find but it is failing complaining about invalid arguments:

Request:
{"destination_amount": "0.05", "source_currencies": [{"currency": "BTC"}], "command": "ripple_path_find", "source_account": "rKLw2zHgA31nRZN3erfZkfR8uhAkQ8s95P", "destination_account": "rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt"}

Response:
{u'status': u'error', u'error_code': 23, u'error_message': u'Invalid parameters.', u'request': {u'destination_amount': u'0.05', u'source_currencies': [{u'currency': u'BTC'}], u'command': u'ripple_path_find', u'source_account': u'rKLw2zHgA31nRZN3erfZkfR8uhAkQ8s95P', u'destination_account': u'rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt'}, u'error': u'invalidParams', u'type': u'response'}

Any clues into what I'm doing wrong?
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 05, 2013, 03:07:14 PM
 #5

Your amount format is wrong. Try this:

{
    "command" : "ripple_path_find",
    "source_account": "rKLw2zHgA31nRZN3erfZkfR8uhAkQ8s95P",
    "destination_account": "rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt",
    "destination_amount": {
        "currency": "BTC",
        "issuer": "rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt",
        "value": "0.05"
    }
}

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 03:18:25 PM
 #6

Thank you, Joel. It works, so please note that the wiki is wrong here Smiley

No on to parsing the result...
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 03:31:32 PM
 #7

Code:
{u'paths_computed':
  [
    [
      {u'currency': u'BTC', u'account': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B'},
      {u'currency': u'BTC', u'account': u'rh7eAu9SrFsE2fVfc1PJTs1WYDHGAUde6V', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rh7eAu9SrFsE2fVfc1PJTs1WYDHGAUde6V'},
      {u'currency': u'BTC', u'account': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E'}
    ],
    [
      {u'currency': u'BTC', u'account': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B'},
      {u'currency': u'BTC', u'account': u'rh7eAu9SrFsE2fVfc1PJTs1WYDHGAUde6V', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rh7eAu9SrFsE2fVfc1PJTs1WYDHGAUde6V'},
      {u'currency': u'BTC', u'account': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E'}
    ],
    [
      {u'currency': u'BTC', u'account': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B'},
      {u'currency': u'BTC', u'account': u'rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW'},
      {u'currency': u'BTC', u'account': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E', u'type': 49,
       u'type_hex': u'0000000000000031', u'issuer': u'rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E'}
    ]
  ],
 u'source_amount': {u'currency': u'BTC', u'value': u'0.0501', u'issuer': u'rKLw2zHgA31nRZN3erfZkfR8uhAkQ8s95P'},
 u'paths_canonical': []}

So I guess I'm getting 3 potential credit lines that could be used, but the first 2 are exactly the same, are they now? Can you help me understand what I'm looking at above (a pointer to the documentation is fine)?

FYI this is one case where I would assume the transfer should be possible but it fails with the partial path error.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 05, 2013, 05:34:06 PM
 #8

So I guess I'm getting 3 potential credit lines that could be used, but the first 2 are exactly the same, are they now? Can you help me understand what I'm looking at above (a pointer to the documentation is fine)?
Duplicate paths are caused by a bug. We've been working on pathfinding heavily for the past week. Expect things to get much better over the next week or so.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 07:07:16 PM
 #9

So the 3 paths I see are actually 2 paths and are alternative ways of proceeding with the transfer? Could you take the first one and convert it to an english sentence?
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 05, 2013, 07:21:43 PM
 #10

So the 3 paths I see are actually 2 paths and are alternative ways of proceeding with the transfer? Could you take the first one and convert it to an english sentence?
In this case, the paths are alternative ways of performing the transfer. In a more complex case, the paths may be needed together to accomplish the full transfer.

Supplying multiple paths also ensures that the transaction succeeds even if the state has changed between when the transaction is formed and when it is processed. When the final transaction is executed, the paths are taken incrementally to ensure the best rate (from those paths) is given. The rate can change as IOUs are pushed through a path for two reasons: 1) If the path includes an order book, you can exhaust the best offer and have to move to an inferior offer. 2) If the path switches from redeeming IOUs to issuing IOUs, the rate can change. (You have to accept your own IOUs at face value, but you can devalue someone else's.)

To explain 2 in a little more detail, say I hold your IOUs and payment path goes from me to you. So long as I hold your IOUs, I can give them to you and you must honor them at face value. However, when I run out of IOUs, if you have extended credit to me, the path can still work with you getting my IOUs instead of me giving you yours. This can result in a different rate because the issuer of the IOUs and the direction they move has changed.

Quote
                 {
                     "account" : "rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B",
                     "currency" : "BTC",
                     "issuer" : "rvYAfWj5gh67oV6fW32ZzP3Aw4Eubs59B",
                     "type" : 49,
                     "type_hex" : "0000000000000031"
                  },
                  {
                     "account" : "rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW",
                     "currency" : "BTC",
                     "issuer" : "rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW",
                     "type" : 49,
                     "type_hex" : "0000000000000031"
                  },
                  {
                     "account" : "rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E",
                     "currency" : "BTC",
                     "issuer" : "rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E",
                     "type" : 49,
                     "type_hex" : "0000000000000031"
                  }
This is an expanded path showing things like currencies even when they don't change. It's a rippling path with no exchanges involved, all in the same currency. The first node is Bitstamp. So the sender is going to return Bitstamp IOUs to Bitstamp and in return Bitsamp will issue IOUs to rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW. In exchange for the Bitstamp IOUs, rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW will issue their own IOUs to rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E, who will in turn issue their own IOUs to the recipient.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 11:41:46 PM
 #11

In this case, the paths are alternative ways of performing the transfer. In a more complex case, the paths may be needed together to accomplish the full transfer.

Supplying multiple paths also ensures that the transaction succeeds even if the state has changed between when the transaction is formed and when it is processed. When the final transaction is executed, the paths are taken incrementally to ensure the best rate (from those paths) is given. The rate can change as IOUs are pushed through a path for two reasons: 1) If the path includes an order book, you can exhaust the best offer and have to move to an inferior offer. 2) If the path switches from redeeming IOUs to issuing IOUs, the rate can change. (You have to accept your own IOUs at face value, but you can devalue someone else's.)

To explain 2 in a little more detail, say I hold your IOUs and payment path goes from me to you. So long as I hold your IOUs, I can give them to you and you must honor them at face value. However, when I run out of IOUs, if you have extended credit to me, the path can still work with you getting my IOUs instead of me giving you yours. This can result in a different rate because the issuer of the IOUs and the direction they move has changed.


In 1) will order books ever be used automatically, i.e. if I send USD but only have BTC, or is that limited to when dealing directly with orders?

As for 2) that is pretty smart, I must say. It does not apply to the case where both sides of a trade trust a common exchange but not one another, right?
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 05, 2013, 11:49:50 PM
 #12

This is an expanded path showing things like currencies even when they don't change. It's a rippling path with no exchanges involved, all in the same currency. The first node is Bitstamp. So the sender is going to return Bitstamp IOUs to Bitstamp and in return Bitsamp will issue IOUs to rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW. In exchange for the Bitstamp IOUs, rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW will issue their own IOUs to rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E, who will in turn issue their own IOUs to the recipient.

So rKLw2zHgA31nRZN3erfZkfR8uhAkQ8s95P wants to send to rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt. rKLw returns the IOUs it holds from Bitstamp and Bitstamp issues a new set of IOUs to rhR5j5LXsujSuwG2bcn1P39utdTA79ceaW. This in turn issues own IOUs to rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E which issues own IOUs to rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt.

Why the extra hops if rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt trusts Bitstamp? And these IOUs from rGwUWgN5BEg3QGNY3RX2HfYowjUTZdid3E, are they still Bitstamp IOUs as far as the recipient is concerned?

It might just be late here, but I fail to understand why this "complicated" path would ever need to take place, unless there's already an outstanding balance between the multiple nodes that would get settled with this transaction.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 05, 2013, 11:50:14 PM
 #13

In 1) will order books ever be used automatically, i.e. if I send USD but only have BTC, or is that limited to when dealing directly with orders?
Order books will always be used when a change of currency is required since that's the only way to change currencies. Otherwise, order books will be used where available. For example, if you hold bitstamp BTC IOUs and are trying to pay someone who only accepts WeExchange BTC IOUs, it will try to ripple through an account, but it will also look for both a direct order book between the two IOUs and an indirect path through two order books.

Quote
As for 2) that is pretty smart, I must say. It does not apply to the case where both sides of a trade trust a common exchange but not one another, right?
(I assume by "exchange" you mean an issuer, not a change in currencies.) Assuming the issuer hasn't issued the first one credit, then it cannot apply. (And gateways don't usually extend credit to anyone.) It can only apply when you are returning IOUs to someone who has extended you credit. At first you can give them their own IOUs and then you can issue your own IOUs.

The network doesn't currently handle all of these cases quite as well as it could when finding paths. We are working on this now.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 06, 2013, 12:01:34 AM
 #14


Quote
As for 2) that is pretty smart, I must say. It does not apply to the case where both sides of a trade trust a common exchange but not one another, right?
(I assume by "exchange" you mean an issuer, not a change in currencies.) Assuming the issuer hasn't issued the first one credit, then it cannot apply. (And gateways don't usually extend credit to anyone.) It can only apply when you are returning IOUs to someone who has extended you credit. At first you can give them their own IOUs and then you can issue your own IOUs.

The network doesn't currently handle all of these cases quite as well as it could when finding paths. We are working on this now.


Yes, issuer, still getting used to the semantics here. The cases that are failing for me (including the one I pasted the RPC call for) are all V shaped. Both ends trust one 3rd party issuer, none trust the other, so I am trying to move Bitstamp IOUs from walletA to walletB, where none of the wallets has extended credit to the other, but both have for Bitstamp.

If walletB had extended credit to walletA, the transfer would succeed but instead of sending the Bitstamp IOUs walletA would issue its own IOUs to walletB. This is all fine, except that having done that I was then not capable redeeming walletA's IOU from walletB to Bitstamp.

I am always working under the assumption that both the V shaped transfer and the latter paragraph example (full circle?) are legal and are only failing due to something wrong with the found paths. The specific error 'Path could not send full amount.' (tecPATH_PARTIAL) appears to say that there aren't enough IOUs in the path to move the full amount transfered, which could be easily solved by taking the shortest path of no hops other than the wallets involved + the issuer, but I haven't looked at the actual code, so this is just a wild guess.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 06, 2013, 12:26:06 AM
 #15

I am always working under the assumption that both the V shaped transfer and the latter paragraph example (full circle?) are legal and are only failing due to something wrong with the found paths. The specific error 'Path could not send full amount.' (tecPATH_PARTIAL) appears to say that there aren't enough IOUs in the path to move the full amount transfered, which could be easily solved by taking the shortest path of no hops other than the wallets involved + the issuer, but I haven't looked at the actual code, so this is just a wild guess.
We just identified a bug that could cause the tecPATH_PARTIAL case. It could also affect the other cases. I'll add some paths as you describe to my test cases.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
Arvicco
Hero Member
*****
Offline Offline

Activity: 574
Merit: 501


Please bear with me


View Profile WWW
March 06, 2013, 08:41:47 PM
 #16

Joel,

Is there any documentation (other than the source) describing the keypairs and signing? 

I'm mostly interested in constructing/signing a raw transaction locally without JavaScript, and then transmitting it as a tx_blob via 'submit' API. The other way of submitting a transaction with tx_json and secret on the wire is REALLY unsafe.

JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 06, 2013, 09:33:55 PM
 #17

Joel,

Is there any documentation (other than the source) describing the keypairs and signing? 

I'm mostly interested in constructing/signing a raw transaction locally without JavaScript, and then transmitting it as a tx_blob via 'submit' API. The other way of submitting a transaction with tx_json and secret on the wire is REALLY unsafe.
The JavaScript client now has code to build a raw binary transaction and sign it locally. Secrets are no longer sent over the wire.

There is documentation on the binary format in the wiki, but it's kind of hard to get a high-level overview from there.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
ahbritto
Full Member
***
Offline Offline

Activity: 132
Merit: 100


Ripple


View Profile WWW
March 06, 2013, 09:39:43 PM
 #18

We have this:

* https://npmjs.org/package/ripple-lib

We push it irregularly. It is undocumented. It is subject to change without notice. It does local signing.

It should be pretty easy to make a command line interface with it.

More information here:

* https://ripple.com/wiki/Ripple_JavaScript_library
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 07, 2013, 11:29:36 AM
 #19

Looking at account_tx now, trying the full range of ledger indexes (0 to 323284 currently) I get 6 results back, all of them failed attempts at sending (the tecPARTIAL_PATH I've reported). There are 4 events in the account history (receiving XRP, receiving BTC and sending a custom currency) that aren't referenced at all in account_tx. Is this expected?

I assumed account_tx would be a filtered version of tx_history, but I may be confusing the command semantics.
Arvicco
Hero Member
*****
Offline Offline

Activity: 574
Merit: 501


Please bear with me


View Profile WWW
March 07, 2013, 08:27:52 PM
 #20

What is the date format used in Ripple API? For example, account_tx returns "tx":  {"date":415507200}

Wiki says Unix epoch but it can't be right. 415507200 sec since Unix = 1983-03-03 07:17:40. Must be some offset or something?

JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 07, 2013, 08:40:21 PM
 #21


Ripple network times are seconds since 1/1/2000, 00:00:00 UTC.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
Arvicco
Hero Member
*****
Offline Offline

Activity: 574
Merit: 501


Please bear with me


View Profile WWW
March 07, 2013, 08:45:18 PM
 #22


Ripple network times are seconds since 1/1/2000, 00:00:00 UTC.


Thanks, you need to update https://ripple.com/wiki/JSON_API then:

time: An unsigned integer. Number of seconds since the UNIX epoch: Janurary 1, 1970.

JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 08, 2013, 02:34:26 AM
 #23

Yeah. Sorry about that. We considered changing it to seconds since the UNIX Epoch or text times, but that would make things awkward when we're reading JSON. (We'd have to know if an integer was a time in order to know whether to apply the offset.)

The documentation has been fixed.


I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
Arvicco
Hero Member
*****
Offline Offline

Activity: 574
Merit: 501


Please bear with me


View Profile WWW
March 08, 2013, 07:31:29 PM
 #24


Ripple network times are seconds since 1/1/2000, 00:00:00 UTC.


BTW, your timestamp formats are inconsistent across events. Such as, "ledgerClosed" event has its timestamp in Unix epoch, rather than Ripple epoch. This is confusing.

JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 08, 2013, 09:16:51 PM
 #25

Nice catch. I'll check to make sure it won't break anything and see if it can be changed to be consistent with all the other times. The "ledger" command also reports close times like "2013-Mar-08 21:16:10", which is a bit annoying (I don't even know if that's local time or UTC, *sigh*). We wanted the JSON format to be mostly human comprehensible, but times get ugly because we don't always know that something is a time when it's being parsed. (And the JSON output and input formats are supposed to be compatible.)

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 10, 2013, 11:28:04 PM
 #26

For my current use case I need a polling approach to following up transactions (as opposed to subscribing to events). This is all fine for payments received but I can't see any reference to payments sent from the source account. I can in fact see the transaction received on the destination.

Is this by design? Shouldn't payments sent also appear in account_tx?
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 12:24:31 AM
 #27

For my current use case I need a polling approach to following up transactions (as opposed to subscribing to events). This is all fine for payments received but I can't see any reference to payments sent from the source account. I can in fact see the transaction received on the destination.

Is this by design? Shouldn't payments sent also appear in account_tx?
Payments sent should appear in account_tx. Perhaps you're trying to query for too many transactions at once? The limit is currently 200 transactions in most cases. The transactions appear with the most recent transactions first, that might be throwing you.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 12:49:04 AM
 #28

Nope, I'm asking for 2 tx only. I'll send you the exact query tomorrow (work computer already off for today).
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 09:35:11 AM
 #29

@JoelKatz it appears this only happens agains my local rippled, so something must have gone wrong with it. I'm recompiling to the latest, but if it keeps happening is there something you'd want me do before I nuke everything out and start from scratch?
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 09:54:11 AM
 #30

@JoelKatz it appears this only happens agains my local rippled, so something must have gone wrong with it. I'm recompiling to the latest, but if it keeps happening is there something you'd want me do before I nuke everything out and start from scratch?
By default, the server only acquires 256 back ledgers. You can set the history higher if you want (search the config file for "ledger_history"). If you don't have the ledger a transaction is in, you can't provide the transaction to clients who ask. If you look in the 'server_info' output, you'll see a field called 'complete_ledgers'. This is the range of ledgers your server has fully indexed and locally available.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 10:05:50 AM
 #31

@JoelKatz it appears this only happens agains my local rippled, so something must have gone wrong with it. I'm recompiling to the latest, but if it keeps happening is there something you'd want me do before I nuke everything out and start from scratch?
By default, the server only acquires 256 back ledgers. You can set the history higher if you want (search the config file for "ledger_history"). If you don't have the ledger a transaction is in, you can't provide the transaction to clients who ask. If you look in the 'server_info' output, you'll see a field called 'complete_ledgers'. This is the range of ledgers your server has fully indexed and locally available.

Sure, I'm kind of, sort of aware of that (I didn't know to look for complete_ledgers) but since I get one half of it, the received payment tx, I should have the ledger, right?

This does present a separate issue, though; how much is enough? If I start the server with history to 256, does that mean it prunes history to last 256 while running, or just that on startup it requests only the past 256 ledgers (if they aren't already local).
I guess my question is how to properly handle long periods between polling the server for new tx.
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 10:39:11 AM
 #32

"complete_ledgers" : "339007-341774"

source @f3b837eea654deec3572a89a000f2b6e13156f6a, same issue. I'll nuke and start afresh, probably something went sideways with my deploy.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 11:02:18 AM
 #33

Are you saying there's a transaction you know was placed in a ledger inside the range of completed ledgers you have and account_tx isn't showing it? Or are you saying your server isn't acquiring a ledger it's supposed to acquire?

You can use account_tx against our server to get the 'inLedger' field to see what ledger a transaction was included in. Then check your server to see if it has that ledger. If it doesn't, make sure it's configured such that it should. If it should have it but doesn't check server_info every few minutes to see if it's still acquiring ledgers.


I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 11:06:30 AM
 #34

I have nuked the db and changed ledger_history to 65536, now waiting for rippled to catch up with the ledger in question.

What was happening was: I had the ledger (339675), a 'rippled ledger 339675' would show it (in full default with 'full'), and then:

./rippled account_tx rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt 339675 339675
would show the transaction

./rippled account_tx rfYv1TXnwgDDK4WQNbFALykYuEBnrR4pDX 339675 339675
would not (but not with a 'ledger not found' error, should wouldn't show any tx detail).

./rippled account_tx rfYv1TXnwgDDK4WQNbFALykYuEBnrR4pDX 339675
./rippled account_tx rLDDq8ZGhCaVBP8MyYTifhCJzgWPQBjwpt 339675
Both the above commands would also not show errors nor anything about the tx.

Running any of these commands against s1.ripple.com does work as expected, so it is something local to me.
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 11:13:48 AM
 #35

Same difference,

./rippled account_tx rfYv1TXnwgDDK4WQNbFALykYuEBnrR4pDX 339675 339675

Connecting to: 127.0.0.1:5005
{
   "result" : {
      "account" : "rfYv1TXnwgDDK4WQNbFALykYuEBnrR4pDX",
      "status" : "success"
   }
}

Oh, well, gona build from a fresh checkout now
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 11:49:14 AM
 #36

This could potentially happen if you wiped your transaction database but kept your ledger database. The ledger database would tell the server that it had processed the ledger and didn't need to re-process it. The 'account_tx' command uses part of the transaction database to find all transactions that affected the selected account.

Data scrubbing is on our road map. This will also allow us to remove old data that's no longer referenced, if desired. (This is not as simple as it sounds because we know when we first fetched data, but whether that data is still in use or not in use, it sits there just the same. So we'd need to do background scrubbing to determine when we stopped needing it.)

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 02:56:58 PM
 #37

I removed the whole db directory, so I'm sure that's not it. I have now built from a clean checkout, started without any database, ledger_history is 65536 and... it still happens

If I issue the same account_tx to s1.ripple.com it returns the correct data, but against my local ripple it gives me nothing (for the source address, it is correct for the destination address). I have a small debug log that I will provide you, but it gives you little to nothing from what I could read.

Can you tell me the exact revision that is running on s1.ripple.com? Are there other public servers I can try the account_tx command against?
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 03:45:44 PM
 #38

I removed the whole db directory, so I'm sure that's not it. I have now built from a clean checkout, started without any database, ledger_history is 65536 and... it still happens
You waited until your server reported that it had the ledger? That's the only thing I can think of. I'll look over the code just in case there's something I might be missing.

Quote
Can you tell me the exact revision that is running on s1.ripple.com? Are there other public servers I can try the account_tx command against?
It's the head of the git repository as of this morning:

commit 92318a47bd59659e2dbfde174b14893ee2fe4afa
Date:   Sun Mar 10 19:15:28 2013 -0700

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 05:16:43 PM
 #39

You waited until your server reported that it had the ledger? That's the only thing I can think of. I'll look over the code just in case there's something I might be missing.
Yes, I did. Had I not done it there would be an error 'No such ledger'. It also dawned on me I'm using a very low end 32bit machine for this, not all that much ram also. I will try to run this on a separate system that has proper server specs.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 11, 2013, 10:34:43 PM
 #40

Yes, I did. Had I not done it there would be an error 'No such ledger'.
I don't think account_tx will ever give you "no such ledger". You can query for transactions from ledger zero to ledger one million and it will give you the latest 200. (Arguably, it probably should tell you which ledgers it has in the reply, but you can always check, it's published to the "server" stream.)

Quote
It also dawned on me I'm using a very low end 32bit machine for this, not all that much ram also. I will try to run this on a separate system that has proper server specs.
We don't test extensively on 32-bit machines. 1GB should be sufficient. If you have less than 1.5GB, leave your "node_size" at "small" (the default). This will cause the system to use less RAM in exchange (mostly) for more disk I/O.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 11, 2013, 10:37:04 PM
 #41

Yes, I did. Had I not done it there would be an error 'No such ledger'.
I don't think account_tx will ever give you "no such ledger". You can query for transactions from ledger zero to ledger one million and it will give you the latest 200. (Arguably, it probably should tell you which ledgers it has in the reply, but you can always check, it's published to the "server" stream.)

Oh, but it does, at least if you do "account_tx account ledger" for a specific ledger id (as opposed to a range).

Still haven't had a chance to try this over a better machine but I'll report on that once I get it done.
JoelKatz
Legendary
*
Offline Offline

Activity: 1596
Merit: 1012


Democracy is vulnerable to a 51% attack.


View Profile WWW
March 12, 2013, 12:27:42 AM
 #42

Oh, but it does, at least if you do "account_tx account ledger" for a specific ledger id (as opposed to a range).
You're correct. It's kind of a quirk in the code. If you specify a ledger range, special code for ranges of ledgers handles it. If you specify one specific ledger, generic code that fails if the ledger isn't available handles it.

I am an employee of Ripple. Follow me on Twitter @JoelKatz
1Joe1Katzci1rFcsr9HH7SLuHVnDy2aihZ BM-NBM3FRExVJSJJamV9ccgyWvQfratUHgN
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 12, 2013, 08:35:06 PM
 #43

Tried on a new, much better system, same difference. Opened an issue with the command outputs and config file.
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 13, 2013, 09:35:27 AM
 #44

Oh, but it does, at least if you do "account_tx account ledger" for a specific ledger id (as opposed to a range).
You're correct. It's kind of a quirk in the code. If you specify a ledger range, special code for ranges of ledgers handles it. If you specify one specific ledger, generic code that fails if the ledger isn't available handles it.

@JoelKatz with the db fix mentioned in the issue I opened everything works great, thank you!

To infinity and beyond!

Now, a small doubt on workflow; I submit a payment for which no trust line exists, I assume this transaction never gets submitted to a ledger, right? At least I could not find my test one. The issue I'm having is one of atomicity, where I submit a tx and note the request ID to match the response, but say between send and response my ws dies, and don't get this response at all.

Can I poll for it afterwards? Or is the assumption that if I don't see it in a ledger for a set period of time I can just resubmit the request? The latter feels dangerous if the network is busy and lags a bit.
ahbritto
Full Member
***
Offline Offline

Activity: 132
Merit: 100


Ripple


View Profile WWW
March 13, 2013, 10:30:48 AM
 #45

Now, a small doubt on workflow; I submit a payment for which no trust line exists, I assume this transaction never gets submitted to a ledger, right? At least I could not find my test one. The issue I'm having is one of atomicity, where I submit a tx and note the request ID to match the response, but say between send and response my ws dies, and don't get this response at all.

Can I poll for it afterwards? Or is the assumption that if I don't see it in a ledger for a set period of time I can just resubmit the request? The latter feels dangerous if the network is busy and lags a bit.

If a transaction could possibly apply, it is forwarded to the rest of the network. If a trust line doesn't exist, that won't stop a transaction from propagating because another transaction could create the line.

Take a look at this: https://ripple.com/wiki/Robustly_submitting_a_transaction

This describes how to submit transactions in an idempotent way. Generally, you want to build and store your transactions before you send them. Then you can send them as many times as you want. If you are coming back after a power failure, you could look for transactions that have been applied to your account since a specific ledger.
nelisky (OP)
Legendary
*
Offline Offline

Activity: 1540
Merit: 1001


View Profile
March 13, 2013, 11:12:29 AM
 #46

Now, a small doubt on workflow; I submit a payment for which no trust line exists, I assume this transaction never gets submitted to a ledger, right? At least I could not find my test one. The issue I'm having is one of atomicity, where I submit a tx and note the request ID to match the response, but say between send and response my ws dies, and don't get this response at all.

Can I poll for it afterwards? Or is the assumption that if I don't see it in a ledger for a set period of time I can just resubmit the request? The latter feels dangerous if the network is busy and lags a bit.

If a transaction could possibly apply, it is forwarded to the rest of the network. If a trust line doesn't exist, that won't stop a transaction from propagating because another transaction could create the line.

Take a look at this: https://ripple.com/wiki/Robustly_submitting_a_transaction

This describes how to submit transactions in an idempotent way. Generally, you want to build and store your transactions before you send them. Then you can send them as many times as you want. If you are coming back after a power failure, you could look for transactions that have been applied to your account since a specific ledger.

Of course, I knew that from my first read of the documentation... thank you for pointing out just how spread I am Smiley
Pages: 1 2 3 [All]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!