Quickseller
Copper Member
Legendary
Offline
Activity: 2968
Merit: 2370
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 28, 2014, 01:35:27 AM |
|
What did you do to make this? How is data pulled from the forum servers?
|
|
|
|
williamj2543
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 28, 2014, 01:43:26 AM |
|
What did you do to make this? How is data pulled from the forum servers?
Well I don't know how he did it exactly, but I did an html_dom, so basically I get text info from your profile page. It finds the post count, username, activity, address, etc. rows for the table, and finds the corresponding data in the next column over. OP, if you want a more accurate fast system I could show you mine, it doesn't take any time to load. Heres my currently working system: http://cryptobets.co/bitcointalkuser.php?id=191301Thats my id, just replace it with yours after the question mark. Its more of a testing thing I did for a possible future signature campaign, and I could easily add a table or fancy graphics. Although I don't really know how the owner of the system on this thread got post history.
|
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 29, 2014, 08:06:48 PM |
|
Right, the loading time is due to process of fetching the info. I could implement it cached, loading times would be lighting fast, but the post could get off if not updated every 5-10 minutes. I think the most important is correct data, but I'll have a look at it. Currently only table3.php shows the correct data, http://talkera.org.cp-in-1.webhostbox.net/pd/table3.php?uid=334898 ), the others have to be updated yet, but updates are coming very soon. All of the requests/suggestions mentioned here will be implemented. Do you Have any news @ virtualx ?''
|
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 30, 2014, 01:28:04 AM |
|
Right, the loading time is due to process of fetching the info. I could implement it cached, loading times would be lighting fast, but the post could get off if not updated every 5-10 minutes. I think the most important is correct data, but I'll have a look at it. Currently only table3.php shows the correct data, http://talkera.org.cp-in-1.webhostbox.net/pd/table3.php?uid=334898 ), the others have to be updated yet, but updates are coming very soon. All of the requests/suggestions mentioned here will be implemented. Do you Have any news @ virtualx ?'' I'm updating the system on this local machine, this week there will be an update. I have a lot of prototypes and making much progress What did you do to make this? How is data pulled from the forum servers?
Well I don't know how he did it exactly, but I did an html_dom, so basically I get text info from your profile page. It finds the post count, username, activity, address, etc. rows for the table, and finds the corresponding data in the next column over. OP, if you want a more accurate fast system I could show you mine, it doesn't take any time to load. Heres my currently working system: http://cryptobets.co/bitcointalkuser.php?id=191301Thats my id, just replace it with yours after the question mark. Its more of a testing thing I did for a possible future signature campaign, and I could easily add a table or fancy graphics. Although I don't really know how the owner of the system on this thread got post history. Basically its using an algorithm which takes some time to load. A less detailed view would be much faster, but it would be good for signature campaign users to see the earnings per day. To improve speed the new version will use caching and be updated automatically - say ever hour or 10 minutes - so the loading time from the user side would be very short.
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
williamj2543
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 30, 2014, 01:29:37 AM |
|
How did you get the post count per day though? The forum does not supply this information. One way would to view the profile caches per day and get the post count per day, but that would take forever, wouldn't it?
|
██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 30, 2014, 01:40:33 AM |
|
How did you get the post count per day though? The forum does not supply this information. One way would to view the profile caches per day and get the post count per day, but that would take forever, wouldn't it?
The algorithm uses some tricks to get the post count. The method you mentioned is one possible method, but I'm sure there are many. The method itself doesn't matter to much - as long as the loading time is low and the data is correct. Using a cached version, it would take the user only around half a second to load everything - I guess that's the way satoshi would implement it. At the moment the code is too messy to make open source, but I might do it later.
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
July 30, 2014, 10:27:32 AM |
|
How did you get the post count per day though? The forum does not supply this information. One way would to view the profile caches per day and get the post count per day, but that would take forever, wouldn't it?
The algorithm uses some tricks to get the post count. The method you mentioned is one possible method, but I'm sure there are many. The method itself doesn't matter to much - as long as the loading time is low and the data is correct. Using a cached version, it would take the user only around half a second to load everything - I guess that's the way satoshi would implement it. At the moment the code is too messy to make open source, but I might do it later. Ok, thanks for everything. we Expect more news!
|
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 01, 2014, 08:54:41 PM |
|
Update: I got a lot of code now, some for caching, some for just collecting the data etc.. the project is growing much bigger than I initially thought. Any user can get their post count in simple JSON format now, for example: http://talkera.org.cp-in-1.webhostbox.net/pd/src/API.php?uid=348910The output in this case is ['July 30',2,0.0008], ['July 27',1,0.0004], ['July 25',6,0.0024], ['July 24',6,0.0024], ['July 23',5,0.002], ['July 21',1,0.0004], ['July 17',1,0.0004], ['July 15',2,0.0008], ['July 14',1,0.0004], ['July 08',1,0.0004], ['July 05',1,0.0004], ['June 30',1,0.0004], ['June 28',1,0.0004], ['June 27',3,0.0012],
The elements are date, posts, btc. The loading time is really slow, but anyone could put it in their own interface, or cache the data for all members. For example, this table interface uses the data: http://talkera.org.cp-in-1.webhostbox.net/pd/view.php?uid=348910![](https://ip.bitcointalk.org/?u=http%3A%2F%2Ftalkera.org.cp-in-1.webhostbox.net%2Fpd%2FPRIMEDICE.png&t=665&c=5uqHn7b-mqfZHQ) I do not have that much control of this server to cache it, and run a program every 10min-1hour to update it. And so, I'm willing to make the code open source- I'm sure an even greater project will come out of it. ![Cool](https://bitcointalk.org/Smileys/default/cool.gif) Is there any svn service these days? I've never been a big fan of git.
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 01, 2014, 09:31:58 PM |
|
Update: I got a lot of code now, some for caching, some for just collecting the data etc.. the project is growing much bigger than I initially thought. Any user can get their post count in simple JSON format now, for example: http://talkera.org.cp-in-1.webhostbox.net/pd/src/API.php?uid=348910The output in this case is ['July 30',2,0.0008], ['July 27',1,0.0004], ['July 25',6,0.0024], ['July 24',6,0.0024], ['July 23',5,0.002], ['July 21',1,0.0004], ['July 17',1,0.0004], ['July 15',2,0.0008], ['July 14',1,0.0004], ['July 08',1,0.0004], ['July 05',1,0.0004], ['June 30',1,0.0004], ['June 28',1,0.0004], ['June 27',3,0.0012],
The elements are date, posts, btc. The loading time is really slow, but anyone could put it in their own interface, or cache the data for all members. For example, this table interface uses the data: http://talkera.org.cp-in-1.webhostbox.net/pd/view.php?uid=348910![](https://ip.bitcointalk.org/?u=http%3A%2F%2Ftalkera.org.cp-in-1.webhostbox.net%2Fpd%2FPRIMEDICE.png&t=665&c=5uqHn7b-mqfZHQ) I do not have that much control of this server to cache it, and run a program every 10min-1hour to update it. And so, I'm willing to make the code open source- I'm sure an even greater project will come out of it. ![Cool](https://bitcointalk.org/Smileys/default/cool.gif) Is there any svn service these days? I've never been a big fan of git. ![Shocked](https://bitcointalk.org/Smileys/default/shocked.gif) This is fantastic , goo job ! Very good job .
|
|
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 01, 2014, 10:46:53 PM |
|
Caching should be used, which is possible if someone downloads and updates the dataset regulary. This could be done automatically using a script, and simply by collection all of the JSON data for each member. I simply don't have enough rights on the server. Now all of the algorithm runs straight, but if cached would take only 1 second ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif)
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
ranochigo
Legendary
Offline
Activity: 3038
Merit: 4418
Crypto Swap Exchange
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 02, 2014, 05:06:40 AM |
|
Caching should be used, which is possible if someone downloads and updates the dataset regulary. This could be done automatically using a script, and simply by collection all of the JSON data for each member. I simply don't have enough rights on the server. Now all of the algorithm runs straight, but if cached would take only 1 second ![Smiley](https://bitcointalk.org/Smileys/default/smiley.gif) Caching is not as accurate as live data, it takes a few minutes between each cache. They cannot cache everyone's info into their server. It would take the server to load everyone's stats, occupying unnecessary space if the user never use them.
|
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 08:55:42 PM |
|
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 08:58:33 PM |
|
Oh , very nice work .. give me your btc address I will send you a small donation !
|
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 09:06:33 PM |
|
Wow, thanks! ![Cheesy](https://bitcointalk.org/Smileys/default/cheesy.gif) My address is 125tQCeg89h8d3ccgBsipbjGUJpqx4c3H1 (just to be sure of the address, current balance is 0.000755 BTC) A small donation sent , thank you for this " web-tool" .
|
|
|
|
|
redsn0w
Legendary
Offline
Activity: 1778
Merit: 1043
#Free market
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 09:45:13 PM |
|
You have just to wait 1 minute ![Wink](https://bitcointalk.org/Smileys/default/wink.gif) .. for me work !
|
|
|
|
bigasic
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 09:55:03 PM |
|
i cant seem to get it to work, i have a mac... any advice?
|
|
|
|
virtualx (OP)
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 10:05:57 PM |
|
i cant seem to get it to work, i have a mac... any advice?
might be some browser issue, does http://talkera.org.cp-in-1.webhostbox.net/pd3/src/API.php?uid=348910 work for you? It should show something like this ['',5,0.002], ['August 01',2,0.0008], ['July 30',2,0.0008], ['July 27',1,0.0004], ['July 25',6,0.0024], ['July 24',6,0.0024], ['July 23',5,0.002], ['July 21',1,0.0004], ['July 17',1,0.0004], ['July 15',2,0.0008], ['July 14',1,0.0004], ['July 08',1,0.0004], ['July 05',1,0.0004], ['June 30',1,0.0004], ['June 28',1,0.0004], ['June 27',3,0.0012],
|
...loteo...
DIGITAL ERA LOTTERY | ║ ║ ║ | | r | ▄▄███████████▄▄ ▄███████████████████▄ ▄███████████████████████▄ ▄██████████████████████████▄ ▄██ ███████▌ ▐██████████████▄ ▐██▌ ▐█▀ ▀█ ▐█▀ ▀██▀ ▀██▌ ▐██ █▌ █▌ ██ ██▌ ██▌ █▌ █▌ ██▌ ▐█▌ ▐█ ▐█ ▐█▌ ▐██ ▄▄▄██ ▐█ ▐██▌ ▐█ ██▄ ▄██ █▄ ██▄ ▄███▌ ▀████████████████████████████▀ ▀██████████████████████████▀ ▀███████████████████████▀ ▀███████████████████▀ ▀▀███████████▀▀
| r | | ║ ║ ║ | RPLAY NOWR
BE A MOON VISITOR! |
[/center]
|
|
|
DannyElfman
|
![](https://bitcointalk.org/Themes/custom1/images/post/xx.gif) |
August 05, 2014, 10:07:56 PM |
|
Very cool gadget! Will keep using it in the future.
|
This spot for rent.
|
|
|
|