Bitcoin Forum
February 28, 2020, 08:18:54 AM *
News: Latest Bitcoin Core release: 0.19.0.1 [Torrent]
 
   Home   Help Search Login Register More  
Pages: 1 2 3 [All]
  Print  
Author Topic: "Show All" on long topics  (Read 1330 times)
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
February 20, 2019, 09:44:55 PM
Last edit: November 18, 2019, 02:45:59 PM by LoyceV
Merited by xandry (5), Welsh (5), malevolent (4), suchmoon (4), Crypto-DesignService (3), bones261 (2), BitMaxz (2), TMAN (2), OgNasty (1), vapourminer (1), fronti (1), Daniel91 (1), goaldigger (1), qwertyup23 (1), iasenko (1), taikuri13 (1), Coolcryptovator (1)
 #1

The reason
This topic: Thread printing got me thinking:
"Show all" is so convenient to CTRL-F something.

Scraper
I made a script that scrapes all posts from a thread, and uploads it as one page to loyce.club/showall/

How to use it
1. Get the topic ID from the URL. I'll use 5102296 as an example.
2. Post the following link, but replace the topic ID by the one you want: http://loyce.club/showall/5102296.html on any public board, so my scraper can find it. This has to be a new post, edits are ignored.
3. Wait a bit (about 10 seconds + 5 seconds per page)
4. Click the link you created
5. Feel free to delete your post again

The link created at #2 is not an archive: any new request will overwrite it. If you want, you can archive it.

Bugs
Probably, it was a quick job Cheesy Please post them here.

Limitations
I scrape a maximum of 500 pages per topic (sorry Wall Observer). I can increase this if needed, but this is probably enough to kill your browser already.
During scraping, all new requests are ignored. I didn't account for multitasking. If someone else occupies my scraper, you'll have to post your request again a bit later.
There's an additional 2 minute delay between requests.
Don't quote this post!



See [overview] LoyceV's useful data on Bitcointalk for more of my forum-related topics

AWARD-WINNING
CRYPTO CASINO
ASKGAMBLERS
PLAYERS CHOICE 2019
PROUD
PARTNER OF
1500+
GAMES
2 MIN
CASH-OUTS
24/7
SUPPORT
100s OF
FREE SPINS
PLAY NOW
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1582877934
Hero Member
*
Offline Offline

Posts: 1582877934

View Profile Personal Message (Offline)

Ignore
1582877934
Reply with quote  #2

1582877934
Report to moderator
1582877934
Hero Member
*
Offline Offline

Posts: 1582877934

View Profile Personal Message (Offline)

Ignore
1582877934
Reply with quote  #2

1582877934
Report to moderator
theymos
Administrator
Legendary
*
Offline Offline

Activity: 3668
Merit: 7581


View Profile
February 20, 2019, 10:57:20 PM
Merited by Welsh (1), LoyceV (1), BitMaxz (1), DireWolfM14 (1)
 #2

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
DireWolfM14
Copper Member
Hero Member
*****
Offline Offline

Activity: 686
Merit: 1404


https://bitcoin.watfordfc.com


View Profile WWW
February 20, 2019, 11:23:42 PM
 #3

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

That's good to know, I was not aware of that.  The "show all" feature is cool, but it often triggers the cloudflare page.

  ▄▄█████▄▄███████▄▄
 ███████████
     ▀▀███▄
█████████████        ▀██▄
█████████████          ██▄
███████████            ██▄
██▀▀█████▀▀              ██
██                       ██
██                       ██
▀██                     ██▀
 ▀██                   ██▀
  ▀██▄               ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
███████   INDUSTRY LEADING CRYPTO SPORTSBOOK   ███████
MULTI
CURRENCY
ONLINE
   CASINO  
DAILY PRICE
BOOSTS
FAST & SECURE
PAYMENTS
█████████████████████████
███████▀▀       ▀▀███████
████▀   ▄ ▀███▀ ▄   ▀████
███  ▄████▄ ▀ ▄████▄  ███
██  ▄ ▀███▀ ▄ ▀███▀ ▄  ██
█  ▄██ ▀▀ ▄███▄ ▀▀ ██▄  █
█  █▀ ▄█ ███████ █▄ ▀█  █
█   ▄███▄ █████ ▄███▄   █
██  ████▀ ▄▄▄▄▄ ▀████  ██
███  ▀ ▄ ▀█████▀ ▄ ▀  ███
████▄  ▀▀▄ ███ ▄▀▀  ▄████
███████▄▄       ▄▄███████
█████████████████████████
█████████████████████████
███████▀▀ █████ ▀▀███████
████▀    ▄█████▄    ▀████
█████▄▄█▀▀ ▄▄▄ ▀▀█▄▄█████
██▀███▀ ▄███▀███▄ ▀███▀██
█   █ ▄██▀     ▀██▄ █   █
█   █ ██         ██ █   █
█   █ ▀██▄▄█ █▄▄██▀ █   █
██▄███▄ ▀██▄▄▄██▀ ▄███▄██
█████▀▀█▄▄ ▀▀▀ ▄▄█▀▀█████
████▄    ▀█████▀    ▄████
███████▄▄ █████ ▄▄███████
█████████████████████████
.
.REGISTER NOW!.
Report To
eldrin
Full Member
***
Offline Offline

Activity: 644
Merit: 138



View Profile WWW
February 21, 2019, 12:40:45 AM
 #4

This is a great alternative, thank you! This is good for what you're looking for, search through thread for some words, for example. I don't know if it would be possible, but it would be better if the page would end up into something like this, the returned output would just be the user and the content of the post, it would not just be a printer-friendly output but also spreadsheet-friendly one. Apologies for being demanding Cheesy

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
February 21, 2019, 10:27:50 AM
 #5

if you use the search box in the upper right while in a topic, it searches only that topic.
It's so obvious, and yet I don't think I've ever used it. Thanks! Shame on me Tongue

it would be better if the page would end up into something like this
Agreed! If this is going to be used, I'll change it. However, nobody has tested it yet. Come on, don't be shy Tongue

r1s2g3
Sr. Member
****
Offline Offline

Activity: 658
Merit: 388


I am alive but in hibernation.


View Profile
February 21, 2019, 12:02:02 PM
Merited by YOSHIE (1)
 #6

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

But still below feature need lot of improvement.


I am not able to find things when I try to search in my own/someone post history.
Nor did it look search button is effective when you search on the pages like this:
https://bitcointalk.org/index.php?action=profile;u=1446224;sa=showPosts

I am alive
Xal0lex
Staff
Legendary
*
Offline Offline

Activity: 924
Merit: 1152


https://blockchain.com.do


View Profile
February 21, 2019, 01:50:39 PM
 #7

Maybe I'm doing something wrong?



I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0


██   ██   ██████████

  ██   █████████████

   ██   ████████████

 ██   ██   █████████

   ██   ████████████

      ██   █████████
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Blockchain.com.do.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████
      ▄▄▄▄▄▄▄▄▄▄▄▄▄
▀ ▀ ▀▀█   █       █▄
 ▀ ▀▀▀█▄▄▀      ▄█ ▄▀█▄
   ▀ ▀█▄▄       ██ ▄▀██▀▄
  ▀ ▀▀█  ▀▄      ▀▄▄█▀   ▀▄
 ▀▀ ▀ █▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄█▄
 ▄▄▄▄▄▄▄▄▄▄▄      █▄▄▄▄▄▄▄▄▄█
 █▄▄▄▄▄▄▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄
  ▀▄     ▄▄▄       █   █▀▀ ▀ ▀
    ▀▄ ▄█ ▄▄█▄      ▀▄▄█▀▀▀ ▀
      ▀██▄▄ ██       ▄▄█▀ ▀
        ▀▄▄▄▀      ▄▀  █▀▀ ▀
          ▀▄▄▄▄▄▄▄▄█▄▄▄█ ▀ ▀▀
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Exchange Bitcoin Quickly.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████




██████████   ██   ██

█████████████   ██

████████████   ██

█████████   ██   ██

████████████   ██

█████████   ██
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
February 21, 2019, 01:58:07 PM
Last edit: February 23, 2019, 09:37:39 PM by LoyceV
 #8

Maybe I'm doing something wrong?
You'll have to post the link you're trying to see on the forum first. Not the link to the topic, but the link to loyce.club.

Quote
That's the wrong link, if you post http://loyce.club/showall/994018.html it works.
Update: it's still scraping. It's done, but it totally fails on Russian Sad
I don't have time at the moment, so I'll look into it later. Sorry for that. Fixed.

madnessteat
Hero Member
*****
Offline Offline

Activity: 714
Merit: 942



View Profile
February 21, 2019, 05:40:43 PM
 #9

Very interesting idea. We will wait when you can add support for Russian. If I archive some user's posts in this form, will they be evidence on the forum?


.Mix coins.
 
Your BITCOIN Transaction
made Truly ANONYMOUS
 

███████
█████████████████
████████████████████████
█████████████████████████████
██████████                    ██████████
█████████                          █████████
███████                                    ████████
          ███████                                        ███████        ██
          █████████████████████████████████████      ███
          █████████████████████████████████████  █████
          ████████████████████████████████████  ██████
        ██████████████████████████████████████████
  █████████████████████████████████████████
█████                                        ████████████
                                  ██████████████
██                          █████████████████████          ██     
█████              ███████████████████████          ████     
█████████████████████████████            █████████     
██████████████████                      ████████████       
██████████████████████████████████████     
████████████████████████  ███████ 
    ██████████              █████
                      ████████
      ████████████████    █
        ██████████████████
                  █████      ███
                      █████
                    ████

Blender
 
The ULTIMATE BITCOIN Mixer
with an ADVANCED TECHNOLOGY
 

███████████   
███████████████████████   
████████████████████████████     
██████████████████████████████     
██████████████████████████████     
 

                                          █████████████
                                          ███████████
██████
                                          ███████████
██████
                                          ███████████
        ██
                                        ████████████
        ██
    ██                              █████████████
        ██
    ████                      ███████████████
        ██
    ██████              █████████████████
        ██
    █████████████████████████████
          ██ 
      ███████████████████████████
          ███
      ██████  ████████████████████
        ████
      ██████████████████████████
  ██████
      ██████████████████████████
███████
  █████      ████████████████████
███
████          ████████████████
████ 
████          ████████████████       
████████████████████████
 

   
█████████████████████████       
███████████████████████████       
█████████    ██    ███    ██████████     
███████████    ██    ███     ██████████       
███████████    ██    ███     ███████████     
██████████████████████████████████     
██████████████████████████████████       
legendster
Hero Member
*****
Offline Offline

Activity: 1428
Merit: 727


www.V.systems


View Profile
February 21, 2019, 06:42:27 PM
 #10

That's good to know, I was not aware of that.  The "show all" feature is cool, but it often triggers the cloudflare page.

Yes cloudflare is an issue. To the extent of my knowledge, all leading cdn's would be triggered on the same issue. I ran into this problem when trying to implement a similar "show all" feature on a flarum based forum. Another dev did confirm that Akami, Fastly, Amazon CF all were triggering the warning page. It wasn't a real hindrance to workflow just a bit annoying.


I am not able to find things when I try to search in my own/someone post history.
Nor did it look search button is effective when you search on the pages like this:
https://bitcointalk.org/index.php?action=profile;u=1446224;sa=showPosts

The search function here is famously broken for quite some time. I try to make do with whatever I want to search by using google search by using "bitcointalk.org" in quotes. It usually works. When I have to search my inbox I try to use common vowel letters. That usually pulls up the required content almost 9 out of 10 times.

**This is a signature**
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
February 23, 2019, 09:37:50 PM
Merited by Xal0lex (2)
 #11

Maybe I'm doing something wrong?
I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0

Sorry for the delay, it works again. See http://loyce.club/showall/994018.html.

Xal0lex
Staff
Legendary
*
Offline Offline

Activity: 924
Merit: 1152


https://blockchain.com.do


View Profile
February 23, 2019, 11:52:58 PM
 #12

Maybe I'm doing something wrong?
I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0

Sorry for the delay, it works again. See http://loyce.club/showall/994018.html.

Yea! It`s cool. Thanks.


██   ██   ██████████

  ██   █████████████

   ██   ████████████

 ██   ██   █████████

   ██   ████████████

      ██   █████████
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Blockchain.com.do.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████
      ▄▄▄▄▄▄▄▄▄▄▄▄▄
▀ ▀ ▀▀█   █       █▄
 ▀ ▀▀▀█▄▄▀      ▄█ ▄▀█▄
   ▀ ▀█▄▄       ██ ▄▀██▀▄
  ▀ ▀▀█  ▀▄      ▀▄▄█▀   ▀▄
 ▀▀ ▀ █▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄█▄
 ▄▄▄▄▄▄▄▄▄▄▄      █▄▄▄▄▄▄▄▄▄█
 █▄▄▄▄▄▄▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄
  ▀▄     ▄▄▄       █   █▀▀ ▀ ▀
    ▀▄ ▄█ ▄▄█▄      ▀▄▄█▀▀▀ ▀
      ▀██▄▄ ██       ▄▄█▀ ▀
        ▀▄▄▄▀      ▄▀  █▀▀ ▀
          ▀▄▄▄▄▄▄▄▄█▄▄▄█ ▀ ▀▀
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Exchange Bitcoin Quickly.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████




██████████   ██   ██

█████████████   ██

████████████   ██

█████████   ██   ██

████████████   ██

█████████   ██
Xal0lex
Staff
Legendary
*
Offline Offline

Activity: 924
Merit: 1152


https://blockchain.com.do


View Profile
February 24, 2019, 11:30:03 AM
 #13

Is it possible to make the script work on the page "Show the last posts of this person"?


BTW, the authors of the big topics might be posting these links in the first posts of their topics. Then users will not need to post such links.


██   ██   ██████████

  ██   █████████████

   ██   ████████████

 ██   ██   █████████

   ██   ████████████

      ██   █████████
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Blockchain.com.do.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████
      ▄▄▄▄▄▄▄▄▄▄▄▄▄
▀ ▀ ▀▀█   █       █▄
 ▀ ▀▀▀█▄▄▀      ▄█ ▄▀█▄
   ▀ ▀█▄▄       ██ ▄▀██▀▄
  ▀ ▀▀█  ▀▄      ▀▄▄█▀   ▀▄
 ▀▀ ▀ █▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄█▄
 ▄▄▄▄▄▄▄▄▄▄▄      █▄▄▄▄▄▄▄▄▄█
 █▄▄▄▄▄▄▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄
  ▀▄     ▄▄▄       █   █▀▀ ▀ ▀
    ▀▄ ▄█ ▄▄█▄      ▀▄▄█▀▀▀ ▀
      ▀██▄▄ ██       ▄▄█▀ ▀
        ▀▄▄▄▀      ▄▀  █▀▀ ▀
          ▀▄▄▄▄▄▄▄▄█▄▄▄█ ▀ ▀▀
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Exchange Bitcoin Quickly.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████




██████████   ██   ██

█████████████   ██

████████████   ██

█████████   ██   ██

████████████   ██

█████████   ██
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
February 24, 2019, 08:14:24 PM
 #14

Is it possible to make the script work on the page "Show the last posts of this person"?
I can, but I need some time to do so.

Quote
BTW, the authors of the big topics might be posting these links in the first posts of their topics. Then users will not need to post such links.
If someone quotes the link, it triggers an update. I didn't exclude quotes yet (because it's more work).

Last of the V8s
Legendary
*
Offline Offline

Activity: 1400
Merit: 2967


Be a bank


View Profile
March 12, 2019, 10:32:55 AM
 #15

This is really great of course.
I would really love one that showed all someone's posts, but I expect that's very hard, even for a masterscraper.

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
March 12, 2019, 10:38:05 AM
 #16

I would really love one that showed all someone's posts
I paused that project due to lack of time.

crwth
Copper Member
Hero Member
*****
Offline Offline

Activity: 1232
Merit: 795


Automate your Trading! https://gunbot.ph


View Profile WWW
March 12, 2019, 10:43:51 AM
 #17

Having to be able to see someones posts, for example, when you are verifying a persons staked address in Tomatocage’s thread or showing all the OP’s replies or something. It would make it a lot easier but as the limitation states on the page count, we’ll see

This is how I knew that the search bar searches only in that thread. I searched my previous post about my staked address.


.Mix coins.
 
Your BITCOIN Transaction
made Truly ANONYMOUS
 

███████
█████████████████
████████████████████████
█████████████████████████████
██████████                    ██████████
█████████                          █████████
███████                                    ████████
          ███████                                        ███████        ██
          █████████████████████████████████████      ███
          █████████████████████████████████████  █████
          ████████████████████████████████████  ██████
        ██████████████████████████████████████████
  █████████████████████████████████████████
█████                                        ████████████
                                  ██████████████
██                          █████████████████████          ██      
█████              ███████████████████████          ████      
█████████████████████████████            █████████      
██████████████████                      ████████████        
██████████████████████████████████████      
████████████████████████  ███████  
    ██████████              █████
                      ████████
      ████████████████    █
        ██████████████████
                  █████      ███
                      █████
                    ████

Blender
 
The ULTIMATE BITCOIN Mixer
with an ADVANCED TECHNOLOGY
 

███████████    
███████████████████████    
████████████████████████████      
██████████████████████████████      
██████████████████████████████     
 

                                          █████████████
                                          ███████████
██████
                                          ███████████
██████
                                          ███████████
        ██
                                        ████████████
        ██
    ██                              █████████████
        ██
    ████                      ███████████████
        ██
    ██████              █████████████████
        ██
    █████████████████████████████
          ██  
      ███████████████████████████
          ███
      ██████  ████████████████████
        ████
      ██████████████████████████
  ██████
      ██████████████████████████
███████
  █████      ████████████████████
███
████          ████████████████
████ 
████          ████████████████        
████████████████████████
 

   
█████████████████████████        
███████████████████████████        
█████████    ██    ███    ██████████      
███████████    ██    ███     ██████████        
███████████    ██    ███     ███████████      
██████████████████████████████████      
██████████████████████████████████       
coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
April 03, 2019, 09:34:55 PM
 #18

I can't get the all list of this topic

https://bitcointalk.org/index.php?topic=2459832.0

http://loyce.club/showall/2459832.html




LoyceMobile
Full Member
***
Offline Offline

Activity: 677
Merit: 156


Hover mouse above my sig to find many useful links


View Profile WWW
April 03, 2019, 09:43:59 PM
 #19

Your link works fine, I think you were too fast.

coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
April 04, 2019, 12:53:22 PM
 #20

Maybe, I loaded the page on your website and got the instant error and I left the site I wasn't thinking about the loading the data since I haven't got the message about it.

LoyceMobile
Full Member
***
Offline Offline

Activity: 677
Merit: 156


Hover mouse above my sig to find many useful links


View Profile WWW
May 07, 2019, 09:27:31 PM
 #21

It works fine, but needs some time to scrape all pages.
You can post this in any thread, is doesn't have to be in this topic.

coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
May 07, 2019, 09:34:08 PM
Last edit: May 07, 2019, 10:28:01 PM by coinlocket$
 #22

Yeah I saw that It took more than expected, also I removed my last post on this thread when I've noticed that the website has done the upload.

Thread archived here: http://archive.fo/1ecbv  Wink


coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
May 28, 2019, 09:20:51 PM
 #23

Hey mate I've tried with http://loyce.club/showall/873587.html

I waited around 2 hours but it cannot be loaded.

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
May 28, 2019, 09:23:51 PM
 #24

I waited around 2 hours but it cannot be loaded.
My scraper didn't catch your first try (where did you post it?), but it's currently scraping based on the post I now quoted.

Update: done!

coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
May 28, 2019, 09:29:41 PM
 #25

I waited around 2 hours but it cannot be loaded.
My scraper didn't catch your first try (where did you post it?), but it's currently scraping based on the post I now quoted.

Update: done!

uhm if I'm not wrong I was on the browser history with an old link and I've overwritten the new topic ID and pressed ENTER but I'm not sure about it.

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
May 28, 2019, 09:36:44 PM
 #26

uhm if I'm not wrong I was on the browser history with an old link and I've overwritten the new topic ID and pressed ENTER but I'm not sure about it.
You'll have to post the link somewhere in a new post on Bitcointalk, in order for my scraper to catch it. So when you mentioned it here, it activated.

coinlocket$
Hero Member
*****
Offline Offline

Activity: 840
Merit: 1204


One of the world's leading Bitcoin-powered casinos


View Profile WWW
May 28, 2019, 09:38:26 PM
 #27

uhm if I'm not wrong I was on the browser history with an old link and I've overwritten the new topic ID and pressed ENTER but I'm not sure about it.
You'll have to post the link somewhere in a new post on Bitcointalk, in order for my scraper to catch it. So when you mentioned it here, it activated.

ok, got it. Thanks

Initscri
Hero Member
*****
Offline Offline

Activity: 1260
Merit: 733


https://bitcoin.watfordfc.com


View Profile WWW
May 29, 2019, 05:27:52 AM
Merited by Vod (1)
 #28

So, I was bored & wanted to code something else than the projects I was working on for an hour. I made a script to make this a whole lot easier to use (I hope that's ok LoyceV, if not LMK):

https://github.com/Initscri/LoyceV-ShowAll-ChromeExtension/

It's a chrome extension, but can be used for Firefox as well, or you can simply just pull the script from content-script.js and use in GreaseMonkey or something.

For those who don't know how to load Unpacked Extensions, visit this link here: https://stackoverflow.com/questions/24577024/install-chrome-extension-not-in-the-store

  ▄▄█████▄▄███████▄▄
 ███████████
     ▀▀███▄
█████████████        ▀██▄
█████████████          ██▄
███████████            ██▄
██▀▀█████▀▀              ██
██                       ██
██                       ██
▀██                     ██▀
 ▀██                   ██▀
  ▀██▄               ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
██████████████████████
██████████████████████
██████████████████████
██████████████████
████████████████████
██████████████████████
██████████████████
▀██████████████████████▀
▀██████████████████▀
▀██████████████▀
▀██████████▀
▀████▀
▀██▀
MAIN CLUB
PARTNER of
W A T F O R D  FC
Industry Leading Crypto Sportsbook
|
DAILY
PROMOS
& BOOSTS
|
PLAY
POKER
& CASINO
|
▄▄█████████▄▄
▄█████████████████▄
▄██████████▀▀▀▀███████▄
▄█████████▀     ████████▄
▄██████████   ████████████▄
█████████        ██████████
█████████▄▄   ▄▄███████████
███████████   █████████████
▀██████████   ████████████▀
▀█████████   ███████████▀
▀████████▄▄▄██████████▀
▀█████████████████▀
▀▀█████████▀▀
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
May 29, 2019, 08:34:34 AM
 #29

I made a script to make this a whole lot easier to use
I'm very careful with browser extensions and barely ever install them, but if someone is brave enough, they can try it out.
I'm curious though if you've tested it yourself, I didn't see any newly added topics on http://loyce.club/showall/?MA.

Initscri
Hero Member
*****
Offline Offline

Activity: 1260
Merit: 733


https://bitcoin.watfordfc.com


View Profile WWW
May 29, 2019, 04:02:35 PM
 #30

I made a script to make this a whole lot easier to use
I'm very careful with browser extensions and barely ever install them, but if someone is brave enough, they can try it out.
I'm curious though if you've tested it yourself, I didn't see any newly added topics on http://loyce.club/showall/?MA.

Source code is fully available. I did, seems to 404 when clicking on a thread which hasn't been archived. IDK if that's normal or not.

I know you said the links have to be posted on BitcoinTalk in order to be picked up, so they can just copy the link if it 404s I guess? It just gets the user to the URL structure you recommended in the opening post.

  ▄▄█████▄▄███████▄▄
 ███████████
     ▀▀███▄
█████████████        ▀██▄
█████████████          ██▄
███████████            ██▄
██▀▀█████▀▀              ██
██                       ██
██                       ██
▀██                     ██▀
 ▀██                   ██▀
  ▀██▄               ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
██████████████████████
██████████████████████
██████████████████████
██████████████████
████████████████████
██████████████████████
██████████████████
▀██████████████████████▀
▀██████████████████▀
▀██████████████▀
▀██████████▀
▀████▀
▀██▀
MAIN CLUB
PARTNER of
W A T F O R D  FC
Industry Leading Crypto Sportsbook
|
DAILY
PROMOS
& BOOSTS
|
PLAY
POKER
& CASINO
|
▄▄█████████▄▄
▄█████████████████▄
▄██████████▀▀▀▀███████▄
▄█████████▀     ████████▄
▄██████████   ████████████▄
█████████        ██████████
█████████▄▄   ▄▄███████████
███████████   █████████████
▀██████████   ████████████▀
▀█████████   ███████████▀
▀████████▄▄▄██████████▀
▀█████████████████▀
▀▀█████████▀▀
Xal0lex
Staff
Legendary
*
Offline Offline

Activity: 924
Merit: 1152


https://blockchain.com.do


View Profile
May 29, 2019, 04:40:56 PM
 #31

So, I was bored & wanted to code something else than the projects I was working on for an hour. I made a script to make this a whole lot easier to use (I hope that's ok LoyceV, if not LMK):

I tried installing the script in Tampermonkey. Here's the result:



██   ██   ██████████

  ██   █████████████

   ██   ████████████

 ██   ██   █████████

   ██   ████████████

      ██   █████████
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Blockchain.com.do.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████
      ▄▄▄▄▄▄▄▄▄▄▄▄▄
▀ ▀ ▀▀█   █       █▄
 ▀ ▀▀▀█▄▄▀      ▄█ ▄▀█▄
   ▀ ▀█▄▄       ██ ▄▀██▀▄
  ▀ ▀▀█  ▀▄      ▀▄▄█▀   ▀▄
 ▀▀ ▀ █▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄█▄
 ▄▄▄▄▄▄▄▄▄▄▄      █▄▄▄▄▄▄▄▄▄█
 █▄▄▄▄▄▄▄▄▄█▄▄▄▄▄▄▄▄▄▄▄▄
  ▀▄     ▄▄▄       █   █▀▀ ▀ ▀
    ▀▄ ▄█ ▄▄█▄      ▀▄▄█▀▀▀ ▀
      ▀██▄▄ ██       ▄▄█▀ ▀
        ▀▄▄▄▀      ▄▀  █▀▀ ▀
          ▀▄▄▄▄▄▄▄▄█▄▄▄█ ▀ ▀▀
██████  ██
██
██
██
██
██
██
██
██
██
██
██
██████  ██
.Exchange Bitcoin Quickly.██  ██████
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
        ██
██  ██████




██████████   ██   ██

█████████████   ██

████████████   ██

█████████   ██   ██

████████████   ██

█████████   ██
Initscri
Hero Member
*****
Offline Offline

Activity: 1260
Merit: 733


https://bitcoin.watfordfc.com


View Profile WWW
May 29, 2019, 04:41:53 PM
 #32

So, I was bored & wanted to code something else than the projects I was working on for an hour. I made a script to make this a whole lot easier to use (I hope that's ok LoyceV, if not LMK):

I tried installing the script in Tampermonkey. Here's the result:



Hmm, weird, I'll test and touch base once I figure out why.

  ▄▄█████▄▄███████▄▄
 ███████████
     ▀▀███▄
█████████████        ▀██▄
█████████████          ██▄
███████████            ██▄
██▀▀█████▀▀              ██
██                       ██
██                       ██
▀██                     ██▀
 ▀██                   ██▀
  ▀██▄               ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
██████████████████████
██████████████████████
██████████████████████
██████████████████
████████████████████
██████████████████████
██████████████████
▀██████████████████████▀
▀██████████████████▀
▀██████████████▀
▀██████████▀
▀████▀
▀██▀
MAIN CLUB
PARTNER of
W A T F O R D  FC
Industry Leading Crypto Sportsbook
|
DAILY
PROMOS
& BOOSTS
|
PLAY
POKER
& CASINO
|
▄▄█████████▄▄
▄█████████████████▄
▄██████████▀▀▀▀███████▄
▄█████████▀     ████████▄
▄██████████   ████████████▄
█████████        ██████████
█████████▄▄   ▄▄███████████
███████████   █████████████
▀██████████   ████████████▀
▀█████████   ███████████▀
▀████████▄▄▄██████████▀
▀█████████████████▀
▀▀█████████▀▀
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
May 30, 2019, 07:36:45 AM
 #33

I know you said the links have to be posted on BitcoinTalk in order to be picked up, so they can just copy the link if it 404s I guess? It just gets the user to the URL structure you recommended in the opening post.
I only scrape long topics after it has been requested, so there's not much use for showing links without posting them.

I can't help thinking manually editing the URL and posting it is much easier than installing a browser extension for this.

Initscri
Hero Member
*****
Offline Offline

Activity: 1260
Merit: 733


https://bitcoin.watfordfc.com


View Profile WWW
May 31, 2019, 03:49:12 AM
 #34

I know you said the links have to be posted on BitcoinTalk in order to be picked up, so they can just copy the link if it 404s I guess? It just gets the user to the URL structure you recommended in the opening post.
I only scrape long topics after it has been requested, so there's not much use for showing links without posting them.

I can't help thinking manually editing the URL and posting it is much easier than installing a browser extension for this.

I was under the impression it worked for every URL generated, not when pasted into a post.

Either way though, it's there if people wish to use it. It makes it faster to copy/paste because you can just "Copy Link Address", and quicker to see if it's already been loaded into your script.

It didn't take me long to create since I had similar scripts available based on threads / urls.

  ▄▄█████▄▄███████▄▄
 ███████████
     ▀▀███▄
█████████████        ▀██▄
█████████████          ██▄
███████████            ██▄
██▀▀█████▀▀              ██
██                       ██
██                       ██
▀██                     ██▀
 ▀██                   ██▀
  ▀██▄               ▄██▀
    ▀███▄▄       ▄▄███▀
       ▀▀█████████▀▀
██████████████████████
██████████████████████
██████████████████████
██████████████████
████████████████████
██████████████████████
██████████████████
▀██████████████████████▀
▀██████████████████▀
▀██████████████▀
▀██████████▀
▀████▀
▀██▀
MAIN CLUB
PARTNER of
W A T F O R D  FC
Industry Leading Crypto Sportsbook
|
DAILY
PROMOS
& BOOSTS
|
PLAY
POKER
& CASINO
|
▄▄█████████▄▄
▄█████████████████▄
▄██████████▀▀▀▀███████▄
▄█████████▀     ████████▄
▄██████████   ████████████▄
█████████        ██████████
█████████▄▄   ▄▄███████████
███████████   █████████████
▀██████████   ████████████▀
▀█████████   ███████████▀
▀████████▄▄▄██████████▀
▀█████████████████▀
▀▀█████████▀▀
Quickseller
Copper Member
Legendary
*
Offline Offline

Activity: 2072
Merit: 1877


BountyPortals.com


View Profile WWW
June 09, 2019, 10:31:46 PM
 #35

http://loyce.club/showall/5110093.html

Thanks

Signatures seem to be disabled/not displayed. Is there a way to have signatures displayed? The lack of a signature is a quick indicator that someone might be banned without cross referencing their handle/UID with any kind of list.

The head executive of the executive office of the department of the redundancy department’s office
TryNinja
Legendary
*
Offline Offline

Activity: 1302
Merit: 1834



View Profile
June 09, 2019, 10:53:34 PM
 #36

I tried installing the script in Tampermonkey. Here's the result:

https://i.imgur.com/8Fhxqj8.png
It worked for me. Try this:

Code:
// ==UserScript==
// @name         @name LoyceV ShowAll Chrome Extension
// @include      https://bitcointalk.org/*
// @author       @author Initscri (https://bitcointalk.org/index.php?action=profile;u=186520)
// ==/UserScript==

function getQueryParams(qs) {
    qs = qs.split('+').join(' ');

    var params = {},
        tokens,
        re = /[?&]?([^=]+)=([^&]*)/g;

    while (tokens = re.exec(qs)) {
        params[decodeURIComponent(tokens[1])] = decodeURIComponent(tokens[2]);
    }

    return params;
}

var LoyceVShowAllChromeExtension = {
  logPrefix: 'LoyceV Show All Chrome Extension V1.0',
  verbose: true, // I mean, you should probably leave this on.
  logIfVerbose: function(message) {
    var controller = this;

    if(controller.verbose == true) {
      console.log(controller.logPrefix + ': ' + message);
    }
  },
  checkIfOnTopicPage: function() {
    var controller = this;

    // Get the mirrortab_back container.
    var mirrortab_b = document.getElementsByClassName("mirrortab_back")[0];
    var url_identifier = '//bitcointalk.org/index.php?topic='; // This also ensures this only runs on BitcoinTalk, and not similar SMF forums which share the same URL structure.
    var cur_url = window.location.href;

    // Check if mirrortab is avail, and on topic page.
    if(mirrortab_b != null && cur_url.indexOf(url_identifier) !== -1) {
      // on a BitcoinTalk.org topic page, proceed.
      console.log('Adding Show All Link');
      controller.addShowAllLinkForTopic(mirrortab_b);
    }
  },
  addShowAllLinkForTopic(location) {
    var controller = this;

    // First, get the topic ID from the url.
    var queryParams = getQueryParams(document.location.href);
    var topicId = queryParams['https://bitcointalk.org/index.php?topic'];
    topicId = topicId.substring(0, topicId.indexOf('.'));
    topicId = topicId.replace('.', '');

    if(location != null && topicId != null) {
      // Adding!
      var nodeSeperator = document.createElement('span');
      nodeSeperator.innerHTML = '  |  ';

      // Create the link
      var nodeLink = document.createElement('a');
      nodeLink.setAttribute('href', 'http://loyce.club/showall/' + topicId + '.html');
      nodeLink.setAttribute('target', '_BLANK');

      // Set the icon of the link.
      nodeLink.innerHTML = 'Show All';

      // And set the link!
      controller.logIfVerbose('Setting the link!');
      location.appendChild(nodeSeperator);
      location.appendChild(nodeLink);

      controller.logIfVerbose('All Done!');
    }
  },
  init: function() {
    var controller = this;

    // Check if on topic page.
    // And this runs everything else.
    controller.checkIfOnTopicPage();
  }
}

setTimeout(function() {
LoyceVShowAllChromeExtension.init();
}, 500);

It adds this option (I had to look for it):



sorry for the off-topic

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
June 10, 2019, 09:24:48 AM
Last edit: August 17, 2019, 08:13:42 AM by LoyceV
 #37

Signatures seem to be disabled/not displayed. Is there a way to have signatures displayed? The lack of a signature is a quick indicator that someone might be banned without cross referencing their handle/UID with any kind of list.
I only copy the posts, which was easiest and gives a cleaner page. I could strike out the usernames for banned users, but considering "showall" is only used a few times per month, I'd rather spend my time on other things (such as my long delayed dtview-viewer).

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
August 17, 2019, 08:08:44 AM
 #38

I wanted to move this to my VPS, but I've postponed that (it was unreliable). I did continue my quest for less manual work, so this runs from a cronjob now.

Testing: http://loyce.club/showall/5112444.html

Let me know if anything fails!

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
September 19, 2019, 05:45:33 PM
Last edit: September 19, 2019, 08:00:37 PM by LoyceV
 #39

Bump

f*ck

This is disabled until I remove session keys.


Enabled again Smiley

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
October 01, 2019, 09:43:49 AM
 #40

Does the script still work?
Yes Smiley

Quote
You could have just clicked https://bitcointalk.org/index.php?topic=5023605.0;all , this topic is short enough Tongue

Veleor
Hero Member
*****
Offline Offline

Activity: 882
Merit: 1461


Èñj𥠆hê §ïlêñ¢ê


View Profile
October 01, 2019, 09:54:44 AM
 #41

Quote
You could have just clicked https://bitcointalk.org/index.php?topic=5023605.0;all , this topic is short enough Tongue

I know, I didn't want to load extra resources for a testing.  Smiley
NeuroticFish
Legendary
*
Online Online

Activity: 2142
Merit: 1496


First 100% Liquid Stablecoin Backed by Gold


View Profile
October 01, 2019, 10:13:37 AM
Merited by vapourminer (1)
 #42

Limitations
I scrape a maximum of 500 pages per topic (sorry Wall Observer).

If it's not too much work, maybe you can do something like http://loyce.club/show/178336.html?last=10 to help people catch up easier with the last couple of pages (it can be useful exactly for WO) without killing your scrapper or the browser?

LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
October 01, 2019, 12:17:11 PM
Merited by NeuroticFish (4), vapourminer (1)
 #43

Limitations
I scrape a maximum of 500 pages per topic (sorry Wall Observer).
If it's not too much work, maybe you can do something like http://loyce.club/show/178336.html?last=10 to help people catch up easier with the last couple of pages (it can be useful exactly for WO) without killing your scrapper or the browser?
And another 80 minutes wasted Tongue
How's this:
All posts in the Wall Observer thread in the past 1h, 24h and 7d
All posts in the past 1 hour: WallObserver.tk/1h.html (updated every 3 minutes).
All posts in the past 24 hours: WallObserver.tk (updated every 30 minutes).
All posts in the past 7 days: WallObserver.tk/7d.html (updated every 2 hours).

Domain WallObserver.tk expires a year from now, I'll see by then if anyone uses it.
All times are set to Amsterdam time.

Limitations:
Images still go through Bitcointalk's image proxy, and they don't load when there are too many on one page. In Firefox a fix is to go to the URL-bar a few times, and press Enter. Just "Reload" doesn't help.

NeuroticFish
Legendary
*
Online Online

Activity: 2142
Merit: 1496


First 100% Liquid Stablecoin Backed by Gold


View Profile
October 01, 2019, 12:45:37 PM
Last edit: October 25, 2019, 06:44:38 AM by NeuroticFish
 #44

And another 80 minutes wasted Tongue
How's this:
All posts in the Wall Observer thread in the past 1h, 24h and 7d
All posts in the past 1 hour: WallObserver.tk/1h.html (updated every 3 minutes).
All posts in the past 24 hours: WallObserver.tk (updated every 30 minutes).
All posts in the past 7 days: WallObserver.tk/7d.html (updated every 2 hours).

Domain WallObserver.tk expires a year from now, I'll see by then if anyone uses it.

Wow, thank you very much. I didn't expect it'll take so much time, but on the other hand you've taken quite a different (and much more optimized) approach.
At least one person will use it.


Edit 2019-10-25: and thanks for updating to a more readable font  Grin

CookieFactory
Member
**
Offline Offline

Activity: 91
Merit: 14


View Profile
October 01, 2019, 10:22:15 PM
 #45

Limitations
I scrape a maximum of 500 pages per topic (sorry Wall Observer).
If it's not too much work, maybe you can do something like http://loyce.club/show/178336.html?last=10 to help people catch up easier with the last couple of pages (it can be useful exactly for WO) without killing your scrapper or the browser?
And another 80 minutes wasted Tongue
How's this:
All posts in the Wall Observer thread in the past 1h, 24h and 7d
All posts in the past 1 hour: WallObserver.tk/1h.html (updated every 3 minutes).
All posts in the past 24 hours: WallObserver.tk (updated every 30 minutes).
All posts in the past 7 days: WallObserver.tk/7d.html (updated every 2 hours).

Domain WallObserver.tk expires a year from now, I'll see by then if anyone uses it.
All times are set to Amsterdam time.

Limitations:
Images still go through Bitcointalk's image proxy, and they don't load when there are too many on one page. In Firefox a fix is to go to the URL-bar a few times, and press Enter. Just "Reload" doesn't help.

This is awesome. I have a "sentiment analysis" bot that currently runs on tweets and subreddits but I'll incorporate this feed as well. Thanks!
LoyceV
Legendary
*
Offline Offline

Activity: 1778
Merit: 5875


Thick-Skinned Gang Leader


View Profile WWW
January 01, 2020, 04:31:02 PM
Merited by NeuroticFish (1)
 #46

In case there are still people who love scrolling, hate clicking, and don't yet know that I made this:
All posts in the Wall Observer thread in the past 1h, 24h and 7d
All posts in the past 1 hour: WallObserver.tk/1h.html (updated every 3 minutes).
All posts in the past 24 hours: WallObserver.tk (updated every 30 minutes).
All posts in the past 7 days: WallObserver.tk/7d.html (updated every 2 hours).

Domain WallObserver.tk expires a year from now, I'll see by then if anyone uses it.
All times are set to Amsterdam time.
I moved it to a new server, and it's back online now Smiley

Pages: 1 2 3 [All]
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!