Bitcoin Forum
June 26, 2019, 11:19:41 AM *
News: Latest Bitcoin Core release: 0.18.0 [Torrent] (New!)
 
   Home   Help Search Login Register More  
Pages: [1] 2 »  All
  Print  
Author Topic: "Show All" on long topics  (Read 989 times)
LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
February 20, 2019, 09:44:55 PM
Last edit: May 31, 2019, 06:23:29 AM by LoyceV
Merited by xandry (5), Welsh (5), suchmoon (4), Crypto-DesignService (3), bones261 (2), BitMaxz (2), TMAN (2), OgNasty (1), fronti (1), Daniel91 (1), qwertyup23 (1), goaldigger (1), iasenko (1), taikuri13 (1), Coolcryptovator (1)
 #1

The reason
This topic: Thread printing got me thinking:
"Show all" is so convenient to CTRL-F something.

Scraper
I made a script that scrapes all posts from a thread, and uploads it as one page to loyce.club/showall/

How to use it
1. Get the topic ID from the URL. I'll use 5102296 as an example.
2. Post the following link, but replace the topic ID by the one you want: http://loyce.club/showall/5102296.html on any public board, so my scraper can find it. This has to be a new post, edits are ignored.
3. Wait a bit (about 10 seconds + 5 seconds per page)
4. Click the link you created
5. Feel free to delete your post again

The link created at #2 is not an archive: any new request will overwrite it. If you want, you can archive it.

Bugs
Probably, it was a quick job Cheesy Please post them here.

Limitations
I scrape a maximum of 500 pages per topic (sorry Wall Observer). I can increase this if needed, but this is probably enough to kill your browser already.
During scraping, all new requests are ignored. I didn't account for multitasking. If someone else occupies my scraper, you'll have to post your request again a bit later.
There's an additional 1 minute delay between requests.
Don't quote this post!

PLAY NOW
Advertised sites are not endorsed by the Bitcoin Forum. They may be unsafe, untrustworthy, or illegal in your jurisdiction. Advertise here.
1561547981
Hero Member
*
Offline Offline

Posts: 1561547981

View Profile Personal Message (Offline)

Ignore
1561547981
Reply with quote  #2

1561547981
Report to moderator
1561547981
Hero Member
*
Offline Offline

Posts: 1561547981

View Profile Personal Message (Offline)

Ignore
1561547981
Reply with quote  #2

1561547981
Report to moderator
theymos
Administrator
Legendary
*
Offline Offline

Activity: 3430
Merit: 5934


View Profile
February 20, 2019, 10:57:20 PM
Merited by Welsh (1), LoyceV (1), BitMaxz (1), DireWolfM14 (1)
 #2

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

1NXYoJ5xU91Jp83XfVMHwwTUyZFK64BoAD
DireWolfM14
Sr. Member
****
Offline Offline

Activity: 434
Merit: 604



View Profile WWW
February 20, 2019, 11:23:42 PM
 #3

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

That's good to know, I was not aware of that.  The "show all" feature is cool, but it often triggers the cloudflare page.
eldrin
Full Member
***
Offline Offline

Activity: 644
Merit: 138



View Profile WWW
February 21, 2019, 12:40:45 AM
 #4

This is a great alternative, thank you! This is good for what you're looking for, search through thread for some words, for example. I don't know if it would be possible, but it would be better if the page would end up into something like this, the returned output would just be the user and the content of the post, it would not just be a printer-friendly output but also spreadsheet-friendly one. Apologies for being demanding Cheesy

LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
February 21, 2019, 10:27:50 AM
 #5

if you use the search box in the upper right while in a topic, it searches only that topic.
It's so obvious, and yet I don't think I've ever used it. Thanks! Shame on me Tongue

it would be better if the page would end up into something like this
Agreed! If this is going to be used, I'll change it. However, nobody has tested it yet. Come on, don't be shy Tongue

r1s2g3
Sr. Member
****
Offline Offline

Activity: 574
Merit: 379


I am alive but in hibernation.


View Profile
February 21, 2019, 12:02:02 PM
Merited by YOSHIE (1)
 #6

"Show all" is so convenient to CTRL-F something.

It is nice to be able to do that, though note that if you use the search box in the upper right while in a topic, it searches only that topic.

But still below feature need lot of improvement.


I am not able to find things when I try to search in my own/someone post history.
Nor did it look search button is effective when you search on the pages like this:
https://bitcointalk.org/index.php?action=profile;u=1446224;sa=showPosts

.
Game that
pays for
Playing











A
blockchain
based game
Ask me anything
about the game
in Bitcointalk.











A game
that recognize
your ownership
Join the
exciting game
of splinterlands











              ▄▄▄▄▄▄██████▄▄▄▄▄▄
          ▄▄██████████████████████▄▄
        ███████▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀███████
     ████████▌    ▄▄▄▄▄▄▄▄    ▐████████
   ▄██████████▌  █████ ▀ ▀███  ▐██████████▄
  ▐███████████▌ ████▄▄ ██ ▐███ ▐███████████▌
  ████████████▌▐█████     ████▌▐████████████
 █████████████▌ ███▄▄ ██ ▐████ ▐█████████████
 █████████████▌  ███    ▄████  ▐█████████████
██████████████▌    ▀▀▀▀▀▀▀▀    ▐██████████████
██████████████████████████████████████████████
▀████████████████████████████████████████████▀
   ▐██████████████████████████████████████▌
   ▐█████████████▀▀▀▀▀▀▀▀▀▀▀▀█████████████▌
    ▀▀██████▀▀                  ▀▀██████▀▀
Xal0lex
Staff
Hero Member
*****
Offline Offline

Activity: 672
Merit: 764



View Profile
February 21, 2019, 01:50:39 PM
 #7

Maybe I'm doing something wrong?



I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0

......
.L I V E C O I N . N E T.
.
..PROFITBOX..
██  █████████████████████████
  █████████▄      ▄██████████
█████████████▄  ▄████████████
    █████████████████████████
  ██████████▀    ▀█ ▀████████
████  █████▀  ▄▄  ▀█  ▀██████
  ████████▀  ▄██▄  ▀█   ▀████
    ██████   ▀██▀   ██   ████
  █████████▄      ▄██████████
██  █████████▄  ▄████████████
  ███████████████████████████
██  █████████████████████████
  █████████████████████▀ ███
█████████████████████▀   ███
    █████████████▀     ████
  █████████████▀   ██    ████
████  █████▀     ██    ████
  ███████▀   ██    ██    ████
    █████    ██    ██    ████
  ███████    ██    ██    ████
██  █████    ██    ██    ████
  ███████████████████████████
.....
LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
February 21, 2019, 01:58:07 PM
Last edit: February 23, 2019, 09:37:39 PM by LoyceV
 #8

Maybe I'm doing something wrong?
You'll have to post the link you're trying to see on the forum first. Not the link to the topic, but the link to loyce.club.

Quote
That's the wrong link, if you post http://loyce.club/showall/994018.html it works.
Update: it's still scraping. It's done, but it totally fails on Russian Sad
I don't have time at the moment, so I'll look into it later. Sorry for that. Fixed.

madnessteat
Sr. Member
****
Offline Offline

Activity: 462
Merit: 353



View Profile
February 21, 2019, 05:40:43 PM
 #9

Very interesting idea. We will wait when you can add support for Russian. If I archive some user's posts in this form, will they be evidence on the forum?

legendster
Hero Member
*****
Offline Offline

Activity: 1190
Merit: 642



View Profile
February 21, 2019, 06:42:27 PM
 #10

That's good to know, I was not aware of that.  The "show all" feature is cool, but it often triggers the cloudflare page.

Yes cloudflare is an issue. To the extent of my knowledge, all leading cdn's would be triggered on the same issue. I ran into this problem when trying to implement a similar "show all" feature on a flarum based forum. Another dev did confirm that Akami, Fastly, Amazon CF all were triggering the warning page. It wasn't a real hindrance to workflow just a bit annoying.


I am not able to find things when I try to search in my own/someone post history.
Nor did it look search button is effective when you search on the pages like this:
https://bitcointalk.org/index.php?action=profile;u=1446224;sa=showPosts

The search function here is famously broken for quite some time. I try to make do with whatever I want to search by using google search by using "bitcointalk.org" in quotes. It usually works. When I have to search my inbox I try to use common vowel letters. That usually pulls up the required content almost 9 out of 10 times.

If you want to hire me or anyone else on this forum, consider posting your requirements here:

JOBS & SERVICE REQUEST BOARD (Post Hiring requests HERE) [ t.me/CrptoLoversWW ]
LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
February 23, 2019, 09:37:50 PM
Merited by Xal0lex (2)
 #11

Maybe I'm doing something wrong?
I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0

Sorry for the delay, it works again. See http://loyce.club/showall/994018.html.

Xal0lex
Staff
Hero Member
*****
Offline Offline

Activity: 672
Merit: 764



View Profile
February 23, 2019, 11:52:58 PM
 #12

Maybe I'm doing something wrong?
I'm trying to see this topic https://bitcointalk.org/index.php?topic=994018.0

Sorry for the delay, it works again. See http://loyce.club/showall/994018.html.

Yea! It`s cool. Thanks.

......
.L I V E C O I N . N E T.
.
..PROFITBOX..
██  █████████████████████████
  █████████▄      ▄██████████
█████████████▄  ▄████████████
    █████████████████████████
  ██████████▀    ▀█ ▀████████
████  █████▀  ▄▄  ▀█  ▀██████
  ████████▀  ▄██▄  ▀█   ▀████
    ██████   ▀██▀   ██   ████
  █████████▄      ▄██████████
██  █████████▄  ▄████████████
  ███████████████████████████
██  █████████████████████████
  █████████████████████▀ ███
█████████████████████▀   ███
    █████████████▀     ████
  █████████████▀   ██    ████
████  █████▀     ██    ████
  ███████▀   ██    ██    ████
    █████    ██    ██    ████
  ███████    ██    ██    ████
██  █████    ██    ██    ████
  ███████████████████████████
.....
Xal0lex
Staff
Hero Member
*****
Offline Offline

Activity: 672
Merit: 764



View Profile
February 24, 2019, 11:30:03 AM
 #13

Is it possible to make the script work on the page "Show the last posts of this person"?


BTW, the authors of the big topics might be posting these links in the first posts of their topics. Then users will not need to post such links.

......
.L I V E C O I N . N E T.
.
..PROFITBOX..
██  █████████████████████████
  █████████▄      ▄██████████
█████████████▄  ▄████████████
    █████████████████████████
  ██████████▀    ▀█ ▀████████
████  █████▀  ▄▄  ▀█  ▀██████
  ████████▀  ▄██▄  ▀█   ▀████
    ██████   ▀██▀   ██   ████
  █████████▄      ▄██████████
██  █████████▄  ▄████████████
  ███████████████████████████
██  █████████████████████████
  █████████████████████▀ ███
█████████████████████▀   ███
    █████████████▀     ████
  █████████████▀   ██    ████
████  █████▀     ██    ████
  ███████▀   ██    ██    ████
    █████    ██    ██    ████
  ███████    ██    ██    ████
██  █████    ██    ██    ████
  ███████████████████████████
.....
LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
February 24, 2019, 08:14:24 PM
 #14

Is it possible to make the script work on the page "Show the last posts of this person"?
I can, but I need some time to do so.

Quote
BTW, the authors of the big topics might be posting these links in the first posts of their topics. Then users will not need to post such links.
If someone quotes the link, it triggers an update. I didn't exclude quotes yet (because it's more work).

Last of the V8s
Legendary
*
Online Online

Activity: 1176
Merit: 2233


Be a bank.


View Profile
March 12, 2019, 10:32:55 AM
 #15

This is really great of course.
I would really love one that showed all someone's posts, but I expect that's very hard, even for a masterscraper.
LoyceV
Legendary
*
Offline Offline

Activity: 1526
Merit: 4150


Self-made Legendary outside Meta!


View Profile WWW
March 12, 2019, 10:38:05 AM
 #16

I would really love one that showed all someone's posts
I paused that project due to lack of time.

crwth
Copper Member
Hero Member
*****
Online Online

Activity: 980
Merit: 540


Automate your trading https://gunbot.ph


View Profile WWW
March 12, 2019, 10:43:51 AM
 #17

Having to be able to see someones posts, for example, when you are verifying a persons staked address in Tomatocage’s thread or showing all the OP’s replies or something. It would make it a lot easier but as the limitation states on the page count, we’ll see

This is how I knew that the search bar searches only in that thread. I searched my previous post about my staked address.

       ▄██▀ ▄█████▄
     ▄██▀ ▄███▀ ▐███▄
   ▄██▀ ▄███▀    █████▄
 ▄██▀ ▄███▀    ▄██▀  ▀██▄
██▀ ▄███▀    ▄██▀      ▀██▄
██ ███▀    ▄██▀    █▄    ███
██ ███    ███    ▄███    ███
██ ███▄    ▀███▄███▀     ███
██▄ ▀███▄    ▀███▀    ▄███▀
 ▀██▄ ▀███▄         ▄███▀
   ▀██▄ ▀███▄     ▄███▀
     ▀██▄ ▀███▄ ▄███▀
       ▀██▄ ▀█████▀
████
████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████

████  ████  ████
████  ████  ████

████  ████  ████
████  ████  ████
████
████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████
████  ████  ████

████  ████  ████
████  ████  ████

████  ████  ████
████  ████  ████
         ▄█████▄ ▀██▄
       ▄███▀ ▐███▄ ▀██▄
     ▄███▀    █████▄ ▀██▄
   ▄███▀    ▄██▀  ▀██▄ ▀██▄
 ▄███▀    ▄██▀      ▀██▄ ▀██
███▀    ▄██▀    █▄    ███ ██
███    ███    ▄███    ███ ██
███▄    ▀███▄███▀    ▄███ ██
 ▀███▄    ▀███▀    ▄███▀ ▄██
   ▀███▄         ▄███▀ ▄██▀
     ▀███▄     ▄███▀ ▄██▀
       ▀███▄ ▄███▀ ▄██▀
         ▀█████▀ ▄██▀
coinlocket$
Hero Member
*****
Offline Offline

Activity: 588
Merit: 1047


One of the world's leading Bitcoin-powered casinos


View Profile WWW
April 03, 2019, 09:34:55 PM
 #18

I can't get the all list of this topic

https://bitcointalk.org/index.php?topic=2459832.0

http://loyce.club/showall/2459832.html




LoyceMobile
Member
**
Offline Offline

Activity: 378
Merit: 69

theymos: "scammers are often pretty stupid"


View Profile
April 03, 2019, 09:43:59 PM
 #19

Your link works fine, I think you were too fast.

LoyceV on the road (or in bed, couch, ....). Don't deal with this account (there's one exception), I use it on untrusted devices.
<<Lightning Network: Tip me 1 satoshi>>. <<Exchange Bitcoin LN for onchain coins>>  <<LN faucets>>
<<"When AI roams free, that's LoyceV...on mobile" (by Steamtyme)>>  <<your text here for free? come up with something good>>
Look at me, I'm DT3!
[/size]
coinlocket$
Hero Member
*****
Offline Offline

Activity: 588
Merit: 1047


One of the world's leading Bitcoin-powered casinos


View Profile WWW
April 04, 2019, 12:53:22 PM
 #20

Maybe, I loaded the page on your website and got the instant error and I left the site I wasn't thinking about the loading the data since I haven't got the message about it.

Pages: [1] 2 »  All
  Print  
 
Jump to:  

Sponsored by , a Bitcoin-accepting VPN.
Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2006-2009, Simple Machines Valid XHTML 1.0! Valid CSS!