Bash script to filter new topics created by high rank members only

[spirali]

As a newbie, I often realized that topics created by high rank members (Legenday or Hero Member for instance) were more interesting to read. I searched for a way to filter these topics but I couldn't find it so I wrote a little bash script for that. Here it is:

Code:

#!/bin/bash
################# SETTINGS #################
LOGFILE=$HOME/log_bitcointalk_high_rank_topics
#Generating "BOARDS_ARRAY" containing the ID's of interesting boards
declare -a BOARDS_ARRAY=(1 6 14 4 12 7 5 8 24 34 39 9 250 67 159 160 161 224)
MAX_PAGES_TO_CHECK_PER_BOARD=5
############################################

#Creating the temporary directory to store queries
TEMP_DIRECTORY=/tmp/bitcointalk_high_rank_topics
if [ ! -d $TEMP_DIRECTORY ]; then
  mkdir $TEMP_DIRECTORY
fi

#Creating the log file if it doesn't exist
if [ ! -f $LOGFILE ]; then
  touch $LOGFILE
fi

boards_array_length=${#BOARDS_ARRAY[@]}
boards_array_entry=0
while [ $boards_array_entry -lt $boards_array_length ]; do
  #Retrieving the ID of the last topic checked for the selected board (or setting it to 0 if absent)
  last_topic_checked=$(grep "last_topic_checked_on_board${BOARDS_ARRAY[$boards_array_entry]}=" "$LOGFILE" | awk -F '=' '{print $2}')
  if [ -z $last_topic_checked ]; then
    echo "last_topic_checked_on_board${BOARDS_ARRAY[$boards_array_entry]}=0" >> "$LOGFILE"
    last_topic_checked=0
  fi
  #Retrieving the first pages of the selected board in the temporary directory (until we find the last topic checked last time the script was run or until MAX_PAGES_TO_CHECK_PER_BOARD is reached)
  last_topic_checked_found=false
  board_page=0
  while [ $last_topic_checked_found = false ] && [ $board_page -lt $MAX_PAGES_TO_CHECK_PER_BOARD ]; do
    from_topic_position=$((board_page*40))
    curl -s -N "https://bitcointalk.org/index.php?board=${BOARDS_ARRAY[$boards_array_entry]}.$from_topic_position;sort=first_post;desc" | iconv -f ISO-8859-1 -t UTF-8 >> "$TEMP_DIRECTORY/board${BOARDS_ARRAY[$boards_array_entry]}"
    cat "$TEMP_DIRECTORY/board${BOARDS_ARRAY[$boards_array_entry]}" | grep -q "https://bitcointalk.org/index.php?topic=$last_topic_checked"
    if [ $? = 0 ]; then
      last_topic_checked_found=true
    fi
    ((board_page++))
  done
  board_title=$(cat "$TEMP_DIRECTORY/board${BOARDS_ARRAY[$boards_array_entry]}" | grep -m 1 -o -P '(?<=<meta name="description" content=).*(?=\ />)' | perl -MHTML::Entities -pe 'use open ":std", ":utf8"; decode_entities($_);')
  echo -e "\n"
  echo "Checking board ${BOARDS_ARRAY[$boards_array_entry]}: $board_title"
  #Generating "topics_array" containing the ID's of the 40 last topics in this board sorted by date of creation descending 
  declare -a topics_array=($(cat "$TEMP_DIRECTORY/board${BOARDS_ARRAY[$boards_array_entry]}" | grep "https://bitcointalk.org/index.php?topic=" | awk '{print $2}' | grep -o -P '(?<=topic=).*(?=\.)' | sort -n | uniq | sed '/^$/d'))
  topics_array_length=${#topics_array[@]}
  topics_array_entry=0
  while [ $topics_array_entry -lt $topics_array_length ]; do
    if [ ${topics_array[$topics_array_entry]} -gt $last_topic_checked ]; then
      #Updating the log file with the last topic ID checked on this board
      sed -i "s/^last_topic_checked_on_board${BOARDS_ARRAY[$boards_array_entry]}=.*/last_topic_checked_on_board${BOARDS_ARRAY[$boards_array_entry]}=${topics_array[$topics_array_entry]}/" "$LOGFILE"
      #echo "checking topic https://bitcointalk.org/index.php?topic=${topics_array[$topics_array_entry]}"
      #Checking if the topic was created by a Legendary or Hero Member (excluding the topics "MOVED")
      curl -s -N -o "$TEMP_DIRECTORY/topic${topics_array[$topics_array_entry]}" "https://bitcointalk.org/index.php?topic=${topics_array[$topics_array_entry]}" | iconv -f ISO-8859-1 -t UTF-8
      topic_title=$(cat "$TEMP_DIRECTORY/topic${topics_array[$topics_array_entry]}" | grep -o -P '(?<=<meta name="description" content=).*(?=\ />)' | perl -MHTML::Entities -pe 'use open ":std", ":utf8"; decode_entities($_);')
      OP_name=$(cat "$TEMP_DIRECTORY/topic${topics_array[$topics_array_entry]}" | grep -m 1 -o -P '(?<=View the profile of ).*(?=\")')
      OP_profile=$(cat "$TEMP_DIRECTORY/topic${topics_array[$topics_array_entry]}" | grep -m 1 -A10 "View the profile of")
      if [[ $OP_profile = *"Legendary"* ]] && [[ $topic_title != *"MOVED: "* ]]; then
        echo "Legendary ($OP_name): $topic_title https://bitcointalk.org/index.php?topic=${topics_array[$topics_array_entry]}"
      fi
      if [[ $OP_profile = *"Hero Member"* ]] && [[ $topic_title != *"MOVED: "* ]]; then
        echo "Hero Member ($OP_name): $topic_title https://bitcointalk.org/index.php?topic=${topics_array[$topics_array_entry]}"
      fi
    sleep 1
    fi
    ((topics_array_entry++))
  done
sleep 1
((boards_array_entry++))
done
#rm -r $TEMP_DIRECTORY

Note some syntax is not shell compliant so you need to run it with bash only (eg "bash myscript.sh")

The very first time you run the script, it will retrieve the first page of the last 40 topics in the selected boards (set in the "BOARDS_ARRAY" constant) and it will print the link of the topic if it was created by a Legendary or Hero Member. The script logs what it already checked in "$HOME/log_bitcointalk_high_rank_topics" so if you run it a few days later, it will only check the new topics created since last time. I commented the last line "rm -r $TEMP_DIRECTORY" to be on the safe side in case someone sets TEMP_DIRECTORY=$HOME (for example) but you can uncomment it if you leave the default value (TEMP_DIRECTORY=/tmp/bitcointalk_high_rank_topics). Also, it sleeps 1 second between each queries so it's gentle with bitcointalk.

Hopefully you'll find it as helpful as I do.

Spirali

[LoyceV]

Can you upload the results somewhere, including the thread titles and regular updates? That would make it much easier to use.

[shield132]

Don't know what this script is but want to say just something (also looking from others to confirm this works and is safe):
To my mind ignoring of some users is great idea when you want to get rid of spammers and one reputable member has huge list of them (can't remember name but will edit this post once I find him). On another hand new users mustn't be ignored, most of them need help in mining and they register for that reason. Community is to help them and everyone interested in bitcoin.

[Don Pedro Dinero]

On another hand new users mustn't be ignored, most of them need help in mining and they register for that reason. Community is to help them and everyone interested in bitcoin.

Yes, and there are some, actually few of them, that there are here because they have real interest in bitcoin and cryptos. If people filter their comments, they are going to find themselves buried into hundreds of shitposts, instead of getting occasional answers from more experienced members.

I think it is better the Jet Cash approach: first read them, and then decide if you put them in your ignore list.

[spirali]

Don't know what this script is but want to say just something (also looking from others to confirm this works and is safe):
To my mind ignoring of some users is great idea when you want to get rid of spammers and one reputable member has huge list of them (can't remember name but will edit this post once I find him). On another hand new users mustn't be ignored, most of them need help in mining and they register for that reason. Community is to help them and everyone interested in bitcoin.

I totally agree with you and I am used to help in other open source communities. But as a newbie on bitcointalk, I need to know what I am talking about before pretending I can help others. But the amount of daily new topics on this forum is overwhelming, so filtering new topics by the OP rank is quite handy at the moment.

[Piggy]

Well done, this is surely something that may come handy to everyone  

Plus can be customized quite easily to obtain different results if needed.

[spirali]

Well done, this is surely something that may come handy to everyone  

Plus can be customized quite easily to obtain different results if needed.

Thanks. It's very easy to customize indeed (to get whatever you want).

Can you upload the results somewhere, including the thread titles and regular updates? That would make it much easier to use.

I just updated the script to include the topic titles as you requested. While I was on it, I also added the board titles and OP names. Here is the result if you run it now for the first time. Note some topics are old but are retrieved by the script only because they still are in the 40 latest topic of their respective board (sorted by creation date descending). So you can check the topics with and ID higher than 47XXXXX if you want recent ones only. It occurs only the first time you run the script as next time you run it, it will only print the new topics since last time you ran it. I'll run it again tomorrow and I'll post the result here so you can see the difference.

------------------------------------------------------------------------------------
Checking board 1: "Bitcoin Discussion"
Legendary (hilariousandco): "Low quality topics do not belong here." https://bitcointalk.org/index.php?topic=1649348
Legendary (achow101): "Bitcoin Core 0.16.2 Released" https://bitcointalk.org/index.php?topic=4773040
Legendary (European Central Bank): "1 year on from the BCH fork, 61% of the coins around at the fork have moved" https://bitcointalk.org/index.php?topic=4800601


Checking board 6: "Development & Technical Discussion"
Legendary (jubalix): "Why can Core not Scale at a 0.X% for blocksize in the S-Curve f(X)" https://bitcointalk.org/index.php?topic=4691017
Legendary (d5000): "Printable one-time coupons (to be used as "cash")" https://bitcointalk.org/index.php?topic=4702304


Checking board 14: "Mining"
Legendary (-ck): "BITCOIN MINING INTRO & RULES OF THIS SUBFORUM - READ BEFORE POSTING" https://bitcointalk.org/index.php?topic=2415854
Hero Member (HagssFIN): "Notes from my journey to Beijing & visiting Canaan Creative in March 10-15, 2018" https://bitcointalk.org/index.php?topic=3169292
Legendary (gentlemand): "The State of Cryptocurrency Mining by David Vorick (Sia)" https://bitcointalk.org/index.php?topic=3776561
Hero Member (buwaytress): "BetterHash Mining Protocol BIP" https://bitcointalk.org/index.php?topic=4436756
Hero Member (buwaytress): "Miners: What kind of fee information do you see on our transactions?" https://bitcointalk.org/index.php?topic=4598742


Checking board 4: "Bitcoin Technical Support"
Legendary (achow101): "[READ] Post Armory, MultiBit, Electrum, or online wallets in the right forum" https://bitcointalk.org/index.php?topic=1628282
Legendary (achow101): "[READ BEFORE POSTING] Tech Support Help Request Format" https://bitcointalk.org/index.php?topic=1741772
Legendary (achow101): "All about "stuck" transactions and what you can do to fix them" https://bitcointalk.org/index.php?topic=1802212
Legendary (achow101): "[READ] About blockchain.info's web wallet and "stuck" transactions" https://bitcointalk.org/index.php?topic=1914800
Legendary (jackg): "Is there an operating system that syncs Bitcoin core the fastest? " https://bitcointalk.org/index.php?topic=4636729
Legendary (jackg): "Issues with raspberry pi and bitcoin core" https://bitcointalk.org/index.php?topic=4700233


Checking board 12: "Project Development"
Legendary (Mitchell): "PSA: Altcoin related threads (including ICO's) and ANN's do not belong here" https://bitcointalk.org/index.php?topic=2028417
Legendary (mocacinno): "lightning mainnet walktrough: feeback?" https://bitcointalk.org/index.php?topic=4787376
Hero Member (btc_enigma): "[ANN] Bitcoin opencart plugin -  free, open source" https://bitcointalk.org/index.php?topic=4795932


Checking board 7: "Economics"
Hero Member (thefiniteidea): "Inflation and Deflation of Price and Money Supply" https://bitcointalk.org/index.php?topic=140793
Legendary (kryptqnick): "SEC chairman being very conservative and thus hostile to cryptos" https://bitcointalk.org/index.php?topic=4762630
Hero Member (Hydrogen): "U.S. Growth Hits 4.1%, Fastest Since 2014" https://bitcointalk.org/index.php?topic=4762743
Hero Member (Hydrogen): "Almost 80% of US workers live from paycheck to paycheck. Here's why" https://bitcointalk.org/index.php?topic=4779269
Legendary (Stedsm): "We are still into flushing out weak hands!" https://bitcointalk.org/index.php?topic=4779982
Hero Member (Hydrogen): "Trade War Casualties: Factories Shifting Out Of China" https://bitcointalk.org/index.php?topic=4792738
Hero Member (Hydrogen): "WSJ: US Faces $5 Trillion Pension Hole, Same Size as Japan's Economy  " https://bitcointalk.org/index.php?topic=4794442
Legendary (darkangel11): "The real estate bubble" https://bitcointalk.org/index.php?topic=4800649


Checking board 5: "Marketplace"
Legendary (Lauda): "Beware of Malware: Think Before Acting!" https://bitcointalk.org/index.php?topic=1652275
Legendary (BayAreaCoins): "List of companies that will ban your account if they know you're mess with BTC." https://bitcointalk.org/index.php?topic=4587626
Hero Member (Hydrogen): "What is the best "non coinbase" method to purchase BTC in the USA via us dollar?" https://bitcointalk.org/index.php?topic=4608028
Hero Member (pornluver): "hitbtc removes coins from trading without announcement?" https://bitcointalk.org/index.php?topic=4628569
Hero Member (AngelSky): "Some people trolling the bitcoin and its potential. Stay away from them" https://bitcointalk.org/index.php?topic=4753811


Checking board 8: "Trading Discussion"
Legendary (Maged): "AML/KYC Explained" https://bitcointalk.org/index.php?topic=454795


Checking board 24: "Meta"
Legendary (theymos): "Do not post off-topic replies" https://bitcointalk.org/index.php?topic=14356
Legendary (John (John K.)): "Forum ranks/positions/badges (What do those shiny coins under my name mean?)" https://bitcointalk.org/index.php?topic=178608
Legendary (theymos): "Security bounties" https://bitcointalk.org/index.php?topic=309785
Legendary (theymos): "Recovering hacked accounts or accounts with lost passwords" https://bitcointalk.org/index.php?topic=497545
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=703657
Legendary (hilariousandco): "FAQ: Everything you need to know about forum 'activity, account ranks and merit" https://bitcointalk.org/index.php?topic=2766177
Legendary (Jet Cash): "Sponsoring a political article, and where to host it." https://bitcointalk.org/index.php?topic=4779121
Hero Member (WaffleMaster): "How to report Member, not post?" https://bitcointalk.org/index.php?topic=4795118
Legendary (kryptqnick): "Is managing an official account as well as my own count towards alt accounts?" https://bitcointalk.org/index.php?topic=4795349
Hero Member (LeGaulois): " The link between quality altcoins and Bitcointalk" https://bitcointalk.org/index.php?topic=4799172
Legendary (Vod): "How recognized are you in the bitcointalk world?" https://bitcointalk.org/index.php?topic=4799468


Checking board 34: "Politics & Society"
Legendary (Flying Hellfish): "Politics & Society Introduction and Rules - READ BEFORE POSTING" https://bitcointalk.org/index.php?topic=3357782
Legendary (Flying Hellfish): "Request For Help From P&S Regulars" https://bitcointalk.org/index.php?topic=4099972
Legendary (Alik Bahshi): "Helsinki: Boa and Rabbit" https://bitcointalk.org/index.php?topic=4778583
Legendary (Alik Bahshi): "CAR, and here is not without Putin" https://bitcointalk.org/index.php?topic=4799986


Checking board 39: "Beginners & Help"
Legendary (theymos): "Verifying Bitcoin Core" https://bitcointalk.org/index.php?topic=1588906
Legendary (Lauda): "[General] How to earn Bitcoins - Part 1" https://bitcointalk.org/index.php?topic=1629118
Legendary (Lauda): "[General] Bitcoin Wallets - Which, what, why?" https://bitcointalk.org/index.php?topic=1631151
Legendary (Lauda): "Newbies - Read before posting" https://bitcointalk.org/index.php?topic=1689727
Legendary (Jet Cash): "The junior blog about your Bitcoin and crypto activity today." https://bitcointalk.org/index.php?topic=4779033
Legendary (Jet Cash): "[iMerits] Why do some coins fork from the Bitcoin chain?" https://bitcointalk.org/index.php?topic=4780538
Legendary (Cluster2k): "ICO LIST - ICO TRACKER WITH NEW ICOs, UPCOMING ICOs & more! " https://bitcointalk.org/index.php?topic=4790542


Checking board 9: "Off-topic"
Legendary (grue): "Low quality topics and posts are not allowed" https://bitcointalk.org/index.php?topic=1334019


Checking board 250: "Serious discussion"
Legendary (theymos): "Rules for Serious Discussion and Ivory Tower" https://bitcointalk.org/index.php?topic=2722359
Legendary (Jet Cash): "[SMART] Europe introduces four new Galileo satellites" https://bitcointalk.org/index.php?topic=4744506
Hero Member (KonstantinosM): "Is the BCH (fake Bitcoin) Bitcoin Cash wallet free of viruses?" https://bitcointalk.org/index.php?topic=4797286


Checking board 67: "Altcoin Discussion"
Legendary (Gavin Andresen): "Alternative Block Chains : be safe!" https://bitcointalk.org/index.php?topic=42465
Legendary (theymos): "Giveaway threads are not allowed" https://bitcointalk.org/index.php?topic=434310
Legendary (grue): "Beware of Increasingly Sophisticated Malware Infection Attempts" https://bitcointalk.org/index.php?topic=935898
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=1042322
Legendary (mprep): ">>> ATTENTION: PRICE SPECULATION DISCUSSIONS DO NOT BELONG HERE <<<" https://bitcointalk.org/index.php?topic=1572439
Legendary (Abiky): "TRON vs EOS" https://bitcointalk.org/index.php?topic=4799442


Checking board 159: "Announcements (Altcoins)"
Legendary (BadBear): "Advertising spam is not allowed" https://bitcointalk.org/index.php?topic=718124
Legendary (mprep): "Giveaway threads are not allowed" https://bitcointalk.org/index.php?topic=780979
Legendary (mprep): "Beware of Increasingly Sophisticated Malware Infection Attempts" https://bitcointalk.org/index.php?topic=936724
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=1042324
Legendary (mprep): "Users posting "joined" type posts when not required or allowed will be BANNED" https://bitcointalk.org/index.php?topic=2284373
Legendary (mprep): "Incentivising posting within one or more threads is not allowed" https://bitcointalk.org/index.php?topic=2559282
Legendary (theymos): "[ANN][PRE-ICO]HoweyCoins: the only BitcoinTalk-endorsed ICO - GUARANTEED PROFIT" https://bitcointalk.org/index.php?topic=3920469


Checking board 160: "Mining (Altcoins)"
Legendary (mprep): "Cloud mining service threads don't belong in here." https://bitcointalk.org/index.php?topic=833063
Legendary (mprep): "Beware of Increasingly Sophisticated Malware Infection Attempts" https://bitcointalk.org/index.php?topic=936726
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=1042326
Legendary (swogerino): "Nicehash vs Winminer" https://bitcointalk.org/index.php?topic=4790512
Hero Member (killyou72): "Help me build a GPU mining rig" https://bitcointalk.org/index.php?topic=4791796
Hero Member (igotek): "cryptonight or Ethash ?" https://bitcointalk.org/index.php?topic=4795860
Legendary (Abiky): "Hottest CPU-only coins to mine right now?" https://bitcointalk.org/index.php?topic=4799398


Checking board 161: "Marketplace (Altcoins)"
Legendary (Tomatocage): "Be safe: READ THIS BEFORE TRADING/BUYING/USING YOUR MONEY" https://bitcointalk.org/index.php?topic=199130
Hero Member (BitJohn): "ALT coin Stores and Services List" https://bitcointalk.org/index.php?topic=273148
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=1042327
Legendary (mprep): "Giveaway threads are not allowed" https://bitcointalk.org/index.php?topic=2103586
Hero Member (Frizz23): "(WTS) LAPO masternode" https://bitcointalk.org/index.php?topic=4779541
Legendary (IBGigglin): "Why trade on just one DEX?  TOTLE combines fragmented orderbooks!" https://bitcointalk.org/index.php?topic=4782733
Hero Member (alamin99): "[ANN] Get Your Own Masternode Coin! 50% Discount for first 5 Clients!" https://bitcointalk.org/index.php?topic=4799727


Checking board 224: "Speculation (Altcoins)"
Legendary (mprep): "Unofficial list of (official) Bitcointalk.org rules, guidelines, FAQ" https://bitcointalk.org/index.php?topic=1572451
Legendary (jeffthebaker): "Team Behind GameCredits and MobileGO has Split" https://bitcointalk.org/index.php?topic=4767641
Legendary (cellard): "Anyone shorting BCash?" https://bitcointalk.org/index.php?topic=4781065

[spirali]

I'll run it again tomorrow and I'll post the result here so you can see the difference.

OK, I just ran the script again. Here is the result so you see how it works. Basically, it publishes only the new topics since last time so you can run it once a day for a daily digest or once a week for a weekly digest for example.

------------------------------------------------------------------------------------------------------------
Checking board 1: "Bitcoin Discussion"


Checking board 6: "Development & Technical Discussion"


Checking board 14: "Mining"


Checking board 4: "Bitcoin Technical Support"


Checking board 12: "Project Development"


Checking board 7: "Economics"


Checking board 5: "Marketplace"


Checking board 8: "Trading Discussion"
Hero Member (Bardman): "Is there any reason to day trade anything other than bitcoin?" https://bitcointalk.org/index.php?topic=4800703


Checking board 24: "Meta"
Legendary (lumeire): "Is this forum bug? " https://bitcointalk.org/index.php?topic=4802758
Legendary (mocacinno): "Is this a copy/paster?" https://bitcointalk.org/index.php?topic=4803097


Checking board 34: "Politics & Society"


Checking board 39: "Beginners & Help"


Checking board 9: "Off-topic"


Checking board 250: "Serious discussion"


Checking board 67: "Altcoin Discussion"


Checking board 159: "Announcements (Altcoins)"


Checking board 160: "Mining (Altcoins)"


Checking board 161: "Marketplace (Altcoins)"


Checking board 224: "Speculation (Altcoins)"

[spirali]

I just ran the script again. Here is an update of the new topics since last time (August 2nd):

------------------------------------------------------------------------------------------------------------
Checking board 1: "Bitcoin Discussion"
Hero Member (r32godzilla): "The next halving  2020" https://bitcointalk.org/index.php?topic=4955526


Checking board 6: "Development & Technical Discussion"
Hero Member (bitcoinore.com): "Decentralized + trustless technology Preventing premature access to data" https://bitcointalk.org/index.php?topic=4883383
Legendary (Gyrsur): "TLA+ for the design of BitCoin" https://bitcointalk.org/index.php?topic=4887820
Legendary (Gyrsur): "Smart Contracts - just a buzzword or are they applicable to the real world?" https://bitcointalk.org/index.php?topic=4923049
Hero Member (BitcoinSupremo): "And now the 1 mln dollar question " https://bitcointalk.org/index.php?topic=4951221


Checking board 14: "Mining"


Checking board 4: "Bitcoin Technical Support"
Legendary (Fakhoury): "Problem with Importing a private key" https://bitcointalk.org/index.php?topic=4835263
Legendary (samspaces): "P2SH electrum addresses insecure seed backup" https://bitcointalk.org/index.php?topic=4879731
Hero Member (kzv): "[TUTORIAL] Build 0.16.2 Bitcoin with one console command" https://bitcointalk.org/index.php?topic=4912326
Legendary (Carlton Banks): "Incoming connections/Port forwarding: is bind=<IP> needed, and why?" https://bitcointalk.org/index.php?topic=4929552
Hero Member (BitcoinSupremo): "How useful is Backup against Ransomware " https://bitcointalk.org/index.php?topic=4950345


Checking board 12: "Project Development"
Hero Member (AdolfinWolf): "Would there be a market/interest for such a service?" https://bitcointalk.org/index.php?topic=4955530


Checking board 7: "Economics"
Hero Member (ttookk): "Why not just a parallel Bitcoin economy?" https://bitcointalk.org/index.php?topic=4919511
Hero Member (Hydrogen): "Trump asks SEC to study HODL shift to 6 month corporate earning reports" https://bitcointalk.org/index.php?topic=4929137
Legendary (cellard): "No threads on the ETN?" https://bitcointalk.org/index.php?topic=4933392
Legendary (cellard): "ETF rejected" https://bitcointalk.org/index.php?topic=4941456
Hero Member (Hydrogen): "Stop worrying about how much energy bitcoin uses" https://bitcointalk.org/index.php?topic=4943336
Hero Member (Hydrogen): "What If Banks Were Publicly Owned?" https://bitcointalk.org/index.php?topic=4943463
Hero Member (Hydrogen): "JP Morgan to unveil investing app offering free trades(like crypto exchanges)" https://bitcointalk.org/index.php?topic=4943508
Legendary (OROBTC): "N. N. Taleb's Book: "Fooled by Randomness"" https://bitcointalk.org/index.php?topic=4952382


Checking board 5: "Marketplace"
Legendary (FortuneJack): "Monday Bonus!" https://bitcointalk.org/index.php?topic=4829457
Legendary (Kemarit): "OKEx Coin Delisting" https://bitcointalk.org/index.php?topic=4900233


Checking board 8: "Trading Discussion"


Checking board 24: "Meta"
Legendary (examplens): "Altcoin Discussion section can change name to Trash discussion" https://bitcointalk.org/index.php?topic=4933112
Legendary (TradeFortress): "CloudFlare? What changed? " https://bitcointalk.org/index.php?topic=4933592
Legendary (BitMaxz): "List of spam bots" https://bitcointalk.org/index.php?topic=4934764
Legendary (Quickseller): "Blazed pot involved in alleged escrow scam, 2k+BTC in dispute, DT remove?" https://bitcointalk.org/index.php?topic=4942549
Legendary (Jet Cash): "Thread comparison between Beginners and serious discussion" https://bitcointalk.org/index.php?topic=4943180
Legendary (Vod): "Please show Trust in Project Development" https://bitcointalk.org/index.php?topic=4946294
Legendary (LFC_Bitcoin): "Spamming?" https://bitcointalk.org/index.php?topic=4948692
Legendary (bg002h): "Follow users by merit instead of threads" https://bitcointalk.org/index.php?topic=4953382
Legendary (whywefight): "Cobras recent moves and possible effects" https://bitcointalk.org/index.php?topic=4955261
Legendary (notbatman): "Don't tread on me" https://bitcointalk.org/index.php?topic=4955545


Checking board 34: "Politics & Society"


Checking board 39: "Beginners & Help"


Checking board 9: "Off-topic"


Checking board 250: "Serious discussion"
Legendary (Jet Cash): "The world's first mining TV" https://bitcointalk.org/index.php?topic=4812479
Legendary (Jet Cash): "How do these Bitcoin transactions confirm so quickly?" https://bitcointalk.org/index.php?topic=4884274
Legendary (theymos): "What would the top coins be if people were sane?" https://bitcointalk.org/index.php?topic=4905191


Checking board 67: "Altcoin Discussion"


Checking board 159: "Announcements (Altcoins)"


Checking board 160: "Mining (Altcoins)"
Legendary (Abiky): "Easiest cryptocurrency to mine on a smartphone?" https://bitcointalk.org/index.php?topic=4947078


Checking board 161: "Marketplace (Altcoins)"
Hero Member (legendster): "[WTB] EQUAL 260500 will pay ETH/BTC" https://bitcointalk.org/index.php?topic=4935296


Checking board 224: "Speculation (Altcoins)"

[khufuking]

The effort you made is appreciated but the purpose of this script is totally wrong and the cons of using this script are outweighing the benefits of using it.

1- You are simply excluding 70%+  users of this forum ( If you are using it).
2- As a high-rank member with Merit it is your responsibility to reward new users for their good posts with Merit. Or else you helping to defeat the purpose of the Merit system.
3- If you are DT member you will not be as effective if you are to use this script. You will miss a lot of users that need to be tagged.

The only benefit of this script is to use it in Alt-Coin section and you will still gonna see all posts from bought high-rank accounts all over the place there.

I still like the effort you made.

[LoyceV]

1- You are simply excluding 70%+  users of this forum ( If you are using it).

I think it's closer to 99% than 70%. The large majority of all accounts are Newbie and Jr. Member spammers.
But if you only have limited time, and can't read all threads anyway, it makes sense to first filter the threads that are most likely to be good.

2- As a high-rank member with Merit it is your responsibility to reward new users for their good posts with Merit. Or else you helping to defeat the purpose of the Merit system.

OP isn't a high-rank member.

3- If you are DT member you will not be as effective if you are to use this script. You will miss a lot of users that need to be tagged.

Being on DT doesn't mean you have to hunt for bad guys all the time.

I still like the effort you made.

Me too It's a necessary evil thanks to the massive spam.

[bitart]

The effort you made is appreciated but the purpose of this script is totally wrong and the cons of using this script are outweighing the benefits of using it.
...

It's always up to the member to decide if he/she wants to use the script or not.
Maybe it's possible to create a daily routine like this:
1)Run the script and check the new topics of the high ranked users
and
2)after if the member still has some time, it's still possible to check the forum without the script to check for newbies looking for help or to check the other topics started by lower ranked members
This script is not a kind of ignore list, and only a few members using the ignore list function (with too much users on ignore)
People usually ignore some members, but if the ignore lists will be made available for the public, it could change.

[LoyceV]

I just ran the script again. Here is an update of the new topics since last time (August 2nd):
~
Checking board 24: "Meta"

You missed at least one thread made by me (this one). Is that a bug?

[spirali]

Nice work, now i have reason to regularly visit Altcoin section and basic editing such as remove/add boards ID or choose specific rank is very easy

I wonder if it's possible to filter/choose thread which is self-moderated. Also, where is "$HOME/log_bitcointalk_high_rank_topics" on windows OS?

I guess "$HOME/log_bitcointalk_high_rank_topics" would be "C:\Users\<username>\log_bitcointalk_high_rank_topics" on Windows but this script is meant to be run on Linux machines (I never used bash on Windows).

Filtering self-moderated topics is easy to implement (with grep "This is a self-moderated topic. If you do not want to be moderated by the person who started this topic, create a new topic." in "$TEMP_DIRECTORY/topic${topics_array[$topics_array_entry]}") but can you explain why it would be interesting?

The effort you made is appreciated but the purpose of this script is totally wrong and the cons of using this script are outweighing the benefits of using it.

1- You are simply excluding 70%+  users of this forum ( If you are using it).
2- As a high-rank member with Merit it is your responsibility to reward new users for their good posts with Merit. Or else you helping to defeat the purpose of the Merit system.
3- If you are DT member you will not be as effective if you are to use this script. You will miss a lot of users that need to be tagged.

The only benefit of this script is to use it in Alt-Coin section and you will still gonna see all posts from bought high-rank accounts all over the place there.

I still like the effort you made.

Fair arguments. In my case, it has been useful to catch up knowledge by reading "more interesting topics" (I am still new to this world) but I agree this can be converted to an evil tool if wrongly used by old members.

You missed at least one thread made by me (this one). Is that a bug?

Correct, your topic was dismissed because it was already in the second page of results in its board (meta) when I ran it yesterday (sorted by date of creation descending): https://bitcointalk.org/index.php?board=24.0;sort=first_post;desc. My idea when I wrote it was to run it daily so checking the first page (which lists the 40 last topics of each selected board) was enough to not forget topics.

But I just modified it (updated in my first post) to check page 2, 3, 4... until it reaches the last ID checked last time it was run (with a limit of pages set with the new constant MAX_PAGES_TO_CHECK_PER_BOARD=5 so that it doesn't crawl the entire forum the very first time you run it or if the topic was deleted).

[Piggy]

This script can be easily modified to show threads started by any rank, can be extremely usefull if you plan to spot meritable threads being a source imo