I've been running a giveaway in Games and Rounds for a few months, and I would like to build a script to download all entries once a day. I know some threads are processed that way, so it must be possible.
I can download the source per page, but it has a lot of unnecessary data in there. Is there a way to view topics without all extra data?
What I need is the user's name, level ("Member") and the post.
The source looks like this:
<b><a href="https://bitcointalk.org/index.php?action=profile;u=849998" title="View the profile of salsa321">salsa321</a></b>
<div class="smalltext">
Member<br />
<img src="https://bitcointalk.org/Themes/custom1/images/star.gif" alt="*" border="0" /><img src="https://bitcointalk.org/Themes/custom1/images/star.gif" alt="*" border="0" /><br />
<a href="https://bitcointalk.org/index.php?action=pm;sa=send;u=849998" title="Personal Message (Online)"><img src="https://bitcointalk.org/Themes/custom1/images/useron.gif" alt="Online" border="0" style="margin-top: 2px;" /></a><span class="smalltext"> Online</span><br /><br />
Activity: 98<br /><br />
Design and dice<br />
<br />
<br />
<a href="https://bitcointalk.org/index.php?action=profile;u=849998"><img src="https://bitcointalk.org/Themes/custom1/images/icons/profile_sm.gif" alt="View Profile" title="View Profile" border="0" /></a>
<a href="https://bitcointalk.org/index.php?action=pm;sa=send;u=849998" title="Personal Message (Online)"><img src="https://bitcointalk.org/Themes/custom1/images/im_on.gif" alt="Personal Message (Online)" border="0" /></a><br /><a href="https://bitcointalk.org/index.php?action=trust;u=849998">Trust:</a> <span class="trustscore" style="color:black"><b>0</b>: -0 / +0</span>
<br /><a href=& [b]DO NOT POST SESC LINKS[/b] ">Ignore</a>
</div>
</td>
<td valign="top" width="85%" height="100%" style="padding: 2px;" class="td_headerandpost">
<table width="100%" border="0"><tr>
<td valign="middle"><a href="https://bitcointalk.org/index.php?topic=1497343.msg16122945#msg16122945"><img src="https://bitcointalk.org/Themes/custom1/images/post/xx.gif" alt="" border="0" /></a></td>
<td valign="middle">
<div style="font-weight: bold;" class="subject" id="subject_16122945">
<a href="https://bitcointalk.org/index.php?topic=1497343.msg16122945#msg16122945">Re: ¦¯¦¦¯¦ Rollin.io Free Lottery Tickets ¦¯¦¦¯¦ Daily Giveaway ¦¯¦¦¯¦ 526 mBTC won</a>
</div>
<div class="smalltext"><b>Today</b> at 03:08:12 PM</div></td>
<td align="right" valign="middle" height="20" style="font-size: smaller; padding-top: 4px;" class="td_buttons" ><div id="ignmsgbttns3145" style="visibility: visible;">
<a href=& [b]DO NOT POST SESC LINKS[/b] "><img src="https://bitcointalk.org/Themes/custom1/images/frostee/frostee_quote.png" alt="Reply with quote" class="reply_button" style="vertical-align: middle;" /></a> <a class="message_number" style="vertical-align: middle;" href="https://bitcointalk.org/index.php?topic=1497343.msg16122945#msg16122945">#3145</a>
</div>
</td>
</tr></table>
<hr width="100%" size="1" class="hrcolor" style="margin-top: 4px;" />
<div class="post">username : maleficent</div>
And this was just one post! I can probably dissect this, but it would be a lot easier if it's not necessary.
Any suggestions where to start searching would be much appreciated.