Are automated accounts allowed?

[hatshepsut93]

I want to make a bot that notifies people when their posts are quoted. To do this I want to utilize the forum search function, and to do that I need to use an account. Are these types of accounts allowed, or will they get banned? And if they are allowed, are there any rate limits for searches? Thanks.

[1715437045]

1715437045

[1715437045]

1715437045

[bL4nkcode]

Yes, some of them have their own bot for notification purposes you can ask @LoyceV and @Piggy for that. This telegram bot can help you as well, credits to @Piggy

[hatshepsut93]

Yes, some of them have their own bot for notification purposes you can ask @LoyceV and @Piggy for that. This telegram bot can help you as well, credits to @Piggy

Thanks, I plan to do things a bit differently, instead of requiring messages to have specific pattern for mentioning, it will just look for posts that include quotes from subscribed users. I already do this by searching for my name with day limit set to 1, to see if someone has quoted my posts, now I want to automate it, add push notifications and release it to the public.

[bL4nkcode]

Thanks, I plan to do things a bit differently, instead of requiring messages to have specific pattern for mentioning, it will just look for posts that include quotes from subscribed users. I already do this by searching for my name with day limit set to 1, to see if someone has quoted my posts, now I want to automate it, add push notifications and release it to the public.

Great! Where you want the user/subscriber to receive the notification alert. Is it a thread, email, or a telegram account as well.

[PrimeNumber7]

To do this I want to utilize the forum search function, and to do that I need to use an account.

This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server.

A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:

Code:

[quote=
AND
]

While looking at all instances of

Code:

[quote=

[hatshepsut93]

To do this I want to utilize the forum search function, and to do that I need to use an account.

This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server.

A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:

Code:

[quote=
AND
]

While looking at all instances of

Code:

[quote=

That would put me at constant 6 thread requests per minute, while the search can be configured with any rate you want, with a tradeoff of creating delays, potentially very long if there's too much subscribers.

If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.

[LoyceV]

If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.

My scraper (see Viewing unedited posts and deleted posts is currently not uploading (I need some time to test it), but when it works, I could provide you with a daily snapshot of all new posts.

[Upgrade00]

This is an old comment from theymos, but it could be useful in your research.

The contextual search in the upper right searches the board you're on and all subboards. If this results in more than 5 boards, but less than all boards, then you get that error (recently-added for performance reasons).

If you want to search the entire forum, you can go to the search page or to the front page. The limitation I added is that you can either search all boards, or limit it to 1-5 boards. But you can't limit it to 10 boards, for example.

The limitation applies to the forum inbuilt search engine and not the Google search option.

[PrimeNumber7]

To do this I want to utilize the forum search function, and to do that I need to use an account.

This will not work very well. The forum search function is a) not very good, and b) very resource intensive on the forum's server.

A superior way of doing this would be to use the See unread posts since last visit page, review the last pages of threads on this page until your bot gets to a post it has already reviewed, and check for keywords in-between the following:

Code:

[quote=
AND
]

While looking at all instances of

Code:

[quote=

That would put me at constant 6 thread requests per minute, while the search can be configured with any rate you want, with a tradeoff of creating delays, potentially very long if there's too much subscribers.

If 6 threads per minute is not too much, then I'll stick to it, otherwise I'll use my original idea.

I believe an average of 1 page request per second is the stated is maximum frequently bots can access the forum. I believe short bursts in excess of this rate is acceptable. One request every 10 seconds should not cause issues.

[hatshepsut93]

I'm working on this project right now, and I've almost finished the backend and the frontend, so the scraping is the last thing left. I've found that there's this interesting page - https://bitcointalk.org/index.php?action=recent

Is this all I need to setup my scraping? @LoyceV, is that how you save your posts? This page doesn't even require registration, so I don't have to create account for my app.

[LoyceMobile]

I use "recent" indeed.