15s ... I can see that getting it banned shortly ... I've implemented limits in CKDB for failure events and activity events.
That was around where I was gonna limit it per hour.
There's no useful information gained by polling the site more than about once a minute.
This will also cause even more problems if you are logging in to a single account with everyone running the app ... since that will hit the account ... lots ...
If someone needs to poll the site 5760 times a day they need to find something better to do with their time ...
Even with identifying idle workers, you can't do that more than once every 30s
A worker is expected, to average, 3 shares a day of 13.33s or more.
Shares are avg 18SPM, so 1 share is avg 1.333s
a 1000% CDF is:
0.99995460007024 1000.000% 1 in 22026.5
i.e. expect to average one in 22026.5 shares to be 10x1.333s = 13.33s
and 22026.5 shares is expected to average 8.16 hours at average 13.33s per share
FYI there's already more than 400k hits a day ...
Felt like I was about to bump that limit soon. I will slow it down.
I did notice that 30 second worker change frequency and don't pull more frequently than 30s per worker.
The data is normalized through a summary node on my end to make data from kano.is look like data from solo.ckpool.
This also allows for notifications, and controls hits to your system at a known rate. And if the downstream changes (new pool, new data, changed data structure), I don't need to wait for apple review to make adjustments.
aggregation node thread:
1 pool thread - Every 15 second, watches for block found changes. collect pool related information
3 thread poll user status - pulls data per user if user's stamp is older than 30 seconds. which is the minimum useful limit you noted.
(misc thread to watch status file, blockchain payout, etc)
The pool thread triggers the pool block push notification
The user thread triggers user push notifications
Every running app hits was suppose to hit the aggregation node a bit harder as it acts as the relay, and broadcast one-to-many for the 15s poll.
The user notification part has been bothering me too. I would like not to scrap earlier notification but if it causes stress or ban, that's no good.
The target hits to kano.is:
240x times per hour for pool block change thread.
120x times per hour per app install for user thread.
Here's a few strays that are causing the numbers to be higher, and what i'm doing to address those.
If someone adds in 20 worker addresses and don't use an account. I did not expect a lot of those, but it is there.
- solo.ckpool does a worker list in the user view. i can adjust this the same to save lots of hits should the parent user view eventually show workers.
User Typos.
- looking to apply retry suppression. there's no stamp to baseline off. need to better distinguish it from no api data, zero api data during restarts.
Also playing with an out of band block change notification.
The 15s/30s can be tuned as required immediately.
