How do other RSS services handle rate limiting or bans?

I’ve noticed in the responses to issues the fix is to contact the publisher and try and get un-banned. How do other services prevent a ban in the first place? If Newsblur is slamming the site with requests I can see how this might raise a red flag. How do other services prevent this?

We have exponential backoff in the case of failing sites, so when we do get blocked, we repeatedly fetch from the site with a greater and greater delay until we’re able to fetch again. Mostly this works, but sometimes it requires manual intervention on the part of the publisher.


Yep always good to back off :+1:

Also wanted to add, that I’m noticing that sometimes Reddit returns a response code that isn’t a 429. Is Newsblur backing off in cases where the response code isn’t a 429?