I just started using NewsBlur, using the ReadKit app. I have it set to check every 30 minutes, and yet I get a 429 error all the time. Is this just par for the course or is there something I should be doing differently. I previously used Inoreader with Readkit and never had any issues like this.
I did search the forum, but it sounds like the 429 is more linked to API usage, which I am not doing. Any suggestions or sharing of similar experiences would be helpful.
Ok I emailed Balazs, the author of ReadKit, to ask him to make some changes:
Hey Balazs,
A ReadKit user with 745 saved stories posted on our forum about getting 429s on every sync, and I traced it to the saved stories sync. Here is what we see from the server for that account.
Each sync makes about 75 requests to /reader/starred_stories, each returning 10 stories, which is one full pass over the saved list 10 stories at a time. That endpoint is limited to 50 requests per 5 minutes per session, so the pass gets cut off around request 50 with a 429. The client then starts a new sync 60 to 90 seconds later, which burns whatever budget has freed up, so the account stays pinned at the limit indefinitely. Over six hours we logged 1,200 saved story requests from this one account, with the feed list and story hash calls every couple of minutes rather than the 30 minutes the user configured.
Three changes on your side would fix it, and any one of them would help a lot:
Use /reader/starred_story_hashes (with include_timestamps=true) to diff against what you already have, then fetch only the new or changed stories with /reader/starred_stories?h=HASH&h=HASH, up to 100 hashes per request. That is what the hashes endpoint is for and it makes a routine sync a single request.
If you keep walking pages, pass limit=100 on /reader/starred_stories. The default is 10. With 100 per page the 745 story library is 8 requests instead of 75.
Treat a 429 as a signal to back off. We don’t send Retry-After on that response today, but the window is 5 minutes, so waiting until the next scheduled sync instead of retrying within a minute is enough.
The API docs for both endpoints are at https://newsblur.com/api. Happy to answer questions or test a build against a heavy account.
Sam
Thank you for the added details in the email Sam! I unsaved all my articles (for now) and the 429 errors have indeed stopped completely. I’ll await an update to Readkit from Balazs 