Is there any reason that the newsblur crawler only scans some feeds once a week? Getting 20 feeds of everything that’s happeened during the past week on a site every Saturday is less useful than you’d think.
2 Likes
What’s the NewsBlur URL of the feeds? I’d love to take a look. Premium users’ feeds should update every hour, unless they average fewer than 1 story a month, in which case they update every 1-12 hours.
Looks like it’s anything tumblr-related.
My own tumblr, http://fancycwabs.com/rss has had nothing new for a couple of days, but the rss shows the most recent entries (a couple today) and Google Reader seems to be picking 'em up.
Looks like your feed is both in real-time and being fetched every 4-5 hours (to catch the fallout from real-time status not being sent by tumblr – which has been a notorious source of frustration in terms of not sending real-time notifications).
Got any more NewsBlur URLs? Try not to Insta-fetch one of them so I can see if it’s related to a bug around the scheduling of feeds not re-calculating.
Pretty much “$anything.tumblr.com/rss” where $anything could be “merlin” or “mydrunkkitchen” or pretty much any other tumblr has the same symptoms–once a week, I get 20 (max) posts all at once, and nothing the rest of the time.
I have noticed this issue with youtube rss feeds, for example this was last updated 4 days ago until I did my own refresh (4 new items showed up). https://www.newsblur.com/site/1315040…
Next time you see one of these feeds, don’t Insta-fetching it. Give me the NewsBlur URL so I can figure it out. I have an inkling, but I need a smoking gun. More the merrier.
https://www.newsblur.com/site/2121/do… (Almost a week)
https://www.newsblur.com/site/207030/… (2+ weeks)
Strangely for all of these the site settings claims it has updated multiple times today alone.
I’m guessing there are more buried in my feed list, but I was able to come up with those pretty quickly.
I had about 5 I clicked Insta-fetch on before seeing the request not to.
Good news! I fixed the bug and it should slowly resolve all of those feeds. As soon as one of those feeds has a new story, the decay to refetch will be recalculated to fix the issue. Unfortunately it will take a few days to fix all of the feeds, but if you want to immediately fix it, just request statistics for the slow feeds. It will immediately kickstart the feed. This should happen automatically for all of your feeds over the next few days.
Excellent! Thanks!
I’m having trouble with this feed and was wondering if you could have a look? http://www.newsblur.com/site/1283078/…
It seems to only pull in 20 feeds once a week even though there should be over 50 per day. It has only started doing this in the past few weeks.
I haven’t done an insta-fetch for approx 24 hours.
I’ve tried requesting the statistics but it hasn’t helped.
I’m having a similar problem with these feeds:
http://www.newsblur.com/site/1472189/…
http://www.newsblur.com/site/387592/m…
http://www.newsblur.com/site/679671/d…
They all seem statistics-ly fine, but every few days I have just a big backlog of stories.
That’s bizarre. I subscribe to the It’s Okay To Be Smart feed and it updates regularly. And taking a look at its Statistics, it seems to be running smooth on both real-time and regular updates.
OK, I looked much more deeply into this and discovered that these feeds were not being auto-resubscribed to their PubSubHubbub hubs. I went a bit overboard and forced them to resubscribe and now all of those slowly updating feeds should be back to real-time! Please let me know if that’s the case. It’ll take a few hours to roll out systemwide, but you should notice those stories coming in immediately.
I know! It’s weird. I resolve to keep an even closer eye on it, maybe take screenshots.