Hi. I noticed that some feeds become inactive and I can only see this if I press on “show all sites”. Thing is, that the websites come back after I press “retry fetching” in the dialog.
-
I would like to see as soon as possible that a site has failed and Newsblur has given up on it so I can take action. The site might have stuff that I would like to read as soon as the stories appear. Maybe it would be good not to hide the failed sites, but show them with the red marker in the “show only unread” view.
-
It would be even better if Newsblur would retry those failed sites by itself at a regular interval, let’s say once a day and finaly give up after 2 weeks or so, leaving the site with the red marker visible so the user can take action (check the site, see if the feed has dissapeared or has changed URL)?
1 Like
NewsBlur actually tries failed sites in a geometric decay. Once it’s declared failed, another fetch is scheduled at 2x the normal rate (so if it was fetched every 4 minutes, it’ll try in 8 minutes), then 2x that, then 2x that, until it’s effectively being fetched once a day. Now, this requires the feed is right itself eventually (think Tumblr being down for 24 hours).
Much more likely are the feeds that change URLs on you without setting up a 301/302 redirect to the new URL. I think showing these feeds immediately is a great idea, but the problem is that during the import process, about 5% of a user’s sites will have exceptions. I’d hate for those to show up. And some feeds just go dark. I would hate to force users to delete those feeds. I need a good way to distinguish between momentary fault, permanent fault, and dismissable fault. Not an easy task.
Well, I don’t know if Newsblur is actually trying to fetch the failed feeds once a day. I have feeds that were failed and remained failed for weeks, even if they came back. I don’t know how to actually reproduce this… Maybe I’ll setup an RSS feed myself, add it to Newsblur and then remove it.
Now, it crossed my mind that Newsblur might not be taking the feeds out of the red marker if the feeds don’t have new stories, even if it noticed that they were back. But I had feeds where I forced a feed reparse and new stories appeared.
Anyway, there’s something wrong in the dead feed retry stuff. Right now I have a feed which has been declared dead, last update was 18 days ago, but it has stories published just 2 days ago.
Then it’s definitely a bug and I need to take a look at it. I’m trying to ship social next week, so I’m not going to be able to look at it for a bit. If this bites you again, or if anybody else notices this issue, just holler and I may have some free time to actually look into what may be causing the code that I wrote to not properly resurrect broken feeds.
Wow! You made the failed feeds to appear with the red marker when viewing only the unread sites. Thank you for this. I’ll keep an eye on them to see if the red marker disappeares and they come back.
Nope, that wasn’t me. That feed has unread stories, which is why it’s showing up.
Ok. This is still happening and I think you should have a thorough look into it.
I have a feed, “#366DaysOfMusic - A Phlow MP3 Music Podcast” (http://feed.phlow-magazine.com/phlow-…) which is marked with red for more than two weeks now and it doesn’t come back. The feed is working right now, but I haven’t clicked “Retry eftching and parsing” on purpose so that you can investigate what’s happening.
Great job not touching the feed. I was able to see exactly how and why it died and why it didn’t get restarted.
Using this feed as a canary, I’m going to make a rather substantial migration tonight to account for feed exceptions in a much nicer way. I’ll run the migration tonight and it should bring about 100k feeds back into the fold.
Tomorrow morning your site should be back to normal. Let me know either way.
Oh, wow. I’m keeping my fingers crossed on getting this fixed once and for all.
OK, I haven’t been able to deploy the fix for this yet, but it’s all written. (See the diff: https://github.com/samuelclay/NewsBlu…)
It’s a bit late tonight, and because this carries even a small risk (but barely), I’m not going to deploy it until tomorrow night. Luckily I’m still working 14-hour days, 7 days a week, so it’ll get done.
I did just launch the river blurblogs (Click on All Blurblog Stories to test it out). It’s highly probably that there is at least one issue with that, so test that out too. But I’ll get the failed feed issue here fixed tomorrow, in exactly the capacity you described above.
Ok. No problem. I know you have other priorities too.