New 301-following behavior stuck on "18 to go"

problem
#1

It looks like the new “20 tries” behavior for resolving 301 redirects gets stuck on 18 (or keeps resetting?)

Check out this feed: https://morning-retreat-1911.herokuapp.com/feed/playlist.php?id=UU0iwHRFpv2_fpojZgQhElEQ
In Newsblur: https://newsblur.com/site/5926740/uploads-from-noisey

Here’s what the statistics page says:
2015-06-15 17:48:11 OK (200)
2015-06-15 17:48:10 HTTP Redirect (18 to go) (301)
2015-06-15 16:37:56 OK (200)
2015-06-15 16:37:52 HTTP Redirect (18 to go) (301)
2015-06-15 15:32:03 OK (200)

It’s been saying “18 to go” for a few days, and it still says it if I manually “insta-fetch,” though the timestamps get updated.

1 Like
#2

Actually the 200s it is seeing are resetting it. Also, it looks like I’m off by 1. It should say 19 to go.

#3

Are the 200s from the new URLs at the end of the redirect?

#4

I’m still seeing this behavior here and there, by the way.

#5

I checked the feed and it only shows 200s.

#6

Sorry, I forgot to link:
Check out this feed: https://morning-retreat-1911.herokuapp.com/feed/playlist.php?id=UUHpFyLQgg4h9VZuFyby7RbQ
Which does the same 301 redirect (indeed it’s the same script as the one above)

http://www.newsblur.com/site/5935829/uploads-from-jstsciencechannel

#7

Any update on this one? The link above ( http://www.newsblur.com/site/5935829/uploads-from-jstsciencechannel ) is still showing “18 to go” alternating with “ok 200”

#8

This could be affecting the Cloudflare blog feed too:
http://www.newsblur.com/site/5858344/cloudflare - https://blog.cloudflare.com/rss/ - 41 subscribers
http://www.newsblur.com/site/605215/cloudflare - https://blog.cloudflare.com/rss/ - 356 subscribers - except for this one, the “HTTP Redirect (18 to go) (301)” is showing in the “Website URL” block, so Newsblur thinks they’re different feeds?

#9

You’re seeing what I call a branched feed. They are the same feed, but one is a branch of the other. It’s what happens when a user changes the feed address of a feed that isn’t having any exceptions. So instead of forcing all users to switch, NewsBlur branches the feed and keeps a link back to the original.

Both feeds work, so what are you looking to fix? The 18 to go issue is that the blog keeps alternating between errors and 200 OK status codes.

#10

Ah, I didn’t know about the “branched feed” behavior, so you can ignore that last post.
Returning to the original issue: It looks to me like the 200/OK happens after Newsblur follows the redirect, and that 200 resets the counter, so it never goes past 18 (really 19, like you said above).

To clarify: example.old sends a 301 redirect to example.new, and example.new would send a 200, as you’d expect. 

Whatever the cause of the reset, it means none of these redirects are being made permanent.

#11

Ok, I pushed out a fix for it. I moved it down to 10 bad fetches, and I now keep the history if there are *any* exceptions in it, resulting in a successful page change after 10 bad fetches among the past 25 fetches.

#12

Thanks! That sounds like it’ll have the desired effect, although it doesn’t address the 18 vs. 19 to go bug or the logic of the checks… I’ll keep an eye on a few feeds and see how it goes :slight_smile:

#13

Yes, this fixes the 18 to go bug. Let me know how it works.