Duplicated posts with "The Smoking Gun" feed

Thanks, Samuel – I dug into the troublesome feed a bit more, and it looks like I’m getting the same story with slightly different GUIDs…I looked at the NewsBlur JSON and I see “ID” values of:

  • “254263 at http:// thesmokinggun .com”
  • “254263 at http:// www.thesmokinggun .com”
  • “254263 at https:// www.thesmokinggun .com”
  • “254263 at https:// thesmokinggun .com”

(extra spaces added in the URLs beause it complained that I am too new a user to put a bunch of links in a post :wink: )

I think this is a problem on The Smoking Gun, not NewsBlur. I went to the feed about 20 minutes apart, using the same URL, in the same browser, and got the feed delivered with different GUID variants.

I assume the content is just different enough that NewsBlur doesn’t detect them as duplicates. I’ve included the relevant excerpt from the JSON below so you can take a look. I’ll also send a message to them with a link to this discussion, but I have no idea if that’ll accomplish anything :wink:

{
	"story_hash": "8151809:80951b",
	"story_tags": [
		"internet",
		"crime",
		"friday photo fun"
	],
	"story_date": "2025-05-23 12:42:05",
	"story_timestamp": "1748004125",
	"story_authors": "wbastone",
	"story_title": "Friday Photo Fun Match Game",
	"story_content": "<div class=\"field field-type-text field-field-subtitle\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    Pair up mug shots with each suspect&#039;s alleged crime        </div>\n        </div>\n</div>\n<div class=\"field field-type-filefield field-field-photo\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    <img alt=\"\" class=\"imagecache imagecache-175xUnlimited imagecache-default imagecache-175xUnlimited_default\" height=\"265\" src=\"http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg\" title=\"\" width=\"175\" />        </div>\n        </div>\n</div>\n<p>Examine the booking photos of five arrested individuals and align the defendants with their respective alleged infractions of the law.</p>\n<p><a href=\"http://thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174\" target=\"_blank\">read more</a></p>",
	"story_permalink": "http://thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174",
	"image_urls": [
		"http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg",
		"http://thesmokinggun.com/sites/default/files/assets/may2325465contart.jpg"
	],
	"secure_image_urls": {
		"http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/sc,s-8QI5SkjZrbezDXlxpon76QoS3HAjymubVQc2zjnFsc=/http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg",
		"http://thesmokinggun.com/sites/default/files/assets/may2325465contart.jpg": "https://imageproxy.newsblur.com/sc,sr1TBbBG2fz1DzS1TiIm2rrh0a2ZQB2iX19E4JA5bIJY=/http://thesmokinggun.com/sites/default/files/assets/may2325465contart.jpg"
	},
	"secure_image_thumbnails": {
		"http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/192,sc,s-8QI5SkjZrbezDXlxpon76QoS3HAjymubVQc2zjnFsc=/http://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg",
		"http://thesmokinggun.com/sites/default/files/assets/may2325465contart.jpg": "https://imageproxy.newsblur.com/192,sc,sr1TBbBG2fz1DzS1TiIm2rrh0a2ZQB2iX19E4JA5bIJY=/http://thesmokinggun.com/sites/default/files/assets/may2325465contart.jpg"
	},
	"story_feed_id": 8151809,
	"has_modifications": false,
	"comment_count": null,
	"comment_user_ids": [],
	"share_count": null,
	"share_user_ids": [],
	"guid_hash": "80951b",
	"id": "254263 at http://thesmokinggun.com",
	"friend_comments": [],
	"friend_shares": [],
	"public_comments": [],
	"reply_count": 0,
	"short_parsed_date": "23 May 2025, 8:42am",
	"long_parsed_date": "Friday, May 23rd 8:42am",
	"read_status": 0,
	"intelligence": {
		"feed": 0,
		"author": 0,
		"tags": 0,
		"title": 0
	},
	"score": 0
},
{
	"story_hash": "8151809:4208d2",
	"story_tags": [
		"internet",
		"crime",
		"friday photo fun"
	],
	"story_date": "2025-05-23 12:42:05",
	"story_timestamp": "1748004125",
	"story_authors": "wbastone",
	"story_title": "Friday Photo Fun Match Game",
	"story_content": "<div class=\"field field-type-text field-field-subtitle\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    Pair up mug shots with each suspect&#039;s alleged crime        </div>\n        </div>\n</div>\n<div class=\"field field-type-filefield field-field-photo\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    <img alt=\"\" class=\"imagecache imagecache-175xUnlimited imagecache-default imagecache-175xUnlimited_default\" height=\"265\" src=\"http://www.thefgun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg\" title=\"\" width=\"175\" />        </div>\n        </div>\n</div>\n<p>Examine the booking photos of five arrested individuals and align the defendants with their respective alleged infractions of the law.</p>\n<p><a href=\"http://www.thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174\" target=\"_blank\">read more</a></p>",
	"story_permalink": "http://www.thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174",
	"image_urls": [
		"http://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	],
	"secure_image_urls": {
		"http://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/sc,sriKXeBm2F9TGQD2st4-x3TobwO8IYmxTTxYg8M-huM4=/http://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"secure_image_thumbnails": {
		"http://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/192,sc,sriKXeBm2F9TGQD2st4-x3TobwO8IYmxTTxYg8M-huM4=/http://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"story_feed_id": 8151809,
	"has_modifications": false,
	"comment_count": null,
	"comment_user_ids": [],
	"share_count": null,
	"share_user_ids": [],
	"guid_hash": "4208d2",
	"id": "254263 at http://www.thesmokinggun.com",
	"friend_comments": [],
	"friend_shares": [],
	"public_comments": [],
	"reply_count": 0,
	"short_parsed_date": "23 May 2025, 8:42am",
	"long_parsed_date": "Friday, May 23rd 8:42am",
	"read_status": 0,
	"intelligence": {
		"feed": 0,
		"author": 0,
		"tags": 0,
		"title": 0
	},
	"score": 0
},
{
	"story_hash": "8151809:0a21ae",
	"story_tags": [
		"internet",
		"crime",
		"friday photo fun"
	],
	"story_date": "2025-05-23 12:42:05",
	"story_timestamp": "1748004125",
	"story_authors": "wbastone",
	"story_title": "Friday Photo Fun Match Game",
	"story_content": "<div class=\"field field-type-text field-field-subtitle\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    Pair up mug shots with each suspect&#039;s alleged crime        </div>\n        </div>\n</div>\n<div class=\"field field-type-filefield field-field-photo\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    <img alt=\"\" class=\"imagecache imagecache-175xUnlimited imagecache-default imagecache-175xUnlimited_default\" height=\"265\" src=\"https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg\" title=\"\" width=\"175\" />        </div>\n        </div>\n</div>\n<p>Examine the booking photos of five arrested individuals and align the defendants with their respective alleged infractions of the law.</p>\n<p><a href=\"https://www.thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174\" target=\"_blank\">read more</a></p>",
	"story_permalink": "https://www.thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174",
	"image_urls": [
		"https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	],
	"secure_image_urls": {
		"https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/sc,sH-8S_pWjnRjMormMb5aQtRoE8hsHHrgOnHm9iA7TsQ4=/https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"secure_image_thumbnails": {
		"https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/192,sc,sH-8S_pWjnRjMormMb5aQtRoE8hsHHrgOnHm9iA7TsQ4=/https://www.thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"story_feed_id": 8151809,
	"has_modifications": false,
	"comment_count": null,
	"comment_user_ids": [],
	"share_count": null,
	"share_user_ids": [],
	"guid_hash": "0a21ae",
	"id": "254263 at https://www.thesmokinggun.com",
	"friend_comments": [],
	"friend_shares": [],
	"public_comments": [],
	"reply_count": 0,
	"short_parsed_date": "23 May 2025, 8:42am",
	"long_parsed_date": "Friday, May 23rd 8:42am",
	"read_status": 0,
	"intelligence": {
		"feed": 0,
		"author": 0,
		"tags": 0,
		"title": 0
	},
	"score": 0
},
{
	"story_hash": "8151809:e3bd21",
	"story_tags": [
		"internet",
		"crime",
		"friday photo fun"
	],
	"story_date": "2025-05-23 12:42:05",
	"story_timestamp": "1748004125",
	"story_authors": "wbastone",
	"story_title": "Friday Photo Fun Match Game",
	"story_content": "<div class=\"field field-type-text field-field-subtitle\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    Pair up mug shots with each suspect&#039;s alleged crime        </div>\n        </div>\n</div>\n<div class=\"field field-type-filefield field-field-photo\">\n    <div class=\"field-items\">\n            <div class=\"field-item odd\">\n                    <img alt=\"\" class=\"imagecache imagecache-175xUnlimited imagecache-default imagecache-175xUnlimited_default\" height=\"265\" src=\"https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg\" title=\"\" width=\"175\" />        </div>\n        </div>\n</div>\n<p>Examine the booking photos of five arrested individuals and align the defendants with their respective alleged infractions of the law.</p>\n<p><a href=\"https://thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174\" target=\"_blank\">read more</a></p>",
	"story_permalink": "https://thesmokinggun.com/documents/internet/friday-photo-fun-match-game-339174",
	"image_urls": [
		"https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	],
	"secure_image_urls": {
		"https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/sc,snn4Xs-z_cpIm7IJlj5XV22bVDWV5J2X5B1mTJYmNSm0=/https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"secure_image_thumbnails": {
		"https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg": "https://imageproxy.newsblur.com/192,sc,snn4Xs-z_cpIm7IJlj5XV22bVDWV5J2X5B1mTJYmNSm0=/https://thesmokinggun.com/sites/default/files/imagecache/175xUnlimited/photos/contart523front175.jpg"
	},
	"story_feed_id": 8151809,
	"has_modifications": false,
	"comment_count": null,
	"comment_user_ids": [],
	"share_count": null,
	"share_user_ids": [],
	"guid_hash": "e3bd21",
	"id": "254263 at https://thesmokinggun.com",
	"friend_comments": [],
	"friend_shares": [],
	"public_comments": [],
	"reply_count": 0,
	"short_parsed_date": "23 May 2025, 8:42am",
	"long_parsed_date": "Friday, May 23rd 8:42am",
	"read_status": 0,
	"intelligence": {
		"feed": 0,
		"author": 0,
		"tags": 0,
		"title": 0
	},
	"score": 0
}