Self-hosting documentation


#1

Hey @samuelclay, is there more robust Documentation on configuring and admin’ing a self-hosted NewBlur instance? It really is a great project and while I did find the notes in the readmes on GitHub as well as some bits in GitHub issues and this forum, I’m still having issues finding my way around.

While I’m doing my best to filter through it to figure out what I need to learn, trying to get a NewBlur instance running feels a lot like running through a house of mirrors at full speed so I figured I’d see if maybe I missed some docs page or folder, especially given how great this series is.


#2

You must have read my mind. I’ve spent the past week thinking about how nice it would be to Docker-ize NewsBlur’s install to make it easier for anybody to get running.

For now you just need to have all of the database servers running and ready to go (postgres/mysql, mongodb, redis, and elasticsearch). Then either run a development server or configure your nginx/haproxy to point to your gunicorn instance. It’s not for beginners right now but it’s not all too bad.

What are you having an issue with? Are you seeing error messages that you don’t know how to resolve?


#3

Thank you for the quick reply! I’d be happy to help test whenever that happens. I have a docker stack up and running with base functionality (feeds can be added and forced to update, searching works) using the docker image from the NewsBlur repo with minimal modifications.

I don’t mind using a little elbow grease to get this working, but I’m not familiar with a lot of the tools/frameworks used to make it so I appreciate all the help I can get. Here are a few of the issues I’m in the process of working through:

  • Feeds aren’t auto-updating - I found your pointer about celery workers and figured out that I can use e.g. ./manage.py celery status from inside the app container (Error: No nodes replied within time constraint.). It looks like a celery beat config comes with settings.py and the issue may simply require getting a worker up and running when the container starts but I’ve yet to work out the incantation(s) for this.

  • My instance can’t send email - I found this thread and your mention of mailgun in regards to newsletters but I was hoping for SMTP support. Can I easily configure an SMTP server instead or will that require some patching?

  • Limiting user registration - I’d like to control which users get created. I did see AUTO_ENABLE_NEW_USERS but my hunch is that this simply requires a user to verify their email before allowing them to login. Is there a way to disable user registration?

  • Couldn't find spam.py when running ./manage.py commands - I noticed spam.py is listed in .gitignore but I’m not sure – and am curious – how I would generate my own. I suspect this isn’t much of an issue if I’m able to control user creation+trust my users though I have yet to suss out what havoc an anonymous user can wreak.

  • Turning off Debug causes assets to fail to load - I flipped debug mode to False (with DEBUG_ASSETS = True) and assets no longer load, though they otherwise load okay with DEBUG = True.

Any pointers you can lend are greatly appreciated; even in its current state, using NewsBlur is like a dream compared to my FreshRSS instance.


#4

For those playing the home game:

I’ve yet to divine anything useful enough to resolve the above issues, but I’ll keep plugging away when I have the time, at least until this game of hide and seek become untenable. To finagle a somewhat usable state I’m simply running a handful of commands every ten minutes:
./manage.py refresh_feeds --force
./manage.py collect_stats
./manage.py index_feeds

and then restarting the container regularly (since DEBUG mode leaks memory like Snowden).

However I have noticed that older feed items have begun to disappear – I’ll have to find some time to dig up some useful data on this, but I noticed the number of unread updates decreasing upon a refresh.

What I do know is that I have several feeds that previously had some amount of unread items from the past week and today a bunch of them simply vanished. Looking closer at this, it seems that Newsblur simply deleted all items in some of my feeds from a few hours ago.

Probably a good thing I still have that FreshRSS instance kicking around :joy:


#5

There’s a setting for this in settings.py:

DAYS_OF_UNREAD          = 30
DAYS_OF_UNREAD_FREE     = 14
DAYS_OF_STORY_HASHES    = 30

As for stories to keep around, edit apps/rss_feeds/models.py and change story_cutoff to 5000 instead of 500.


#6

Huzzah! Thank you @samuelclay! I’ll throw those in and give it a good kick :comet:

Any more hot tips on the other issues I have from last week?


#7

For those playing the home game:

I did go into apps/rss_feeds/models.py and noticed that simply changing cutoff = 500 to cutoff = 5000 likely won’t do quite what I want because the rest of the story_cutoff function will immediately tune the cutoff value based on the number of premium subscribers as well as how ‘well read’ the feed is [I think] so I wholesale replaced the function to simply be
def story_cutoff(self): return 5000
and redeployed, so hopefully that will do the trick.:crossed_fingers:

I didn’t get much of a chance to verify this because my instance has been failing to update on the reg. :tired_face: I noticed that the ./manage.py refresh_feeds --force command that I run (as a workaround to the celery worker issue mentioned above) hangs every few days or so without an indication of an error. Unsurprisingly, not much progress on why this happens (despite the DEBUG mode bug, funnily enough :joy:) but I’ve certainly got my work cut out for me!