So, Front Page Category didn’t cut it for me. It was to “exclusive”, so to say.
The plugin did exclude the category “Blog” from the frontpage, but it also made it impossible for the category posts widget to display posts from the category. That wasn’t good, cause I do want the blog to be more visible than it would with only the tab in the navigation bar.
I found the plugin Simply Exclude instead. It’s actually a bit confusing to use, but once I got my head around it, it does what I want it to. I’ll recommend it for sure. It seems to allow for some good fine tuning.