Tech Question about my Linux Server

I know we have a lot of techies here, so maybe someone can help or give advice…

I am running my own small linux cloud server thingy with ubuntu/plesk because … [long history] It’s just a tiny little thing running a webserver for my website, a few mini services and my own mailserver with postfix/dovecot. My linux knowledge used to be better than it is right now. I haven’t needed it much I admit.

I’ve been experiencing memory issues with the mailserver and looking into the logfiles I’ve been seeing failed login attempts to postfix from an IP block. The danger is not from them getting into the server, they are just trying random/spammer-known mail addresses that do not correspond to any existing credentials. They are just using up my cpu/memory…

I’ve configured fail2ban to trigger faster and keep them away longer but I would kind of like to get rid of the whole block of addresses permanently for my postfix (I checked they all appear to be from the same provider–not mine, not even my country). Can I do that? Is it advisable?

2 Likes

That should be fine. (Though double and triple check the address details. It is entirely possible accidentally to include one’s own remote admin address in a ban. Not that I would ever do something that stupid. Ahem.)

I don’t speak Plesk; I just do straight command line stuff. But setting an IP-level ban with iptables/whatever filtering tool should be cheaper on your CPU time than allowing an SMTP connection that then gets rejected.

2 Likes

I am quite … ahem familiar with my hosting provider’s rescue system setup for … some reason :wink:

Ah, iptables, my great nemesis… but yeah that sounds like an idea.

edit:

I expected nothing else :stuck_out_tongue_winking_eye:

2 Likes

The downside obviously is that if someone sends you legit email from that block it’ll bounce. Depending on the block, that may be unlikely. One of my servers is hosted with OVH, because it’s very cheap… but nobody would reasonably expect anyone to accept email from there.

2 Likes

It’s already looking much nicer and my mail log looks like right now it’s just me and my partner and the occasional spam email arriving, as it should be… well not the spam but even with my setup it’s hard to get rid of all of it.

2 Likes

And today… I accidentally destroyed my nextcloud installation while trying to update. This is not a question. I fixed it the old fashioned way: setting up a fresh install on a different url because the old one didn’t work anymore not even for a fresh install…

And I still don’t know why, luckily we are not using it extensively so far and I did have a database backup from right before because I could kind of see this coming from a mile away.

The server installation is four and a half years old (yes yes, I update it obviously but still…) maybe it is time to start over from scratch. So much ugliness hidden in the configurations I don’t even want to look at it. At least this time I didn’t try to run my own hypervisor (figuring out KVM was a special experience I do not care to repeat)

I really am not an IT admin. I shouldn’t be running a server on the internet. BUT I AM :smiley: I got used to having access to internet connected servers at uni…

2 Likes

The modern SA methodology is focused around automated deployment of templates. In many organizations, it is completely normal to look at something, think “hmm, this isn’t working the way it’s supposed to… I’ll just wipe it and redeploy” and then do just that.

I’ve used Docker for almost all of my recent (last ~6-7 years) server configuration/deployments. Then, I make sure I backup the data for the applications (which is organized by app in a predicable manner), the Dockerfiles and any necessary deployment scripts.

I’m probably going to experiment using UnRaid in the near future as a supplement to my primary Docker host. UnRaid, allegedly, has some pretty nice WUI wizards to manage VMs and containers, along with automated backups. It’s been on my todo list for a while, but I actually haven’t set my servers back up since I moved (I need to tear down some shelving that was here when I bought the house in order to get my server cabinet where I want it… recently bought a reciprocating saw to do the demo with)

1 Like

I have experimented with docker on my old dedicated server in addition to KVM. (RIP 2016… just as pokemongo started… has nothing to do with pokemon I just remember playing a lot after the server died)

It was fun while it lasted…

I just upgraded my virtual server slice from 2GB to 8GB Ram so now I can do a little more than I used to. But I am not sure I can docker around on such a smallish VM and with plesk on top because I really need someone competent to handle mailserver installation. And the mailserver is the main reason for this because we never could find a mail provider that does what we want.

  • handle dozens and dozens of different addresses with automatic filtering into folders (I have written a tool that allows us to create addresses on the fly add them to a category let’s say “games” and all mails to this address will automatically land in the “games” folder
  • provide a current webmail experience
  • handle all the privacy issues like I want

The first I could have had on the server I am still part of with our uni friends but that runs on debian-stale and never got around to a current version of roundcube.

And the first part is crucial to how we handle email and there is NO email provider that allows for unlimited mail addresses and the filtering we love.

Why unlimited?

Each website that needs me to create an account and provide a mail address get’s their own address. So I always know which ones were hacked or sold my data. Mostly they are being hacked: adobe, dropbox and kickstarter have all been moved to instance “2”… so far I haven’t had to issue a 3rd address for any website.

2 Likes

I’m a tad disappointed. There was a fantastic resource that I used to point people towards that would have made my self-education a lot easier. It looks as though it’s targeting a more experienced audience now and doesn’t look at newbie friendly, but I still highly recommend lookin at https://geek-cookbook.funkypenguin.co.nz, and perhaps How to run Nextcloud in Docker (behind Traefik) |・∀・ specifically?

I use Google Apps for Domains (or whatever it’s called now). I’m on an old, grandfathered plan though. Gmail for domains allows you to set a “catchall” Inbox. So if you sent an email to somenewemailthatsneverbeenusedbefore@[mydomain], it would land in my user’s inbox on that account. And then you can use Gmail rules to automatically do things based on the email address that was used.

In practice, I don’t actually leverage this as much as I should.

1 Like

Oh catchalls exist for a variety of providers. But I specifically do not want a catchall either… I reject everything that is not configured. I hate spam.

1 Like

So the setup is something that took me a bit to figure out especially when moving from exim to postfix with this incarnation.

  1. create new entry to “mail database” via php script
  2. select domain (we have several obviously) and category
  3. besides mysql storage the script writes a nicely formatted alias file for postfix where the category becomes part of the “target” address for each alias
  4. incron notices the file has changed and moves it over to the postfix configuration directory
  5. postfix recognizes the categories and writes them into a mail header
  6. mail filters are configured to filter by the mail header “x-category”
  7. never sort emails anymore

There’s a whitelist for private mails not that I get more than a dozen of those in a year anymore…

1 Like

I think your mail server is probably a bit too much to try to host in an environment like Docker or Kubernetes.

But it’s probably still important to be able to reproduce a valid deployment of it rather than having to maintain the existing deployment (I really don’t care to describe the many, many cases where there’s a really, really old server that is mission critical and it has to be nursed along well beyond its lifetime because it’s running software/configuration in such a way that nobody knows how to recreate it)

Foreman might be the way to go. It’s like docker (in a way) but for servers rather than containers.

1 Like

At least I’ve got some documentation of how I created the filter header:

//not the solution - Create File /etc/postfix/addheader: /(.+filter).+@(.+)/ PREPEND X-Keyword-Filter: $1

use some web tools to check general mail server health security verify that it is not an open relay!
Test headerchecks with postmap -q “To: xxx-filter-user@domain” regexp:/etc/postfix/addheader

//not the solution - Alternative attempt: use mysql → to access remove chroot flag for smtpd in master.cf (oops) + allow localhost connects through firewall for mysql

//mysql header: user=\ password=\ dbname=\ query=select concat(‘X-Keyword-Filter: ‘,category,’, ‘, destination) from Address where address=’%s’ and protected=0;

Solution: create hard coded file from php script containing regex and prepend lines as above and include in main.cf (incron should fetch this the same way)

main.cf: smtpd_recipient_restrictions = (…) check_recipient_access regexp:/opt/postfix/categories I did it even if it looks like this:

I’ll see myself to the corner for horrible admins :smiley:

I really try to get by with minimal work on the server. I saw that hetzner now has such small VMs that I could have one for just 3€ a month to play around with to figure out how to have a better solution than what I am doing now. Because right now everything is a bit precarious (has been for years)

1 Like

Job N, 1998: “You’d like to have a permanently-connected server, right? Just bring in your hardware and put it on the shelf there.”
Job N+1, 1999, with a competitor company: “What, a permanently-connected server? Hmm, we’ve never done that before…”

I can see the virtues but I like to get my hands dirty and see why a thingy is failing, rather than just reboot/retry/reinstall.

But then there’s that whole desire for privacy. Yeah, if you give that up it all gets much simpler…

2 Likes

I have a server that I literally built over a decade ago. I’ve kept up on software updates and whatnot, but there’s a good amount on there that I just don’t remember anymore.

A good portion of that functionality has now been moved to docker. Why? Because docker becomes self-documenting (I’m a network engineer… self-documentation is usually about all you get when you are a network engineer… that and a Visio diagram that an idiot (sometimes I’m the idiot) made before the work was actually done)

2 Likes

If I were a hobbyist with free time, this would be a resounding: “hear hear!”

Businesses can save tons of engineer-hours by wiping it and redeploying it; if that doesn’t work, you kick it up to the grey beards to dive in and figure it out (and then update the deployment templates so that it can be wiped clean again and redeployed)

I can save tons of engineer-hours by not having to think about it unless a wipe/redeploy fails to fix the problem, which is good for me because all of my home server SA time competes with my other hobbies (that I can do once my kids are in bed). And since I run a Plex server with a number of movies and shows that placate my children, being able to wipe-redeploy in about 5 minutes is a lot better while also dealing with toddler-meltdowns.

TL;DR - if I had more free time, I probably wouldn’t have embraced the “wipe/redeploy” methodology as much as I did. But now I’m thankful for it.

2 Likes

It’s been like that ever since a windows reboot would fix windows… and probably before.
At least Linux gives us a bit more access into the why’s of it all.

If I was working on this server professionally none of what I am doing would “pass” … if there was anyone but my partner and me depending on that server I couldn’t handle it like I am doing. Even just one person outside the household would make it so I’d have to be more professional.

Absolutely, but alas the last Cambridge Analytica has been a while and everyone is back to saying “I have nothing to hide” and I live in a country that is a bit famous for people valueing their privacy. and they really do they just focus on the wrong pieces and so we have a Corona app that doesn’t even have an opt-in to share data and we pay cash at the bakery while everyone uses gmail – no they don’t they use gmx.de and web.de not knowing those two are the same company :wink: but they use facebook and whatsapp—

3 Likes

I think (hope) that privacy rights are visited soon in the US. I have reason to believe they might due to recent political events (which I don’t want to get into).

But that is a different topic altogether.

1 Like

can you give me a pointer where to start reading up on this? My “google” (*) fu is apparently suffering.

I spontaneously added another server to my “project” and found out that plesk is not available for new cloud servers @ hetzner and so I am in search for a fresh solution. because my attempts at cleaning up the old installation reminded me it is an old ubuntu version and there is so much stuff there that isn’t nice… …

(*) when I say “google” I mean duckduckgo or another search engine. I don’t google much anymore

Sorry, I certainly did not give enough information. And shame on Foreman for choosing such a terrible product name (like Chef did)

https://theforeman.org/

Automatic lifecycle management for physical and virtual servers

1 Like