How to Make Your Boss Angry — Bad Linux Sysadmin Practices

“You’re fired!” yells a red-headed, more or less bald guy in a suit. You’ll probably recognize this from any of a dozen comedy series. Would you like to play the role of the employee in such a scene, in real life? If you happen to be a Linux sysadmin, it’s your lucky day! We’ve got just the right tips for you. Read on for a comprehensive list of ways to make your boss hit the roof and bounce back again.

1. Write undocumented scripts

This one always works wonders. Be sure to do this with scripts that are 1) important for others to be able to work and 2) likely to need an update in about 6 months, but no earlier. Those 6 months will help your memory to forget the tricky parts of the script, so you’re actually treating yourself to a nice challenge in the future! And if somebody else might ever try to replace you, he will have a ball figuring out what all your scripts are supposed to do. Even better: do not remove or mark old versions and unused scripts, and force him to examine them all!

2. Do not keep any back-ups, or only local ones

Back-ups are cumbersome. There is a good reason that many people intend to set them up but don’t actually do it, and the reason is that they’re cumbersome. Did I say that back-ups are cumbersome? Anyway, some bosses may actually keep nagging you about back-ups until you have them configured, so if you really can’t get away without setting them up, be sure to only make back-ups within a server. Why go through all the hassle of setting up remote authentication if you can just add a simple line of cp to your personal crontab? Problem solved! Nowadays hard disks don’t fail much, right?

3. Install updates on live systems

Installing software updates is easy, right? Just whack “sudo apt-get update” and then “sudo apt-get upgrade” into the server terminal, press enter without looking, now run the command again and press “y” before enter without looking because some blundering fool made “no” the default answer, and you’re done. What’s that? The company website is down? Well, it can’t have been the update you just installed, because newer versions are always better. Go check the power cables or something. Even better: blame the employee who was just editing the site using the CMS back-end. You never trusted that software anyway.

4. Install no updates at all

So, installing updates on a live system almost got you fired for bringing the company website down. Fine, let them experience life without updates! Be sure to expose the exact version string of the web server software in the site’s HTML, then wait until the next useful exploit is found for that version. And if the script kiddies don’t find the vulnerable server fast enough, post a challenge on some obscure hacker forum offering a list of all the company’s active e-mail addresses to the first person able to deface the company website. That’ll teach ’em!

5. Put your personal settings in global configuration files

Have you ever had to work on somebody else’s account for a short while? Having to miss all your useful aliases, color settings, vi-style hotkeys and 60 characters long PS1 is just unbearable, right? So why not put those settings in the server’s /etc/profile? That way, you can enjoy your favorite terminal settings on everyone’s account! Doesn’t life just smile at you now?

6. Do not implement redundancy

Redundancy is lame. The very word suggests what’s wrong with it: you’re wasting hardware that could be used for something awesome, like powering your BitTorrent downloads so you can watch Star Trek tonight. And what about RAID? All RAID configurations are lame, except RAID 0, because it actually makes your I/O faster. So don’t do failover configurations, versioned back-ups, mirroring disks or UPS power supplies. Say it’s against your religion or something. Bloody waste.

7. Run a different distro on every server you manage

Let’s do some simple math. Every distro has its strengths and weaknesses, and every server is used for something different. So, install a different distro on every server, to match distro strengths with function requirements. Brilliant! Make sure to complete your collection of different package managers, so that every time you need to install or update software, you get confused between apt/dpkg, apt/rpm, apt/tgz, yum/rpm, pacman, emerge and even pisi. Also, make sure to “accidentally” include a distro that barely receives any updates and delight in the mess of incompatibility that ensues. And of course, enjoy the thrill of solving a single problem in 9 different ways!

8. Eat above your keyboard

Not only does it personalize your keyboard by making it feel extra greasy and causing it to smell after a while, but also it helps you to avoid boring conversations with colleagues who do not understand your ravings about the advantages of emacs over vim. Moreover, you don’t have to hear their complaints about the current IT situation, which are only caused by their own lack of computer skills. Besides, if they really want to talk to you, they can always join you on IRC in your favorite Freenode channel, right?

9. Don’t mind file permissions

File permissions are overprotective. And cumbersome. If people can read each others’ files, there is more social control, resulting in a better working climate. Besides, nobody is stupid enough to store a password in a file. If you can remember your own 40-random-character password, they should be able to remember their variations on abc123 and qwertyuiop. Also, nobody ever stores private files on work network shares, or sensitive files for that matter. No sense in fencing things off.

10. Always keep the default configuration on software

Especially if the comment says “DO NOT LEAVE THIS ON IN A PRODUCTION ENVIRONMENT”. Default settings are always good, because if you have a problem with the software and you Google for it, people’s answers are assuming that you’re running the default settings. Makes find a solution just that little bit easier. And if you have to change a setting, do it at the bottom of the configuration file without commenting the original default setting. Much easier than having to find where that particular setting is already listed.

11. Make all servers depend on a single one to function

Single points of failure are cool. Every sysadmin needs to experience a total breakdown of the IT setup of his/her company at least once, and the more single points of failure there are, the faster that experience will come. So, by all means, make a single login server that is contacted on every login, let servers use their configuration files from a network share directly, and put critical servers in a subnet protected by some outdated version of a firewall distro. In the meanwhile, make sure all internet traffic is routed through a proxy server that blocks traffic based on some 4 kilobyte regex you once conjured up.

12. Do not monitor your servers and certainly don’t set alerts

An appropriate supplement to the previous point! Once your carefully designed single point of failure has done its job, the lack of monitoring and alerts ensures that it takes much longer before things are back up again. Make sure to save your dullest face for the moment the first colleague walks in asking why he hasn’t been able to get anything done for over an hour. Then start (slowly) walking through the default checklist of “have you tried turning it off and on again”, “is the power cord plugged in” and “did you recently see a living unicorn”. Then, when you’re finally fixing the actual problem, the lack of monitoring and alerts ensures you won’t notice it if you break something else in the process. Good times!

13. Give your friend ssh access to the server

So, your friend would also like to do nightly BitTorrent downloads over your company’s huge internet connection. Sure thing! Just give him an ssh account, preferably without double-checking the permissions it has, and let him download away! Nobody’s bothered, right? What could possibly go wrong?

14. Do not use a firewall

Firewalls are nasty. They always get in the way when you want to do something cool, like mounting the server’s “/” directory to your phone using sshfs, or playing EVE Online at work, or streaming the webcam in the company kitchen to Linux is secure by definition, so firewalls are redundant, and we already established that redundancy is lame. Also, don’t do port scans, as they might advise you to change the default configuration, which is against tip #10.

15. Copy huge files over the company internet connection without bandwidth limits

Sometimes, you have to copy a large amount of data to some machine outside the company. It could be a back-up, but that would violate tip #2, so let’s say your external hard drive is full and you want to copy the latest 1080p uncompressed Doctor Who episode to your home computer over the internet. Well, doesn’t that seem like an excellent moment to test the upload bandwidth of the company internet connection? Not only is this a useful statistic to know, but it will also get your 300 gigabytes of sci-fi goodness home that much sooner! Perhaps it will make Reddit load a bit slow on your pc, but fortunately, there’s an app for that!

16. Take home confidential files on an unencrypted usb pendrive

And by tip #2, don’t have a back-up of them either. You won’t be as stupid as those people who lost important pendrives in public transport or things like that. Also, pendrives never fail. And encryption just wastes space and CPU cycles.

17. Always leave the physical terminal of every server logged in

It’ll save you a few keystrokes and some racking your brain for the password. No one else will ever even think of touching the server’s physical terminal, right? No colleagues, cleaners, janitors and certainly no hackers, who always play by the rules and hack over the internet. Completely safe, this.

18. Find the cheapest hosting for the company website

And put the company personnel database on the same hosting account. How could a dollar a month ever be wrong? Besides, shared hosting makes the company website feel less lonely. And if it loads slow (if at all) or suddenly gets hacked, you have the hosting provider to point at instead of taking the blame yourself. Think about it! No need to apply updates, no configuration files to crack your brains over, easy phpMyAdmin access (preferably without SSL) and a fancy control panel to click around in. Oh, and “99% uptime guaranteed” means the website isn’t down for more than 7.44 hours a month, or you’ll get your dollar back!

19. Use the same password on every server

And preferably a password that you also use in your private life. Look at how fast you can type this password because you type it all day! Share this achievement with all your Facebook friends and be considered the most awesome geek they know!

20. Enable root login in sshd_config

Having to su or type sudo all the time is cumbersome bothersome (back-ups are cumbersome, remember?). Therefore, enable root login for ssh, keep it running on the default port (saves typing -p every time) and use a short root password for convenience. Now, when you see strange failed authentication warnings for the root account in the sshd logs (presuming you actually check those at all), comfort yourself by saying that they will never guess that your short root password is l3tm31n instead of letmein.

21. Change the server network configuration over ssh

And don’t add a timed reversal script or anything like that. Locking yourself out won’t happen to you like it happened to so many others, because of your secret years of ninja training. Or whatever. Bonus points if you don’t have physical access to the server.

22. Don’t invest in your Linux knowledge

Stay away from the other articles on this site, and especially from Block it in your firewall! Wait, you don’t have one because of tip #14. Ahem! Change the ip to in /etc/hosts! This site is insolent enough to presume that you could possibly learn anything about Linux. The arrogance! Your vast knowledge is unparalleled! Besides, copy-pasting solutions from Google always works, so there’s no need to understand anything about them.