Having a tool to warn me that I'm about to do something I shouldn't be allowed to do is okay. Having proper access control would be better.
(Fun: 'boulette' is French for 'dumb mistake')
Some of us live on the "granted" side of that access control. I'm an SRE; I have access to a fair number of things. I worry about making dumb mistakes like that in the article due to sleep deprivation. The warning here might very well be a warning that you're about to do something that you are allowed to do.
In the specific example … sometimes rebooting a production machine is what needs doing. Ideally, there's a second VM, ideally, the services on the machine would gracefully drain, ideally, it wouldn't matter. Ideally. But the reality is that often they don't, whether because of developer laziness, incompetence, some ugly confluence of bullshit bugs that conspire to prevent sanity, etc., more often than not I want to be careful even when the process/theory would like to say "this is a fully automated & completely safe operation" — and half-asleep is not careful, and accidental allegedly safe commands while half asleep is where Murphy likes to rear his head.
I've definitely used techniques like this. Red prompts that scream "You're SSH'd into PROD dummy" also come to mind. I keep my kubectl context fixed to "local laptop" so that I am forced to always type out --context the_production_cluster when I want that.
Temet nosce.
Fun followup: the image is from the French movie "Le dîner de cons", where the character depicted says "Oh la boulette", after making a gaffe.
Is this for ephemeral dev boxes? Does shutdown suspend billing on AWS/cloud type hosts?
I my case, I use atuin a lot and automatically retype long commands and sometimes as root.
The regularity of this actions tend to lower my attention, but the fact that many users depend on what I do on big machines and with lots of priviledges isn't less true.
Danger comes from getting used to it.
It doesn’t if you use a tool like Mosh [1] (I’m not affiliated)
[1]: https://mosh.org/
Is it as secure as regular SSH when configured properly?
Here's a conversation on HN about it. Mosh uses an initial SSH connection to establish a session, and I located elsewhere that the communication thereafter is handled via AES-128 encrypted UDP traffic. The server process itself seems to only live the life of the session, and doesn't require escalated permissions.
I can't imagine enterprise or government adding it to their stacks, but for connecting to personal stuff doesn't really seem like a big risk.
> Mosh doesn't listen on network ports or authenticate users. The mosh client logs in to the server via SSH, and users present the same credentials (e.g., password, public key) as before. Then Mosh runs the mosh-server remotely and connects to it over UDP.
...
> ... Then Mosh runs the mosh-server remotely and connects to it over UDP.
Are UDP sockets immune from port scanning? Regardless of the answer to that, the sentence should be rewritten because it sounds like nonsense as is
The number of software systems I've seen designed to allow the clueless to bumble their way through operations is much higher than the number of companies I've seen that attempts to train better operators.
Maybe its the experience of working on tractors, where the PTO has fun side effects like "degloving" where, sure theres a guard but more importantly theres attention not to put your hand, hair or clothing near the spinning torque monster of doom. Theres no option to make that purely safe economically. Same goes for machine tools and other heavy equipment. The people who work around such things have been to my experience more capable at accomplishing difficult, diverse tasks.
Where was the last company that had anything equivalent to the lowly "forklift certified" for prod? Its a very rare shop I've seen invest in any sort of across the board training for command line skills, outages, pre-requirements for delicate operations. We don't invest in people being better, being more capable. I think because we have internalized an owner/management point of view that workers are fungible and training is a waste, while software system guards are investments.
As a worker, I don't agree with that. I don't agree with building systems to be powered by the lowest educated, lowest paid meat popsicles yet I think thats the strategy behind this.
Next time you go to build a system like this, consider who's logged into production and can they be trained to be more capable, more attentive operators. In the long run, I think it will end up with a better industry overall.
This is something that could conceivably happen to people who are properly trained. This also means it doesn't fall prey to the usual "do you really want to do this? yes - no" prompt, where you just get used to automatically hitting "yes." Even if you habitually enter the hostname, it'll be wrong if you execute the command in the wrong terminal.
But as an idiot, I’m here to tell you it isn’t idiot proof. In a moment of suboptimal attention, I could easily ask myself “What host am I on?” rather than the more appropriate question “What host am I trying to shutdown?”
In my case it may be quite a bit worse because I have hostnames in PS1 specifically to avoid running any command on the wrong host. With the hostname right in front of me I could easily accidentally habitualize just typing the hostname I see.
Over time, I imagine this would befome more likely rather than less, so although I think it probably helps, I am sympathetic to GP’s view of futility.
/s
I agree, and that's a problem. Now with LLM, we're training everybody to not know how to find information by themselves (we call that "prompt engineering" so it cannot be bad, right? /s).
(This isn't to say that training is useless, just that more training isn't the best and only solution to all problems, nor will a lack of careful interface design magically create more capable operators)
Separately the only reason tractors don't get the same treatment is because society doesn't care about rural men in the same way they don't care about soldiers. In comparison cars have been largely regulated for safety, because the people who die in car crashes come from a wider swath of society.
When you are working with the PTO you know you are working with the PTO. You are standing next to the tractor. The PTO should always be handled with the understanding that it can be dangerous. Here people are doing something mundane, everyday and not dangerous (shutting down their own laptop) and suddenly it becomes dangerous because they mixed up the terminal they typed their command in.
It is as if sometimes your alarm clock would be replaced with a PTO. Your alarm beeps, you reach out to turn it off as always and bam your hand is gone. That is the situation we are talking about here.
> I think because we have internalized an owner/management point of view that workers are fungible and training is a waste
This is not something you can train out of people. Your best trained, and most skillful operator can do this mixup. This is not that someone doesn't know what "sudo shutdown -h now" does. They do very much know it. Like the palm of their hand. What they don't notice is that they are typing it into the wrong terminal.
Like, yes, I am a console jockey and prefer working in shells with a tiling window manager, keyboard only control and such. I will however always shutdown my workstation with the mouse (or trackball rather) through some UI of the desktop manager, or a desktop manager specific way.
Otherwise, I am normalizing the use of sla-dangerous commands like "shutdown". It takes that little bit of fear and respect out of those commands if you use them daily for no good reason or if better choices exist. Like, don't turn your alarm off by cutting it's wire and re-soldering it later.
And similar, if I need to reboot production systems, I'll much rather reach to some control interface of the virtualization, or use something like ansible to dry-run these dangerous tasks first.
Or it terrifies me how care-free some people are with "sudo rm -rf". I've caused myself so much pain with rm. "sudo -u app-user rm -rf" is right there, or even better, "sudo -u app-user find -name foo -print > stuff; cat stuff" and later some "xargs -i rm < stuff" and "xargs -v rm < stuff". Yes it takes a minute more to do, but it prevents ... accidents.
This is a cool ability of dpkg that is not well known. You can use it without making a package as well.
Ps. At Adaptive (http://adaptive.live), we have kind of productized something like this.
Reduces number of fat fingering disasters.
Also, the problem with a Y/N question is that when you are bored and/or in a hurry, you only skim the question and muscle memory takes over, and you hit Y and then you realise a few seconds later that you rebooted the wrong machine. This is why molly-guard makes you enter the hostname of the host you want to shut down.
Maybe this would help:
https://serverfault.com/questions/136515/read-only-bind-moun...
mahler ~ # mkdir /roroot
mahler ~ # mount -o bind,ro / /roroot
mahler ~ # chroot /roroot
mahler / # passwd
Enter new password:
Re-type new password:
passwd: Authentication token lock busy
passwd: password unchanged sudo systemd-nspawn --directory=/ --read-only --ephemeral --volatile=yes
I believe this specific set of options relies on (BTRFS) file system snapshots for performance. It's possible that you can get it to work on non-BTRFS systems by providing another combination of command line variables, but the default setting is to copy the file system tree to a temporary path.
You can also pass parameters like --volatile=state (so you can write to /var) and --volatile=overlay (so you can "write" to state, but all changes are discarded after the container exits). --volatile=state is useful for extracting data from a temporary read-only system, --volatile=overlay is useful for running tools that crash if they run on a read-only filesystem.
You can use overlayfs + chroot.
I created a seccomp DSL to make this kind of stuff easier [0] (an example of dropping network access is at [1])
[0] https://chiselapp.com/user/rkeene/repository/bash-drop-netwo...
[1] https://chiselapp.com/user/rkeene/repository/bash-drop-netwo...
If you're in a mental state where you can shutdown a prod server while thinking you're on a dev one, you certainly won't think to type "wtf command", no matter "wtf" actually is.
#!/bin/bash
if [[ $SSH_TTY ]]; then
read -p 'You are in SSH. Are you sure (enter hostname for yes)? '
[[ $REPLY == $(hostname) ]] || exit 2
fi
exec "$@"
alias shutdown='protect.sh shutdown'
alias reboot='protect.sh'
alias sudo='sudo ' # Don't allow sudo to bypass the protection. Can do the same with doasSince it's so old, it is present in all the systems - "sudo apt install molly-guard" on your server, and your shutdown, reboot, etc.. are all protected, no need for 3rd party tools.
For the specific purpose of shutdown, there has been a solution for decades: molly-guard.
Moi ça m'a fait sourire !