Curlbomb - Server Automation for the Wayfaring Hacker
May 19, 2016 at 10:13 AM | categories: curlbomb | View Comments
The industry is full of configuration frameworks: chef, puppet, fabric, salt, ansible, etc. I've also personally worked on several proprietary ones too. Most of these frameworks are pretty good at what they do. It's a good time to be alive if you're into devops.
All of these frameworks require some amount of setup to get going (some more than others.) In this way, they are geared towards running in a preconfigured environment, and are not easily used with fresh hardware that is outside of this environment. Mostly I mean having SSH keys installed, software dependencies setup, and (usually) the client daemon for your chosen automation framework running:
- chef - Requires chef-client preinstalled on the client.
- puppet - Requires puppet agent preinstalled on the client.
- fabric - Runs entirely through SSH, no client is necessary, but does require SSH keys to be installed.
- salt - Requires salt-minion preinstalled, or to use salt-ssh which requires SSH keys to be preinstalled.
- ansible - Runs via SSH, so requires SSH keys to be preinstalled.
If you're running an enterprise, this extra setup is not a problem. You just bake the setup into your OS image, including the SSH keys, software dependencies, and a systemd service file (I don't get the hate) to run your client. Spin up a new instance with that image, and chef, puppet, or whatever can see it and control it.
But I'm a wayfaring hacker. I encounter new systems, on new networks, all the time. Here's a few examples where it is not so easy to setup these tools:
- Installing Linux on a computer using a Live CD. Live CDs typically have a bare minimum of packages installed and don't have an SSH service running, let alone have your SSH keys installed.
- Being given hardware from an IT organization, or from a partner company. In these cases I've usually been told to use IPMI or I have been given a passphrase rather than an SSH key.
- Using an Open Source image for your virtualization platform. Maybe you don't trust yourself to perform security updates on your images. Maybe you feel safer running the base image provided by the distribution.
- Cloud instances where you forgot to give it your SSH key. Yea, typically I just smack my head and redeploy with the key, but this happens to me fairly regularly.
- Embedded systems, or Virtual Machines, where an SSH service and automation client is overkill.
- You're at someone elses desk, they're logged into the system with their own key. Maybe you don't even have an account.
It's not that hard to copy your SSH key into a
~/.ssh/authorized_keys
file, so what am I ranting on about here? I'm
about to propose something even easier.
The wayfaring hacker is very different than the enterprise. He has more limited resources, more limited time, and is doing things very often in a one-off fashion. The wayfaring hacker doesn't have time to create images for all the systems he uses. He may not have the resources to dedicate a server to configuration managment and prefers to just use his laptop. Wayfaring hackers still find homes within enterprises, but his default way of working is to ignore enterprise standards of setting up data centers and creating system images. The wayfaring hacker bootstraps from default Linux distros. The wayfaring hacker regularly tries novel things on cloud instances for only a few hours, then shuts them down, only to rebuild them again a bit later.
Ad-hoc server automation
I've been using the following method of scripting the bootstrap of new machines for several years now. I have to assume I'm not the only one who does it this way, but I don't see it referenced too often. It doesn't use any of the above mentioned configuration management tools. When you're doing something simple, on just one machine, they're overkill. I just do this:
gpg -d ~/git/machine_scripts/my_server_setup.sh.gpg | ssh my_server
This one-liner is great. Let me enumerate the ways:
- It uses tools commonly installed almost everywhere.
- It uses ssh, so everything is encrypted on the wire.
- The script is executed directly, without saving it on the remote client.
- The script is safely stored, gpg encrypted, on my laptop. It is only ever decrypted right before being sent over the wire.
- I edit the script in Emacs, which supports gpg encryption out of the box.
- I store the script in a git repository, so it's always ready to use. If I need to redeploy, just power cycle the server, wipe the drives (or spin up a new cloud instance), and rerun the script.
- Passphrases or other credentials can be put directly into the script because of the security this affords.
This makes it really easy for me to write a script for a specific machine, and keep that script safe on my own laptop (where it's easy to edit with any editor I have installed locally I might add.) I can redeploy my cloud instance, and rerun my script on it to do whatever I want. It's got none of the niceties (or error handling) of the big configuration management tools, but it's great for simple scripts (including scripts that kick off jobs for those other tools.)
But this still has the same problem as before, the machine still requires an SSH server, and it requires that my key is setup.
Server automation with curlbomb
The gpg | ssh
trick is very similar to another well-known trick:
curl | bash
. It's probably the most popular way of installing open
source software these days. It's everywhere (a few examples:
docker,
sandstorm, even
Salt.)
The great thing about curl | bash
is that it has very minimal
requirements on the client end... you guessed it: curl and bash. You
don't need an SSH server (nor ssh keys). All you have to do is type
the command and stuff happens. Magic! There's
a lot of
reasons
to
not do that
though (and some that make a pretty good argument it's
ok),
which this article doesn't really get into, I'm just planting
the idea.
curlbomb is a twist on this idea. Instead of hosting an install script on a public webserver (that anyone can download and run), curlbomb starts a private webserver that requires an authentication token to download a script. You run this on your personal computer, where you have stored a script:
curlbomb run ~/git/machine_scripts/my_server.sh.gpg
Which outputs the following text to the screen:
Paste this command on the client:
KNOCK=KfG2LkxEkrj4 bash <(curl -LSs http://192.0.2.100:56423)
curlbomb stays running until you paste that bit of text into another
computer's shell. That computer will connect to your server, provide
the preshared key (KNOCK), download the script, and pipe it into
bash. Once the script has been served one time, the server shuts
down. If you ever want to run the script again, you have to restart
curlbomb. In essence, curlbomb is a private, one-time-use, curl |
bash
that only you can run. You write the script, and curlbomb
ensures that it's transported securely to your client machine over TLS
(SSL.) You gain security benefits similar to the gpg | ssh
trick
(GnuPG+OpenSSL+knock vs GnuPG+OpenSSH), and you also gain the dependency-free
nature of curl | bash
.
curlbomb can forward it's HTTP port through another server via SSH. This allows you to run the server on your laptop, from anywhere in the world, as long as you can ssh to another server you control. Any client, that can access the internet, can then download the script from your laptop by using the server as a proxy.
Let's look at the same scenarios I said was hard to automate with traditional configuration tools, this time with curlbomb:
- Installing Linux on a computer using a Live CD - Stick the usb drive/CD into the computer, setup the network connection, and type a curl command.
- Being given hardware from an IT organization, or from a partner company - Log in via IPMI, type a curl command.
- Using an Open Source image for your virtualization platform - Log in, type a curl command.
- Cloud instances where you forgot to give it your SSH key - Log in via the vendor provided console, type a curl command.
- Embedded systems, or Virtual Machines, where an SSH service and automation client is overkill - Log in, type a curl command.
- You're at someone elses desk, they're logged into the system with their own key - Type a curl command.
In essence, your dependencies for curlbomb on the client are:
- a network connection.
- curl installed.
- a way to type the curl command.
Isn't that easy? No opening up ports to the client. No running daemons on the client. No passwords or SSH keys need setting up. No maintaining images. No need to ask questions on the client to fill in passwords or other setup questions (just put them in the script.)
Consider the following script I use to do the initial setup for a server. This creates my user account, sets my password, sets up my SSH key, and installs salt-minion so that I can now control the machine with saltstack:
#!/bin/bash user=ryan pass=hunter2 sshkey="ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAmo9sQ6iAHDH8Gg1vd1js3R...." # Create user account: create_user() { useradd $user mkdir /home/$user chown $user:$user /home/$user chpasswd <<EOF $user:$pass EOF # Install SSH Key: mkdir /home/$user/.ssh chmod 700 /home/$user/.ssh echo "$sshkey" > /home/$user/.ssh/authorized_keys chmod 600 /home/$user/.ssh/authorized_keys chown -R $user:$user /home/$user/.ssh } install_salt_minion() { # Saltstack GPG key from https://repo.saltstack.com/apt/debian/8/amd64/latest/SALTSTACK-GPG-KEY.pub cat <<EOF | apt-key add - -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG v2.0.22 (GNU/Linux) mQENBFOpvpgBCADkP656H41i8fpplEEB8IeLhugyC2rTEwwSclb8tQNYtUiGdna9 m38kb0OS2DDrEdtdQb2hWCnswxaAkUunb2qq18vd3dBvlnI+C4/xu5ksZZkRj+fW tArNR18V+2jkwcG26m8AxIrT+m4M6/bgnSfHTBtT5adNfVcTHqiT1JtCbQcXmwVw WbqS6v/LhcsBE//SHne4uBCK/GHxZHhQ5jz5h+3vWeV4gvxS3Xu6v1IlIpLDwUts kT1DumfynYnnZmWTGc6SYyIFXTPJLtnoWDb9OBdWgZxXfHEcBsKGha+bXO+m2tHA gNneN9i5f8oNxo5njrL8jkCckOpNpng18BKXABEBAAG0MlNhbHRTdGFjayBQYWNr YWdpbmcgVGVhbSA8cGFja2FnaW5nQHNhbHRzdGFjay5jb20+iQE4BBMBAgAiBQJT qb6YAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRAOCKFJ3le/vhkqB/0Q WzELZf4d87WApzolLG+zpsJKtt/ueXL1W1KA7JILhXB1uyvVORt8uA9FjmE083o1 yE66wCya7V8hjNn2lkLXboOUd1UTErlRg1GYbIt++VPscTxHxwpjDGxDB1/fiX2o nK5SEpuj4IeIPJVE/uLNAwZyfX8DArLVJ5h8lknwiHlQLGlnOu9ulEAejwAKt9CU 4oYTszYM4xrbtjB/fR+mPnYh2fBoQO4d/NQiejIEyd9IEEMd/03AJQBuMux62tjA /NwvQ9eqNgLw9NisFNHRWtP4jhAOsshv1WW+zPzu3ozoO+lLHixUIz7fqRk38q8Q 9oNR31KvrkSNrFbA3D89uQENBFOpvpgBCADJ79iH10AfAfpTBEQwa6vzUI3Eltqb 9aZ0xbZV8V/8pnuU7rqM7Z+nJgldibFk4gFG2bHCG1C5aEH/FmcOMvTKDhJSFQUx uhgxttMArXm2c22OSy1hpsnVG68G32Nag/QFEJ++3hNnbyGZpHnPiYgej3FrerQJ zv456wIsxRDMvJ1NZQB3twoCqwapC6FJE2hukSdWB5yCYpWlZJXBKzlYz/gwD/Fr GL578WrLhKw3UvnJmlpqQaDKwmV2s7MsoZogC6wkHE92kGPG2GmoRD3ALjmCvN1E PsIsQGnwpcXsRpYVCoW7e2nW4wUf7IkFZ94yOCmUq6WreWI4NggRcFC5ABEBAAGJ AR8EGAECAAkFAlOpvpgCGwwACgkQDgihSd5Xv74/NggA08kEdBkiWWwJZUZEy7cK WWcgjnRuOHd4rPeT+vQbOWGu6x4bxuVf9aTiYkf7ZjVF2lPn97EXOEGFWPZeZbH4 vdRFH9jMtP+rrLt6+3c9j0M8SIJYwBL1+CNpEC/BuHj/Ra/cmnG5ZNhYebm76h5f T9iPW9fFww36FzFka4VPlvA4oB7ebBtquFg3sdQNU/MmTVV4jPFWXxh4oRDDR+8N 1bcPnbB11b5ary99F/mqr7RgQ+YFF0uKRE3SKa7a+6cIuHEZ7Za+zhPaQlzAOZlx fuBmScum8uQTrEF5+Um5zkwC7EXTdH1co/+/V/fpOtxIg4XO4kcugZefVm5ERfVS MA== =dtMN -----END PGP PUBLIC KEY BLOCK----- EOF cat <<EOF > /etc/apt/sources.list.d/saltstack.list deb http://repo.saltstack.com/apt/debian/8/amd64/latest jessie main EOF apt-get update apt-get install -y salt-minion } main() { create_user install_salt_minion systemctl start salt-minion } # It's a really good idea to create main functions and call them at # the very bottom of the file whenever you are streaming a shell script # from a remote location into your shell. What happens if you want to # run the command "rm -rf /home/guy_who_doesnt_work_here_anymore" and # the connection dies right as it gets to the first slash? I'm developing # a buffering feature into curlbomb, to prevent this, but it's not done yet # (although the encryption features do accomplish that as a side effect.) main
curlbomb is useful for these kinds of quick scripts that setup the environment for a more robust configuration management tool. It's also a way to keep your entire server setup automated and documented, without having to resort to system images.
This article only scratched the surface of what curlbomb can do. Learn about it in more detail at the curlbomb github project page.