Do not copy paste the code from the core text of this post. These are one year old scripts. An updated version is available in my GitHub Repository.
[/UPDATE]
This article describes how to run your private VPN gateway in Amazon’s cloud. Although this article describes a 100% automatic (scripted) method to start and configure your VPN server, it assumes some basic knowledge ofAmazon’s EC2 platform and – obviously – requires you to have an account on EC2.
If you are totally new to EC2, I strongly advise you to follow a Getting Started guide before going through this article.
The VPN server I am using for the purpose of this article is based on IPSec / L2TP security protocols implemented by open source projects OpenSWAN and XL2LTP.
For the impatient, the scripts are available on github, along with basic configuration and setup information. Should you need more details, I encourage you to read the following.
Why a private VPN server ?
Sometime, it is legitimate to create an encrypted tunnel of data to another machine on the internet. Think about situations like
- Being connected on a public network in an hotel, a conference, a restaurant or coffee shop
- Willing to escape your ISP or Service Provider limitations (Belgium DNS Blocking, French Hadopi, …)
- Accessing services not being distributed in your country (Deezer, Spotify etc ..)
- or simply to ensure no one can snoop your network traffic
How to start a customised machine on EC2 ? Some Background.
AWS provides several ways to start customised machines. Either you can create your own virtual machine image (AMI) based on one of the many images available. Either you can start a standard image and run a script at startup time to customise it. Either you can boot from an EBS backed machine image (AMI) and create a snapshot of your root volume.
The first method is more labor intensive (install the software, maintain the image, …) and more expensive (you have to pay for the storage of your customised image) but has the advantage of faster startup times as the image does not need to install and to configure required softwares at boot time.
Running a script at boot time is easier as you do not need to enter into the details of creating and maintaining custom images. It is cheaper as you do no need to store that custom image. But the machine is slower to boot as it requires to download, install and configure required softwares at every boot. This is the method I choose to setup the VPN server.
The latter method (EBS Snapshot of root volume) is described in extenso in the documentation and – based on my own experience – provides the best ratio between labour, price and effectiveness. This is probably the method I would recommend for production workloads.
But … How to start a customisation / installation script just after booting a standard linux distribution or one of the prepared Amazon Machine Image ? This is where cloud-init kicks in.
Cloud-Init is an open source library initiated by Canonical (the maker Ubuntu) to initialise Virtual Machines running in the cloud. It allows, amongst others, to do post-boot configuration like
- setting the correct locale
- setting the hostname
- initialising (or installing) ssh keys
- setup mount points
- etc …
It also allows to pass a user defined script to the instance to perform any additional setup and configuration tasks. This is the technique I am using to download, install, configure and start IPSec and L2TP daemons on the server.
Cloud-Init is included by default in Ubuntu machine images and in Amazon Linux machine images on EC2.
For the purpose of this article, I choose to use the Amazon Linux machine image because it is lightweight and specifically designed to run on EC2.
This is enough background information, let’s start to do real stuffs.
How to start a machine from your command line ?
To start an EC2 instance form your machine command line, you will need the following :
- an Amazon Web Service account and a credit card
- to create a SSH key pair
- to create a VPN security group
The VPN Security Group must allow TCP and UPD port 500 and UPD port 4500 as shown on the screenshot below.
Please refer to the getting started guide to learn how to perform these. Be sure to write down the name of your key pair and the name of your security group as we will need these later.
Once your basic setup of EC2 is done, you will need to install and configure EC2 Command line tools on your machine.
- Download and Install EC2 Command Line Tools
- Configure your environment
To configure your environment, you will need to setup a couple of environment variables, typically in $HOME/.profile
- EC2_HOME environment variable points to command line tools
- EC2_URL environment variables contains AWS endpoint (http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region)
- AWS_ACCESS_KEY environment variable contains your AWS access key
- AWS_SECRET_KEY environment variable contains your AWS secret key
Do not change the name of these environment variables as these are used in the script.
For example, here is my own .profile file (on Mac OS X) :
<span style="font-weight: bold; color: #7a0874;">export</span> <span style="color: #007800;">JAVA_HOME</span>=<span style="font-weight: bold;">`/</span>usr<span style="font-weight: bold;">/</span>libexec<span style="font-weight: bold;">/</span>java_home<span style="font-weight: bold;">`</span> <span style="font-weight: bold; color: #7a0874;">export</span> <span style="color: #007800;">EC2_HOME</span>=<span style="font-weight: bold;">/</span>Users<span style="font-weight: bold;">/</span>sst<span style="font-weight: bold;">/</span>Projects<span style="font-weight: bold;">/</span>aws<span style="font-weight: bold;">/</span>ec2-api-tools-latest <span style="font-weight: bold; color: #7a0874;">export</span> <span style="color: #007800;">AWS_ACCESS_KEY</span>=<span style="font-weight: bold;">&</span>lt;access key<span style="font-weight: bold;">&</span>gt; <span style="font-weight: bold; color: #7a0874;">export</span> <span style="color: #007800;">AWS_SECRET_KEY</span>=<span style="font-weight: bold;">&</span>lt;secret key<span style="font-weight: bold;">&</span>gt; <span style="font-weight: bold; color: #7a0874;">export</span> <span style="color: #007800;">EC2_URL</span>=http:<span style="font-weight: bold;">//</span>ec2.eu-west-1.amazonaws.com |
Once this setup is done, you can start to use the EC2 command line tools as demonstrated in the script below :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
<span style="font-style: italic; color: #666666;">#to be run on my laptop</span>
<span style="font-style: italic; color: #666666;"># create and start an instance</span>
<span style="font-style: italic; color: #666666;">#AMI = AMZN Linux 64 Bits</span>
<span style="font-style: italic; color: #666666;">#AMI_DESCRIPTION="amazon/amzn-ami-pv-2012.09.0.x86_64-ebs"</span>
<span style="color: #007800;">AMI_ID</span>=ami-c37474b7
<span style="color: #007800;">KEY_ID</span>=sst-ec2
<span style="color: #007800;">SEC_ID</span>=VPN
<span style="color: #007800;">BOOTSTRAP_SCRIPT</span>=vpn-ec2-install.sh
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Starting Instance..."</span>
<span style="color: #007800;">INSTANCE_DETAILS</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-run-instances <span style="color: #007800;">$AMI_ID</span> <span style="color: #660033;">-k</span> <span style="color: #007800;">$KEY_ID</span> <span style="color: #660033;">-t</span> t1.micro <span style="color: #660033;">-g</span> <span style="color: #007800;">$SEC_ID</span> <span style="color: #660033;">-f</span> <span style="color: #007800;">$BOOTSTRAP_SCRIPT</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE<span style="font-weight: bold;">`</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #007800;">$INSTANCE_DETAILS</span>
<span style="color: #007800;">AVAILABILITY_ZONE</span>=<span style="font-weight: bold;">`</span><span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #007800;">$INSTANCE_DETAILS</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $9}'</span><span style="font-weight: bold;">`</span>
<span style="color: #007800;">INSTANCE_ID</span>=<span style="font-weight: bold;">`</span><span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #007800;">$INSTANCE_DETAILS</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $2}'</span><span style="font-weight: bold;">`</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #007800;">$INSTANCE_ID</span> <span style="font-weight: bold;">&</span>gt; <span style="color: #007800;">$HOME</span><span style="font-weight: bold;">/</span>vpn-ec2.id
<span style="font-style: italic; color: #666666;"># wait for instance to be started</span>
<span style="color: #007800;">DNS_NAME</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-describe-instances <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"image-id=<span style="color: #007800;">$AMI_ID</span>"</span> <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"instance-state-name=running"</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $4}'</span><span style="font-weight: bold;">`</span>
<span style="font-weight: bold;">while</span> <span style="font-weight: bold; color: #7a0874;">[</span> <span style="color: #660033;">-z</span> <span style="color: #ff0000;">"<span style="color: #007800;">$DNS_NAME</span>"</span> <span style="font-weight: bold; color: #7a0874;">]</span>
<span style="font-weight: bold;">do</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Waiting for instance to start...."</span>
<span style="font-weight: bold; color: #c20cb9;">sleep</span> 5
<span style="color: #007800;">DNS_NAME</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-describe-instances <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"image-id=<span style="color: #007800;">$AMI_ID</span>"</span> <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"instance-state-name=running"</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $4}'</span><span style="font-weight: bold;">`</span>
<span style="font-weight: bold;">done</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Instance started"</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Instance ID = "</span> <span style="color: #007800;">$INSTANCE_ID</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"DNS = "</span> <span style="color: #007800;">$DNS_NAME</span> <span style="color: #ff0000;">" in availability zone "</span> <span style="color: #007800;">$AVAILABILITY_ZONE</span>
|
You will need to slightly customise this script to make it run :
- Line 5 : check what is the AMI ID in your geography
- Line 6 : replace “sst-ec2″ with the name of your ssh key pair
- Line 7 : replace “VPN” with the name you choose for your Security Group
How is it working ?
10 11 12 |
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Starting Instance..."</span> <span style="color: #007800;">INSTANCE_DETAILS</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-run-instances <span style="color: #007800;">$AMI_ID</span> <span style="color: #660033;">-k</span> <span style="color: #007800;">$KEY_ID</span> <span style="color: #660033;">-t</span> t1.micro <span style="color: #660033;">-g</span> <span style="color: #007800;">$SEC_ID</span> <span style="color: #660033;">-f</span> <span style="color: #007800;">$BOOTSTRAP_SCRIPT</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE<span style="font-weight: bold;">`</span> <span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #007800;">$INSTANCE_DETAILS</span> |
This script starts an EC2 instance (line 11) of the given type with the specified SSH key pair and Security Group. It uses the “-f” option to pass a cloud-init user data script that will download install and configure IPSec and L2TP once the machine is booted.
18 19 20 21 22 23 24 25 26 |
<span style="font-style: italic; color: #666666;"># wait for instance to be started</span>
<span style="color: #007800;">DNS_NAME</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-describe-instances <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"image-id=<span style="color: #007800;">$AMI_ID</span>"</span> <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"instance-state-name=running"</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $4}'</span><span style="font-weight: bold;">`</span>
<span style="font-weight: bold;">while</span> <span style="font-weight: bold; color: #7a0874;">[</span> <span style="color: #660033;">-z</span> <span style="color: #ff0000;">"<span style="color: #007800;">$DNS_NAME</span>"</span> <span style="font-weight: bold; color: #7a0874;">]</span>
<span style="font-weight: bold;">do</span>
<span style="font-weight: bold; color: #7a0874;">echo</span> <span style="color: #ff0000;">"Waiting for instance to start...."</span>
<span style="font-weight: bold; color: #c20cb9;">sleep</span> 5
<span style="color: #007800;">DNS_NAME</span>=<span style="font-weight: bold;">`</span><span style="color: #007800;">$EC2_HOME</span><span style="font-weight: bold;">/</span>bin<span style="font-weight: bold;">/</span>ec2-describe-instances <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"image-id=<span style="color: #007800;">$AMI_ID</span>"</span> <span style="color: #660033;">--filter</span> <span style="color: #ff0000;">"instance-state-name=running"</span> <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">grep</span> INSTANCE <span style="font-weight: bold;">|</span> <span style="font-weight: bold; color: #c20cb9;">awk</span> <span style="color: #ff0000;">'{print $4}'</span><span style="font-weight: bold;">`</span>
<span style="font-weight: bold;">done</span>
|
The script then waits for the machine to be ready (lines 19-26) and, once available, the script reports the machine public DNS name (to be used to configure your VPN client software) (line 30 – 31)
How to Install and to Configure VPN into your new machine ?
Now that the machine is started, it receives the customisation script through the -f option. Cloud-Init will execute this script to finalise the setup of the machine.
Here is the script allowing to install and configure IPSec and L2TP automatically. Some details are given after the code.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
<span style="font-style: italic; color: #666666;">#!/bin/sh</span> <span style="font-style: italic; color: #666666;"># Please define your own values for those variables</span> <span style="color: #007800;">IPSEC_PSK</span>=SharedSecret <span style="color: #007800;">VPN_USER</span>=username <span style="color: #007800;">VPN_PASSWORD</span>=password <span style="font-style: italic; color: #666666;"># Those two variables will be found automatically</span> <span style="color: #007800;">PRIVATE_IP</span>=<span style="font-weight: bold;">`</span><span style="font-weight: bold; color: #c20cb9;">wget</span> <span style="color: #660033;">-q</span> <span style="color: #660033;">-O</span> - <span style="color: #ff0000;">'http://instance-data/latest/meta-data/local-ipv4'</span><span style="font-weight: bold;">`</span> <span style="color: #007800;">PUBLIC_IP</span>=<span style="font-weight: bold;">`</span><span style="font-weight: bold; color: #c20cb9;">wget</span> <span style="color: #660033;">-q</span> <span style="color: #660033;">-O</span> - <span style="color: #ff0000;">'http://instance-data/latest/meta-data/public-ipv4'</span><span style="font-weight: bold;">`</span> <span style="font-weight: bold; color: #c20cb9;">yum install</span> <span style="color: #660033;">-y</span> <span style="color: #660033;">--enablerepo</span>=epel openswan xl2tpd <span style="font-weight: bold; color: #c20cb9;">cat</span> <span style="font-weight: bold;">></span> <span style="font-weight: bold;">/</span>etc<span style="font-weight: bold;">/</span>ipsec.conf <span style="font-style: italic; color: #cc0000;"><<EOF version 2.0 config setup dumpdir=/var/run/pluto/ nat_traversal=yes virtual_private=%v4:10.0.0.0/8,%v4:192.168.0.0/16,%v4:172.16.0.0/12,%v4:25.0.0.0/8,%v6:fd00::/8,%v6:fe80::/10 oe=off protostack=netkey nhelpers=0 interfaces=%defaultroute conn vpnpsk auto=add left=$PRIVATE_IP leftid=$PUBLIC_IP leftsubnet=$PRIVATE_IP/32 leftnexthop=%defaultroute leftprotoport=17/1701 rightprotoport=17/%any right=%any rightsubnetwithin=0.0.0.0/0 forceencaps=yes authby=secret pfs=no type=transport auth=esp ike=3des-sha1 phase2alg=3des-sha1 dpddelay=30 dpdtimeout=120 dpdaction=clear EOF cat > /etc/ipsec.secrets <<EOF $PUBLIC_IP %any : PSK "$IPSEC_PSK" EOF cat > /etc/xl2tpd/xl2tpd.conf <<EOF [global] port = 1701 ;debug avp = yes ;debug network = yes ;debug state = yes ;debug tunnel = yes [lns default] ip range = 192.168.42.10-192.168.42.250 local ip = 192.168.42.1 require chap = yes refuse pap = yes require authentication = yes name = l2tpd ;ppp debug = yes pppoptfile = /etc/ppp/options.xl2tpd length bit = yes EOF cat > /etc/ppp/options.xl2tpd <<EOF ipcp-accept-local ipcp-accept-remote ms-dns 8.8.8.8 ms-dns 8.8.4.4 noccp auth crtscts idle 1800 mtu 1280 mru 1280 lock connect-delay 5000 EOF cat > /etc/ppp/chap-secrets <<EOF # Secrets for authentication using CHAP # client server secret IP addresses $VPN_USER l2tpd $VPN_PASSWORD * EOF iptables -t nat -A POSTROUTING -s 192.168.42.0/24 -o eth0 -j MASQUERADE echo 1 > /proc/sys/net/ipv4/ip_forward iptables-save > /etc/iptables.rules cat > /etc/network/if-pre-up.d/iptablesload <<EOF #!/bin/sh iptables-restore < /etc/iptables.rules echo 1 > /proc/sys/net/ipv4/ip_forward exit 0 EOF</span> service ipsec start service xl2tpd start chkconfig ipsec on chkconfig xl2tpd on |
As promised, here are some details
- Lines 4-6 defines your security credentials for the VPN. They must be changed before executing this script.
- Line 12 uses yum to install IPSec & L2TP implementation (OpenSWAN and xl2tpd) from the Amazon’s provided EPEL repository
- Lines 14-93 creates IPSec and L2TP configuration files, reusing the credentials you provided at the head of the script.
- Lines 95-96 setup proper network NATing
- Lines 98-105 ensure the network NATing settings will be restored in case the network interface is shutdown and up again.
- Finally, lines 107-110 start required services and ensure they will be restarted in case of reboot.
Congrats for those of you still reading. You now should have a valid VPN server running in the cloud. If everything went well, you should now be able to configure your VPN client.
How to connect from Mac OS X ?
Once the server is up and running, you simply add a VPN interface in your Network Preferences
Then, use the public DNS hostname as server address and your username, as shown below
Finally, click on “Authentication Settings” to enter the shared secret and your password.
Then click on Apply, then Connect
If everything is OK, you should connect to your new VPN Server.
How to be sure you’re connecting through the VPN ?
The easiest way to check that indeed all your network traffic is routed through the VPN tunnel is to connect to one of the many IP Address Geolocalisation web sites.
The web site I found on Google reported an Amazon IP address from Ireland, which is the geographical region I choose to deploy my VPN server.
A note for Windows users
Microsoft published an extensive technical note describing the details of setting up a IPSec client on Windows.
Also, Windows does not support IPsec NAT-T by default, which is used whenever the server is behind a NAT (as in this case). You have to add a registry key to enable this – see http://support.microsoft.com/kb/926179/en-us (still applies to Windows 8)
How to hook up a DNS alias to avoid to change client configuration ?
Every time you will startup a new VPN server, you will need to enter its public DNS name to your VPN client configuration. It is possible to avoid this if you have a domain name of your own, just by creating a DNS CNAME record pointing to the public DNS address of your server, such as
vpn.mydomain.com CNAME ec2-176-34-71-204.eu-west-1.compute.amazonaws.com.
If you are using Amazon’s Route 53 DNS service, this step can be entirely automated using scripts. More about this in another article.
Congrats if you manage to read this article to the end. Once again, the script source code is available on GitHub.
Enjoy !