DevOps Zone is brought to you in partnership with:

A friendly Finnish hacker. I am technology consultant, open source advocator and entrepreneur. My expertise areas cover HTML5, Python, Plone, Javascript, WebGL,UNIX and mobile web. Mikko likes sushi, Angry Birds and dislikes winter. Mikko is a DZone MVB and is not an employee of DZone and has posted 43 posts at DZone. You can read more from them at their website. View Full User Profile

Migrating from BIND DNS servers to Amazon Route 53 by using cli53

11.23.2012
| 7405 views |
  • submit to reddit

Amazon Route 53 offers DNS as a hosted service. They maintain robust DNS service for you with easy to user web interface and API.

Name servers are part of any serious hosting infrastructure. They have been traditionally run on BIND or similar UNIX daemons. The usual hosting tasks including mapping domain names and subdomains (A, CNAME, et. al records) to servers and delivering email (MX and SPF records).

Usually a basic hosting plan is only good for setting A records for few subdomains like www, but on the instant you need something more advanced like routing MX records to Google Apps email you need to start considering a real DNS service. GoDaddy and other big name registrants provide tools for this, but they are usually specific only for top level domains (TLDs) the registrant supports. If you come from a cold country like .fi you cannot centralize your DNS management to big popular domain registrants.

The bad thing with running your own name servers is that when the service goes down the whole your infrastructure becomes virtually inaccessible – web sites, email, anything. Thus you usually run at least two BIND daemons on two physically separated servers just in case one blows up the other can still keep DNS records running. Also DNS, being core part of internet infrastructure, is subject for many kind of attacks and keeping your skills and knowledge up-to-date with BIND may be time consuming.

Like with email servers, hosting name servers is a thing you really don’t want to do. Enter Amazon Route 53. (tha name Route 53 probably comes from the famous U.S. road  Route 67 and DNS port 53?).

Route 53 is ridiculous easy to use. Just go to the site and register one for yourself. However if you have legacy BIND servers running around there exist ways to import data automatically instead of manually reading through zone files and typing them in to Route 53 by hand.

1. Setting up cli53

cli53 is a Python based command line interface for Route 53. It uses buildout for installation (of Python package dependencies).

These instructions have been tested with Ubuntu 8.04 server. Prerequisitements for using this stuff is knowing basic Ubuntu server management from the command line.

Run everything as root.

Install wget and unzip

apt-get install wget unzip

Download ZIP copy of cli53 from Github (we don’t do git clone here – no need to install git on the server):

wget --no-check-certificate https://github.com/barnybug/cli53/zipball/master

Extract

unzip master

Install as user local under the extracted folder (note that the folder varies across GitHub exports)

cd barnybug-cli53-3b468b7
python bootstrap.py
bin/buildout

This will generate the local bin/cli53 command.

Now you need to set up AWS credentials. Route 53, and other AWS service, are operated using access keys which you can get from Security Credentials page in AWS web console. Set the access keys for your shell/SSH session using enviroment variables:

export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Side note: as this point you need to check that your server clock is in correct time your AWS remote operations will fail (300 seconds thresold).. Use date command to see the server clock time and correct it with the same date command using this obscure date command syntax.

Now you can run a test by listing your Amazon Route 53 Hosted Zones (Hosted Zone = zonefile = one per top level domain usually) using list command:

bin/cli53 list

The proper output for one domain which has been created throught-the-web in Route 53 control panel is something like:

HostedZones:
  - CallerReference: 9265CCC3-9C41-98CE-8820-E26C0356C478
    Config:
      Comment: test
    Id: /hostedzone/Z2L3NT0WCS8OWA
    Name: xxx.fi.

2. Importing zone files

Now when cli53 is running and working it is time to rock’n'roll.

On our server configuration BIND zonefiles lived in /srv/dns/zones though this is not the default location for Ubuntu BIND. There is one file per each zone and the filename is the domain name.

[root@foobar][23:02][/srv/dns/zones]# ls -l
total 176
-rw-r--r-- 1 root root  425 Dec 30  2008 abc.com
-rw-r--r-- 1 root root 1713 Dec 30  2008 abcabc.com
-rw-r--r-- 1 root root 1719 Dec 30  2008 abcabcabc.com

We can simple create an one-liner shell script which will loop through all the files and import them to Amazon Route 53. However – there is a trick. You’ll get unknown origin exception if your zone files lack a certain line:

Traceback (most recent call last):
  File "/root/barnybug-cli53-3b468b7/bin/cli53", line 22, in <module>
    cli53.cli53.main()
  File "/root/barnybug-cli53-3b468b7/src/cli53/cli53.py", line 495, in main
    args.func(args)
  File "/root/barnybug-cli53-3b468b7/src/cli53/cli53.py", line 268, in cmd_import
    raise Exception, 'Could not find origin'
Exception: Could not find origin

This is because the zone file doesn’t have $ORIGIN directive. it’s optional with BIND as BIND uses the filename as $ORIGIN if it’s not defined, but the directive is mandatory  for cli53 (cli53 doesn’t yet synthetize $ORIGIN).

You can simple add $ORIGIN to the beginning of each file and copy files to a working directory with the following shell one-liner (it’s always safe to make copies of processed files than try to fix them in place):./transfer/$i ; done

mkdir ../transfer
for i in * ; do echo "\$ORIGIN $i." > ../transfer/$i ; cat $i >> ../transfer/$i ; done

Then do a test run for one import:

~/barnybug-cli53-3b468b7/bin/cli53 create xxx.fi
~/barnybug-cli53-3b468b7/bin/cli53 import xxx.fi --file xxx.fi --replace --wait

Note: deleting zones in Route 53 control panel is painful difficult, so make sure you import zones you only really need.

If everything looks good we are ok to upload everything to Route 53. Again, a shell one-liner does the trick for us:

for i in * ; do ~/barnybug-cli53-3b468b7/bin/cli53 create $i ; ~/barnybug-cli53-3b468b7/bin/cli53 import $i --file $i --wait ; done

This will take abour 15-20 seconds per domain.

Note: if you need to re-import add –replace flag.

3. The ugly part

Each zone has Source of Authority record with authoritative nameserves (NS records). These will be change from your BIND server IP addresses to Amazon ones. Route 53 will re-assign its own name servers for each imported Hosted Zone. However, you cannot know beforehand what name servers the dice has chosen for your Hosted Zone, so automatizing this process is little bit difficult. NS records servers are overriden when the zone file is uploaded to Route 53.

You can see new SOA and NS records when you choose the domain in Route 53 control panel and press Go to Record Sets.

To make things easier later on here is a script which will dump all the name servers of all domains you have in Route 53 and collect are info to domain-info.txt file:

for i in * ; do ~/barnybug-cli53-3b468b7/bin/cli53 info $i >> /tmp/domain-info.txt ; done

4. Testing Route 53 DNS

Browser through imported domain records in Route 53 control panel and see if they look ok.

You can also test the actual DNS functionality. When you use ping command, your web browser or any other “normal” mean which queries DNS records they use the Source of Authority name servers as set by in your domain registrant settings. When in the middle of migration to Route 53, the authorative name servers are still pointing to the old name servers in this point.

dig command can be used to query a specific name server for DNS records.

  • In Route 53 control panel pick any domain and any of its name servers
  • Use dig command to query the records of this domain

Example:

dig -t ANY @ns-1340.awsdns-39.org yourdomainname.com

Output should be something like this:

; <<>> DiG 9.4.2-P1 <<>> -t ANY @ns-1340.awsdns-39.org yourdomain.com
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 9352
;; flags: qr aa rd; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;yourdomain.com.            IN    ANY

;; ANSWER SECTION:
yourdomain.com.        172800    IN    NS    ns-470.awsdns-58.com.
yourdomain.com.        172800    IN    NS    ns-553.awsdns-05.net.
yourdomain.com.        172800    IN    NS    ns-1340.awsdns-39.org.
yourdomain.com.        172800    IN    NS    ns-1706.awsdns-21.co.uk.
yourdomain.com.        900    IN    SOA    ns-1340.awsdns-39.org. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400
yourdomain.com.        0    IN    MX    10 aspmx.l.google.com.
yourdomain.com.        0    IN    MX    20 alt1.aspmx.l.google.com.
yourdomain.com.        0    IN    MX    20 alt2.aspmx.l.google.com.
yourdomain.com.        0    IN    TXT    "v=spf1 a mx a:smtp.something.fi a:auth-smtp.something.fi include:aspmx.googlemail.com ~all"
yourdomain.com.        0    IN    A    84.20.128.49

;; Query time: 32 msec
;; SERVER: 205.251.197.60#53(205.251.197.60)
;; WHEN: Wed Nov 23 00:04:53 2011
;; MSG SIZE  rcvd: 413

(subdomains are not listed)

5. Finalizing it

Go to your domain name registrant and update the domain name servers point to your Route 53 servers.

  • For each domain in domain-info.txt
    • Go to the corresponding registrant from whom you obtained the domain (e.g. domain.ficora.fi)
    • Update the authoritative name servers to be as stated in the file
  • Wait 24-48 hours or whatever time-to-live time you have for your name servers – usually this is longish period
  • See that your services still run
  • Take down your old name servers

6. More info



Published at DZone with permission of Mikko Ohtamaa, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)