Home Page
Archive > Posts > Tags > Networking
Archive > Posts > Tags > Networking
Search:
RABiD BUNNY FEVER
K.T.K

Warning: you do not have javascript enabled. This WILL cause layout glitches.

Optionally encrypted TCP class for Google's Go
Yet another new language to play with

I wanted to play around with Google's go language a little so I ended up decided on making a simple class that helps create a TCP connection between a server and client that is encrypted via TLS, or not, depending upon a flag. Having the ability to not encrypt a connection is useful for debugging and testing purposes, especially if other people are needing to create clients to connect to your server.


The example server.go file listens on port 16001 and for every set of data it receives, it sends the reversed string back to the client. (Note there are limitations to the string lengths in the examples due to buffer and packet payload length restrictions).


The example client.go file connects to the server (given via the 1st command line parameter), optionally encrypts the connection (depending upon the 2nd command line parameter), and sends the rest of the parameters to the server as strings.


The encryptedtcp.go class has the following exported functions:
  • StartServer: Goes into a connection accepting loop. Whenever a connection is accepted, it checks the data stream for either the "ENCR" or "PTXT" flags, which control whether a TLS connection is created or not. The passed "clientHandler" function is called once the connection is completed.
  • StartClient: Connects to a server, passes either the "ENCR" or "PTXT" flag as noted above, and returns the finished connection.

Connections are returned as "ReadWriteClose" interfaces. Creating the pem and key certificate files is done via openssl. You can just google for examples.


server.go:
package main
import ( "./encryptedtcp"; "fmt"; "log" )

func main() {
	if err := encryptedtcp.StartServer("server.pem", "server.key", "0.0.0.0:16001", handleClient); err != nil {
		log.Printf("%q\n", err) }
}

func handleClient(conn encryptedtcp.ReadWriteClose) {
	buf := make([]byte, 512)
	for {
		//Read data
		n, err := conn.Read(buf)
		if err != nil {
			log.Printf("Error Reading: %q\n", err); break }
		fmt.Printf("Received: %q\n", string(buf[:n]))

		//Reverse data
		for i, m := 0, n/2; i<m; i++ { //Iterate over half the list
			buf[i], buf[n-i-1] = buf[n-i-1], buf[i] } //Swap first and half of list 1 char at a time

		//Echo back reversed data
		n, err = conn.Write(buf[:n])
		if err != nil {
			log.Printf("Error Writing: %q\n", err); break }
		fmt.Printf("Sent: %q\n", string(buf[:n]))
	}
}

client.go:
package main
import ( "./encryptedtcp"; "fmt"; "log"; "os" )

func main() {
	//Confirm parameters, and if invalid, print the help
	if len(os.Args) < 4 || (os.Args[2] != "y" && os.Args[2] != "n") {
		log.Print("First Parameter: ip address to connect to\nSecond Parameter: y = encrypted, n = unencrypted\nAdditional Parameters (at least 1 required): messages to send\n"); return }

	//Initialize the connection
	conn, err := encryptedtcp.StartClient("client.pem", "client.key", os.Args[1]+":16001", os.Args[2]=="y" )
	if err != nil {
		log.Printf("%q\n", err); return }
	defer conn.Close()

	//Process all parameters past the first
	buf := make([]byte, 512)
	for _, msg := range os.Args[3:] {
		//Send the parameter
		if(len(msg)==0) {
			continue }
		n, err := conn.Write([]byte(msg))
		if err != nil {
			log.Printf("Error Writing: %q\n", err); break }
		fmt.Printf("Sent: %q\n", msg[:n])

		//Receive the reply
		n, err = conn.Read(buf)
		if err != nil {
			log.Printf("Error Reading: %q\n", err); break }
		fmt.Printf("Received: %q\n", string(buf[:n]))
	}
}

encryptedtcp/encryptedtcp.go:
//A simple TCP client/server that can be encrypted (via tls) or not, depending on a flag passed from the client

package encryptedtcp

import ( "crypto/rand"; "crypto/tls"; "net"; "log" )

//Goes into a loop to accept clients. Returns a string on error
func StartServer(certFile, keyFile, listenOn string, clientHandler func(ReadWriteClose)) (error) {
	//Configure the certificate information
	cert, err := tls.LoadX509KeyPair(certFile, keyFile)
	if err != nil {
		return MyError{"Cannot Load Keys", err} }
	conf := tls.Config{Certificates:[]tls.Certificate{cert}, ClientAuth:tls.RequireAnyClientCert, Rand:rand.Reader}

	//Create the listener
	listener, err := net.Listen("tcp", listenOn)
	if err != nil {
		return MyError{"Cannot Listen", err} }
	defer listener.Close()

	//Listen and dispatch clients
	for {
		conn, err := listener.Accept()
		if err != nil {
			return MyError{"Cannot Accept Client", err} }
		go startHandleClient(conn, &conf, clientHandler)
	}

	//No error to return - This state is unreachable in the current library
	return nil
}

//Return the io stream for the connected client
func startHandleClient(conn net.Conn, conf* tls.Config, clientHandler func(ReadWriteClose)) {
	defer conn.Close()

	//Confirm encrypted connection flag (ENCR = yes, PTXT = no)
	isEncrypted := make([]byte, 4)
	amountRead, err := conn.Read(isEncrypted)
	if err != nil {
		log.Printf("Cannot get Encrypted Flag: %q\n", err); return }
	if amountRead != 4 {
		log.Printf("Cannot get Encrypted Flag: %q\n", "Invalid flag length"); return }
	if string(isEncrypted) == "PTXT" { //If plain text, just pass the net.Conn object to the client handler
		clientHandler(conn); return
	} else if string(isEncrypted) != "ENCR" { //If not a valid flag value
		log.Printf("Invalid flag value: %q\n", isEncrypted); return }

	//Initialize the tls session
	tlsconn := tls.Server(conn, conf)
	defer tlsconn.Close()
	if err := tlsconn.Handshake(); err != nil {
		log.Printf("TLS handshake failed: %q\n", err); return }

	//Pass the tls.Conn object to the client handler
	clientHandler(tlsconn)
}

//Start a client connection
func StartClient(certFile, keyFile, connectTo string, isEncrypted bool) (ReadWriteClose, error) {
	//Configure the certificate information
	cert, err := tls.LoadX509KeyPair(certFile, keyFile)
	if err != nil {
		return nil, MyError{"Cannot Load Keys", err} }
	conf := tls.Config{Certificates:[]tls.Certificate{cert}, InsecureSkipVerify:true}

	//Connect to the server
	tcpconn, err := net.Dial("tcp", connectTo)
	if err != nil {
		return nil, MyError{"Cannot Connect", err} }

	//Handle unencrypted connections
	if !isEncrypted {
		tcpconn.Write([]byte("PTXT"))
		return tcpconn, nil //Return the base tcp connection
	}

	//Initialize encrypted connections
	tcpconn.Write([]byte("ENCR"))
	conn := tls.Client(tcpconn, &conf)
	conn.Handshake()

	//Confirm handshake was successful
	state := conn.ConnectionState()
	if !state.HandshakeComplete || !state.NegotiatedProtocolIsMutual {
		conn.Close()
		if !state.HandshakeComplete {
			return nil, MyError{"Handshake did not complete successfully", nil}
		} else {
			return nil, MyError{"Negotiated Protocol Is Not Mutual", nil} }
	}

	//Return the tls connection
	return conn, nil
}

//Error handling
type MyError struct {
	Context string
	TheError error
}
func (e MyError) Error() string {
	return e.Context+": "+e.TheError.Error(); }

//Interface for socket objects (read, write, close)
type ReadWriteClose interface {
	Read(b []byte) (n int, err error)
	Write(b []byte) (n int, err error)
	Close() error
}
The Pitfalls and Use of VPN and TOR
AKA Privacy Online

A friend of mine recently asked me about the TOR network because of a PC world article he had read. First, I’d like to state that the article actually has a lot of good general information, covering a lot of general security problems with solutions to them that have been time proven and useful to millions of people (VPNs, privacy/incognito mode in browsers, cookie management, bugmenot, etc). However, I think the article does not cover the realities of TOR and VPNs at all, so I figured I’d write up an article on these topics that I could share with my inquisitive friend and anyone else who is interested.


I used TOR back in the early 2000s and it’s not cracked up to what the article would have you think. Basically, it securely routes your connection through a few other people’s internet connections (we’ll say 3 for examples sake). The computers/nodes between you and the “exit node” in the route can’t read what your traffic data says because it’s all encrypted, but the final person/computer (the “exit node”) literally sees, in clear text, 100% of your data as if you were sending/receiving it out of your own machine without the TOR network. So if you are doing anything that isn’t natively encrypted (instant message chatting without OTR, going to a site via http instead of https) the exit node can snoop on everything you do. They can even see the domain (not the entire URL) of WHERE you are going with https1. If I recall, you can’t really control the exit node as, I think, it semi-randomly picks it from any person in the world running a TOR router node.


So all TOR really does for you is make servers that you connect to not know from where you are coming. So one day it may think you are coming from Michigan, and another day, from Singapore. And honestly, for most people that isn’t even really all that important. Do you really care if servers you go to on the internet know you are coming in from your home town? (They generally can’t pinpoint further than that without getting a warrant and asking the ISP). All that's really done with this data is correlation. Seeing that someone from this IP address that went to this one website also went to this other website.


And even worse, TOR is known for being ungodly slow. Back when I was using it I was LUCKY to get 15KB/s throughput on my connections, and I doubt it has changed much (though you could get lucky too on your “randomly” chosen connection nodes). This means to download a normal webpage (~1.5MB for arguments sake) it would take ~2 minutes to download the page instead of 1-2 seconds for normal broadband users.


The more important thing (than anonymity) for online security is making sure everything you do is encrypted end point to end point (privacy). That means using securely encrypted (usually SSL) connections (https is SSL on top of http). That makes it so no one can snoop on conversations between your computer and the server you are communicating with. Location anonymity isn’t really that important unless you have something to hide that you think someone may try to find you for, though taking appropriate precautions (next few paragraphs) could never hurt. TOR is actually probably more hurtful in the long run since the exit node is an untrusted user who can spy on your unencrypted traffic.


Now, if you really wanted an appropriate solution for privacy (not anonymity), you only ever let your unencrypted traffic exit out of trusted networks. This generally means your house (and maybe your office), though even from those places their ISPs could easily “spy” on your unencrypted traffic. And technically, any router in between you and the server you are connected to can spy on your unencrypted traffic, though there is too much traffic going on for anyone in between ISPs to really even want to try this sort of thing. So it’s not a bad idea to set up a VPN server at a secure locations for yourself so you can connect in and route your traffic through the secure location when you are anywhere on the planet. For this I would recommend OpenVPN, and make sure you configure your client to route all traffic through the VPN tunnel. This approach could severely reduce your connection speed as most broadband connections have a much lower upload than download (meaning when your VPN server sends data back to you, it’s most likely slower than you would normally get it).

However, the speed issue can be solved by setting up your VPN server at a collocation (or on a cloud like Amazon’s), as these collocation ISPs route through so much traffic it would be unfeasible for them to snoop, nor often would they have as much inclination to do so. This wouldn’t give great anonymity since only a handful of people would most likely be using these VPNs, and they will generally exit from the same IP address, but it gives a great amount of privacy when on untrusted (or any) internet connection, and there are no noticeable speed decreases if at a good collocation.


The best solution is to use a paid-for VPN service. However, you would have to of course trust this service to not be spying on your unencrypted traffic, which they generally wouldn’t do. These services are good because they (should be) fast, they are secure exit points, and best of all they can be anonymous to a large degree. Since so many people are coming from the same exit points, and your exit point’s IP could change in between each connection with these VPNs, there’s no easy way to know who the traffic is coming from on a monitoring perspective outside of the VPN provider.


However, there are also downsides to using these VPN services since many providers depend and filter based on location data. For example:

  • If you are coming from outside of the country many services inside the USA may block you
  • Providers needing your location to provide a service for you would have the wrong location. For example, Google Maps wouldn’t know what area to search around when you asked for “restaurants”. You would have to specify “restaurants around my address
  • Some banks and services check to make sure you are always coming in from the same IP addresses. If you aren’t, it makes you go through additional, often convoluted, security checks.


Some networks you may connect to (hotels for example) may also block VPNs, which can be a major pain. However, I can usually get through using dynamic SSH tunnels (“ssh -D” for a socks proxy) at the very least.


If I were to recommend a paid-for VPN service, it would be PirateBay’s ipredator. This service was set up to help the people of Sweden get around some bad laws passed regarding user privacy. I’m sure they have enough users so you would become one of the crowd, and The Pirate Bay has proven themselves to be trustworthy advocates of internet freedom.


1Modern browsers include the domain you are visiting in the https connection packet in plain text, via the Server Name Indication TLS extension. This means if someone is snooping on your packets, they will see the domain you are visiting via https.

OpenVPN Authentication and Gateway Configuration
Securing oneself is a never ending battle

For a number of years now when on insecure network connections I have been routing my computer to the Internet through secure tunnels and VPNs, but I’ve been interested in trying out different types of VPN software lately so I can more easily help secure friends who ask of it. This would mainly include ease of installation and enabling, which partly requires no extra software for them to install.

Unfortunately, Windows 7 and Android (and probably most other software) only support PPTP and L2TP/IPSEC out of the box. While these protocols are good for what they do, everything I have read says OpenVPN is superior to them as a protocol. I was very frustrated to find out how little support OpenVPN actually has today as a standard in the industry, which is to say, you have to use third party clients and it is rarely, if ever, included by default in OSes. The OpenVPN client and server aren’t exactly the easiest to set up either for novices.


So on to the real point of this post. The sample client and server configurations for OpenVPN were set up just how I needed them except they did not include two important options for me: User authentication and full client Internet forwarding/tunneling/gateway routing. Here is how to enable both.


Routing all client traffic (including web-traffic) through the VPN:
  • Add the following options to the server configuration file:
    • push "redirect-gateway def1" #Tells the client to use the server as its default gateway
    • push "dhcp-option DNS 10.8.0.1" #Tells the client to use the server as its DNS Server (DNS Server's IP address dependent on configuration)
  • Run the following commands in bash:
    • iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE #This command assumes that the VPN subnet is 10.8.0.0/24 (taken from the server directive in the OpenVPN server configuration) and that the local ethernet interface is eth0.
    • echo '1' > /proc/sys/net/ipv4/ip_forward #Enable IP Forwarding (This one is not mentioned in the OpenVPN howto)

Adding user authentication (Using alternative authentication methods)

To set up username/password authentication on the server, an authorization script is needed that receives the username/password and returns whether the login information was successful (0) or failed (1). The steps to set up this process are as follows:

  • Add the following options to the server configuration file:
    • auth-user-pass-verify verify.php via-env #The third argument (method) specifies whether to send the username and password through either a temporary file (via-file) or environment variables (via-env)
    • script-security 3 system #Allows OpenVPN to run user scripts and executables and send password authentication information through environment variables. While "system" is deprecated, I had to use it or external commands like ifconfig and route were failing with "failed: could not execute external program"
  • Add the following options to the client configuration file:
    • auth-user-pass #Request user credentials to log in
  • The final step is to create the verify.php (see auth-user-pass-verify configuration above) script which returns whether it was successful, and also outputs its success to stdout, which is added to the OpenVPN log file.
    #!/usr/bin/php -q
    <?
    //Configuration
    $ValidUserFile='users.txt'; //This file must be in htpasswd SHA1 format (htpasswd -s)
    $Method='via-env'; //via-file or via-env (see auth-user-pass-verify configuration above for more information)
    
    //Get the login info
    if($Method=='via-file') //via-file method
    {
    	$LoginInfoFile=trim(file_get_contents('php://stdin')); //Get the file that contains the passed login info from stdin
    	$LoginInfo=file_get_contents($LoginInfoFile); //Get the passed login info
    	file_put_contents($LoginInfoFile, str_repeat('x', strlen($LoginInfo))); //Shred the login info file
    	$LoginInfo=explode("\n", $LoginInfo); //Split into [Username, Password]
    	$UserName=$LoginInfo[0];
    	$Password=$LoginInfo[1];
    }
    else //via-env method
    {
    	$UserName=$_ENV['username'];
    	$Password=$_ENV['password'];
    }
    
    //Test the login info against the valid user file
    $UserLine="$UserName:{SHA}".base64_encode(sha1($Password, TRUE)); //Compile what the user line should look like
    foreach(file($ValidUserFile, FILE_IGNORE_NEW_LINES) as $Line) //Attempt to match against each line in the file
    	if($UserLine==$Line) //If credentials match, return success
    	{
    		print "Logged in: $UserName\n";
    		exit(0);
    	}
    
    //Return failure
    print "NOT Logged in: $UserName\n";
    exit(1);
    ?>
    		
Auto initializing SSH Remote Tunnel
SSH is so incredibly useful

I often find SSH tunnels absolutely indispensable in my line of work for multiple reasons including secure proxies (tunneling) over insecure connections and connecting computers and programs together over difficult network setups involving NATs and firewalls.

One such example of this I ran into recently is that I have a server machine (hereby called “the client”) that I wanted to make sure I have access to no matter where it is. For this I created an auto initializing SSH remote port tunnel to a server with a static IP Address (hereby called “the proxy server”) which attempts to keep itself open when there is problems.

The first step of this was to create the following bash script on the client that utilizes the OpenSSH’s client to connect to an OpenSSH server on the proxy server for tunneling:

#!/bin/bash
for ((;;)) #Infinite loop to keep the tunnel open
do
	ssh USERNAME@PROXYSERVER -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG #Create the SSH tunnel
	echo "Restarting: " `date` >> TUNNEL_LOG #Write to the log file "TUNNEL_LOG" whenever a restart is attempted
	sleep 1 #Wait 1 second in between connection attempts
done
The parts of the command that create the SSH tunnel are as follows:
Part of Command Description
ssh The OpenSSH client application
USERNAME@PROXYSERVER The proxy server and username on said server to connect to
-o ExitOnForwardFailure=yes Automatically terminate the SSH session if the remote port forward fails
-o TCPKeepAlive=yes
-o ServerAliveCountMax=2
-o ServerAliveInterval=10
Make sure the SSH connection is working, and if not, reinitialize it. The connection fails if server keepalive packets that are sent every 10 seconds are not received twice in a row, or if TCP protocol keepalive fails
-N “Do not execute a remote command. This is useful for just forwarding ports” (This means no interactive shell is run)
-R PROXYPORT:localhost:22 Establish a port of PROXYPORT on the proxy server that sends all data to port 22 (ssh) on the client (localhost)
&>> TUNNEL_LOG Send all output from both stdin and stderr to the log file “TUNNEL_LOG”

For this to work, you will need to set up public key authentication between the client and utilized user on the proxy server. To do this, you have to run “ssh-keygen” on the client. When it has finished, you must copy the contents of your public key file (most likely at “~/.ssh/id_rsa.pub”) to the “~/.ssh/authorized_keys” file on the user account you are logging into on the proxy server. You will also have to log into this account at least once from the client so the proxy server’s information is in the client’s “known_hosts” file.

For security reasons you may want the user on the proxy server to have a nologin shell set since it is only being used for tunneling, and the above mentioned public key will allow access without a password.

For network access reasons, it might also be worth it to have the proxy port and the ssh port on the proxy server set to commonly accessible ports on all network setups (that a firewall wouldn’t block). Or you may NOT want to have it on common ports for other security reasons :-).

If you want the proxy port on the proxy server accessible from other computers (not only the proxy server), you will have to turn on “GatewayPorts” (set to “yes”) in the proxy server’s sshd config, most likely located at “/etc/ssh/sshd_config”.


The next step is to create a daemon that calls this script. The normal method for this in Linux would be to use inittab. This can be a bit cumbersome though with Linux distributions that use upstart, like Ubuntu, so I just cheated in it and created the following script to initialize a daemon through rc.d:

#!/bin/bash
echo -e '#!/bin/bash\nfor ((;;))\ndo\n  ssh USERNAME@PROXYSERVER -o TCPKeepAlive=yes -o ExitOnForwardFailure=yes -o ServerAliveCountMax=2 -o ServerAliveInterval=10 -N -R PROXYPORT:localhost:22 &>> TUNNEL_LOG\n  echo "Restarting: " `date` >> TUNNEL_LOG\n  sleep 1\ndone' > TUNNEL_SCRIPT_PATH #This creates the above script
echo -e '#!/bin/bash\ncd TUNNEL_SCRIPT_DIRECTORY\n./TUNNEL_SCRIPT_EXECUTABLE &' > /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #This creates the init.d daemon script. I have the script set the working path before running the executable so the log file stays in the same directory
chmod u+x TUNNEL_SCRIPT_PATH /etc/init.d/TUNNEL_SCRIPT_SERVICE_NAME #Set all scripts as executable
update-rc.d TUNNEL_SCRIPT_SERVICE_NAME defaults #Set the run levels at which the script runs
Internet Connections Abroad
America: Definitely not the most wired or wireless country

Wow, I’m sitting here at 11:45 at night (PST 1/30/2009) with no Internet [this post has been delayed going live until an Internet connection was established]. The hotel we are staying at (The Anaheim Park Hotel at Harbor and 91 in Fullerton) had advertised when reservations were made much earlier that they had free Internet. Alas, they do not. I called the front desk and asked about this, and they told me their free Internet was broken and that they have been trying to fix it. I told them I was a networking expert and might be able to help, to which they replied something to the extent of “we already have networking experts who have been working on it for a few days, but thanks”. This was an obvious lie because it would not take any “networking expert” more than a day to fix this kind of problem, but I did not feel like pushing the issue.

The pay for wireless-Internet service for the hotel is done through Wayport, and it has been very finicky. When I tried to get on earlier tonight the process was breaking after my credit card was submitted for processing. I submitted it 5 or 6 times over a period of time and it broke at the same place each time with an error message something to the extent of “there was an error processing your request. Please try again”. Later on, I couldn’t even get past the main Wayport page to any other section. All pages were returned as zero byte files with no header, which was confirmed through packet sniffing.

0030                    48 54  54 50 2f 31 2e 31 20 33         HT TP/1.1 3
0040  30 34 20 4e 6f 74 20 4d  6f 64 69 66 69 65 64 0d   04 Not M odified.
0050  0a 45 78 70 69 72 65 73  3a 20 53 61 74 2c 20 33   .Expires : Sat, 3
0060  31 20 4a 61 6e 20 32 30  30 39 20 30 37 3a 35 33   1 Jan 20 09 07:53
0070  3a 33 37 20 47 4d 54 0d  0a 4c 61 73 74 2d 4d 6f   :37 GMT. .Last-Mo
0080  64 69 66 69 65 64 3a 20  57 65 64 2c 20 31 31 20   dified:  Wed, 11 
0090  41 70 72 20 32 30 30 37  20 31 34 3a 33 32 3a 34   Apr 2007  14:32:4
00a0  34 20 47 4d 54 0d 0a 4d  49 4d 45 2d 56 65 72 73   4 GMT..M IME-Vers
00b0  69 6f 6e 3a 20 31 2e 30  0d 0a 44 61 74 65 3a 20   ion: 1.0 ..Date: 
00c0  53 61 74 2c 20 33 31 20  4a 61 6e 20 32 30 30 39   Sat, 31  Jan 2009
00d0  20 30 37 3a 34 38 3a 33  37 20 47 4d 54 0d 0a 53    07:48:3 7 GMT..S
00e0  65 72 76 65 72 3a 20 41  4f 4c 73 65 72 76 65 72   erver: A OLserver
00f0  2f 34 2e 35 2e 30 0d 0a  43 6f 6e 74 65 6e 74 2d   /4.5.0.. Content-
0100  4c 65 6e 67 74 68 3a 20  30 0d 0a 43 6f 6e 6e 65   Length:  0..Conne
0110  63 74 69 6f 6e 3a 20 6b  65 65 70 2d 61 6c 69 76   ction: k eep-aliv
0120  65 0d 0a 0d 0a                                     e....            

I blame the fact that the server is “AOLServer” :-).


So, anyways, I called the front desk who had a “technician” come up and he told me to call Wayport. He also told me they did not provide free Internet anymore, and hadn’t for a while, directly contradicting the front desk representative....

So I called the provided Wayport tech support number... which gave me a beeping error signal. I think it was an invalid phone number beeping, but it could have been a busy signal. I tried multiple times, and got the same error response each time, so I couldn’t get through to technical support that way. I tried the general Wayport phone number, and went to the tech support section that way, which asked me to leave a voice message. How crappy and unprofessional :-\. I am giving up for the night in hopes that I can get online in the morning.


I had contemplated getting a Verizon Aircard (wireless Internet for computers over wireless phone waves) for this 10 day trip, but decided I probably wouldn’t need it as most places would provide wireless. How wrong I was. I wish I had now, as it would have been especially helpful at airports and many other times. I have had previous experience with using Verizon Aircards over many months and was very very pleased with the service. Even an iPhone might have been nice... if only it wasn’t only AT&T >:-(.

Diagnosing DNS Problems
Digging until you find the root

Yesterday I wrote a bit about the DNS system being rather fussy, so I thought today I’d go a bit more into how DNS works, and some good tools for problem solving in this area.


First, some technical background on the subject is required.
  • A network is simply a group of computers hooked together to communicate with each other. In the old days, all networking was done through physical wires (called the medium), but nowadays much of it is done through wireless connections. Wired networking is still required for the fastest communications, and is especially important for major backbones (the super highly utilized lines that connect networks together across the world).
  • A LAN is a local network of all computers connected together in one physical location, whether it be a single room, a building, or a city. Technically, a LAN doesn’t have to be localized in one area, but it is preferred, and we will just assume it is so for arguments sake :-).
  • A WAN is a Wide (Area) Network that connects multiple LANs together. This is what the Internet is.
  • The way one computer finds another computer on a network is through its IP Address [hereby referred to as IPs in this post only]. There are other protocols, but this (TCP/IP) is by far the most widely utilized and is the true backbone of the Internet. IPs are like a house’s address (123 Fake Street, Theoretical City, Made Up Country). To explain it in a very simplified manner (this isn’t even remotely accurate, as networking is a complicated topic, but this is a good generalization), IPs have 4 sections of numbers ranging from 0-255 (1 byte). For example, 67.45.32.28 is a (class 4) IP. Each number in that address is a broader location, so the “28” is like a street address, “32” is the street, “45” is the city, and “67” is the country. When you send a packet from your computer, it goes to your local (street) router which then passes it to the city router and so on until it reaches its destination. If you are in the same city as the final destination of the packet, then it wouldn’t have to go to the country level.
  • The final important part of networking (for this post) is the domain system (DNS) itself. A domain is a label for an IP Address, like calling “1600 Pennsylvania Avenue” as “The White House”. As an example, “www.castledragmire.com” just maps to my web server at “209.85.115.128” (this is the current IP, it will change if the site is ever moved to a new server).

Next is a brief lesson on how DNS itself works:
  • The root DNS servers (a.root-servers.net through m.root-servers.net) point to the servers that hold top-level-domain information (.com, .org., .net, .jp, etc)
    Examples of these servers are as follows:
    auns1.audns.net.au
    bizE.GTLD.biz
    caCA04.CIRA.ca
    cnA.DNS.cn
    com&netA.GTLD-SERVERS.NET
    deZ.NIC.de
    euU.NIC.eu
    infoB9.INFO.AFILIAS-NST.ORG
    orgTLD1.ULTRADNS.NET
    tvC5.NSTLD.COM
  • Next, these root name servers (like A.GTLD-SERVERS.NET through M.GTLD-SERVERS.NET for .com) hold two main pieces of information for ALL domains under their top-level-domain jurisdiction:
    • The registrar where the domain was registered
    • The name server(s) that are responsible for the domain
    Only registrars can talk to these root servers, so you have to go through the registrar to change the name server information.
  • The final lowest rung in the DNS hierarchy is name servers. Name servers hold all the actual addressing information for a domain and can be run by anyone. The 2 most important (or maybe relevant is a better word...) types of DNS records are:
    • A: There should be many of these, each pointing a domain or subdomain (castledragmire.com, www.castledragmire.com, info.castledragmire.com, ...) to a specific IP address (version 4)
    • SOA: Start of Authority - There is only one of these records per domain, and it specifies authoritative information including the primary name server, the domain administrator’s email, the domain serial number, and several timeout values relating to refreshing domain information.

Now that we have all the basics down, on to the actual reason for this post. It’s really a nuisance trying to explain to people why their domain isn’t working, or is pointing to the wrong place. So here’s why it happens!

Back in the old days, it often took days for DNS propagation to happen after you made changes at your registrar or elsewhere, but fortunately, this problem is of the past. The reason for this is that ISPs and/or routers cached domain lookups and only refreshed them according to the metrics in the SOA record mentioned above, as they were supposed to. This was done for network speed reasons, as I believe older OSs might not have cached domains (wild speculation), and ISPs didn’t want to look up the address for a domain every time it was requested. Now, though, I rarely see caching on any level except at the local computer; not only on the OS level, but even some programs cache domains, like FireFox.

So the answer for when a person is getting the wrong address for a domain, and you know it is set correctly, is usually to just reboot. Clearing the DNS cache works too (for the OS level), but explaining how to do that is harder than saying “just reboot” ^_^;.

To clear the DNS cache in XP, enter the following into your “run” menu or in the command prompt: “ipconfig /flushdns”. This does not ALWAYS work, but it should work.


If your domain is still resolving to the wrong address when you ping it after your DNS cache is cleared, the next step is to see what name servers are being used for the information. You can do a whois on your domain to get the information directly form the registrar who controls the domain, but be careful where you do this as you never know what people are doing with the information. For a quick and secure whois, you can use “whois” from your linux command line, which I have patched through to a web script here. This script gives both normal and extended information, FYI.

Whois just tells you the name servers that you SHOULD be contacting, it doesn’t mean these are the ones you are asking, as the root DNS servers may not have updated the information yet. This is where our command line programs come into play.

In XP, you can use “nslookup -query=hinfo DOMAINNAME” and “nslookup -query=soa DOMAINNAME” to get a domain’s name servers, and then “nslookup NAMESERVER DOMAINNAME” to get the IP the name server points too. For example: (Important information in the following examples are bolded and in white)

C:\>nslookup -query=hinfo castledragmire.com
Server:  dns-redirect-lb-01.texas.rr.com
Address:  24.93.41.127

castledragmire.com
        primary name server = ns3.deltaarc.com
        responsible mail addr = admins.deltaarc.net
        serial  = 2007022713
        refresh = 14400 (4 hours)
        retry   = 7200 (2 hours)
        expire  = 3600000 (41 days 16 hours)
        default TTL = 86400 (1 day)

C:\>nslookup -query=soa castledragmire.com
Server:  dns-redirect-lb-01.texas.rr.com
Address:  24.93.41.127

Non-authoritative answer:
castledragmire.com
        primary name server = ns3.deltaarc.com
        responsible mail addr = admins.deltaarc.net
        serial  = 2007022713
        refresh = 14400 (4 hours)
        retry   = 7200 (2 hours)
        expire  = 3600000 (41 days 16 hours)
        default TTL = 86400 (1 day)

castledragmire.com      nameserver = ns4.deltaarc.com
castledragmire.com      nameserver = ns3.deltaarc.com
ns3.deltaarc.com        internet address = 216.127.92.71

C:\>nslookup ns3.deltaarc.com castledragmire.com
Server:  ev1s-209-85-115-128.theplanet.com
Address:  209.85.115.128

Name:    ns3.deltaarc.com
Address:  216.127.92.71

Nslookup is also available in Linux, but Linux has a better tool for this, as nslookup itself doesn’t always seem to give the correct answers, for some reason. So I recommend you use dig if you have it or Linux available to you. So with dig, we just start at the root name servers and work our way up to the SOA name server to get the real information of where the domain is resolving to and why.

root@www [~]# dig @a.root-servers.net castledragmire.com

; <<>> DiG 9.2.4 <<>> @a.root-servers.net castledragmire.com
; (2 servers found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 5587
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 14

;; QUESTION SECTION:
;castledragmire.com.            IN      A

;; AUTHORITY SECTION:
com.                    172800  IN      NS      H.GTLD-SERVERS.NET.
com.                    172800  IN      NS      I.GTLD-SERVERS.NET.
com.                    172800  IN      NS      J.GTLD-SERVERS.NET.
com.                    172800  IN      NS      K.GTLD-SERVERS.NET.
com.                    172800  IN      NS      L.GTLD-SERVERS.NET.
com.                    172800  IN      NS      M.GTLD-SERVERS.NET.
com.                    172800  IN      NS      A.GTLD-SERVERS.NET.
com.                    172800  IN      NS      B.GTLD-SERVERS.NET.
com.                    172800  IN      NS      C.GTLD-SERVERS.NET.
com.                    172800  IN      NS      D.GTLD-SERVERS.NET.
com.                    172800  IN      NS      E.GTLD-SERVERS.NET.
com.                    172800  IN      NS      F.GTLD-SERVERS.NET.
com.                    172800  IN      NS      G.GTLD-SERVERS.NET.

;; ADDITIONAL SECTION:
A.GTLD-SERVERS.NET.     172800  IN      A       192.5.6.30
A.GTLD-SERVERS.NET.     172800  IN      AAAA    2001:503:a83e::2:30
B.GTLD-SERVERS.NET.     172800  IN      A       192.33.14.30
B.GTLD-SERVERS.NET.     172800  IN      AAAA    2001:503:231d::2:30
C.GTLD-SERVERS.NET.     172800  IN      A       192.26.92.30
D.GTLD-SERVERS.NET.     172800  IN      A       192.31.80.30
E.GTLD-SERVERS.NET.     172800  IN      A       192.12.94.30
F.GTLD-SERVERS.NET.     172800  IN      A       192.35.51.30
G.GTLD-SERVERS.NET.     172800  IN      A       192.42.93.30
H.GTLD-SERVERS.NET.     172800  IN      A       192.54.112.30
I.GTLD-SERVERS.NET.     172800  IN      A       192.43.172.30
J.GTLD-SERVERS.NET.     172800  IN      A       192.48.79.30
K.GTLD-SERVERS.NET.     172800  IN      A       192.52.178.30
L.GTLD-SERVERS.NET.     172800  IN      A       192.41.162.30

;; Query time: 240 msec
;; SERVER: 198.41.0.4#53(198.41.0.4)
;; WHEN: Sat Aug 23 04:15:28 2008
;; MSG SIZE  rcvd: 508

root@www [~]# dig @a.gtld-servers.net castledragmire.com

; <<>> DiG 9.2.4 <<>> @a.gtld-servers.net castledragmire.com
; (2 servers found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35586
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;castledragmire.com.            IN      A

;; AUTHORITY SECTION:
castledragmire.com.     172800  IN      NS      ns3.deltaarc.com.
castledragmire.com.     172800  IN      NS      ns4.deltaarc.com.

;; ADDITIONAL SECTION:
ns3.deltaarc.com.       172800  IN      A       216.127.92.71
ns4.deltaarc.com.       172800  IN      A       209.85.115.181

;; Query time: 58 msec
;; SERVER: 192.5.6.30#53(192.5.6.30)
;; WHEN: Sat Aug 23 04:15:42 2008
;; MSG SIZE  rcvd: 113

root@www [~]# dig @ns3.deltaarc.com castledragmire.com

; <<>> DiG 9.2.4 <<>> @ns3.deltaarc.com castledragmire.com
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26198
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 0

;; QUESTION SECTION:
;castledragmire.com.            IN      A

;; ANSWER SECTION:
castledragmire.com.     14400   IN      A       209.85.115.128

;; AUTHORITY SECTION:
castledragmire.com.     14400   IN      NS      ns4.deltaarc.com.
castledragmire.com.     14400   IN      NS      ns3.deltaarc.com.

;; Query time: 1 msec
;; SERVER: 216.127.92.71#53(216.127.92.71)
;; WHEN: Sat Aug 23 04:15:52 2008
;; MSG SIZE  rcvd: 97

Linux also has the “host” command, but I prefer and recommend “dig”.


And that’s how you diagnose DNS problems! :-). For reference, two common DNS configuration problems are not having your SOA and NS records properly set for the domain on your name server.


I also went ahead and added dig to the “Useful Bash commands and scripts” post.

Windows Hosts File
When DNS decides to be finicky

Another of my favorite XP hacks is modifying domain addresses through XP’s Hosts file. You can remap where a domain points on your local computer by adding an IP address followed by a domain in the “c:\windows\system32\drivers\etc\hosts” file.

Domain names are locally controlled, looked up, and cached on your computer at the OS level, so there are simple hacks like this for other OSs too.

I often utilize this solution as a server admin who controls a lot of domains (Over 100, and I control most of them at the registrar level too ^_^). The domain system itself across the web is incredibly fastidious and prone to problems if not perfectly configured, so this hack is a wonderful time saver and diagnostic tool until things resolve and work properly.