about


Scribe:

chrono

blog

Time

science!

book musings

gov

'punk

missle defense

jams

antiques

cultcha blog


Construct:

scripts & programs

on the town

snaps


Self:

creds (PDF)

key

missive


Traceroute: Finding a Lost Connection

Recently, my Web site seemingly stopped responding. Or at least it seemed that way from where I was, about 100 miles away at the time. I could not SSL in, nor could I get the Web page for the site. Even the Ping command did not work. Was my server down? Or was the network down?

I used the traceroute command to determine where the problem spot is. As it turned out, Traceroute did not solve the problem entirely, though it allowed me to define the troublespot with more precision.

Traceroute maps each hop along the journey from my computer to the end point that computer is trying to reach. On Windows, the command is “tracert [host name].” On Unix/Linux, it is “traceroute [host name].”

These were the results:

Tracing route to joabj.com [75.148.24.2]
over a maximum of 30 hops:

....snip first few hops...

  5    12 ms    12 ms    12 ms  so-15-0-0-0.RES-BB-RTR1-RE1.28.81.130.in-addr.arpa [130.81.28.248] 
  6    13 ms    12 ms    12 ms  0.so-1-2-0.XL3.IAD8.ALTER.NET [152.63.37.117] 
  7    14 ms    12 ms    11 ms  0.ge-6-0-0.BR2.IAD8.ALTER.NET [152.63.41.149] 
  8    12 ms    12 ms    11 ms  te-11-3-0.edge1.Washington4.level3.net [4.68.63.169] 
  9    14 ms    14 ms    13 ms  ae-1-69.edge1.Washington1.Level3.net [4.68.17.16] 
 10    14 ms    14 ms    15 ms  COMCAST-IP.edge1.Washington1.Level3.net [4.79.231.14] 
 11    14 ms    15 ms    14 ms  68.85.130.53 
 12    15 ms    14 ms    14 ms  68.85.130.54 
 13     *        *        *     Request timed out.
 14     *        *        *     Request timed out.   (repeated until hop number reached 30)
 
Trace complete.

Each successive line represents the next hop on the network. In the above set of data, the first column is the count of nodes from the client making the request. The following three times (in milliseconds) indicates the the time that node took to respond to the traceroute request (about 500 milliseconds is the upper-end of acceptable response time. And the last column is the Internet Protocol (IP) number and, if available, the domain name of the node at that location.

As this page helpfully explained, it appeared as if a node at least one hop away from my own server was not not responding (75.148.24.2).

This was indicated by the * * * in the output. The asterisk means that the node did not respond to the traceroute request in a timely manner. A single asterisk can be typical, especially for a busy server, though all asterisks indicates trouble is afoot.

What was odd was that my own server was not the last IP number listed. So it could be that it was down, or it could be that there was another hop. As the tutorial suggested, I did a traceroute from another location, in this case from Network-Tools(A whole list of others could be found here). From this location, it appeared as if my server was responding fine, but the hop before it was not responding, I.e.:

Hop	(ms)	(ms)	(ms)		IP Address	Host name
1	28	6	8		72.249.134.177	-
2	7	6	8		8.9.232.73	xe-5-3-0.edge3.dallas1.level3.net
3	9	6	6		4.71.198.14	comcast-ip.edge3.dallas1.level3.net
4	29	30	42		68.86.85.254	pos-0-13-0-0-cr01.atlanta.ga.ibone.comcast.net
5	47	45	42		68.86.85.238	pos-0-12-0-0-cr01.mclean.va.ibone.comcast.net
6	46	48	59		68.85.130.53	-
7	54	48	47		68.85.130.54	-
8	Timed out	Timed out	Timed out			-
9	59	50	53		75.148.24.2	www.joabj.com

Trace complete

So, whatever that hop is between 68.85.130.54 and my own site (75.85.130.54) was not responding. But my own site was! Most peculiar.

Adding to the mystery even further, I found I could access my Web site just fine, using an anonymous public proxies (public proxies are third-party sites you can go to visit other Web sites, without either that site or your own network administrator knowing the IP details of either party). This meant that other parties could access my site, but no computer from the IP address I was could.

This other site notes that the no information coming back from a hop (“* * *”) means only that that hop doesn't respond to traceroute requests, not that it is not operating correctly. It could not be returning information for security reasons, for instance. This would not be the case if it were the last hop that was not responding in the traceroute—this is why it is important to do traceroute from alternate locations. (And this page points out that sometimes timed-out requests do not, at least in some cases, represent, nodes; rather they are artifacts of the traceroute reporting mechanism.)

So, summary (for now): Something at the location one hop away from my server is blocking access to that server, though only for my own remote location. According to this page, this may be my IP provider (Comcast) blocking any traffic from my own IP address (maybe because of the large number of files I had been transferring between the two locations).

The good news that I am working from a location with dynamic IP, so getting another IP address address should do the trick. (Update: I regained access from this location the next day).