When I run an trace from China to Singapore, I usually have about 25 hops. I understood that the latency from the final hop is the closest thing to overall latency from point A to poing B.

In our run, the middle hops (where are problem starts) have latency that is much higher than the final hop. How can this happen if the data has to run through the entire path to get to the destination router. Below is an example from the txt file:

"10/31/2002 3:32:32 PM",0,3,3,2,5,4,4,7,5,9,10,29,32,34,29,64,64,68,*,35,894,891,894,938,275,78

Any help would be appreciated. I am actually using the trial version of your tool, but if I can make sense of this, I am interested in buying quite a few copies as I've found it very helpful identifying bottlenecks for our Internet users.

Thanks

There are several possibilities for why this could be happening. Before we start talking about them, I just want to reaffirm that the final hop's latency is the latency to get from you to that site, and then back again. The final hop is the *most* important number - as all of the intermediate data is just helpful information that may be useful for troubleshooting.

1) Intermediate hops may be down-prioritizing ICMP echo requests or replies. This is pretty normal - but not to the scale that you're seeing. Normally, this will demonstrate itself in one or two (or several) routers that jump up in latency occasionally, but not constantly - and usually no more than 2X the latency of the final hop (this is a general observation - not a rule - sometimes it may drop packets completely, or add a lot of latency).

2) The intermediate hops are using a different return path. This is more likely - that the packets in the middle are coming back via a different route than the packets at the end. Unfortunately, there's not any good way of figuring out what route this is - unless these routers are under your control to the degree that you can run a traceroute from those routers back to yourself. We recently had a bit of a discussion about this that may help explain how "asynchronous routing" works.

There are some other more remote possibilities, but option 2 above is the most likely culprit on what results you are seeing. This becomes even more convincing if the routers that are reporting bad latency are all inside a specific network provider's boundries - and the hops that are reporting normal latencies are outside these boundries. If that's the case, then the return routing rules for that provider probably are going through a non-optimal link.