Greetings! What a fantastic product thanks!

In reading the documentation I got the idea (correct ?) that with respect to packet loss I want to rely on what is showing at the destination/last hop and use loss on other hops to try and narrow things down. I don't really get this. If I have something like this:

2% loss to 2nd hop (ISP default gateway)

0%-5% over say 10 next hops

1% at last hop/destination

What does this mean? I am running multiple VOIP connections and am finding them very vulnerable to packet loss. Imagine hypothetically that these values are static... what is the implication...do I go by the destination/last hop and say that my VOIP packets are suffering only 1% packet loss? I am guessing that what happens is that packets are re-transmitted and that the only thing I should really be concerned with is what I am seeing on the last hop? Also that perhaps I should be running PP back the other way and see if I am getting 2% when trying to send to this end?

In practicality what I have found is quite interesting. I went from a business connection that "tweaked" their service after many inquiries (including packet loss to default gateway... they got than down to 0% and basically almost always 0% to the endpoint!) however I had issues with complete service outages. I then switched to a supposedly more robust service at nearly 4 X the cost and I see some packet loss over almost all the hops and WAY more red lines in PP (albeit thin ones) in this connection than the previous! My users are not reporting much in the way of call quality issues however dialing is a problem! My conclusion is that DTMF tones are being interupted by packet loss and beeeeep comes through as beee---------eeeep which gets interpretaded on my end as two digits instead of one!

Thanks for any comments/advice!

Hello, Voiper.

You always look at the final destination first, then look upstream for the first hops that don't show similar loss. If you're not seeing packet loss at the final destination, then none of the other intermediate data matters - those are just routers delivering data. We have an analogy on why this happens like this in our VoIP troubleshooting guide. If your final destination is showing 1% loss, then that's the number you report - not hop 4 that's showing 40% loss, or hop 1 that's showing no lost packets.

There are certainly nuances to this. You should definitely make sure you're examining enough data to have your packet loss numbers be statistically relevant - if you're only looking at 100 samples, then that's not statistically enough data to trust the 1%, 0%, 2% and 5% numbers. If you're looking at 1000 samples, then you start to get more statistical redundancy that makes you believe the numbers more.

So, let's say you're looking at 10000 samples and you're seeing 2% at hop 2, 0% at hop 3, 5% at hop 7, 0% at hop 10, and 1% at the final destination (hop 11).

First, you have to look at the difference between 0% and 1% and see if these are really close with rounding (let's say you have 55 lost packets at the final destination and 45 at hop 11). Those are pretty close to the same, and you'd have to look upstream for roughly 1/2 of 1% packet loss and see which is the first hop that doesn't show that same loss.

Another scenario is that you've lost 1 packet at hop 11, but 145 at hop 12 - that's pretty statistically compelling - you know hop 11 isn't the culprit. If you have other intermediate hops that also have ~145 lost packets, then you start to suspect the return route. If hop 9 and hop 12 have ~145 lost packets, but hop 10 and 11 have < 10, then you suspect that the packets from hop 12 are going back through a different provider than hop 10 or 11, but hop 9 is using that same one. In that case, tracing the route back from the other end is hugely helpful (if you have control of the destination and you want to do that with PingPlotter Pro, you can set up a remote agent at the far end and trace both directions in one instance of Pro).

VoIP does not retransmit - and neither does PingPlotter. If a packet doesn't make it, it doesn't make it. In PingPlotter, this is shown as a lost packet. In VoIP, it causes a loss of voice data, which can sometimes be smoothed over, and sometimes not (especially if there are enough of them).

I'm not enough of a VoIP expert to validate your theory on packet loss interrupting DTMF tones. DTMF is built to be noise-resistant, though, and I'd be surprised if a single lost packet would cause that sort of problems. Are you using G.711 or G.729? G.729 is a lot more susceptible to packet loss and network call quality issues.

If you want us to look at your situation in specific, collect a few hours of data (especially good if you do that when you're using the line and can report in problems through the comment feature) and then send the .pp2 save file to support@pingplotter.com. We can analyze it, extract the best picture to show what we think is important, and then post a picture back here (or just keep it in email if you prefer).

- Pete

Ohh yeah here was the original/old connection:




and the new one:

OK - based on your pictures, that problem is all about hop 2. Statistically, those are close enough that you point at hop 2 for all the problems here. 7 lost packets vs 4 lost packets is not enough data to really say that *for sure*, but it looks very, very similar. I wouldn't think that amount of packet loss would cause DTMF disturbances either.

Hi Peter. Thanks for the fast replies... almost like chat!

Regarding retransmission I believe that this is because VOIP is typically UDP right? In my case it is encapsulated by IPSEC so I THINK this is wrapped in TCP and therefore would get retransmitted which is probably something that we don't want with RTP anyway.

I am 99.9% sure on the DTMF issue. It happens only when the connection(s) are "worse." and on the FXO end I clearly always see a double digit dialed when the user reports that they are having issues dialing. This is probably hardware specific and I know there are a tonne of settings on these boxes (out of band dialing) that I should check.

Anyway on the 1st pic the 1st hop is the default gateway as it is connected directly to the PC. On the second/new connection Hop 2 is the ISP's default gateway where I am typically seeing a consistant 1-2% loss over 200 samples. Peter the results are similar over a large sample collection and that would be great if I can send the captures.

Thanks!

DTMF tones can be sent in two ways (that I know of). One way is to treat the tones like voice packets, and the other is to embed the tones as commands to the intended system (PBX). Check which one you are using and try the other.
Also, read this:
http://www.voiploop.com/index.php?option...mp;amp;Itemid=0
Dave

IPSEC is only the security blanket used for the VPN that you are travelling through. VOIP packets (yes UDP on RTP) are never resent. It would not make sense to resend voice packets as the delay could be unreal.
It occurs to me that you may be experiencing a nasty amount of jitter

Dave M

OK, maybe I'm nuts!
You used the term IPSEC. This to me means that you are using a VPN. If yes, how come the pictures you sent have so many hops? Maybe you are using PP outside/beside the router? Last time I used PP in this way I had 3 hops because I put my laptop on the voice network. Even though there may have been many hops to get to the other end, PP could only see three. The gateway, the router at the other end, and my voice card.

Also, what gear are you using? FXO/FXS are terms used to define analog sets over IP...

Dave M

Dave not nuts lol. Yes am running IPSEC VPN - appreciate any additional comments on this as far as I can tell this is TCP traffic and the WAN has no way to know that it is UDP inside therefore errors packets dropped over VPN will get retransmitted no?

Obviously running PP over the VPN is only going to show the source and destination that is why I did the testing outside the tunnel to see whats happening through the hops that the VPN runs over.

Hmmm...
Well, it is my understanding that your (and all of our) VOIP packets will NEVER be resent EVER. The packets sent are ALWAYS UDP over RTP and cannot (that I know of) be changed to TCP. This would take too much time to do and would add to delay problems. So...

a) IPSEC has to re-negotiate a security key every so often which interupts data flow. What is the interval?
b) What type of routers are you using?
c) how many VPN's are you using?
d) Is this a teleworker app or pbx networking?
e) G711 or G729?
f) how many concurrent calls?

Dave M