Topic Options
#3889 - 09/21/21 10:00 AM Huge packet loss from hop one through to final hop
Auto Gnome Offline


Registered: 09/21/21
Posts: 3
Hello all, new to Pingplotter and this forum - Have spent a lot of time reading through the manual and Knowledge Base, really informative content and feel like I've learned lots already. However.. I'm still having trouble with pinpointing my issue.

Regardless of the destination, we are seeing averages sometimes in excess of 70% packet loss all the way from Hop 1 through to the final hop.

In my testing I have disconnected all clients and network devices from the router, turned off wireless radios, and only connected my laptop (running Pingplotter) directly to the router via gigabit ethernet. I have also tried another laptop of different (and equally high) spec, with a different (new) ethernet cable, yet the same issue and results persist.

At first I thought the router hardware was at fault, as the PL was originating at Hop 1 - However after doing some reading I learned that many routers de-prioritise ICMP traffic, therefore resulting in PL. At this point I increased the Interval from 2.5s to 5s, and even 10s; but the same level of packet loss remained after collecting 30-60mins of data.

It is worth reiterating again that my laptop was the only device on the network at the time - I would have thought that the router would drop less ICMP packets in this case as there is nothing else on the network for it to prioritise? I am now starting to believe the packet loss from Hop 1 could be a false-positive and the root cause may reside further up the route.

Our provider is a 'Business class' ISP. The service is FTTC and we receive good speeds of 60-80Mb down/ 20Mb up.

The reason we are using Pingplotter - We are a bar/ restaurant and use a cloud-based app for taking orders + payments/ printing receipts etc. Every day we get intermittent freezing of the app, which we believe is due to internet connectivity issues. However we don't tend to see any other issues if using other internet based apps such as streaming Youtube/ Spotify, or loading web pages etc. The tablets that we run this app on do not lose WiFi connectivity and when the issue occurs I am still able to successfully load a webpage to prove the internet connectivity has not dropped out.

When we originally raised our concerns with the ISP (before we did any detailed testing using Pingplotter), they actually replaced the router for a new unit, this was around 3 weeks ago. The only thing that hasn't been replaced/ tested so far is the RJ11 cable to the telephone socket - Is it worth doing this anyway or is it likely the issue resides on the WAN/ ISP infrastructure after hop 2?

If I connect another device up to the LAN then I get 0% packet loss when running PP to that host.

A summary of all the targets i have tested:
[img]https://www.dropbox.com/s/c0jjrdglc81u3cz/All%20Targets%20Summary.png?raw=1[/img]

Share Page for 24hrs - talktalk.co.uk
https://share.pingplotter.com/bTEb1EK6NNf

Share page for 24hrs - pingplotter.com
https://share.pingplotter.com/VzNKyAHy879

Really interested to get to the bottom of this, and hope that we can! I can export/ provide any other data that might be required.

Thanks for reading smile


Edited by Auto Gnome (09/21/21 10:01 AM)

Top
#3890 - 09/21/21 12:39 PM Re: Huge packet loss from hop one through to final hop [Re: Auto Gnome]
Poe Offline
Pingman Tools Support


Registered: 02/11/19
Posts: 77
Hi Auto Gnome,

Thanks for writing into the forum!

I appreciate your detailed troubleshooting report, it sounds like you've done a lot already.

The easiest way to know if you're getting false positive packet loss is to trace directly to the hop where you suspect a false positive from. In this case, your router (192.168.1.1). This will send an ICMP packet with a TTL greater than 0 so it won't drop those packets and will prioritize responding to them. If hop #1 comes back clean then I'd suggest trying hop #2 and see how it responds. If hop #1 is clean but there is an issue at hop #2 that means that it's either hop #2 or between hop #1 and #2. If you're seeing packet loss directly to your router with a known good ethernet cable then it's the router still. I think that using a known good cable would be one of the next things I'd check. Do you have a modem that you could plug directly into and bypass the router all together to see what that looks like?

After looking at your talktalk.co.uk pp2 file I see that there are some issues that appear to start at the router and go all the way through the route. These do not appear to be false positives (see images below) since you see the pattern of the same blocks of red packet loss going all the way through the route starting at hop #1.

Image_1

Image_2

It's possible that there may be an issue with NIC on the machine that you're testing from but it sounds like you tried more than one machine with the same results so that also seems unlikely. You can always trace to the NICs IP address or your loopback address (127.0.0.1) to verify that the results are clean.

Let me know what you find (I'm always so curious about tricky situations like this) and if you have any further questions!

Thanks,

-Poe

Top
#3891 - 09/21/21 03:44 PM Re: Huge packet loss from hop one through to final hop [Re: Poe]
Auto Gnome Offline


Registered: 09/21/21
Posts: 3
Hi Poe thanks for the quick response!

That's really good to know. I've actually already done the test directly to the router and was indeed getting huge packet loss as you can see here:

https://share.pingplotter.com/8xgpSBMcjZn

Weirdly though you will notice there was little to no packet loss between 1700 and 1830.

I have just ran the trace to my interface on the loopback address for 30 mins and all squeaky green with 0 PL.

Going into a little more detail on my original post.. as the ISP has already replaced the router (for a later/ 'better' model too) I imagine it's extremely unlikely that the router itself is at fault right? I am now starting to think something environmental such as EMI. The router is on a middle shelf under the till station - Directly behind it (within 6 inches) is a power strip with 3 or 4 DC power supplies connected to it for the various equipment that is on the shelves above and below (NVR for wired CCTV, NVR for wireless CCTV, audio amplifier, etc!) I actually think we have 5 6-gang power strips daisychained from one single wall socket eek Also I can't be certain whether the ethernet cables I have used to hook my laptop up to the router are shielded.

So do you think we can be confident the packet loss issue is on the LAN side of the router? Meaning it's not worth replacing the ADSL cable at this point? (appreciate this would usually be quick and easy to do in a normal domestic environment, but the cable for our setup runs in some trunking under the floor back to the ADSL socket)

I guess maybe the next step would be to move the router to an interference-free location and plug the power directly into an outlet on the wall?


Really interested to find out what the cause of this PL is.. but even more so I hope it also turn out to be triggering the issues we're getting with our order system app! crazy

Cheers

AG

Ps. I'm unable to see anything from your links (it goes to a page showing 'Access Denied')


Edited by Auto Gnome (09/21/21 03:45 PM)

Top
#3892 - 09/22/21 06:09 PM Re: Huge packet loss from hop one through to final hop [Re: Auto Gnome]
Auto Gnome Offline


Registered: 09/21/21
Posts: 3
Ok so some interesting results from testing today.

I left the router where it was but removed the power connector from the power strip and plugged it directly into a socket on the wall.

Ran a trace directly to the Router IP at 0.5ms interval and only returned an average PL of 0.5% over....I was about to write "1000 counts" but as I was typing this post it started dropping packets like crazy again from 800 counts onwards (averaging 40% PL over a 60sec focus)

The data is here - https://share.pingplotter.com/SYhUaYLoRSd

I have added comments at various points to detail key points of the test such as when the power was switched back to the power strip again.

One more thing I did was also connect the tablets (running the app experiencing the problems) up to my phone's 4G hotspot for the evening. We had it running like this for ~4 hours and the app did not lock up or return the issue once!

Some good/ semi-conclusive results I think. I guess the next step is to completely replace the router and go from there. I have a Draytek turning up tomorrow so am hoping to have this installed and working over the weekend.

Thanks again for your help on this Poe! smile


Edited by Auto Gnome (09/22/21 06:11 PM)

Top

Search

Who's Online
0 registered (), 8 Guests and 2 Spiders online.
Key: Admin, Global Mod, Mod