Topic Options
#2317 - 02/04/12 05:53 PM Opinion on my assessment would be appreciated.
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Hi,

Man! If there's one person that needed your software that would be me.

I've had problems dating back since Sept 2011 that still haven't been resolved between two other parties.

In order for you to make sense of what I need an opinion on, I will post the last four emails I sent to the parties in question.

The main questions I'd like to ask are;
>Did I do enough to try and prove where the problem is?
>Could I do more?
>Am I correct in my assessment of the results in PingPlotter?
>What other questions/reasons may there be that I didn't submit to the parties that could cause this problem? A few good ones would help if I need to send them an email again.

I will ask more questions later if you don't mind.

I couldn't post my email attachment or pics here because of the size. I hope you don't mind that I add a link to my server to view it. It's a .doc file containing four emails from the first to the last in that order.

http://ispzone.ca/pingplotter-forum/re-support-inquiry-ticket.doc

I'm going to post some updates I sent to the parties here just incase you respond before my 48 Hr test is complete. This will provide you with a current status of events.

Update 1

http://ispzone.ca/pingplotter-forum/updated-2-4-2012_6-20-25_PM.jpg

Update 2

http://ispzone.ca/pingplotter-forum/updated-2-5-2012_3-41-04_AM.jpg

Update 3

http://ispzone.ca/pingplotter-forum/updated-2-5-2012_11-27-31_AM.jpg

Important Update 4

http://ispzone.ca/pingplotter-forum/updated-2-5-2012_2-13-32_PM.jpg

Update 5

http://ispzone.ca/pingplotter-forum/updated-2-5-2012_4-21-29_PM.jpg

Update 6 - Final till I hear from either tech department.

http://ispzone.ca/pingplotter-forum/to-whom-it-may-concern.doc

One thing about all of this is that my packet loss far out ways yours and my latency times are much smaller but they are both still directly proportionate to one another. In your screenshot example;

http://www.pingplotter.com/screenshot.html

My screenshot example with the same sample time, graph time and packet loss graph as yours;

http://ispzone.ca/pingplotter-forum/2-6-2012_8-15-29_AM.jpg

The packet loss in yours is way below your higher latency compared to mine and proportional.

>Do both scenarios still yield the same results? Saturated Bandwidth problem?
>Can you tell me what difference it makes in the link?
>Why is there a dip every morning between hops 11 & 12-13 at exactly 2:00AM till 8:45AM in the graph? It's not like everyone is dropping off that pipe at the same time every morning right? Why is the link becoming unsaturated?
>Am I causing the packet loss on the final hop? My latencies really seem to influence the packet loss, especially when I downloaded as can be seen with the new tracert I started.
>Am I still on the right track with what I stated to the techs? Is hop 11 really the culprite?

http://ispzone.ca/pingplotter-forum/bbol.embanet.com-PingPlotterStandard-first-tracerts.pp2

http://ispzone.ca/pingplotter-forum/bbol.embanet.com-PingplotterPro-new-second-tracert.pp2

Thanks


Edited by acdcmike1 (02/06/12 02:29 PM)

Top
#2321 - 02/06/12 10:59 PM Re: Opinion on my assessment would be appreciated. [Re: acdcmike1]
Pete Ness Offline



Registered: 08/30/99
Posts: 1106
Loc: Boise, Idaho
Lots of information to parse through here.

Have you correlated your results with other embanet users? This is often a good way to isolate (and/or eliminate) components from the problem. If multiple users have similar experiences - even from different geographic areas (with different providers), then you can focus on just the common elements as problems. Alternately, if others do *not* have the problem, you can focus on just the differences.

It looks a lot like bandwidth saturation on their end. The challenge about saying "it's bandwidth saturation on the part of embanet" is that the further you get away from your own network, the more opportunities you have for return routes to impact your data - and you can't see the return route in PingPlotter or any other trace route tool - at least not without having an agent on the remote end of things. Sometimes your provider (the one at hop 2) might have multiple peers and the outbound one works great but the return one (which you can't see) doesn't work nearly as well. This might *look* like an embanet problem (and it very well could be), but in the end it turns out to be an issue with peering closer to home.

The fact that there is a correlation between the close-to-home latency and the final-destination packet loss is a pretty strong indication that your issue starts closer to home rather than at the final destination.

This isn't really much help to you, except to tell you other things you might look for, and to tell you that there may be other things in play here that you can't see.

Your situation is not entirely simple (obviously!), and the quantity of data you've collected and long summaries may be making it more difficult for you to get action. I think the sheer mass and sprawling case you're trying to make might be working against you in some ways.

Based on the data I see here, I can't yet rule out your own provider as a significant contributor to your problem.

I would do the following (not knowing what exactly you've tried and done over the last few months):

* Gather data from other users of the final destination to see if they are experiencing the same.
* Gather data about your provider (DSL Reports, for example) to see if there are any complaints about your provider.
* Start isolating things that you can easily control. If things aren't working well on your home PC / network, take your laptop to a nearby coffee shop and see if it's a problem from there too - or maybe from a friend's house.
* Focus more on the effects of the problem (the inability to connect / use) rather than quite as much disparate data. This depends *a lot* on the provider you're working with, but it's hard to sort through all the data if it's not what they normally deal with.

The amount of packet loss you're seeing is pretty huge for it to be a problem with a pretty widely deployed and used cloud-based application like you're dealing with (I think). It's certainly possible that it's a problem at the far end, but you've really got to work under the assumption it's very close to you first.

Sorry if I've not been much help here.

- Pete

Top
#2325 - 02/07/12 07:39 AM Re: Opinion on my assessment would be appreciated. [Re: Pete Ness]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Thanks a ton for the info Pete. I know I had a lot. frown

I've been doing other tracerts and downloads etc, to dig deeper. I too felt that the problem may still be at my end. I started second guessing myself. I always kept in mind to work from the last hop through to the second though. It really does look like hop 11-13 is the problem.

Finding other users won't be easy, not public info.

I'm on a WIFI (rural) 3.5 GHz system.

The pattern I found really puzzles me. There is absolutely no change at my end computer wise and the packet loss on the 11-13 link goes down to 0% - 10% from 2 - 9 AM! Everyday!

Well, I will keep working at this, I never give up.

Mike


Edited by acdcmike1 (02/07/12 08:33 AM)
_________________________
I will prevail.

Top
#2326 - 02/07/12 03:49 PM A test. [Re: acdcmike1]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Hi Pete,

I've been working on the problems all day and communicating with Embanet. Nothing concrete from them yet.

I performed a simple test to your site.
I started to downloaded a 1GB file and stopped it part way through while tracing to your server.

http://ispzone.ca/pingplotter-forum/www.pingplotter.com-test1.pp2



>I didn't have any packet loss in hop 1, I bypassed my router and went dynamic, made no difference. Great!
>Hop 4 shows 6% loss between xplornet and atlas, xplornet and link to atlas is to blame.
>No loss showing in the final destination column. Only looks bad because I magnified the window, there was a 1% loss at three points. This is negligible.
>Because I was downloading at full KB's from another server, it affected my latencies starting from hop 1, and reflected onto the concurrent hops to the destination which is normal.

All in all this is a very good connection and I assessed this correctly I hope right?

Thanks


Edited by acdcmike1 (02/09/12 07:25 AM)
_________________________
I will prevail.

Top
#2327 - 02/07/12 04:08 PM Re: A test. [Re: acdcmike1]
Pete Ness Offline



Registered: 08/30/99
Posts: 1106
Loc: Boise, Idaho
Yup - looks solid, and exactly what you'd hope to have happen.

What this doesn't do, though, is help understand the return route. If your provider has a peer that is bandwidth saturated but is the preferred provider by data coming from Embanet (but not *to* Embanet - these routes are determined independently), then you might still be getting packet loss at the boundary between your provider and that peer without seeing the problem when targeting other servers. This is the challenge with asymmetrical routing. I don't know if that's what's happening, but it's a credible scenario that makes it difficult to pin the problem on one host. This usually only happens in the short-term, though, because everyone figures out that one peer is saturated and starts asking for data to be prioritized to another peer.

It would be really nice to get a traceroute / PingPlotter dataset from the Embanet server back to you - this is usually not possible, but depending on the network situation at Embanet they may be able to do this with you. If the server is hosted in the same facility where support works, and they share a network, then this becomes possible. If the server is hosted in a datacenter with no other support machines available, it's a lot closer to impossible.

I'm kind of taking the provider's viewpoint here just to help you understand what they *might* be thinking so you can investigate more thoroughly.

- Pete

Top
#2328 - 02/07/12 04:27 PM Re: A test. [Re: Pete Ness]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Had a hard time getting that pic to register. A little rusty with tags.:)

I read up on peering today and I understand what you mean, saw some peering diagrams, thanks for the extra knowledge, it will stick. I sent this to Embanet today;

http://ispzone.ca/pingplotter-forum/please-investigate-this-issue.htm

I will work my way up to a dataset. If I'm on their backs long enaough they may buckle, you know, do the Columbo thing. smile

Mike


Edited by acdcmike1 (02/07/12 04:41 PM)
_________________________
I will prevail.

Top
#2330 - 02/08/12 02:45 PM Some headway. [Re: acdcmike1]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
I received a response from Embanet yesterday evening. This is what they had to say;

Incident Thread
Response (Aaron) 02/07/2012 05:33 PM
Hello Michel,

Thank you for taking the time to contact us.

We are following up with you in regards to your case; we have received a workaround from Xplornet for the connectivity issues you are encountering. Please use the following steps to add proxy settings to your browser:

For Internet Explorer:

1. Access "Tools" and select "Internet Options"
2. Select the "Connections" Tab and select "LAN" settings
3. Select the check box beside “Use a proxy server” and enter the following information:
Address: proxy.ripnet.com
Port: 8888
4. Select the check box beside “Bypass proxy server for local addresses”
5. Select "OK"
6. Select "OK"

For Mozilla FireFox:

1. Access "Tools" and select "Options".
2. Select the "Advanced" tab and select "Network".
3. Under "Connection" select "Settings".
4. Select "Manual Proxy Configuration".
5. Enter in the following information in the appropriate areas:
Address: proxy.ripnet.com
Port: 8888
6. Select "OK" and select "OK" again.


If the steps above don't work please communicate this with Xplornet.

If we may be of any further assistance, please do not hesitate to contact us.

Regards,

Aaron
Helpdesk Support Specialist

You may reply to this email to update your ticket.
Answers to most frequently asked questions may also be found at http://supportcenter.embanet.com/ontariolearn

Response (Aaron) 02/05/2012 12:33 PM


It didn't work. The address that is, and I find this to be a really lame fix. Go onto proxy with higher ms times and downtimes, real smart! I'ts not really a fix, just another evasive way to not work and correct the real problem at hand.

I contacted Xplornet and they have no record of this fix, they say they never sent an email to Embanet regarding this. Hummmmmm! I then proceeded to force the level one tech to patch me through to a supervisor. Normally you can never get to one, I did this time. Dan was bent out of shape at first about speaking with a client but warmed up real fast when I summarized the events that led to me speaking with him. He and I finally hit it off technically and I gave him access to the web based side of PingPlotter Pro and his first reaction was WOOOOOOoooooow!
He used this program once to trace within a network for another company he worked for and loved it. He's fallen in love with PingPlotter Pro again. smile He started taking snapshots of the conditions in PingPlotter, even logging after we got off the phone!, and assured me that he would get it to his engineer friend in Xplornet to finally fix this issue. I have since been tracing and keeping a record of things for both he, and I. He does agree with my conclusions regarding the problems I've been having and deeply apologized for the length of time it's taken to resolve this issue, he has some techs he will be speaking to.

I will tear a strip off of Embanet when the time comes too.

Once again, I cannot begin to express my gratitude for PingPlotter Pro and Pete Ness. Both your program and you ROCK! smile

Mike


Edited by acdcmike1 (02/08/12 02:51 PM)
_________________________
I will prevail.

Top
#2331 - 02/08/12 05:28 PM Found something really interesting! [Re: acdcmike1]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
*Interesting Development - On 2/8/2012 at approximately 4:00PM*

Packet loss is at its high point in hop 13 with low latency, and hop 1 packet loss has disappeared!

So, when my packet loss is high in hop 2 , hop 13 subtracts that loss from losses already there. When packet loss is low in hop 2, packet loss is added to losses already there in hop 13. If you look along the graph you can see this.

I would have never seen this anomaly if it hadn't of happened today in front off my eyes and comparing them to other tracerts I did. This would explain the lower differences in hop 13 behavior today. Packet loss was always 0%-3% Max in hop 2, or if it did get higher it wasn't there long enough for me to see the relationship in either hop. There was no effect from this in hops 3-11.

It's as though the packet loss from hop 2 and hop 13 is somehow switching places.

What's going on?

http://ispzone.ca/pingplotter-forum/bbol.embanet.com.pp2

http://ispzone.ca/pingplotter-forum/tracert-info.pdf

Updating these files about every hour.

Short video clips of login issues at different packet levels;

http://ispzone.ca/pingplotter-forum/10/up-to-10-percent-packet-loss.html

http://ispzone.ca/pingplotter-forum/40/up-to-40-percent-packet-loss.html

Cheers,

Mike

An Update





Edited by acdcmike1 (02/10/12 02:44 PM)
_________________________
I will prevail.

Top
#2333 - 02/13/12 07:38 PM Update [Re: acdcmike1]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Update, end of this tracert.



_________________________
I will prevail.

Top
#2338 - 02/17/12 09:45 PM Re: Update [Re: acdcmike1]
acdcmike1 Offline


Registered: 02/04/12
Posts: 14
Loc: Mountain, ON
Well,

I sent another letter yesterday to the parties involved except to Embanet.

Hi Dan and Brandon, (Xplornet and Sungard employees)

Thank you for taking the time to look into this for me.

I see that the problem still exists though, and I’m not hopeful this will be resolved anytime soon. I’ve had this issue since Sept of last year and to be quite frank with you, I’m at my wits end.

I’ve decided to go by way of litigation to seek restitution for the lack of support, personal damages including my health and welfare, and other reasons. As you can imagine, I’ve built quite a case regarding this issue and I feel very confident that I will have no problem convincing the courts that I’m a victim.

My main target of course is Embanet/Ontario Learn. Their failure to resolve this problem and having made me veer off the beaten path to investigate, has seriously taken time away from my studies. This has not only affected my train of thought and grades, but my health as well. I pretty much had to change my entire life to adapt to the login times available to me, login times by the way that I had to figure out myself. I’m used to getting up at 8:00AM, not going to bed at this time. This has stressed me in many other areas as well, from finances too my relationship and family, too the very way I live my life. I am on a timeline regarding courses I must take and my future, this timeline has been crippled.

As it stands, I’m still waiting for responses from both Xplornet and Sungard regarding the high packet loss on the link between Sungard and Embanet. There’s a reason for this packet loss other than my computer or connection, and it needs to at least be found and or corrected.
I will ask my lawyer to seek an external audit of the networks involved regarding this issue, if need be. As far as I’m concerned, it was Embanet’s responsibility to correct this problem from the very beginning.

I hope that one of you can provide me with a concrete answer as to what is causing the high packet loss, “we know where it’s happening”. Also, if it will be corrected.

Regards,

Mike Thellend



Then I tried to login to my system during the day today and voila!

Here’s a 7 day scale tracert from the separate dynamic IP, computer, and Xplornet account I have, packet loss disappeared;




Here is my static IP and system on a 30 minute scale;
This problem was corrected at exactly 3:55.42PM today. I noticed a lot of action regarding IP changes and new routes from hop 6 to hop 15.




I didn't receive any email from them telling me it had been repaired. I still don't know who is responsible or what exactly was wrong. I decided to send one final email.


OK,

I take this back! “I see that the problem still exists though, and I’m not hopeful this will be resolved anytime soon”


Gentlemen, I feel that I’m entitled to know a few of the W5’s here for the months of trouble I went through.

Who was responsible to correct this problem? Was it Embanet seeing as that I was a student through them or Xplornet my ISP?

What was the problem? We know it was packet loss but was the cause bandwidth saturation on the link between Sungard and Embanet, a hardware issue or something else?

Where was the problem? I think I had that right too but was it somewhere else or a combination of two or more places? I noticed 19 hops now instead of 13, different route midpoint now but still Sungard to Embanet.

I would appreciate that these questions be answered,

Regards,

Mike Thellend


So now I'm going to wait and see.


Without concrete evidence there was a problem to begin with I would have probably never got the point across to them.

Thanks PingPlotter!


Edited by acdcmike1 (02/17/12 09:55 PM)
_________________________
I will prevail.

Top

Search

Who's Online
0 registered (), 18 Guests and 1 Spider online.
Key: Admin, Global Mod, Mod