Well, it's not 100% clear. Hop 2 has some definite issues that looks to be significantly impacting your performance. Hop 2 is the first hop inside your ISP, so that could be the issue. Based on when you restarted your router, it looks like your biggest pain was happening when you have high latency at hop 2 - which would be between hop 1 and hop 2 (or your main connection).

The packet loss and latency you have at downstream hops (past hop 2) really needs to be correlated with the experience you're feeling. Does your internet experience match what you're seeing at hop 2 more, or what you're seeing at hop 4. If it's a closer match to hop 4, then the problem could be inside your ISP. If it's more like hop 2, then it's between you and the ISP.

Let us know where this leaves you.

- Pete