Ideally, you should have 0ms and no packet loss when you ping your modem. That's not always what happens - it's often an issue with pinging a router that's not really built to be pinged. It's definitely worth looking in to a bit, but keep in mind that a router is not a great endpoint.

You might try pinging a final destination that you're having issues with, and run another instance of PingPlotter to your router. Correlate both sets of data with problems you're having (VoIP, XBox gaming, etc). See if the bad data matches.