Turning on the graph at hop 1 also shows the exact same pattern at hop 1. Hop 1 is inside your network (your modem).
So there are a couple of possibilities here ...
First, reboot your modem (do I sound like your Comcast support tech? :)) and see if that changes anything.
Second, if you actually have 2 computers on your network (or maybe you have a fridge or a thermostat or some other device that responds to ping and is wired to your network), I would try pinging from one to the other directly. You're trying to eliminate variables, and when you directly ping another computer, you're not using the gateway in your cable modem. If you have a router plugged in to your cable modem, then traffic isn't even touching your cable modem.
If that still shows the same problem (you'll only have one hop), then the issue is inside your own network (maybe a bad network cable, network card, bad cable modem itself, or some massive amount of network use by something inside your network that's saturating your internal network).
If the latency goes away when you ping another computer / device on your own network, then it's likely an issue with the cable modem, or with the connection between the cable modem and the outside world. The packets are not leaving your own network, but maybe the cable modem is doing something (like retraining, or trying to mitigate some problem that it sees with its connection) that is delaying packets for some reason. It's probably time to get in touch with Comcast and see what they have to say. They may be able to see something in your modem configuration that is solvable.