Having tested online multiplayer with two xbox consoles set up in my home, with a neighbor living across the street, and with simulated latency, I felt very confident that my network code would hold up online. I was wrong.
After completing a series of playtests, I decided to conduct a test with my nephew who lives 15 miles from my house. To my dismay, the game fell completely out of sync. Some of the bugs were caused by timing issues that simply don't occur under simulated latency or low latency, but did with the latency spikes experienced in the real world. Other bugs were caused by my misconception that network session properties would get updated as frequently as I sent other packets.
So I owe a big thank you to my nephew LeMonte and my friend David for helping me test online multiplayer! My next step...peer review!