Verio strikes again. About half the IPs in our C block seemingly went dead the other day. Web servers on those IPs couldn’t be hit from outside and we couldn’t ping anything beyond our own router from them. Something along these lines happened last year and it turned out someone at Verio had decided to subclass our C block and assign part of it to a new DSL customer without bothering to tell us (or even pinging a few of the IPs to see if anyone else might be using them). I called Verio and was able to confirm my theory fairly quicly – it was Verio’s doing. But this time they simply put a bogus static route in one of their routers that mis-routed half our C block for no apparent reason. It was suprisingly easy to get the tech-support person to understand what the problem was too. That’s very unusual as most Verio tech-support personal seem to pride themselves on their total lack of comprehension of anything technical and how to support it. I did have to tell them which router the problem was in but I’m used to that.

Getting it fixed was classic Verio. The tech-support department can’t actually fix anything themselves (and I don’t blame them for that – I wouldn’t let those people near any of my servers either!). They have to email a central problem resolution center and request that a third group be contacted that might actually fix the problem. They claim there is no way for them to contact the second group directly because that group has no telephones. And they claim the third group, the ones that actually fix problems, also have no telephones. Both groups can only be contacted by email. Of course, they have no idea how long or even if the other groups will take in responding to their email. It blows my mind that they actually expect anyone to believe that a corporation the size of Verio has entire departments of people who can’t be contacted by telephone.

From the time I first contacted Verio, it took them around three hours to fix the problem. That’s probably two hours and fifty nine minutes of waiting for people to get the email requesting the fix and one minute for someone to log into the router and remove the bogus route.