No Love For Central Office Techs

As it says in my little bio over there:

I have a lot of experience with enterprise wide area networks (WANs). This means I've spent more than a few hours, evenings, weekends, and holidays arguing with CO techs about circuits.

I wrote a blog about MPLS Carrier Diversity a few months back which included this:

The local loop is where most people assume problems will occur, and generally, they are correct. Basic T-1s delivered by LECs are prone to outages due to several reasons. Many T-1s are really just 2-wire DSL lines that are converted to 4-wire T-1s at your office. So, you get to pay for a T-1 and get all the love of a DSL line. Next, T-1s are a CO techs least concern. CO techs will bounce them, loop them, or reprogram them without blinking an eye. Have you wondered why your T-1 is so stable at night and bounces during the day?

Well, that little outburst about the professional honor of CO techs led to an e-mail the other day from an ex-CO tech. My new friend, let's call him Ron, wrote:

----- Original Message ----
From: Ron
To: mjmorris@yahoo.com
Sent: Friday, March 14, 2008 10:46:49 PM
Subject: offended

Hello,
Having been a Central Office switch tech for years, I am offended by your
remarks concerning the "attitudes" of CO techs towards T-1 ckts. T-1's are
the bread and butter of the industry still and in my experience were treated
as such. In my opinion, the people who never learned the "5 9's" of service
(99.999% reliability in case you are 'that' young) are the on prem IT
people. I currently am a phone system vendor and am constantly amazed at the
willingness of IT staff to bring service down, without notification and with
known traffic on the circuit.
It's all a matter of perspective, and I believe that yours is skewed.
Thank you for your time,
Ron

I replied to Ron with a few questions. These came from my experience over the years working in NOCs and dealing with outages, often cause by rogue CO techs:

• Have you ever or have you seen another tech bounce an entire channelized DS-3 to fix a single T-1? (Thus taking down all the other circuits)
• Have you ever or have you seen another tech leave a T-1 looped for hours by mistake only to blame something else when the problem is found?
• Have you or have you seen another tech ever down the active side of a SONET ring without checking the protect?
• Have you or have you seen another tech ever swapped a card in a MUX with a single protect card before checking if any other cards were using the protect?

His answers surprised me in that he prefaced them with clarifying that he was not an old, grumpy, union CO tech working for a Bell. He worked for a non-union long distance company. He often dealt with union employees that refused to work with him. It got me thinking that a lot of my dislike for CO techs came from dealing with union operators, who often gave me the impression they could care less about the customer being affected (in most cases, me). I worked at AT&T as a "management" employee in a NOC and dealing with union CO techs was very frustrating - be it AT&T, Sprint, PacBell, Bellsouth, SBC or some other LEC. They knew their union rules and were not going to go one step beyond to help a customer. But this ultimately hurt these employees. Being rigid and inflexible in today's economy was a recipe for disaster. How much CO work has been automated today away from union labor? Union employees seemed to have forgotten who the customer was.

I'm not a big fan of unions because I don't think they really help employees or the economy. Sure, you may think you're getting great benefits and a solid job from the union, but just ask UAW members how they feel now. Unions don't have a place in today's global economy, especially in telecommunications. Why do you think there's no Cisco Tech Union? We know we have to compete in the global economy or get outsourced to India. We respond to our customers quickly or they get new vendors. The same is happening in telecommunications.

Let the hate mail begin. ;-)

PS - Here are Ron's answers:

• Not on purpose, and not ever between the hours of 8 and 5.
• Yes to the first part of the question, including myself.
• Only had a one SONET ring, so really "NA"
• I think it probably happened, I'd be surprised if it didn't

So, for now, I'm sticking to my story.

More >From the Field blog entries:

Do You Have an Architecture Review Board?

NX-OS's Best Feature: Virtual Device Contexts (VDCs)

Come Visit Me at FutureNet

* Tips on spending your time well at Cisco Networkers, plus: bring back the CCIE party!!

* NX-OS - Some Software For all that New Nexus 7000 Hardware

* A CCIE job that only offers $150K - ummm...maybe...well...no.....

* The DC3....err....Nexus 7000 brings some exciting hardware to the DC LAN

Go to Cisco Subnet for more Cisco news, blogs, discussion forums, security alerts, book giveaways, and more.

Recent Cisconet blog entries

20 useful sites for Cisco networking professionals

Network World's IT Buyer's Guide: Cisco products

Subscribe to Network World's Cisco Alert, which includes a weekly digest of all Cisco Subnet items