Author: ccie14023

TAC Tales #19: Butt-in-Chair

Posted by | Dec 17, 2019 | |

This one falls into the category of, “I probably shouldn’t post this, especially now that I’m at Cisco again,” but what the heck. I’ve often mentioned, in this series, the different practices of “backbone TAC” (or WW-TAC) and High Touch Technical Support (HTTS), the group I was a part of.  WW-TAC was the larger TAC organization, where the vast majority of the cases landed.  HTTS was (and still is) a specialized TAC group dedicated to Cisco’s biggest customers, who generally pay for the additional service.  HTTS was supposed to provide a deeper knowledge of the specifics of customer networks and practices, but generally worked the same as TAC.  We had our own queues, and when a high-touch customer would open a case, Cisco’s entitlement tool would automatically route their case to HTTS based on the contract number. Unlike WW-TAC, HTTS did not use the “follow the sun” model.  This meant that regular TAC cases would be picked up by a region where it was currently daytime, and when a TAC agent’s shift ended, they would find another agent in the next timezone over to pick up a live (P1/P2) case.  At HTTS, we had US-based employees only, at the time, and they had to work P1/P2 cases to resolution.  This meant if your shift ended at 6pm, and a P1 case came in at 5:55, you might be stuck...

Read More

Where does time go?

Posted by | Dec 9, 2019 | |

Two things can almost go without saying: If you start a blog, you need to commit time to writing it. When you move up in the corporate world, time becomes a precious commodity. When I started this blog several years ago, I was a network architect at Juniper with a fair amount of time on my hands.  Then I came to Cisco as a Principal TME, with a lot less time on my hands.  Then I took over a team of TMEs.  And now I have nearly 40 people reporting to me, and responsibility for technical marketing for Cisco’s entire enterprise software portfolio.  That includes ISE, Cisco DNA Center, SD-Access, SD-WAN (Viptela), and more.  With that kind of responsibility and that many people depending on me, writing TAC Tales becomes a lower priority. In addition, when you advance in the corporate hierarchy, expressing your opinions freely becomes more dangerous.  What if I say something I shouldn’t?  Or, do I really want to bare my soul on a blog when an employee is reading it?  Might they be offended, or afraid I would post something about them?  Such concerns don’t exist when you’re an individual contributor, even at the director level, which I was. I can take some comfort in the fact that this blog is not widely read.  The handful of people who stumble across it probably will not...

Read More

Before the Internet: The Bulletin Board System

Posted by | Aug 15, 2019 | |

It’s inevitable, as we get older, that we look back on the past with a certain nostalgia.  That said, I think that computing in the era when I was growing up, the 1980’s, was more fun and interesting than it is now.  Personal computers were starting to become common, but were not omnipresent as they are now.  They were quite mysterious boxes.  An error might throw you into a screen that started displaying hexadecimal with no apparent meaning.  Each piece of software had its own interface which you had to learn, since there were really no set standards.  For some, there might be a menu-driven interface.  For others there might be control keys you used to navigate.  Some programs required text commands.  Even working with devices that had only 64 Kilobytes of memory, there was always a sense of adventure. I got my start in network engineering in high school, in fact.  Computer networks as we understand them today didn’t really exist back then, in the 1980’s, except in some universities and the Defense Department.  Still, we found ways to connect computers together and get them to communicate,  the most common of which was the Bulletin Board System, or BBS. The BBS was an individual computer equipped with a modem, into which other computer users could dial.  For those who aren’t familiar with the concept of a modem, this was...

Read More

TAC Tales #18: All at once

Posted by | Jul 5, 2019 | |

The case came into the routing protocols queue, even though it was simply a line card crash.  The RP queue in HTTS was the dumping ground for anything that did not fit into one of the few other specialized queues we had.  A large US service provider had a Packet over SONET (PoS) line card on a GSR 12000-series router crashing over and over again. Problem Details: 8 Port ISE Packet Over SONET card continually crashing due to SLOT 2:Aug 3 03:58:31: %EE48-3-ALPHAERR: TX ALPHA: error: cpu int 1 mask 277FFFFF SLOT 2:Aug 3 03:58:31: %EE48-4-GULF_TX_SRAM_ERROR: ASIC GULF: TX bad packet header detected. Details=0x4000 1234 Problem Details: 8 Port ISE Packet Over SONET card continually crashing due to SLOT 2:Aug  3 03:58:31: %EE48-3-ALPHAERR: TX ALPHA: error: cpu int 1 mask 277FFFFFSLOT 2:Aug  3 03:58:31: %EE48-4-GULF_TX_SRAM_ERROR: ASIC GULF: TX bad packet header detected. Details=0x4000 A previous engineer had the case, and he did what a lot of TAC engineers do when faced with an inexplicable problem:  he RMA’d the line card.  As I have said before, RMA is the default option for many TAC engineers, and it’s not a bad one.  Hardware errors are frequent and replacing hardware often is a quick route to solving the problem.  Unfortunately the RMA did not fix the problem, the case got requeued to another engineer, and he…RMA’d the line card.  Again.  When that didn’t work, he...

Read More

Interviewing #2: Why do we interview?

Posted by | Jun 27, 2019 | |

In the last article on technical interviewing, I told the story of how I got my first networking job.  The interview was chaotic and unorganized, and resulted in me getting the job and being quite successful.  In this post, I’d like to start with a very basic question:  Why is it that we interview job candidates in first place? This may seem like an obvious question, but if you think about it face-to-face interviewing is not necessarily the best way to assess a candidate for a networking position.  To evaluate their technical credentials, why don’t we administer a test? Or, force network engineering candidates to configure a small network? (Some places do!)  What exactly is it that we hope to achieve by sitting down for an hour and talking to this person face-to-face? Interviewing is fundamentally a subjective process.  Even when an interviewer attempts to bring objectivity to the interview by, say, asking right/wrong questions, interviews are just not structured as objective tests.  The interviewer feedback is usually derived from gut reactions and feelings as much as it is from any objective criteria.  The interviewer has a narrow window into the candidate’s personality and achievements, and frequently an interviewer will make an incorrect assessment in either direction: By turning down a candidate who is qualified for the job.  When I worked at TAC, I remember declining a candidate who...

Read More

Recent Comments