AllDayDevOps 2018: Progressive Testing to Meet the Performance Imperative

Mostly as an appendix for references and readings, but whenever I can I like to have a self-hosted post to link everything back to about a particular presentation.

My slides for the presy: https://docs.google.com/presentation/d/1OpniWRDgdbXSTqSs8g4ofXwRpE78RPjLmTkJX03o0Gg/edit?usp=sharing

Video stream: https://www.youtube.com/watch?v=DexfpnbBFn8

A few thoughts from my journal today (spelling and grammar checks off):

Will update as the conversation unfolds.

 

Adhocathon: Selenium Conf Chicago 2018 – Rstats in Grid

The goal of this time-boxed, scoped hackathon is to visualize the health and activities of a Selenium Grid instance by contributing the code necessary to report key metrics via Statsd. This supports observability and operational reliability requirements of continuous testing “at scale”.

Date/time: Thu Oct 18th, 6-8pm in the LaSalle room

Sign up here: https://goo.gl/forms/SztY8j7j83RZQEkJ2

Why Selenium Grid and Statsd?

Out of the box, visibility into how Selenium Grid is working/performing can be…limiting. This is a problem, because when a test fails (maybe multiple times), how can you quickly know where the issue originated (app, test, environment, data) if all of these things are not easily observable?

…and because lots of people use both of these technologies, or at least should, and they are both premised on separation of concerns at architecturally significant points, such as test-vs-environment and business-logic-vs-logging. Also, this is a stated potential area of focus for improvement by a few of the Selenium core contributors, and because the Statsd protocol has extremely wide-spread support in the industry.

Why So Last Minute at the Conference?

A few weeks ago while helping to organize DevOps Days Boston, I had a mental breakthrough about why testing gets talked about very little in organic DevOps communities: much of automated functional and non-functional testing doesn’t speak the same lingo as development and operations.

One of the core DoD organizers, Laura Stone, had just published a set of training videos on Statsd which I was impressed with, learned something from, and it struck home that we don’t have telemetry for Selenium Grid, a focus of my recent pet project during the day.

Many large organizations I work with use WebDriver extensively, and in order to fit something as critical as functional testing into a DevOps-minded delivery pipeline, software components and services must be operationally visible. The health of supportive dependencies, as they contribute affinities and deficiencies, are often as important as the health of primary deliverables.

So I had the courage to reach out to my volunteers’ contact at Selenium Conf Chicago about an idea I had on contributing to the Selenium project, namely layering in Statsd functionality into Selenium Grid to improve the observability.

Why Is This Not a Hackathon?

My ideas are shit. Not always, but most of life is an opportunity to learn what you don’t know. A majority of hackathons typically structure as a competition for prizes, and typically gravitate around some narrow vendor offering or are staged solely to promote some new service. Some business sets the tone and parameters, then judges contributions based on a set of meritable characteristics. This is not what I want.

In every area of my professional life, I look for opportunities to listen, learn, and deliver value. What better way to do those things that get a lot of other people to do them with you, multiplying the chances of exfoliating great ideas rapidly! The closest I know to this idea is a hackathon, but much of the work to getting the best ideas out quickly isn’t coding; it’s necessary, but often not the most relevant outcomes. So I needed a new name: “ADHOCATHON” is an iterative, time-boxed design-code-deliver session format.

How Is It Going to Go?

I have no idea. I often get these things right, like last month when I got 40+ managers together as a meetup to have a “forward dialog” about hiring and recruiting challenges and tips in the current Boston tech climate. The point is not that you know what to expect, but that you can facilitate the “best” outcomes to become a useful contribution.

That said, the conference organizers were happy to help however they could, and now we have a room at the venue right next to the first-day reception area. I have a number of customers and colleagues signed up to be there. There will be tweets going out via the conference handle. And I will have found my way into the place that Tim O’Reilly describes as “creating more value than you capture”.

For the next few days, I will be scrambling to provide minimalistic resources to accomplish this goal, such as:

Whatever happens, it will definitely be interesting. I’ll publish a follow-up article on what happened and outcomes here after the conference ends.

 

Engineering Is About More Than Code

Curiosity is what drives engineers, and is equal parts curse and companion. An engineer isn’t limited to development or operations. An engineer would be a problem-solver in both areas, probably more. Curiosity is a surprisingly rare quality in people, even in technology.

If you want to know how something works, take it apart and observe. My first digital systems disassembly was a Sony Discman in 1989. Whatever I did, I fixed it. The feeling was powerful. It just took me 25 years to realize that there are many broken things in the world and to prioritize which one’s I involve myself with. Understanding the problem is crucial.

This is how I approach many conversations, navigating purposely and politely until there’s a useful reframing. People aren’t things, so be kind, be sensitive, and be patient. When you engage, learn about their biggest challenges, how they approach things, and what drives them. Just start with that.

[If my 80’s discman was still around, it would be like “yup, those were the days”. My 8086 XT clone next to it would splutter out some op codes. My Mega Man game watch would be waterlogged and stuck in a loop. Which all lead me to the next action…]

Put things back together again so that they work, hopefully, better than before. It’s just courtesy. In commit-worthy code that’s called hygiene. In conversation, that’s called maintaining a shared view or vision. Do these things enough and you’ll find that the way to a common goal is easy easier than with clutter obscurring your journey, and for others’. When you can row in the same direction, you get to your destination a whole lot faster.

Put it with other things to see where it doesn’t work. That’s integration and it’s not always easy, especially if it’s your own code or new auto-scaling configuration that causes unforeseen things to blow up. Get to know what your thing does before and after you put it out in the wild. Be honest with yourself and others about the time this takes.

There are some things you can take apart, and some things you can’t. If you can’t or if it’s too much effort for not enough value, move on to another learning tool, but remain committed to your goal. In the light of fundamental flaws in how we think about security, privacy, and basic human welfare right now, seek something you’re proud and grateful to do. [There are some very worthy things happening in Boston right now.]

Subverting Social Bad Behavior with Community Service

Summary: personal aha…spend equal time on your community as you do on social media, and everyone will be better for it.

The Local Buffoonery of Garden Variety

When I woke, it was because of the shouting. That usually doesn’t happen in my neighborhood at two in the morning. I don’t mean like “hey, you forgot your beer on top of your car” kind of shouting, I mean the kind with curses and violence into the darkness of night. When I finally sat up to look down onto the street, between the fuzzy patches I saw a local taxi service driver furiously shaking his fist at something down the street yelling things.

As he used his mobile phone to call the cops, there was time to lite a…non-uniform cigarette…and pace around right in the middle of the blindest corner in town. Four more real cigarettes later and five flashing, silent police vehicles quickly swarmed what’s typically a peaceful daytime intersection that my apartment windows overlook. Two at a time, the tatted-up Ford Explorers sped away in the fisterly direction, leaving the final officer to take what I can only imagine was a confusing deposition and also a flashlight look-around to validate the driver’s story.

Today, Sunday, I drove downtown to ask the daytime taxi dispatcher what happened. I was expecting that I’d also have to stop up at the police station to get their official statement of what happened, but the guy at the desk knew the whole story and was happy enough to spill. Here it is.

After an overly complicated girlfriend-plus-other-friend pickup, some mentally imbalanced passenger, seeing a fare that surprised him as far too high, escalated an argument about the fare to an unexpected roundhouse punch to the driver’s head. That’s right, with two other people in the back, he temporarily disabled someone with the responsibility of driving them all somewhere on Eastern Point.

How doped up or insane does a passenger have to be to attack the driver? Why is this ragtag group of weirdos proceeding to somewhere on Eastern Point (very affluent)? Movie stupid, very upset, either or both. The world is really, truly crawling with crazy, fucked up people at all levels (especially in politics). And, like taxi drivers, sometimes you don’t have the economic luxury of pushing back on psychotic behavior when it comes your way because of what you do.

The Political Buffoonery of Friends and Neighbors

Assault is of course a crime…depending on who you are these days. It should be, it doesn’t work at very large scale in the commons where most of us live. But for some reason, everyone from the well-educated to the god-fearing seem to think that it’s open season on assaulting others who don’t share their beliefs. When it happens verbally on local social media (in particular, Facebook), I step back and ask, “what side of all of this do you want to be on?”

Take this guy I’d hope to call a friend, Jam. Jam is all kinds of good and crazy and well-intentioned. He runs a local blog, or maybe a cemetery where overly opinionated blog posts go to die, I don’t know at this point. Whatever he is, he’s complicated. Sadly, and rightly so, the current political climate has turned him into an-eye-for-an-eye asshole on social media. A counterpoint-liberalist, we’ll call him the Cook, recently said some things in response to Jam and other local liberals’ boycotting a local business for some insensitive things the conservative owner said. This is like Spanky and Our Gang vs. the Little Rascals type bullshit. “Not beef”, as some might call what this is.

This unwelcome display of reverse-perspective outreach sparked a knock-down, drag-out social media war between two individuals that are very reasonable when not provoked. But neither even saw what they were doing: focusing on the meta over the impact. Impact of their meta, more polarization. Opportunity cost of the meta over of useful action. Long-term impact of rejecting others’ views and position on their journey, a.k.a. karma. Acceptance that there is a shared journey. Acknowledgement that they may be wrong, no matter how much they think they’re right. That they are most certainly wrong if they think they’re the only one’s who can see something right. Complete subjectivity instead of focus on objective measures.

The result is that at least one of these parties is regretful and walking around hurt by it all. The worst thing is that I don’t think it’s my friend Jam. He still hasn’t worked his ego problems out before diving headlong into ‘other’ advocacy with women’s movement and anything else the overwhelmingly liberal-biased feeds feed him. All those things are good, but without love and respect, what good does it do for Jam to constantly interject himself?

It’s all buffoonery until we do something useful with others, not flashy or geeky or memorable or unusual. Not something that gets you a group photo with Captain Picard at a fancy inherited wealth house or the following to argue to against Trump multiple times a day instead of showing up to community roundtables about how to improve local education. Speaking from this experience, it was nice to be in a place where everyone practiced listening as much as they practiced having a spine and a heart. This is what I and others did yesterday from 3-5pm downtown. I don’t feel at all that way about the online shouting match Jam and the Cook participated in.

Don’t Let Nationalism Buffoonery Legitimize Local Buffoonery

You may think, “I’m so different than what’s going on politically right now, I need to speak out!” Please find more useful ways to argue with people over social media. Need solid reasons not to? How about that you won’t be permanently quoted in exquisite online conversational detail by an article like the one you’re reading now. How about, instead of counter-trolling, you could be sipping something nice right now. Fuck, anything but arguing with neighbors online.

Instead of escaping from the real world, think about how the way you view things affects those around you. How about that you don’t want your check-in with captured moments of friends and family to be perforated by “what’s trending” bullshit between town neighbors. Sure, share stories and articles that you’ve validated and make sense to you in various forums; but once an online conversation looks like it’s getting nasty, politely move along. Don’t let something as transparent and egotistical as “my blog swings 2,300 local votes” be your hood ornament. If people are being victimized, eject the violator and report their poor behavior.

In short, and to bring it full circle, don’t let a culture precipitated by a talentless, washed-up genital wart of a President and his cargo cult nationalist, supremacy-retrograding base suck you in to the human-capital machines that Jack Doorsey and Mark Zuckerberg built but now take it with their pants on from Russian mafia-state. Find me instead and we’ll identify something far more useful in our town to work on together. I’ve been collecting people’s challenges, many of them just take a little bit of help from others to overcome.

If you can’t be bothered to focus on useful activity over timespend on social media, at least try this: every minute you spend on social media this week, spend equal on something useful for the people who live around you. Can’t think of what that might be? I’ve got some ideas handy:

  • Donate supplies to a program that need it (this week, it was tampons)
  • Walk around your neighborhood (or just your block) and pick up trash
  • Clean up, repair something in, or just post pictures of a great little local park
  • Engage others to write about their local views over just your small clique
  • Prioritize and facilitate actionable community projects needing nothing more than hands and time to complete

When I see people not doing stuff like this, it’s usually the folks that maybe don’t deserve a seat at our community table. They like to bark, but don’t like to dig. They look influential, but are really inconsequential. They are simply taxpayers, not community members.

How to “Hire” into a “DevOps” Market

Let’s be clear. I’m mostly writing this so that I don’t have to have another bullshit conversation with a high-level agency “technical” recruiter that doesn’t really know what the hell DevOps really is. But before you misinterpret what’s to come as a horn-rimmed trash session on recruiters (only a motif, I promise), consider that had you not read to the end, you would never have learned how to hire more efficiently, effectively, and ethically.

Boston, We Have a Big Problem

Aside from the occasional scuffle at a Java meetup, I’m in the Greater Boston Area, and Boston isn’t exactly known as a shining example of the tech boom. Sure, we have Facebook, Amazon, ZipCar, TripAdvisor, Chewy, RaizLabs, and Pivotal amongst others. We also have Oracle, IBM, Salesforce, and a plethora of other institutionalized madness from the Fortune 500 which typically drags our communal technical proficiency rating down year after year. We also have some of the most dedicated, seasoned professionals which you’d be hard-pressed to find in Silicon Valley during a fictional OSCON meets AWS Re:invent meets RSA all rolled into one.

Our problem is hiring. Everyone everywhere does not have the problem quite like we do. Its potency is not diminished in the DevOps space simply because its a generally applicable point to make for any highly skilled market. When something is as culturally intertwined as True DevOps mindset is in high performing teams, traditional recruiting approaches will always fall flat on their face. What people are calling DevOps positions right now are a loose collection of uninformed guesses, buzzwords, poorly crafted hiring pitches, philosophical paradoxes, voodoo, and outright corporate misunderstandings.

Recruiter, Know Thyself

What’s required is simply a sea change in recruiting mindset: the fastest way to qualify people is for you to qualify yourself. I don’t mean credentials, I mean the way you ‘smell’. Not by your morning shower and shave, but by what real practitioners need to hear from your mouths in the first 30 seconds: “I’ve really been thinking about how to right-fit you and a few of my clients,” or something equally real and challenging. How about “I’m interested to know what kind of work you’d like to be doing…” or “Which do you like better, people or code?” (that one is for all you recruiters looking to place the even more elusive ‘DevOps Manager’ position, which as it turns out is just a normal technical manager that’s passionate about coaching and improvement and customers and building future leaders.)

Or you could just leave us really good hires out of it, focus on the email overload flowing from opaque B2B recruiting firms currently choking your inbox (yes, I have recruiter friends and we talk), and then you’d be bidding for the lowest common denominators and margins, not the highest ones. Good luck with that approach, it leads to burnout and we DevOps people know a million ways to get burnt out. But unless you’re ethically fine with placing unqualified people with unqualified teams for a buck, the first option is better for you and for everyone you churn through week after week.

Be Transparent..to a Fault

Recruiters and hiring managers, there is nothing worthwhile to hide from someone like me that isn’t abundantly already transparent through what you don’t know about the organization you represent. I don’t have to ask questions about the org, just about you and your relationship with actual decision-makers, and what you don’t understand or know enough about to hire on behalf of True DevOps teams properly.

To get over cursory technical qualifications, ask people for examples of their work, or better yet look for them first; the simple act of a prospect answering you with examples of their expertise (or simply knowledge to-date) in particular areas is something unqualified people can’t do and unmotivated people simply won’t do. If your candidate hasn’t used any of a dozen or more social platforms (like Stack Overflow, Medium, LinkedIn, etc.) to publish their own stuff, encourage them to so you can pass it along to the real decision makers.

Once these minimum-viable qualification hoops are behind us, bring something to the table. Have a spine and a brain and a perspective about the challenge you’re looking for my help to solve in an org. Understand the broader issues the organization you’re representing is having, then ask us which ones we think we actually have the desire and a real shot at helping to move forward.

Suspend disbelief for a moment…

Drop the bullshit. Tell us who your client is when we ask, how many and which positions they have open, how many other candidates you’re playing off each other for some high-dollar plot of glory, and for god’s sake be prepared to describe your client’s *engineering culture* like it was your own family. Go to a meetup every so often, or better, help to organize one. Sit through and listen, stop emailing from your phone through the presentations. Absorb what’s happening, how people respond to certain topics, and if you don’t find yourself startled awake by clapping, you may just learn something.

Against all corporate recruiting conventional wisdom, when a potential candidate comes at you with such Maslowian questions as “Can you tell me a bit about their culture?” and “What challenges there would I even be interested in?”, definitely don’t talk about benefits packages (that are mostly all the same at a certain level anyway) and please don’t use things like “occupational environment”, “team synergy”, or “interpersonal skills”. Please be real, explain to us what you do so we can understand if you do it better than anyone else we will talk to that week, and leave the qualification script at the door.

You’re selling you to us first, then you earn the right to sell your client to us. If the order of those two things is in reverse, it’s equally transparent where your placement priorities lie.

Hiring Managers: Who’s Doing Things Differently?

Off the top of my head, I can think of a few folks in the local Boston DevOps meetup that are good examples of how to place highly qualified practitioners:

  • Dave Fredricks, Founder, eninjia.io
    This guy is legit. Hard working, constantly advocating, and an early organizer of DevOps Days Boston, amongst other dedicated individuals.
    .
  • Sam Oliver, 3yrs self-employed recruiter, now FTE at PathAI
    This woman really rolls up her sleeves, as a co-organizer of the Boston DevOps meetups, and smartly carved out the “we’re hiring” pitches from the “let’s talk tech” conversation that this crowd is known to like as separate things. #listening
    .
  • Kara Lehman, Principal Recruitment Consultant, Huxley
    Her stock is climbing with me. We first chatted it up in 2017 and when I asked her what she was doing at the nerd-fest of a meetup at Pivotal, she said that she “needs to understand this thing called DevOps better”, which is far more enlightened of a response that about 99% of other local recruiters.

If your name isn’t in the above list, nothing personal, I’m writing this at 10pm on a Monday. Let’s have a conversation where I vet you out and then maybe I’ll write about you too. In the mean time, prepare yourself because it will be me who’s interviewing you.

Apologies and Thanks

If you got this far, my gift to you is my true gratefulness for feedback you may have and maybe a retroactive apology for saying things harshly. We need to cut through bullshit, especially the corporate flavors of it, and this is my personal blog anyway.

Recruiters: if you “can’t do these kinds of things” in the position you’re currently in, consider that you would probably earn far more by taking a new  approach (like the one in this article) on your own and making 100% commission on your own closes instead of a measly 50k/year plus.

Also, you can also reach out to me via LinkedIn and let’s talk about your challenges in hiring, training, managing, or fostering a DevOps-minded team. I’m much nicer in person than in this post.

More on DevOps:

Open Spaces at Swiftfest Boston

I’ll be hosting an open spaces “in the wild” session at Banyan Bar from 6pm-9pm at the end of Swiftfest Boston Tuesday June 19th 2018. Look for me and for people with party beads like this:

Location and Timing

We will be meeting at Banyan Bar right around the corner from the conference. Feel free to walk over after the conference ends at 6pm, the conversation will start at 7pm, and I’ll be handing out free drink tickets at around 8pm after we are done.

What is open spaces?

Open Spaces is an “unconferencey” way to have small group conversations on topics that are most relevant to our community at just the right time. You submit topics (http://bit.ly/openswift2018) that you want to discuss in a group setting, then before the event, these topics are grouped into categories that help people self-organize into right-fit groups.

What Will We Discuss?

Because this open spaces event is the last workshop session of SwiftFest Boston 2018 (http://swiftfest.io/), it’s likely that topics will gravitate towards technically-focused iOS development, testing, and operations, and team management. Great for conference attendees, great for Mobile Tea members too!

However, a key affinity of the open spaces approach is to let the community figure out what matters most right now, and there’s a lot going on in the broader tech scene in Boston today, like DevOps leadership, sustainable startup practices, effective technical recruitment, municipal transportation and infrastructure support, etc. You never know, and that’s the fun part!

Socratic Method for Advocacy

In disassembly of how I approach a zero-knowledge situation, a few key dynamics emerge. The goal of this simple framework is to accelerate the bond-forming process of dialog between individuals.

Not everyone has an appetite for the full menu of techniques here, and right-fitting takes sizing up others in real-time, which itself takes practice. If you’re really interested in how to have more effective conversations that leave all parties feeling positive, this is what I can currently offer.

  • Introductions
  • Exploratory Listening
    • Open-ended questions (a.k.a. “double-clicking”)
    • Challenging to understand how topics relate and handling skills
    • Confirming what was shared with close-ended questions
    • Synthesizing the crucial point
  • Identifying the “as-is” current state
    • Share “the Summit”
    • Assess the journey to date
    • Materialize motivations (unearth drive)
  • Planning the “to-be” desired state
    • Identify milestones, past and future
    • Hunt for and prioritize gaps in order of risk to reach the summit
    • Summarize approach and obtain permission
  • Co-develop a plan of action
    • Goal-Strategy-Objective-Tactic
    • Visualize the timeline to reverse-engineer milestones
    • Socialize with key stakeholders

Introductions

Be cordial, clear about your role, short about your background, and quickly move to questions that help the other person(s) engage about themselves.

Present well. Shave, brush your teeth, wear a bit of a smile. Smell like someone you’d want to be around. Be attentive, particularly in the eyes, and suppress the urge to look like you’re anywhere else but this conversation right now.

Make sure that the person has time to talk a little. If not, politely ask when. If so, ask them how they arrived here, where they’re headed, and what’s got them going this direction. Build a basic rapport in the first few moments. Start off well and build on the previous moment at every opportunity

Exploratory Listening

Ask far more questions than the number of statements you make. Extroverted learning prefers information you don’t know over suppositions you could make alone.

Use open-ended questions when you want to explore directions. If you don’t know enough about what someone’s describing, open more windows. When the way of conversation is unknown, let them talk and learn how they communicate. There’s a lot of ‘meta’ information in human speech.

It’s important to challenge people, in no abrasive manner, but through asking how the current conversation point or branch relates to another topic or branch. The dichotomy of human behavior is that what is unknown represents simultaneously a source of both intrigue and fear. Questions can either encourage people to engage or to retreat, and our job is to engage.

Asking a question that someone doesn’t have an answer for leads to insight no matter what. How someone deals with unknowns will become useful later. When the question is right-timed and right-fit to the context, people without an answer are likely to explore with you. Poorly-timed or out of context, a question where no one has the answer feels awkward and often causes people to retreat.

When something seems very important, narrow down the scope of the questions you ask, never coerce. Close-ended questions confirm that what you think you heard was actually what was said. Don’t ignore non-verbal cues. Look for expressions of emotion surrounding quantities. Form a mental map of how these points affect their recipient and which seem relevant to them, not just you. Verify their relevance to others with close-ended questions.

Once a branch of exploration is sufficiently developed, synthesize the crucial point uncovered back into the main theme of the conversation. By understanding it’s impact, you can bring the point of the branch back to the goal of the conversation: to understand and support what the person is currently working on, suffering from, or driving towards.

Anyone can voice the crucial point, but it’s better if it’s someone else. You don’t have to be the one with an answer. This is why being curious about their perspective is so incredibly effective. Questions (open or closed) guide the conversation, even though people tend to think that an idea is original and their own simply because they voiced it. When someone thinks the idea is theirs, they tend to champion it with banners and bugles. Questions help steer champions in the right direction.

Identifying the “as-is” current state

The first move in any consultation should be to gain situational awareness. In other words, qualification of what dynamics and decisions are currently in place. Before a hypothesis and plan of action can be formed, observation.

To make broad and holistic observations, you must share the summit. As the landscape of context emerges from your listening expedition and as you process that landscape together into a shared construct, a key state will emerge, “what we have accomplished so far.” This is often coupled with pains and challenges regarding where the person currently is in their journey. Point and click with the camera in your mind because later we’ll be doing a before-and-after photo collage. The highest point achieved is a summit, even if it’s not the tallest peak visible.

You may need to know more about how they arrived at their current summit in order to fully flesh out gaps in context. This sense is highly subjective and experientially developed, so practice earning the right to get to this point. It’s important to step back and assess their journey every so often. This helps to uncover information gaps.

These gaps of context, though you may be the one to bring them up, must be accepted by others before they can be addressed. You can’t convince someone of a solution if they don’t think there’s a problem. And if something really isn’t a problem, it’s important for you to know that too. This is just another form of permission. The best way to ensure people show up to a party is to bring them with you.

Equal parts collected context and ‘meta’ (appetite, capabilities, deficiencies) makes for a very good peep into what motivates someone, either to stay in the current state, migrate to a future state, or even drive a future state. Motivations are imperative to materialize about others. An accurate understanding of someone’s motivations is worth more than developing a trust between you and them.

Planning the “to-be” desired state

Moving from what is to what could/should be the summit, start with our friend, an open-ended question, about where they want to be. In a group, this may get complicated; everyone communicates differently, and everyone has a different perspective. Quantize. Start with one person at a time. Give them non-verbal cues about length, but let them finish. The smaller the group, the easier this process is.

With a shared vision of the future summit, ask what it would take to get there. Searching for important, measurable future achievements is called identifying milestones. Again, not something everyone knows up front, and while some people have immediate intuitions or ideas, others take time to form their thoughts and answers. Since humans are notoriously bad at predicting the future, extrapolating future milestones may be difficult. If stuck, revisit past milestones, the ones that brought them to the current summit. This will help form a ‘meta’ understanding about how people categorize just-busy-work from notable achievements.

Make sure that the flow of questions to answers at this stage is 100% you questioning to 100% them answering. Questions may come up, but anything can be put in the “parking lot” provided it is suggested politely by you and agreed to by them.

With a set of milestones laid out, reorder them based on which events are dependent upon others. Ask about which milestones enable other milestones. Build the most efficient decision tree. As always, pause at junctures that feel important to obtain agreement. For those that don’t obtain consensus, circle back once, but otherwise parking lot it to keep the conversation flowing in the forward direction.

Now that you have a map of the landscape and have sketched out a path to the new summit, hunt down gaps (unstated objections, parking lotted details, well-known deficiencies) to arrive at MECE (mutually exclusive, collectively exhaustive) on a list of variables that exist between knowns. There will always be another iteration, so “exhaustive” simply means when people seem like they are happy with moving on.

With this list of gaps, ask how people would prioritize them. Root the conversation in terms of establishing priority based on relative risk to achieving the milestones and accomplishing the goal, getting to the summit.

There is no one right answer so long as those in the conversation express their perspectives on what order things should be prioritized. The how and the why they prioritize things the way they do is not key to this logistics exercise. Though useful meta information about individuals, justifications and disputes drag down the goal, which is alignment and on an approach to confidently arriving at positive outcomes together.

With a sense of how to prioritize milestones and gaps, circle back verbally to summarize the approach to moving from the old summit to the new one. This should take no more than 60 seconds. By now you should have mentally wordsmithed the goal, the milestones, and the approach such that you can do this. Those involved in the conversation up to this point will very likely unanimously agree because the summary incorporates their perspectives and terminology. When you do this effectively, you fade into their view of how to accomplish the goal. You are part of their team. You’re in.

Co-develop a Plan of Action

With all this context and alignment under your belt, it’s time to whip out some artisan management tools and have at it.

Crystalize the new summit into a single statement/slide/executive summary structured as a Goal-Strategy-Objective-Tactic (GSOT) framing. This progression properly organizes information in an approachable hierarchy that stakeholders and sponsors can grok quickly. Everything in it will be defensible because it came from the people that know how to accomplish the new goal and they’re already aligned with the details.

There typically should be only one goal, otherwise, the approach serves more than one master and the purpose of the objectives become muddied. An approach (or strategy) should express a strong point-of-view about the objectives that is fundamentally different from the prior approach which led to the current summit, even if the prior approach was good at decision time.

Every leg of the journey requires a pivot to ensure the direction is correct. Once a GSOT is constructed, make it easy for humans to visualize the timeline of objectives and key tactical events that support forward motion. These should be expressed in relative timeframes. Ranges, not absolute dates/times, are best at this point.

Reverse-engineer where/when to place milestones progressively. Start by placing the biggest rocks in dependent-events order, then layer in tactics that support various objectives. Adjust as necessary when tactics are tight (risk of slippage) or clustered too closely together (work bottlenecks). Do this with one slide, and you have the most effective meeting with an executive sponsor you’ve ever had.

Finally, with your ducks in a row, with everyone aligned, it’s time to socialize with stakeholders one at a time. Don’t present an idea for the first time in a boardroom. Objections are a fixture of the boardroom, so make it hard for them to maximize negative impact. Start with individual stakeholders. Do this under the auspices of collecting feedback, but then actually incorporate the feedback so you’re not a liar.

Pick your early stakeholder reviews carefully. A troll will turn around and proclaim your incompetence, but people with the most to gain will become your champion army when it comes time to fight the trolls. I wish you zero trolls, but living under bridges makes it hard to see them in advance.

On a Personal Note

The narrative above is a reflection of my observations and collaboration with technical engineering teams, boardroom executives, investors, sociologists, bartenders, and wolves. I broke my finger pretty badly two months ago which helped me deeply understand what would really hurt someone else. Likewise, as we exercise empathy and effective patterns of communication, we better understand how they improve us and their importance in our work together.

Thank you for your time, and as you may guess, I’m also on a journey. We all are. If you found anything in this post useful, connect with me so we can journey a bit together.

What a Site Reliability Engineer Really Does…in DevOps

We really, really build ourselves into a corner with the internet and mobile and cloud and Agile “at scale”. Good news is, we’re engineers that can invent ourselves out of anything, or at least that’s what’s made all this money so far.

What Is a Site Reliability Engineer?

Srsly. Wikipedia. Too lazy? Fine, from Wikipedia (please donate):

Site Reliability Engineering (SRE) is a discipline that incorporates aspects of software engineering and applies that to IT operations problems. The main goals are to create ultra-scalable and highly reliable software systems. According to Ben Treynor, founder of Google’s Site Reliability Team, SRE is “what happens when a software engineer is tasked with what used to be called operations.”[1]

What kind of this ninja trickery is this? Using common sense to make learn how to hire the best people in technology? Why would Google spill the beans on this hiring secret? Maybe they’re sick of dealing with our broken shit.

Our digital systems are ALL distributed and complex now. How can we still expect that having some ignorant code-jockey in a cubicle who never uses what they make control the entire business with the stroke of a keyboard? Because: we are cost-accounting brainwashed and forget that the job to do needs the right experience and skill to do it well. Meanwhile, we keep under-hiring operations and over-hire developers such that there’s a 1-to-who-knows ratio between the people that press one button and the people that press another.

If You’re Offended By What I’ve Described, Congratulations!

I am too. Things that are so complex no one person can understand them, those things are dangerous. Banking apps that aren’t secure, mapping apps that get us lost and late, social media apps that show our kids their first porn, CGM devices that cost more in maintenance fees than their worth…it offends me when these things don’t work. Technology that works is how I make sure I have money for a family, sponsored, biological, or otherwise.

Our tech industry should be hiring people that can comprehend the things they deliver. People pay for things that work. If you don’t care about others, at least you’ll care about making money, and “right” software in a customer-obsessed market makes the most money.

It’s particularly offensive when the hybrid phoenix of a job title that ‘Site Reliability Engineer’ embodies goes largely unnoticed in high tech corporate mindsets. “What the hell is that, your latest professional title advancement scheme? Just because you mashed these words together doesn’t mean you deserve a raise!!!” If you know the following things, you deserve a salary that rivals an enterprise VP of marketing:

  • What your software should do
  • How your software does what it does
  • How to communicate the value of the things you’re working on
  • Don’t mind being woken up when it’s broken for someone
  • Ignore those around you that don’t think the above is relevant to do their jobs

Go forth and make your first salary million in a few years, y’all who can. Do this well and grow.

Why Do We See Site Reliability Engineering on the Rise?

The tech industry is now at the point where we completely forgot that the persons who build software should know how to operate that software when it other people depend on it. Big money, consumer insatiability, customer centricity, and digital transformation has skyrocketed the imperative to make the modern enterprise business engine their engineering teams. We build shiny, complicated, and highly profitable things. What did we expect?

We, the nerds, lured jocks in with our shiny things such as the Altair, BBS, and the entire mobile revolution…and they brought their friends. CFOs, ‘professional CEOs’, and other people that look at a hoodie like its pajamas that violate the corporate dress code. We allowed things to get this way #waterfall #agile #WomenInTech by being egotistical, lazy, impatient, and unkind. These are our chickens coming home to roost.

And now we have to reinvent a way out of the ‘shallow engineering’ tech culture that looks skeptically at #DevOps as a management problem. I don’t mean that everyone on your engineering team has to code, but the people who do code should understand the impact of what they do. This is ethical and this is practical. This is how you make your next billions.

This is the new horizon for impactful, profitable, and scalable tech culture:

On #InternationalWomensDay, I guess that is all for now.

 

FUEL for Your Brain: On Focus, Usefulness, Execution, Learning

I hate acronyms. My dad used to use them far too much, the kind of guy that was more smart in retrospect than the kind of boy I was understood at the time. Kind, thoughtful, quiet, and invested in people around him.

My current thought product is an acronym, “FUEL”, based on a few key practices that I find are valuable to my current line of work as a Change Agent. These practices take time to develop and are only truly useful when used in parallel.

  • Focus: ability to right-size activities to close the task(s) currently underway
  • Usefulness: ability to gauge effectiveness of work and reprioritize based on new ideas/objectives/activities
  • Execution: ability to match skill to task, collaborate the plan, and resolve blockers as they arise
  • Learning: ability to observe outcomes and refactor them into useful ways to improve all of the above

Real-time Example of FUEL

Last night after a meetup, I had a beer with someone I’d met before at a local conference but hadn’t dived into. The opportunity presented itself, so I stayed a little later than I normally would. They are a CTO for a 50-person startup in town. Net-net:

Paul: “What’s weighing on you right now man, work related?” (L)
Them: “Kind of glad someone asked…we have people issues.” (E)
Paul: “You’re not alone…what kind?” (E/F)
Them: “There are a few ‘senior’ engineers that don’t produce like others.” (U)
Paul: “What’s your plan for them and the rest of your team?” (U)
Them: “We just laid off one after giving him a path, but the other two, I don’t know…maybe add metrics, visibility…they’re kind of SPOFs.” (U)
Paul: “…so you can quantify what you already know? How did we arrive here?” (U/L)
Them: “They were here at the beginning, hence ‘senior’, but one guy hasn’t committed code since Sept (5 months)!” (E)
Paul: “Got it, they don’t ‘git’ it. [laughs] How are you and the leadership team  helping to coach other junior engineers?” (L)
Them: “Well that’s the problem maybe. We don’t exactly have a culture yet, but our C-level relationships with each other are solid.” (U/E)
Paul: “I once heard that great leaders define their success by fostering other leaders. Do you know who your real ‘senior’ engineers are?” (F/L)
Them: “Well I kind of already know who deserves the chance to step up.” (E)
Paul: “That’s good, but not enough. People often hesitate on new things simply because they haven’t experienced how it works yet. Your coaching needs to help those people get over any blockers to proving one way or the other if they can do the job well/right/better.” (FUEL)
Them: “I’m going to talk to our CEO about this. Can I get your card?”
Paul: “Only if you intend to use it.”

Good enough exercise and learning for me for one night.

 

Technical Recruitment 101 – Advocates vs. Evangelists

What’s the difference between a technical evangelist and an advocate? What are these terms even? If you’re a technical recruiter, I encourage you to know the answer to all of these questions by reading this piece for the next 4mins.

What is a “Technical Evangelist”?

A way to simplify the definition of an evangelist is to onboard people to their particular topic, do whatever it takes (hackathons, sponsorships, contribution, enablement sessions, flights…lots and lots of flights…blogging, interviews, code samples, research, t-shirt design, customer support, server setup). They wear all the hats, usually at the same time.

What is an “Advocate”?

An advocate defines their job based on the success of the customer (or anyone really) at the job they most need to do. Deep listening, directed dialog, metrics extraction, change impact quantification, being kind, and work framing are all tools in an advocate’s toolbelt. It’s caring about the other person first, then of course yourself secondly.

Advocacy vs. Evangelism

Advocates are good facilitators. Unlike evangelists, they don’t assume that because a hat needs a wearer, that it must be them to wear it. They see the whole field, not just the ball in-front of them. Advocates identify what needs to be done cross-functionally and help to match people who want to and can do a thing to get that done. They think about the problem that people are trying to solve, and put resources into motion to reach that goal.

Naturally, when I frame advocacy this way, it’s easy to see why evangelism leads to burnout. Advocacy leads to faster burndown (e.g. sprint burndown charts) because facilitation, a scalability reinforcer, is at the core of advocacy. What problem are you trying to solve? What’s required to get there? What challenges will arise? How can we most effectively help each other?

How to Qualify Who Should Apply For Which Role?

If someone demonstrates that they can differentiate between facilitating needs over taking on things as a personal responsibility to accomplish, you may just have the potential for an advocate. If a candidate lists off all the things they do without a clear definition of why first, you’re better off funneling them into an evangelist role so they can either learn a better balance or burn out and find something else.

Advocates build plans and often play a significant role in driving those plans to completion. Internal, external, independent, or consultative. When they do take ownership over a goal, you know that it will get closed, even if it wasn’t achieved to expectation, it will get closed and retro’d.

In Practice, I Say “Advocacy over Evangelism”

Evangelists talk about their product and rarely take full responsibility for anything. They are often driven by others to do things, go places, speak under sponsored time, build samples, and be engaged with customers. It’s right in the job title. “Evangelism” in Latin can be translated as “messenger”. In other words, they have been told to deliver a message. Usually the messenger has nothing to do with the crafting process of the message, and would you want someone whose job is to talk rather than listen involved in what your message to people should say? I wouldn’t.

Put another way, an evangelist mindset attempts to frame the problems of the world into terms that their product can solve. Their focus is the product that they’re incentivized to proliferate. If it doesn’t quite fit a customer’s situation, too bad, we’ll make it fit. I don’t know the job you’re trying to accomplish because I didn’t bother to ask, but I’m going to offer you a random solution anyway. And naturally, introducing a wrong-fit solution produces negative consequences (like lost time, lost money, and lost opportunity).

An advocacy mindset exfoliates information required to make decisions about how to best accomplish a customer’s true goal. Your product is not your software. Your product is a right-fit of people, process, and technology for a customer to successfully accomplish their goal. Your overlap to their situation in strictly wheelhouse terms might only by 10% or just one specific job. But if you can understand how important (or not) what you do is to completing the job they have at hand, then you can quickly fit what you offer to their needs.

This is the heart of the way product teams should “go to market”. They proceed as explorers and caretakers, not “disruptors” simply because it sounds cool. In this world, caring about how to help someone with their job better every day is disruptive. It’s honestly disruptive, to sales, to marketing, to product management, and to vision holders in technology.