Holiday IoT and the Performance Imperative

A few words to manufacturers and vendors of tech toys: to really be ready for the holiday, if your product requires software updates in order to work or is in any way internet connected, make sure your site stays up. Otherwise, you just shipped coal.

  • Provide more than one distribution point for downloadable updates/binaries
  • Rely on CDNs for static assets (like installers and documentation)
  • If your update process must rely on live services, make sure they’re scalable
    • Load test subcomponents/microservices AND the end-to-end process
  • Be prepared for damage control by:
    • Monitoring site uptime and availability to know when things are broken
    • Proactively establish a communication channel with customers during issues
    • Properly staff IT and support for during AND post-season issues
  • Make sure the cost of downtime is factored into your next sales cycle

Santa Brought Us a Brick?

Let me start by admitting how 1st-world this example is. Robots as play-things are still not exactly ‘so easy, a child could do it’, and Roomba’s have been around for almost two decades, but we still have yet to see a really down-to-earth home robotics project that really works for children under 10. I don’t just mean toys that are not pre-assembled, even the right-out-of-the-box kind often require firmware updates or online services to really work as expected.

Case in point, the Meccano MAX. Though it only took about 3hrs total to put together, this morning when we finally turned it on and went to connect for a firmware update, the vendor’s website was down…hard. The instructions said, before anything else, update the ‘MeccaMind’ and voice commands weren’t working without, so, blocker.

As an ops nerd, I slapped an uptime monitor on it to know when (if ever) it was back up:

That didn’t stop the whole multi-day experience from deflating to a dud. We all worked on this thing together and then before it can do anything, we are stuck guessing about when we can actually enjoy it. Don’t blame Santa, blame the geniuses at Meccano.

Performance, of your product, of your service, of your site, is imperative to delivering what you sold people. Availability, uptime, scalability, and reliability matter by default now. Everyone has downtime, but 4hr recovery time on your corporate domain isn’t just irresponsible and costly, it’s plain embarrassing…and transparent.

Why Is This Even Important?

My 7yr old is crazy into coding right now. Granted, we use a visual Code Block Editor mostly for lights and tones, but it’s a great way to introduce concepts like flow control and formal logic. As soon as she saw an example I built that used function blocks to encapsulate and reuse logic, she instantly understood and started refactoring her programs.

But finding the right project for varying stages of aptitude, appetite, and enjoyment is a real challenge. No thanks to marketing, but also unanticipated road-blocks like service and subscription dependencies are hard for consumers to factor in when purchasing. Even when you do find a right-fit project, if bone-headed problems like website downtime occur, it can become a negative experience for the child (or student).

It’s important for STEM product manufacturers and software vendors to really think about the impact of what they’re selling, how they’re delivering it, and how to support people who paid them money for something to accomplish a goal. If you don’t have the optimal consumer’s experience in mind, it will eventually cost you.

Need the Robot Software Updater?

I can’t archive everything on their site, and it’s their job to provide reliable content distribution, but in case you find yourself stuck like I was, here are links to at least the firmware updater tool:

Also, never ask a consumer if they want to choose (null).

And if the updater gives you flashbacks to DirectX drivers from 1997, don’t worry. It only looked like it bricked my robot for about 4mins before providing UI feedback:


Once you do get back to the modern era, a 98.6MB mobile app to control it shouldn’t be too hard on your data plan. They also need to know your GPS location, phone contacts, and file storage for some reason.

Value Chain in DevOps

Foreward: Since I highly doubt the following concepts will see the light of day in the final draft of IEEE 2675, I wanted to document that in fact I pushed this to the group June 15th 2017. During a subsequent review, it got huge push-back from our Curator in Chief, the early first of a future string of events that lead me to publish this primary work on my personal blog.

What is a ‘value chain’?

value chain is a set of activities that a firm operating in a specific industry performs in order to deliver a valuable product or service for the market. The concept comes through business management and was first described by Michael Porter in his 1985 publication, Competitive Advantage: Creating and Sustaining Superior Performance.[1]

The idea of the value chain is based on the process view of organizations, the idea of seeing a manufacturing (or service) organization as a system, made up of subsystems each with inputs, transformation processes and outputs. Inputs, transformation processes, and outputs involve the acquisition and consumption of resources – money, labor, materials, equipment, buildings, land, administration and management. How value chain activities are carried out determines costs and affects profits.— IfM, Cambridge[2]

[Wikipedia: Value Chain (Porter)]

How does it related to IEEE 2675?

As related to DevOps, the Value Chain for Software Delivery is an application of a lifecycle perspective that scopes standards adherence to only certain individuals based on their participation in primary or supporting activities related to a particular software product.

DevOps is about continuously delivering value to users/customers/consumers. A value chain perspective disaggregates activities from an organizational funnel, making it easier for teams and consumers of a particular product or service to ask “does this thing meet the standard” without unintentionally including other unrelated teams or products, were the question be phrased as “does this organization meet the standard”.

What problem are we solving by using ‘value chain’?

Adoption. In large organizations with many independent groups or product teams, DevOps principals may apply across the value chain of one product, but not another, provided these products are completely independent from one another. For an organization or team to claim that a particular software product adheres to this standard, all aspects of that product’s value chain must implement the principals and practices set forth in this standard.

Examples of where the broadness of using “organization” presents a challenge to adoption:

  • Consulting agencies with many independent project teams on working on separate contracts/products
    • If only one of those contacts require adherence to 2675, does this require every team/contract (both in the future and retroactively) to do the same?
    • Using “value chain” would scope 2675 adherence to any and all parties performing activities germane to delivering that specific contract/product
    • We know that if even one team implements DevOps per 2675 and sees success, organizations are likely to grow that out to other teams over time; “value chain” helps adoption of the standard. 

  • Enterprises in the midst of transformation to DevOps
    • Can they claim adherence on a specific product if the “whole organization” can’t yet?
    • Much like the above agencies argument, when the scope of adherence is based on activities relating to delivery of a project, enterprises are far more capable of becoming “DevOps ready” because they can grow the practice out *over time*
    • In DevOps, key success factors are determined – driven – by customers, the end user.

  • Organization’s requirements from non-technical internal agencies
    • Can we expect that the legal or HR departments are also “DevOps”? 
      This would of course need to be defined by activities that support the delivery side of the business (i.e. billing, purchasing, etc.), begetting an activities-based perspective on implementation over organizational labeling

Why is this of importance to IEEE 2675?

In layman’s terms, adoption of IEEE 2675 at an organizational level can’t happen overnight, especially in large enterprises with many teams. Fortunately, it doesn’t have to, provided we adequately scope ‘shall’ statements with a perspective that A) is reasonable in scope and impact on the org, and B) enables parties to agree on what it means for a software product to have been developed and delivered using this standard.

How and where would we use ‘value chain’?

Places in the text where use of the word ‘organization’ could infer that IEEE 2675 must be implemented across the whole organization before any single team or product could claim adherence to the standard. For instance:

  • Too broad:

    “Organizations shall implement effective continuous delivery procedures aligned with the architecture definition in a manner that meets the business needs of the system.” … “DevOps itself requires effective architecture across the organization to ensure that application build, package and deployment procedures are implemented in a robust and consistent manner.” (6.4.4)

  • Scoped: 

    “Organizations shall implement effective continuous delivery procedures within a particular value chain that are aligned with the architecture definition and in a manner that meets the business needs of the system.” … “DevOps itself requires effective architecture across the whole value chain to ensure that application build, package and deployment procedures are implemented in a robust and consistent manner.”

  • Too broad: 

    “Organizations shall maintain an accurate record of both code deployed and fully automated mechanisms in order to remove obsolete code.”

  • Scoped:

    “Organizations shall maintain an accurate record of both code deployed and fully automated mechanisms in a value chain in order to remove obsolete code.”

Additional references:

Wanting vs. Having for My 7yr Old

I make a lot of mistakes. I try not to do it on Github where commits are a permanent record of your competencies. So are children, only the impact of a word said harshly or missed opportunity for positive reinforcement take immediate effect and often have lasting effects on their lives.

Taking my son and daughter out one-on-one to get gifts for the people in their lives is important to me, and despite sometimes feeling like it in the moment, is never a mistake in the ideal sense. Having a positive experience doing so is also part of a successful outing, because if it’s a negative experience, the whole thing is useless.

A few weekends ago, I went out with my daughter. When we finally ended up at the toy store to get something for her brother, naturally she found something for herself. Because she’s seven, and no matter how zoned in of a quick pow-pow we had outside before entering about who she’s here to shop for, she still looks at the world with her-eyes. Arguably we all do this, regardless of our age, defaulting to looking out for number one in all manner of situations. That day, it was a cheap, pink, plastic set of glam rings.

I said no, but I’ll think about it, and asked her to put them back and focus on her brother’s gift. What a mistake. She didn’t throw a fit or get weepy or shut down. I took a picture (for when I would come back in search of stocking stuffers) and moved on. The Big Day is a week away, and I realize there was a missed opportunity there and I have to find a moment with her when she gets them in her stocking to make this memory-point:

In life, the moment you want a thing and the moment you get it are often different.

If I had bought her that thing, what next? Should she have it right then, as inevitably she would. This is just kicking the ‘delayed gratification’ can down the road. And in juxtaposition with the notion of immediate positive reinforcements, her want wasn’t connected to a prior goal or reward plan. So no, I don’t just buy things because my kids want them, even when I can. It’s not just the principle that matters, my choices (a.k.a. commits to their codebase) have an immediate impact. The manner in which I roll those decisions out matters as much as the very moment of desire in their hearts.

So what will I do with this new hypothesis? As with all hypotheses, I will test it. On the morning of the Big Day, after all the things are unwrapped and people chill the fark back down and people have 2nd coffees and there’s bandwidth for sharing that learning, I will say to her:

“Love, I too seek the desires of your heart and will always support you in them. In life, the moment you want a thing and the moment you get it are often different times. Remember this thing, what you wanted weeks ago? Well I remembered and it’s been waiting for you the whole time. But there’s a time and a place, which is today! Trust in me, one who has gone before, that I can help you learn to deal with the waiting time between when you desire something and when you achieve it. Remember that my love for you is something you don’t always know how to see, but never doubt that it will always be there.”