Welcome!

@CloudExpo Authors: Elizabeth White, Rene Buest, Liz McMillan, Mehdi Daoudi, Astadia CloudGPS

Related Topics: @CloudExpo, SDN Journal

@CloudExpo: Blog Post

Training Wheels and Protective Gear By @PlexxiInc | @CloudExpo [#SDN]

This balancing act is part of what as made networking as complex as it has become

Throughout the development cycle of new features and functions for any network platform (or probably most other products not targeted at the mass market consumer) this one question will always come up: should we protect the user of our product from doing this? And “this” is always something that would allow the user of the product to really mess things up if not done right. As a product management organization you almost have to take a philosophical stand when it comes to these questions.

Protect the user
Sure enough, the question came up last week as part of the development of one our features. When putting the finishing touches on a feature that allows very direct control over some of the fundamental portions of what creates a Plexxi fabric, our QA team (very appropriately) raised the concern: if the user does this, bad things can happen, should we not allow the user to change this portion of the feature?

This balancing act is part of what as made networking as complex as it has become. As an industry we have been extremely flexible in what we have exposed to our users. We have given access to portions of our products that 99.9% of customers will never need, but unfortunately because of that 0.1% every networking product has tons of these little tweaks and knobs that could wreak havoc if used the wrong way.

We take a lot of pride in creating a network solution that is simple to use, simple to interact with, but extremely powerful under the hood. Direct access to all that power will lead to not only giving the customer a powerful weapon, but also the ammunition to use it. And like handling any weapon, you can really hurt yourself if you are not careful. Which comes back to the question at hand, how many safety valves do you put in place to make sure the user cannot hurt themselves?

The reason why
Some of these controls are buried fairly deep inside our products. They are meant for true power users and for the support teams of the vendors. And even beyond the support teams, there are tools and tricks inside our products that only the engineering teams know about, hidden even beyond the knowledge of support teams. Several years ago (in a previous job), we had a customer with a complex problem. Traffic was inconsistently forwarded and the belief was that there were communication problems between line cards and the main CPU card that would create inconsistant tables (the biggest challenge for any chassis based system).

Of course our development teams had tools embedded in the code to carefully examine and manipulate the tables and communications between these cards. Not exposed to a regular user, because they were potentially dangerous. And we proved that they were. During the execution of the command by one of my developers, he made a small typo in one of the arguments and boom went the switch. Crash and reboot. Customer very upset (for good reason, this was a production network), executive management very upset (also for good reason) and worse, the problem disappeared without us collecting the information we needed to attempt to fix it.

Different Answers for Different Tools
There is a difference between debug tools that allow engineers to look deep inside the switch versus common features that may have significant service consequences if not in expert hands. No matter how hard we try, the first category will continue to exist. As vendors we will bring portions of these tools to the user or support visible spectrum, but at the same time we will create new ones buried deep.

The latter category though is one where I favor a less protective approach. There are many ways by which you can completely disrupt your network service. Most of the services your network provide have been created with your own hands through provisioning and configuration and can therefore be disrupted by those same actions. When we create features and functions that are potentially dangerous, it is on us the vendor to make sure it is properly documented and explained. This way when you do make that mistake (and it will happen) we can refer to that 4 letter “read the documentation” response.

Off come the training wheels
When it comes to user configurable features and functions, every single one of them has the potential to disrupt service when used the wrong way. We as vendors should not shy away from giving you all the tools you need to create (and destroy) the service you need. And I do not believe anyone wants to step through one “Are you sure (Y/N)?” after another. Of course we need to make creating services easier for you. If you are a frequent reader of our blogs you know that is what we stand for. But we should not take away features because we are afraid you can shoot yourself in the foot. Any time in the past where we opted to give you a gun but keep the bullets behind a locked door, we have found someone that legitimately explained that he or she needed the bullets to solve their specific problem. And we unlocked the door.

There are ways to teach someone how to ride a bike without providing permanent training wheels. Documentation (for those few that read it), workflow based provisioning and configuration and solid default behaviors with predictable results can steer you clear of the dangers we have provided. And when you do fall off the bike and hurt your knee or elbow, well, you are less likely to try that maneuver again next time. That is how most of us learn. Including those developers that crash a customer production switch during a debug session. For every one of those “oops” moments there will be many where those hidden gems may have saved your network from disaster. Just like there is one customer for whom having the bullets makes the difference between a working service and one that just limps along.

[Today's fun fact: You burn more calories sleeping than watching TV. I enjoy combining the two, especially during some of the last few Thursday night NFL games.]

The post Training Wheels and Protective Gear appeared first on Plexxi.

Read the original blog entry...

More Stories By Marten Terpstra

Marten Terpstra is a Product Management Director at Plexxi Inc. Marten has extensive knowledge of the architecture, design, deployment and management of enterprise and carrier networks.

@CloudExpo Stories
"We're here to tell the world about our cloud-scale infrastructure that we have at Juniper combined with the world-class security that we put into the cloud," explained Lisa Guess, VP of Systems Engineering at Juniper Networks, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Your homes and cars can be automated and self-serviced. Why can't your storage? From simply asking questions to analyze and troubleshoot your infrastructure, to provisioning storage with snapshots, recovery and replication, your wildest sci-fi dream has come true. In his session at @DevOpsSummit at 20th Cloud Expo, Dan Florea, Director of Product Management at Tintri, provided a ChatOps demo where you can talk to your storage and manage it from anywhere, through Slack and similar services with...
As enterprise cloud becomes the norm, businesses and government programs must address compounded regulatory compliance related to data privacy and information protection. The most recent, Controlled Unclassified Information and the EU’s GDPR have board level implications and companies still struggle with demonstrating due diligence. Developers and DevOps leaders, as part of the pre-planning process and the associated supply chain, could benefit from updating their code libraries and design by in...
"Peak 10 is a hybrid infrastructure provider across the nation. We are in the thick of things when it comes to hybrid IT," explained Michael Fuhrman, Chief Technology Officer at Peak 10, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Calligo, an innovative cloud service provider offering mid-sized companies the highest levels of data privacy and security, has been named "Bronze Sponsor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Calligo offers unparalleled application performance guarantees, commercial flexibility and a personalised support service from its globally located cloud plat...
"We are an IT services solution provider and we sell software to support those solutions. Our focus and key areas are around security, enterprise monitoring, and continuous delivery optimization," noted John Balsavage, President of A&I Solutions, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We were founded in 2003 and the way we were founded was about good backup and good disaster recovery for our clients, and for the last 20 years we've been pretty consistent with that," noted Marc Malafronte, Territory Manager at StorageCraft, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
There is a huge demand for responsive, real-time mobile and web experiences, but current architectural patterns do not easily accommodate applications that respond to events in real time. Common solutions using message queues or HTTP long-polling quickly lead to resiliency, scalability and development velocity challenges. In his session at 21st Cloud Expo, Ryland Degnan, a Senior Software Engineer on the Netflix Edge Platform team, will discuss how by leveraging a reactive stream-based protocol,...
"We are focused on SAP running in the clouds, to make this super easy because we believe in the tremendous value of those powerful worlds - SAP and the cloud," explained Frank Stienhans, CTO of Ocean9, Inc., in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"DivvyCloud as a company set out to help customers automate solutions to the most common cloud problems," noted Jeremy Snyder, VP of Business Development at DivvyCloud, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
"As we've gone out into the public cloud we've seen that over time we may have lost a few things - we've lost control, we've given up cost to a certain extent, and then security, flexibility," explained Steve Conner, VP of Sales at Cloudistics,in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
Internet of @ThingsExpo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The Internet of Things (IoT) is the most profound change in personal and enterprise IT since the creation of the Worldwide Web more than 20 years ago. All major researchers estimate there will be tens of billions devic...
"The Striim platform is a full end-to-end streaming integration and analytics platform that is middleware that covers a lot of different use cases," explained Steve Wilkes, Founder and CTO at Striim, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
"We want to show that our solution is far less expensive with a much better total cost of ownership so we announced several key features. One is called geo-distributed erasure coding, another is support for KVM and we introduced a new capability called Multi-Part," explained Tim Desai, Senior Product Marketing Manager at Hitachi Data Systems, in this SYS-CON.tv interview at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
21st International Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Me...
SYS-CON Events announced today that DXWorldExpo has been named “Global Sponsor” of SYS-CON's 21st International Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Digital Transformation is the key issue driving the global enterprise IT business. Digital Transformation is most prominent among Global 2000 enterprises and government institutions.
SYS-CON Events announced today that Datera, that offers a radically new data management architecture, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Datera is transforming the traditional datacenter model through modern cloud simplicity. The technology industry is at another major inflection point. The rise of mobile, the Internet of Things, data storage and Big...
Kubernetes is an open source system for automating deployment, scaling, and management of containerized applications. Kubernetes was originally built by Google, leveraging years of experience with managing container workloads, and is now a Cloud Native Compute Foundation (CNCF) project. Kubernetes has been widely adopted by the community, supported on all major public and private cloud providers, and is gaining rapid adoption in enterprises. However, Kubernetes may seem intimidating and complex ...