Welcome!

@CloudExpo Authors: Pat Romanski, Elizabeth White, Yeshim Deniz, Liz McMillan, Aruna Ravichandran

Related Topics: @CloudExpo, Open Source Cloud

@CloudExpo: Blog Feed Post

Designing Startup Metrics to Drive Successful Behavior

The Secret to Success

Great companies are almost always run by great management teams. And great management teams know that the only way to improve a process is to start by measuring it. Good metrics should also be actionable, and drive successful behavior. In this post I hope to help show how to figure out which metrics matter the most, and how to design them in such a way as to drive behavior that will lead to the results that you want.

This post is applicable to any kind of business. In a follow up post, I will use this technique to walk through the design of a set of metrics for a SaaS company. Since SaaS businesses (or any other subscription-based business) are different from standard software businesses, there are some interesting elements that we will uncover.

image

Think of your company as a machine
One way to look at how companies work is to imagine them as a machine that has Outputs, and Levers that you, the management team, can pull to affect it’s behavior.

Weak management teams have only a limited understanding of how their machines work, and what levers are available to affect performance. The better the management team, the better they will understand how that machine works, and how they can optimize its performance (what levers they can pull).

When we look to design metrics, we are looking to deepen our understanding of the machinery, and how it works. Well designed metrics will automatically drive behavior to optimize output from the machine.

Example of a bad Board Meeting
Here is an example of a bad board meeting, which happens far more frequently than you might imagine. The company has just missed its quarterly revenue forecast. Good board members want to know two things:

  • Why that happened?
  • What can be done to avoid the problem going forward?

As they ask management what happened, a common answer will be that the market was really tough, and deals just didn’t close the way that they hoped. They also don’t have a great plan for what they are going to do differently next quarter, other than hope that the market improves, and that more deals will close. There is a great saying for situations like this: Hope is not a strategy.

Example of a good Board Meeting
The better management teams answer those questions differently. They will gradually peel back the covers of the machine, like peeling the layers of an onion, and expose the true nature of the problem, which of course will also highlight what levers need to be pulled to fix the problem. Lets take an example, and look at how they might do this:

    1. They will be able to tell you that revenue is composed of deals. To compute revenue, you multiply average deal size by number of deals. They may tell you that they were targeting to grow their average deal size to $x, and were successful in hitting this target. But the number of deals that they closed was below target.
    2. They will then peel back the onion one more layer, and tell you that the reason that the number of deals was below target was because 1/3rd of the salesforce missed their targets.
    3. They will then peel back another layer, and tell you that the reason those salespeople missed their targets was because they were not handed the required number of trials from marketing. However, for the trials that they did receive they were successful at converting them to closed deals at the expected conversion rate. So we know from this that the problem is not the quality of those sales people.
    4. Peeling back another layer, they will tell you that the number of trials is equal to the visitors to the site x the conversion rate of those visitors to trials. They may tell you that the number of visitors was on target, but the conversion rate fell below the previous levels.
    5. Peeling back one more level, they may tell you that they ran three major campaigns to drive visitors to the site, as well as relying on the normal levels of word of mouth traffic.  They may then reveal the true source of the problem: the ads that they had started running on Facebook were delivering a far lower conversion rate to trials than in prior months.

The contrast between the two approaches is stark. In the second case, it is clear that management will know how to fix the problem (by adding new traffic generation programs). They also know precisely how much additional traffic will need to be generated to reach the growth targets, and how many sales people are needed at a given productivity level, etc. etc.

What is surprising is just how few management teams really have their act in order in this area. For Web and SaaS businesses with smaller transactions at higher volumes, this kind of modeling and tracking is much easier, as web-based lead generation and marketing have easy to implement measurements, and the greater the volume of transactions, the more clearly patterns emerge. This is a little harder to do for channel sales, but still extremely valuable. And a little harder than that for direct sales situations with large deal sizes.

The Secret to Success
The secret to successful design of metrics is to start with the end goal and work backwards. In most companies, the end goals that matter the most are:

  • Profit/(Loss)
  • Growth
  • Good cash flow

(You may wonder why we don’t have Revenue in this list, but read further, and and it will soon become clear.)

Let’s take the first of these, Profitability, and work backwards. Working backwards means looking at the components that make up Profitability:

Profits (EBITDA) = Revenue – Cost of Goods Sold – Expenses

So to focus the management team on driving profitability, we should also track and measure Revenue, CoGS, and Expenses. Obvious, isn’t it? Well the good news is that this same principle can be applied over and over again focusing on the components of Revenue, CoGS, and Expenses where needed.

So the next step is to take Revenue, CoGS, and Expenses, and break them down to the key components. Bookings is the pre-cursor to Revenue. So let’s look at Bookings as an example:

Bookings =No of deals closed * Average Deal Size

For Reseller Channels, we might be looking at something different like this:

Revenue = No of productive resellers * average productivity per reseller

(Note: in many businesses there are several categories of deals. e.g. there could be large deals, and smaller deals. Or their could be deals from two or more different categories of customers. So the formula may have more elements to it than shown above.)

Peeling back another level, we might find the following:

No of deals closed = No of productive sales people * Average Productivity per Sales person

There will also likely be another formula to compute this, which will look like the following:

No of deals closed = No of Trials * Average Conversion Rate

These two formulae clearly indicate some of the levers that we have available to increase Bookings. We can grow the number of trials, or grow the number productive sales people, or we could try to increase the average productivity of our sales people. However we need to make sure that we grow them both together, otherwise we could end up out of balance, and have too many sales people and not enough trials to feed them, or too many trials and not enough productive sales people to close them.

The next step would be to peel back the onion a few more layers:

No of trials = No of visitors to the web site * Average Conversion Rate to Trials

No of Visitors to the web site = Normal traffic + for each traffic generation campaign: target audience of each campaign * Conversion Rate to visitors

Each time we peel back a layer to expose the components, we gain a better understanding of our machine and the levers that we can pull to make it work better. For example:

In the above two formulae, we can see that a big driver of the model is visitors to the web site. But this can be expensive to increase. So the other variable that we can try to increase is the conversion rate for each campaign, and the conversion rate to trials. We can try to do this by altering campaign messaging and landing pages and using A/B testing to find the optimum creative content.

We might also decide to focus our efforts on increasing the average deal size. We could do this in several ways:

  • Cross sell to add additional products
  • Up sell to add seats, or premium features
  • Develop a scalable pricing matrix that does a better job of charging higher end customers that are willing to pay more. This might involve several new axes that increase pricing, such as charging per seat, or charging per 1,000 data elements tracked, or charging for 24×7 support, etc.

As with many good ideas in business, all of the ideas above are obvious, and follow common sense.  However, you would be shocked to discover how rare it is to actually see businesses that have fully peeled back the onion to expose all the major variables and levers, and then implemented appropriate metrics to track these over time.

Trend based analysis
For every major variable that matters in our model, we will want to track how this varies over time. This will show us if we are succeeding in our efforts to improve things, and also give us early warning signs of any negative trends.

For most stages in a sales and marketing pipeline, we will want to track two metrics: how many prospects we put through that stage, and how effective were we at converting them to the next stage.Peeling back the Onion on Inside Sales performance

Another area where metrics can be extremely useful is in managing an inside sales (telesales) organization. Starting with the overall sales number achieved by the whole group, let’s peel this back layer by layer, to see what we can learn:

  • Overall group performance = Sum (individual contributor performance).

Not surprisingly we need to look at how each individual has done relative to the average levels to understand the strong performers, and the weak performers.

  • Individual performance = No of deals closed * Average Deal Size

For the weak performers, it is likely that the number of deals closed will be lower than we want. The question is why? So what are the components that make up the number of deals that an individual closes? Assuming a sales process where each inside sales person is handed a queue of marketing qualified leads, and then calls these to try to schedule a demo, and the post the demo tries to close a sale, the components will be:

  1. Calls made per sales person (if this is low, they will quickly react to peer pressure when they see other sales people’s call rates)
  2. Conversion rate to returned calls. (If this is low, it means the sales person is not leaving compelling voicemails, and should be given training by someone that has a high conversion rate.)
  3. Conversion rate from phone calls to Demos. (If this is low, it means the sales person’s ability to convey the value proposition is weak, and they should be given training by someone with high conversion rates.)
  4. Conversion rate from Demos to Closed Deals. (If this is low, it means the sales person needs better demo training.)
  5. Average Deal Size. (If this is low, it could mean the sales person needs better training on cross selling, or up selling.)

The above may not mirror your inside sales process, but hopefully the method of working backwards from the end goal, and peeling back the layers to expose the components will enable you to map out the metrics that matter to you.

Sales and marketing funnel – summary metrics
We will also want to look at some metrics that cover the entire sales and marketing funnel from top to bottom. Here are some example metrics that are important at this overall level:

Lead source effectiveness:

  • CAC by lead source
  • ROI by lead source (takes into consideration cost, conversion rates to closed deals, and lifetime value of customers that came through that particular lead source)

What not to track
Some categories like Expenses are made up of many line items, and we very likely don’t want to bother with metrics for every line item, we need to answer the question: How deep should we go with our analysis? The answer to this is pretty much common sense:

  • Prioritize the components that have the biggest effect
  • Don’t put much effort into tracking things that you can’t affect
  • Don’t bother tracking items that are small, or that don’t vary much. Leave these to accounting.

Conclusions
There is nothing in this article that should be surprising or earth shattering. It is all obvious. However, as is often the case in business, it is really easy to have the vision of what to do, but far harder to execute on that vision. In my experience the mark of a really well run business is that they actually have the systems in place to automatically produce these metrics. And they use those metrics as part of the management process to run the business.

The Benefits: Good Metrics drive Actions and Behavior
One of the greatest things about putting in place the right metrics is that showing them to people will automatically change their behavior to try to improve the metrics.  Furthermore, the metrics make it clear what levers they can use to change performance.

Well designed metrics make it clear what actions are needed to hit plan
Working backwards from a specific Revenue target, management will be able to understand all the other elements that have to be put in place to reach that target. For example, if you want to hit $xm in bookings for the quarter, you can work out:

  • How many sales people are required
  • How many leads are required to feed those sales people
  • What marketing campaign spend is needed to generate those leads

If you are in a channel model, you can work out how many productive resellers are required, and given a known conversion rate from newly signed resellers going through an on-boarding process, you will be able to work out how many new resellers are required, and how many on-boarding sales training sessions need to be run. Etc.

Acknowledgements
I have had the very good fortune to work with some excellent management teams that have helped teach me these lessons. In particular, I would like to thank the teams at HubSpot and JBoss who were very advanced in their use of metrics.

Follow up blog post
Watch out for my follow up blog post on SaaS Metrics and Levers to see what happens when I drill down on the key metrics for a SaaS business. (This is applicable to other subscription businesses such as Open Source.)

Read the original blog entry...

More Stories By David Skok

David Skok joined Matrix Partners as a General Partner in May 2001. He has a wealth of experience running companies. He started his first company in 1977 at age 22. Since then he has founded a total of four separate companies and performed one turn-around. Three of these companies went public.

Skok joined Matrix from SilverStream Software, which he founded in June 1996. Prior to its July 2002 acquisition by Novell, SilverStream was a public company that had reached a revenue run rate in excess of $100M, with approximately 800 employees and offices in more than 20 countries around the world. His work as a value added investor is best known for helping JBoss take its Open Source business to a successful exit with its sale to Red Hat, and for helping AppIQ, Tabblo and Diligent Technologies, which have all had successful exits, from their inceptions to their acquisitions by HP and IBM.

He serves on the boards of Digium (makers of the very popular Asterisk Open Source PBX/telephony software), CloudSwitch, Enservio, OpenSpan, Solidworks, VideoIQ, and HubSpot. In addition to his broad focus on enterprise software, he is specifically focused on the areas of cloud computing, Open Source, Software as a Service (SaaS), marketing automation, virtualization, storage, and data center automation.

@CloudExpo Stories
SYS-CON Events announced today that N3N will exhibit at SYS-CON's @ThingsExpo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. N3N’s solutions increase the effectiveness of operations and control centers, increase the value of IoT investments, and facilitate real-time operational decision making. N3N enables operations teams with a four dimensional digital “big board” that consolidates real-time live video feeds alongside IoT sensor data a...
In his session at 21st Cloud Expo, Raju Shreewastava, founder of Big Data Trunk, will provide a fun and simple way to introduce Machine Leaning to anyone and everyone. Together we will solve a machine learning problem and find an easy way to be able to do machine learning without even coding. Raju Shreewastava is the founder of Big Data Trunk (www.BigDataTrunk.com), a Big Data Training and consulting firm with offices in the United States. He previously led the data warehouse/business intellige...
In a recent survey, Sumo Logic surveyed 1,500 customers who employ cloud services such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). According to the survey, a quarter of the respondents have already deployed Docker containers and nearly as many (23 percent) are employing the AWS Lambda serverless computing framework. It’s clear: serverless is here to stay. The adoption does come with some needed changes, within both application development and operations. Tha...
SYS-CON Events announced today that Avere Systems, a leading provider of enterprise storage for the hybrid cloud, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Avere delivers a more modern architectural approach to storage that doesn't require the overprovisioning of storage capacity to achieve performance, overspending on expensive storage media for inactive data or the overbui...
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Data scientists must access high-performance computing resources across a wide-area network. To achieve cloud-based HPC visualization, researchers must transfer datasets and visualization results efficiently. HPC clusters now compute GPU-accelerated visualization in the cloud cluster. To efficiently display results remotely, a high-performance, low-latency protocol transfers the display from the cluster to a remote desktop. Further, tools to easily mount remote datasets and efficiently transfer...
Digital transformation is changing the face of business. The IDC predicts that enterprises will commit to a massive new scale of digital transformation, to stake out leadership positions in the "digital transformation economy." Accordingly, attendees at the upcoming Cloud Expo | @ThingsExpo at the Santa Clara Convention Center in Santa Clara, CA, Oct 31-Nov 2, will find fresh new content in a new track called Enterprise Cloud & Digital Transformation.
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
Though cloud is the future of enterprise computing, a smooth transition of legacy applications and systems is critical for seamless business operations. IT professionals are eager to start leveraging the cost, scale and other benefits of cloud, but with massive investments already in place in existing infrastructure and a number of compliance and resource hurdles, it can be challenging to move to a cloud-based infrastructure.
Most technology leaders, contemporary and from the hardware era, are reshaping their businesses to do software. They hope to capture value from emerging technologies such as IoT, SDN, and AI. Ultimately, irrespective of the vertical, it is about deriving value from independent software applications participating in an ecosystem as one comprehensive solution. In his session at @ThingsExpo, Kausik Sridhar, founder and CTO of Pulzze Systems, will discuss how given the magnitude of today's applicati...
Smart cities have the potential to change our lives at so many levels for citizens: less pollution, reduced parking obstacles, better health, education and more energy savings. Real-time data streaming and the Internet of Things (IoT) possess the power to turn this vision into a reality. However, most organizations today are building their data infrastructure to focus solely on addressing immediate business needs vs. a platform capable of quickly adapting emerging technologies to address future ...
Amazon is pursuing new markets and disrupting industries at an incredible pace. Almost every industry seems to be in its crosshairs. Companies and industries that once thought they were safe are now worried about being “Amazoned.”. The new watch word should be “Be afraid. Be very afraid.” In his session 21st Cloud Expo, Chris Kocher, a co-founder of Grey Heron, will address questions such as: What new areas is Amazon disrupting? How are they doing this? Where are they likely to go? What are th...
SYS-CON Events announced today that NetApp has been named “Bronze Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. NetApp is the data authority for hybrid cloud. NetApp provides a full range of hybrid cloud data services that simplify management of applications and data across cloud and on-premises environments to accelerate digital transformation. Together with their partners, NetApp emp...
The dynamic nature of the cloud means that change is a constant when it comes to modern cloud-based infrastructure. Delivering modern applications to end users, therefore, is a constantly shifting challenge. Delivery automation helps IT Ops teams ensure that apps are providing an optimal end user experience over hybrid-cloud and multi-cloud environments, no matter what the current state of the infrastructure is. To employ a delivery automation strategy that reflects your business rules, making r...
Join IBM November 1 at 21st Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Cognitive analysis impacts today’s systems with unparalleled ability that were previously available only to manned, back-end operations. Thanks to cloud processing, IBM Watson can bring cognitive services and AI to intelligent, unmanned systems. Imagine a robot vacuum that becomes your personal assistant th...
SYS-CON Events announced today that SkyScale will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. SkyScale is a world-class provider of cloud-based, ultra-fast multi-GPU hardware platforms for lease to customers desiring the fastest performance available as a service anywhere in the world. SkyScale builds, configures, and manages dedicated systems strategically located in maximum-security...
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, will go over the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, applicatio...
Enterprises are adopting Kubernetes to accelerate the development and the delivery of cloud-native applications. However, sharing a Kubernetes cluster between members of the same team can be challenging. And, sharing clusters across multiple teams is even harder. Kubernetes offers several constructs to help implement segmentation and isolation. However, these primitives can be complex to understand and apply. As a result, it’s becoming common for enterprises to end up with several clusters. Thi...
Microsoft Azure Container Services can be used for container deployment in a variety of ways including support for Orchestrators like Kubernetes, Docker Swarm and Mesos. However, the abstraction for app development that support application self-healing, scaling and so on may not be at the right level. Helm and Draft makes this a lot easier. In this primarily demo-driven session at @DevOpsSummit at 21st Cloud Expo, Raghavan "Rags" Srinivas, a Cloud Solutions Architect/Evangelist at Microsoft, wi...
Containers are rapidly finding their way into enterprise data centers, but change is difficult. How do enterprises transform their architecture with technologies like containers without losing the reliable components of their current solutions? In his session at @DevOpsSummit at 21st Cloud Expo, Tony Campbell, Director, Educational Services at CoreOS, will explore the challenges organizations are facing today as they move to containers and go over how Kubernetes applications can deploy with lega...