Welcome!

@CloudExpo Authors: Pat Romanski, Tim Hinds, Xenia von Wedel, William Schmarzo, Elizabeth White

Related Topics: @CloudExpo, Open Source Cloud

@CloudExpo: Blog Feed Post

Designing Startup Metrics to Drive Successful Behavior

The Secret to Success

Great companies are almost always run by great management teams. And great management teams know that the only way to improve a process is to start by measuring it. Good metrics should also be actionable, and drive successful behavior. In this post I hope to help show how to figure out which metrics matter the most, and how to design them in such a way as to drive behavior that will lead to the results that you want.

This post is applicable to any kind of business. In a follow up post, I will use this technique to walk through the design of a set of metrics for a SaaS company. Since SaaS businesses (or any other subscription-based business) are different from standard software businesses, there are some interesting elements that we will uncover.

image

Think of your company as a machine
One way to look at how companies work is to imagine them as a machine that has Outputs, and Levers that you, the management team, can pull to affect it’s behavior.

Weak management teams have only a limited understanding of how their machines work, and what levers are available to affect performance. The better the management team, the better they will understand how that machine works, and how they can optimize its performance (what levers they can pull).

When we look to design metrics, we are looking to deepen our understanding of the machinery, and how it works. Well designed metrics will automatically drive behavior to optimize output from the machine.

Example of a bad Board Meeting
Here is an example of a bad board meeting, which happens far more frequently than you might imagine. The company has just missed its quarterly revenue forecast. Good board members want to know two things:

  • Why that happened?
  • What can be done to avoid the problem going forward?

As they ask management what happened, a common answer will be that the market was really tough, and deals just didn’t close the way that they hoped. They also don’t have a great plan for what they are going to do differently next quarter, other than hope that the market improves, and that more deals will close. There is a great saying for situations like this: Hope is not a strategy.

Example of a good Board Meeting
The better management teams answer those questions differently. They will gradually peel back the covers of the machine, like peeling the layers of an onion, and expose the true nature of the problem, which of course will also highlight what levers need to be pulled to fix the problem. Lets take an example, and look at how they might do this:

    1. They will be able to tell you that revenue is composed of deals. To compute revenue, you multiply average deal size by number of deals. They may tell you that they were targeting to grow their average deal size to $x, and were successful in hitting this target. But the number of deals that they closed was below target.
    2. They will then peel back the onion one more layer, and tell you that the reason that the number of deals was below target was because 1/3rd of the salesforce missed their targets.
    3. They will then peel back another layer, and tell you that the reason those salespeople missed their targets was because they were not handed the required number of trials from marketing. However, for the trials that they did receive they were successful at converting them to closed deals at the expected conversion rate. So we know from this that the problem is not the quality of those sales people.
    4. Peeling back another layer, they will tell you that the number of trials is equal to the visitors to the site x the conversion rate of those visitors to trials. They may tell you that the number of visitors was on target, but the conversion rate fell below the previous levels.
    5. Peeling back one more level, they may tell you that they ran three major campaigns to drive visitors to the site, as well as relying on the normal levels of word of mouth traffic.  They may then reveal the true source of the problem: the ads that they had started running on Facebook were delivering a far lower conversion rate to trials than in prior months.

The contrast between the two approaches is stark. In the second case, it is clear that management will know how to fix the problem (by adding new traffic generation programs). They also know precisely how much additional traffic will need to be generated to reach the growth targets, and how many sales people are needed at a given productivity level, etc. etc.

What is surprising is just how few management teams really have their act in order in this area. For Web and SaaS businesses with smaller transactions at higher volumes, this kind of modeling and tracking is much easier, as web-based lead generation and marketing have easy to implement measurements, and the greater the volume of transactions, the more clearly patterns emerge. This is a little harder to do for channel sales, but still extremely valuable. And a little harder than that for direct sales situations with large deal sizes.

The Secret to Success
The secret to successful design of metrics is to start with the end goal and work backwards. In most companies, the end goals that matter the most are:

  • Profit/(Loss)
  • Growth
  • Good cash flow

(You may wonder why we don’t have Revenue in this list, but read further, and and it will soon become clear.)

Let’s take the first of these, Profitability, and work backwards. Working backwards means looking at the components that make up Profitability:

Profits (EBITDA) = Revenue – Cost of Goods Sold – Expenses

So to focus the management team on driving profitability, we should also track and measure Revenue, CoGS, and Expenses. Obvious, isn’t it? Well the good news is that this same principle can be applied over and over again focusing on the components of Revenue, CoGS, and Expenses where needed.

So the next step is to take Revenue, CoGS, and Expenses, and break them down to the key components. Bookings is the pre-cursor to Revenue. So let’s look at Bookings as an example:

Bookings =No of deals closed * Average Deal Size

For Reseller Channels, we might be looking at something different like this:

Revenue = No of productive resellers * average productivity per reseller

(Note: in many businesses there are several categories of deals. e.g. there could be large deals, and smaller deals. Or their could be deals from two or more different categories of customers. So the formula may have more elements to it than shown above.)

Peeling back another level, we might find the following:

No of deals closed = No of productive sales people * Average Productivity per Sales person

There will also likely be another formula to compute this, which will look like the following:

No of deals closed = No of Trials * Average Conversion Rate

These two formulae clearly indicate some of the levers that we have available to increase Bookings. We can grow the number of trials, or grow the number productive sales people, or we could try to increase the average productivity of our sales people. However we need to make sure that we grow them both together, otherwise we could end up out of balance, and have too many sales people and not enough trials to feed them, or too many trials and not enough productive sales people to close them.

The next step would be to peel back the onion a few more layers:

No of trials = No of visitors to the web site * Average Conversion Rate to Trials

No of Visitors to the web site = Normal traffic + for each traffic generation campaign: target audience of each campaign * Conversion Rate to visitors

Each time we peel back a layer to expose the components, we gain a better understanding of our machine and the levers that we can pull to make it work better. For example:

In the above two formulae, we can see that a big driver of the model is visitors to the web site. But this can be expensive to increase. So the other variable that we can try to increase is the conversion rate for each campaign, and the conversion rate to trials. We can try to do this by altering campaign messaging and landing pages and using A/B testing to find the optimum creative content.

We might also decide to focus our efforts on increasing the average deal size. We could do this in several ways:

  • Cross sell to add additional products
  • Up sell to add seats, or premium features
  • Develop a scalable pricing matrix that does a better job of charging higher end customers that are willing to pay more. This might involve several new axes that increase pricing, such as charging per seat, or charging per 1,000 data elements tracked, or charging for 24×7 support, etc.

As with many good ideas in business, all of the ideas above are obvious, and follow common sense.  However, you would be shocked to discover how rare it is to actually see businesses that have fully peeled back the onion to expose all the major variables and levers, and then implemented appropriate metrics to track these over time.

Trend based analysis
For every major variable that matters in our model, we will want to track how this varies over time. This will show us if we are succeeding in our efforts to improve things, and also give us early warning signs of any negative trends.

For most stages in a sales and marketing pipeline, we will want to track two metrics: how many prospects we put through that stage, and how effective were we at converting them to the next stage.Peeling back the Onion on Inside Sales performance

Another area where metrics can be extremely useful is in managing an inside sales (telesales) organization. Starting with the overall sales number achieved by the whole group, let’s peel this back layer by layer, to see what we can learn:

  • Overall group performance = Sum (individual contributor performance).

Not surprisingly we need to look at how each individual has done relative to the average levels to understand the strong performers, and the weak performers.

  • Individual performance = No of deals closed * Average Deal Size

For the weak performers, it is likely that the number of deals closed will be lower than we want. The question is why? So what are the components that make up the number of deals that an individual closes? Assuming a sales process where each inside sales person is handed a queue of marketing qualified leads, and then calls these to try to schedule a demo, and the post the demo tries to close a sale, the components will be:

  1. Calls made per sales person (if this is low, they will quickly react to peer pressure when they see other sales people’s call rates)
  2. Conversion rate to returned calls. (If this is low, it means the sales person is not leaving compelling voicemails, and should be given training by someone that has a high conversion rate.)
  3. Conversion rate from phone calls to Demos. (If this is low, it means the sales person’s ability to convey the value proposition is weak, and they should be given training by someone with high conversion rates.)
  4. Conversion rate from Demos to Closed Deals. (If this is low, it means the sales person needs better demo training.)
  5. Average Deal Size. (If this is low, it could mean the sales person needs better training on cross selling, or up selling.)

The above may not mirror your inside sales process, but hopefully the method of working backwards from the end goal, and peeling back the layers to expose the components will enable you to map out the metrics that matter to you.

Sales and marketing funnel – summary metrics
We will also want to look at some metrics that cover the entire sales and marketing funnel from top to bottom. Here are some example metrics that are important at this overall level:

Lead source effectiveness:

  • CAC by lead source
  • ROI by lead source (takes into consideration cost, conversion rates to closed deals, and lifetime value of customers that came through that particular lead source)

What not to track
Some categories like Expenses are made up of many line items, and we very likely don’t want to bother with metrics for every line item, we need to answer the question: How deep should we go with our analysis? The answer to this is pretty much common sense:

  • Prioritize the components that have the biggest effect
  • Don’t put much effort into tracking things that you can’t affect
  • Don’t bother tracking items that are small, or that don’t vary much. Leave these to accounting.

Conclusions
There is nothing in this article that should be surprising or earth shattering. It is all obvious. However, as is often the case in business, it is really easy to have the vision of what to do, but far harder to execute on that vision. In my experience the mark of a really well run business is that they actually have the systems in place to automatically produce these metrics. And they use those metrics as part of the management process to run the business.

The Benefits: Good Metrics drive Actions and Behavior
One of the greatest things about putting in place the right metrics is that showing them to people will automatically change their behavior to try to improve the metrics.  Furthermore, the metrics make it clear what levers they can use to change performance.

Well designed metrics make it clear what actions are needed to hit plan
Working backwards from a specific Revenue target, management will be able to understand all the other elements that have to be put in place to reach that target. For example, if you want to hit $xm in bookings for the quarter, you can work out:

  • How many sales people are required
  • How many leads are required to feed those sales people
  • What marketing campaign spend is needed to generate those leads

If you are in a channel model, you can work out how many productive resellers are required, and given a known conversion rate from newly signed resellers going through an on-boarding process, you will be able to work out how many new resellers are required, and how many on-boarding sales training sessions need to be run. Etc.

Acknowledgements
I have had the very good fortune to work with some excellent management teams that have helped teach me these lessons. In particular, I would like to thank the teams at HubSpot and JBoss who were very advanced in their use of metrics.

Follow up blog post
Watch out for my follow up blog post on SaaS Metrics and Levers to see what happens when I drill down on the key metrics for a SaaS business. (This is applicable to other subscription businesses such as Open Source.)

Read the original blog entry...

More Stories By David Skok

David Skok joined Matrix Partners as a General Partner in May 2001. He has a wealth of experience running companies. He started his first company in 1977 at age 22. Since then he has founded a total of four separate companies and performed one turn-around. Three of these companies went public.

Skok joined Matrix from SilverStream Software, which he founded in June 1996. Prior to its July 2002 acquisition by Novell, SilverStream was a public company that had reached a revenue run rate in excess of $100M, with approximately 800 employees and offices in more than 20 countries around the world. His work as a value added investor is best known for helping JBoss take its Open Source business to a successful exit with its sale to Red Hat, and for helping AppIQ, Tabblo and Diligent Technologies, which have all had successful exits, from their inceptions to their acquisitions by HP and IBM.

He serves on the boards of Digium (makers of the very popular Asterisk Open Source PBX/telephony software), CloudSwitch, Enservio, OpenSpan, Solidworks, VideoIQ, and HubSpot. In addition to his broad focus on enterprise software, he is specifically focused on the areas of cloud computing, Open Source, Software as a Service (SaaS), marketing automation, virtualization, storage, and data center automation.

@CloudExpo Stories
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, discussed how data centers of the future will be managed, how the p...
Sometimes I write a blog just to formulate and organize a point of view, and I think it’s time that I pull together the bounty of excellent information about Machine Learning. This is a topic with which business leaders must become comfortable, especially tomorrow’s business leaders (tip for my next semester University of San Francisco business students!). Machine learning is a key capability that will help organizations drive optimization and monetization opportunities, and there have been some...
"Storpool does only block-level storage so we do one thing extremely well. The growth in data is what drives the move to software-defined technologies in general and software-defined storage," explained Boyan Ivanov, CEO and co-founder at StorPool, in this SYS-CON.tv interview at 16th Cloud Expo, held June 9-11, 2015, at the Javits Center in New York City.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, whic...
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory? In her Day 2 Keynote at @DevOpsSummit at 21st Cloud Expo, Aruna Ravichandran, VP, DevOps Solutions Marketing, CA Technologies, was jo...
As Marc Andreessen says software is eating the world. Everything is rapidly moving toward being software-defined – from our phones and cars through our washing machines to the datacenter. However, there are larger challenges when implementing software defined on a larger scale - when building software defined infrastructure. In his session at 16th Cloud Expo, Boyan Ivanov, CEO of StorPool, provided some practical insights on what, how and why when implementing "software-defined" in the datacent...
Blockchain. A day doesn’t seem to go by without seeing articles and discussions about the technology. According to PwC executive Seamus Cushley, approximately $1.4B has been invested in blockchain just last year. In Gartner’s recent hype cycle for emerging technologies, blockchain is approaching the peak. It is considered by Gartner as one of the ‘Key platform-enabling technologies to track.’ While there is a lot of ‘hype vs reality’ discussions going on, there is no arguing that blockchain is b...
Blockchain is a shared, secure record of exchange that establishes trust, accountability and transparency across business networks. Supported by the Linux Foundation's open source, open-standards based Hyperledger Project, Blockchain has the potential to improve regulatory compliance, reduce cost as well as advance trade. Are you curious about how Blockchain is built for business? In her session at 21st Cloud Expo, René Bostic, Technical VP of the IBM Cloud Unit in North America, discussed the b...
You know you need the cloud, but you’re hesitant to simply dump everything at Amazon since you know that not all workloads are suitable for cloud. You know that you want the kind of ease of use and scalability that you get with public cloud, but your applications are architected in a way that makes the public cloud a non-starter. You’re looking at private cloud solutions based on hyperconverged infrastructure, but you’re concerned with the limits inherent in those technologies.
Is advanced scheduling in Kubernetes achievable?Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, answered these questions and demonstrated techniques for implementing advanced scheduling. For example, using spot instances and co...
The cloud era has reached the stage where it is no longer a question of whether a company should migrate, but when. Enterprises have embraced the outsourcing of where their various applications are stored and who manages them, saving significant investment along the way. Plus, the cloud has become a defining competitive edge. Companies that fail to successfully adapt risk failure. The media, of course, continues to extol the virtues of the cloud, including how easy it is to get there. Migrating...
The use of containers by developers -- and now increasingly IT operators -- has grown from infatuation to deep and abiding love. But as with any long-term affair, the honeymoon soon leads to needing to live well together ... and maybe even getting some relationship help along the way. And so it goes with container orchestration and automation solutions, which are rapidly emerging as the means to maintain the bliss between rapid container adoption and broad container use among multiple cloud host...
Imagine if you will, a retail floor so densely packed with sensors that they can pick up the movements of insects scurrying across a store aisle. Or a component of a piece of factory equipment so well-instrumented that its digital twin provides resolution down to the micrometer.
The need for greater agility and scalability necessitated the digital transformation in the form of following equation: monolithic to microservices to serverless architecture (FaaS). To keep up with the cut-throat competition, the organisations need to update their technology stack to make software development their differentiating factor. Thus microservices architecture emerged as a potential method to provide development teams with greater flexibility and other advantages, such as the abili...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settle...
Product connectivity goes hand and hand these days with increased use of personal data. New IoT devices are becoming more personalized than ever before. In his session at 22nd Cloud Expo | DXWorld Expo, Nicolas Fierro, CEO of MIMIR Blockchain Solutions, will discuss how in order to protect your data and privacy, IoT applications need to embrace Blockchain technology for a new level of product security never before seen - or needed.
Leading companies, from the Global Fortune 500 to the smallest companies, are adopting hybrid cloud as the path to business advantage. Hybrid cloud depends on cloud services and on-premises infrastructure working in unison. Successful implementations require new levels of data mobility, enabled by an automated and seamless flow across on-premises and cloud resources. In his general session at 21st Cloud Expo, Greg Tevis, an IBM Storage Software Technical Strategist and Customer Solution Architec...
Nordstrom is transforming the way that they do business and the cloud is the key to enabling speed and hyper personalized customer experiences. In his session at 21st Cloud Expo, Ken Schow, VP of Engineering at Nordstrom, discussed some of the key learnings and common pitfalls of large enterprises moving to the cloud. This includes strategies around choosing a cloud provider(s), architecture, and lessons learned. In addition, he covered some of the best practices for structured team migration an...
In his general session at 21st Cloud Expo, Greg Dumas, Calligo’s Vice President and G.M. of US operations, discussed the new Global Data Protection Regulation and how Calligo can help business stay compliant in digitally globalized world. Greg Dumas is Calligo's Vice President and G.M. of US operations. Calligo is an established service provider that provides an innovative platform for trusted cloud solutions. Calligo’s customers are typically most concerned about GDPR compliance, application p...
Coca-Cola’s Google powered digital signage system lays the groundwork for a more valuable connection between Coke and its customers. Digital signs pair software with high-resolution displays so that a message can be changed instantly based on what the operator wants to communicate or sell. In their Day 3 Keynote at 21st Cloud Expo, Greg Chambers, Global Group Director, Digital Innovation, Coca-Cola, and Vidya Nagarajan, a Senior Product Manager at Google, discussed how from store operations and ...