Welcome!

@CloudExpo Authors: Liz McMillan, Karthick Viswanathan, Jason Bloomberg, Pat Romanski, Yeshim Deniz

Related Topics: @CloudExpo, Open Source Cloud

@CloudExpo: Blog Feed Post

Designing Startup Metrics to Drive Successful Behavior

The Secret to Success

Great companies are almost always run by great management teams. And great management teams know that the only way to improve a process is to start by measuring it. Good metrics should also be actionable, and drive successful behavior. In this post I hope to help show how to figure out which metrics matter the most, and how to design them in such a way as to drive behavior that will lead to the results that you want.

This post is applicable to any kind of business. In a follow up post, I will use this technique to walk through the design of a set of metrics for a SaaS company. Since SaaS businesses (or any other subscription-based business) are different from standard software businesses, there are some interesting elements that we will uncover.

image

Think of your company as a machine
One way to look at how companies work is to imagine them as a machine that has Outputs, and Levers that you, the management team, can pull to affect it’s behavior.

Weak management teams have only a limited understanding of how their machines work, and what levers are available to affect performance. The better the management team, the better they will understand how that machine works, and how they can optimize its performance (what levers they can pull).

When we look to design metrics, we are looking to deepen our understanding of the machinery, and how it works. Well designed metrics will automatically drive behavior to optimize output from the machine.

Example of a bad Board Meeting
Here is an example of a bad board meeting, which happens far more frequently than you might imagine. The company has just missed its quarterly revenue forecast. Good board members want to know two things:

  • Why that happened?
  • What can be done to avoid the problem going forward?

As they ask management what happened, a common answer will be that the market was really tough, and deals just didn’t close the way that they hoped. They also don’t have a great plan for what they are going to do differently next quarter, other than hope that the market improves, and that more deals will close. There is a great saying for situations like this: Hope is not a strategy.

Example of a good Board Meeting
The better management teams answer those questions differently. They will gradually peel back the covers of the machine, like peeling the layers of an onion, and expose the true nature of the problem, which of course will also highlight what levers need to be pulled to fix the problem. Lets take an example, and look at how they might do this:

    1. They will be able to tell you that revenue is composed of deals. To compute revenue, you multiply average deal size by number of deals. They may tell you that they were targeting to grow their average deal size to $x, and were successful in hitting this target. But the number of deals that they closed was below target.
    2. They will then peel back the onion one more layer, and tell you that the reason that the number of deals was below target was because 1/3rd of the salesforce missed their targets.
    3. They will then peel back another layer, and tell you that the reason those salespeople missed their targets was because they were not handed the required number of trials from marketing. However, for the trials that they did receive they were successful at converting them to closed deals at the expected conversion rate. So we know from this that the problem is not the quality of those sales people.
    4. Peeling back another layer, they will tell you that the number of trials is equal to the visitors to the site x the conversion rate of those visitors to trials. They may tell you that the number of visitors was on target, but the conversion rate fell below the previous levels.
    5. Peeling back one more level, they may tell you that they ran three major campaigns to drive visitors to the site, as well as relying on the normal levels of word of mouth traffic.  They may then reveal the true source of the problem: the ads that they had started running on Facebook were delivering a far lower conversion rate to trials than in prior months.

The contrast between the two approaches is stark. In the second case, it is clear that management will know how to fix the problem (by adding new traffic generation programs). They also know precisely how much additional traffic will need to be generated to reach the growth targets, and how many sales people are needed at a given productivity level, etc. etc.

What is surprising is just how few management teams really have their act in order in this area. For Web and SaaS businesses with smaller transactions at higher volumes, this kind of modeling and tracking is much easier, as web-based lead generation and marketing have easy to implement measurements, and the greater the volume of transactions, the more clearly patterns emerge. This is a little harder to do for channel sales, but still extremely valuable. And a little harder than that for direct sales situations with large deal sizes.

The Secret to Success
The secret to successful design of metrics is to start with the end goal and work backwards. In most companies, the end goals that matter the most are:

  • Profit/(Loss)
  • Growth
  • Good cash flow

(You may wonder why we don’t have Revenue in this list, but read further, and and it will soon become clear.)

Let’s take the first of these, Profitability, and work backwards. Working backwards means looking at the components that make up Profitability:

Profits (EBITDA) = Revenue – Cost of Goods Sold – Expenses

So to focus the management team on driving profitability, we should also track and measure Revenue, CoGS, and Expenses. Obvious, isn’t it? Well the good news is that this same principle can be applied over and over again focusing on the components of Revenue, CoGS, and Expenses where needed.

So the next step is to take Revenue, CoGS, and Expenses, and break them down to the key components. Bookings is the pre-cursor to Revenue. So let’s look at Bookings as an example:

Bookings =No of deals closed * Average Deal Size

For Reseller Channels, we might be looking at something different like this:

Revenue = No of productive resellers * average productivity per reseller

(Note: in many businesses there are several categories of deals. e.g. there could be large deals, and smaller deals. Or their could be deals from two or more different categories of customers. So the formula may have more elements to it than shown above.)

Peeling back another level, we might find the following:

No of deals closed = No of productive sales people * Average Productivity per Sales person

There will also likely be another formula to compute this, which will look like the following:

No of deals closed = No of Trials * Average Conversion Rate

These two formulae clearly indicate some of the levers that we have available to increase Bookings. We can grow the number of trials, or grow the number productive sales people, or we could try to increase the average productivity of our sales people. However we need to make sure that we grow them both together, otherwise we could end up out of balance, and have too many sales people and not enough trials to feed them, or too many trials and not enough productive sales people to close them.

The next step would be to peel back the onion a few more layers:

No of trials = No of visitors to the web site * Average Conversion Rate to Trials

No of Visitors to the web site = Normal traffic + for each traffic generation campaign: target audience of each campaign * Conversion Rate to visitors

Each time we peel back a layer to expose the components, we gain a better understanding of our machine and the levers that we can pull to make it work better. For example:

In the above two formulae, we can see that a big driver of the model is visitors to the web site. But this can be expensive to increase. So the other variable that we can try to increase is the conversion rate for each campaign, and the conversion rate to trials. We can try to do this by altering campaign messaging and landing pages and using A/B testing to find the optimum creative content.

We might also decide to focus our efforts on increasing the average deal size. We could do this in several ways:

  • Cross sell to add additional products
  • Up sell to add seats, or premium features
  • Develop a scalable pricing matrix that does a better job of charging higher end customers that are willing to pay more. This might involve several new axes that increase pricing, such as charging per seat, or charging per 1,000 data elements tracked, or charging for 24×7 support, etc.

As with many good ideas in business, all of the ideas above are obvious, and follow common sense.  However, you would be shocked to discover how rare it is to actually see businesses that have fully peeled back the onion to expose all the major variables and levers, and then implemented appropriate metrics to track these over time.

Trend based analysis
For every major variable that matters in our model, we will want to track how this varies over time. This will show us if we are succeeding in our efforts to improve things, and also give us early warning signs of any negative trends.

For most stages in a sales and marketing pipeline, we will want to track two metrics: how many prospects we put through that stage, and how effective were we at converting them to the next stage.Peeling back the Onion on Inside Sales performance

Another area where metrics can be extremely useful is in managing an inside sales (telesales) organization. Starting with the overall sales number achieved by the whole group, let’s peel this back layer by layer, to see what we can learn:

  • Overall group performance = Sum (individual contributor performance).

Not surprisingly we need to look at how each individual has done relative to the average levels to understand the strong performers, and the weak performers.

  • Individual performance = No of deals closed * Average Deal Size

For the weak performers, it is likely that the number of deals closed will be lower than we want. The question is why? So what are the components that make up the number of deals that an individual closes? Assuming a sales process where each inside sales person is handed a queue of marketing qualified leads, and then calls these to try to schedule a demo, and the post the demo tries to close a sale, the components will be:

  1. Calls made per sales person (if this is low, they will quickly react to peer pressure when they see other sales people’s call rates)
  2. Conversion rate to returned calls. (If this is low, it means the sales person is not leaving compelling voicemails, and should be given training by someone that has a high conversion rate.)
  3. Conversion rate from phone calls to Demos. (If this is low, it means the sales person’s ability to convey the value proposition is weak, and they should be given training by someone with high conversion rates.)
  4. Conversion rate from Demos to Closed Deals. (If this is low, it means the sales person needs better demo training.)
  5. Average Deal Size. (If this is low, it could mean the sales person needs better training on cross selling, or up selling.)

The above may not mirror your inside sales process, but hopefully the method of working backwards from the end goal, and peeling back the layers to expose the components will enable you to map out the metrics that matter to you.

Sales and marketing funnel – summary metrics
We will also want to look at some metrics that cover the entire sales and marketing funnel from top to bottom. Here are some example metrics that are important at this overall level:

Lead source effectiveness:

  • CAC by lead source
  • ROI by lead source (takes into consideration cost, conversion rates to closed deals, and lifetime value of customers that came through that particular lead source)

What not to track
Some categories like Expenses are made up of many line items, and we very likely don’t want to bother with metrics for every line item, we need to answer the question: How deep should we go with our analysis? The answer to this is pretty much common sense:

  • Prioritize the components that have the biggest effect
  • Don’t put much effort into tracking things that you can’t affect
  • Don’t bother tracking items that are small, or that don’t vary much. Leave these to accounting.

Conclusions
There is nothing in this article that should be surprising or earth shattering. It is all obvious. However, as is often the case in business, it is really easy to have the vision of what to do, but far harder to execute on that vision. In my experience the mark of a really well run business is that they actually have the systems in place to automatically produce these metrics. And they use those metrics as part of the management process to run the business.

The Benefits: Good Metrics drive Actions and Behavior
One of the greatest things about putting in place the right metrics is that showing them to people will automatically change their behavior to try to improve the metrics.  Furthermore, the metrics make it clear what levers they can use to change performance.

Well designed metrics make it clear what actions are needed to hit plan
Working backwards from a specific Revenue target, management will be able to understand all the other elements that have to be put in place to reach that target. For example, if you want to hit $xm in bookings for the quarter, you can work out:

  • How many sales people are required
  • How many leads are required to feed those sales people
  • What marketing campaign spend is needed to generate those leads

If you are in a channel model, you can work out how many productive resellers are required, and given a known conversion rate from newly signed resellers going through an on-boarding process, you will be able to work out how many new resellers are required, and how many on-boarding sales training sessions need to be run. Etc.

Acknowledgements
I have had the very good fortune to work with some excellent management teams that have helped teach me these lessons. In particular, I would like to thank the teams at HubSpot and JBoss who were very advanced in their use of metrics.

Follow up blog post
Watch out for my follow up blog post on SaaS Metrics and Levers to see what happens when I drill down on the key metrics for a SaaS business. (This is applicable to other subscription businesses such as Open Source.)

Read the original blog entry...

More Stories By David Skok

David Skok joined Matrix Partners as a General Partner in May 2001. He has a wealth of experience running companies. He started his first company in 1977 at age 22. Since then he has founded a total of four separate companies and performed one turn-around. Three of these companies went public.

Skok joined Matrix from SilverStream Software, which he founded in June 1996. Prior to its July 2002 acquisition by Novell, SilverStream was a public company that had reached a revenue run rate in excess of $100M, with approximately 800 employees and offices in more than 20 countries around the world. His work as a value added investor is best known for helping JBoss take its Open Source business to a successful exit with its sale to Red Hat, and for helping AppIQ, Tabblo and Diligent Technologies, which have all had successful exits, from their inceptions to their acquisitions by HP and IBM.

He serves on the boards of Digium (makers of the very popular Asterisk Open Source PBX/telephony software), CloudSwitch, Enservio, OpenSpan, Solidworks, VideoIQ, and HubSpot. In addition to his broad focus on enterprise software, he is specifically focused on the areas of cloud computing, Open Source, Software as a Service (SaaS), marketing automation, virtualization, storage, and data center automation.

@CloudExpo Stories
yperConvergence came to market with the objective of being simple, flexible and to help drive down operating expenses. It reduced the footprint by bundling the compute/storage/network into one box. This brought a new set of challenges as the HyperConverged vendors are very focused on their own proprietary building blocks. If you want to scale in a certain way, let’s say you identified a need for more storage and want to add a device that is not sold by the HyperConverged vendor, forget about it....
SYS-CON Events announced today that Pulzze Systems will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems Inc, provides the software product "The Interactor" that uniquely simplifies building IoT, Web and Smart Enterprise Solutions. It is a Silicon Valley startup funded by US government agencies, NSF and DHS to bring innovative solutions to market.
With Cloud Foundry you can easily deploy and use apps utilizing websocket technology, but not everybody realizes that scaling them out is not that trivial. In his session at 21st Cloud Expo, Roman Swoszowski, CTO and VP, Cloud Foundry Services, at Grape Up, will show you an example of how to deal with this issue. He will demonstrate a cloud-native Spring Boot app running in Cloud Foundry and communicating with clients over websocket protocol that can be easily scaled horizontally and coordinate...
SYS-CON Events announced today that Elastifile will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Elastifile Cloud File System (ECFS) is software-defined data infrastructure designed for seamless and efficient management of dynamic workloads across heterogeneous environments. Elastifile provides the architecture needed to optimize your hybrid cloud environment, by facilitating efficient...
For financial firms, the cloud is going to increasingly become a crucial part of dealing with customers over the next five years and beyond, particularly with the growing use and acceptance of virtual currencies. There are new data storage paradigms on the horizon that will deliver secure solutions for storing and moving sensitive financial data around the world without touching terrestrial networks. In his session at 20th Cloud Expo, Cliff Beek, President of Cloud Constellation Corporation, d...
Businesses and business units of all sizes can benefit from cloud computing, but many don't want the cost, performance and security concerns of public cloud nor the complexity of building their own private clouds. Today, some cloud vendors are using artificial intelligence (AI) to simplify cloud deployment and management. In his session at 20th Cloud Expo, Ajay Gulati, Co-founder and CEO of ZeroStack, discussed how AI can simplify cloud operations. He covered the following topics: why cloud mana...
As more and more companies are making the shift from on-premises to public cloud, the standard approach to DevOps is evolving. From encryption, compliance and regulations like GDPR, security in the cloud has become a hot topic. Many DevOps-focused companies have hired dedicated staff to fulfill these requirements, often creating further siloes, complexity and cost. This session aims to highlight existing DevOps cultural approaches, tooling and how security can be wrapped in every facet of the bu...
DevOps at Cloud Expo, taking place October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 21st Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to w...
SYS-CON Events announced today that CAST Software will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CAST was founded more than 25 years ago to make the invisible visible. Built around the idea that even the best analytics on the market still leave blind spots for technical teams looking to deliver better software and prevent outages, CAST provides the software intelligence that matter ...
SYS-CON Events announced today that Golden Gate University will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Since 1901, non-profit Golden Gate University (GGU) has been helping adults achieve their professional goals by providing high quality, practice-based undergraduate and graduate educational programs in law, taxation, business and related professions. Many of its courses are taug...
Vulnerability management is vital for large companies that need to secure containers across thousands of hosts, but many struggle to understand how exposed they are when they discover a new high security vulnerability. In his session at 21st Cloud Expo, John Morello, CTO of Twistlock, will address this pressing concern by introducing the concept of the “Vulnerability Risk Tree API,” which brings all the data together in a simple REST endpoint, allowing companies to easily grasp the severity of t...
Recently, WebRTC has a lot of eyes from market. The use cases of WebRTC are expanding - video chat, online education, online health care etc. Not only for human-to-human communication, but also IoT use cases such as machine to human use cases can be seen recently. One of the typical use-case is remote camera monitoring. With WebRTC, people can have interoperability and flexibility for deploying monitoring service. However, the benefit of WebRTC for IoT is not only its convenience and interopera...
Translating agile methodology into real-world best practices within the modern software factory has driven widespread DevOps adoption, yet much work remains to expand workflows and tooling across the enterprise. As models evolve from pockets of experimentation into wholescale organizational reinvention, practitioners find themselves challenged to incorporate the culture and architecture necessary to support DevOps at scale.
SYS-CON Events announced today that CA Technologies has been named “Platinum Sponsor” of SYS-CON's 21st International Cloud Expo®, which will take place October 31-November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. CA Technologies helps customers succeed in a future where every business – from apparel to energy – is being rewritten by software. From planning to development to management to security, CA creates software that fuels transformation for companies in the applic...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
In his session at 20th Cloud Expo, Scott Davis, CTO of Embotics, discussed how automation can provide the dynamic management required to cost-effectively deliver microservices and container solutions at scale. He also discussed how flexible automation is the key to effectively bridging and seamlessly coordinating both IT and developer needs for component orchestration across disparate clouds – an increasingly important requirement at today’s multi-cloud enterprise.
WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
IT organizations are moving to the cloud in hopes to approve efficiency, increase agility and save money. Migrating workloads might seem like a simple task, but what many businesses don’t realize is that application migration criteria differs across organizations, making it difficult for architects to arrive at an accurate TCO number. In his session at 21st Cloud Expo, Joe Kinsella, CTO of CloudHealth Technologies, will offer a systematic approach to understanding the TCO of a cloud application...
"With Digital Experience Monitoring what used to be a simple visit to a web page has exploded into app on phones, data from social media feeds, competitive benchmarking - these are all components that are only available because of some type of digital asset," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
SYS-CON Events announced today that Secure Channels, a cybersecurity firm, will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Secure Channels, Inc. offers several products and solutions to its many clients, helping them protect critical data from being compromised and access to computer networks from the unauthorized. The company develops comprehensive data encryption security strategie...