|By Mike Maciag||
|June 2, 2011 04:45 PM EDT||
It's an age-old struggle. IT works hard to provide top-of-the-line infrastructure, while developers juggle the build-test-deploy cycle. Somewhere in the middle things get lost in translation and the two find themselves at odds. Developers vie for control of resources and access to tools, while IT struggles to provide resources that are standardized and can be managed in a secure and consistent way.
Sound familiar? We see it all the time in the enterprises we work with. It really boils down to seemingly disparate goals. Even though they're working for the same organization, IT and developers are trying to achieve different things. IT strives for efficient use of resources to get a better return on infrastructure investment. Software developers, on the other hand, are concerned mainly with efficient development. They demand ready access to infrastructure and need a wide range of tools at their fingertips. The constantly fluctuating demands and hodgepodge of tools make it difficult for IT to keep up with their resource needs, and maintain security and consistency within the organization. It's no wonder they always seem to be butting heads.
A New Approach
With the advent of cloud computing, there's new hope of bridging the gap between IT and development. The processes that benefit most from moving to the cloud are those that are resource-intensive or "bursty" in compute demand - exactly the kinds of processes that abound in the build-test-deploy cycle. Great examples include compiling and building source code, testing on several different operating systems, and load testing.
But is a basic cloud implementation enough to address the Dev - IT divide?
Most cloud implementations leverage virtualization and user self-service as their two cornerstone technologies. Virtualization dramatically improves the utilization of the underlying resource; now your underutilized physical servers can be loaded up with many virtual machines (VM), improving your asset utilization. Virtualization also allows IT to provide standardized resources as templates s servers, applications, databases, etc., to users, which enables fast setup and consistent management of the resources. Self-service gives users IT resources on demand: they can request a new server and voila - a new virtual machine is provided instantaneously. Because they don't have to wait for hours or days to get the compute services they need, productivity and time-to-market can be improved.
While cloud provides a lot of value, it still doesn't address the way development wants to interact with IT. Most development teams today have a software production process, a workflow that starts with developers writing software code, building and testing the software, and culminates with the release/deployment of the customer-ready software. This process, complex to begin with, is becoming even more complex with the adoption of Agile development methodologies that encourage faster and more iterative development of software applications. To improve the productivity and efficiency of development in this fast-evolving landscape, cloud infrastructure and services need to be tightly integrated to the process. The necessary ingredients of this integration include:
- Automated, but seamless, self-service: Developers want self-service, but not in the typical Web interface sense that limits them to setting up one resource at a time. In today's fast-paced and Agile software production process, developers need the ability to set up resources instantaneously and in-context of the software production process. For example, developers want build systems to be automatically provisioned upon start of a build process, and torn down upon successful completion of the process. It is imperative that this process is seamless so it can be iterated multiple times a day; it's also important to automate the process so IT doesn't have to deal with VM sprawl or orphaned VM issues.
- Customized resource and environment: While the cloud provides standard IT compute resources, developers typically want to customize the resources to the requirements of the software production process. This may involve configuring the standard IT-instantiated resources deploying new dev/test-specific applications. Just as important, developers want these changes to be done automatically without manual interventions
- Automatic resource management: Developers want the cloud solution to automatically manage the cloud resource, whether it means creation, deletion or active management of the cloud workload and resources. This lets them focus on what they do best and, more important, it enables the IT organization to manage cloud resources in an optimal and efficient manner and achieve shared service economies.
- Visibility: Developers want their solution to provide them with end-to-end visibility into the software production process and the resources that these processes run on. Whether the process is running on physical, virtual or cloud resources, developers are looking for good analytics to quickly triage software production process errors (which build was broken, what software version passed the tests, etc.).
- Flexibility: Finally, while developers want to leverage the cloud, they don't want to be locked in to any one resource choice. The development team wants to retain the option of using physical, virtual or cloud services (private or public) to best fit their production process.
Today's cloud solutions do a great job of managing the cloud resources from an infrastructure perspective:
- Lab management solutions, such as VMWare's Lab Manager or Citrix VMlogix, manage the VMs and standardized templates that are used by development teams
- Cloud Infrastructure as a Service (IaaS) solutions such as Eucalyptus provision and manage the infrastructure that's used by the development team
- Amazon EC2 and Rightscale provide capabilities to use the public cloud to build and test software applications.
Many of these solutions, however, don't understand the development process /tasks that are run on cloud resources, and it is precisely the lack of these types of integrations that is preventing development teams from widespread adoption of cloud technology. One piece that fills the self-service gap to enable development on the private cloud is a software workflow automation system like Electric Cloud ElectricCommander.
Let's talk about how this looks in practice. Say I need to do some system testing. I'm going to need a bunch of machines. If my IT organization supports virtualization technologies, such as VMware Vsphere or Microsoft HyperV, I can get a number of virtual machines with a specific software configuration on them. This is a big improvement over the old days when I would need to secure physical machines, but it's still the virtual equivalent of a blank rack of servers. I've provisioned the resources, but I haven't provisioned the actual test applications. At this point, I have self-service compute resources, but I don't have a system in place that determines what needs to happen, what workflows it needs to go through, how I'm going to load the software or how I'm going to integrate the tools.
By implementing a software production automation system that is optimized for use on the cloud, I now have a platform that lets me define the steps, the workflow between them, tool integration and resource management.
This solution would provide workflow automation (automating, parallelizing and distributing steps within the workflow), seamless services (automatically setting up and tearing down resources as tasks demand), dev tool integration and end-to-end visibility and reporting (aggregating data from multiple apps to quickly identify software errors).
How It Plays It Out in the "Real World"
One of our customers, a large financial institution, has a development team of more than 5,000 developers spread around the world. They've long employed Agile practices, including continuous integration and test processes, but as the development team grew, its demands overwhelmed the script- and open source-based software build and test system they had relied on. Because individual teams were allowed the discretion to choose development methods and tools that worked best for them, the organization was dealing with a wide variety of tools that had become difficult to manage.
A private development cloud turned out to be an essential part of the solution for this organization. It allowed them to offer software build and test as a service to developers, while staying behind their firewall to maintain the tight security the financial industry demands. They now have a common pool of resources to support build, test and deploy procedures that are always accessible on-demand. Teams are still using the tools they prefer, but the organization can now easily allocate resources as they are needed, while supporting parallel builds across multiple computers with varied operating systems and languages.
This customer implemented the private development cloud as an opt-in service, letting teams choose whether to use it or continue to run builds and tests locally. But as teams began to see the benefits, they were eager to move to the cloud. IT is happier too: managing resources while accommodating development's varied tools is now easier, and they have much better visibility into the development process, which is invaluable for a financial company that has to be ready for audits.
At a macro level, implementing a private development cloud has allowed this organization to increase their productivity and save money. The developers, though, aren't thinking of it as an ROI - they're just glad to have a system that helps them do their jobs as efficiently and easily as possible.
Moving development to the cloud and enabling self-service allows developers and IT to work together more easily, with the end result they're all ultimately looking for: better software that's built, tested and deployed cheaper and faster. It seems the cloud holds the key to ending that age-old struggle once and for all.
SYS-CON Events announced today that Gridstore™, the leader in hyper-converged infrastructure purpose-built to optimize Microsoft workloads, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Gridstore™ is the leader in hyper-converged infrastructure purpose-built for Microsoft workloads and designed to accelerate applications in virtualized environments. Gridstore’s hyper-converged infrastructure is the ...
May. 24, 2015 01:15 AM EDT Reads: 6,186
of cloud, colocation, managed services and disaster recovery solutions, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. TierPoint, LLC, is a leading national provider of information technology and data center services, including cloud, colocation, disaster recovery and managed IT services, with corporate headquarters in St. Louis, MO. TierPoint was formed through the strategic combination of some of t...
May. 24, 2015 01:00 AM EDT Reads: 3,120
Cryptography has become one of the most underappreciated, misunderstood components of technology. It’s too easy for salespeople to dismiss concerns with three letters that nobody wants to question. ‘Yes, of course, we use AES.’ But what exactly are you trusting to be the ultimate guardian of your data? Let’s face it – you probably don’t know. An organic, grass-fed Kobe steak is a far cry from a Big Mac, but they’re both beef, right? Not exactly. Crypto is the same way. The US government require...
May. 24, 2015 01:00 AM EDT Reads: 3,918
SYS-CON Events announced today Sematext Group, Inc., a Brooklyn-based Performance Monitoring and Log Management solution provider, will exhibit at SYS-CON's DevOps Summit 2015 New York, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Sematext is a globally distributed organization that builds innovative Cloud and On Premises solutions for performance monitoring, alerting and anomaly detection (SPM), log management and analytics (Logsene), search analytics (S...
May. 24, 2015 12:45 AM EDT Reads: 5,203
Hadoop as a Service (as offered by handful of niche vendors now) is a cloud computing solution that makes medium and large-scale data processing accessible, easy, fast and inexpensive. In his session at Big Data Expo, Kumar Ramamurthy, Vice President and Chief Technologist, EIM & Big Data, at Virtusa, will discuss how this is achieved by eliminating the operational challenges of running Hadoop, so one can focus on business growth. The fragmented Hadoop distribution world and various PaaS soluti...
May. 24, 2015 12:30 AM EDT Reads: 3,845
SYS-CON Events announced today that Stratoscale, the new data center operating system, will exhibit at SYS-CON's 16th International Cloud Expo®, which will take place on June 9-11, 2015, at the Javits Center in New York City, NY. Based in Herzeliya, Israel, Stratoscale is redefining the data center, developing a hardware-agnostic, software platform hyper-converging compute, storage and networking across the rack or data center. The self-optimizing platform automatically distributes all physical...
May. 24, 2015 12:30 AM EDT Reads: 2,990
When an enterprise builds a hybrid IaaS cloud connecting its data center to one or more public clouds, security is often a major topic along with the other challenges involved. Security is closely intertwined with the networking choices made for the hybrid cloud. Traditional networking approaches for building a hybrid cloud try to kludge together the enterprise infrastructure with the public cloud. Consequently this approach requires risky, deep "surgery" including changes to firewalls, subnets...
May. 24, 2015 12:00 AM EDT Reads: 4,149
SYS-CON Media announced today that Blue Box as launched a popular blog feed on Cloud Computing Journal. Cloud Computing Journal aims to help open the eyes of Enterprise IT professionals to the economics and strategies that utility/cloud computing provides. Blue Box Cloud gives you unequaled agility, without the burden of designing, deploying and managing your own infrastructure. It’s the right choice when public cloud just won’t do. Blue Box Cloud is a managed Private Cloud as a Service (...
May. 24, 2015 12:00 AM EDT Reads: 2,852
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo in Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading in...
May. 23, 2015 11:00 PM EDT Reads: 2,727
MeriTalk, a public-private partnership focused on improving the outcomes of government IT, today announced the results of its new report, "The Agile Advantage: Can DevOps Move Cloud to the Fast Lane?" The study, underwritten by Accenture Federal Services, reveals that approximately two-thirds of Feds say DevOps will help agencies shift into the cloud fast lane - improving IT collaboration and migration speed. But help is needed, with 66 percent saying that their agency needs to move IT services ...
May. 23, 2015 11:00 PM EDT Reads: 1,794
There is little doubt that Big Data solutions will have an increasing role in the Enterprise IT mainstream over time. 8th International Big Data Expo, co-located with 17th International Cloud Expo - to be held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA - has announced its Call for Papers is open. As advanced data storage, access and analytics technologies aimed at handling high-volume and/or fast moving data all move center stage, aided by the cloud computing bo...
May. 23, 2015 08:45 PM EDT Reads: 1,743
Move from reactive to proactive cloud management in a heterogeneous cloud infrastructure. In his session at 16th Cloud Expo, Manoj Khabe, Innovative Solution-Focused Transformation Leader at Vicom Computer Services, Inc., will show how to replace a help desk-centric approach with an ITIL-based service model and service-centric CMDB that’s tightly integrated with an event and incident management platform. Learn how to expand the scope of operations management to service management. He will al...
May. 23, 2015 08:00 PM EDT Reads: 1,527
Every day we read jaw-dropping stats on the explosion of data. We allocate significant resources to harness and better understand it. We build businesses around it. But we’ve only just begun. For big payoffs in Big Data, CIOs are turning to cognitive computing. Cognitive computing’s ability to securely extract insights, understand natural language, and get smarter each time it’s used is the next, logical step for Big Data.
May. 23, 2015 08:00 PM EDT Reads: 2,139
Enterprises are fast realizing the importance of integrating SaaS/Cloud applications, API and on-premises data and processes, to unleash hidden value. This webinar explores how managers can use a Microservice-centric approach to aggressively tackle the unexpected new integration challenges posed by proliferation of cloud, mobile, social and big data projects. Industry analyst and SOA expert Jason Bloomberg will strip away the hype from microservices, and clearly identify their advantages and d...
May. 23, 2015 07:00 PM EDT Reads: 1,889
DevOps Summit, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long developmen...
May. 23, 2015 06:00 PM EDT Reads: 2,412
The 5th International DevOps Summit, co-located with 17th International Cloud Expo – being held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the...
May. 23, 2015 05:30 PM EDT Reads: 4,181
Over the years, a variety of methodologies have emerged in order to overcome the challenges related to project constraints. The successful use of each methodology seems highly context-dependent. However, communication seems to be the common denominator of the many challenges that project management methodologies intend to resolve. In this respect, Information and Communication Technologies (ICTs) can be viewed as powerful tools for managing projects. Few research papers have focused on the way...
May. 23, 2015 05:00 PM EDT Reads: 1,844
As the world moves from DevOps to NoOps, application deployment to the cloud ought to become a lot simpler. However, applications have been architected with a much tighter coupling than it needs to be which makes deployment in different environments and migration between them harder. The microservices architecture, which is the basis of many new age distributed systems such as OpenStack, Netflix and so on is at the heart of CloudFoundry – a complete developer-oriented Platform as a Service (PaaS...
May. 23, 2015 05:00 PM EDT Reads: 1,735
17th Cloud Expo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises a...
May. 23, 2015 05:00 PM EDT Reads: 2,464
High-performing enterprise Software Quality Assurance (SQA) teams validate systems that are ready for use - getting most actively involved as components integrate and form complete systems. These teams catch and report on defects, making sure the customer gets the best software possible. SQA teams have leveraged automation and virtualization to execute more thorough testing in less time - bringing Dev and Ops together, ensuring production readiness. Does the emergence of DevOps mean the end of E...
May. 23, 2015 04:00 PM EDT Reads: 5,247