In earlier days, solutions were associated with getting the technology right. The key was technology, the solution was technology and the business expected and paid for technology. Times have changed. Well, at least for those of us taking notice. Today technology is hardly ever a significant problem. Technically, we have a less complicated world. Over the years we have come to understand that technology is basically an arrangement of Processing, Memory, Networking and Storage. We have mastered utilization by using virtualization. We understand horizontal scaling is ‘better’ than vertical scaling and that we can deliver the PMNS more easily in converged and hyperconverged products that also contain the software solution. We have automated many of the key activities to enable reduction in time and costs.
DevOps – The Cloud paradigm came along and made life easier by helping us to become Service Brokers rather than server admins or network engineers. To the customer we are now Service Brokers; well, we should be. We should be experiencing shorter procurement cycles given that applications and services (the solutions) are delivered from a Service Catalog. Although this can be true in the Public Cloud deployment model and the Software as a Service (SaaS) delivery model, when it comes to Private Cloud procurement we still seem to be stuck in the past and suffer unnecessary delays. Even as Public Cloud services are taken up by more and more businesses the activity of getting the servers, applications and services ‘up there’ still makes for hard going. All the work that is required to design and deliver a Public Cloud hosted environment is still steeped in old-fashioned working practices.
Despite all this change and learning, solution design and implementation is still a thorny job and produces mountains of documentation (some needed, some pointless), endless Gant charts and interminable meetings trying to get the solution in place and delivered. Why is this?
Application Development and Delivery
Application developers use to live in a world of their own. To some extent that is still true. Application development companies don’t usually have network engineers, technical architects and storage SMEs sitting in on the early morning scrums. Applications are developed in isolation and separate from the technical solutions that will need to be created to host, resource and support the application.
In most cases an application is developed for one of two reasons. To provide a solution for an external customer or to provide an application for the business with which it can make money. For instance, a company needs to pay salaries. To do that it needs an application that can pay the salaries, calculate tax and pension information and enter data into a database and then print a payslip all in accordance with the legal framework set out in the Revenue Services ‘rules of engagement’. An application development company will take on that challenge and through a series of iterations it will deliver an application that meets all of the customer and legislative requirements. For a business that wants to make money from an application the scenario is very similar to that for an external customer. The difference is financial in that the business has to justify the cost of having developers on staff creating the application. That cost is set against a forecast of income from the eventual deployment of the application as a service for the business.
In both of the examples there are constants that can make for hard going. In the same way that technical solutions are affected by people, process and politics, so application development is affected by an isolationist practice.
Why Is This?
Across all IT from datacenter infrastructure to applications to cloud there is one problem that affects the smooth, joined-up running of a project and that is ‘silos of activity’.
The silo has long been the black mark of IT. We became so used to operating in silos that we didn’t question whether such an arrangement was productive and cost effective. In fact, even now, the majority of IT organizations operate using silos. Solutioning and development in isolation.
Solution design and application development saw the arrival of Lean and Agile as a really effective way to operate and yet, silos remained. Companies operated Agile but, kept the silo way of doing things. Strange when you think about it. Agile means flexible and able to change without trauma. Silo is a ‘pit’ with high sides that makes change very difficult. So, in essence, Agile and silo worked together and made change difficult. Still does.
Silo
Here is a real-world example of a silo-based traditional IT environment where an application is to be developed and deployed. The process may differ slightly in some companies and the job titles may not be the same but, this has been my experience working for several large IT corporations and it is recognisable as a fairly common procedure.
The Application Developer creates an application from a concept or from a request. A Technical Services (TS) Architect is asked to create a High Level Design (HLD) for the application infrastructure. The TS Architect passes the HLD to the Project Architect to review the design. The Project Architect passes the final HLD back to the TS Architect. The TS Architect explains the design to the application developer and covers off any items that are likely to compromise the application. This is usually done in isolation from other experts. The HLD is signed off buy someone or other and the Project Architect sets about carrying out a due-diligence activity prior to creating the Low Level Design (LLD or Build Doc) for the application infrastructure. The Project Architect has to visit various Subject Matter Experts (SMEs) for Compute, Network, Storage and Disaster Recovery (DR) to find out what technologies and requirements will need to be in the LLD. Details around protocols, routing, security and firewall rules can be complex and can negatively affect the application if not carefully planned. To get this right a Business Impact Analysis expert needs to be consulted to make sure that security and compliance problems, if they exist, can be dealt with or mitigated. Most applications are deployed to virtual infrastructures which require the involvement of virtualization experts to aid provisioning and automation technologies. All in all, the Project Architect has to consult with many different silos of technology/experts. In the course of this activity the Architect has to constantly return to the application developer to check that what is being planned for the infrastructure is not going to ‘damage’ the application design and make the application ineffective when deployed. Finally, the Service Wrap needs to be put in place to support the application and to meet the non-functional requirements in the Service Level Agreements (SLAs). There could easily be twenty people involved in this process. I haven’t included test and development as this usually waits until the end of the main process along with User Acceptance Testing (UAT). Sometimes there is a separate team that handles this part, sometimes it’s carried out by Operations. Application design also includes the dependency tiers that provide the middleware and database layers. It could be that many more people will need to be involved when those services are included. What is true is that each SME is part of a silo. The project has to consult all these silos. Some are helpful, some are not and there are lots of reasons why No! can be the answer to all questions and suggested solutions.
All the silos and all the people involved make the whole project slow and costly. The analogy is the game of Snakes and Ladders.
DevOps
Although the above example is somewhat crude it is a fair assessment of what application development can be like end-to-end. Everyone in the industry knows that this is the ‘normal’ state of affairs and accept that it is less than perfect. DevOps has begun to appear on the scene as the answer to the traditional silo approach. DevOps attempts to remove the silos and replace them with a collaborative and inclusive activity that is the Project. Application Development and Solution Design benefit from DevOps principles.
What needs to be done to remove silos:
- Change the working culture
- Remove the walls between teams (and you remove the silos)
Keys:
- Communication, Collaboration, Integration and Information Sharing
Easy to say and hard to do.
Most SMEs like to keep their information to themselves. Not true of all but, of many. It’s part of the traditional culture that has developed over many years. Working practices have made change difficult. Management of change is one of the most challenging tasks any company can embark on. Resistance will be resilient as it is important that people give up something to gain something. Making it clear what the gains are is imperative. People will change their attitudes and behaviours but, you have to give them really good reasons to do so. I’ve found that running multi-discipline workshops for the SMEs has proven an effective method of encouraging information-sharing and the breaking down of those ‘pit-walls’.
Explaining to the teams what DevOps is and what it is supposed to achieve is the first part of the educational process. The second is what needs to be done.
State specific, measurable objectives:
- Implement an organization structure that is ‘flat’. If we espouse horizontal scaling, why not horizontal organizations?
- Each App-Dev or Solution-Dev is a project and the team is end-to-end across the disciplines
- Implement ongoing informational exchange and reviews
- Make sure that everyone signs up to DevOps and understands the paradigm
What is DevOps
Just like the Cloud paradigm it is simply another way of doing something. Like Cloud it has different definitions depending on to whom you are speaking at the time.
Wikipedia states: Because DevOps is a cultural shift and collaboration between development and operations, there is no single DevOps tool, rather a set or “toolchain” consisting of multiple tools. Generally, DevOps tools fit into one or more categories, which is reflective of the software development and delivery process.
I don’t think that this is all DevOps is. The inference is that DevOps is concerned only with application development and operations. I do not believe that. I believe that DevOps is a paradigm and that like other IT ‘standards’ and paradigms it is relevant to all IT and not just applications. By removing the partitions between each practice in the chain and having all the key players involved from day one, as part of an inclusive and collaborative team, the cycle of application development and solution design becomes a continuous process that doesn’t have to divert to consult each required expert. No-one needs to throw a document over the wall to the next crew. Each document is written within the collaboration process and this has to make the document more relevant and powerful. Imagine that the project team is always in the same room from concept to deployment and each expert is always available to comment on and add to each step of that project. How much better than the traditional method where it can take days to get an answer to a simple question, or to even find the right person to ask.
The mantra is: Develop, Test, Deploy, Monitor, Feedback and so on. This sounds application-orientated. In fact, it can apply to the development of any IT solution. Like ITIL, TOGAF and the Seven Layer Reference Model it can be applied to any and all IT activities from development right through to support services. DevOps puts us all on the same page from the start to the finish.
Don’t allow your company to implement DevOps in isolation and only as a framework for application development. To do that would be to create another silo. Use it for every project and as the default culture for all your teams whether or not they are developers, engineers, architects or operations. And, finally, don’t complicate it. DevOps doesn’t need deep and profound definitions or long and tedious conversations about what it is and how to implement it. Just do it.
