Continuous Lifecycle Conference 2013 Karlsruhe, Germany - An Abstract

Impressed

First of all: I'm impressed. High level quality talks, lot's of hands on, not that much theoretical discussion, nice meals. Met lots of people understanding what they are doing, heard about millions of tools to use, boiled down effective just a dozen are meeting most requirements. And at the latest watching a live demo on changing source code and after commit those changes a real one click mechanism squeezes changes through

• code checkin,
• building,
• static code analysis,
• unit tests,
• performance test,
• ui tests,deploy and
• operation

got my mind thrilled. I WANT THAT TOO! (uppercase is right here - I want to shout it out!). And by the way, restrictions on how fast this pipeline runs through is limited by computing resources. I'm sure you get your new feature out to production within 30 minutes.

Find more on Continuous Lifecycle 2013 homepage or Twitter #ConLi2013 @APGlaeser

The continuous whatever - just another hype?

These were my thoughts too, but what I saw, as a newby on this, were well integrated tools and automatic processes enabling developers still to code but also analyze, test, stage, deploy and monitor new software components. Thats fine, and some of those already sounds revolutionary. But I'm not finished. Developers and operators are permanently sharing the same desk and developers writing their own code for automatic provisioning of needed infrastructure - wo-hoooow, think for testing purposes only? No, for production too!

Ok ok - let's calm down...

As far as I understood - and I saw lots of reliable examples - a continuous pipeline from team composition, coding, versioning, testing and deploying gets your new features out to customers - fast! For this, surely, some things have to be done. And as I understand, there're fundamental changes to organization, tooling and how teams work together required.

But what's the difference between continuous integration (CI) and continuous delivery (CD)? Quite simple:

CI = CD minus D(elivery)

CI does not deliver - thats it. But in detail there're certain opinions on whether delivering to test and integration environments is part of CI. Surely delivering to production is not. Thats part of CD.

Let's get practical

The picture

My favorite picture of CI/CD. All PMs stay tuned - time to market is involved!

Do small releases with potential small impact capability and small risks.

The massive part

Everyone involved in CI/CD has to dive in

PO/PM

• can't scratch surface to define new feature requests
• must dive into details to get common understanding with developers of what a feature is to customers

Developers

• are much more involved in QA, deploy and operations than just throwing buckets of software over the wall
• are experts for tools used to provide CI/CD
• do production deployments on their own

QA

• [QA -eq developer] (unix shell notation)
• (QA == developer) (Java notation)
  No mistake. Developers are QA. Running a separate QA is fine for squeezing out the most of your software. With an established CI/CD QA gets more and more bored and out of work. Maybe some hard to automate UI tasks will be the tiny rest of what is left.

Operators

• are daily involved in development process
• are no more the nerds from basement getting thrown pieces of software or requirements over the wall (while all running away after doing that)

Management

• forget to implement CI/CD without their support

Methods: Scrum

• has to be adjusted to real needs - example: some say, scrum master role must die, some say each scrum team needs its own scrum master

Methods: Developer efficiency monitoring

• No more Excel - push more features in less time with more quality

The facts

• You're leader in your business? It's nearly impossible to be innovator of your own business!
• Software development is driven by customers - neither by processes, tools/technologies used nor ideology
• Rollback is the natural enemy of CI/CD
• Feature ready and testet? Deploy! Forget about (fixed) release cycles
• Nightly deploys? Forget about - do it just before lunch
• Backlog rule: 95% of backlog must not depend on other modules
• Split up monolithic software to small pieces, where each piece is deployable separately
• CI/CD has no compelling answers on how to integrate database changes to structure and content into pipeline
• Releases are non-events

The Teams

Team evolution stage 1

Teams. That's it. No interaction, no knowledge mixture

Team evolution stage 2

Mixed teams. No interaction, but knowledge mixture

Team evolution stage 3

CI teams. Interaction + knowledge mixture

Be careful when thinking about teams. Talking about CI/CD we're not only dealing with several development team, we're dealing with QA, operations and PO/PM too.

A special relationship is treated seperately when talking about CI/CD: Development and Operations. So a special "movement" comes up dealing with issues around DevOps. Find a good description of DevOps at codecentric. Find a simplified as follows:

Establish traffic light systems or team monitors to show everyone current build, verify, test or deploy status. People start asking if Jenkins stays red for 2 days. Do the same for monitoring showing applications current health. Hello to management - do a permanent coaching. Hire specialists for tooling and methods for a given time period to support all involved teams hands on.

Maybe developers have to move a bit more than operatores but both must be open minded for each others work. Developers and operators must share same desk instead of working side on side.

The tools

The most common used tools providing a CI/CD pipeline are:

Tool	What For
Jenkins	CI server
Git	versioning
Sonar	static code analysis
jUnit	unit tests
jBehave	acceptance tests
Selenium	UI tests
jMeter	performance tests
ZAProxy	security tests
Puppet	provisioning
Artifactory	artefact repository
Maven	build server
Liquibase	DB versioning

Defining a CI/CD pipeline which fits your needs and afterwards orchestrating the tools to do what you want is surely one of the most exciting tasks. Start small, but think big.

The time

Start small! Everybody gave this advice. Organize one or two small teams as a pilot for implementing CI/CD. Start with a small real life project/product/release, with their own cycles, methods and infrastructure. Best methos: start from zero. OTTO (otto.de) did so starting to implement their own online shop software as replacement of Intershop Infinity. They succeeded after two years.

The methods

People who established CI/CD within their organisation on a high level of maturity tending to switch from Scrum to Kanban. But starting with Scrum when thinking about implementation of CI/CD is quite a good idea.
Next idea: No rollbacks. Forward development. With CI/CD rollbacks are obsolete. When deploying small changes the effort for a rollback exceeds the effort for just fixing the failed deploy of a change.

The testing

Take a look back to "The tools". Do you think, testing is fundamental to CI/CD?
Treat testing as a backbone if CI/CD. Even before thinking to write a line of code you should have written 2 tests for it. Starting with high integrated unit tests, going through acceptance and GUI tests up to performance and even security tests. All automated. Nice.

The rest and unanswered questions

Doing a vertical dive into discussions about branching would end up with its own book. Branches by feature or release? Or better by task? No best practice on this during conference. As well as lack of information about database versioning and migration. In fact, doing a deploy of a piece of software is not that hard, but what, if database content and/or structure changes? How to deploy without interruptions? Even a blue-green deployment is no solution. And I can not imagine to push whole servers to versioning tools like Git or SVN as some supposed to do. Even the idea behind immutable infrastructure leaves some gaps - how to implement a server as immutable if parameters needs to be changed during incident analysis or just simple support requests?