持续集成/持续交付 Coolblue 的持续部署

laofo · 发布于 2016年9月07日 · 317 次阅读
4

作者 Ben Linders ,译者 夏雪

Coolblue 的技术开拓者Paul de Raaij提出,持续部署会得到更强的责任感和更好的部署质量。代码规范预防代码库混乱,自动化检查很合适完成冗长而无聊的检查,人工检查很合适去检查代码的逻辑和用法实际上是否成立。

De Raaij 写了一篇博文《我们软件的持续部署》,在文章中他解释了Coolblue是如何完成持续部署的。

仅仅开发软件是不够的。你还需要让它投入使用,并确保部署了高质量的软件且未对其造成破坏。

Paul de Raaij 将在敏捷和软件架构专题研讨会2016上讨论Coolblue的持续部署。InfoQ有幸采访了他,请他谈了谈Coolblue的部署、持续部署的优势和缺点、质量保证的技术和实践、代码规范、应用监控,以及Coolblue面临的挑战和他们将如何应对。

InfoQ:什么使Coolblue决定要创造条件让开发人员自己完成部署的?

Paul de Raaij:我们希望软件尽快投入使用。这能帮我们快速得到我们开发的功能的反馈意见,并使用这些反馈去了解是否需要改变方向。我们也喜欢让开发团队对他们的产品有完全的所有权。举例来说,比如发布工程师正在转移责任,其关键就在于拥有完全的所有权。如果你破坏了它,就必须由你来修复它。没有人为你擦屁股。它会使大家拥有更高的责任感,使部署有更高的质量。

InfoQ:目前部署是如何来做的?

De Raaij:很难就此给出一个简单而又全面的答案,简短概括一下:我们使用gitflow 工作流并将GIT作为分布式版本系统工具。当一名开发人员想要把他的新代码提交上线时,会创建一个pull请求。这个pull请求将会得到自动化和人工的审核。全都亮绿灯了?那么就会自动地把它打包并放到部署和验收测试服务器上。人工按一下按钮就会把它部署到生产环境。当然,这里面还有很多的内容,我将在ASAS会议上详细探讨。

InfoQ:持续交付的优势是什么?

De Raaij:它有很多的优势。对于初学者而言,如果做法正确,你就有能力把自己的软件部署到一台服务器上。通过各种检查工具的自动化检查,它将显示明确的质量水平。如果适当地设立了回归测试,你就能保障原有功能在你部署之后仍可正常工作。

InfoQ:有什么缺点吗?

De Raaij:在最后他仍然是个人工过程,是由人来配置的。它能轻松确保愉快路径,但有些东西仍有可能出错。所以在你的持续集成过程配置中有许多值得思考的地方。例如,如果你使用的在线服务(比如GitHub)出故障了,那么会对你的流程造成什么影响?

另外还要想想你的开发团队的规模。500个开发人员在同一流程中工作和只有两个人去做更新肯定会让你的持续集成流水线有完全不同的工作方式。所以,提前思考你的解决方案有怎样的可伸缩性。能否轻松地为代码检查增加额外的代理?在一个pull请求接受检查之前等待半个小时是否可行?

InfoQ:哪些技术和实践被Coolblue拿来用于质量保证?为什么?

De Raaij:我们使用了多种方法,它们也使人工和自动化检查更加多样化了。自动化检查能很好地完成冗长乏味的检查(比如linting)、代码规范的检查、回归测试的执行。人工检查很合适去检查代码的逻辑和用法实际上是否成立,实际编码的内容是否实际解决了全部的使用案例。

InfoQ:如何应用编码规范?这么做会带来什么好处?

De Raaij:编码规范非常重要,能避免你的代码库变得混乱。如果每位开发人员都按自己的缩进、间距和命名规则的话,很快就会变得一团糟。这将极大影响软件和可交付物的质量,也会令开发人员的工作难以开展,迅速产生挫败感而恼怒不已。

在我们的案例中,代码规范大多数是遵循行业规范来定义的。比如我们的PHP规范就主要遵循了PSR的规范,同时又进行了少量的微调。

InfoQ:你们是如何监控应用的?

De Raaij:很多工具都在我们的监控中发挥着作用。针对机器的技术监控,我们严重依赖Nagios作为我们的监控应用。这实际关注的是机器的状态以及应用和守护进程是否在运行。在我们的软件生态中监控极其重要,所有应用都有自己的指标,我们将它们用于健康检查、洞察力和分析。这些指标通过一个StatsD像聚合器一样发送到数据日志。除了指标,我们还会利用ELK栈深入分析日志。

InfoQ:Coolblue正在面临着什么挑战,你们将如何去应对它们?

De Raaij:完全的持续交付对于我们来说可是个大事。我们已经取得了相当的进展,但我们前方还有许多障碍,横在我们走向真正的自动化部署的路上。其中一个大的障碍是如何处理系统中的数据迁移,在我们应用的生态中它可是个大难题。如果我们解决了它,事实上就非常接近真正的持续部署了。

敏捷和软件架构专题研讨会2016将于2016年9月28日在荷兰阿纳姆的GelreDome球场召开:

ASAS是一个具体解决方案驱动的会议,在这里技术带头人、软件架构师、业务分析师和工程主管能够了解和讨论交付软件解决方案的更好的方式,使其更适合于他们的用户。

共收到 1 条回复
4
laofo · #1 · 2016年9月07日

原文如下

Written by Paul de Raaij on 20 July 2015 CONTINUOUS DEPLOYMENT OF OUR SOFTWARE

Just developing software is not enough. You’ll need to get it on production and you want to be sure that you deploy quality software and not breaking software. In this article, part of our microservices journey, I’ll describe how we have set up our deployment pipeline in a way that developers can do their own deployment complying to our standards.

LANGUAGE AGNOSTIC The deployment pipeline as we have created it is language agnostic. Which means that the process to assure quality and deploy software for a PHP application is the same as the process for a .NET application. The used tools for quality assurance and packaging might differ, but the process is equal. This ensures simplicity and reduces the cognitive load on developers to understand the process for all microservices, which might be developed in different languages.

DESIGNING THE DEPLOYMENT PROCESS When we start designing our deployment process we were looking for a solution that could support us in deploying high quality software. At that point we were already using GitHub and their pull-request system for our code versioning. What we wanted was a system that could verify the quality of the pull request, before merging it into the main repository.

If a pull request is validated and passed all checks, it should be allowed to be merged onto the master repository. All steps after the merge should be executed automatically and result eventually into a deployment on production. The diagram below displays a bird overview.

continuous-deployment.002

AUTOMATING THE PROCESS Each build step consists out of a set of actions which we define in the project itself. These actions are defined in ant build scripts and can be anything from creating a folder to calling a command line tool which does the actual inspection. These build scripts are not triggered manually. We use a build server to guide the whole build process. A build server is able to react on triggers and start the appropriate action.

For our build process we’ve decided to use TeamCity as build server. There are many (open-source) alternatives available, but TeamCity fitted our needs perfectly.

CONTINUOUS INTEGRATION This step can be triggered for several reasons. It can be the creation or update of a pull request by the developer or it is caused by the merge of a pull request on the main repository. Either way, the purpose of the step is the same. Validating if the quality of the entire repository matches our standards.

When executing this step, the build server will execute a variety of quality tests which basically are split up into two groups, automated tests and static code analyzers.

The group automated tests consists out of unit testing and functional testing. As an example you can think of tools as NUnit, PHPUnit or Behat. These tests are present to validate if the tested functionalities are still matching the expectations of the tests. They are mainly used for regression testing, which boils down to the question of “does my change do what I expect and does it not break any other functionality”.

Static code analyzers consist out of tools that generate code metrics, check coding violations. As an addition to scripting languages, linting tools like PHPLint will check if the code can be executed at all. These kind of tools do not aim on functionality, but focus on code quality itself. Is your code consistent in indentation, do you consequently use the same naming conventions or do you have a lot of duplicate code in your repository.

PACKAGING If the continuous integration step succeeds we move over to the next step, packaging. The purpose of this step is simply to create a single deliverable which can install itself onto servers.

Within Coolblue we can differentiate two types of packages. For Linux systems we use rpm packages and the RedHat package manager. Windows systems will be packaged into NuGet packages and deployed via Octopus Deploy.

We deliberately have chosen to create packages that are close to the operating system, which makes life a bit easier. For example, by choosing for the RedHat package manager we chose a well known process. Known to developers and system administrators since they all have more than basic knowledge of the yum command. But also a known process for the distribution of packages. As a RedHat user you know how repositories work, how they combine with yum and its process.

There a lot of useful tools in the open source community like Capistrano that could help with distribution of artifacts to a production environment. All of those tools have their own pros and cons. For us it felt that we would add a lot of unnecessary complexity to our pipeline and process.

Packaging and delivering .NET services is something different. There is no problem to package .NET applications into a Windows native package format, but hooking into a generic package management on Windows is not possible. So, we tackled this problem a bit differently.

.NET applications will be packaged into the NuGet package format. NuGet being an open source package manager for Windows environments. The build packages then will be distributed in the later steps. The generation and deployment of NuGet packages is worth a whole article on its own which we will publish in the future.

CONTINUOUS DELIVERY We have created a package containing our inspected code repository, now it is time to get it actually on a server so it actually can be used. This third step of the build pipeline will publish to development and accept servers.

Before I start to explain what we do exactly in this step it is wise to get a clear vision how continuous delivery differs from continuous deployment. The difference between the two is small, but significant.

Continuous delivery are all automated deployments of packages to any except production. Continuous deployment describes the automated deployment of a package to production. So during this step our goal is to get the generated package on different servers so the deliverable can be checked and is available for other team members. The deliveries done by this step might differ per project. For example, some projects might have an acceptance environment we need to publish to, others might not.

CONTINUOUS DEPLOYMENT With the continuous deployment step we reach the end of our deployment pipeline. In this step we actually deploy the application onto a production server. For Linux environments that is adding the package to our internal repository server. Windows environments will push and install the package onto the production nodes.

For us this step currently isn’t automatically triggered. A developer manually needs to trigger this build step to get his changes onto production servers. This isn’t something we want. but is necessary due to the lack of automated post-deployment tests.

In post deployment tests we want to check if the deployment went successful, something we currently do manually. For example by checking if the homepage returns a HTTP status code of 200. We want to check if we are still able to add a product to the shopping cart. Is our checkout process still available. These are the most viable tests we have after an deploy since they are critical to our business.

When we automate these post deployments tests and are able to do an instant revert of the deployment we will automate the trigger to have true continuous deployment.

In a high overview this article describes our continuous deployment process. A process which is working fine for now, but we are still working on to optimize. Objectives on our radar are for example the automated post deployment tests and implementing canary deployment.

http://devblog.coolblue.nl/tech/continuous-deployment-of-our-software/

需要 登录 后方可回复, 如果你还没有账号请点击这里 注册