2010年12月30日 星期四

持續整合樣式:剩下的 Patterns

Dec. 30 20:18~20:39

[續上兩篇的內容] 在 paper 中一共提到 11 個 patterns(參考下表),前兩天 Teddy 介紹(其實也不算介紹,只是把內容貼出來)了其中兩個,今天把剩下的用 Problem-Solution 的簡短形式全部介紹完畢。相信大部分的鄉民們應該都沒把內容看完... 如果是換作 Teddy,應該也是看不下去... XD... 但是切記這幾篇的目的是『作帳』,對於內容是否過於文言文就不要太計較啦。

Project Installation Project, Single Shared Library Project, Interface Project, Cross-Platform Project, Native Project, Patch Project
Build Local Private Build, Remote Private Build, Cross-Platform Integration, Integration Workflow
Good Habit Single Responsible Person

Interface Project


Problem: How to isolate platform and implementation specific code from platform-independent projects? 
Solution: Define programming interfaces to encapsulate platform-dependent services or to allow a service to have multiple implementations. Put the interface definition code in an interface project. Do not mix interface definition code with interface implementation one. “Programming to an interface, not an implementation” and “dependence injection” are well-know software development practices that require clearly defined interfaces. For cross-platform software, strictly separating interfaces from the implementation hides platform-dependent details and enhances the flexibility and maintainability of software systems.


Cross-Platform Project


Problem: How to organize platform-independent code?

Solution: Classify implementation code into two types: platform-dependent and platform-dependent. Put platform-independent code in cross-platform projects. From the continuous integration perspective, cross-platform projects are easier to build and test. They can be built and tested by using virtual machines such as VMWare, VirtualBox, and Xen. Developers can quickly verify cross-platform projects by making a Local Private Build before the projects are committed for integration.


Native Project


Problem: How to organize platform-specific code?

Solution: Classify implementation code into two types: platform-dependent and platform-dependent. Put platform-dependent code in native projects. Native projects can be built on virtual machines if they depend on operating systems rather than on hardware. For hardware dependent native projects, a Remote Private Build can be used before they are committed for integration.

Patch Project


Problem: How to produce quick fixes to resolve problems without reinstalling the whole software?

Solution: Create a patch project to generate patch files. One way to produce a patch file is to perform a “diff” on the two different software versions and package the difference in the patch file. The patch file also contains a script to upgrade necessary files on the system to be patched.


Local Private Build


Problem: How to prevent a broken build?

Solution: Build a project locally before it is committed. You can make a local build by pressing a build button in your IDE or by running a pre-defined build script. Use the same build script executed by the continuous integration system to make a local build can further reduce the possibility of causing a broken build. 


Remote Private Build


Problem: How to prevent a broken build?

Solution: Use a remote agent to build a project before it is committed. The remote agent’s platform provides an environment that is required to build the project. This pattern is usually used to verify a Native Project that requires a build environment that is different from the local development environment. Continuous integrations systems such as JCIS, CruiseControl, Hudson, Buildbot, TeamCity, and Bamboo support this pattern.


Cross-Platform Integration

Problem: How to build cross-platform products?

Solution: Use a continuous integration system that supports cross-platform integration. Apply Integration Workflow to guide the continuous integration system to dispatch projects on suitable remote platforms for integration. Continuous integration systems supporting this pattern include JCIS, CruiseControl, Hudson, Buildbot, TeamCity, and Bamboo.



Integration Workflow


Problem: How to manage the continuous integration process for a cross-platform product containing interdependent projects?

Solution: Design integration workflows to control the continuous integration process. Two types of essential integration workflows are concerned: intra-project and inter-project. The former decides which integration activities (e.g., compilation, testing, test coverage analyzing, and packaging) to be included in a build and the execution order of each integration activities. The latter decides the build order of all projects according to the relations of project references. 


Single Responsible Person


Problem: How to deal with a broken build?

Solution: Assign a person in your team who will be notified once a build was broken. This pattern prevents broken builds from being ignored; especially those caused by overnight build scripts.


看完這三篇並寫出心得者請向 Teddy 兌換綠色乖乖一包。

友藏內心獨白:To 某學弟 -- 『終於有新文章了,啊你有認真在看嗎?』

2010年12月29日 星期三

持續整合樣式:Single Shared Library Project

Dec. 29 20:41~21:02

昨天才提到 Installation Project 這個 pattern,今天 Teddy 在 Continuous Integration 這本書的 191 頁剛好看到一句話,可以呼應 Installation Project 的重要性 。這句話是這樣滴:

If you can't release your software, then it's almost as if it doesn't exist.

噯呀,辛辛苦苦熬夜爆肝所寫出來的程式如果沒有 release ,就好像女方好不容易懷孕但是男方不肯負責,只好去『夾娃娃』,而這個小生命最後等於不存在一樣的那種感覺。

揮一揮鍵盤,只帶走一個月 N 千塊的研究助理費,這不是 太棒  太可惜了嗎!所以,程式寫幾行不是重點,前人留給你多少 bugs 也不是重點,重點是軟體在你手中有沒有(能不能) release。

今天接這看 paper 裡面個另一個 pattern: Single Shared Library Project 。

Single Shared Library Project

Context: A software product is structured as a number of projects. These projects share some common libraries. For example, a software called ezMonitor includes both Java and C++ projects. In the projects written in Java, Apache Log4J is used. You also develop two common libraries for your application: FileCommons for file manipulation and NetworkCommons for network operation. Some of the ezMonitor projects make use of both libraries; some others make use of one of them and still others make use of none. One way to share the libraries is to manually copy them to the project workspace. However, doing so complicates the version control of the share libraries. For example, once a shared library needs to be upgraded, each of the local copies needs to be upgraded as well. This can be error-prone and tedious for a product that has a number of projects sharing common libraries. 
Problem: How do we make share common third-parity and in-house libraries among projects easy?

  • You want to use a single version of a shared library among the projects that use the library.
  • Shared libraries, either third-parity or in-house, are likely to change over time.
  • The version control mechanism for shared libraries should be as easy as possible. 
Solution: Create a project that acts as a centralized repository for shared libraries. Organize the project structure in a way that third-parity libraries, in-house libraries, native (platform-dependent) libraries as well as system drivers, and source code of the libraries (if available) are placed in different folders. Define rules for each type of the libraries: (1) how should these libraries be referenced by other projects; (2) when and who can update a new version of a library; and (3) how does the library update process get initiated (e.g., manually or automatically).

Figure 2. An example of a Single Shared Library Project

Fig. 2 shows an example to organize a Single Shared Library Project named SharedLibraries. The SharedLibraries project acts as a centralized repository for a cross-platform product written in Java and C/C++. There are six different folders in the SharedLibraries project:
  • driver: This folder contains system drivers, primarily for Windows operating systems. Usually, a Windows system needs to be rebooted to enable the driver.
  • native: This folder contains platform-dependent libraries (usually .dll files for Windows and .so files for Linux-based systems) or executable utility programs (e.g., the cfg_x64 program in Fig. 2).
  • in-house: This folder contains the binary code of common in-house libraries that will be used by other projects. For example, if the MyServer project needs to use the FileCommons.jar file, it does so by directly referencing FileCommons.jar from the SharedLibraries. Whenever the FileCommons.jar is updated to a newer version, the MyServer project (and all projects that use the FileCommons.jar) can get the latest version of the FileCommons.jar by simply issuing an “update” command to sync its local data from the code repository.
  • in-house-src: This folder contains the source code of common in-house libraries. The main purpose of placing library source code in the SharedLibraries project is for debugging and tracing. When everything goes well, developers usually do not want to know the internal implementation of a library. However, when things go wrong and an exception is raised, developers may need to locate the problem by following the stack trace. In this situation, jumping into the source code along the stack trace is necessary. If the stack trace contains an invocation to a method in the library, developers need its source code to explore the method implementation. An IDE (Integrated Development Environment) like Eclipse allows developers to link the source code of a library to its binary code so that Eclipse can jump into the implementation of the library while debugging.
  • 3party: The function of this folder is similar to the in-house folder except that this folder contains libraries from third-parties.
  • 3party-src: The function of this folder is similar to the in-house-src folder except that this folder contains source code of third-party libraries.

Resulting Context: One drawback of sharing libraries via a Single Shared Library Project is the “timing” issue when this pattern is applied in IDEs. Suppose that you use the Eclipse IDE to modify code in project A and project B at the same time and project A uses code in the project B. If project A references code in the project B via a Single Shared Library Project, modification in project B cannot be reflected immediately to project A. Modification in project B can be reflected in project A only after (1) a jar file (i.e., the library of project B) is generated and committed to the Single Shared Library Project; (2) project A syncs with the Single Shared Library Project. Thus, if a set of projects is close related, a Single Shared Library Project may not be suitable. In this situation, use project references in your IDE instead. A project reference example is shown in Fig. 3. In the continuous integration system site, the Single Shared Library Project can still be used since project A is built after project B according to their dependency relationships

Figure 3. A screenshot of the Eclipse project reference.

Related Patterns: The build outcomes of Platform Independent Projects, Interface Projects, and Native Projects are stored in a Single Shared Library Project.


 友藏內心獨白:花不到 30 分鐘又寫了一篇,YES。

2010年12月28日 星期二

持續整合樣式:Installation Project

Dec. 28 22:48~23:11

2010 年就快結束了,12 月到目前為止只寫了一篇,不及格。這原因又是說來話長,還是不說了,藉口講太多 Teddy 都有點心虛了。聽說一般的公司年底都有所謂的『作帳行情』,就算是業績不好看也要想辦法在帳面上動點手腳,粉飾一下太平。以上所說,正是 Teddy 接下來要做的事情。

前一陣子 Teddy 和指導教授寫了一篇關於持續整合的論文,想想 Teddy 也有好一陣子沒談這個主題了,所以就拿點論文中的內容來充數一下,讓本月篇數帳面上不要太難看。這篇論文的題目叫做:

Emerging Patterns of Continuous Integration for Cross-Platform Software Development

這篇論文一共提到 11 個關於跨平台軟體開發的持續整合樣式,其中有兩個樣式有列出詳細的內容,其餘的以 problem-solution 的簡短形式呈現。今天先看兩個詳細樣式中的第一個:Installation Project.

Installation Project

Context: Traditionally, the deployment of a software product is considered at the very late stage of the software development, usually before release. For a project that applies incremental and iterative processes as well as continues integration, working software is expected to be ready at the end of iteration. The working software may be deployed internally for testing and for demonstration. It may also be deployed on the customer site for trial use. Since multiple target platforms are involved in a cross-platform software product, the deployment task is made much more difficult. 
Problem: How to provide an easy-to-use and robust method for users to deploy a cross-platform software product?

  • A software product can fail to attract users if it cannot be easily installed.
  • Different operating systems have their own flavors in installation modes, ranging from text-based to GUI-based modes and using different formats such as the Microsoft MSI and the RedHat RPM.
  • You want to simplify the installation procedures and minimize possible deployment problems.

Solution: At an early stage of software development, create an installation project to produce an installer for the cross-platform application under development. Add the project to your CI system. When the CI system builds the installation project either automatically or by request, an installer is produced. Release software at the end of iteration has become a norm for projects applying incremental and iterative development. Thus, it is reasonable to consider the installation problem at the early stage of a product development so that running software can be provided via the installer. Once the installation project can produce an installer, find a way to automate the testing of the installer on all supported operating systems.

Many commercial and open source tools are available for creating installation programs. Some of them support the creation of cross-platform installers while the others are platform-specific. An example of the former category is InstallAnywhere. By using InstallAnywhere, an Installation Project can produce multiple installers for multiple platforms. For example, InstallAnywhere can produce .exe installers for Windows platforms and .bin installers for Linux-based platforms. The installers can be configured in a way that they run in the GUI mode on Windows platforms and in the text mode on Linux-based platforms. InstallAnywhere also supports a silent installation mode. When an installer is run in the silent mode, users need not (and cannot) to interact with the installer to modify the default behavior of the installer (e.g., to change the installation folder). Instead, the installer accepts a pre-defined configuration file that contains all necessary settings to guide the installer.

Developers can also use platform-specific tools to produce platform-specific installers. For example, InstallShield is a popular tool that prodcues .msi installers for the Windows platforms. For Linux-based systems such as RedHat and Ubuntu, tools to package .rpm and .deb modules are used.

It is also possible to package a cross-platform product as the so-called green software, portable applications, or portable software if the product does not contain low-level native drivers which must be installed in a specific location and does not require the systems to reboot. Green software does not require a sophisticated installation procedure. To use the software, merely copy the software (usually an executable file or a folder) or unzip it to a hard disk or flash disk.

Resulting Context: Using sophisticated tools like InstallAnywhere is very convenient to produce cross-platform installers. However, such a tool is usually very expensive. Also, the produced installers may be larger in size than those produced by platform-specific installation tools. Sophisticated installation tools that support complicated scripts and customization functions produce flexible installers, but can lead developers to experience a sharp learning curve and consequently put off the creation of an Installation Project.

Installation Projects produce installers to deploy a software product from scratch. To install a new version of the software with the installers, the old one usually needs to be uninstalled first and then the new one can be installed. If only a minor part of the product is revised, using Installation Projects to produce an installer to upgrade the software may be taxing. In this situation, use a Patch Project instead.

Related Patterns: Installers package components produced by Platform Independent Projects, Interface Projects, Native Projects, and other components made available through a Single Shared Library Project. Apply Cross-Platform Integration to test an installer on all supported platforms. 


整理持續整合樣式的工作目前 Teddy 還在『持續』進行中,歡迎沒有轉台又能夠把上面內容看完且眼睛沒有扭到的鄉民們給 Teddy 一點建議。

友藏內心獨白:為什麼領錢的都是學弟而寫 paper 的都是『老』學長和老師?

2010年12月26日 星期日


Dec. 26 17:21~19:24

今天到華山藝文中心看幾米的展覽,2009年在華山藝文中心也有幾米的展覽,當時 Teddy 是在展期結束之後才知道有這個展覽,殘念。早上搭捷運來到了熟悉的『忠孝新生站』,從忠孝國小出口走到華山藝文中心大概 10 分鐘左右。到了華山藝文中心之後看到這個就知道是展場的入口:

首先買票入場,單人票 250元, 雙人票450元,四人票880元。看完之後保證物超所值。

除了門票以外,還可以選擇一張上面印有一組號碼的紙卡(五選一),上面這組號碼待會可以用 iPad 留言以及買紀念品打折。

館內是可以拍照的,但是不可以使用閃光燈。入館之後首先看到的是這個由華碩主機版所作成的展品,很有趣。整個展覽的小房間模擬成『雪屋』(當然是用保麗龍球模擬,不是真的雪),小朋友進去超開心(這也是悲劇的開始)。還好 Teddy 去的早,不然就差點看不到。為什麼?耐心看下去...

以下是兩張近拍的照片,用 SATA 接頭當作摩天輪超有意思。



來到另一個房間,有幾張很有 fu 的照片。


接下來就是讓你拍照拍到手斷掉的幾米插畫。Teddy 當場聽到好幾個鄉民說相機沒電了,此時 Teddy 想到一句成語:『相機用時電恨少』。還好 Teddy 有多帶一顆電池出門,否則就要學某位鄉民,把沒電的電池拿出來用手搓一搓看看能不能再多拍幾張(Teddy 內心獨白:賭神看太多了嗎?人家搓牌你老兄搓電池)。

在展場中央有一個比較大的房間,裡面有幾台 iPad 可以在上面留言,待會離開之後在展場出口就可以看到你自己的留言。可以順便在這個地方休息一下。PS:這個留言的軟體好像有 bugs,一直說『ID 卡號錯誤,請重新輸入』,但是又沒有顯示可以重新輸入的地方。


Teddy 又繞回『華碩雪屋』,裡面鬧哄哄的,剛好看到好幾個小朋友在地上玩得不逸樂呼。不久之後,『華碩雪屋』熄燈了。據工作人員氣憤的表示,展覽品被人『蓄意破壞』,正在維修(工作人員內心獨白:來人啊,關門放狗)。下面第二張照片就是工作人員摸黑搶救展品,不知道附近的華碩皇家維修中心是否有可以幫忙... XD。

Teddy 替沒看到『華碩雪屋』的鄉民們感到遺憾,這個展覽廳很棒的說。PS:Teddy 並不知道『華碩雪屋』故障和這群小朋友有沒有關連,也有可能是剛好在裡面的其他鄉民不造成的。總之,好像這批人離開之後不久雪屋也剛好熄燈了。


離開展場馬上看到賣吃的地方,Teddy 點了拿鐵和焦糖瑪奇朵,還不錯喝(肚子還很飽,所以沒有點吃的)。賣咖啡的旁邊就有一個寬敞的地方可以邊喝咖啡,邊看鄉民們的留言。

最後還有賣紀念品的地方 Teddy 忘了拍。紀念品有點貴,Teddy 買了幾張明信片和一盒拼圖。

這個展覽從 2010-12-22 ~ 2011-03-27 ,裡面好像有 10 個還是 12 個不同的房間,各有不同的展品。非常推薦喔。

友藏內心獨白:用 8M/640K ADSL 上載照片好慢啊,難道是到了換光世代的時候了嗎?