Building and Customizing Coreos - Creating images for NIFTY Cloud

Building and Customizing CoreOS - Creating images for NIFTY Cloud Yuya Kusakabe - @higebu NIFTY Corp. Nov. 13, 2014 Copyright © NIFTY Corporation All Rights Reserved.

About me Twitter / GitHub: @higebu  Engineer at Nifty Corp.  Desining NIFTY Cloud virtual network services  Hacking VyOS, Packer, Zabbix and CoreOS <- New!  Copyright © NIFTY Corporation All Rights Reserved. Confidential 2

About NIFTY Cloud Launched in 2010.  IaaS, PaaS and SaaS.  Based on VMware.  Original cloud controller like OpenStack.  Copyright © NIFTY Corporation All Rights Reserved. Confidential 3

Why customize CoreOS? To provide CoreOS images on NIFTY Cloud.  We need to handle end-user cloud-config files.  Copyright © NIFTY Corporation All Rights Reserved. Confidential 4

Why build CoreOS?  Because we can not install packages on CoreOS without building. Read-only rootfs  No classic package manager  Copyright © NIFTY Corporation All Rights Reserved. Confidential 5

Building CoreOS ( not customized ) For more detail: https://coreos.com/docs/sdk-distributors/sdk/modifying-coreos/ Copyright © NIFTY Corporation All Rights Reserved.

• https://coreos.com/docs/sdk-distributors/sdk/modifying-coreos/

Install depot_tools % git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git % export PATH="$PATH":`pwd`/depot_tools For more details about depot_tools: http://www.chromium.org/developers/how-tos/depottools Copyright © NIFTY Corporation All Rights Reserved. Confidential 7

• https://chromium.googlesource.com/chromium/tools/depot_tools.git
• http://www.chromium.org/developers/how-tos/depottools

Bootstrap the SDK chroot % mkdir coreos; cd coreos % git config --global color.ui false # avoid interactive question % repo init -u https://github.com/coreos/manifest.git -g minilayout -repo-url https://chromium.googlesource.com/external/repo.git % repo sync Copyright © NIFTY Corporation All Rights Reserved. Confidential 8

Building an image % cros_sdk % cros_sdk --enter % ./set_shared_user_password.sh % echo amd64-usr > .default_board % ./setup_board % ./build_packages % ./build_image prod --group alpha # production and alpha channel % ./image_to_vm.sh --from=../build/images/amd64-usr/latest -board=amd64-usr --prod_image --format vmware # convert to VMware VMX and VMDK % exit Your image is in the following directory. “src/build/images/amd64-usr/latest/” Copyright © NIFTY Corporation All Rights Reserved. Confidential 9

Customizing CoreOS Copyright © NIFTY Corporation All Rights Reserved.

Add support for NIFTY Cloud  Add oem-niftycloud to coreos-overlay   Add niftycloud format to scripts/build_library/vm_image_util.sh   https://github.com/higebu/coreosoverlay/tree/niftycloud-494 https://github.com/higebu/scripts/tree/niftycloud-494 VMware Tools  Experimental support for open-vm-tools at 471.1.0 Copyright © NIFTY Corporation All Rights Reserved. Confidential 11

• https://github.com/higebu/coreos-overlay/tree/niftycloud-494
• https://github.com/higebu/scripts/tree/niftycloud-494

oem-niftycloud  https://github.com/higebu/coreosoverlay/tree/niftycloud-494/coreos-base/oem-niftycloud % tree coreos-base/oem-niftycloud/ coreos-base/oem-niftycloud/ ├── files │ ├── cloud-config.yml # cloud-config for NIFTY Cloud │ ├── coreos-setup-environment # set environment variables │ ├── niftycloud-coreos-cloudinit # run coreos-cloudinit with user-data │ ├── niftycloud-run-startup-scripts # run user scripts │ └── niftycloud-ssh-key # set ssh key └── oem-niftycloud-0.0.1.ebuild 1 directory, 6 files Copyright © NIFTY Corporation All Rights Reserved. Confidential 12

• https://github.com/higebu/coreos-

niftycloud format  https://github.com/higebu/scripts/blob/niftycloud494/build_library/vm_image_util.sh ## niftycloud IMG_niftycloud_DISK_FORMAT=vmdk_scsi IMG_niftycloud_DISK_LAYOUT=vm IMG_niftycloud_CONF_FORMAT=vmx IMG_niftycloud_OEM_PACKAGE=oem-niftycloud Copyright © NIFTY Corporation All Rights Reserved. Confidential 13

• https://github.com/higebu/scripts/blob/niftycloud-

open-vm-tools It’s me! Copyright © NIFTY Corporation All Rights Reserved. Confidential 14

Building CoreOS ( Customized ) For more detail: https://coreos.com/docs/sdk-distributors/sdk/modifying-coreos/ Copyright © NIFTY Corporation All Rights Reserved.

• https://coreos.com/docs/sdk-distributors/sdk/modifying-coreos/

Modify manifests/release.xml • Change repository names and revisions <project groups="minilayout" name="higebu/coreos-overlay" path="src/third_party/coreos-overlay" remote="private" revision="niftycloud494" upstream="refs/heads/master"/> <project groups="minilayout" name="higebu/scripts" path="src/scripts" remote="private" revision="niftycloud-494" upstream="refs/heads/master"/> Copyright © NIFTY Corporation All Rights Reserved. Confidential 16

Install depot_tools % git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git % export PATH="$PATH":`pwd`/depot_tools For more details about depot_tools: http://www.chromium.org/developers/how-tos/depottools Copyright © NIFTY Corporation All Rights Reserved. Confidential 17

• https://chromium.googlesource.com/chromium/tools/depot_tools.git
• http://www.chromium.org/developers/how-tos/depottools

Bootstrap the SDK chroot % mkdir coreos; cd coreos % git config --global color.ui false # avoid interactive question % repo init -u ssh://git@github.com/higebu/manifest.git -b niftycloud494 -g minilayout --repo-url https://chromium.googlesource.com/external/repo.git % repo init –m release.xml % repo sync Copyright © NIFTY Corporation All Rights Reserved. Confidential 18

• https://chromium.googlesource.com/external/repo.git

Building an image • Use niftycloud format • Use “cros_sdk --” for automation % cros_sdk --download % cros_sdk -- "./set_shared_user_password.sh" "core" % echo "amd64-usr" > src/scripts/.default_board % cros_sdk -- "./setup_board" % cros_sdk -- "./build_packages" % cros_sdk -- "./build_image" "prod" "--group" "alpha" % cros_sdk -- "./image_to_vm.sh" "--from=../build/images/amd64usr/latest" "--board=amd64-usr" "--prod_image" "--format" "niftycloud" Your image is in the following directory. “src/build/images/amd64-usr/latest/” Copyright © NIFTY Corporation All Rights Reserved. Confidential 19

NIFTY Cloud CoreOS images! Copyright © NIFTY Corporation All Rights Reserved. Confidential 20

Thank you for listening! We are hiring! http://www.nifty.co.jp/recruit/ Copyright © NIFTY Corporation All Rights Reserved.

• http://www.nifty.co.jp/recruit/

Copyright © NIFTY Corporation All Rights Reserved. Confidential 22