ようへいの日々精進XP

よかろうもん

terraform の Providor Docker を色々と弄った雑なメモ

docker terraform

terraform の Provider から Docker Provider を試してみる。

なぜか terraform の開発環境を一応を整えておく

go を最新版に...

  • 最新の go を用意
$ cd /usr/local/src
$ sudo wget https://storage.googleapis.com/golang/go1.4.2.linux-amd64.tar.gz
$ sudo tar -zxf go1.4.2.linux-amd64.tar.gz
$ sudo mv go /usr/local

GOPATH と GOROOT を設定

  • GOPATH と GOROOT の設定
$ mkdir $HOME/dev/go
$ export GOPATH=$HOME/dev/go
$ export GOROOT=/usr/local/go
$ export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
  • 一応、確認
# バージョン確認
$ go version
go version go1.4.2 linux/amd64

# go env 確認
$ go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/vagrant/dev/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0"
CXX="g++"
CGO_ENABLED="1"

terraform のソースを git clone する

$ git clone https://github.com/hashicorp/terraform.git $GOPATH/src/github.com/hashicorp/terraform

ビルドするかもしれないので依存パッケージを取得しておく

万が一 terraform をイチからビルドしたくなった時の為にビルドに必要な依存パッケージは取得しておく

$ cd $GOPATH/src/github.com/hashicorp/terraform
$ make updatedeps

最新の terraform をビルドする

$ cd $GOPATH/src/github.com/hashicorp/terraform
$ make
$ make dev

上記を実行することで $GOPATH/bin/ 以下に最新の terraform が展開される。

ということで terraform の Docker Providor を少し弄る

terraform のバージョン

$ ./terraform -version
Terraform v0.5.3

とりあえず tf ファイル

とりあえずは以下のドキュメントを参考に...。

www.terraform.io

丸パクリ。

# Configure the Docker provider

# Docker API が有効な環境では API のホストを指定するが、デフォルトでは unix:/run/docker.sock を利用する
#provider "docker" {
#    host = "tcp://127.0.0.1:1234/"
#}

# Create a container
resource "docker_container" "foo" {
    image = "${docker_image.centos.latest}"
    name = "hoge"
    command = ["python", "-m", "SimpleHTTPServer", "1919"]
}

resource "docker_image" "centos" {
    name = "inokappa/centos-base:latest"
}

terraform plan からの apply

  • terraform plan
$ ./terraform plan
Refreshing Terraform state prior to plan...

docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)

The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ docker_container.foo
    bridge:           "" => "<computed>"
    command.#:        "" => "4"
    command.0:        "" => "python"
    command.1:        "" => "-m"
    command.2:        "" => "SimpleHTTPServer"
    command.3:        "" => "1919"
    gateway:          "" => "<computed>"
    image:            "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
    ip_address:       "" => "<computed>"
    ip_prefix_length: "" => "<computed>"
    must_run:         "" => "1"
    name:             "" => "hoge"
  • terraform apply
$ ./terraform apply
docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)
docker_container.foo: Creating...
  bridge:           "" => "<computed>"
  command.#:        "" => "4"
  command.0:        "" => "python"
  command.1:        "" => "-m"
  command.2:        "" => "SimpleHTTPServer"
  command.3:        "" => "1919"
  gateway:          "" => "<computed>"
  image:            "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
  ip_address:       "" => "<computed>"
  ip_prefix_length: "" => "<computed>"
  must_run:         "" => "1"
  name:             "" => "hoge"
docker_container.foo: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate
  • docker ps で確認
$ docker ps
CONTAINER ID        IMAGE                                                                     COMMAND                CREATED             STATUS              PORTS               NAMES
9d9b27f291e4        1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139:latest   "python -m SimpleHTT   3 minutes ago       Up 3 minutes                            hoge

一応、コンテナ出来てる。

Providor Docker で指定可能なリソースを幾つか試す

以下のドキュメントに従って試してみる。

www.terraform.io

以下は docker run のヘルプ。

 docker run --help

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Run a command in a new container

  -a, --attach=[]            Attach to STDIN, STDOUT or STDERR
  --add-host=[]              Add a custom host-to-IP mapping (host:ip)
  -c, --cpu-shares=0         CPU shares (relative weight)
  --cap-add=[]               Add Linux capabilities
  --cap-drop=[]              Drop Linux capabilities
  --cgroup-parent=           Optional parent cgroup for the container
  --cidfile=                 Write the container ID to the file
  --cpuset-cpus=             CPUs in which to allow execution (0-3, 0,1)
  -d, --detach=false         Run container in background and print container ID
  --device=[]                Add a host device to the container
  --dns=[]                   Set custom DNS servers
  --dns-search=[]            Set custom DNS search domains
  -e, --env=[]               Set environment variables
  --entrypoint=              Overwrite the default ENTRYPOINT of the image
  --env-file=[]              Read in a file of environment variables
  --expose=[]                Expose a port or a range of ports
  -h, --hostname=            Container host name
  --help=false               Print usage
  -i, --interactive=false    Keep STDIN open even if not attached
  --ipc=                     IPC namespace to use
  -l, --label=[]             Set meta data on a container
  --label-file=[]            Read in a line delimited file of labels
  --link=[]                  Add link to another container
  --log-driver=              Logging driver for container
  --lxc-conf=[]              Add custom lxc options
  -m, --memory=              Memory limit
  --mac-address=             Container MAC address (e.g. 92:d0:c6:0a:29:33)
  --memory-swap=             Total memory (memory + swap), '-1' to disable swap
  --name=                    Assign a name to the container
  --net=bridge               Set the Network mode for the container
  -P, --publish-all=false    Publish all exposed ports to random ports
  -p, --publish=[]           Publish a container's port(s) to the host
  --pid=                     PID namespace to use
  --privileged=false         Give extended privileges to this container
  --read-only=false          Mount the container's root filesystem as read only
  --restart=no               Restart policy to apply when a container exits
  --rm=false                 Automatically remove the container when it exits
  --security-opt=[]          Security Options
  --sig-proxy=true           Proxy received signals to the process
  -t, --tty=false            Allocate a pseudo-TTY
  -u, --user=                Username or UID (format: <name|uid>[:<group|gid>])
  --ulimit=[]                Ulimit options
  -v, --volume=[]            Bind mount a volume
  --volumes-from=[]          Mount volumes from the specified container(s)
  -w, --workdir=             Working directory inside the container

test.tf（1）

とりあえずシンプルにコンテナを起動してホスト名を出力し続ける tf ファイル。

resource "docker_container" "hoge" {
    image = "${docker_image.centos.latest}"
    name = "hoge"
    hostname = "hoge"
    command = ["/bin/sh", "-c", "while true ; do sleep 1; hostname -s ; done"]
}

resource "docker_image" "centos" {
    name = "inokappa/centos-base:latest"
}

apply してみる。

$ ./terraform apply
docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)
docker_container.hoge: Creating...
  bridge:           "" => "<computed>"
  command.#:        "" => "3"
  command.0:        "" => "/bin/sh"
  command.1:        "" => "-c"
  command.2:        "" => "while true ; do sleep 1; hostname -s ; done"
  gateway:          "" => "<computed>"
  hostname:         "" => "hoge"
  image:            "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
  ip_address:       "" => "<computed>"
  ip_prefix_length: "" => "<computed>"
  must_run:         "" => "1"
  name:             "" => "hoge"
docker_container.hoge: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

docker ps でコンテナ出来ているか確認する。

$ docker ps
CONTAINER ID        IMAGE                                                                     COMMAND                CREATED             STATUS              PORTS               NAMES
8988c724f4c3        1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139:latest   "/bin/sh -c 'while t   6 minutes ago       Up 6 minutes                            hoge

一応、出来ているようなので、ホスト名を出力し続けているはずなので確認してみる。

$ docker attach hoge
hoge
hoge

hoge が出力されているっすね。

test.tf（2）

次にポートを晒してみたいので以下のような tf ファイルを用意する。

resource "docker_container" "hoge" {
  image = "${docker_image.centos.latest}"
  name = "hoge"
  hostname = "hoge"
  command = ["python", "-m", "SimpleHTTPServer", "1919"]
  ports = "1919"
}

resource "docker_image" "centos" {
    name = "inokappa/centos-base:latest"
}

terraform plan してみる。

$ ./terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * docker_container.hoge: ports: should be a list

ports は list で書けとのこと。terraform のシンタックスについては以下を参考に...

www.terraform.io

list については以下のように言及されているので修正する。

Lists of primitive types can be made by wrapping it in []. Example: ["foo", "bar", 42].

以下のように修正。

resource "docker_container" "hoge" {
  image = "${docker_image.centos.latest}"
  name = "hoge"
  hostname = "hoge"
  command = ["python", "-m", "SimpleHTTPServer", "1919"]
  ports = ["1919"]
}

resource "docker_image" "centos" {
    name = "inokappa/centos-base:latest"
}

改めて terraform plan してみる。

$ ./terraform plan
There are warnings and/or errors related to your configuration. Please
fix these before continuing.

Errors:

  * docker_container.hoge: session shutdown
panic: Unknown kind: string
2015/06/27 13:23:19 terraform-provider-docker:
2015/06/27 13:23:19 terraform-provider-docker: goroutine 20 [running]:
2015/06/27 13:23:19 terraform-provider-docker: github.com/hashicorp/terraform/terraform.(*ResourceConfig).get(0xc2080a2c60, 0xc2080a5060, 0x10, 0xc2080a2d80, 0x0, 0x0, 0x1)

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

な、なんだこれは...。

ということで、ドキュメントをちゃんと読むとそもそも ports は以下のように記載する。

  ports {
    internal = 1919
    external = 19190
    ip = (option)
    protocol= (option)
 }

改めて、以下のように tf ファイルを修正からの...

resource "docker_container" "hoge" {
  image = "${docker_image.centos.latest}"
  name = "hoge"
  hostname = "hoge"
  dns = ["8.8.8.8", "8.8.8.4"]
  command = ["/bin/sh", "-c", "hostname -s > /tmp/index.html ; cd /tmp/ ; python -m SimpleHTTPServer 1919"]
  ports {
    internal = 1919
    external = 19190
 }
}

apply してみる。

docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)
docker_container.hoge: Creating...
  bridge:                   "" => "<computed>"
  command.#:                "" => "3"
  command.0:                "" => "/bin/sh"
  command.1:                "" => "-c"
  command.2:                "" => "hostname -s > /tmp/index.html ; cd /tmp/ ; python -m SimpleHTTPServer 1919"
  dns.#:                    "" => "2"
  dns.3817307869:           "" => "8.8.8.8"
  dns.3929092854:           "" => "8.8.8.4"
  gateway:                  "" => "<computed>"
  hostname:                 "" => "hoge"
  image:                    "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
  ip_address:               "" => "<computed>"
  ip_prefix_length:         "" => "<computed>"
  must_run:                 "" => "1"
  name:                     "" => "hoge"
  ports.#:                  "" => "1"
  ports.156446557.external: "" => "19190"
  ports.156446557.internal: "" => "1919"
  ports.156446557.ip:       "" => ""
  ports.156446557.protocol: "" => "tcp"
docker_container.hoge: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

イケた、イケたので...

$ curl localhost:19190
hoge

ちゃんとアクセス出来ることを確認。 尚、外部ポート（external）のアサインを Docker に任せたい場合には以下のように external = 0 を記載する。

resource "docker_container" "hoge" {
  image = "${docker_image.centos.latest}"
  name = "hoge"
  hostname = "hoge"
  dns = ["8.8.8.8", "8.8.8.4"]
  command = ["/bin/sh", "-c", "hostname -s > /tmp/index.html ; cd /tmp/ ; python -m SimpleHTTPServer 1919"]
  ports {
    internal = 1919
    external = 0
  }
}

terraform apply すると以下のように自動的に外部ポートがアサインされている。

$ docker ps
CONTAINER ID        IMAGE                                                                     COMMAND                CREATED             STATUS              PORTS                     NAMES
dbd2553d625d        1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139:latest   "/bin/sh -c 'hostnam   2 minutes ago       Up 2 minutes        0.0.0.0:32770->1919/tcp   hoge

更に複数のポートを晒したい場合には以下のように指定する。

  ports {
    internal = 1919
    external = 0
  }
  ports {
    internal = 1920
    external = 0
  }

terraform apply してみると...

$ ./terraform apply
docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)
docker_container.hoge: Creating...
  bridge:                    "" => "<computed>"
  command.#:                 "" => "3"
  command.0:                 "" => "/bin/sh"
  command.1:                 "" => "-c"
  command.2:                 "" => "hostname -s > /tmp/index.html ; cd /tmp/ ; python -m SimpleHTTPServer 1919 "
  dns.#:                     "" => "2"
  dns.3817307869:            "" => "8.8.8.8"
  dns.3929092854:            "" => "8.8.8.4"
  gateway:                   "" => "<computed>"
  hostname:                  "" => "hoge"
  image:                     "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
  ip_address:                "" => "<computed>"
  ip_prefix_length:          "" => "<computed>"
  must_run:                  "" => "1"
  name:                      "" => "hoge"
  ports.#:                   "" => "2"
  ports.1763054579.external: "" => "0"
  ports.1763054579.internal: "" => "1920"
  ports.1763054579.ip:       "" => ""
  ports.1763054579.protocol: "" => "tcp"
  ports.780279467.external:  "" => "0"
  ports.780279467.internal:  "" => "1919"
  ports.780279467.ip:        "" => ""
  ports.780279467.protocol:  "" => "tcp"
docker_container.hoge: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

以下のように複数のポートを晒すことが出来ている。

$ docker ps
CONTAINER ID        IMAGE                                                                     COMMAND                CREATED              STATUS              PORTS                                              NAMES
34b974db655d        1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139:latest   "/bin/sh -c 'hostnam   About a minute ago   Up About a minute   0.0.0.0:32771->1919/tcp, 0.0.0.0:32772->1920/tcp   hoge

test.tf（3）

DNS サーバーの指定をしてみる。

resource "docker_container" "hoge" {
  image = "${docker_image.centos.latest}"
  name = "hoge"
  hostname = "hoge"
  dns = ["8.8.8.8", "8.8.8.4"]
  command = ["/bin/sh", "-c", "while true ; do sleep 1; curl -s -I hogehuga.inokara.com ; done"]
}

resource "docker_image" "centos" {
    name = "inokappa/centos-base:latest"
}

dns リソースを ["8.8.8.8", "8.8.8.4"] で指定して terraform apply する。

$ ./terraform apply
docker_image.centos: Refreshing state... (ID: 1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139inokappa/centos-base:latest)
docker_container.hoge: Refreshing state... (ID: 91e657f3586ace300616b6d8ffc42d2ed197237fc9384a6cd204a3f3ac3c7edf)
docker_container.hoge: Creating...
  bridge:           "" => "<computed>"
  command.#:        "" => "3"
  command.0:        "" => "/bin/sh"
  command.1:        "" => "-c"
  command.2:        "" => "while true ; do sleep 1; curl -s -I hogehuga.inokara.com ; done"
  dns.#:            "" => "2"
  dns.3817307869:   "" => "8.8.8.8"
  dns.3929092854:   "" => "8.8.8.4"
  gateway:          "" => "<computed>"
  hostname:         "" => "hoge"
  image:            "" => "1ee7b72a440794c44f62ef26a179242dce2325549a42a9bf8fd3d6f1441a5139"
  ip_address:       "" => "<computed>"
  ip_prefix_length: "" => "<computed>"
  must_run:         "" => "1"
  name:             "" => "hoge"
docker_container.hoge: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate

コンテナは起動したので、指定した dns オプションが効いているかを確認する。

$ docker inspect hoge | ruby -r json -e 'r = STDIN.read;j = JSON.parse(r); j.each {|r| puts r["HostConfig"]["Dns"]}'
8.8.8.8
8.8.8.4

ちゃんと指定した通りに設定されている。

ということで...

まだまだ色々と試したいオプション（リソース）があるので引き続き。試している間にバージョンが上がっていたりする可能性もあるが、出来るだけ追従しながら試していきたい。