This is the 2nd part of the series about the virtual build setup behind VPN. The first part can be found her.

Now that the VPN is working we have 3 Nodes communicating via VPN:

  • vpn1 VPN Entry Server
  • teamcity TeamCity Host
  • teamcity-node1 TeamCity Agent

It’s time to fill them with live. We used Docker to easily do installation. Luckily there are already some ready-to-use containers for TeamCity but in order to get a compatible build we had to patch them a little bit.

What should be build

  • Scala Play! Application with some NodeJS dependencies for client-side-code
  • C++-Applications
  • The build should create ready-to-use Docker containers, so we need the privileged Mode
  • The build shall be done inside Docker containers to get them easily reproducible.

Pitfall 1: Swap-File

The cheapest Vultr-VMs comes with only 768MB of RAM. This is OK for the VMs, but Java allocates more memory upon startup. It’s necessary to create a Swap file. This Manual is handy.

Pitfall 2: Docker inside VPN

We do not want the Docker servers to listen on the public eth0 net interface, which it does by default. You can however tell docker to listen on a specific IP Address directly, but this is error prone as you have to tell it on each Docker invocation.

Much more easily you can set Docker to automatically listen on one interface, using following configuration:

 # This comes into /etc/default/docker.io
 # You have to replace 10.8.0.10 with your VPN ip adress.
 DOCKER_OPTS="--ip 10.8.0.10"

It is maybe also possible to restrict all trafic using iptables, but it turns out that docker writes in your iptables too (in order to create FORWARD-Calls to it’s VMs). This is a bad as you cannot just create a meaningful set and then backup it but you also have to check after each Docker call, that your configuration is still Ok.

Main TeamCity Server

We are using the Dockerfile nob13/teamcity-docker. A database server is necessary, we choose Postgres.

The following decking.json-File will start up the TeamCity server together with a Postgres SQL server

{
    "containers":{
        "teamcity":{
            "image":"nob13/teamcity",
            "port":["8111:8111"],
            "mount":["/data/teamcity:/data/teamcity"],
            "dependencies":["postgres:postgres"]
        },
        "postgres":{
            "image":"postgres",
            "mount":["/data/postgres:/var/lib/postgresql/data"]
        }
    },
    "clusters" : {
        "main":["teamcity", "postgres"]
    }
}

You can start it with decking.io:

# As root
apt-get install npm nodejs-legacy # Node.js and NPM
npm install -g decking.io

# The following calls in the same directory as decking.json
decking create
decking start

You can also choose Fig, the configuration would be similar. After starting TeamCity for the first time, you have to cop the Postgres JDBC lib found here into /data/teamcity/lib/jdbc. This cannot be done from Docker but must be done manually. Then just continue the installation (set your master password).

The database user is postgres, there is no user set. The only directory which are needed to backup is /data. It contains the database together will all TeamCity related files.

Team City Build Agent Node

Here we use the following Dockerfile [nob13/teamcity-agent-docker]. It is forked from Sjoerd Mulder because we need the MongoDB-executable inside the container to use MongoFaker for application testing.

The startup doesn’t need any additional Containers, so there is no need to use decking.io, just execute:

docker run -d --name teamcity_agent --privileged -e TEAMCITY_SERVER=http://teamcity.lan:8111 -p 90 90:9090 nob13/teamcity_agent:latest

teamcity.lan is the VPN DNS-Name of the TeamCity Server.

As soon as the Node has been started, you can authorize it inside TeamCity Main server, and voila: The build setup is ready to run.

The node is able to create docker images because it is privileged.