How to Manage GitLab Runner Concurrency For Parallel CI Jobs

How to Deploy a GitLab Server With Docker

GitLab is a leading platform for hosting Git repositories, CI pipelines, and DevOps workflows. It’s available as a SaaS offering on GitLab.com or as a self-managed distribution for private use on your own hardware.

GitLab’s a complex system formed from a web of distinct components and dependencies. Installing GitLab packages directly onto your operating system will add weighty new services to your machine, including PostgreSQL, Redis, Gitaly, and the main Rails-based GitLab web application.

1.JiBnMqyl6S

Deploying GitLab as a Docker container is one way to avoid polluting your environment with all these components. Everything related to GitLab will live within the container, separately from your host’s filesystem.

In this guide we’ll use Docker to deploy a production-ready GitLab instance that you can use to host your source code and collaborate on projects. One thing to consider before proceeding is that Docker doesn’t eliminate GitLab’s basic hardware requirements: you’ll need at least 4GB of free RAM and around 10GB of unused storage.

The Official Docker Image

GitLab offers a pre-built Docker image that comes with everything you need to deploy the software. We’re focusing on this image in this tutorial but it’s worth paying attention to its limitations.

The image is monolithic in nature, bundling all GitLab components so they run in a single container. This simplifies set up but makes it challenging to scale your installation in the future. It goes against containerization best practices by running multiple distinct components in the container.

This means the stock image might not be ideal for busy installations. As an alternative, you could use GitLab’s Helm chart to deploy to a Kubernetes cluster. This launches each service as its own containerized Pod so you can scale components individually.

Deploying GitLab With Docker

Install Docker and set up a DNS A record for your GitLab domain name before continuing. You should point the DNS record at the IP address of your Docker host. We’ll use gitlab.example.com as the domain through the remainder of this guide.

You can start GitLab by running the following command:

docker run -d -p 22:22 -p 80:80 -p 443:443 \
    --name gitlab \
    --hostname gitlab.example.com \
    --restart unless-stopped \
    --shm-size 256m \
    -v gitlab_config:/etc/gitlab \
    -v gitlab_logs:/var/log/gitlab \
    -v gitlab_data:/var/opt/gitlab \
    gitlab/gitlab-ce:14.7.0-ce.0

Docker will download the GitLab Community Edition (CE) image and start a new container using it. It’s best practice to pin to a specific GitLab version by selecting its corresponding image tag, 14.7.0-ce.0 in this case. Here’s an explanation of the flags used in the command:

  • -d – Detach the terminal from the container so it runs in the background.
  • -p – Bind ports 22, 80, and 443 in the container to the corresponding ports on your host; this allows GitLab to receive Git and web traffic over SSH and HTTP/S when it’s directed at your host.
  • --name – Assign the container a friendly name so you can conveniently reference it when running Docker CLI commands in the future.
  • --hostname – Set the container’s hostname; this must match the domain you’re using to access GitLab.
  • --restart – Assign the container a restart policy so it’s automatically restarted when it exits due to failure or a host reboot.
  • --shm-size – This is explained in the next section.
  • -v – Set up Docker volumes to persistently store GitLab’s config files, logs, and generated user data outside of the container. This is vital so you don’t lose your data when the container stops!

1.JiBnMqyl6S

Allow several minutes for GitLab’s first-run configuration to complete. Eventually you should be able to visit gitlab.example.com to see the login page. Login as root; the account’s password is automatically generated and can be retrieved by running the following command:

docker exec -it <gitlab-container-name> grep 'Password:' /etc/gitlab/initial_root_password

Substitute <gitlab-container-name> with the name you assigned when creating your container. This was gitlab in the example above.

1.JiBnMqyl6S

 

Investigating Deployment Issues

It’s normal for the installation scripts to take a long time to complete. GitLab has a lot of components to configure before the service can begin to function. You can monitor the progress by viewing the container’s logs:

docker logs gitlab --follow

The best course of action if GitLab’s web UI isn’t available is to simply wait a bit longer and let the set up procedure run through to completion. If something has gone wrong, you’re seeing 500 errors in your browser, and the container logs aren’t showing new activity, restarting the container with docker restart gitlab can sometimes help.

One common error message you might see in the logs looks similar to this:

writing value to /dev/shm/gitlab/... failed with unmapped file

This occurs because services bundled with GitLab write significant amounts of data to the temporary /dev/shm filesystem. Docker only allocates containers a /dev/shm space of 64 MB by default. This is rarely sufficient to sustain GitLab’s metrics collection via Prometheus, responsible for most of the writes to the filesystem.

The capacity of /dev/shm can be increased using the --shm-size flag when you create your container with docker run. This is shown in the deployment example above where 256 MB is allocated, the minimum recommended by GitLab. Very active installations may need to use a larger size.

Another issue pertains to GitLab’s log files: these quickly become expansive, which can an cause buffer overflow detected errors when starting Docker. This can be prevented by periodically clearing old files in the /var/log/gitlab directory from within your GitLab container:

docker exec -it gitlab rm /var/log/gitlab/*

The log files which Docker collects from the container’s output streams and retains on your host will rapidly grow to large sizes too. Docker’s default json-file log storage driver is inadequate for use with a production GitLab instance. The JSON files it creates are heavy, verbose, and never compressed or rotated. Switch to another storage driver such as local or journald to ensure logs are regularly rotated.

Configuring GitLab

You can supply GitLab config file snippets when you create your container by setting the GITLAB_OMNIBUS_CONFIG environment variable. This should be a valid Ruby string that will be appended to the /etc/gitlab/gitlab.rb file inside the container:

docker run -e GITLAB_OMNIBUS_CONFIG="gitlab_rails['allowed_hosts'] = ['gitlab.example.com]; nginx['redirect_http_to_https'] = true;"

You can also edit /etc/gitlab/gitlab.rb from within the container once it’s running. When you’re done making changes, you must restart the container to apply them:

docker restart gitlab

The GitLab Docker image automatically runs the Omnibus reconfigure script each time it starts. This takes the current config from your gitlab.rb and applies it to your installation. It’s normal for GitLab to be inaccessible for a few seconds after your container restarts while this process completes.

Applying GitLab Updates

GitLab updates are easily applied by stopping your container and starting a new one with the same configuration.

Pull the new image corresponding to your chosen GitLab release:

docker pull gitlab/gitlab-ce:14.7.1-ce.0

Remove your existing container:

docker rm gitlab

Finally, start a new container, repeating your original flags but modifying the image reference:

docker run -d -p 22:22 -p 80:80 -p 443:443 \
    --name gitlab \
    ...
    gitlab/gitlab-ce:14.7.1-ce.0

Your data will be intact as the volumes from the old container will be reattached to the new one.

You can avoid repetition of your docker run flags by encapsulating your configuration in a docker-compose.yml file:

version: "3"
services:
  gitlab:
    image: gitlab/gitlab-ce:14.7.0-ce-0
    hostname: gitlab.example.com
    ports:
      - 22:22
      - 80:80
      - 443:443
    volumes:
      gitlab_config:/etc/gitlab
      gitlab_logs:/var/log/gitlab
      gitlab_data:/var/opt/gitlab
    restart: unless-stopped
volumes:
  gitlab_config:
  gitlab_logs:
  gitlab_data:

When you’re using Docker Compose, you can bring up your GitLab instance by running docker-compose up -d. To update to a new release, change the image field accordingly and repeat the command. Docker Compose will automate the container replacement process.

Irrespective of the mechanism you use, it’s important to make sure your upgrade paths align with GitLab’s supported routes. Occasionally you may need to take manual actions post-upgrade to complete the migration.

Backing Up Your Installation

Regular backups are critical to a successful GitLab deployment. GitLab’s typical role as your organization’s single source of truth means a worst case scenario could see all your work gone forever unless backups are made.

GitLab has an integrated backup utility that can be used to create a complete archive of your installation. This is accessible in the Docker image via the gitlab-backup command:

docker exec -t gitlab gitlab-backup create

Backups are saved to the /var/opt/gitlab/backups directory by default. This means they’ll be stored outside the container, within one of your mounted Docker volumes. For better safety you should configure GitLab to upload backups to a remote object storage provider by adding these lines to your config file:

gitlab_rails['backup_upload_connection'] = {
    "provider" => "AWS",
    "region" => "eu-west-1",
    "aws_access_key_id" => "access_key",
    "aws_secret_access_key" => "secret_key",
    # "endpoint" => "https://..."
}

Now the gitlab-backup create command will automatically upload the archive files it generates, keeping them separate from your GitLab container and Docker host. Your last step should be adding a cron task on your system that periodically runs the backup script:

0 * * * * docker exec -t gitlab gitlab-backup create

Summary

GitLab is a complex piece of software that’s easily deployed with Docker. The docker run example shown in this guide is suitable for production use when combined with the best practice config changes explained above. You should also audit the security of the Docker daemon and your host to ensure your data’s adequately protected.

A Dockerized GitLab deployment is a good way to quickly trial the platform. It’s also an effective strategy for launching and maintaining a smaller GitLab server for long-term use. Other options such as GitLab Omnibus on a dedicated server or a cloud-native Kubernetes install are generally better suited to larger requirements and sustained enterprise adoption.

Original Link

Leave a Comment

Your email address will not be published.