Use Nginx as Load Balancer for GitLab
full step for a SSL-enabled GitLab with external url

Recently I figured out the load balancer integration method for GitLab, and riched to a SSL-enabled, domain-linked GitLab.

Step 0. Prepare

Knowledge and tools needed:

Step 1. Install docker to production server

Follow Docker’s official document for installation:

Setup docker repository:

sudo tee /etc/apt/sources.list.d/docker.list <<< "deb http://mirrors.tuna.tsinghua.edu.cn/docker/apt/repo ubuntu-xenial main"

Install docker:

sudo apt update
sudo apt install -y docker-engine
docker info

Install docker-compose:

sudo curl -L "https://github.com/docker/compose/releases/download/1.10.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose --version

Step 2. Build GitLab application

Please follow the tutorial here:

A example docker-compose.yml:

version: "2"

services:
  redis:
    restart: always
    image: sameersbn/redis:latest
    command:
    - --loglevel warning
    volumes:
    - /srv/docker/gitlab/redis:/var/lib/redis:Z

  postgresql:
    restart: always
    image: sameersbn/postgresql:9.6-2
    volumes:
    - /srv/docker/gitlab/postgresql:/var/lib/postgresql:Z
    environment:
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production
    - DB_EXTENSION=pg_trgm

  gitlab:
    restart: always
    image: sameersbn/gitlab:8.16.3
    depends_on:
    - redis
    - postgresql
    ports:
    - "10080:80"
    - "10022:22"
    - "10443:443"
    volumes:
    - /srv/docker/gitlab/gitlab:/home/git/data:Z
    environment:
    - DEBUG=false

    - DB_ADAPTER=postgresql
    - DB_HOST=postgresql
    - DB_PORT=5432
    - DB_USER=gitlab
    - DB_PASS=password
    - DB_NAME=gitlabhq_production

    - REDIS_HOST=redis
    - REDIS_PORT=6379

    - TZ=Asia/Chongqing
    - GITLAB_TIMEZONE=Chongqing

    - GITLAB_HTTPS=true
    - SSL_SELF_SIGNED=false  # SSL_SELF_SIGNED should be false

    - GITLAB_HOST=git.xxxx.com  # here is the gitlab domain
    - GITLAB_PORT=443           # SSL port 443
    - GITLAB_SSH_PORT=22        # SSH port 22
    - GITLAB_RELATIVE_URL_ROOT=
    - GITLAB_SECRETS_DB_KEY_BASE=xxxx   # xxxx should be generated by yourself
    - GITLAB_SECRETS_SECRET_KEY_BASE=xxxx
    - GITLAB_SECRETS_OTP_KEY_BASE=xxxx

Note that:

Setup GitLab server, docker-compose will pull the image automatically:

docker-compose up -d

Step 3. Domain name resolution

  1. Config DNS in domain name provider: add A-record.
  2. Config virtual server in local router: add record to the port forwarding rules.
    • forward external port 80 to load balancer server, with local port 10080
    • forward external port 443 to load balancer server, with local port 10443
    • forward external port 22 to load balancer server, with local port 10022

Step 4. Load balancer configuration

There’s a lot staff to do:

Setup Nginx in load balancer server:

# Add nginx repository

sudo tee /etc/yum.repos.d/nginx.repo << EOF
# nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
EOF

# Install Nginx

sudo yum makecache
sudo yum install -y nginx

Add stream config in nginx.conf:

sudo mkdir /etc/nginx/stream.d/
sudo tee -a /etc/nginx/nginx.conf <<EOF
stream {
  error_log  /var/log/nginx/stream.error.log info;
  include /etc/nginx/stream.d/*.conf;
}
EOF

Setup gitlab server using nginx load-balancing methods:

sudo tee /etc/nginx/conf.d/git.xxxx.com.conf << EOF
server {
    listen 10080;
    server_name git.clgene.com;

    rewrite ^(.*) https://git.clgene.com$1 permanent;
}

server {
    listen 10443;
    server_name git.clgene.com;
    root /usr/share/nginx/html/;

    ssl                  on;
    ssl_certificate      /etc/nginx/conf.d/git.xxxx.com.crt;
    ssl_certificate_key  /etc/nginx/conf.d/git.xxxx.com.key;

    ssl_prefer_server_ciphers on;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    # intermediate configuration. tweak to your needs.
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';

    location /.well-known {
        root /usr/share/nginx/html/.well-known;
    }
    location / {
        proxy_pass http://192.168.1.85:10080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Ssl on;
    }
}
EOF

sudo tee /etc/nginx/stream.d/git.xxxx.com.conf << EOF
upstream gitlab {
  server 192.168.1.xx:10022;
}
server {
  listen 10022;
  proxy_pass gitlab;
}
EOF

sudo serice nginx restart

Enable ssl using Let’s Encrypt

letsencrypt-rs is just one of implements in Rust Language.

# Rust installation
curl https://sh.rustup.rs -sSf | sh
# Install letsencrypt-rs
cargo install letsencrypt-rs

# Get CA
letsencrypt-rs sign -D git.xxxx.com -P /usr/share/nginx/html/ -k /etc/nginx/conf.d/git.xxxx.com.key -o /etc/nginx/conf.d/git.xxxx.com.crt

# Restart nginx
serice nginx restart

In browser, just type git.xxxx.com and it’ll redirect to https://git.xxxx.com

*****
Written by Huo Linhe on 03 February 2017