GitLab Runner

Overview

GitLab Runner is the agent that executes CI/CD jobs in the GitLab ecosystem. It runs the scripts defined in .gitlab-ci.yml files and reports the results back to GitLab. GitLab Runners can be installed on various platforms, from virtual machines and containers to Kubernetes clusters and serverless environments.

Key Features

Distributed execution: Run jobs on multiple machines simultaneously
Autoscaling: Dynamically provision runners based on workload
Multiple executor options: Shell, Docker, Kubernetes, Virtual Machine, SSH, and more
Job concurrency: Configure the number of concurrent jobs per runner
Custom environment variables: Pass environment-specific data to jobs
Artifact management: Handle job output files automatically
Caching support: Speed up builds by caching dependencies

Installation Guide

Linux Installation

Standard Linux Installation (Debian/Ubuntu)

# Add the GitLab repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

# Install the latest version
sudo apt-get install gitlab-runner

# Register the runner with your GitLab instance
sudo gitlab-runner register

Follow the interactive registration process:

Enter your GitLab instance URL
Enter the registration token from your GitLab project/group settings
Enter a description for the runner
Add tags (optional but recommended for job targeting)
Choose an executor (docker, shell, etc.)

RHEL/CentOS/Fedora Installation

# Add the GitLab repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash

# Install the latest version
sudo dnf install gitlab-runner

# Register the runner
sudo gitlab-runner register

WSL Installation

Installing GitLab Runner in WSL combines the flexibility of Linux with integration into Windows environments:

# Update package lists
sudo apt update

# Install prerequisites
sudo apt install curl

# Download the GitLab Runner binary
sudo curl -L --output /usr/local/bin/gitlab-runner "https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64"

# Set execute permissions
sudo chmod +x /usr/local/bin/gitlab-runner

# Create a GitLab Runner user
sudo useradd --comment 'GitLab Runner' --create-home gitlab-runner --shell /bin/bash

# Install and run as a service
sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
sudo gitlab-runner start

# Register the runner
sudo gitlab-runner register

WSL-Specific Considerations

Ensure that Windows Defender or other security software doesn't block GitLab Runner
Configure WSL to maintain a persistent connection if needed with task scheduler:

# Create a scheduled task to keep WSL running (execute in PowerShell as Admin)
$action = New-ScheduledTaskAction -Execute "wsl.exe" -Argument "-- sleep infinity"
$trigger = New-ScheduledTaskTrigger -AtStartup
$settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -ExecutionTimeLimit 0
Register-ScheduledTask -TaskName "Keep WSL Running" -Action $action -Trigger $trigger -Settings $settings -User "SYSTEM" -RunLevel Highest

NixOS Installation

NixOS offers a declarative approach to system configuration, making GitLab Runner setup reproducible and version-controlled.

NixOS Configuration

Add the following to your configuration.nix file:

{ config, pkgs, ... }:

{
  # Enable GitLab Runner service
  services.gitlab-runner = {
    enable = true;
    
    # Configure runners
    # You can add multiple runners with different configurations
    settings = {
      concurrent = 4;  # Number of concurrent jobs
      check_interval = 0;
    };
    
    # Define a runner
    runners = [
      {
        name = "nixos-runner";
        # Environment variables available to the build
        environment = [ "PATH=/run/wrappers/bin:/nix/var/nix/profiles/default/bin" ];
        # GitLab instance URL - replace with your instance
        url = "https://gitlab.example.com";
        # Registration token from GitLab
        registrationConfigFile = "/var/lib/gitlab-runner/registration-token";
        # Runner executor
        executor = "shell";
        # Tags for job selection
        tagList = [ "nixos" "linux" ];
        # Run as unprivileged user
        runUntagged = true;
        # Whether to run untagged jobs
        protected = false;
      }
    ];
  };
  
  # Open necessary ports for GitLab communication
  networking.firewall.allowedTCPPorts = [ 9252 ];
  
  # Add the gitlab-runner user to docker group if using Docker executor
  users.users.gitlab-runner.extraGroups = [ "docker" ];
  virtualisation.docker.enable = true;  # If using Docker executor
}

Create the registration token file:

sudo mkdir -p /var/lib/gitlab-runner
echo "YOUR_RUNNER_REGISTRATION_TOKEN" | sudo tee /var/lib/gitlab-runner/registration-token
sudo chmod 600 /var/lib/gitlab-runner/registration-token

Apply the configuration:

sudo nixos-rebuild switch

Real-Life Scenarios (2025 Best Practices)

Scenario 1: Multi-Environment CI/CD Pipeline for Microservices

Challenge: A team manages a microservices architecture spanning development, staging, and production environments across multiple cloud providers.

Solution with GitLab Runner:

Environment-Specific Runners:

# .gitlab-ci.yml
stages:
  - test
  - build
  - deploy-dev
  - deploy-staging
  - deploy-prod

variables:
  DOCKER_REGISTRY: "${CI_REGISTRY}"
  
test:
  stage: test
  image: node:18-alpine
  tags:
    - docker
  script:
    - npm ci
    - npm run test
  cache:
    paths:
      - node_modules/

build:
  stage: build
  image: docker:24.0
  tags:
    - docker
  services:
    - docker:24.0-dind
  script:
    - docker build -t ${DOCKER_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA} .
    - docker push ${DOCKER_REGISTRY}/${CI_PROJECT_PATH}:${CI_COMMIT_SHA}
  only:
    - main
    - tags

deploy-dev:
  stage: deploy-dev
  image: 
    name: bitnami/kubectl:1.28
    entrypoint: [""]
  tags:
    - kubernetes
    - dev
  script:
    - echo "${KUBE_CONFIG_DEV}" > kubeconfig
    - export KUBECONFIG=./kubeconfig
    - envsubst < kubernetes/dev.yaml | kubectl apply -f -
  environment:
    name: development
    url: https://dev.example.com
  only:
    - main

# Similar configurations for staging and production stages with different tags

Runner Configuration:

# config.toml for dedicated environment runners
concurrent = 10
check_interval = 0

[[runners]]
  name = "dev-kubernetes-runner"
  url = "https://gitlab.example.com"
  token = "TOKEN"
  executor = "kubernetes"
  [runners.kubernetes]
    namespace = "gitlab-runners"
    image = "ubuntu:22.04"
    service_account = "gitlab-runner"
    service_account_overwrite_allowed = "gitlab-runner"
    pod_annotations_overwrite_allowed = "*"
    cpu_limit = "1"
    memory_limit = "1Gi"
    helper_cpu_limit = "200m"
    helper_memory_limit = "256Mi"
    poll_interval = 5
    resource_availability_check_interval = 1

[[runners]]
  name = "prod-kubernetes-runner"
  url = "https://gitlab.example.com"
  token = "TOKEN"
  executor = "kubernetes"
  tags = ["kubernetes", "production"]
  [runners.kubernetes]
    namespace = "gitlab-runners-prod"
    # Enhanced security for production
    privileged = false
    # Production runner with higher resource limits
    cpu_limit = "2"
    memory_limit = "4Gi"
    # Use node selector for production grade nodes
    node_selector = "env=production"
    service_account = "gitlab-runner-prod"

Scenario 2: Secure Infrastructure as Code Deployment

Challenge: Securely deploy infrastructure changes with appropriate approval gates and compliance checks.

Solution with GitLab Runner:

# .gitlab-ci.yml for Terraform deployment
stages:
  - validate
  - plan
  - apply
  - verify

variables:
  TF_ROOT: ${CI_PROJECT_DIR}/terraform
  TF_STATE_NAME: ${CI_PROJECT_PATH_SLUG}
  TF_CACHE_KEY: ${CI_COMMIT_REF_SLUG}
  TF_VAR_environment: ${CI_ENVIRONMENT_NAME}

# Include secure scanning templates
include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Jobs/Infrastructure-as-Code-Security.gitlab-ci.yml

validate:
  stage: validate
  image: hashicorp/terraform:1.7
  tags:
    - docker
    - secure
  script:
    - cd ${TF_ROOT}
    - terraform init
    - terraform validate
    - terraform fmt -check
  cache:
    key: ${TF_CACHE_KEY}
    paths:
      - ${TF_ROOT}/.terraform

terraform-plan:
  stage: plan
  image: hashicorp/terraform:1.7
  tags:
    - docker
    - secure
  script:
    - cd ${TF_ROOT}
    - terraform init
    - terraform plan -out=tfplan
  artifacts:
    paths:
      - ${TF_ROOT}/tfplan
    expire_in: 1 week
  cache:
    key: ${TF_CACHE_KEY}
    paths:
      - ${TF_ROOT}/.terraform
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

# The apply job requires approval before execution (2025 best practice)
terraform-apply:
  stage: apply
  image: hashicorp/terraform:1.7
  tags:
    - docker
    - secure
  script:
    - cd ${TF_ROOT}
    - terraform init
    - terraform apply -auto-approve tfplan
  dependencies:
    - terraform-plan
  when: manual
  environment:
    name: production
    url: https://console.example.cloud/projects/${TF_VAR_project_id}
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
  cache:
    key: ${TF_CACHE_KEY}
    paths:
      - ${TF_ROOT}/.terraform

# Verify infrastructure after deployment
verify-infrastructure:
  stage: verify
  image: python:3.12-alpine
  tags:
    - docker
  script:
    - pip install pytest boto3 azure-identity azure-mgmt-resource google-cloud-resource-manager
    - cd ${CI_PROJECT_DIR}/tests
    - python -m pytest -xvs infra_tests.py
  dependencies:
    - terraform-apply
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: on_success

Scenario 3: Auto-Scaling Cloud Runners for Cost Optimization

Challenge: Managing pipeline costs while ensuring sufficient capacity for peak times.

Solution with GitLab Runner on AWS/Azure/GCP:

AWS Auto-Scaling Configuration (Terraform)

# AWS auto-scaling GitLab Runners with Terraform

resource "aws_launch_template" "gitlab_runner" {
  name_prefix   = "gitlab-runner-"
  image_id      = "ami-0123456789abcdef0" # Amazon Linux 2 AMI
  instance_type = "c6a.large"  # AMD-based instances for better cost/performance
  
  user_data = base64encode(<<-EOF
    #!/bin/bash
    yum update -y
    amazon-linux-extras install docker -y
    systemctl enable docker
    systemctl start docker
    
    # Install GitLab Runner
    curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | bash
    yum install -y gitlab-runner
    
    # Register the runner with auto-scaling configuration
    gitlab-runner register \
      --non-interactive \
      --url "https://gitlab.example.com/" \
      --registration-token "${var.runner_token}" \
      --executor "docker+machine" \
      --docker-image "alpine:latest" \
      --description "Auto-scaling Runner" \
      --tag-list "aws,auto-scale" \
      --run-untagged="true" \
      --locked="false" \
      --access-level="not_protected" \
      --docker-privileged \
      --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
      --machine-idle-nodes 1 \
      --machine-idle-time 1800 \
      --machine-max-builds 100 \
      --machine-machine-driver "amazonec2" \
      --machine-machine-name "gitlab-docker-machine-%s" \
      --machine-machine-options "amazonec2-instance-type=c6a.large" \
      --machine-machine-options "amazonec2-region=${var.aws_region}" \
      --machine-machine-options "amazonec2-vpc-id=${var.vpc_id}" \
      --machine-machine-options "amazonec2-subnet-id=${var.subnet_id}" \
      --machine-machine-options "amazonec2-use-private-address=true" \
      --machine-machine-options "amazonec2-security-group=${var.security_group}" \
      --machine-machine-options "amazonec2-tags=runner,gitlab,${var.environment}"
  EOF
  )
  
  iam_instance_profile {
    name = aws_iam_instance_profile.gitlab_runner.name
  }
  
  network_interfaces {
    associate_public_ip_address = true
    security_groups             = [var.security_group_id]
  }
  
  tag_specifications {
    resource_type = "instance"
    tags = {
      Name = "GitLab Runner"
      Environment = var.environment
      ManagedBy = "Terraform"
    }
  }
}

resource "aws_autoscaling_group" "gitlab_runner" {
  desired_capacity    = 1
  max_size            = 5
  min_size            = 1
  vpc_zone_identifier = var.subnet_ids
  
  launch_template {
    id      = aws_launch_template.gitlab_runner.id
    version = "$Latest"
  }
  
  tag {
    key                 = "Name"
    value               = "GitLab Runner"
    propagate_at_launch = true
  }
  
  # Scale based on average CPU utilization
  target_tracking_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ASGAverageCPUUtilization"
    }
    target_value = 60.0
  }
}

Scenario 4: Secure GitLab Runners for Sensitive Workloads

Challenge: Running CI/CD pipelines for high-security environments with strict compliance requirements.

Solution:

Configure isolated, dedicated runners with enhanced security settings:

# config.toml for secure runners
concurrent = 4
check_interval = 0

[[runners]]
  name = "secure-compliant-runner"
  url = "https://gitlab.example.com"
  token = "TOKEN"
  executor = "docker"
  [runners.docker]
    tls_verify = true
    image = "alpine:3.19"
    privileged = false
    disable_entrypoint_overwrite = true
    oom_kill_disable = false
    disable_cache = true
    volumes = ["/cache"]
    shm_size = 0
    network_mode = "bridge"
  [runners.cache]
    Type = "s3"
    Shared = false
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      AccessKey = "${S3_ACCESS_KEY}"
      SecretKey = "${S3_SECRET_KEY}"
      BucketName = "gitlab-runner-cache"
      BucketLocation = "us-east-1"
      Insecure = false
      Encrypted = true

Pipeline configuration for sensitive workloads:

# .gitlab-ci.yml for sensitive workloads
stages:
  - security-scan
  - test
  - build
  - deploy

# Include security scanning templates
include:
  - template: Security/SAST.gitlab-ci.yml
  - template: Security/Secret-Detection.gitlab-ci.yml
  - template: Security/Dependency-Scanning.gitlab-ci.yml
  - template: Security/Container-Scanning.gitlab-ci.yml
  - template: Security/License-Scanning.gitlab-ci.yml

variables:
  # Enhance security features
  SECURE_ANALYZERS_PREFIX: "registry.gitlab.com/security-products"
  SCAN_KUBERNETES_MANIFESTS: "true"

# Interactive Application Security Testing
dast:
  stage: security-scan
  tags:
    - secure
    - compliant
  image:
    name: "${SECURE_ANALYZERS_PREFIX}/dast:latest"
  variables:
    DAST_WEBSITE: https://staging.example.com
    DAST_FULL_SCAN_ENABLED: "true"
    DAST_ZAP_USE_AJAX_SPIDER: "true"
    DAST_REQUEST_HEADERS: "Cache-Control: no-cache,User-Agent: DAST/1.0"
    # Add authentication if needed
    # DAST_AUTH_URL: https://staging.example.com/login
    # DAST_USERNAME: ${SECURITY_USERNAME}
    # DAST_PASSWORD: ${SECURITY_PASSWORD}
  allow_failure: false
  artifacts:
    reports:
      dast: gl-dast-report.json

# Secure build process
build:
  stage: build
  tags:
    - secure
    - compliant
  image: docker:24.0
  services:
    - name: docker:24.0-dind
      command: ["--tls=true", "--tlscert=/certs/server.pem", "--tlskey=/certs/server-key.pem"]
  script:
    # Sign the build for supply chain security
    - apk add --no-cache cosign
    - docker build -t ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA} .
    - echo "${CI_REGISTRY_PASSWORD}" | docker login -u ${CI_REGISTRY_USER} --password-stdin ${CI_REGISTRY}
    - docker push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
    - cosign sign --key ${COSIGN_KEY} ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
  artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json

# Deploy using zero-trust principles
deploy:
  stage: deploy
  tags:
    - secure
    - compliant
  image: 
    name: bitnami/kubectl:1.28
    entrypoint: [""]
  variables:
    VERIFY_TLS: "true"
  script:
    # Use short-lived credentials
    - export TOKEN=$(curl -H "X-Vault-Token: $VAULT_TOKEN" -X POST $VAULT_ADDR/v1/auth/kubernetes/login -d '{"role":"gitlab-deploy","jwt":"'$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)'"}' | jq -r '.auth.client_token')
    # Get environment-specific configs from vault
    - export KUBECONFIG=$(curl -H "X-Vault-Token: $TOKEN" -X GET $VAULT_ADDR/v1/secret/data/kubernetes/config | jq -r '.data.data.kubeconfig' > kubeconfig && echo ./kubeconfig)
    # Verify image signature before deployment
    - cosign verify --key ${COSIGN_PUBLIC_KEY} ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
    # Deploy with strict security context
    - envsubst < kubernetes/deploy-secure.yaml | kubectl apply -f -
  environment:
    name: production
    url: https://example.com
  when: manual

Best Practices for GitLab Runners (2025)

Security Best Practices

Zero Trust Architecture:
- Implement Workload Identity Federation instead of static tokens
- Use short-lived credentials for cloud resource access
- Verify all artifacts with digital signatures
Runner Isolation:
- Use dedicated runners for sensitive projects
- Implement network segmentation for runner environments
- Utilize container sandboxing technologies (gVisor, Kata Containers)
Supply Chain Security:
- Sign all container images with Cosign or Notary
- Implement Software Bill of Materials (SBOM) generation
- Use trusted base images with vulnerability scanning

Performance Optimization

Smart Caching Strategies:
- Use distributed caching systems (S3, GCS, Azure Blob)
- Implement layer caching for Docker builds
- Cache package manager dependencies effectively
Resource Allocation:
- Set appropriate resource limits based on job requirements
- Utilize spot/preemptible instances for cost-efficiency
- Implement auto-scaling based on workload patterns
Containerized Execution:
- Use slim, purpose-built containers for jobs
- Implement multi-stage builds to reduce image size
- Optimize dependency management for faster execution

Cost Optimization

Intelligent Scaling:
- Implement predictive auto-scaling based on historical patterns
- Use spot instances with fallback mechanisms
- Scale to zero for non-critical environments
Resource Efficiency:
- Tag runners for specific workloads to optimize resource use
- Set job timeouts to prevent runaway processes
- Implement job concurrency limits
Multi-Cloud Strategy:
- Distribute runners across cloud providers for cost arbitrage
- Use runners in regions with lower costs
- Implement FinOps practices with cost monitoring

Resilience and Reliability

High Availability Setup:
- Deploy runners across multiple availability zones
- Implement circuit breakers for external dependencies
- Use health checks and automatic remediation
Failure Management:
- Implement retry mechanisms for transient failures
- Set up alerting for repeated job failures
- Create runbooks for common runner issues
Disaster Recovery:
- Backup runner configurations regularly
- Test recovery scenarios periodically
- Document recovery procedures

Monitoring GitLab Runners

Key Metrics to Monitor

System Resources:
- CPU, memory, and disk usage
- Network throughput and latency
- Queue length and job wait time
Job Performance:
- Job execution time
- Build success/failure rate
- Cache hit/miss ratio
Cost Metrics:
- Runner uptime
- Resource utilization efficiency
- Job cost by project/branch

Monitoring Tools Integration

# prometheus.yml configuration
scrape_configs:
  - job_name: 'gitlab_runners'
    scrape_interval: 15s
    static_configs:
      - targets: ['gitlab-runner:9252']

Example Grafana dashboard query:

rate(gitlab_runner_jobs_total{status="success"}[5m])

Conclusion

GitLab Runner remains a critical component of modern CI/CD infrastructure in 2025. By following best practices for installation, configuration, security, and scaling, organizations can build resilient and efficient pipelines that support modern software delivery practices. The platform-specific installation guides and real-world scenarios provided here should help teams implement GitLab Runner effectively across different environments.

Additional Resources

PreviousGitLab NextGitOps Overview

Last updated 6 months ago