DevOps help for Cloud Platform Engineers
  • Welcome!
  • Quick Start Guide
  • About Me
  • CV
  • Contribute
  • 🧠DevOps & SRE Foundations
    • DevOps Overview
      • Engineering Fundamentals
      • Implementing DevOps Strategy
      • DevOps Readiness Assessment
      • Lifecycle Management
      • The 12 Factor App
      • Design for Self Healing
      • Incident Management Best Practices (2025)
    • SRE Fundamentals
      • Toil Reduction
      • System Simplicity
      • Real-world Scenarios
        • AWS VM Log Monitoring API
    • Agile Development
      • Team Agreements
        • Definition of Done
        • Definition of Ready
        • Team Manifesto
        • Working Agreement
    • Industry Scenarios
      • Finance and Banking
      • Public Sector (UK/EU)
      • Energy Sector Edge Computing
  • 🛠️DevOps Practices
    • Platform Engineering
    • FinOps
    • Observability
      • Modern Practices
  • 🚀Modern DevOps Practices
    • Infrastructure Testing
    • Modern Development
    • Database DevOps
  • 🛠️Infrastructure as Code (IaC)
    • Terraform
      • Cloud Integrations - Provider-specific implementations
        • Azure Scenarios
          • Azure Authetication
            • Service Principal
            • Service Principal in block
            • Service Principal in env
        • AWS Scenarios
          • AWS Authentication
        • GCP Scenarios
          • GCP Authentication
      • Testing and Validation
        • Unit Testing
        • Integration Testing
        • End-to-End Testing
        • Terratest Guide
      • Best Practices
        • State Management
        • Security
        • Code Organization
        • Performance
      • Tools & Utilities - Enhancing the Terraform workflow
        • Terraform Docs
        • TFLint
        • Checkov
        • Terrascan
      • CI/CD Integration - Automating infrastructure deployment
        • GitHub Actions
        • Azure Pipelines
        • GitLab CI
    • Bicep
      • Getting Started - First steps with Bicep [BEGINNER]
      • Template Specs
      • Best Practices - Guidelines for effective Bicep implementations
      • Modules - Building reusable components [INTERMEDIATE]
      • Examples - Sample implementations for common scenarios
      • Advanced Features
      • CI/CD Integration - Automating Bicep deployments
        • GitHub Actions
        • Azure Pipelines
  • 💰Cost Management & FinOps
    • Cloud Cost Optimization
  • 🐳Containers & Orchestration
    • Containerization Overview
      • Docker
        • Dockerfile Best Practices
        • Docker Compose
      • Kubernetes
        • CLI Tools - Essential command-line utilities
          • Kubectl
          • Kubens
          • Kubectx
        • Core Concepts
        • Components
        • Best Practices
          • Pod Security
          • Security Monitoring
          • Resource Limits
        • Advanced Features - Beyond the basics [ADVANCED]
          • Service Mesh
            • Istio
            • Linkerd
          • Ingress Controllers
            • NGINX
            • Traefik
            • Kong
            • Gloo Edge
            • Contour
        • Tips
          • Status in Pods
          • Resource handling
          • Pod Troubleshooting Commands
        • Enterprise Architecture
        • Health Management
        • Security & Compliance
        • Virtual Clusters
      • OpenShift
  • Service Mesh & Networking
    • Service Mesh Implementation
  • Architecture Patterns
    • Data Mesh
    • Multi-Cloud Networking
    • Disaster Recovery
    • Chaos Engineering
  • Edge Computing
    • Implementation Guide
      • Serverless Edge
      • IoT Edge Patterns
      • Real-Time Processing
      • Edge AI/ML
      • Security Hardening
      • Observability Patterns
      • Network Optimization
      • Storage Patterns
  • 🚀CI/CD & Release Management
    • Continuous Integration
    • Continuous Delivery
      • Deployment Strategies
      • Secrets Management
      • Blue-Green Deployments
      • Deployment Metrics
      • Progressive Delivery
      • Release Management for DevOps/SRE (2025)
  • CI/CD Platforms
    • Tekton
      • Build and Push Container Images
      • Tekton on NixOS Setup
    • Flagger
    • Azure DevOps
      • Pipelines
        • Stages
        • Jobs
        • Steps
        • Templates - Reusable pipeline components
        • Extends
        • Service Connections - External service authentication
        • Best Practices for 2025
        • Agents and Runners
        • Third-Party Integrations
        • Azure DevOps CLI
      • Boards & Work Items
    • GitHub Actions
      • GitHub SecOps: DevSecOps Pipeline
    • GitLab
      • GitLab Runner
  • GitOps
    • GitOps Overview
      • Modern GitOps Practices
      • GitOps Patterns for Multi-Cloud (2025)
      • Flux
        • Progressive Delivery
        • Use GitOps with Flux, GitHub and AKS
  • Source Control
    • Source Control Overview
      • Git Branching Strategies
      • Component Versioning
      • Kubernetes Manifest Versioning
      • GitLab
      • Creating a Fork
      • Naming Branches
      • Pull Requests
      • Integrating LLMs into Source Control Workflows
  • ☁️Cloud Platforms
    • Cloud Strategy
      • AWS to Azure
      • Azure to AWS
      • GCP to Azure
      • AWS to GCP
      • GCP to AWS
    • Landing Zones in Public Clouds
      • AWS Landing Zone
      • GCP Landing Zone
      • Azure Landing Zones
    • Azure
      • Best Practices
        • Azure Best Practices Overview
        • Azure Architecture Best Practices
        • Azure Naming Standards
        • Azure Tags
        • Azure Security Best Practices
      • Services
        • Azure Active Directory (AAD)
        • Azure Monitor
        • Azure Key Vault
        • Azure Service Bus
        • Azure DNS
        • Azure App Service
        • Azure Batch
        • Azure Machine Learning
        • Azure OpenAI Service
        • Azure Cognitive Services
        • Azure Kubernetes Service (AKS)
        • Azure Databricks
        • Azure SQL Database
      • Monitoring
      • Administration Tools - Platform management interfaces
        • Azure PowerShell
        • Azure CLI
      • Tips & Tricks
    • AWS
      • Authentication
      • Best Practices
      • Tips & Tricks
      • Services
        • AWS IAM (Identity and Access Management)
        • Amazon CloudWatch
        • Amazon SNS (Simple Notification Service)
        • Amazon SQS (Simple Queue Service)
        • Amazon Route 53
        • AWS Elastic Beanstalk
        • AWS Batch
        • Amazon SageMaker
        • Amazon Bedrock
        • Amazon Comprehend
    • Google Cloud
      • Services
        • Cloud CDN
        • Cloud DNS
        • Cloud Load Balancing
        • Google Kubernetes Engine (GKE)
        • Cloud Run
        • Artifact Registry
        • Compute Engine
        • Cloud Functions
        • App Engine
        • Cloud Storage
        • Persistent Disk
        • Filestore
        • Cloud SQL
        • Cloud Spanner
        • Firestore
        • Bigtable
        • BigQuery
        • VPC (Virtual Private Cloud)
  • 🔐Security & Compliance
    • DevSecOps Overview
      • DevSecOps Pipeline Security
      • DevSecOps
        • Real-life Examples
        • Scanning & Protection - Automated security tooling
          • Dependency Scanning
          • Credential Scanning
          • Container Security Scanning
          • Static Code Analysis
            • Best Practices
            • Tool Integration Guide
            • Pipeline Configuration
        • CI/CD Security
        • Secrets Rotation
      • Supply Chain Security
        • SLSA Framework
        • Binary Authorization
        • Artifact Signing
      • Security Best Practices
        • Threat Modeling
        • Kubernetes Security
      • SecOps
      • Zero Trust Model
      • Cloud Compliance
        • ISO/IEC 27001:2022
        • ISO 22301:2019
        • PCI DSS
        • CSA STAR
      • Security Frameworks
      • SIEM and SOAR
  • Security Architecture
    • Zero Trust Implementation
      • Identity Management
      • Network Security
      • Access Control
  • 🔍Observability & Monitoring
    • Observability Fundamentals
  • 🧪Testing Strategies
    • Testing Overview
      • Modern Testing Approaches
      • End-to-End Testing
      • Unit Testing
      • Performance Testing
        • Load Testing
      • Fault Injection Testing
      • Integration Testing
      • Smoke Testing
  • 🤖AI Integration
    • AIops Overview
      • Workflow Automation
      • Predictive Analytics
      • Code Quality
  • 🧠AI & LLM Integration
    • Overview
      • Claude
        • Installation Guide
        • Project Guides
        • MCP Server Setup
        • LLM Comparison
      • Ollama
        • Installation Guide
        • Configuration
        • Models and Fine-tuning
        • DevOps Usage
        • Docker Setup
        • GPU Setup
        • Open WebUI
      • Copilot
        • Installation Guide
        • VS Code Integration
        • CLI Usage
      • Gemini
        • Installation Guides - Platform-specific setup
          • Linux Installation
          • WSL Installation
          • NixOS Installation
        • Gemini 2.5 Features
        • Roles and Agents
        • NotebookML Guide
        • Cloud Infrastructure Deployment
        • Summary
  • 💻Development Environment
    • DevOps Tools
      • Pulumi
      • Operating Systems - Development platforms
        • NixOS
          • Install NixOS: PC, Mac, WSL
          • Nix Language Deep Dive
          • Nix Language Fundamentals
            • Nix Functions and Techniques
            • Building Packages with Nix
            • NixOS Configuration Patterns
            • Flakes: The Future of Nix
          • NixOS Generators: Azure & QEMU
        • WSL2
          • Distributions
          • Terminal Setup
      • Editor Environments
      • CLI Tools
        • Azure CLI
        • PowerShell
        • Linux Commands
          • SSH - Secure Shell)
            • SSH Config
            • SSH Port Forwarding
        • Linux Fundametals
        • Cloud init
          • Cloud init examples
        • YAML Tools
          • How to create a k8s yaml file - How to create YAML config
          • YQ the tool
  • 📚Programming Languages
    • Python
    • Go
    • JavaScript/TypeScript
    • Java
    • Rust
  • Platform Engineering
    • Implementation Guide
  • FinOps
    • Implementation Guide
  • AIOps
    • LLMOps Guide
  • Should Learn
    • Should Learn
    • Linux
      • Commands
      • OS
      • Services
    • Terraform
    • Getting Started - Installation and initial setup [BEGINNER]
    • Cloud Integrations
    • Testing and Validation - Ensuring infrastructure quality
      • Unit Testing
      • Integration Testing
      • End-to-End Testing
      • Terratest Guide
    • Best Practices - Production-ready implementation strategies
      • State Management
      • Security
      • Code Organization
      • Performance
    • Tools & Utilities
    • CI/CD Integration
    • Bicep
    • Kubernetes
      • kubectl
    • Ansible
    • Puppet
    • Java
    • Rust
    • Azure CLI
  • 📖Documentation Best Practices
    • Documentation Strategy
      • Project Documentation
      • Release Notes
      • Static Sites
      • Documentation Templates
      • Real-World Examples
  • 📋Reference Materials
    • Glossary
    • Tool Comparison
    • Tool Decision Guides
    • Recommended Reading
    • Troubleshooting Guide
    • Development Setup
Powered by GitBook
On this page
  • Why Use yq in DevOps & SRE?
  • Real-Life Examples
  • 1. Update Image Tag in All Kubernetes Deployments
  • 2. Extract All Resource Limits for Audit
  • 3. Patch a Value in a CI/CD Pipeline (GitHub Actions)
  • 4. Merge Multiple YAML Files for GitOps
  • 5. Use Environment Variables for Dynamic Values
  • Best Practices (2025)
  • Common Pitfalls
  • Install
Edit on GitHub
  1. Development Environment
  2. DevOps Tools
  3. CLI Tools
  4. YAML Tools

YQ the tool

yq is a lightweight, portable command-line YAML, JSON, and XML processor. It is essential for DevOps and SRE engineers working with Kubernetes, Terraform, Ansible, and CI/CD pipelines across AWS, Azure, GCP, Linux, NixOS, and WSL environments.

Why Use yq in DevOps & SRE?

  • Automate YAML/JSON edits: Update Kubernetes manifests, Terraform variables, and CI/CD configs programmatically.

  • Bulk Operations: Apply changes across multiple files for GitOps, policy enforcement, or compliance.

  • CI/CD Integration: Use yq in GitHub Actions, Azure Pipelines, or GitLab CI/CD for validation, patching, and templating.

  • Cloud-Native: Works seamlessly with cloud IaC and configuration workflows.

Real-Life Examples

1. Update Image Tag in All Kubernetes Deployments

grep -rl 'image:' ./k8s | xargs -I{} yq -i '.spec.template.spec.containers[0].image = "nginx:1.25.0"' {}

2. Extract All Resource Limits for Audit

find ./manifests -name '*.yaml' | xargs -I{} yq '.spec.template.spec.containers[].resources.limits' {}

3. Patch a Value in a CI/CD Pipeline (GitHub Actions)

- name: Patch image tag in deployment
  run: yq -i '.spec.template.spec.containers[0].image = "myrepo/app:${{ github.sha }}"' k8s/deployment.yaml

4. Merge Multiple YAML Files for GitOps

yq ea '. as $item ireduce ({}; . * $item )' overlays/*.yml > merged.yaml

5. Use Environment Variables for Dynamic Values

export VERSION=1.2.3
yq -i '.app.version = strenv(VERSION)' values.yaml

Best Practices (2025)

  • Always validate YAML after edits: kubectl apply --dry-run=client -f file.yaml

  • Use yq in CI/CD for repeatable, automated changes

  • Document yq commands in README or pipeline logs

  • Prefer explicit paths to avoid accidental overwrites

  • Use yq with version control for traceability

Common Pitfalls

  • Overwriting files without backup (-i is destructive)

  • Not validating YAML after bulk edits

  • Using ambiguous paths (be specific to avoid wrong fields)

  • Forgetting to quote strings with special characters

Install

wget

Use wget to download, gzipped pre-compiled binaries:

For instance, VERSION=v4.2.0 and BINARY=yq_linux_amd64

Compressed via tar.gz

wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - |\
  tar xz && mv ${BINARY} /usr/bin/yq
```plaintext

**Plain binary**

```sh
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq
```plaintext

**Latest version**

```sh
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq &&\
    chmod +x /usr/bin/yq
```plaintext

#### MacOS / Linux via Homebrew:

Using [Homebrew](https://brew.sh/)

```shell
brew install yq
```plaintext

#### Linux via snap:

```shell
snap install yq
```plaintext

**Snap notes**

`yq` installs with [_strict confinement_](https://docs.snapcraft.io/snap-confinement/6233) in snap, this means it doesn't have direct access to root files. To read root files you can:

```sh
sudo cat /etc/myfile | yq '.a.path'
```plaintext

And to write to a root file you can either use [sponge](https://linux.die.net/man/1/sponge):

```shell
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo sponge /etc/myfile
```plaintext

or write to a temporary file:

```sh
sudo cat /etc/myfile | yq '.a.path = "value"' | sudo tee /etc/myfile.tmp
sudo mv /etc/myfile.tmp /etc/myfile
rm /etc/myfile.tmp
```plaintext

#### Run with Docker or Podman

**Oneshot use:**

```sh
docker run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
```plaintext

Note that you can run `yq` in docker without network access and other privileges if you desire, namely `--security-opt=no-new-privileges --cap-drop all --network none`.

```shell
podman run --rm -v "${PWD}":/workdir mikefarah/yq [command] [flags] [expression ]FILE...
```plaintext

**Pipe in via STDIN:**

You'll need to pass the `-i\--interactive` flag to docker:

```sh
docker run -i --rm mikefarah/yq '.this.thing' < myfile.yml
```plaintext

```sh
podman run -i --rm mikefarah/yq '.this.thing' < myfile.yml
```plaintext

**Run commands interactively:**

```sh
docker run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```plaintext

```sh
podman run --rm -it -v "${PWD}":/workdir --entrypoint sh mikefarah/yq
```plaintext

It can be useful to have a bash function to avoid typing the whole docker command:

```shell
yq() {
  docker run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```plaintext

```shell
yq() {
  podman run --rm -i -v "${PWD}":/workdir mikefarah/yq "$@"
}
```plaintext

**Running as root:**

`yq`'s container image no longer runs under root ([#860](https://github.com/mikefarah/yq/pull/860)). If you'd like to install more things in the container image, or you're having permissions issues when attempting to read/write files you'll need to either:

```sh
docker run --user="root" -it --entrypoint sh mikefarah/yq
```plaintext

```sh
podman run --user="root" -it --entrypoint sh mikefarah/yq
```plaintext

Or, in your Dockerfile:

```shell
FROM mikefarah/yq

USER root
RUN apk add --no-cache bash
USER yq
```plaintext

**Missing timezone data**

By default, the alpine image yq uses does not include timezone data. If you'd like to use the `tz` operator, you'll need to include this data:

```shell
FROM mikefarah/yq

USER root
RUN apk add --no-cache tzdata
USER yq
```plaintext

**Podman with SELinux**

If you are using podman with SELinux, you will need to set the shared volume flag `:z` on the volume mount:

```sh
-v "${PWD}":/workdir:z
```plaintext

#### GitHub Action

```shell
  - name: Set foobar to cool
    uses: mikefarah/yq@master
    with:
      cmd: yq -i '.foo.bar = "cool"' 'config.yml'
  - name: Get an entry with a variable that might contain dots or spaces
    id: get_username
    uses: mikefarah/yq@master
    with:
      cmd: yq '.all.children.["${{ matrix.ip_address }}"].username' ops/inventories/production.yml
  - name: Reuse a variable obtained in another step
    run: echo ${{ steps.get_username.outputs.result }}
```plaintext

See [https://mikefarah.gitbook.io/yq/usage/github-action](https://mikefarah.gitbook.io/yq/usage/github-action) for more.

#### Go Install:

```sh
go install github.com/mikefarah/yq/v4@latest
```plaintext

---

> **yq Joke:**
> Why did the SRE love yq? Because it could fix YAML faster than they could break it!
PreviousHow to create a k8s yaml file - How to create YAML configNextPython

Last updated 19 days ago

💻
Download the latest binary