Integrating LLMs into Source Control Workflows

Large Language Models (LLMs) are transforming DevOps practices by enhancing automation, improving code quality, and streamlining source control workflows. This guide provides practical examples for implementing LLMs in modern source control processes across AWS, Azure, and GCP environments.

Use Cases for LLMs in Source Control

1. Pull Request Enhancement

LLMs can significantly improve the pull request process by:

• Automatically generating PR descriptions

• Summarizing code changes

• Identifying potential issues

• Suggesting improvements

• Documenting changes more effectively

2. Commit Message Quality

LLMs can help create structured, informative commit messages by:

• Enforcing conventional commits format

• Expanding terse commit messages

• Linking to relevant issues/documentation

• Suggesting better descriptive text

• Validating commit message quality

3. Code Review Assistance

LLMs can augment human code reviews by:

• Detecting common bugs and anti-patterns

• Ensuring consistency with coding standards

• Identifying security vulnerabilities

• Suggesting optimizations

• Explaining complex code segments

4. Documentation Generation

LLMs can automate documentation tasks by:

• Creating/updating README files

• Generating API documentation

• Documenting module interfaces

• Explaining code functionality

• Creating examples and usage guides

Implementation Patterns

GitHub Copilot for PRs

GitHub Copilot can be integrated directly into the PR workflow:

# .github/workflows/copilot-pr.yml
name: Copilot PR Assistant

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  enhance-pr:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
          
      - name: Generate PR summary
        id: summary
        uses: actions/github-script@v6
        with:
          script: |
            const body = context.payload.pull_request.body || '';
            if (body.includes('[AI-ASSISTED]')) {
              console.log('PR already enhanced by Copilot');
              return;
            }
            
            // Call GitHub Copilot API
            const response = await github.rest.pulls.get({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number
            });
            
            const files = await github.rest.pulls.listFiles({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number
            });
            
            // This is where you'd integrate with the Copilot API
            // For demonstration, we'll just use a template
            const summary = `## [AI-ASSISTED] PR Summary
            
            This PR makes the following changes:
            - Modified ${files.data.length} files
            - Added feature X
            - Fixed bug Y
            
            ### Impact
            [Description of impact would be generated by LLM]
            
            ### Testing
            [Testing recommendations would be generated by LLM]`;
            
            github.rest.pulls.update({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number,
              body: summary + '\n\n' + body
            });

Azure OpenAI for Commit Message Enhancement

Integrate Azure OpenAI with Git hooks to improve commit messages:

#!/bin/bash
# .git/hooks/prepare-commit-msg
# Make executable with: chmod +x .git/hooks/prepare-commit-msg

COMMIT_MSG_FILE=$1
COMMIT_SOURCE=$2

# Don't modify merge or template commit messages
if [ "$COMMIT_SOURCE" = "merge" ] || [ "$COMMIT_SOURCE" = "template" ]; then
  exit 0
fi

# Read the current commit message
current_msg=$(cat "$COMMIT_MSG_FILE")

# Skip if commit message starts with a conventional commit prefix
if [[ "$current_msg" =~ ^(feat|fix|docs|style|refactor|perf|test|chore|build|ci|revert)(\(.+\))?: ]]; then
  exit 0
fi

# Call Azure OpenAI API to enhance the commit message
enhanced_msg=$(curl -s -X POST \
  -H "Content-Type: application/json" \
  -H "api-key: $AZURE_OPENAI_API_KEY" \
  -d "{
    \"messages\": [
      {
        \"role\": \"system\",
        \"content\": \"You are a helpful assistant that improves git commit messages to follow conventional commits format. Convert the provided message into the format: type(scope): description\"
      },
      {
        \"role\": \"user\",
        \"content\": \"$current_msg\"
      }
    ],
    \"temperature\": 0.2,
    \"max_tokens\": 100
  }" \
  "https://$AZURE_OPENAI_ENDPOINT/openai/deployments/$AZURE_OPENAI_MODEL/chat/completions?api-version=2023-05-15" | jq -r '.choices[0].message.content')

# Update the commit message file
echo "$enhanced_msg" > "$COMMIT_MSG_FILE"

GitLab LLM Integration for Code Reviews

Set up an automated code review system using GitLab and self-hosted LLM:

# .gitlab-ci.yml
stages:
  - test
  - review

automated-code-review:
  stage: review
  image: python:3.10
  script:
    - pip install requests
    - pip install transformers torch
    - |
      cat > review_code.py << 'EOF'
      #!/usr/bin/env python
      import os
      import sys
      import requests
      import json
      
      # Connect to your local Ollama server or other LLM API
      LLM_API_URL = os.environ.get("LLM_API_URL", "http://ollama:11434/api/generate")
      
      def get_diff():
          """Get the diff from GitLab CI environment"""
          merge_request_iid = os.environ.get("CI_MERGE_REQUEST_IID")
          project_id = os.environ.get("CI_PROJECT_ID")
          gitlab_token = os.environ.get("GITLAB_TOKEN")
          gitlab_api_url = os.environ.get("CI_API_V4_URL")
          
          url = f"{gitlab_api_url}/projects/{project_id}/merge_requests/{merge_request_iid}/changes"
          headers = {"PRIVATE-TOKEN": gitlab_token}
          response = requests.get(url, headers=headers)
          return response.json()
      
      def analyze_code(diff_data):
          """Use LLM to analyze code changes"""
          changes = []
          for change in diff_data.get("changes", []):
              file_path = change.get("new_path")
              diff = change.get("diff")
              
              # Skip if not code file
              if not any(file_path.endswith(ext) for ext in ['.py', '.js', '.ts', '.go', '.java', '.cs', '.tf', '.yaml', '.yml']):
                  continue
              
              prompt = f"""
              Review this code diff and provide feedback on:
              1. Potential bugs or issues
              2. Security concerns
              3. Performance improvements
              4. Code style suggestions
              
              File: {file_path}
              
              ```
              {diff}
              ```
              
              Provide specific, actionable feedback in bullet points.
              """
              
              response = requests.post(
                  LLM_API_URL,
                  json={
                      "model": "codellama",
                      "prompt": prompt,
                      "temperature": 0.2,
                      "max_tokens": 500
                  }
              )
              
              result = response.json().get("response", "No feedback generated")
              changes.append({"file": file_path, "feedback": result})
          
          return changes
      
      def post_comment(feedback):
          """Post feedback as comment on merge request"""
          merge_request_iid = os.environ.get("CI_MERGE_REQUEST_IID")
          project_id = os.environ.get("CI_PROJECT_ID")
          gitlab_token = os.environ.get("GITLAB_TOKEN")
          gitlab_api_url = os.environ.get("CI_API_V4_URL")
          
          url = f"{gitlab_api_url}/projects/{project_id}/merge_requests/{merge_request_iid}/notes"
          headers = {"PRIVATE-TOKEN": gitlab_token}
          
          comment = "## 🤖 AI Code Review\n\n"
          for item in feedback:
              comment += f"### {item['file']}\n\n{item['feedback']}\n\n"
          
          response = requests.post(
              url, 
              headers=headers,
              json={"body": comment}
          )
          return response.status_code
      
      if __name__ == "__main__":
          diff_data = get_diff()
          feedback = analyze_code(diff_data)
          post_comment(feedback)
      EOF
    - python review_code.py
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

Multi-Cloud Infrastructure Review with LLMs

Create a system that evaluates infrastructure-as-code changes across cloud providers:

# iac_reviewer.py for use in CI/CD pipelines
import os
import requests
import json
import subprocess
import glob
import sys

def get_changed_files():
    """Get files changed in the current PR/MR"""
    if os.environ.get("GITHUB_EVENT_NAME") == "pull_request":
        # GitHub Actions
        event_path = os.environ.get("GITHUB_EVENT_PATH")
        with open(event_path) as f:
            event_data = json.load(f)
        
        pr_number = event_data["pull_request"]["number"]
        repo = event_data["repository"]["full_name"]
        
        # Use GitHub API to get changed files
        token = os.environ.get("GITHUB_TOKEN")
        url = f"https://api.github.com/repos/{repo}/pulls/{pr_number}/files"
        headers = {"Authorization": f"token {token}"}
        response = requests.get(url, headers=headers)
        files = [item["filename"] for item in response.json()]
        
    elif os.environ.get("CI_MERGE_REQUEST_IID"):
        # GitLab CI
        project_id = os.environ.get("CI_PROJECT_ID")
        mr_id = os.environ.get("CI_MERGE_REQUEST_IID")
        gitlab_token = os.environ.get("GITLAB_TOKEN")
        gitlab_api_url = os.environ.get("CI_API_V4_URL")
        
        url = f"{gitlab_api_url}/projects/{project_id}/merge_requests/{mr_id}/changes"
        headers = {"PRIVATE-TOKEN": gitlab_token}
        response = requests.get(url, headers=headers)
        files = [item["new_path"] for item in response.json().get("changes", [])]
    
    else:
        # Fallback to git diff
        cmd = ["git", "diff", "--name-only", "HEAD~1", "HEAD"]
        files = subprocess.check_output(cmd).decode().splitlines()
    
    return files

def classify_cloud_resources(files):
    """Classify files by cloud provider"""
    aws_files = []
    azure_files = []
    gcp_files = []
    
    for file in files:
        if any(aws_pattern in file.lower() for aws_pattern in ["aws", "amazon", "dynamodb", "lambda", "ec2"]):
            aws_files.append(file)
        elif any(azure_pattern in file.lower() for azure_pattern in ["azure", "microsoft", "appservice", "cosmosdb", "azurerm"]):
            azure_files.append(file)
        elif any(gcp_pattern in file.lower() for gcp_pattern in ["gcp", "google", "gke", "cloudfunctions", "bigtable"]):
            gcp_files.append(file)
    
    return {"aws": aws_files, "azure": azure_files, "gcp": gcp_files}

def analyze_with_llm(file_path, cloud_provider):
    """Analyze IaC file with LLM"""
    try:
        with open(file_path, "r") as f:
            content = f.read()
    except Exception as e:
        return f"Error reading file: {str(e)}"
    
    # OpenAI API (could be replaced with any LLM API)
    api_key = os.environ.get("OPENAI_API_KEY")
    url = "https://api.openai.com/v1/chat/completions"
    
    prompt = f"""
    Review this {cloud_provider} infrastructure as code file and provide feedback on:
    1. Security best practices
    2. Cost optimization opportunities
    3. Compliance considerations
    4. Resilience and reliability improvements
    
    File: {file_path}
    
    ```
    {content}
    ```
    
    Format your response as bullet points under each category.
    """
    
    headers = {
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json"
    }
    
    data = {
        "model": "gpt-4",
        "messages": [
            {"role": "system", "content": f"You are an expert DevOps engineer specializing in {cloud_provider} infrastructure."},
            {"role": "user", "content": prompt}
        ],
        "temperature": 0.1
    }
    
    try:
        response = requests.post(url, headers=headers, json=data)
        return response.json()["choices"][0]["message"]["content"]
    except Exception as e:
        return f"Error calling LLM API: {str(e)}"

def main():
    changed_files = get_changed_files()
    cloud_files = classify_cloud_resources(changed_files)
    
    results = {}
    
    for provider, files in cloud_files.items():
        if not files:
            continue
        
        provider_results = []
        for file in files:
            if os.path.exists(file) and any(file.endswith(ext) for ext in [".tf", ".hcl", ".yaml", ".json", ".bicep", ".arm"]):
                analysis = analyze_with_llm(file, provider)
                provider_results.append({"file": file, "analysis": analysis})
        
        results[provider] = provider_results
    
    # Output results - could be posted as PR comment, stored as artifact, etc.
    print(json.dumps(results, indent=2))
    
    # Write to report file
    with open("iac_review_report.md", "w") as f:
        f.write("# Infrastructure as Code Review Report\n\n")
        
        for provider, analyses in results.items():
            f.write(f"## {provider.upper()} Resources\n\n")
            
            if not analyses:
                f.write("No resources identified for review.\n\n")
                continue
                
            for item in analyses:
                f.write(f"### {item['file']}\n\n")
                f.write(f"{item['analysis']}\n\n")

if __name__ == "__main__":
    main()

Real-World Integration Examples

Example 1: GitHub Actions with OpenAI-Enhanced PRs

This workflow uses OpenAI to summarize pull requests and suggest reviewers:

# .github/workflows/enhance-prs.yml
name: Enhance Pull Requests

on:
  pull_request:
    types: [opened, synchronize]

jobs:
  enhance-pr:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: actions/checkout@v3
        with:
          fetch-depth: 0
      
      - name: Diff changes
        id: get-diff
        run: |
          git fetch origin ${{ github.base_ref }}:${{ github.base_ref }}
          DIFF_OUTPUT=$(git diff --stat ${{ github.base_ref }}..HEAD)
          echo "DIFF_OUTPUT<<EOF" >> $GITHUB_ENV
          echo "$DIFF_OUTPUT" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV
      
      - name: Enhance PR with OpenAI
        id: openai
        run: |
          # Call OpenAI API
          response=$(curl -s \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer ${{ secrets.OPENAI_API_KEY }}" \
            -d '{
              "model": "gpt-4",
              "messages": [
                {
                  "role": "system",
                  "content": "You are a helpful AI assistant for DevOps teams. Analyze the git diff stats and PR title to create a comprehensive summary and suggest appropriate reviewers based on the files changed."
                },
                {
                  "role": "user",
                  "content": "PR Title: ${{ github.event.pull_request.title }}\n\nDiff Stats:\n${{ env.DIFF_OUTPUT }}"
                }
              ],
              "temperature": 0.2
            }' \
            https://api.openai.com/v1/chat/completions)
          
          # Extract the generated content
          content=$(echo $response | jq -r '.choices[0].message.content')
          
          # Create output variables
          echo "SUMMARY<<EOF" >> $GITHUB_ENV
          echo "$content" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV
      
      - name: Update PR description
        uses: actions/github-script@v6
        with:
          script: |
            const summary = process.env.SUMMARY;
            const currentBody = context.payload.pull_request.body || '';
            
            // Don't add the summary again if it's already been added
            if (currentBody.includes('## AI-Generated Summary')) {
              return;
            }
            
            const newBody = `## AI-Generated Summary

${summary}

---

${currentBody}`;
            
            github.rest.pulls.update({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number,
              body: newBody
            });

Example 2: Automated Code Documentation for Terraform Modules

This script uses LLMs to generate comprehensive documentation for Terraform modules:

# terraform_docs_generator.py
import os
import glob
import subprocess
import requests
import json
import re

def find_terraform_files(directory):
    """Find all Terraform files in a directory"""
    return glob.glob(os.path.join(directory, "**/*.tf"), recursive=True)

def extract_terraform_components(file_path):
    """Extract resources, variables, outputs from Terraform file"""
    with open(file_path, 'r') as f:
        content = f.read()
    
    # Simple regex-based parsing (a proper parser would be better in production)
    resources = re.findall(r'resource\s+"([^"]+)"\s+"([^"]+)"\s+{', content)
    variables = re.findall(r'variable\s+"([^"]+)"\s+{', content)
    outputs = re.findall(r'output\s+"([^"]+)"\s+{', content)
    
    return {
        'resources': resources,
        'variables': variables,
        'outputs': outputs
    }

def get_module_description(directory, components):
    """Generate module description using LLM"""
    # Get main.tf content for context
    main_tf_path = os.path.join(directory, "main.tf")
    if os.path.exists(main_tf_path):
        with open(main_tf_path, 'r') as f:
            main_content = f.read()
    else:
        main_content = "No main.tf found"
    
    # Prepare component summaries
    resources_str = "\n".join([f"- {r[0]} \"{r[1]}\"" for r in components['resources']])
    variables_str = "\n".join([f"- {v}" for v in components['variables']])
    outputs_str = "\n".join([f"- {o}" for o in components['outputs']])
    
    # Call LLM API (Azure OpenAI example)
    api_key = os.environ.get("AZURE_OPENAI_API_KEY")
    endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")
    deployment = os.environ.get("AZURE_OPENAI_DEPLOYMENT")
    
    url = f"{endpoint}/openai/deployments/{deployment}/chat/completions?api-version=2023-05-15"
    
    prompt = f"""
    Create comprehensive documentation for this Terraform module:
    
    Directory: {directory}
    
    Resources:
    {resources_str}
    
    Variables:
    {variables_str}
    
    Outputs:
    {outputs_str}
    
    Main.tf content excerpt:
    ```hcl
    {main_content[:1500]}  # Limit content length
    ```
    
    Generate the following sections:
    1. Module Description - A clear explanation of what this module creates and its purpose
    2. Architecture - A high-level description of the architecture this module implements
    3. Usage Example - A practical example of how to use this module
    4. Best Practices - Tips for using this module effectively
    
    Format the output in Markdown.
    """
    
    headers = {
        "Content-Type": "application/json",
        "api-key": api_key
    }
    
    data = {
        "messages": [
            {"role": "system", "content": "You are a DevOps documentation expert who creates clear, comprehensive documentation for Terraform modules."},
            {"role": "user", "content": prompt}
        ],
        "temperature": 0.2,
        "max_tokens": 1500
    }
    
    try:
        response = requests.post(url, headers=headers, json=data)
        result = response.json()
        return result['choices'][0]['message']['content']
    except Exception as e:
        return f"Error generating documentation: {str(e)}"

def main():
    # Directory containing Terraform modules
    modules_dir = "terraform/modules"
    
    for module_dir in glob.glob(os.path.join(modules_dir, "*")):
        if not os.path.isdir(module_dir):
            continue
        
        print(f"Processing module: {module_dir}")
        
        # Collect all Terraform files in this module
        tf_files = find_terraform_files(module_dir)
        
        # Extract components from all files
        all_components = {
            'resources': [],
            'variables': [],
            'outputs': []
        }
        
        for tf_file in tf_files:
            components = extract_terraform_components(tf_file)
            all_components['resources'].extend(components['resources'])
            all_components['variables'].extend(components['variables'])
            all_components['outputs'].extend(components['outputs'])
        
        # Generate module documentation
        documentation = get_module_description(module_dir, all_components)
        
        # Write documentation to README.md
        readme_path = os.path.join(module_dir, "README.md")
        with open(readme_path, 'w') as f:
            f.write(documentation)
        
        print(f"Generated documentation saved to {readme_path}")

if __name__ == "__main__":
    main()

Example 3: Commit Quality Analysis for Git Repositories

This script analyzes the quality of commit messages in a repository:

# analyze_commits.py
import subprocess
import re
import json
import requests
import os
import matplotlib.pyplot as plt
from datetime import datetime, timedelta

def get_recent_commits(days=30):
    """Get commits from the last N days"""
    since_date = (datetime.now() - timedelta(days=days)).strftime('%Y-%m-%d')
    cmd = ["git", "log", "--since", since_date, "--pretty=format:%H|%an|%at|%s"]
    output = subprocess.check_output(cmd).decode()
    
    commits = []
    for line in output.strip().split("\n"):
        parts = line.split("|", 3)
        if len(parts) == 4:
            commits.append({
                "hash": parts[0],
                "author": parts[1],
                "timestamp": int(parts[2]),
                "message": parts[3]
            })
    
    return commits

def analyze_commit_quality(message):
    """Analyze commit message quality using LLM"""
    # Use locally hosted Ollama for inference
    url = "http://localhost:11434/api/generate"
    
    prompt = f"""
    Analyze this git commit message and rate its quality:
    
    "{message}"
    
    Rate the message on a scale of 1-10 for:
    1. Clarity (Is it clear what changes were made?)
    2. Specificity (Does it provide specific details?)
    3. Conventional format (Does it follow conventional commits format?)
    4. Context (Does it explain why the change was made?)
    
    Return a JSON object with these ratings and an overall score.
    ```json
    {{
      "clarity": 0,
      "specificity": 0,
      "conventional_format": 0,
      "context": 0,
      "overall": 0,
      "suggestions": ""
    }}
    ```
    """
    
    data = {
        "model": "codellama:latest",
        "prompt": prompt,
        "stream": False
    }
    
    try:
        response = requests.post(url, json=data)
        text = response.json()["response"]
        
        # Extract JSON from response
        json_match = re.search(r'```json\n(.*?)\n```', text, re.DOTALL)
        if json_match:
            json_str = json_match.group(1)
            try:
                return json.loads(json_str)
            except:
                pass
        
        # Try to find JSON without markdown code blocks
        json_match = re.search(r'{[\s\S]*?}', text)
        if json_match:
            json_str = json_match.group(0)
            try:
                return json.loads(json_str)
            except:
                pass
        
        return {
            "clarity": 0,
            "specificity": 0,
            "conventional_format": 0,
            "context": 0,
            "overall": 0,
            "suggestions": "Failed to parse LLM response"
        }
    except Exception as e:
        return {
            "clarity": 0,
            "specificity": 0,
            "conventional_format": 0,
            "context": 0,
            "overall": 0,
            "suggestions": f"Error: {str(e)}"
        }

def generate_report(commit_analyses):
    """Generate a markdown report with visualizations"""
    # Calculate averages by author
    author_stats = {}
    for commit in commit_analyses:
        author = commit["author"]
        analysis = commit["analysis"]
        
        if author not in author_stats:
            author_stats[author] = {
                "count": 0,
                "clarity": 0,
                "specificity": 0,
                "conventional_format": 0,
                "context": 0,
                "overall": 0
            }
        
        author_stats[author]["count"] += 1
        author_stats[author]["clarity"] += analysis.get("clarity", 0)
        author_stats[author]["specificity"] += analysis.get("specificity", 0)
        author_stats[author]["conventional_format"] += analysis.get("conventional_format", 0)
        author_stats[author]["context"] += analysis.get("context", 0)
        author_stats[author]["overall"] += analysis.get("overall", 0)
    
    # Calculate averages
    for author, stats in author_stats.items():
        count = stats["count"]
        stats["clarity"] /= count
        stats["specificity"] /= count
        stats["conventional_format"] /= count
        stats["context"] /= count
        stats["overall"] /= count
    
    # Generate visualizations
    create_charts(author_stats, "commit_quality_charts")
    
    # Generate markdown report
    report = "# Git Commit Quality Report\n\n"
    report += f"Analysis of {len(commit_analyses)} commits\n\n"
    
    # Add stats by author
    report += "## Commit Quality by Author\n\n"
    report += "| Author | Commits | Clarity | Specificity | Convention | Context | Overall |\n"
    report += "| ------ | ------- | ------- | ----------- | ---------- | ------- | ------- |\n"
    
    for author, stats in author_stats.items():
        report += f"| {author} | {stats['count']} | {stats['clarity']:.1f} | {stats['specificity']:.1f} | {stats['conventional_format']:.1f} | {stats['context']:.1f} | {stats['overall']:.1f} |\n"
    
    # Add recommendations
    report += "\n## Top Recommendations\n\n"
    
    # Find worst areas
    all_scores = []
    for commit in commit_analyses:
        analysis = commit["analysis"]
        for metric in ["clarity", "specificity", "conventional_format", "context"]:
            all_scores.append({
                "commit": commit["hash"][:7],
                "message": commit["message"],
                "metric": metric,
                "score": analysis.get(metric, 0),
                "suggestion": analysis.get("suggestions", "")
            })
    
    # Sort by score (ascending)
    all_scores.sort(key=lambda x: x["score"])
    
    # Get 5 worst scores
    for score_data in all_scores[:5]:
        report += f"### Improve {score_data['metric']} (Score: {score_data['score']})\n\n"
        report += f"**Commit**: {score_data['commit']} \"{score_data['message']}\"\n\n"
        report += f"**Suggestion**: {score_data['suggestion']}\n\n"
    
    # Add chart references
    report += "\n## Charts\n\n"
    report += "![Commit Quality by Author](./commit_quality_charts/overall_quality.png)\n\n"
    report += "![Metrics Breakdown](./commit_quality_charts/metrics_breakdown.png)\n\n"
    
    # Write report to file
    with open("commit_quality_report.md", "w") as f:
        f.write(report)
    
    return report

def create_charts(author_stats, output_dir):
    """Create visualization charts for the report"""
    os.makedirs(output_dir, exist_ok=True)
    
    # Overall quality by author
    authors = list(author_stats.keys())
    overall_scores = [stats["overall"] for stats in author_stats.values()]
    
    plt.figure(figsize=(10, 6))
    plt.bar(authors, overall_scores)
    plt.title("Overall Commit Quality by Author")
    plt.xlabel("Author")
    plt.ylabel("Score (0-10)")
    plt.ylim(0, 10)
    plt.xticks(rotation=45, ha="right")
    plt.tight_layout()
    plt.savefig(f"{output_dir}/overall_quality.png")
    plt.close()
    
    # Metrics breakdown
    metrics = ["clarity", "specificity", "conventional_format", "context"]
    
    fig, ax = plt.subplots(figsize=(12, 8))
    x = np.arange(len(authors))
    width = 0.2
    
    for i, metric in enumerate(metrics):
        values = [stats[metric] for stats in author_stats.values()]
        ax.bar(x + (i - 1.5) * width, values, width, label=metric.capitalize())
    
    ax.set_title("Commit Quality Metrics by Author")
    ax.set_xlabel("Author")
    ax.set_ylabel("Score (0-10)")
    ax.set_ylim(0, 10)
    ax.set_xticks(x)
    ax.set_xticklabels(authors, rotation=45, ha="right")
    ax.legend()
    
    plt.tight_layout()
    plt.savefig(f"{output_dir}/metrics_breakdown.png")
    plt.close()

def main():
    # Get recent commits
    commits = get_recent_commits(days=30)
    print(f"Analyzing {len(commits)} commits...")
    
    # Analyze each commit
    commit_analyses = []
    for i, commit in enumerate(commits):
        print(f"Analyzing commit {i+1}/{len(commits)}: {commit['hash'][:7]}")
        analysis = analyze_commit_quality(commit["message"])
        
        commit_analyses.append({
            "hash": commit["hash"],
            "author": commit["author"],
            "message": commit["message"],
            "analysis": analysis
        })
    
    # Generate report
    report_path = generate_report(commit_analyses)
    print(f"Report generated: {report_path}")
    
    # Save raw data
    with open("commit_analyses.json", "w") as f:
        json.dump(commit_analyses, f, indent=2)

if __name__ == "__main__":
    main()

Best Practices for LLM Integration

1. Use LLMs Responsibly

• Keep sensitive information out of LLM prompts

• Validate LLM output before acting on it

• Monitor LLM usage for cost control

• Use private or air-gapped models for sensitive code bases

2. Balance Automation and Human Oversight

• Use LLMs to augment human workflows, not replace them

• Keep humans in the loop for critical decisions

• Implement approval gates for automated changes

• Provide options to skip or override LLM suggestions

3. Monitor and Improve Performance

• Collect feedback on LLM-generated content

• Track metrics on acceptance rates of suggestions

• Adjust prompts and models based on feedback

• Retrain or fine-tune models for your specific domain

4. Integrate with Existing Tools

• Leverage Git hooks for seamless integration

• Connect with CI/CD pipelines for automated analysis

• Integrate with project management systems

• Add LLM capabilities to existing code review tools

Conclusion

LLM integration into source control workflows offers significant productivity improvements, from better code reviews to enhanced documentation and more informative commit messages. By following the patterns and examples in this guide, DevOps teams can build more effective, efficient, and collaborative development processes.

As LLM technology continues to evolve, these integrations will become more sophisticated, providing even greater value to development teams while maintaining the necessary human oversight and governance required for enterprise applications.