Scaling et CI/CD du site Do-It

Ce script permet de réencoder les images en WebP, les vidéos en MP4 et les audios en Opus. Il permet également de mettre à jour les références des médias dans les fichiers HTML et de réécrire l'historique Git pour supprimer les médias originaux.

#!/bin/bash

if [[ "$1" == "--rewrite-git-history" ]]; then
    REWRITE_GIT_HISTORY=true
    shift
fi

if [[ "$1" == "-h" || "$1" == "--help" ]]; then
    echo "Usage: optimizer.bash <image|video|audio>"
    echo "Optimize media files in the current directory."
    exit 0
fi

temp_file=$(mktemp) # Temporary file to store paths for git filter-repo

if [[ "$1" == "image" || "$2" == "image" || "$3" == "image" ]]; then
    # Process image files (JPG, JPEG, PNG)
    find src -type f \( -name "*.jpg" -o -name "*.jpeg" -o -name "*.png" -o -name "*.JPG" -o -name "*.JPEG" -o -name "*.PNG" \) -print0 | while IFS= read -r -d '' img; do
        ffmpeg -y -i "$img" -q:v 75 "${img%.*}.webp" && \
        echo "$img" >> "$temp_file" && \
        rm "$img"

        if [[ "$REWRITE_GIT_HISTORY" == "true" ]]; then
             git rm --cached "$img"
        fi
    done

    # Update image references in HTML files
    find src -type f -name "*.md" -exec sed -i \
        -e '/http/! s/\.png/\.webp/g' \
        -e '/http/! s/\.PNG/\.webp/g' \
        -e '/http/! s/\.jpg/\.webp/g' \
        -e '/http/! s/\.JPG/\.webp/g' \
        -e '/http/! s/\.jpeg/\.webp/g' \
        -e '/http/! s/\.JPEG/\.webp/g' {} +

elif [[ "$1" == "video" || "$2" == "video" || "$3" == "video" ]]; then
    # Process video files (MP4, MOV, MKV)
    find src -type f \( -name "*.mp4" -o -name "*.mov" -o -name "*.mkv" \) -print0 | while IFS= read -r -d '' video; do
        ffmpeg -y -i "$video" -c:v libx264 -preset veryslow -c:a aac -b:a 96k "${video%.*}.compressed.mp4" && \
        echo "$video" >> "$temp_file" && \
        rm "$video" && \
        mv "${video%.*}.compressed.mp4" "${video%.*}.mp4"

        if [[ "$REWRITE_GIT_HISTORY" == "true" ]]; then
             git rm --cached "$video"
        fi
    done

    # Update video references in HTML files
    find src -type f -name "*.md" -exec sed -i \
        -e '/http/! s/\.mp4/\.compressed\.mp4/g' \
        -e '/http/! s/\.mov/\.compressed\.mp4/g' \
        -e '/http/! s/\.mkv/\.compressed\.mp4/g' {} +


elif [[ "$1" == "audio" || "$2" == "audio" || "$3" == "audio" ]]; then
    # Process audio files (WAV, FLAC, MP3, OGG)
    find src -type f \( -name "*.wav" -o -name "*.flac" -o -name "*.mp3" -o -name "*.ogg" \) -print0 | while IFS= read -r -d '' audio; do
        ffmpeg -y -i "$audio" -c:a libopus -b:a 96k "${audio%.*}.opus" && \
        echo "$audio" >> "$temp_file" && \
        rm "$audio"

        if [[ "$REWRITE_GIT_HISTORY" == "true" ]]; then
             git rm --cached "$audio"
        fi
    done

    # Update audio references in HTML files
    find src -type f -name "*.md" -exec sed -i \
        -e '/http/! s/\.wav/\.opus/g' \
        -e '/http/! s/\.flac/\.opus/g' \
        -e '/http/! s/\.mp3/\.opus/g' \
        -e '/http/! s/\.ogg/\.opus/g' {} +

else
    echo "Usage: optimizer.bash <image|video|audio>"
    exit 1
fi

# Rewrite git history for all stored paths
if [[ -s "$temp_file" && "$REWRITE_GIT_HISTORY" == "true" ]]; then
    git filter-repo --invert-paths --paths-from-file "$temp_file" --force
fi

# Cleanup temporary file
rm "$temp_file"

Cette pipeline GitHub Actions utilise du caching pour accélérer le build et lance tous nos tests de conformité.
Elle ne se lance que lorsqu'une merge request est acceptée ou lorsqu'un submodule est mis à jour. Finito les push sur main!

name: check-compliance

on:
  repository_dispatch:
    types: [submodule-update]
  pull_request:
    types:
      - closed
    branches:
      - main

env:
  NODE_VERSION: "20.7" # Define the Node.js version here

jobs:
  check-compliance:
    if: github.event.pull_request.merged == true
    runs-on: ubuntu-latest
    steps:
      # Checkout code without initializing submodules
      - uses: actions/checkout@v4
        with:
          submodules: false # Defer submodule initialization

      # Restore Git Submodules Cache
      - name: Restore Git Submodules Cache
        uses: actions/cache@v4
        with:
          path: .git/modules
          key: submodules-$
          restore-keys: |
            submodules-$
            submodules-

      # Git Submodule Update
      - name: Git Submodule Update
        run: |
          git submodule sync
          git submodule update --init --recursive

      # Cache Git Submodules
      - name: Cache Git Submodules
        uses: actions/cache@v4
        with:
          path: .git/modules
          key: submodules-$

      # Restore Node.js Modules Cache
      - name: Restore Node.js Modules Cache
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: npm-cache-$-$
          restore-keys: |
            npm-cache-$-$
            npm-cache-$-
            npm-cache-

      # Use Node.js
      - name: Use Node.js $
        uses: actions/setup-node@v4
        with:
          node-version: $

      # Install dependencies
      - name: Install dependencies
        run: npm ci

      # Save Node.js Modules Cache
      - name: Save Node.js Modules Cache
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: npm-cache-$-$

      - name: Check Compliance
        run:  npm run check-compliance

Cette pipeline GitHub Actions construit le site Do-It à partir des submodules Git et applique les changements sur la branche gh-pages pour déployer le site. Elle ne conserve que la dernière version du build pour éviter de faire grossir le repository Git inutilement.

name: build

on:
  workflow_run:
    workflows: ["check-compliance"]
    types:
      - completed

env:
  NODE_VERSION: "20.7" # Define the Node.js version here

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      # Checkout code without initializing submodules
      - uses: actions/checkout@v4
        with:
          submodules: false # Defer submodule initialization

      # Restore Git Submodules Cache
      - name: Restore Git Submodules Cache
        uses: actions/cache@v4
        with:
          path: .git/modules
          key: submodules-$
          restore-keys: |
            submodules-$
            submodules-

      # Git Submodule Update
      - name: Git Submodule Update
        run: |
          git submodule sync
          git submodule update --init --recursive

      # Cache Git Submodules
      - name: Cache Git Submodules
        uses: actions/cache@v4
        with:
          path: .git/modules
          key: submodules-$

      # Restore Node.js Modules Cache
      - name: Restore Node.js Modules Cache
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: npm-cache-$-$
          restore-keys: |
            npm-cache-$-$
            npm-cache-$-
            npm-cache-

      # Use Node.js
      - name: Use Node.js $
        uses: actions/setup-node@v4
        with:
          node-version: $

      # Install dependencies
      - name: Install dependencies
        run: npm ci

      # Save Node.js Modules Cache
      - name: Save Node.js Modules Cache
        uses: actions/cache@v4
        with:
          path: ~/.npm
          key: npm-cache-$-$

      - name: Build
        run: |
          npm run build-github
          npm run node-modules-front
          npm run build-tailwind

    outputs:
      build-dir: ./dist

  deploy:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Deploy
        uses: peaceiris/actions-gh-pages@v4
        with:
          publish_dir: $NaN
          github_token: $
          force_orphan: true
          publish_branch: gh-pages