DSCI

**File structure** ``` . ├── .dsci │ ├── jobs.yaml │ ├── job_test │ │ ├── job.bash │ │ └── tasks │ │ ├── run_tests │ │ │ └── task.bash │ │ └── check_coverage │ │ └── task.py │ ├── job_build │ │ ├── job.bash │ │ └── tasks │ │ └── docker_build │ │ └── task.bash │ └── job_push │ ├── job.bash │ └── tasks │ └── docker_push │ └── task.bash └── Dockerfile # (placeholder – you will add your Perl app Dockerfile later) ``` --- ## 1️⃣ `jobs.yaml` ```yaml # .dsci/jobs.yaml jobs: - id: test path: job_test/ params: minimal: 85 # minimal coverage percentage (can be overridden) - id: build path: job_build/ params: version: "1.0.0" # image tag version (can be overridden) registry: "registry.example.com" image_name: "perl-app" - id: push path: job_push/ params: version: "1.0.0" registry: "registry.example.com" image_name: "perl-app" ``` --- ## 2️⃣ Test job – runs unit‑tests, generates coverage and validates it ### `job_test/job.bash` ```bash #!/bin/bash # .dsci/job_test/job.bash # execute the two tasks sequentially run_task "run_tests" run_task "check_coverage" ``` ### `job_test/tasks/run_tests/task.bash` ```bash #!/bin/bash # .dsci/job_test/tasks/run_tests/task.bash set -e # ----------------------------------------------------------------- # 1) Install Perl test dependencies (placeholder – adapt to your project) # ----------------------------------------------------------------- # cpanm --quiet --installdeps . # ----------------------------------------------------------------- # ----------------------------------------------------------------- # 2) Run the test suite # ----------------------------------------------------------------- echo "=== Running Perl test suite ===" prove -l t/ || { echo "Tests failed"; exit 1; } # ----------------------------------------------------------------- # 3) Generate coverage report (Devel::Cover) # ----------------------------------------------------------------- echo "=== Generating coverage report ===" cover -report text > ~/artifacts/coverage.txt echo "Coverage report saved to ~/artifacts/coverage.txt" ``` ### `job_test/tasks/check_coverage/task.py` ```python #!/usr/bin/python3 # .dsci/job_test/tasks/check_coverage/task.py import re import sys from pathlib import Path # -------------------------------------------------------------- # Read pipeline parameters # -------------------------------------------------------------- cfg = config() # default to 80 % if the param is missing minimal = float(cfg.get("minimal", 80)) # -------------------------------------------------------------- # Load the coverage report created by the previous task # -------------------------------------------------------------- cov_file = Path.home() / "artifacts/coverage.txt" try: report = cov_file.read_text() except FileNotFoundError: print("❌ Coverage report not found at", cov_file) sys.exit(1) # -------------------------------------------------------------- # Extract the overall coverage percentage. # Devel::Cover's text report contains a line like: # TOTAL 123 98.7% # -------------------------------------------------------------- match = re.search(r"TOTAL\s+\d+\s+([\d\.]+)%", report) if not match: print("❌ Unable to parse coverage from report") sys.exit(1) coverage = float(match.group(1)) print(f"📈 Coverage found: {coverage}% (minimum required: {minimal}%)") # -------------------------------------------------------------- # Fail the job if coverage is below the required threshold # -------------------------------------------------------------- if coverage < minimal: print(f"❌ FAIL: Coverage {coverage}% < required {minimal}%") sys.exit(1) print("✅ PASS: Coverage meets the minimal requirement") ``` --- ## 3️⃣ Build job – builds the Docker image ### `job_build/job.bash` ```bash #!/bin/bash # .dsci/job_build/job.bash run_task "docker_build" ``` ### `job_build/tasks/docker_build/task.bash` ```bash #!/bin/bash # .dsci/job_build/tasks/docker_build/task.bash set -e # ----------------------------------------------------------------- # Read parameters supplied via the pipeline # ----------------------------------------------------------------- REGISTRY=$(config registry) IMAGE_NAME=$(config image_name) VERSION=$(config version) # Provide sensible defaults if the user did not override them : "${REGISTRY:=registry.example.com}" : "${IMAGE_NAME:=perl-app}" : "${VERSION:=latest}" FULL_TAG="${REGISTRY}/${IMAGE_NAME}:${VERSION}" echo "🔨 Building Docker image ${FULL_TAG}" docker build -t "${FULL_TAG}" . # (optional) you could expose the tag to downstream jobs via state, # but the push job rebuilds the same tag from the same parameters. ``` --- ## 4️⃣ Push job – pushes the built image to a container registry ### `job_push/job.bash` ```bash #!/bin/bash # .dsci/job_push/job.bash run_task "docker_push" ``` ### `job_push/tasks/docker_push/task.bash` ```bash #!/bin/bash # .dsci/job_push/tasks/docker_push/task.bash set -e # ----------------------------------------------------------------- # Read the same parameters that were used during the build step # ----------------------------------------------------------------- REGISTRY=$(config registry) IMAGE_NAME=$(config image_name) VERSION=$(config version) : "${REGISTRY:=registry.example.com}" : "${IMAGE_NAME:=perl-app}" : "${VERSION:=latest}" FULL_TAG="${REGISTRY}/${IMAGE_NAME}:${VERSION}" echo "🚀 Pushing Docker image ${FULL_TAG}" docker push "${FULL_TAG}" ``` --- ## How the pipeline works 1. **`test` job** – *Runs the Perl test suite (`prove`) and creates a coverage report (`cover`).* The `check_coverage` Python task reads the `minimal` parameter (in %); if the actual coverage is lower, the task exits with a non‑zero status, causing the whole job (and thus the pipeline) to fail. 2. **`build` job** –