DSCI Report: dsci@2169
15:43:53 :: [repository] - index updated from http://sparrowhub.io/repo/api/v1/index run stage: main [task run: task.bash - create directory scm] [task stdout] 15:43:54 :: directory path: scm 15:43:54 :: directory owner: <ubuntu> 15:43:54 :: directory group: <ubuntu> 15:43:54 :: directory access rights: drwxr-xr-x scm: http://localhost:8080/test1.git sha: 1c9fb55 message: feat: add or update file via dsci web by dsci@sparrowhub.io - manual run container runtime: podman raku version Raku++ (rakupp) 3.25.0 — a Raku interpreter and compiler in C++ Implements Raku 6.d, and 6.e under `use v6.e.PREVIEW`. Build v3.25.0 (2026-09-03), x86_64-linux, clang 18.1.3 Home https://raku.online — docs, a tour of the language, and a browser playground sparrow version 0.0.94 runner version: 0.1.1 ======= [task run: task.bash - bash: git config --global advice.detachedHead false] [task stdout] [task run: task.bash - bash: git clone http://localhost:8080/test1.git .] [task stdout] 15:43:54 :: /home/ubuntu/.dsci/.sparky/.cache/1c9fb55.1789573431/.sparrowdo/scm [task stderr] 15:43:54 :: Cloning into '.'... [task run: task.bash - bash: git checkout 1c9fb55] [task stdout] 15:43:55 :: /home/ubuntu/.dsci/.sparky/.cache/1c9fb55.1789573431/.sparrowdo/scm [task stderr] 15:43:55 :: HEAD is now at 1c9fb55 feat: add or update file via dsci web pipeline mode: localhost [task run: task.bash - bash: podman stop -t 1 dsci-agent || :] [task stdout] 15:43:56 :: dsci-agent [task stderr] 15:43:56 :: time="2026-09-16T15:43:56Z" level=warning msg="StopSignal SIGTERM failed to stop container dsci-agent in 1 seconds, resorting to SIGKILL" [task run: task.bash - bash: podman rm dsci-agent || :] [task stdout] [task stderr] 15:43:57 :: Error: no container with ID or name "dsci-agent" found: no such container run dsci agent container from image: melezhik/dsci-agent-rakupp podman run --network host -dit --rm --name dsci-agent --entrypoint /bin/sh melezhik/dsci-agent-rakupp [task run: task.bash - bash: podman run --network host -dit --rm --name dsci-a ...] [task stdout] 15:43:57 :: d249ea91688ee29abae76f297b21708c2cff05db0640c16bc18494c0f3415280 [task stderr] 15:43:57 :: time="2026-09-16T15:43:57Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly" ai_agent_message set: 437265617465206473636920706970656c696e6520666f722052616b756c616e672070726f6a6563742c207573652070726f76653620666f7220756e6974207465737473202c20696e7374616c6c2073716c69746520616e64206375726c2061732073797374656d20646570732e20496e7374616c6c20616e642072756e206d6172696164622061732073657276696365 >>> POST http://127.0.0.1:8181/stash >>> POST http://127.0.0.1:8181/queue >>> wait for jobs, timeout: 600 sec >>> GET http://127.0.0.1:8181/stash/job.run/jyxwgvrandbqlfecisuk.3912954 job test ... [OK] ... >>> GET job report, status=200 15:44:03 :: [repository] - index updated from http://sparrowhub.io/repo/api/v1/index run stage: job-run raku version Raku++ (rakupp) 3.25.0 — a Raku interpreter and compiler in C++ Implements Raku 6.d, and 6.e under `use v6.e.PREVIEW`. Build v3.25.0 (2026-09-03), x86_64-linux, clang 18.1.3 Home https://raku.online — docs, a tour of the language, and a browser playground sparrow version 0.0.94 runner version: 0.1.1 run job: test === >>> GET http://127.0.0.1:8181/stash/job.run/jyxwgvrandbqlfecisuk.3912954 [task run: task.bash - create directory scm] [task stdout] 15:44:04 :: directory path: scm 15:44:04 :: directory owner: <ubuntu> 15:44:04 :: directory group: <ubuntu> 15:44:04 :: directory access rights: drwxr-xr-x [task run: task.bash - bash: git clone http://localhost:8080/test1.git .] [task stdout] 15:44:04 :: /home/ubuntu/.dsci/.sparky/.cache/jyxwgvrandbqlfecisuk.3912954/.sparrowdo/scm [task stderr] 15:44:04 :: Cloning into '.'... [task run: task.bash - bash: git checkout 1c9fb55] [task stdout] 15:44:04 :: /home/ubuntu/.dsci/.sparky/.cache/jyxwgvrandbqlfecisuk.3912954/.sparrowdo/scm [task stderr] 15:44:04 :: HEAD is now at 1c9fb55 feat: add or update file via dsci web [task run: task.bash - bash: prepare dsci scripts] [task stdout] >>> GET http://127.0.0.1:8181/stash/job.run/jyxwgvrandbqlfecisuk.3912954 [task run: task.bash - .] [task stdout] 15:44:05 :: total 8 15:44:05 :: -rw-r--r-- 1 ubuntu ubuntu 109 Sep 16 15:44 jobs.yaml 15:44:05 :: -rw-r--r-- 1 ubuntu ubuntu 301 Sep 16 15:44 task.bash 15:44:05 :: 437265617465206473636920706970656c696e6520666f722052616b756c616e672070726f6a6563742c207573652070726f76653620666f7220756e6974207465737473202c20696e7374616c6c2073716c69746520616e64206375726c2061732073797374656d20646570732e20496e7374616c6c20616e642072756e206d6172696164622061732073657276696365 15:44:12 :: === 15:44:12 :: Create dsci pipeline for Rakulang project, use prove6 for unit tests , install sqlite and curl as system deps. Install and run mariadb as service 15:44:12 :: === 15:44:12 :: ## 1️⃣ Repository layout for the **Rakulang** DSCI pipeline 15:44:12 :: 15:44:12 :: ``` 15:44:12 :: . 15:44:12 :: ├── jobs.yaml # pipeline definition (list of jobs) 15:44:12 :: └── .dsci 15:44:12 :: ├── install_deps/ 15:44:12 :: │ ├── job.bash # job that installs system deps 15:44:12 :: │ └── tasks/ 15:44:12 :: │ └── sys_packages/ 15:44:12 :: │ └── task.bash # installs sqlite3 & curl 15:44:12 :: ├── mariadb/ 15:44:12 :: │ ├── config.yaml # default DB credentials (can be overridden) 15:44:12 :: │ ├── job.bash # job that installs & starts MariaDB 15:44:12 :: │ └── tasks/ 15:44:12 :: │ └── start/ 15:44:12 :: │ └── task.bash # installs mariadb, creates DB & exports state 15:44:12 :: └── test/ 15:44:12 :: ├── job.bash # job that runs unit‑tests with prove6 15:44:12 :: └── tasks/ 15:44:12 :: └── run_prove/ 15:44:12 :: └── task.bash # executes `prove6 -r t/` 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ## 2️⃣ `jobs.yaml` – only the **job list** (no logic, no matrices) 15:44:12 :: 15:44:12 :: ```yaml 15:44:12 :: # .dsci/jobs.yaml 15:44:12 :: jobs: 15:44:12 :: - id: install_deps 15:44:12 :: path: .dsci/install_deps/ 15:44:12 :: - id: mariadb 15:44:12 :: path: .dsci/mariadb/ 15:44:12 :: # you could override defaults here, e.g. 15:44:12 :: # params: 15:44:12 :: # db_user: ci_user 15:44:12 :: # db_pass: super_secret 15:44:12 :: # db_name: rakulang 15:44:12 :: - id: test 15:44:12 :: path: .dsci/test/ 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: *Each entry only declares the job id and where its source lives. 15:44:12 :: All the real work lives in the `job.*` / `task.*` files.* 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ## 3️⃣ Job **install_deps** – Bash job that calls a single task 15:44:12 :: 15:44:12 :: ### `install_deps/job.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/install_deps/job.bash 15:44:12 :: 15:44:12 :: # Run the only task in this job 15:44:12 :: run_task "sys_packages" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: ### `install_deps/tasks/sys_packages/task.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/install_deps/tasks/sys_packages/task.bash 15:44:12 :: 15:44:12 :: set -euo pipefail 15:44:12 :: 15:44:12 :: echo "=== Installing system dependencies (sqlite3 & curl) ===" 15:44:12 :: 15:44:12 :: # DSCI jobs usually run in an Alpine container – use apk. 15:44:12 :: # If you run on Debian/Ubuntu replace with apt‑get. 15:44:12 :: if command -v apk >/dev/null 2>&1; then 15:44:12 :: apk update 15:44:12 :: apk add --no-cache sqlite curl 15:44:12 :: else 15:44:12 :: apt-get update -y 15:44:12 :: DEBIAN_FRONTEND=noninteractive apt-get install -y sqlite3 curl 15:44:12 :: fi 15:44:12 :: 15:44:12 :: echo "=== System deps installed ===" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: *No state is needed for this job – it just prepares the environment.* 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ## 4️⃣ Job **mariadb** – installs MariaDB, starts it as a background service and shares connection info via **state** 15:44:12 :: 15:44:12 :: ### `mariadb/config.yaml` *(default parameters, can be overridden in `jobs.yaml`)* 15:44:12 :: 15:44:12 :: ```yaml 15:44:12 :: # .dsci/mariadb/config.yaml 15:44:12 :: db_user: rakulang_user 15:44:12 :: db_pass: rakulang_pass 15:44:12 :: db_name: rakulang 15:44:12 :: db_port: 3306 # default MariaDB port inside the container 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: ### `mariadb/job.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/mariadb/job.bash 15:44:12 :: 15:44:12 :: # Run the start task – it will install MariaDB, create DB/user, 15:44:12 :: # start the server and export the connection data as state. 15:44:12 :: run_task "start" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: ### `mariadb/tasks/start/task.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/mariadb/tasks/start/task.bash 15:44:12 :: 15:44:12 :: set -euo pipefail 15:44:12 :: 15:44:12 :: # ------------------------------------------------------------------ 15:44:12 :: # 1️⃣ Read job parameters (defaults are in config.yaml) 15:44:12 :: # ------------------------------------------------------------------ 15:44:12 :: DB_USER=$(config db_user) 15:44:12 :: DB_PASS=$(config db_pass) 15:44:12 :: DB_NAME=$(config db_name) 15:44:12 :: DB_PORT=$(config db_port) 15:44:12 :: 15:44:12 :: echo "=== Installing MariaDB ===" 15:44:12 :: if command -v apk >/dev/null 2>&1; then 15:44:12 :: apk update 15:44:12 :: apk add --no-cache mariadb mariadb-client 15:44:12 :: else 15:44:12 :: apt-get update -y 15:44:12 :: DEBIAN_FRONTEND=noninteractive apt-get install -y mariadb-server mariadb-client 15:44:12 :: fi 15:44:12 :: 15:44:12 :: echo "=== Initialising MariaDB data directory ===" 15:44:12 :: # Initialise the data directory if it is empty 15:44:12 :: if [ ! -d /var/lib/mysql/mysql ]; then 15:44:12 :: mariadb-install-db --user=mysql --datadir=/var/lib/mysql 15:44:12 :: fi 15:44:12 :: 15:44:12 :: echo "=== Starting MariaDB in the background ===" 15:44:12 :: # Run MariaDB as a daemon (no systemd) – we keep the PID to stop later if needed 15:44:12 :: mysqld --datadir=/var/lib/mysql --socket=/var/lib/mysql/mysql.sock & 15:44:12 :: MYSQL_PID=$! 15:44:12 :: 15:44:12 :: # Give the server a few seconds to become ready 15:44:12 :: sleep 5 15:44:12 :: 15:44:12 :: echo "=== Securing & creating DB/user ===" 15:44:12 :: # Basic security (no root password) and our test user 15:44:12 :: mysql -uroot <<SQL 15:44:12 :: DELETE FROM mysql.user WHERE User=''; 15:44:12 :: CREATE USER '${DB_USER}'@'%' IDENTIFIED BY '${DB_PASS}'; 15:44:12 :: GRANT ALL PRIVILEGES ON ${DB_NAME}.* TO '${DB_USER}'@'%'; 15:44:12 :: FLUSH PRIVILEGES; 15:44:12 :: SQL 15:44:12 :: 15:44:12 :: # Create the database if it does not exist 15:44:12 :: mysql -uroot -e "CREATE DATABASE IF NOT EXISTS ${DB_NAME};" 15:44:12 :: 15:44:12 :: echo "=== MariaDB is up and running (PID $MYSQL_PID) ===" 15:44:12 :: 15:44:12 :: # ------------------------------------------------------------------ 15:44:12 :: # 2️⃣ Export connection details as **state** so downstream jobs can use them 15:44:12 :: # ------------------------------------------------------------------ 15:44:12 :: # NOTE: update_state expects a *single* JSON object – otherwise previous keys are lost. 15:44:12 :: # The DSCI SDK makes `update_state` available in Bash. 15:44:12 :: update_state "$(cat <<EOF 15:44:12 :: { 15:44:12 :: "host": "127.0.0.1", 15:44:12 :: "port": ${DB_PORT}, 15:44:12 :: "user": "${DB_USER}", 15:44:12 :: "password": "${DB_PASS}", 15:44:12 :: "database": "${DB_NAME}" 15:44:12 :: } 15:44:12 :: EOF 15:44:12 :: )" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: *The task stores the connection data under the key `"_dsci_"]["mariadb"]` automatically (the SDK prefixes it). 15:44:12 :: Down‑stream jobs can read it with `config | jq '._dsci_.mariadb'`.* 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ## 5️⃣ Job **test** – runs the Rakulang unit‑tests with **prove6** 15:44:12 :: 15:44:12 :: (Assumes the project already contains a `t/` directory with Perl test files.) 15:44:12 :: 15:44:12 :: ### `test/job.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/test/job.bash 15:44:12 :: 15:44:12 :: # 1️⃣ Pull MariaDB connection details from upstream job state (if needed) 15:44:12 :: # – we just demonstrate reading them; the test suite can use them via env vars. 15:44:12 :: DB_INFO=$(config | jq -r '._dsci_.mariadb') 15:44:12 :: export DB_HOST=$(echo "$DB_INFO" | jq -r '.host') 15:44:12 :: export DB_PORT=$(echo "$DB_INFO" | jq -r '.port') 15:44:12 :: export DB_USER=$(echo "$DB_INFO" | jq -r '.user') 15:44:12 :: export DB_PASSWORD=$(echo "$DB_INFO" | jq -r '.password') 15:44:12 :: export DB_NAME=$(echo "$DB_INFO" | jq -r '.database') 15:44:12 :: 15:44:12 :: echo "=== DB connection for tests ===" 15:44:12 :: echo " host=$DB_HOST port=$DB_PORT user=$DB_USER db=$DB_NAME" 15:44:12 :: 15:44:12 :: # 2️⃣ Run the test task (prove6) 15:44:12 :: run_task "run_prove" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: ### `test/tasks/run_prove/task.bash` 15:44:12 :: 15:44:12 :: ```bash 15:44:12 :: #!/usr/bin/env bash 15:44:12 :: # .dsci/test/tasks/run_prove/task.bash 15:44:12 :: 15:44:12 :: set -euo pipefail 15:44:12 :: 15:44:12 :: echo "=== Installing test runner (prove6) ===" 15:44:12 :: # prove6 is a Perl testing harness – install it via cpanminus or the system package. 15:44:12 :: if command -v cpanm >/dev/null 2>&1; then 15:44:12 :: cpanm Test::More Test::Simple 15:44:12 :: cpanm App::Prove 15:44:12 :: else 15:44:12 :: # Alpine has a package named perl-app-prove, Debian/Ubuntu: perl-prove 15:44:12 :: if command -v apk >/dev/null 2>&1; then 15:44:12 :: apk add --no-cache perl-app-prove 15:44:12 :: else 15:44:12 :: apt-get update -y 15:44:12 :: DEBIAN_FRONTEND=noninteractive apt-get install -y perl-prove 15:44:12 :: fi 15:44:12 :: fi 15:44:12 :: 15:44:12 :: echo "=== Running unit tests with prove6 ===" 15:44:12 :: # `prove6 -r` runs recursively all *.t files under the given directory. 15:44:12 :: prove6 -r t/ | tee ~/artifacts/test_output.txt 15:44:12 :: 15:44:12 :: # Optionally capture the exit code as state (e.g., for a later “publish” job) 15:44:12 :: TEST_STATUS=$? 15:44:12 :: update_state "$(cat <<EOF 15:44:12 :: { 15:44:12 :: "test_exit_code": $TEST_STATUS 15:44:12 :: } 15:44:12 :: EOF 15:44:12 :: )" 15:44:12 :: ``` 15:44:12 :: 15:44:12 :: *The test output is stored in `~/artifacts/test_output.txt` so any later job can read it.* 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ## 6️⃣ How the pipeline works (high‑level flow) 15:44:12 :: 15:44:12 :: 1. **install_deps** – Guarantees `sqlite3` and `curl` are present for the rest of the pipeline. 15:44:12 :: 2. **mariadb** – Installs MariaDB, starts it, creates a database/user, and **exports** the connection information as state (`_dsci_.mariadb`). 15:44:12 :: 3. **test** – Reads the DB connection info from state, exports it as environment variables, installs `prove6`, and runs the Rakulang Perl test suite. Test logs are saved as an artifact, and the exit status is also stored in state for potential downstream jobs. 15:44:12 :: 15:44:12 :: All jobs are **self‑contained** Bash scripts that only use the DSCI SDK helpers: 15:44:12 :: 15:44:12 :: * `config` – read job parameters or upstream state 15:44:12 :: * `run_task` – invoke a task inside the same job 15:44:12 :: * `update_state` – share data with later jobs 15:44:12 :: 15:44:12 :: No complex YAML, no external CI‑system actions, and no hidden magic – just plain Bash (and a tiny bit of Python could be added later if you need more sophisticated processing). 15:44:12 :: 15:44:12 :: --- 15:44:12 :: 15:44:12 :: ### 🎉 Ready to go! 15:44:12 :: 15:44:12 :: Add the files exactly as shown, commit them to your repository, and DSCI will execute the three jobs in order, providing a clean, maintainable pipeline for the **Rakulang** project. [task stderr] 15:44:12 :: 15:44:12 :: real 0m7.002s 15:44:12 :: user 0m0.674s 15:44:12 :: sys 0m0.131s saving job artifacts ... >> PUT http://127.0.0.1:8181/file/project/dsci/job/1c9fb55.1789573431/filename/answer.md >>> POST http://127.0.0.1:8181/stash --- ---