refactor CI into a single job and cache the dora CLI
validate / validate (pull_request) Successful in 59m51s
validate / validate (pull_request) Successful in 59m51s
The two jobs each paid for a checkout, an apt-get update and an OpenCV shared library install -- about twenty seconds apiece -- to run gates that together take about a second. Merging them pays that once. The dora CLI is a static binary pinned to DORA_VERSION, so it caches exactly.
This commit is contained in:
@@ -1,6 +1,11 @@
|
|||||||
# Two gates, neither of which needs a camera or a display: the example dataflow
|
# Three gates, none of which needs a camera or a display: the example dataflow
|
||||||
# still matches the typed contract in dora-node.yml and still runs end to end,
|
# still matches the typed contracts in the two dora-node.yml files, it still
|
||||||
# and the node's own unit tests pass.
|
# runs end to end, and the node's own unit tests pass.
|
||||||
|
#
|
||||||
|
# All three share one job on purpose. The work itself is trivial -- the unit
|
||||||
|
# tests take under a second -- and is dwarfed by preparing the runner, so
|
||||||
|
# splitting the gates across jobs mostly buys a second checkout and a second
|
||||||
|
# OpenCV install. A step that fails is still named in the log.
|
||||||
name: validate
|
name: validate
|
||||||
|
|
||||||
# A pull request would otherwise run this twice on the same commit: once for
|
# A pull request would otherwise run this twice on the same commit: once for
|
||||||
@@ -15,13 +20,15 @@ env:
|
|||||||
DORA_TARGET: x86_64-unknown-linux-gnu
|
DORA_TARGET: x86_64-unknown-linux-gnu
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
dataflow:
|
validate:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
# The opencv-python wheel links against libGL at import time, so cv2
|
# The opencv-python wheel links against libGL at import time, so cv2
|
||||||
# cannot even be imported on the bare runner image.
|
# cannot even be imported on the bare runner image. This is the most
|
||||||
|
# expensive step in the job by a wide margin; see AGENTS.md for the
|
||||||
|
# headless-wheel alternative that would remove it.
|
||||||
- name: Install the OpenCV shared libraries
|
- name: Install the OpenCV shared libraries
|
||||||
run: |
|
run: |
|
||||||
sudo apt-get update
|
sudo apt-get update
|
||||||
@@ -31,20 +38,43 @@ jobs:
|
|||||||
with:
|
with:
|
||||||
enable-cache: true
|
enable-cache: true
|
||||||
|
|
||||||
|
# The CLI is a single static binary pinned to DORA_VERSION, so the cache
|
||||||
|
# hit is exact and the download only happens when the version changes.
|
||||||
|
- name: Cache the dora CLI
|
||||||
|
id: dora-cache
|
||||||
|
uses: actions/cache@v4
|
||||||
|
with:
|
||||||
|
path: ~/.local/bin/dora
|
||||||
|
key: dora-cli-${{ env.DORA_VERSION }}-${{ env.DORA_TARGET }}
|
||||||
|
|
||||||
- name: Install the dora CLI
|
- name: Install the dora CLI
|
||||||
|
if: steps.dora-cache.outputs.cache-hit != 'true'
|
||||||
run: |
|
run: |
|
||||||
curl -fsSL -o dora-cli.tar.gz \
|
curl -fsSL -o dora-cli.tar.gz \
|
||||||
"https://github.com/dora-rs/dora/releases/download/${DORA_VERSION}/dora-cli-${DORA_TARGET}.tar.gz"
|
"https://github.com/dora-rs/dora/releases/download/${DORA_VERSION}/dora-cli-${DORA_TARGET}.tar.gz"
|
||||||
tar xzf dora-cli.tar.gz
|
tar xzf dora-cli.tar.gz
|
||||||
mkdir -p "$HOME/.local/bin"
|
mkdir -p "$HOME/.local/bin"
|
||||||
install -m 755 "dora-cli-${DORA_TARGET}/dora" "$HOME/.local/bin/dora"
|
install -m 755 "dora-cli-${DORA_TARGET}/dora" "$HOME/.local/bin/dora"
|
||||||
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
|
||||||
rm -rf dora-cli.tar.gz "dora-cli-${DORA_TARGET}"
|
rm -rf dora-cli.tar.gz "dora-cli-${DORA_TARGET}"
|
||||||
|
|
||||||
|
- name: Put the dora CLI on PATH
|
||||||
|
run: echo "$HOME/.local/bin" >> "$GITHUB_PATH"
|
||||||
|
|
||||||
- run: dora --version
|
- run: dora --version
|
||||||
|
|
||||||
|
# --locked fails if uv.lock is out of date with pyproject.toml, so a
|
||||||
|
# dependency change that was never locked cannot reach main.
|
||||||
|
- name: Install
|
||||||
|
run: uv sync --locked --group dev
|
||||||
|
|
||||||
|
- run: uv run ruff check .
|
||||||
|
|
||||||
|
- run: uv run pytest -q
|
||||||
|
|
||||||
# strict_types: true in the dataflow turns a type warning into a non-zero
|
# strict_types: true in the dataflow turns a type warning into a non-zero
|
||||||
# exit, so a mismatched annotation fails here rather than at runtime.
|
# exit, so a mismatched annotation fails here rather than at runtime.
|
||||||
|
# Both nodes are path nodes with a manifest, so both contracts are
|
||||||
|
# injected and compared against each other.
|
||||||
- name: Validate the dataflow
|
- name: Validate the dataflow
|
||||||
run: dora validate dataflow.yml
|
run: dora validate dataflow.yml
|
||||||
|
|
||||||
@@ -55,26 +85,3 @@ jobs:
|
|||||||
env:
|
env:
|
||||||
SOURCE_FRAMES: 30
|
SOURCE_FRAMES: 30
|
||||||
run: dora run dataflow.yml --uv
|
run: dora run dataflow.yml --uv
|
||||||
|
|
||||||
test:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Install the OpenCV shared libraries
|
|
||||||
run: |
|
|
||||||
sudo apt-get update
|
|
||||||
sudo apt-get install -y --no-install-recommends libgl1 libglib2.0-0
|
|
||||||
|
|
||||||
- uses: astral-sh/setup-uv@v5
|
|
||||||
with:
|
|
||||||
enable-cache: true
|
|
||||||
|
|
||||||
# --locked fails if uv.lock is out of date with pyproject.toml, so a
|
|
||||||
# dependency change that was never locked cannot reach main.
|
|
||||||
- name: Install
|
|
||||||
run: uv sync --locked --group dev
|
|
||||||
|
|
||||||
- run: uv run ruff check .
|
|
||||||
|
|
||||||
- run: uv run pytest -q
|
|
||||||
|
|||||||
Reference in New Issue
Block a user