Installation pipeline

osdk separates version resolution from artifact installation. The CLI owns request orchestration, concurrency, and shims; each backend builds an installation plan; the shared pipeline downloads, verifies, extracts, ingests into CAS, and commits the installation.

CLI orchestration

install_requests merges -o key=value options and installs different tools with bounded concurrency from [settings].jobs. Each request enters install_one_without_shims: If the request set contains npm, pnpm, Yarn, or npm:<package> without Node, the CLI first injects Node. Scheduling completes every Node request serially before admitting the remaining requests to bounded concurrency, preventing a dynamic npm tool from racing its runtime. The same orchestration requires exactly one exact managed Rust request for any cargo: tool. Rust resolves and installs before Cargo tools enter the concurrent remainder, and its exact result is bound into each Cargo install identity. Go command packages use the same dependency barrier: one explicit or configured managed go request resolves and installs before every go: request, and its exact result is bound into each Go-tool identity.

1. optionally refresh source probes for SDK-installing calls; the lock, outdated, and list-remote resolution-only paths do not execute this flag;
2. find the backend, expand the version alias, and resolve an exact version;
3. if the complete marker exists, run only idempotent ensure_post_install;
4. otherwise call the backend's install;
5. after all installs finish, generate shims and sort results by backend name.

A bare project install that finds a current-platform lock only replays it. If that section is absent and the command actually resolves project configuration, it rebuilds the platform tool table from the exact results after the whole batch and shim generation succeed. Explicit install TOOL... remains a one-shot shared install and writes no project lock. Project use publishes [tools] and the corresponding lock upsert as one project-metadata transaction: lock failure restores configuration, while the already completed immutable shared install is left available for a retry.

Except for that Node dependency barrier, different tools may run concurrently. Pipeline or backend locks serialize fixed-backend writes to one tool@version and dynamic writes to one complete install identity. If any member of a batch fails, try_collect returns the error and the final shim-generation phase is not entered. The compatibility isolated npm path used by explicit install/exec bypasses the archive CAS pipeline below and uses a managed npm subprocess with an isolated install root and the shared osdk-owned npm cache/store. Project-aware and global use can instead select npm or pnpm during planning; see npm developer tool implementation. Cargo developer tools also bypass the archive CAS pipeline. Their native lifecycle holds an identity lock across a sibling stage, prefers a controlled eligible cargo-binstall, falls back to cargo install only for exit 94, and atomically publishes validated binaries; see Cargo developer tool implementation. Go developer tools reuse that native transaction and invoke one exact managed go install with a staged GOBIN; see Go developer tool implementation.

Backend plans

The common contract is Backend. An archive backend normally calls ranked_source_list, then constructs an InstallPlan containing the tool, exact version, candidate URLs, filename, archive kind, optional checksum, strip-root flag, and optional safe subdirectory. node.rs is a representative implementation.

A request restored from the lockfile first uses locked_install_plan, carrying the locked URL, filename, checksum, and subdirectory without querying a release registry again. Built-in archive backends and data-only declarative backends share this path. pnpm's complete JavaScript distribution, Yarn, Deno, Bun, and standalone npm are driven by npm registry packages and SRI. The generic GitHub backend may additionally require Sigstore/Rekor attestation. Rust is the major exception: it delegates toolchain installation to isolated rustup/Cargo homes and lets osdk maintain the completion marker and shims.

Shared pipeline transaction

pipeline::run_with_attestation performs these steps:

1. Acquire <tool>/<version>.lock for a fixed backend. A dynamic backend instead holds an identity-qualified lock through post-processing, .osdk-install.json publication, and the completion marker.
2. If the pipeline is invoked directly and .osdk-complete exists, it returns early; when that invocation carries attestation, it reverifies and merges evidence. Normal CLI install usually short-circuits earlier in install_one_without_shims and runs only ensure_post_install.
3. Remove a stale install directory that has no completion marker.
4. Use a stable artifact-cache path; fail immediately on an offline cache miss.
5. Download URLs in plan order. A single URL gets up to three transient-error attempts before the pipeline advances to the next source.
6. Verify the supplied or previously persisted checksum, and verify attestation plus its authenticated digest when present. Fail when require_checksums=true and neither is available.
7. Extract into a scratch directory under the cache. An optional subdirectory must be relative, contain no .. or platform prefix, and remain beneath scratch.
8. Ingest files into the BLAKE3 CAS, then materialize the install tree using reflink, hardlink, or copy according to configuration.
9. Write .osdk-artifact.json, then write .osdk-complete last. A version is listed as installed only after that final commit marker exists.

Resume and caching

pipeline/download.rs writes to a sibling .partial file and atomically renames it on success. It sends Range plus If-Range only when partial metadata has an ETag or Last-Modified validator for the same URL. A server that ignores ranges, changes the object, or returns a mismatched Content-Range causes a safe restart or failure rather than blind concatenation. Sensitive headers apply only to the initial request and are not retained on cross-host redirects. This paragraph concerns download headers carried by a generic artifact download plan. Source.headers separately applies to osdk metadata/source probes under an origin boundary. The managed npm/pnpm delegates currently do not forward arbitrary Source.headers. Project operations may use native trusted configuration; global npm-tool installs reject authenticated/private native pass-through while running in their isolated prefix.

An artifact-cache hit during an actual pipeline run or reinstall still runs the applicable checksum or attestation verification; mere cache-file existence is not trusted. Offline reinstall can reuse a persisted checksum, but it never falls back to the network when the artifact cache is absent. Ordinary osdk install reuses an already complete installation before entering the pipeline and does not revalidate its receipt, checksum, or installed bytes.

Failure semantics and caveats

• Each source candidate must complete download, checksum/attestation, extraction, and required-directory validation before it is accepted. Failure in those stages cleans up that attempt and advances to the next source. CAS ingestion, final materialization, and publication happen after a candidate has passed content validation; failures in those source-independent stages do not retry another source.
• The pipeline removes stale install trees and scratch at the start of the next attempt and removes scratch after successful materialization; failed extraction or materialization may leave scratch temporarily. It keeps .partial downloads for validated resume.
• Checksums are policy-dependent unless the backend supplies one, attestation supplies an authenticated digest, or require_checksums is enabled. Guarantees therefore differ by backend.
• ensure_post_install may have additional side effects. On a fresh Node install, Corepack failure removes the installation tree; on the already-installed fast path, ensure_post_install may fail while the existing completion marker remains.
• Delegate backends such as Rust do not traverse the complete archive pipeline; Cargo and Go developer tools also use their own native stage/receipt/seal transaction. Inspect those backends for their exact idempotency and verification boundaries.
• For backends with a generic artifact receipt, a locked artifact URL fixes artifact identity and enables metadata-free reinstall. Reinstallation applies any available or policy-required checksum/attestation checks; with no digest/evidence and require_checksums=false, it may proceed without cryptographic integrity verification. npm tools do not use generic artifact receipts. Current schema 4 retains scope, installer, and optional native-lock identity while leaving the dependency-graph payload under installer ownership. Schema 2 graph sidecars remain a compatibility-read path only; see npm developer tool implementation. Cargo and Go native metadata likewise describe runtime/source/replay identity rather than a complete graph, so cold offline installation is unsupported; see Cargo developer tool implementation and Go developer tool implementation.

Core coverage is in pipeline/mod.rs, pipeline/download.rs, backend/contract.rs, and end-to-end isolated_cli.rs.