Compare commits

..

8 Commits

Author SHA1 Message Date
Tõnis Tiigi
257815a6fb Merge pull request #2698 from tonistiigi/v0.17-cli-vendor
[v0.17] update docker/cli to 48a2cdff97
2024-09-13 08:03:56 -07:00
Tõnis Tiigi
dbccfa60a7 Merge pull request #2690 from crazy-max/v0.17.1_cherry-picks
[v0.17] cherry-picks for v0.17.1
2024-09-13 08:03:24 -07:00
Tonis Tiigi
59cb959195 [v0.17] update docker/cli to 48a2cdff97
Brings in fix for telemetry under WSL2

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-09-13 07:42:49 -07:00
CrazyMax
dd0d53efd5 ci: fix golvulncheck job permissions
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 120578091f)
2024-09-12 15:25:05 +02:00
CrazyMax
e8ceaad0a8 builder: do not set network.host entitlement flag if already set in buildkitd conf
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 617d59d70b)
2024-09-11 22:51:10 +02:00
CrazyMax
e5a6b8e140 bake: fix missing omitempty and optional tags for network field
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 9fb8b04b64)
2024-09-11 14:51:26 +02:00
CrazyMax
78c8c28cf5 Merge pull request #2681 from crazy-max/v0.17.0_update-buildkit
[v0.17 backport] vendor: update buildkit to v0.16.0
2024-09-10 18:40:29 +02:00
CrazyMax
4173281da3 vendor: update buildkit to v0.16.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 8201d301d5)
2024-09-10 18:19:31 +02:00
3396 changed files with 165516 additions and 291925 deletions

View File

@@ -188,89 +188,6 @@ To generate new vendored files with go modules run:
$ make vendor $ make vendor
``` ```
### Generate profiling data
You can configure Buildx to generate [`pprof`](https://github.com/google/pprof)
memory and CPU profiles to analyze and optimize your builds. These profiles are
useful for identifying performance bottlenecks, detecting memory
inefficiencies, and ensuring the program (Buildx) runs efficiently.
The following environment variables control whether Buildx generates profiling
data for builds:
```console
$ export BUILDX_CPU_PROFILE=buildx_cpu.prof
$ export BUILDX_MEM_PROFILE=buildx_mem.prof
```
When set, Buildx emits profiling samples for the builds to the location
specified by the environment variable.
To analyze and visualize profiling samples, you need `pprof` from the Go
toolchain, and (optionally) GraphViz for visualization in a graphical format.
To inspect profiling data with `pprof`:
1. Build a local binary of Buildx from source.
```console
$ docker buildx bake
```
The binary gets exported to `./bin/build/buildx`.
2. Run a build and with the environment variables set to generate profiling data.
```console
$ export BUILDX_CPU_PROFILE=buildx_cpu.prof
$ export BUILDX_MEM_PROFILE=buildx_mem.prof
$ ./bin/build/buildx bake
```
This creates `buildx_cpu.prof` and `buildx_mem.prof` for the build.
3. Start `pprof` and specify the filename of the profile that you want to
analyze.
```console
$ go tool pprof buildx_cpu.prof
```
This opens the `pprof` interactive console. From here, you can inspect the
profiling sample using various commands. For example, use `top 10` command
to view the top 10 most time-consuming entries.
```plaintext
(pprof) top 10
Showing nodes accounting for 3.04s, 91.02% of 3.34s total
Dropped 123 nodes (cum <= 0.02s)
Showing top 10 nodes out of 159
flat flat% sum% cum cum%
1.14s 34.13% 34.13% 1.14s 34.13% syscall.syscall
0.91s 27.25% 61.38% 0.91s 27.25% runtime.kevent
0.35s 10.48% 71.86% 0.35s 10.48% runtime.pthread_cond_wait
0.22s 6.59% 78.44% 0.22s 6.59% runtime.pthread_cond_signal
0.15s 4.49% 82.93% 0.15s 4.49% runtime.usleep
0.10s 2.99% 85.93% 0.10s 2.99% runtime.memclrNoHeapPointers
0.10s 2.99% 88.92% 0.10s 2.99% runtime.memmove
0.03s 0.9% 89.82% 0.03s 0.9% runtime.madvise
0.02s 0.6% 90.42% 0.02s 0.6% runtime.(*mspan).typePointersOfUnchecked
0.02s 0.6% 91.02% 0.02s 0.6% runtime.pcvalue
```
To view the call graph in a GUI, run `go tool pprof -http=:8081 <sample>`.
> [!NOTE]
> Requires [GraphViz](https://www.graphviz.org/) to be installed.
```console
$ go tool pprof -http=:8081 buildx_cpu.prof
Serving web UI on http://127.0.0.1:8081
http://127.0.0.1:8081
```
For more information about using `pprof` and how to interpret the call graph,
refer to the [`pprof` README](https://github.com/google/pprof/blob/main/doc/README.md).
### Conventions ### Conventions
@@ -426,4 +343,4 @@ The rules:
If you are having trouble getting into the mood of idiomatic Go, we recommend If you are having trouble getting into the mood of idiomatic Go, we recommend
reading through [Effective Go](https://golang.org/doc/effective_go.html). The reading through [Effective Go](https://golang.org/doc/effective_go.html). The
[Go Blog](https://blog.golang.org) is also a great resource. [Go Blog](https://blog.golang.org) is also a great resource.

50
.github/SECURITY.md vendored
View File

@@ -1,44 +1,12 @@
# Security Policy # Reporting security issues
The maintainers of Docker Buildx take security seriously. If you discover The project maintainers take security seriously. If you discover a security
a security issue, please bring it to their attention right away! issue, please bring it to their attention right away!
## Reporting a Vulnerability **Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Please **DO NOT** file a public issue, instead send your report privately Security reports are greatly appreciated, and we will publicly thank you for it.
to [security@docker.com](mailto:security@docker.com). We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
Reporter(s) can expect a response within 72 hours, acknowledging the issue was ruling it out in the future.
received.
## Review Process
After receiving the report, an initial triage and technical analysis is
performed to confirm the report and determine its scope. We may request
additional information in this stage of the process.
Once a reviewer has confirmed the relevance of the report, a draft security
advisory will be created on GitHub. The draft advisory will be used to discuss
the issue with maintainers, the reporter(s), and where applicable, other
affected parties under embargo.
If the vulnerability is accepted, a timeline for developing a patch, public
disclosure, and patch release will be determined. If there is an embargo period
on public disclosure before the patch release, the reporter(s) are expected to
participate in the discussion of the timeline and abide by agreed upon dates
for public disclosure.
## Accreditation
Security reports are greatly appreciated and we will publicly thank you,
although we will keep your name confidential if you request it. We also like to
send gifts - if you're into swag, make sure to let us know. We do not currently
offer a paid security bounty program at this time.
## Supported Versions
Once a new feature release is cut, support for the previous feature release is
discontinued. An exception may be made for urgent security releases that occur
shortly after a new feature release. Buildx does not offer LTS (Long-Term Support)
releases. Refer to the [Support Policy](https://github.com/docker/buildx/blob/master/PROJECT.md#support-policy)
for further details.

5
.github/labeler.yml vendored
View File

@@ -96,11 +96,6 @@ area/hack:
- changed-files: - changed-files:
- any-glob-to-any-file: 'hack/**' - any-glob-to-any-file: 'hack/**'
# Add 'area/history' label to changes in history command
area/history:
- changed-files:
- any-glob-to-any-file: 'commands/history/**'
# Add 'area/tests' label to changes in test files # Add 'area/tests' label to changes in test files
area/tests: area/tests:
- changed-files: - changed-files:

View File

@@ -1,14 +1,5 @@
name: build name: build
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.ref }} group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
@@ -28,15 +19,15 @@ on:
- 'docs/**' - 'docs/**'
env: env:
SETUP_BUILDX_VERSION: "edge" BUILDX_VERSION: "latest"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest" BUILDKIT_IMAGE: "moby/buildkit:latest"
SCOUT_VERSION: "1.11.0" SCOUT_VERSION: "1.11.0"
REPO_SLUG: "docker/buildx-bin" REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin" DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test" TEST_CACHE_SCOPE: "test"
TESTFLAGS: "-v --parallel=6 --timeout=30m" TESTFLAGS: "-v --parallel=6 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose" GOTESTSUM_FORMAT: "standard-verbose"
GO_VERSION: "1.23" GO_VERSION: "1.22"
GOTESTSUM_VERSION: "v1.9.0" # same as one in Dockerfile GOTESTSUM_VERSION: "v1.9.0" # same as one in Dockerfile
jobs: jobs:
@@ -54,9 +45,9 @@ jobs:
- master - master
- latest - latest
- buildx-stable-1 - buildx-stable-1
- v0.20.1 - v0.14.1
- v0.19.0 - v0.13.2
- v0.18.2 - v0.12.5
worker: worker:
- docker-container - docker-container
- remote - remote
@@ -76,26 +67,6 @@ jobs:
- worker: docker+containerd # same as docker, but with containerd snapshotter - worker: docker+containerd # same as docker, but with containerd snapshotter
pkg: ./tests pkg: ./tests
mode: experimental mode: experimental
- worker: "docker@27.5"
pkg: ./tests
- worker: "docker+containerd@27.5" # same as docker, but with containerd snapshotter
pkg: ./tests
- worker: "docker@27.5"
pkg: ./tests
mode: experimental
- worker: "docker+containerd@27.5" # same as docker, but with containerd snapshotter
pkg: ./tests
mode: experimental
- worker: "docker@26.1"
pkg: ./tests
- worker: "docker+containerd@26.1" # same as docker, but with containerd snapshotter
pkg: ./tests
- worker: "docker@26.1"
pkg: ./tests
mode: experimental
- worker: "docker+containerd@26.1" # same as docker, but with containerd snapshotter
pkg: ./tests
mode: experimental
steps: steps:
- -
name: Prepare name: Prepare
@@ -106,7 +77,7 @@ jobs:
fi fi
testFlags="--run=//worker=$(echo "${{ matrix.worker }}" | sed 's/\+/\\+/g')$" testFlags="--run=//worker=$(echo "${{ matrix.worker }}" | sed 's/\+/\\+/g')$"
case "${{ matrix.worker }}" in case "${{ matrix.worker }}" in
docker | docker+containerd | docker@* | docker+containerd@*) docker | docker+containerd)
echo "TESTFLAGS=${{ env.TESTFLAGS_DOCKER }} $testFlags" >> $GITHUB_ENV echo "TESTFLAGS=${{ env.TESTFLAGS_DOCKER }} $testFlags" >> $GITHUB_ENV
;; ;;
*) *)
@@ -131,14 +102,13 @@ jobs:
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }} driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug buildkitd-flags: --debug
- -
name: Build test image name: Build test image
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
source: .
targets: integration-test targets: integration-test
set: | set: |
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }} *.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
@@ -152,7 +122,7 @@ jobs:
- -
name: Send to Codecov name: Send to Codecov
if: always() if: always()
uses: codecov/codecov-action@v5 uses: codecov/codecov-action@v4
with: with:
directory: ./bin/testreports directory: ./bin/testreports
flags: integration flags: integration
@@ -179,16 +149,11 @@ jobs:
matrix: matrix:
os: os:
- ubuntu-24.04 - ubuntu-24.04
- macos-14 - macos-12
- windows-2022 - windows-2022
env: env:
SKIP_INTEGRATION_TESTS: 1 SKIP_INTEGRATION_TESTS: 1
steps: steps:
-
name: Setup Git config
run: |
git config --global core.autocrlf false
git config --global core.eol lf
- -
name: Checkout name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
@@ -229,7 +194,7 @@ jobs:
- -
name: Send to Codecov name: Send to Codecov
if: always() if: always()
uses: codecov/codecov-action@v5 uses: codecov/codecov-action@v4
with: with:
directory: ${{ env.TESTREPORTS_DIR }} directory: ${{ env.TESTREPORTS_DIR }}
env_vars: RUNNER_OS env_vars: RUNNER_OS
@@ -250,88 +215,27 @@ jobs:
name: test-reports-${{ env.TESTREPORTS_NAME }} name: test-reports-${{ env.TESTREPORTS_NAME }}
path: ${{ env.TESTREPORTS_BASEDIR }} path: ${{ env.TESTREPORTS_BASEDIR }}
test-bsd-unit: govulncheck:
runs-on: ubuntu-22.04 runs-on: ubuntu-24.04
continue-on-error: true permissions:
strategy: # required to write sarif report
fail-fast: false security-events: write
matrix: # required to check out the repository
os: contents: read
- freebsd
- netbsd
- openbsd
steps: steps:
-
name: Prepare
run: |
echo "VAGRANT_FILE=hack/Vagrantfile.${{ matrix.os }}" >> $GITHUB_ENV
# Sets semver Go version to be able to download tarball during vagrant setup
goVersion=$(curl --silent "https://go.dev/dl/?mode=json&include=all" | jq -r '.[].files[].version' | uniq | sed -e 's/go//' | sort -V | grep $GO_VERSION | tail -1)
echo "GO_VERSION=$goVersion" >> $GITHUB_ENV
- -
name: Checkout name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4
-
name: Cache Vagrant boxes
uses: actions/cache@v4
with:
path: ~/.vagrant.d/boxes
key: ${{ runner.os }}-vagrant-${{ matrix.os }}-${{ hashFiles(env.VAGRANT_FILE) }}
restore-keys: |
${{ runner.os }}-vagrant-${{ matrix.os }}-
-
name: Install vagrant
run: |
set -x
wget -O - https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update
sudo apt-get install -y libvirt-dev libvirt-daemon libvirt-daemon-system vagrant vagrant-libvirt ruby-libvirt
sudo systemctl enable --now libvirtd
sudo chmod a+rw /var/run/libvirt/libvirt-sock
vagrant plugin install vagrant-libvirt
vagrant --version
-
name: Set up vagrant
run: |
ln -sf ${{ env.VAGRANT_FILE }} Vagrantfile
vagrant up --no-tty
-
name: Test
run: |
vagrant ssh -- "cd /vagrant; SKIP_INTEGRATION_TESTS=1 go test -mod=vendor -coverprofile=coverage.txt -covermode=atomic ${{ env.TESTFLAGS }} ./..."
vagrant ssh -c "sudo cat /vagrant/coverage.txt" > coverage.txt
-
name: Upload coverage
if: always()
uses: codecov/codecov-action@v5
with:
files: ./coverage.txt
env_vars: RUNNER_OS
flags: unit,${{ matrix.os }}
token: ${{ secrets.CODECOV_TOKEN }}
env:
RUNNER_OS: ${{ matrix.os }}
govulncheck:
runs-on: ubuntu-24.04
permissions:
# same as global permission
contents: read
# required to write sarif report
security-events: write
steps:
- -
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }} driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug buildkitd-flags: --debug
- -
name: Run name: Run
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
targets: govulncheck targets: govulncheck
env: env:
@@ -385,8 +289,8 @@ jobs:
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }} driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug buildkitd-flags: --debug
- -
name: Build name: Build
@@ -412,14 +316,8 @@ jobs:
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }} if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps: steps:
- -
name: Free disk space name: Checkout
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1 uses: actions/checkout@v4
with:
android: true
dotnet: true
haskell: true
large-packages: true
swap-storage: true
- -
name: Set up QEMU name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v3
@@ -427,8 +325,8 @@ jobs:
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }} driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug buildkitd-flags: --debug
- -
name: Docker meta name: Docker meta
@@ -451,11 +349,11 @@ jobs:
password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }} password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }}
- -
name: Build and push image name: Build and push image
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
files: | files: |
./docker-bake.hcl ./docker-bake.hcl
cwd://${{ steps.meta.outputs.bake-file }} ${{ steps.meta.outputs.bake-file }}
targets: image-cross targets: image-cross
push: ${{ github.event_name != 'pull_request' }} push: ${{ github.event_name != 'pull_request' }}
sbom: true sbom: true
@@ -467,13 +365,14 @@ jobs:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
if: ${{ github.ref == 'refs/heads/master' && github.repository == 'docker/buildx' }} if: ${{ github.ref == 'refs/heads/master' && github.repository == 'docker/buildx' }}
permissions: permissions:
# same as global permission
contents: read
# required to write sarif report # required to write sarif report
security-events: write security-events: write
needs: needs:
- bin-image - bin-image
steps: steps:
-
name: Checkout
uses: actions/checkout@v4
- -
name: Login to DockerHub name: Login to DockerHub
uses: docker/login-action@v3 uses: docker/login-action@v3
@@ -496,9 +395,6 @@ jobs:
release: release:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
permissions:
# required to create GitHub release
contents: write
needs: needs:
- test-integration - test-integration
- test-unit - test-unit
@@ -528,7 +424,7 @@ jobs:
- -
name: GitHub Release name: GitHub Release
if: startsWith(github.ref, 'refs/tags/v') if: startsWith(github.ref, 'refs/tags/v')
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2.2.1 uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191 # v2.0.8
env: env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with: with:

View File

@@ -1,14 +1,5 @@
name: codeql name: codeql
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on: on:
push: push:
branches: branches:
@@ -16,16 +7,17 @@ on:
- 'v[0-9]*' - 'v[0-9]*'
pull_request: pull_request:
permissions:
actions: read
contents: read
security-events: write
env: env:
GO_VERSION: "1.23" GO_VERSION: "1.22"
jobs: jobs:
codeql: codeql:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
security-events: write
steps: steps:
- -
name: Checkout name: Checkout

View File

@@ -1,14 +1,5 @@
name: docs-release name: docs-release
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on: on:
workflow_dispatch: workflow_dispatch:
inputs: inputs:
@@ -19,17 +10,10 @@ on:
types: types:
- released - released
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs: jobs:
open-pr: open-pr:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
if: ${{ (github.event.release.prerelease != true || github.event.inputs.tag != '') && github.repository == 'docker/buildx' }} if: ${{ (github.event.release.prerelease != true || github.event.inputs.tag != '') && github.repository == 'docker/buildx' }}
permissions:
contents: write
pull-requests: write
steps: steps:
- -
name: Checkout docs repo name: Checkout docs repo
@@ -50,13 +34,9 @@ jobs:
- -
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
- -
name: Generate yaml name: Generate yaml
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ env.RELEASE_NAME }} source: ${{ github.server_url }}/${{ github.repository }}.git#${{ env.RELEASE_NAME }}
targets: update-docs targets: update-docs
@@ -77,7 +57,7 @@ jobs:
VENDOR_MODULE: github.com/docker/buildx@${{ env.RELEASE_NAME }} VENDOR_MODULE: github.com/docker/buildx@${{ env.RELEASE_NAME }}
- -
name: Create PR on docs repo name: Create PR on docs repo
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 uses: peter-evans/create-pull-request@8867c4aba1b742c39f8d0ba35429c2dfa4b6cb20 # v7.0.1
with: with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }} token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io push-to-fork: docker-tools-robot/docker.github.io

View File

@@ -3,15 +3,6 @@
# https://github.com/docker/docker.github.io/blob/98c7c9535063ae4cd2cd0a31478a21d16d2f07a3/docker-bake.hcl#L34-L36 # https://github.com/docker/docker.github.io/blob/98c7c9535063ae4cd2cd0a31478a21d16d2f07a3/docker-bake.hcl#L34-L36
name: docs-upstream name: docs-upstream
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.ref }} group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
@@ -29,24 +20,21 @@ on:
- '.github/workflows/docs-upstream.yml' - '.github/workflows/docs-upstream.yml'
- 'docs/**' - 'docs/**'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs: jobs:
docs-yaml: docs-yaml:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
steps: steps:
-
name: Checkout
uses: actions/checkout@v4
- -
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: latest
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
- -
name: Build reference YAML docs name: Build reference YAML docs
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
targets: update-docs targets: update-docs
provenance: false provenance: false
@@ -65,7 +53,7 @@ jobs:
retention-days: 1 retention-days: 1
validate: validate:
uses: docker/docs/.github/workflows/validate-upstream.yml@main uses: docker/docs/.github/workflows/validate-upstream.yml@6b73b05acb21edf7995cc5b3c6672d8e314cee7a # pin for artifact v4 support: https://github.com/docker/docs/pull/19220
needs: needs:
- docs-yaml - docs-yaml
with: with:

View File

@@ -1,14 +1,5 @@
name: e2e name: e2e
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.ref }} group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
@@ -26,25 +17,23 @@ on:
- 'docs/**' - 'docs/**'
env: env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
DESTDIR: "./bin" DESTDIR: "./bin"
K3S_VERSION: "v1.32.2+k3s1" K3S_VERSION: "v1.21.2-k3s1"
jobs: jobs:
build: build:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
steps: steps:
- name: Checkout
uses: actions/checkout@v4
- -
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: latest
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
- -
name: Build name: Build
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
targets: binaries targets: binaries
set: | set: |
@@ -65,7 +54,7 @@ jobs:
retention-days: 7 retention-days: 7
driver: driver:
runs-on: ubuntu-24.04 runs-on: ubuntu-20.04
needs: needs:
- build - build
strategy: strategy:
@@ -153,7 +142,7 @@ jobs:
- -
name: Install k3s name: Install k3s
if: matrix.driver == 'kubernetes' if: matrix.driver == 'kubernetes'
uses: crazy-max/.github/.github/actions/install-k3s@7730d1434364d4b9aded32735b078a7ace5ea79a uses: crazy-max/.github/.github/actions/install-k3s@fa6141aedf23596fb8bdcceab9cce8dadaa31bd9
with: with:
version: ${{ env.K3S_VERSION }} version: ${{ env.K3S_VERSION }}
- -
@@ -177,78 +166,3 @@ jobs:
DRIVER_OPT: ${{ matrix.driver-opt }} DRIVER_OPT: ${{ matrix.driver-opt }}
ENDPOINT: ${{ matrix.endpoint }} ENDPOINT: ${{ matrix.endpoint }}
PLATFORMS: ${{ matrix.platforms }} PLATFORMS: ${{ matrix.platforms }}
bake:
runs-on: ubuntu-24.04
needs:
- build
env:
DOCKER_BUILD_CHECKS_ANNOTATIONS: false
DOCKER_BUILD_SUMMARY: false
strategy:
fail-fast: false
matrix:
include:
-
# https://github.com/docker/bake-action/blob/v5.11.0/.github/workflows/ci.yml#L227-L237
source: "https://github.com/docker/bake-action.git#v5.11.0:test/go"
overrides: |
*.output=/tmp/bake-build
-
# https://github.com/tonistiigi/xx/blob/2fc85604e7280bfb3f626569bd4c5413c43eb4af/.github/workflows/ld.yml#L90-L98
source: "https://github.com/tonistiigi/xx.git#2fc85604e7280bfb3f626569bd4c5413c43eb4af"
targets: |
ld64-static-tgz
overrides: |
ld64-static-tgz.output=type=local,dest=./dist
ld64-static-tgz.platform=linux/amd64
ld64-static-tgz.cache-from=type=gha,scope=xx-ld64-static-tgz
ld64-static-tgz.cache-to=type=gha,scope=xx-ld64-static-tgz
-
# https://github.com/moby/buildkit-bench/blob/54c194011c4fc99a94aa75d4b3d4f3ffd4c4ce27/docker-bake.hcl#L154-L160
source: "https://github.com/moby/buildkit-bench.git#54c194011c4fc99a94aa75d4b3d4f3ffd4c4ce27"
targets: |
tests-buildkit
envs: |
BUILDKIT_REFS=v0.18.2
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Expose GitHub Runtime
uses: crazy-max/ghaction-github-runtime@v3
-
name: Environment variables
if: matrix.envs != ''
run: |
for l in "${{ matrix.envs }}"; do
echo "${l?}" >> $GITHUB_ENV
done
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Install buildx
uses: actions/download-artifact@v4
with:
name: binary
path: /home/runner/.docker/cli-plugins
-
name: Fix perms and check
run: |
chmod +x /home/runner/.docker/cli-plugins/docker-buildx
docker buildx version
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v6
with:
source: ${{ matrix.source }}
targets: ${{ matrix.targets }}
set: ${{ matrix.overrides }}

View File

@@ -1,14 +1,5 @@
name: labeler name: labeler
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.ref }} group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
@@ -18,12 +9,10 @@ on:
jobs: jobs:
labeler: labeler:
runs-on: ubuntu-latest
permissions: permissions:
# same as global permission
contents: read contents: read
# required for writing labels
pull-requests: write pull-requests: write
runs-on: ubuntu-latest
steps: steps:
- -
name: Run name: Run

View File

@@ -1,14 +1,5 @@
name: validate name: validate
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency: concurrency:
group: ${{ github.workflow }}-${{ github.ref }} group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true cancel-in-progress: true
@@ -25,10 +16,6 @@ on:
paths-ignore: paths-ignore:
- '.github/releases.json' - '.github/releases.json'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs: jobs:
prepare: prepare:
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
@@ -94,16 +81,17 @@ jobs:
if [ "$GITHUB_REPOSITORY" = "docker/buildx" ]; then if [ "$GITHUB_REPOSITORY" = "docker/buildx" ]; then
echo "GOLANGCI_LINT_MULTIPLATFORM=1" >> $GITHUB_ENV echo "GOLANGCI_LINT_MULTIPLATFORM=1" >> $GITHUB_ENV
fi fi
-
name: Checkout
uses: actions/checkout@v4
- -
name: Set up Docker Buildx name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
with: with:
version: ${{ env.SETUP_BUILDX_VERSION }} version: latest
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
- -
name: Validate name: Validate
uses: docker/bake-action@v6 uses: docker/bake-action@v5
with: with:
targets: ${{ matrix.target }} targets: ${{ matrix.target }}
set: | set: |

View File

@@ -1,55 +1,26 @@
run: run:
timeout: 30m timeout: 30m
modules-download-mode: vendor modules-download-mode: vendor
# default uses Go version from the go.mod file, fallback on the env var
# `GOVERSION`, fallback on 1.17: https://golangci-lint.run/usage/configuration/#run-configuration
go: "1.23"
linters: linters:
enable: enable:
- bodyclose
- depguard
- forbidigo
- gocritic
- gofmt - gofmt
- goimports
- gosec
- gosimple
- govet - govet
- depguard
- goimports
- ineffassign - ineffassign
- makezero
- misspell - misspell
- noctx - unused
- nolintlint
- revive - revive
- staticcheck - staticcheck
- testifylint
- typecheck - typecheck
- unused - nolintlint
- whitespace - gosec
- forbidigo
disable-all: true disable-all: true
linters-settings: linters-settings:
gocritic:
disabled-checks:
- "ifElseChain"
- "assignOp"
- "appendAssign"
- "singleCaseSwitch"
- "exitAfterDefer" # FIXME
importas:
alias:
# Enforce alias to prevent it accidentally being used instead of
# buildkit errdefs package (or vice-versa).
- pkg: "github.com/containerd/errdefs"
alias: "cerrdefs"
# Use a consistent alias to prevent confusion with "github.com/moby/buildkit/client"
- pkg: "github.com/docker/docker/client"
alias: "dockerclient"
- pkg: "github.com/opencontainers/image-spec/specs-go/v1"
alias: "ocispecs"
- pkg: "github.com/opencontainers/go-digest"
alias: "digest"
govet: govet:
enable: enable:
- nilness - nilness
@@ -72,27 +43,14 @@ linters-settings:
desc: The io/ioutil package has been deprecated. desc: The io/ioutil package has been deprecated.
forbidigo: forbidigo:
forbid: forbid:
- '^context\.WithCancel(# use context\.WithCancelCause instead)?$'
- '^context\.WithDeadline(# use context\.WithDeadline instead)?$'
- '^context\.WithTimeout(# use context\.WithTimeoutCause instead)?$'
- '^ctx\.Err(# use context\.Cause instead)?$'
- '^fmt\.Errorf(# use errors\.Errorf instead)?$' - '^fmt\.Errorf(# use errors\.Errorf instead)?$'
- '^platforms\.DefaultString(# use platforms\.Format(platforms\.DefaultSpec()) instead\.)?$' - '^platforms\.DefaultString(# use platforms\.Format(platforms\.DefaultSpec()) instead\.)?$'
gosec: gosec:
excludes: excludes:
- G204 # Audit use of command execution - G204 # Audit use of command execution
- G402 # TLS MinVersion too low - G402 # TLS MinVersion too low
- G115 # integer overflow conversion (TODO: verify these)
config: config:
G306: "0644" G306: "0644"
testifylint:
disable:
# disable rules that reduce the test condition
- "empty"
- "bool-compare"
- "len"
- "negative-positive"
issues: issues:
exclude-files: exclude-files:

View File

@@ -1,27 +1,20 @@
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
ARG GO_VERSION=1.23 ARG GO_VERSION=1.22
ARG ALPINE_VERSION=3.21 ARG XX_VERSION=1.4.0
ARG XX_VERSION=1.6.1
# for testing # for testing
ARG DOCKER_VERSION=28.0.0 ARG DOCKER_VERSION=27.1.1
ARG DOCKER_VERSION_ALT_27=27.5.1
ARG DOCKER_VERSION_ALT_26=26.1.3
ARG DOCKER_CLI_VERSION=${DOCKER_VERSION} ARG DOCKER_CLI_VERSION=${DOCKER_VERSION}
ARG GOTESTSUM_VERSION=v1.12.0 ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.3 ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.20.1 ARG BUILDKIT_VERSION=v0.14.1
ARG UNDOCK_VERSION=0.9.0 ARG UNDOCK_VERSION=0.7.0
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS golatest FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS golatest
FROM moby/moby-bin:$DOCKER_VERSION AS docker-engine FROM moby/moby-bin:$DOCKER_VERSION AS docker-engine
FROM dockereng/cli-bin:$DOCKER_CLI_VERSION AS docker-cli FROM dockereng/cli-bin:$DOCKER_CLI_VERSION AS docker-cli
FROM moby/moby-bin:$DOCKER_VERSION_ALT_27 AS docker-engine-alt27
FROM moby/moby-bin:$DOCKER_VERSION_ALT_26 AS docker-engine-alt26
FROM dockereng/cli-bin:$DOCKER_VERSION_ALT_27 AS docker-cli-alt27
FROM dockereng/cli-bin:$DOCKER_VERSION_ALT_26 AS docker-cli-alt26
FROM registry:$REGISTRY_VERSION AS registry FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM crazymax/undock:$UNDOCK_VERSION AS undock FROM crazymax/undock:$UNDOCK_VERSION AS undock
@@ -84,7 +77,6 @@ RUN --mount=type=bind,target=. \
set -e set -e
xx-go --wrap xx-go --wrap
DESTDIR=/usr/bin VERSION=$(cat /buildx-version/version) REVISION=$(cat /buildx-version/revision) GO_EXTRA_LDFLAGS="-s -w" ./hack/build DESTDIR=/usr/bin VERSION=$(cat /buildx-version/version) REVISION=$(cat /buildx-version/revision) GO_EXTRA_LDFLAGS="-s -w" ./hack/build
file /usr/bin/docker-buildx
xx-verify --static /usr/bin/docker-buildx xx-verify --static /usr/bin/docker-buildx
EOT EOT
@@ -103,10 +95,7 @@ FROM scratch AS binaries-unix
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx
FROM binaries-unix AS binaries-darwin FROM binaries-unix AS binaries-darwin
FROM binaries-unix AS binaries-freebsd
FROM binaries-unix AS binaries-linux FROM binaries-unix AS binaries-linux
FROM binaries-unix AS binaries-netbsd
FROM binaries-unix AS binaries-openbsd
FROM scratch AS binaries-windows FROM scratch AS binaries-windows
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx.exe COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx.exe
@@ -131,21 +120,16 @@ COPY --link --from=gotestsum /out /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/ COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=docker-engine / /usr/bin/ COPY --link --from=docker-engine / /usr/bin/
COPY --link --from=docker-cli / /usr/bin/ COPY --link --from=docker-cli / /usr/bin/
COPY --link --from=docker-engine-alt27 / /opt/docker-alt-27/
COPY --link --from=docker-engine-alt26 / /opt/docker-alt-26/
COPY --link --from=docker-cli-alt27 / /opt/docker-alt-27/
COPY --link --from=docker-cli-alt26 / /opt/docker-alt-26/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/ COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/ COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=undock /usr/local/bin/undock /usr/bin/ COPY --link --from=undock /usr/local/bin/undock /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/ COPY --link --from=binaries /buildx /usr/bin/
ENV TEST_DOCKER_EXTRA="docker@27.5=/opt/docker-alt-27,docker@26.1=/opt/docker-alt-26"
FROM integration-test-base AS integration-test FROM integration-test-base AS integration-test
COPY . . COPY . .
# Release # Release
FROM --platform=$BUILDPLATFORM alpine:${ALPINE_VERSION} AS releaser FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work WORKDIR /work
ARG TARGETPLATFORM ARG TARGETPLATFORM
RUN --mount=from=binaries \ RUN --mount=from=binaries \
@@ -160,7 +144,7 @@ COPY --from=releaser /out/ /
# Shell # Shell
FROM docker:$DOCKER_VERSION AS dockerd-release FROM docker:$DOCKER_VERSION AS dockerd-release
FROM alpine:${ALPINE_VERSION} AS shell FROM alpine AS shell
RUN apk add --no-cache iptables tmux git vim less openssh RUN apk add --no-cache iptables tmux git vim less openssh
RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx
COPY ./hack/demo-env/entrypoint.sh /usr/local/bin COPY ./hack/demo-env/entrypoint.sh /usr/local/bin

View File

@@ -2,13 +2,11 @@ package bake
import ( import (
"context" "context"
"encoding"
"io" "io"
"os" "os"
"path" "path"
"path/filepath" "path/filepath"
"regexp" "regexp"
"slices"
"sort" "sort"
"strconv" "strconv"
"strings" "strings"
@@ -27,7 +25,9 @@ import (
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb" "github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/session/auth/authprovider" "github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/entitlements"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/tonistiigi/go-csvvalue"
"github.com/zclconf/go-cty/cty" "github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert" "github.com/zclconf/go-cty/cty/convert"
) )
@@ -45,7 +45,6 @@ type File struct {
type Override struct { type Override struct {
Value string Value string
ArrValue []string ArrValue []string
Append bool
} }
func defaultFilenames() []string { func defaultFilenames() []string {
@@ -53,8 +52,8 @@ func defaultFilenames() []string {
names = append(names, composecli.DefaultFileNames...) names = append(names, composecli.DefaultFileNames...)
names = append(names, []string{ names = append(names, []string{
"docker-bake.json", "docker-bake.json",
"docker-bake.hcl",
"docker-bake.override.json", "docker-bake.override.json",
"docker-bake.hcl",
"docker-bake.override.hcl", "docker-bake.override.hcl",
}...) }...)
return names return names
@@ -193,7 +192,7 @@ func ListTargets(files []File) ([]string, error) {
return dedupSlice(targets), nil return dedupSlice(targets), nil
} }
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string, ent *EntitlementConf) (map[string]*Target, map[string]*Group, error) { func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, map[string]*Group, error) {
c, _, err := ParseFiles(files, defaults) c, _, err := ParseFiles(files, defaults)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
@@ -207,24 +206,23 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
m := map[string]*Target{}
targetsMap := map[string]*Target{} n := map[string]*Group{}
groupsMap := map[string]*Group{}
for _, target := range targets { for _, target := range targets {
ts, gs := c.ResolveGroup(target) ts, gs := c.ResolveGroup(target)
for _, tname := range ts { for _, tname := range ts {
t, err := c.ResolveTarget(tname, o, ent) t, err := c.ResolveTarget(tname, o)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
if t != nil { if t != nil {
targetsMap[tname] = t m[tname] = t
} }
} }
for _, gname := range gs { for _, gname := range gs {
for _, group := range c.Groups { for _, group := range c.Groups {
if group.Name == gname { if group.Name == gname {
groupsMap[gname] = group n[gname] = group
break break
} }
} }
@@ -232,26 +230,25 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
} }
for _, target := range targets { for _, target := range targets {
if _, ok := groupsMap["default"]; ok && target == "default" { if target == "default" {
continue continue
} }
if _, ok := groupsMap["default"]; !ok { if _, ok := n["default"]; !ok {
groupsMap["default"] = &Group{Name: "default"} n["default"] = &Group{Name: "default"}
} }
groupsMap["default"].Targets = append(groupsMap["default"].Targets, target) n["default"].Targets = append(n["default"].Targets, target)
} }
if g, ok := groupsMap["default"]; ok { if g, ok := n["default"]; ok {
g.Targets = dedupSlice(g.Targets) g.Targets = dedupSlice(g.Targets)
sort.Strings(g.Targets)
} }
for name, t := range targetsMap { for name, t := range m {
if err := c.loadLinks(name, t, targetsMap, o, nil, ent); err != nil { if err := c.loadLinks(name, t, m, o, nil); err != nil {
return nil, nil, err return nil, nil, err
} }
} }
return targetsMap, groupsMap, nil return m, n, nil
} }
func dedupSlice(s []string) []string { func dedupSlice(s []string) []string {
@@ -478,44 +475,36 @@ func (c Config) expandTargets(pattern string) ([]string, error) {
return names, nil return names, nil
} }
func (c Config) loadLinks(name string, t *Target, m map[string]*Target, o map[string]map[string]Override, visited []string, ent *EntitlementConf) error { func (c Config) loadLinks(name string, t *Target, m map[string]*Target, o map[string]map[string]Override, visited []string) error {
visited = append(visited, name) visited = append(visited, name)
for _, v := range t.Contexts { for _, v := range t.Contexts {
if strings.HasPrefix(v, "target:") { if strings.HasPrefix(v, "target:") {
target := strings.TrimPrefix(v, "target:") target := strings.TrimPrefix(v, "target:")
if target == name { if target == t.Name {
return errors.Errorf("target %s cannot link to itself", target) return errors.Errorf("target %s cannot link to itself", target)
} }
if slices.Contains(visited, target) { for _, v := range visited {
return errors.Errorf("infinite loop from %s to %s", name, target) if v == target {
return errors.Errorf("infinite loop from %s to %s", name, target)
}
} }
t2, ok := m[target] t2, ok := m[target]
if !ok { if !ok {
var err error var err error
t2, err = c.ResolveTarget(target, o, ent) t2, err = c.ResolveTarget(target, o)
if err != nil { if err != nil {
return err return err
} }
t2.Outputs = []*buildflags.ExportEntry{ t2.Outputs = []string{"type=cacheonly"}
{Type: "cacheonly"},
}
t2.linked = true t2.linked = true
m[target] = t2 m[target] = t2
} }
if err := c.loadLinks(target, t2, m, o, visited, ent); err != nil { if err := c.loadLinks(target, t2, m, o, visited); err != nil {
return err return err
} }
// entitlements are inherited from linked targets
for _, ent := range t2.Entitlements {
if !slices.Contains(t.Entitlements, ent) {
t.Entitlements = append(t.Entitlements, ent)
}
}
if len(t.Platforms) > 1 && len(t2.Platforms) > 1 { if len(t.Platforms) > 1 && len(t2.Platforms) > 1 {
if !isSubset(t.Platforms, t2.Platforms) { if !sliceEqual(t.Platforms, t2.Platforms) {
return errors.Errorf("target %s can't be used by %s because its platforms %v are not a subset of %v", target, name, t.Platforms, t2.Platforms) return errors.Errorf("target %s can't be used by %s because it is defined for different platforms %v and %v", target, name, t2.Platforms, t.Platforms)
} }
} }
} }
@@ -527,12 +516,9 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
m := map[string]map[string]Override{} m := map[string]map[string]Override{}
for _, v := range v { for _, v := range v {
parts := strings.SplitN(v, "=", 2) parts := strings.SplitN(v, "=", 2)
keys := strings.SplitN(parts[0], ".", 3)
skey := strings.TrimSuffix(parts[0], "+")
appendTo := strings.HasSuffix(parts[0], "+")
keys := strings.SplitN(skey, ".", 3)
if len(keys) < 2 { if len(keys) < 2 {
return nil, errors.Errorf("invalid override key %s, expected target.name", skey) return nil, errors.Errorf("invalid override key %s, expected target.name", parts[0])
} }
pattern := keys[0] pattern := keys[0]
@@ -545,7 +531,8 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
return nil, err return nil, err
} }
okey := strings.Join(keys[1:], ".") kk := strings.SplitN(parts[0], ".", 2)
for _, name := range names { for _, name := range names {
t, ok := m[name] t, ok := m[name]
if !ok { if !ok {
@@ -553,15 +540,12 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
m[name] = t m[name] = t
} }
override := t[okey] o := t[kk[1]]
// IMPORTANT: if you add more fields here, do not forget to update
// docs/reference/buildx_bake.md (--set) and https://docs.docker.com/build/bake/overrides/
switch keys[1] { switch keys[1] {
case "output", "cache-to", "cache-from", "tags", "platform", "secrets", "ssh", "attest", "entitlements", "network", "annotations": case "output", "cache-to", "cache-from", "tags", "platform", "secrets", "ssh", "attest", "entitlements", "network":
if len(parts) == 2 { if len(parts) == 2 {
override.Append = appendTo o.ArrValue = append(o.ArrValue, parts[1])
override.ArrValue = append(override.ArrValue, parts[1])
} }
case "args": case "args":
if len(keys) != 3 { if len(keys) != 3 {
@@ -572,7 +556,7 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
if !ok { if !ok {
continue continue
} }
override.Value = v o.Value = v
} }
fallthrough fallthrough
case "contexts": case "contexts":
@@ -582,11 +566,11 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
fallthrough fallthrough
default: default:
if len(parts) == 2 { if len(parts) == 2 {
override.Value = parts[1] o.Value = parts[1]
} }
} }
t[okey] = override t[kk[1]] = o
} }
} }
return m, nil return m, nil
@@ -634,8 +618,8 @@ func (c Config) group(name string, visited map[string]visit) ([]string, []string
return targets, groups return targets, groups
} }
func (c Config) ResolveTarget(name string, overrides map[string]map[string]Override, ent *EntitlementConf) (*Target, error) { func (c Config) ResolveTarget(name string, overrides map[string]map[string]Override) (*Target, error) {
t, err := c.target(name, map[string]*Target{}, overrides, ent) t, err := c.target(name, map[string]*Target{}, overrides)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -651,7 +635,7 @@ func (c Config) ResolveTarget(name string, overrides map[string]map[string]Overr
return t, nil return t, nil
} }
func (c Config) target(name string, visited map[string]*Target, overrides map[string]map[string]Override, ent *EntitlementConf) (*Target, error) { func (c Config) target(name string, visited map[string]*Target, overrides map[string]map[string]Override) (*Target, error) {
if t, ok := visited[name]; ok { if t, ok := visited[name]; ok {
return t, nil return t, nil
} }
@@ -668,7 +652,7 @@ func (c Config) target(name string, visited map[string]*Target, overrides map[st
} }
tt := &Target{} tt := &Target{}
for _, name := range t.Inherits { for _, name := range t.Inherits {
t, err := c.target(name, visited, overrides, ent) t, err := c.target(name, visited, overrides)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -680,7 +664,7 @@ func (c Config) target(name string, visited map[string]*Target, overrides map[st
m.Merge(tt) m.Merge(tt)
m.Merge(t) m.Merge(t)
tt = m tt = m
if err := tt.AddOverrides(overrides[name], ent); err != nil { if err := tt.AddOverrides(overrides[name]); err != nil {
return nil, err return nil, err
} }
tt.normalize() tt.normalize()
@@ -702,61 +686,59 @@ type Target struct {
// Inherits is the only field that cannot be overridden with --set // Inherits is the only field that cannot be overridden with --set
Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional" cty:"inherits"` Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional" cty:"inherits"`
Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"` Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"`
Attest buildflags.Attests `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"` Attest []string `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"` Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"` Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"` Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"`
DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional" cty:"dockerfile-inline"` DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional" cty:"dockerfile-inline"`
Args map[string]*string `json:"args,omitempty" hcl:"args,optional" cty:"args"` Args map[string]*string `json:"args,omitempty" hcl:"args,optional" cty:"args"`
Labels map[string]*string `json:"labels,omitempty" hcl:"labels,optional" cty:"labels"` Labels map[string]*string `json:"labels,omitempty" hcl:"labels,optional" cty:"labels"`
Tags []string `json:"tags,omitempty" hcl:"tags,optional" cty:"tags"` Tags []string `json:"tags,omitempty" hcl:"tags,optional" cty:"tags"`
CacheFrom buildflags.CacheOptions `json:"cache-from,omitempty" hcl:"cache-from,optional" cty:"cache-from"` CacheFrom []string `json:"cache-from,omitempty" hcl:"cache-from,optional" cty:"cache-from"`
CacheTo buildflags.CacheOptions `json:"cache-to,omitempty" hcl:"cache-to,optional" cty:"cache-to"` CacheTo []string `json:"cache-to,omitempty" hcl:"cache-to,optional" cty:"cache-to"`
Target *string `json:"target,omitempty" hcl:"target,optional" cty:"target"` Target *string `json:"target,omitempty" hcl:"target,optional" cty:"target"`
Secrets buildflags.Secrets `json:"secret,omitempty" hcl:"secret,optional" cty:"secret"` Secrets []string `json:"secret,omitempty" hcl:"secret,optional" cty:"secret"`
SSH buildflags.SSHKeys `json:"ssh,omitempty" hcl:"ssh,optional" cty:"ssh"` SSH []string `json:"ssh,omitempty" hcl:"ssh,optional" cty:"ssh"`
Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional" cty:"platforms"` Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional" cty:"platforms"`
Outputs buildflags.Exports `json:"output,omitempty" hcl:"output,optional" cty:"output"` Outputs []string `json:"output,omitempty" hcl:"output,optional" cty:"output"`
Pull *bool `json:"pull,omitempty" hcl:"pull,optional" cty:"pull"` Pull *bool `json:"pull,omitempty" hcl:"pull,optional" cty:"pull"`
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"` NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"network,omitempty" hcl:"network,optional" cty:"network"` NetworkMode *string `json:"network,omitempty" hcl:"network,optional" cty:"network"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"` NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
ShmSize *string `json:"shm-size,omitempty" hcl:"shm-size,optional" cty:"shm-size"` ShmSize *string `json:"shm-size,omitempty" hcl:"shm-size,optional"`
Ulimits []string `json:"ulimits,omitempty" hcl:"ulimits,optional" cty:"ulimits"` Ulimits []string `json:"ulimits,omitempty" hcl:"ulimits,optional"`
Call *string `json:"call,omitempty" hcl:"call,optional" cty:"call"` Call *string `json:"call,omitempty" hcl:"call,optional" cty:"call"`
Entitlements []string `json:"entitlements,omitempty" hcl:"entitlements,optional" cty:"entitlements"` Entitlements []string `json:"entitlements,omitempty" hcl:"entitlements,optional" cty:"entitlements"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides/AddOverrides and docs/bake-reference.md. // IMPORTANT: if you add more fields here, do not forget to update newOverrides/AddOverrides and docs/bake-reference.md.
// linked is a private field to mark a target used as a linked one // linked is a private field to mark a target used as a linked one
linked bool linked bool
} }
var ( var _ hclparser.WithEvalContexts = &Target{}
_ hclparser.WithEvalContexts = &Target{} var _ hclparser.WithGetName = &Target{}
_ hclparser.WithGetName = &Target{} var _ hclparser.WithEvalContexts = &Group{}
_ hclparser.WithEvalContexts = &Group{} var _ hclparser.WithGetName = &Group{}
_ hclparser.WithGetName = &Group{}
)
func (t *Target) normalize() { func (t *Target) normalize() {
t.Annotations = removeDupesStr(t.Annotations) t.Annotations = removeDupes(t.Annotations)
t.Attest = t.Attest.Normalize() t.Attest = removeAttestDupes(t.Attest)
t.Tags = removeDupesStr(t.Tags) t.Tags = removeDupes(t.Tags)
t.Secrets = t.Secrets.Normalize() t.Secrets = removeDupes(t.Secrets)
t.SSH = t.SSH.Normalize() t.SSH = removeDupes(t.SSH)
t.Platforms = removeDupesStr(t.Platforms) t.Platforms = removeDupes(t.Platforms)
t.CacheFrom = t.CacheFrom.Normalize() t.CacheFrom = removeDupes(t.CacheFrom)
t.CacheTo = t.CacheTo.Normalize() t.CacheTo = removeDupes(t.CacheTo)
t.Outputs = t.Outputs.Normalize() t.Outputs = removeDupes(t.Outputs)
t.NoCacheFilter = removeDupesStr(t.NoCacheFilter) t.NoCacheFilter = removeDupes(t.NoCacheFilter)
t.Ulimits = removeDupesStr(t.Ulimits) t.Ulimits = removeDupes(t.Ulimits)
if t.NetworkMode != nil && *t.NetworkMode == "host" { if t.NetworkMode != nil && *t.NetworkMode == "host" {
t.Entitlements = append(t.Entitlements, "network.host") t.Entitlements = append(t.Entitlements, "network.host")
} }
t.Entitlements = removeDupesStr(t.Entitlements) t.Entitlements = removeDupes(t.Entitlements)
for k, v := range t.Contexts { for k, v := range t.Contexts {
if v == "" { if v == "" {
@@ -815,19 +797,20 @@ func (t *Target) Merge(t2 *Target) {
t.Annotations = append(t.Annotations, t2.Annotations...) t.Annotations = append(t.Annotations, t2.Annotations...)
} }
if t2.Attest != nil { // merge if t2.Attest != nil { // merge
t.Attest = t.Attest.Merge(t2.Attest) t.Attest = append(t.Attest, t2.Attest...)
t.Attest = removeAttestDupes(t.Attest)
} }
if t2.Secrets != nil { // merge if t2.Secrets != nil { // merge
t.Secrets = t.Secrets.Merge(t2.Secrets) t.Secrets = append(t.Secrets, t2.Secrets...)
} }
if t2.SSH != nil { // merge if t2.SSH != nil { // merge
t.SSH = t.SSH.Merge(t2.SSH) t.SSH = append(t.SSH, t2.SSH...)
} }
if t2.Platforms != nil { // no merge if t2.Platforms != nil { // no merge
t.Platforms = t2.Platforms t.Platforms = t2.Platforms
} }
if t2.CacheFrom != nil { // merge if t2.CacheFrom != nil { // merge
t.CacheFrom = t.CacheFrom.Merge(t2.CacheFrom) t.CacheFrom = append(t.CacheFrom, t2.CacheFrom...)
} }
if t2.CacheTo != nil { // no merge if t2.CacheTo != nil { // no merge
t.CacheTo = t2.CacheTo t.CacheTo = t2.CacheTo
@@ -862,9 +845,7 @@ func (t *Target) Merge(t2 *Target) {
t.Inherits = append(t.Inherits, t2.Inherits...) t.Inherits = append(t.Inherits, t2.Inherits...)
} }
func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementConf) error { func (t *Target) AddOverrides(overrides map[string]Override) error {
// IMPORTANT: if you add more fields here, do not forget to update
// docs/bake-reference.md and https://docs.docker.com/build/bake/overrides/
for key, o := range overrides { for key, o := range overrides {
value := o.Value value := o.Value
keys := strings.SplitN(key, ".", 2) keys := strings.SplitN(key, ".", 2)
@@ -875,7 +856,7 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Dockerfile = &value t.Dockerfile = &value
case "args": case "args":
if len(keys) != 2 { if len(keys) != 2 {
return errors.Errorf("invalid format for args, expecting args.<name>=<value>") return errors.Errorf("args require name")
} }
if t.Args == nil { if t.Args == nil {
t.Args = map[string]*string{} t.Args = map[string]*string{}
@@ -883,7 +864,7 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Args[keys[1]] = &value t.Args[keys[1]] = &value
case "contexts": case "contexts":
if len(keys) != 2 { if len(keys) != 2 {
return errors.Errorf("invalid format for contexts, expecting contexts.<name>=<value>") return errors.Errorf("contexts require name")
} }
if t.Contexts == nil { if t.Contexts == nil {
t.Contexts = map[string]string{} t.Contexts = map[string]string{}
@@ -891,122 +872,36 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Contexts[keys[1]] = value t.Contexts[keys[1]] = value
case "labels": case "labels":
if len(keys) != 2 { if len(keys) != 2 {
return errors.Errorf("invalid format for labels, expecting labels.<name>=<value>") return errors.Errorf("labels require name")
} }
if t.Labels == nil { if t.Labels == nil {
t.Labels = map[string]*string{} t.Labels = map[string]*string{}
} }
t.Labels[keys[1]] = &value t.Labels[keys[1]] = &value
case "tags": case "tags":
if o.Append { t.Tags = o.ArrValue
t.Tags = append(t.Tags, o.ArrValue...)
} else {
t.Tags = o.ArrValue
}
case "cache-from": case "cache-from":
cacheFrom, err := buildflags.ParseCacheEntry(o.ArrValue) t.CacheFrom = o.ArrValue
if err != nil {
return err
}
if o.Append {
t.CacheFrom = t.CacheFrom.Merge(cacheFrom)
} else {
t.CacheFrom = cacheFrom
}
for _, c := range t.CacheFrom {
if c.Type == "local" {
if v, ok := c.Attrs["src"]; ok {
ent.FSRead = append(ent.FSRead, v)
}
}
}
case "cache-to": case "cache-to":
cacheTo, err := buildflags.ParseCacheEntry(o.ArrValue) t.CacheTo = o.ArrValue
if err != nil {
return err
}
if o.Append {
t.CacheTo = t.CacheTo.Merge(cacheTo)
} else {
t.CacheTo = cacheTo
}
for _, c := range t.CacheTo {
if c.Type == "local" {
if v, ok := c.Attrs["dest"]; ok {
ent.FSWrite = append(ent.FSWrite, v)
}
}
}
case "target": case "target":
t.Target = &value t.Target = &value
case "call": case "call":
t.Call = &value t.Call = &value
case "secrets": case "secrets":
secrets, err := parseArrValue[buildflags.Secret](o.ArrValue) t.Secrets = o.ArrValue
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
if o.Append {
t.Secrets = t.Secrets.Merge(secrets)
} else {
t.Secrets = secrets
}
for _, s := range t.Secrets {
if s.FilePath != "" {
ent.FSRead = append(ent.FSRead, s.FilePath)
}
}
case "ssh": case "ssh":
ssh, err := parseArrValue[buildflags.SSH](o.ArrValue) t.SSH = o.ArrValue
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
if o.Append {
t.SSH = t.SSH.Merge(ssh)
} else {
t.SSH = ssh
}
for _, s := range t.SSH {
ent.FSRead = append(ent.FSRead, s.Paths...)
}
case "platform": case "platform":
if o.Append { t.Platforms = o.ArrValue
t.Platforms = append(t.Platforms, o.ArrValue...)
} else {
t.Platforms = o.ArrValue
}
case "output": case "output":
outputs, err := parseArrValue[buildflags.ExportEntry](o.ArrValue) t.Outputs = o.ArrValue
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
if o.Append {
t.Outputs = t.Outputs.Merge(outputs)
} else {
t.Outputs = outputs
}
for _, o := range t.Outputs {
if o.Destination != "" {
ent.FSWrite = append(ent.FSWrite, o.Destination)
}
}
case "entitlements": case "entitlements":
t.Entitlements = append(t.Entitlements, o.ArrValue...) t.Entitlements = append(t.Entitlements, o.ArrValue...)
for _, v := range o.ArrValue {
if v == string(EntitlementKeyNetworkHost) {
ent.NetworkHost = true
} else if v == string(EntitlementKeySecurityInsecure) {
ent.SecurityInsecure = true
}
}
case "annotations": case "annotations":
t.Annotations = append(t.Annotations, o.ArrValue...) t.Annotations = append(t.Annotations, o.ArrValue...)
case "attest": case "attest":
attest, err := parseArrValue[buildflags.Attest](o.ArrValue) t.Attest = append(t.Attest, o.ArrValue...)
if err != nil {
return errors.Wrap(err, "invalid value for attest")
}
t.Attest = t.Attest.Merge(attest)
case "no-cache": case "no-cache":
noCache, err := strconv.ParseBool(value) noCache, err := strconv.ParseBool(value)
if err != nil { if err != nil {
@@ -1014,19 +909,11 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
} }
t.NoCache = &noCache t.NoCache = &noCache
case "no-cache-filter": case "no-cache-filter":
if o.Append { t.NoCacheFilter = o.ArrValue
t.NoCacheFilter = append(t.NoCacheFilter, o.ArrValue...)
} else {
t.NoCacheFilter = o.ArrValue
}
case "shm-size": case "shm-size":
t.ShmSize = &value t.ShmSize = &value
case "ulimits": case "ulimits":
if o.Append { t.Ulimits = o.ArrValue
t.Ulimits = append(t.Ulimits, o.ArrValue...)
} else {
t.Ulimits = o.ArrValue
}
case "network": case "network":
t.NetworkMode = &value t.NetworkMode = &value
case "pull": case "pull":
@@ -1167,9 +1054,7 @@ func (t *Target) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(
func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) { func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) {
// make sure local credentials are loaded multiple times for different targets // make sure local credentials are loaded multiple times for different targets
dockerConfig := config.LoadDefaultConfigFile(os.Stderr) dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
authProvider := authprovider.NewDockerAuthProvider(authprovider.DockerAuthProviderConfig{ authProvider := authprovider.NewDockerAuthProvider(dockerConfig, nil)
ConfigFile: dockerConfig,
})
m2 := make(map[string]build.Options, len(m)) m2 := make(map[string]build.Options, len(m))
for k, v := range m { for k, v := range m {
@@ -1221,44 +1106,62 @@ func updateContext(t *build.Inputs, inp *Input) {
t.ContextState = &st t.ContextState = &st
} }
func isRemoteContext(t build.Inputs, inp *Input) bool { // validateContextsEntitlements is a basic check to ensure contexts do not
if build.IsRemoteURL(t.ContextPath) { // escape local directories when loaded from remote sources. This is to be
return true // replaced with proper entitlements support in the future.
func validateContextsEntitlements(t build.Inputs, inp *Input) error {
if inp == nil || inp.State == nil {
return nil
} }
if inp != nil && build.IsRemoteURL(inp.URL) && !strings.HasPrefix(t.ContextPath, "cwd://") { if v, ok := os.LookupEnv("BAKE_ALLOW_REMOTE_FS_ACCESS"); ok {
return true if vv, _ := strconv.ParseBool(v); vv {
return nil
}
} }
return false
}
func collectLocalPaths(t build.Inputs) []string {
var out []string
if t.ContextState == nil { if t.ContextState == nil {
if v, ok := isLocalPath(t.ContextPath); ok { if err := checkPath(t.ContextPath); err != nil {
out = append(out, v) return err
} }
if v, ok := isLocalPath(t.DockerfilePath); ok {
out = append(out, v)
}
} else if strings.HasPrefix(t.ContextPath, "cwd://") {
out = append(out, strings.TrimPrefix(t.ContextPath, "cwd://"))
} }
for _, v := range t.NamedContexts { for _, v := range t.NamedContexts {
if v.State != nil { if v.State != nil {
continue continue
} }
if v, ok := isLocalPath(v.Path); ok { if err := checkPath(v.Path); err != nil {
out = append(out, v) return err
} }
} }
return out return nil
} }
func isLocalPath(p string) (string, bool) { func checkPath(p string) error {
if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") { if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
return "", false return nil
} }
return strings.TrimPrefix(p, "cwd://"), true p, err := filepath.EvalSymlinks(p)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
p, err = filepath.Abs(p)
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return err
}
rel, err := filepath.Rel(wd, p)
if err != nil {
return err
}
parts := strings.Split(rel, string(os.PathSeparator))
if parts[0] == ".." {
return errors.Errorf("path %s is outside of the working directory, please set BAKE_ALLOW_REMOTE_FS_ACCESS=1", p)
}
return nil
} }
func toBuildOpt(t *Target, inp *Input) (*build.Options, error) { func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
@@ -1298,6 +1201,9 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
// it's not outside the working directory and then resolve it to an // it's not outside the working directory and then resolve it to an
// absolute path. // absolute path.
bi.DockerfilePath = path.Clean(strings.TrimPrefix(bi.DockerfilePath, "cwd://")) bi.DockerfilePath = path.Clean(strings.TrimPrefix(bi.DockerfilePath, "cwd://"))
if err := checkPath(bi.DockerfilePath); err != nil {
return nil, err
}
var err error var err error
bi.DockerfilePath, err = filepath.Abs(bi.DockerfilePath) bi.DockerfilePath, err = filepath.Abs(bi.DockerfilePath)
if err != nil { if err != nil {
@@ -1334,6 +1240,10 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
} }
} }
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath t.Context = &bi.ContextPath
args := map[string]string{} args := map[string]string{}
@@ -1390,35 +1300,24 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
} }
bo.Platforms = platforms bo.Platforms = platforms
secrets := t.Secrets secrets, err := buildflags.ParseSecretSpecs(t.Secrets)
if isRemoteContext(bi, inp) { if err != nil {
if _, ok := os.LookupEnv("BUILDX_BAKE_GIT_AUTH_TOKEN"); ok { return nil, err
secrets = append(secrets, &buildflags.Secret{
ID: llb.GitAuthTokenKey,
Env: "BUILDX_BAKE_GIT_AUTH_TOKEN",
})
}
if _, ok := os.LookupEnv("BUILDX_BAKE_GIT_AUTH_HEADER"); ok {
secrets = append(secrets, &buildflags.Secret{
ID: llb.GitAuthHeaderKey,
Env: "BUILDX_BAKE_GIT_AUTH_HEADER",
})
}
} }
secrets = secrets.Normalize() secretAttachment, err := controllerapi.CreateSecrets(secrets)
bo.SecretSpecs = secrets.ToPB()
secretAttachment, err := controllerapi.CreateSecrets(bo.SecretSpecs)
if err != nil { if err != nil {
return nil, err return nil, err
} }
bo.Session = append(bo.Session, secretAttachment) bo.Session = append(bo.Session, secretAttachment)
bo.SSHSpecs = t.SSH.ToPB() sshSpecs, err := buildflags.ParseSSHSpecs(t.SSH)
if len(bo.SSHSpecs) == 0 && buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL)) { if err != nil {
bo.SSHSpecs = []*controllerapi.SSH{{ID: "default"}} return nil, err
} }
if len(sshSpecs) == 0 && (buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL))) {
sshAttachment, err := controllerapi.CreateSSH(bo.SSHSpecs) sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
sshAttachment, err := controllerapi.CreateSSH(sshSpecs)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -1434,14 +1333,23 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
} }
} }
if t.CacheFrom != nil { cacheImports, err := buildflags.ParseCacheEntry(t.CacheFrom)
bo.CacheFrom = controllerapi.CreateCaches(t.CacheFrom.ToPB()) if err != nil {
} return nil, err
if t.CacheTo != nil {
bo.CacheTo = controllerapi.CreateCaches(t.CacheTo.ToPB())
} }
bo.CacheFrom = controllerapi.CreateCaches(cacheImports)
bo.Exports, bo.ExportsLocalPathsTemporary, err = controllerapi.CreateExports(t.Outputs.ToPB()) cacheExports, err := buildflags.ParseCacheEntry(t.CacheTo)
if err != nil {
return nil, err
}
bo.CacheTo = controllerapi.CreateCaches(cacheExports)
outputs, err := buildflags.ParseExports(t.Outputs)
if err != nil {
return nil, err
}
bo.Exports, err = controllerapi.CreateExports(outputs)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -1456,7 +1364,11 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
} }
} }
bo.Attests = controllerapi.CreateAttestations(t.Attest.ToPB()) attests, err := buildflags.ParseAttests(t.Attest)
if err != nil {
return nil, err
}
bo.Attests = controllerapi.CreateAttestations(attests)
bo.SourcePolicy, err = build.ReadSourcePolicy() bo.SourcePolicy, err = build.ReadSourcePolicy()
if err != nil { if err != nil {
@@ -1471,7 +1383,9 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
} }
bo.Ulimits = ulimits bo.Ulimits = ulimits
bo.Allow = append(bo.Allow, t.Entitlements...) for _, ent := range t.Entitlements {
bo.Allow = append(bo.Allow, entitlements.Entitlement(ent))
}
return bo, nil return bo, nil
} }
@@ -1480,7 +1394,7 @@ func defaultTarget() *Target {
return &Target{} return &Target{}
} }
func removeDupesStr(s []string) []string { func removeDupes(s []string) []string {
i := 0 i := 0
seen := make(map[string]struct{}, len(s)) seen := make(map[string]struct{}, len(s))
for _, v := range s { for _, v := range s {
@@ -1497,76 +1411,106 @@ func removeDupesStr(s []string) []string {
return s[:i] return s[:i]
} }
func setPushOverride(outputs []*buildflags.ExportEntry, push bool) []*buildflags.ExportEntry { func removeAttestDupes(s []string) []string {
if !push { res := []string{}
// Disable push for any relevant export types m := map[string]int{}
for i := 0; i < len(outputs); { for _, v := range s {
output := outputs[i] att, err := buildflags.ParseAttest(v)
switch output.Type { if err != nil {
case "registry": res = append(res, v)
// Filter out registry output type continue
outputs[i], outputs[len(outputs)-1] = outputs[len(outputs)-1], outputs[i]
outputs = outputs[:len(outputs)-1]
continue
case "image":
// Override push attribute
output.Attrs["push"] = "false"
}
i++
}
return outputs
}
// Force push to be enabled
setPush := true
for _, output := range outputs {
if output.Type != "docker" {
// If there is an output type that is not docker, don't set "push"
setPush = false
} }
// Set push attribute for image if i, ok := m[att.Type]; ok {
if output.Type == "image" { res[i] = v
output.Attrs["push"] = "true" } else {
m[att.Type] = len(res)
res = append(res, v)
} }
} }
return res
if setPush {
// No existing output that pushes so add one
outputs = append(outputs, &buildflags.ExportEntry{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
})
}
return outputs
} }
func setLoadOverride(outputs []*buildflags.ExportEntry, load bool) []*buildflags.ExportEntry { func parseOutput(str string) map[string]string {
fields, err := csvvalue.Fields(str, nil)
if err != nil {
return nil
}
res := map[string]string{}
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) == 2 {
res[parts[0]] = parts[1]
}
}
return res
}
func parseOutputType(str string) string {
if out := parseOutput(str); out != nil {
if v, ok := out["type"]; ok {
return v
}
}
return ""
}
func setPushOverride(outputs []string, push bool) []string {
var out []string
setPush := true
for _, output := range outputs {
typ := parseOutputType(output)
if typ == "image" || typ == "registry" {
// no need to set push if image or registry types already defined
setPush = false
if typ == "registry" {
if !push {
// don't set registry output if "push" is false
continue
}
// no need to set "push" attribute to true for registry
out = append(out, output)
continue
}
out = append(out, output+",push="+strconv.FormatBool(push))
} else {
if typ != "docker" {
// if there is any output that is not docker, don't set "push"
setPush = false
}
out = append(out, output)
}
}
if push && setPush {
out = append(out, "type=image,push=true")
}
return out
}
func setLoadOverride(outputs []string, load bool) []string {
if !load { if !load {
return outputs return outputs
} }
setLoad := true
for _, output := range outputs { for _, output := range outputs {
switch output.Type { if typ := parseOutputType(output); typ == "docker" {
case "docker": if v := parseOutput(output); v != nil {
// if dest is not set, we can reuse this entry and do not need to set load // dest set means we want to output as tar so don't set load
if output.Destination == "" { if _, ok := v["dest"]; !ok {
return outputs setLoad = false
break
}
} }
case "image", "registry", "oci": } else if typ != "image" && typ != "registry" && typ != "oci" {
// Ignore
default:
// if there is any output that is not an image, registry // if there is any output that is not an image, registry
// or oci, don't set "load" similar to push override // or oci, don't set "load" similar to push override
return outputs setLoad = false
break
} }
} }
if setLoad {
outputs = append(outputs, &buildflags.ExportEntry{ outputs = append(outputs, "type=docker")
Type: "docker", }
})
return outputs return outputs
} }
@@ -1584,9 +1528,14 @@ func sanitizeTargetName(target string) string {
return strings.ReplaceAll(target, ".", "_") return strings.ReplaceAll(target, ".", "_")
} }
func isSubset(s1, s2 []string) bool { func sliceEqual(s1, s2 []string) bool {
for _, item := range s1 { if len(s1) != len(s2) {
if !slices.Contains(s2, item) { return false
}
sort.Strings(s1)
sort.Strings(s2)
for i := range s1 {
if s1[i] != s2[i] {
return false return false
} }
} }
@@ -1600,24 +1549,3 @@ func toNamedContexts(m map[string]string) map[string]build.NamedContext {
} }
return m2 return m2
} }
type arrValue[B any] interface {
encoding.TextUnmarshaler
*B
}
func parseArrValue[T any, PT arrValue[T]](s []string) ([]*T, error) {
outputs := make([]*T, 0, len(s))
for _, text := range s {
if text == "" {
continue
}
output := new(T)
if err := PT(output).UnmarshalText([]byte(text)); err != nil {
return nil, err
}
outputs = append(outputs, output)
}
return outputs, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,14 +5,13 @@ import (
"fmt" "fmt"
"os" "os"
"path/filepath" "path/filepath"
"slices" "sort"
"strings" "strings"
"github.com/compose-spec/compose-go/v2/consts" "github.com/compose-spec/compose-go/v2/consts"
"github.com/compose-spec/compose-go/v2/dotenv" "github.com/compose-spec/compose-go/v2/dotenv"
"github.com/compose-spec/compose-go/v2/loader" "github.com/compose-spec/compose-go/v2/loader"
composetypes "github.com/compose-spec/compose-go/v2/types" composetypes "github.com/compose-spec/compose-go/v2/types"
"github.com/docker/buildx/util/buildflags"
dockeropts "github.com/docker/cli/opts" dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/pkg/errors" "github.com/pkg/errors"
@@ -103,12 +102,6 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
shmSize = &shmSizeStr shmSize = &shmSizeStr
} }
var networkModeP *string
if s.Build.Network != "" {
networkMode := s.Build.Network
networkModeP = &networkMode
}
var ulimits []string var ulimits []string
if s.Build.Ulimits != nil { if s.Build.Ulimits != nil {
for n, u := range s.Build.Ulimits { for n, u := range s.Build.Ulimits {
@@ -120,16 +113,14 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
} }
} }
var ssh []*buildflags.SSH var ssh []string
for _, bkey := range s.Build.SSH { for _, bkey := range s.Build.SSH {
sshkey := composeToBuildkitSSH(bkey) sshkey := composeToBuildkitSSH(bkey)
ssh = append(ssh, sshkey) ssh = append(ssh, sshkey)
} }
slices.SortFunc(ssh, func(a, b *buildflags.SSH) int { sort.Strings(ssh)
return a.Less(b)
})
var secrets []*buildflags.Secret var secrets []string
for _, bs := range s.Build.Secrets { for _, bs := range s.Build.Secrets {
secret, err := composeToBuildkitSecret(bs, cfg.Secrets[bs.Source]) secret, err := composeToBuildkitSecret(bs, cfg.Secrets[bs.Source])
if err != nil { if err != nil {
@@ -145,16 +136,6 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
labels[k] = &v labels[k] = &v
} }
cacheFrom, err := buildflags.ParseCacheEntry(s.Build.CacheFrom)
if err != nil {
return nil, err
}
cacheTo, err := buildflags.ParseCacheEntry(s.Build.CacheTo)
if err != nil {
return nil, err
}
g.Targets = append(g.Targets, targetName) g.Targets = append(g.Targets, targetName)
t := &Target{ t := &Target{
Name: targetName, Name: targetName,
@@ -171,9 +152,9 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
val, ok := cfg.Environment[val] val, ok := cfg.Environment[val]
return val, ok return val, ok
})), })),
CacheFrom: cacheFrom, CacheFrom: s.Build.CacheFrom,
CacheTo: cacheTo, CacheTo: s.Build.CacheTo,
NetworkMode: networkModeP, NetworkMode: &s.Build.Network,
SSH: ssh, SSH: ssh,
Secrets: secrets, Secrets: secrets,
ShmSize: shmSize, ShmSize: shmSize,
@@ -192,6 +173,7 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
c.Targets = append(c.Targets, t) c.Targets = append(c.Targets, t)
} }
c.Groups = append(c.Groups, g) c.Groups = append(c.Groups, g)
} }
return &c, nil return &c, nil
@@ -310,12 +292,10 @@ type xbake struct {
// https://github.com/docker/docs/blob/main/content/build/bake/compose-file.md#extension-field-with-x-bake // https://github.com/docker/docs/blob/main/content/build/bake/compose-file.md#extension-field-with-x-bake
} }
type ( type stringMap map[string]string
stringMap map[string]string type stringArray []string
stringArray []string
)
func (sa *stringArray) UnmarshalYAML(unmarshal func(any) error) error { func (sa *stringArray) UnmarshalYAML(unmarshal func(interface{}) error) error {
var multi []string var multi []string
err := unmarshal(&multi) err := unmarshal(&multi)
if err != nil { if err != nil {
@@ -332,7 +312,7 @@ func (sa *stringArray) UnmarshalYAML(unmarshal func(any) error) error {
// composeExtTarget converts Compose build extension x-bake to bake Target // composeExtTarget converts Compose build extension x-bake to bake Target
// https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension // https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension
func (t *Target) composeExtTarget(exts map[string]any) error { func (t *Target) composeExtTarget(exts map[string]interface{}) error {
var xb xbake var xb xbake
ext, ok := exts["x-bake"] ext, ok := exts["x-bake"]
@@ -349,45 +329,23 @@ func (t *Target) composeExtTarget(exts map[string]any) error {
t.Tags = dedupSlice(append(t.Tags, xb.Tags...)) t.Tags = dedupSlice(append(t.Tags, xb.Tags...))
} }
if len(xb.CacheFrom) > 0 { if len(xb.CacheFrom) > 0 {
cacheFrom, err := buildflags.ParseCacheEntry(xb.CacheFrom) t.CacheFrom = dedupSlice(append(t.CacheFrom, xb.CacheFrom...))
if err != nil {
return err
}
t.CacheFrom = t.CacheFrom.Merge(cacheFrom)
} }
if len(xb.CacheTo) > 0 { if len(xb.CacheTo) > 0 {
cacheTo, err := buildflags.ParseCacheEntry(xb.CacheTo) t.CacheTo = dedupSlice(append(t.CacheTo, xb.CacheTo...))
if err != nil {
return err
}
t.CacheTo = t.CacheTo.Merge(cacheTo)
} }
if len(xb.Secrets) > 0 { if len(xb.Secrets) > 0 {
secrets, err := parseArrValue[buildflags.Secret](xb.Secrets) t.Secrets = dedupSlice(append(t.Secrets, xb.Secrets...))
if err != nil {
return err
}
t.Secrets = t.Secrets.Merge(secrets)
} }
if len(xb.SSH) > 0 { if len(xb.SSH) > 0 {
ssh, err := parseArrValue[buildflags.SSH](xb.SSH) t.SSH = dedupSlice(append(t.SSH, xb.SSH...))
if err != nil { sort.Strings(t.SSH)
return err
}
t.SSH = t.SSH.Merge(ssh)
slices.SortFunc(t.SSH, func(a, b *buildflags.SSH) int {
return a.Less(b)
})
} }
if len(xb.Platforms) > 0 { if len(xb.Platforms) > 0 {
t.Platforms = dedupSlice(append(t.Platforms, xb.Platforms...)) t.Platforms = dedupSlice(append(t.Platforms, xb.Platforms...))
} }
if len(xb.Outputs) > 0 { if len(xb.Outputs) > 0 {
outputs, err := parseArrValue[buildflags.ExportEntry](xb.Outputs) t.Outputs = dedupSlice(append(t.Outputs, xb.Outputs...))
if err != nil {
return err
}
t.Outputs = t.Outputs.Merge(outputs)
} }
if xb.Pull != nil { if xb.Pull != nil {
t.Pull = xb.Pull t.Pull = xb.Pull
@@ -407,30 +365,35 @@ func (t *Target) composeExtTarget(exts map[string]any) error {
// composeToBuildkitSecret converts secret from compose format to buildkit's // composeToBuildkitSecret converts secret from compose format to buildkit's
// csv format. // csv format.
func composeToBuildkitSecret(inp composetypes.ServiceSecretConfig, psecret composetypes.SecretConfig) (*buildflags.Secret, error) { func composeToBuildkitSecret(inp composetypes.ServiceSecretConfig, psecret composetypes.SecretConfig) (string, error) {
if psecret.External { if psecret.External {
return nil, errors.Errorf("unsupported external secret %s", psecret.Name) return "", errors.Errorf("unsupported external secret %s", psecret.Name)
} }
secret := &buildflags.Secret{} var bkattrs []string
if inp.Source != "" { if inp.Source != "" {
secret.ID = inp.Source bkattrs = append(bkattrs, "id="+inp.Source)
} }
if psecret.File != "" { if psecret.File != "" {
secret.FilePath = psecret.File bkattrs = append(bkattrs, "src="+psecret.File)
} }
if psecret.Environment != "" { if psecret.Environment != "" {
secret.Env = psecret.Environment bkattrs = append(bkattrs, "env="+psecret.Environment)
} }
return secret, nil
return strings.Join(bkattrs, ","), nil
} }
// composeToBuildkitSSH converts secret from compose format to buildkit's // composeToBuildkitSSH converts secret from compose format to buildkit's
// csv format. // csv format.
func composeToBuildkitSSH(sshKey composetypes.SSHKey) *buildflags.SSH { func composeToBuildkitSSH(sshKey composetypes.SSHKey) string {
bkssh := &buildflags.SSH{ID: sshKey.ID} var bkattrs []string
bkattrs = append(bkattrs, sshKey.ID)
if sshKey.Path != "" { if sshKey.Path != "" {
bkssh.Paths = []string{sshKey.Path} bkattrs = append(bkattrs, sshKey.Path)
} }
return bkssh
return strings.Join(bkattrs, "=")
} }

View File

@@ -12,7 +12,7 @@ import (
) )
func TestParseCompose(t *testing.T) { func TestParseCompose(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
db: db:
build: ./db build: ./db
@@ -33,7 +33,7 @@ services:
cache_to: cache_to:
- type=local,dest=path/to/cache - type=local,dest=path/to/cache
ssh: ssh:
- key=/path/to/key - key=path/to/key
- default - default
secrets: secrets:
- token - token
@@ -74,14 +74,14 @@ secrets:
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile) require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args)) require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"]) require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, []string{"type=local,src=path/to/cache"}, stringify(c.Targets[1].CacheFrom)) require.Equal(t, []string{"type=local,src=path/to/cache"}, c.Targets[1].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache"}, stringify(c.Targets[1].CacheTo)) require.Equal(t, []string{"type=local,dest=path/to/cache"}, c.Targets[1].CacheTo)
require.Equal(t, "none", *c.Targets[1].NetworkMode) require.Equal(t, "none", *c.Targets[1].NetworkMode)
require.Equal(t, []string{"default", "key=/path/to/key"}, stringify(c.Targets[1].SSH)) require.Equal(t, []string{"default", "key=path/to/key"}, c.Targets[1].SSH)
require.Equal(t, []string{ require.Equal(t, []string{
"id=aws,src=/root/.aws/credentials",
"id=token,env=ENV_TOKEN", "id=token,env=ENV_TOKEN",
}, stringify(c.Targets[1].Secrets)) "id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name) require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, "dir", *c.Targets[2].Context) require.Equal(t, "dir", *c.Targets[2].Context)
@@ -89,7 +89,7 @@ secrets:
} }
func TestNoBuildOutOfTreeService(t *testing.T) { func TestNoBuildOutOfTreeService(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
external: external:
image: "verycooldb:1337" image: "verycooldb:1337"
@@ -103,7 +103,7 @@ services:
} }
func TestParseComposeTarget(t *testing.T) { func TestParseComposeTarget(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
db: db:
build: build:
@@ -129,7 +129,7 @@ services:
} }
func TestComposeBuildWithoutContext(t *testing.T) { func TestComposeBuildWithoutContext(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
db: db:
build: build:
@@ -153,7 +153,7 @@ services:
} }
func TestBuildArgEnvCompose(t *testing.T) { func TestBuildArgEnvCompose(t *testing.T) {
dt := []byte(` var dt = []byte(`
version: "3.8" version: "3.8"
services: services:
example: example:
@@ -179,7 +179,7 @@ services:
} }
func TestInconsistentComposeFile(t *testing.T) { func TestInconsistentComposeFile(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
webapp: webapp:
entrypoint: echo 1 entrypoint: echo 1
@@ -190,7 +190,7 @@ services:
} }
func TestAdvancedNetwork(t *testing.T) { func TestAdvancedNetwork(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
db: db:
networks: networks:
@@ -215,7 +215,7 @@ networks:
} }
func TestTags(t *testing.T) { func TestTags(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
example: example:
image: example image: example
@@ -233,7 +233,7 @@ services:
} }
func TestDependsOnList(t *testing.T) { func TestDependsOnList(t *testing.T) {
dt := []byte(` var dt = []byte(`
version: "3.8" version: "3.8"
services: services:
@@ -269,7 +269,7 @@ networks:
} }
func TestComposeExt(t *testing.T) { func TestComposeExt(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
addon: addon:
image: ct-addon:bar image: ct-addon:bar
@@ -283,7 +283,7 @@ services:
tags: tags:
- ct-addon:baz - ct-addon:baz
ssh: ssh:
key: /path/to/key key: path/to/key
args: args:
CT_ECR: foo CT_ECR: foo
CT_TAG: bar CT_TAG: bar
@@ -336,23 +336,23 @@ services:
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "CT_TAG": ptrstr("bar")}, c.Targets[0].Args) require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "CT_TAG": ptrstr("bar")}, c.Targets[0].Args)
require.Equal(t, []string{"ct-addon:baz", "ct-addon:foo", "ct-addon:alp"}, c.Targets[0].Tags) require.Equal(t, []string{"ct-addon:baz", "ct-addon:foo", "ct-addon:alp"}, c.Targets[0].Tags)
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[0].Platforms) require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[0].Platforms)
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom)) require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheTo)) require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
require.Equal(t, []string{"default", "key=/path/to/key", "other=path/to/otherkey"}, stringify(c.Targets[0].SSH)) require.Equal(t, []string{"default", "key=path/to/key", "other=path/to/otherkey"}, c.Targets[0].SSH)
require.Equal(t, newBool(true), c.Targets[0].Pull) require.Equal(t, newBool(true), c.Targets[0].Pull)
require.Equal(t, map[string]string{"alpine": "docker-image://alpine:3.13"}, c.Targets[0].Contexts) require.Equal(t, map[string]string{"alpine": "docker-image://alpine:3.13"}, c.Targets[0].Contexts)
require.Equal(t, []string{"ct-fake-aws:bar"}, c.Targets[1].Tags) require.Equal(t, []string{"ct-fake-aws:bar"}, c.Targets[1].Tags)
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,src=/local/secret2"}, stringify(c.Targets[1].Secrets)) require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,src=/local/secret2"}, c.Targets[1].Secrets)
require.Equal(t, []string{"default"}, stringify(c.Targets[1].SSH)) require.Equal(t, []string{"default"}, c.Targets[1].SSH)
require.Equal(t, []string{"linux/arm64"}, c.Targets[1].Platforms) require.Equal(t, []string{"linux/arm64"}, c.Targets[1].Platforms)
require.Equal(t, []string{"type=docker"}, stringify(c.Targets[1].Outputs)) require.Equal(t, []string{"type=docker"}, c.Targets[1].Outputs)
require.Equal(t, newBool(true), c.Targets[1].NoCache) require.Equal(t, newBool(true), c.Targets[1].NoCache)
require.Equal(t, ptrstr("128MiB"), c.Targets[1].ShmSize) require.Equal(t, ptrstr("128MiB"), c.Targets[1].ShmSize)
require.Equal(t, []string{"nofile=1024:1024"}, c.Targets[1].Ulimits) require.Equal(t, []string{"nofile=1024:1024"}, c.Targets[1].Ulimits)
} }
func TestComposeExtDedup(t *testing.T) { func TestComposeExtDedup(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
webapp: webapp:
image: app:bar image: app:bar
@@ -383,9 +383,9 @@ services:
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"ct-addon:foo", "ct-addon:baz"}, c.Targets[0].Tags) require.Equal(t, []string{"ct-addon:foo", "ct-addon:baz"}, c.Targets[0].Tags)
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom)) require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheTo)) require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
require.Equal(t, []string{"default", "key=path/to/key"}, stringify(c.Targets[0].SSH)) require.Equal(t, []string{"default", "key=path/to/key"}, c.Targets[0].SSH)
} }
func TestEnv(t *testing.T) { func TestEnv(t *testing.T) {
@@ -396,7 +396,7 @@ func TestEnv(t *testing.T) {
_, err = envf.WriteString("FOO=bsdf -csdf\n") _, err = envf.WriteString("FOO=bsdf -csdf\n")
require.NoError(t, err) require.NoError(t, err)
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:
@@ -424,7 +424,7 @@ func TestDotEnv(t *testing.T) {
err := os.WriteFile(filepath.Join(tmpdir, ".env"), []byte("FOO=bar"), 0644) err := os.WriteFile(filepath.Join(tmpdir, ".env"), []byte("FOO=bar"), 0644)
require.NoError(t, err) require.NoError(t, err)
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:
@@ -443,7 +443,7 @@ services:
} }
func TestPorts(t *testing.T) { func TestPorts(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
foo: foo:
build: build:
@@ -664,7 +664,7 @@ target "default" {
} }
func TestComposeNullArgs(t *testing.T) { func TestComposeNullArgs(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:
@@ -680,7 +680,7 @@ services:
} }
func TestDependsOn(t *testing.T) { func TestDependsOn(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
foo: foo:
build: build:
@@ -711,7 +711,7 @@ services:
`), 0644) `), 0644)
require.NoError(t, err) require.NoError(t, err)
dt := []byte(` var dt = []byte(`
include: include:
- compose-foo.yml - compose-foo.yml
@@ -740,7 +740,7 @@ services:
} }
func TestDevelop(t *testing.T) { func TestDevelop(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:
@@ -759,7 +759,7 @@ services:
} }
func TestCgroup(t *testing.T) { func TestCgroup(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:
@@ -772,7 +772,7 @@ services:
} }
func TestProjectName(t *testing.T) { func TestProjectName(t *testing.T) {
dt := []byte(` var dt = []byte(`
services: services:
scratch: scratch:
build: build:

View File

@@ -2,25 +2,17 @@ package bake
import ( import (
"bufio" "bufio"
"cmp"
"context" "context"
"fmt" "fmt"
"io" "io"
"io/fs"
"os" "os"
"path/filepath"
"slices" "slices"
"strconv"
"strings" "strings"
"syscall"
"github.com/containerd/console" "github.com/containerd/console"
"github.com/docker/buildx/build" "github.com/docker/buildx/build"
"github.com/docker/buildx/util/osutil"
"github.com/moby/buildkit/util/entitlements" "github.com/moby/buildkit/util/entitlements"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/tonistiigi/go-csvvalue"
) )
type EntitlementKey string type EntitlementKey string
@@ -28,7 +20,6 @@ type EntitlementKey string
const ( const (
EntitlementKeyNetworkHost EntitlementKey = "network.host" EntitlementKeyNetworkHost EntitlementKey = "network.host"
EntitlementKeySecurityInsecure EntitlementKey = "security.insecure" EntitlementKeySecurityInsecure EntitlementKey = "security.insecure"
EntitlementKeyDevice EntitlementKey = "device"
EntitlementKeyFSRead EntitlementKey = "fs.read" EntitlementKeyFSRead EntitlementKey = "fs.read"
EntitlementKeyFSWrite EntitlementKey = "fs.write" EntitlementKeyFSWrite EntitlementKey = "fs.write"
EntitlementKeyFS EntitlementKey = "fs" EntitlementKeyFS EntitlementKey = "fs"
@@ -41,7 +32,6 @@ const (
type EntitlementConf struct { type EntitlementConf struct {
NetworkHost bool NetworkHost bool
SecurityInsecure bool SecurityInsecure bool
Devices *EntitlementsDevicesConf
FSRead []string FSRead []string
FSWrite []string FSWrite []string
ImagePush []string ImagePush []string
@@ -49,11 +39,6 @@ type EntitlementConf struct {
SSH bool SSH bool
} }
type EntitlementsDevicesConf struct {
All bool
Devices map[string]struct{}
}
func ParseEntitlements(in []string) (EntitlementConf, error) { func ParseEntitlements(in []string) (EntitlementConf, error) {
var conf EntitlementConf var conf EntitlementConf
for _, e := range in { for _, e := range in {
@@ -67,22 +52,6 @@ func ParseEntitlements(in []string) (EntitlementConf, error) {
default: default:
k, v, _ := strings.Cut(e, "=") k, v, _ := strings.Cut(e, "=")
switch k { switch k {
case string(EntitlementKeyDevice):
if v == "" {
conf.Devices = &EntitlementsDevicesConf{All: true}
continue
}
fields, err := csvvalue.Fields(v, nil)
if err != nil {
return EntitlementConf{}, errors.Wrapf(err, "failed to parse device entitlement %q", v)
}
if conf.Devices == nil {
conf.Devices = &EntitlementsDevicesConf{}
}
if conf.Devices.Devices == nil {
conf.Devices.Devices = make(map[string]struct{}, 0)
}
conf.Devices.Devices[fields[0]] = struct{}{}
case string(EntitlementKeyFSRead): case string(EntitlementKeyFSRead):
conf.FSRead = append(conf.FSRead, v) conf.FSRead = append(conf.FSRead, v)
case string(EntitlementKeyFSWrite): case string(EntitlementKeyFSWrite):
@@ -98,8 +67,10 @@ func ParseEntitlements(in []string) (EntitlementConf, error) {
conf.ImagePush = append(conf.ImagePush, v) conf.ImagePush = append(conf.ImagePush, v)
conf.ImageLoad = append(conf.ImageLoad, v) conf.ImageLoad = append(conf.ImageLoad, v)
default: default:
return conf, errors.Errorf("unknown entitlement key %q", k) return conf, errors.Errorf("uknown entitlement key %q", k)
} }
// TODO: dedupe slices and parent paths
} }
} }
return conf, nil return conf, nil
@@ -119,99 +90,21 @@ func (c EntitlementConf) Validate(m map[string]build.Options) (EntitlementConf,
func (c EntitlementConf) check(bo build.Options, expected *EntitlementConf) error { func (c EntitlementConf) check(bo build.Options, expected *EntitlementConf) error {
for _, e := range bo.Allow { for _, e := range bo.Allow {
k, rest, _ := strings.Cut(e, "=")
switch k {
case entitlements.EntitlementDevice.String():
if rest == "" {
if c.Devices == nil || !c.Devices.All {
expected.Devices = &EntitlementsDevicesConf{All: true}
}
continue
}
fields, err := csvvalue.Fields(rest, nil)
if err != nil {
return errors.Wrapf(err, "failed to parse device entitlement %q", rest)
}
if expected.Devices == nil {
expected.Devices = &EntitlementsDevicesConf{}
}
if expected.Devices.Devices == nil {
expected.Devices.Devices = make(map[string]struct{}, 0)
}
expected.Devices.Devices[fields[0]] = struct{}{}
}
switch e { switch e {
case entitlements.EntitlementNetworkHost.String(): case entitlements.EntitlementNetworkHost:
if !c.NetworkHost { if !c.NetworkHost {
expected.NetworkHost = true expected.NetworkHost = true
} }
case entitlements.EntitlementSecurityInsecure.String(): case entitlements.EntitlementSecurityInsecure:
if !c.SecurityInsecure { if !c.SecurityInsecure {
expected.SecurityInsecure = true expected.SecurityInsecure = true
} }
} }
} }
rwPaths := map[string]struct{}{}
roPaths := map[string]struct{}{}
for _, p := range collectLocalPaths(bo.Inputs) {
roPaths[p] = struct{}{}
}
for _, p := range bo.ExportsLocalPathsTemporary {
rwPaths[p] = struct{}{}
}
for _, ce := range bo.CacheTo {
if ce.Type == "local" {
if dest, ok := ce.Attrs["dest"]; ok {
rwPaths[dest] = struct{}{}
}
}
}
for _, ci := range bo.CacheFrom {
if ci.Type == "local" {
if src, ok := ci.Attrs["src"]; ok {
roPaths[src] = struct{}{}
}
}
}
for _, secret := range bo.SecretSpecs {
if secret.FilePath != "" {
roPaths[secret.FilePath] = struct{}{}
}
}
for _, ssh := range bo.SSHSpecs {
for _, p := range ssh.Paths {
roPaths[p] = struct{}{}
}
if len(ssh.Paths) == 0 {
if !c.SSH {
expected.SSH = true
}
}
}
var err error
expected.FSRead, err = findMissingPaths(c.FSRead, roPaths)
if err != nil {
return err
}
expected.FSWrite, err = findMissingPaths(c.FSWrite, rwPaths)
if err != nil {
return err
}
return nil return nil
} }
func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Writer) error { func (c EntitlementConf) Prompt(ctx context.Context, out io.Writer) error {
var term bool var term bool
if _, err := console.ConsoleFromFile(os.Stdin); err == nil { if _, err := console.ConsoleFromFile(os.Stdin); err == nil {
term = true term = true
@@ -220,93 +113,35 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
var msgs []string var msgs []string
var flags []string var flags []string
// these warnings are currently disabled to give users time to update
var msgsFS []string
var flagsFS []string
if c.NetworkHost { if c.NetworkHost {
msgs = append(msgs, " - Running build containers that can access host network") msgs = append(msgs, " - Running build containers that can access host network")
flags = append(flags, string(EntitlementKeyNetworkHost)) flags = append(flags, "network.host")
} }
if c.SecurityInsecure { if c.SecurityInsecure {
msgs = append(msgs, " - Running privileged containers that can make system changes") msgs = append(msgs, " - Running privileged containers that can make system changes")
flags = append(flags, string(EntitlementKeySecurityInsecure)) flags = append(flags, "security.insecure")
} }
if c.Devices != nil { if len(msgs) == 0 {
if c.Devices.All {
msgs = append(msgs, " - Access to CDI devices")
flags = append(flags, string(EntitlementKeyDevice))
} else {
for d := range c.Devices.Devices {
msgs = append(msgs, fmt.Sprintf(" - Access to device %s", d))
flags = append(flags, string(EntitlementKeyDevice)+"="+d)
}
}
}
if c.SSH {
msgsFS = append(msgsFS, " - Forwarding default SSH agent socket")
flagsFS = append(flagsFS, string(EntitlementKeySSH))
}
roPaths, rwPaths, commonPaths := groupSamePaths(c.FSRead, c.FSWrite)
wd, err := os.Getwd()
if err != nil {
return errors.Wrap(err, "failed to get current working directory")
}
wd, err = filepath.EvalSymlinks(wd)
if err != nil {
return errors.Wrap(err, "failed to evaluate working directory")
}
roPaths = toRelativePaths(roPaths, wd)
rwPaths = toRelativePaths(rwPaths, wd)
commonPaths = toRelativePaths(commonPaths, wd)
if len(commonPaths) > 0 {
for _, p := range commonPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Read and write access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFS)+"="+p)
}
}
if len(roPaths) > 0 {
for _, p := range roPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Read access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFSRead)+"="+p)
}
}
if len(rwPaths) > 0 {
for _, p := range rwPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Write access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFSWrite)+"="+p)
}
}
if len(msgs) == 0 && len(msgsFS) == 0 {
return nil return nil
} }
fmt.Fprintf(out, "Your build is requesting privileges for following possibly insecure capabilities:\n\n") fmt.Fprintf(out, "Your build is requesting privileges for following possibly insecure capabilities:\n\n")
for _, m := range slices.Concat(msgs, msgsFS) { for _, m := range msgs {
fmt.Fprintf(out, "%s\n", m) fmt.Fprintf(out, "%s\n", m)
} }
for i, f := range flags { for i, f := range flags {
flags[i] = "--allow=" + f flags[i] = "--allow=" + f
} }
for i, f := range flagsFS {
flagsFS[i] = "--allow=" + f
}
if term { if term {
fmt.Fprintf(out, "\nIn order to not see this message in the future pass %q to grant requested privileges.\n", strings.Join(slices.Concat(flags, flagsFS), " ")) fmt.Fprintf(out, "\nIn order to not see this message in the future pass %q to grant requested privileges.\n", strings.Join(flags, " "))
} else { } else {
fmt.Fprintf(out, "\nPass %q to grant requested privileges.\n", strings.Join(slices.Concat(flags, flagsFS), " ")) fmt.Fprintf(out, "\nPass %q to grant requested privileges.\n", strings.Join(flags, " "))
} }
args := slices.Clone(os.Args) args := append([]string(nil), os.Args...)
if v, ok := os.LookupEnv("DOCKER_CLI_PLUGIN_ORIGINAL_CLI_COMMAND"); ok && v != "" { if v, ok := os.LookupEnv("DOCKER_CLI_PLUGIN_ORIGINAL_CLI_COMMAND"); ok && v != "" {
args[0] = v args[0] = v
} }
@@ -314,33 +149,7 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
if idx != -1 { if idx != -1 {
fmt.Fprintf(out, "\nYour full command with requested privileges:\n\n") fmt.Fprintf(out, "\nYour full command with requested privileges:\n\n")
fmt.Fprintf(out, "%s %s %s\n\n", strings.Join(args[:idx+1], " "), strings.Join(slices.Concat(flags, flagsFS), " "), strings.Join(args[idx+1:], " ")) fmt.Fprintf(out, "%s %s %s\n\n", strings.Join(args[:idx+1], " "), strings.Join(flags, " "), strings.Join(args[idx+1:], " "))
}
fsEntitlementsEnabled := true
if isRemote {
if v, ok := os.LookupEnv("BAKE_ALLOW_REMOTE_FS_ACCESS"); ok {
vv, err := strconv.ParseBool(v)
if err != nil {
return errors.Wrapf(err, "failed to parse BAKE_ALLOW_REMOTE_FS_ACCESS value %q", v)
}
fsEntitlementsEnabled = !vv
}
}
v, fsEntitlementsSet := os.LookupEnv("BUILDX_BAKE_ENTITLEMENTS_FS")
if fsEntitlementsSet {
vv, err := strconv.ParseBool(v)
if err != nil {
return errors.Wrapf(err, "failed to parse BUILDX_BAKE_ENTITLEMENTS_FS value %q", v)
}
fsEntitlementsEnabled = vv
}
if !fsEntitlementsEnabled && len(msgs) == 0 {
return nil
}
if fsEntitlementsEnabled && !fsEntitlementsSet && len(msgsFS) != 0 {
fmt.Fprintf(out, "To disable filesystem entitlements checks, you can set BUILDX_BAKE_ENTITLEMENTS_FS=0 .\n\n")
} }
if term { if term {
@@ -364,296 +173,3 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
return errors.Errorf("additional privileges requested") return errors.Errorf("additional privileges requested")
} }
func isParentOrEqualPath(p, parent string) bool {
if p == parent || parent == "/" {
return true
}
if strings.HasPrefix(p, filepath.Clean(parent+string(filepath.Separator))) {
return true
}
return false
}
func findMissingPaths(set []string, paths map[string]struct{}) ([]string, error) {
set, allowAny, err := evaluatePaths(set)
if err != nil {
return nil, err
} else if allowAny {
return nil, nil
}
paths, err = evaluateToExistingPaths(paths)
if err != nil {
return nil, err
}
paths, err = dedupPaths(paths)
if err != nil {
return nil, err
}
out := make([]string, 0, len(paths))
loop0:
for p := range paths {
for _, c := range set {
if isParentOrEqualPath(p, c) {
continue loop0
}
}
out = append(out, p)
}
if len(out) == 0 {
return nil, nil
}
slices.Sort(out)
return out, nil
}
func dedupPaths(in map[string]struct{}) (map[string]struct{}, error) {
arr := make([]string, 0, len(in))
for p := range in {
arr = append(arr, filepath.Clean(p))
}
slices.SortFunc(arr, func(a, b string) int {
return cmp.Compare(len(a), len(b))
})
m := make(map[string]struct{}, len(arr))
loop0:
for _, p := range arr {
for parent := range m {
if strings.HasPrefix(p, parent+string(filepath.Separator)) {
continue loop0
}
}
m[p] = struct{}{}
}
return m, nil
}
func toRelativePaths(in []string, wd string) []string {
out := make([]string, 0, len(in))
for _, p := range in {
rel, err := filepath.Rel(wd, p)
if err == nil {
// allow up to one level of ".." in the path
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)+"..") {
out = append(out, rel)
continue
}
}
out = append(out, p)
}
return out
}
func groupSamePaths(in1, in2 []string) ([]string, []string, []string) {
if in1 == nil || in2 == nil {
return in1, in2, nil
}
slices.Sort(in1)
slices.Sort(in2)
common := []string{}
i, j := 0, 0
for i < len(in1) && j < len(in2) {
switch {
case in1[i] == in2[j]:
common = append(common, in1[i])
i++
j++
case in1[i] < in2[j]:
i++
default:
j++
}
}
in1 = removeCommonPaths(in1, common)
in2 = removeCommonPaths(in2, common)
return in1, in2, common
}
func removeCommonPaths(in, common []string) []string {
filtered := make([]string, 0, len(in))
commonIndex := 0
for _, path := range in {
if commonIndex < len(common) && path == common[commonIndex] {
commonIndex++
continue
}
filtered = append(filtered, path)
}
return filtered
}
func evaluatePaths(in []string) ([]string, bool, error) {
out := make([]string, 0, len(in))
allowAny := false
for _, p := range in {
if p == "*" {
allowAny = true
continue
}
v, err := filepath.Abs(p)
if err != nil {
logrus.Warnf("failed to evaluate entitlement path %q: %v", p, err)
continue
}
v, rest, err := evaluateToExistingPath(v)
if err != nil {
return nil, false, errors.Wrapf(err, "failed to evaluate path %q", p)
}
v, err = osutil.GetLongPathName(v)
if err != nil {
return nil, false, errors.Wrapf(err, "failed to evaluate path %q", p)
}
if rest != "" {
v = filepath.Join(v, rest)
}
out = append(out, v)
}
return out, allowAny, nil
}
func evaluateToExistingPaths(in map[string]struct{}) (map[string]struct{}, error) {
m := make(map[string]struct{}, len(in))
for p := range in {
v, _, err := evaluateToExistingPath(p)
if err != nil {
return nil, errors.Wrapf(err, "failed to evaluate path %q", p)
}
v, err = osutil.GetLongPathName(v)
if err != nil {
return nil, errors.Wrapf(err, "failed to evaluate path %q", p)
}
m[v] = struct{}{}
}
return m, nil
}
func evaluateToExistingPath(in string) (string, string, error) {
in, err := filepath.Abs(in)
if err != nil {
return "", "", err
}
volLen := volumeNameLen(in)
pathSeparator := string(os.PathSeparator)
if volLen < len(in) && os.IsPathSeparator(in[volLen]) {
volLen++
}
vol := in[:volLen]
dest := vol
linksWalked := 0
var end int
for start := volLen; start < len(in); start = end {
for start < len(in) && os.IsPathSeparator(in[start]) {
start++
}
end = start
for end < len(in) && !os.IsPathSeparator(in[end]) {
end++
}
if end == start {
break
} else if in[start:end] == "." {
continue
} else if in[start:end] == ".." {
var r int
for r = len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
break
}
}
if r < volLen || dest[r+1:] == ".." {
if len(dest) > volLen {
dest += pathSeparator
}
dest += ".."
} else {
dest = dest[:r]
}
continue
}
if len(dest) > volumeNameLen(dest) && !os.IsPathSeparator(dest[len(dest)-1]) {
dest += pathSeparator
}
dest += in[start:end]
fi, err := os.Lstat(dest)
if err != nil {
// If the component doesn't exist, return the last valid path
if os.IsNotExist(err) {
for r := len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
return dest[:r], in[start:], nil
}
}
return vol, in[start:], nil
}
return "", "", err
}
if fi.Mode()&fs.ModeSymlink == 0 {
if !fi.Mode().IsDir() && end < len(in) {
return "", "", syscall.ENOTDIR
}
continue
}
linksWalked++
if linksWalked > 255 {
return "", "", errors.New("too many symlinks")
}
link, err := os.Readlink(dest)
if err != nil {
return "", "", err
}
in = link + in[end:]
v := volumeNameLen(link)
if v > 0 {
if v < len(link) && os.IsPathSeparator(link[v]) {
v++
}
vol = link[:v]
dest = vol
end = len(vol)
} else if len(link) > 0 && os.IsPathSeparator(link[0]) {
dest = link[:1]
end = 1
vol = link[:1]
volLen = 1
} else {
var r int
for r = len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
break
}
}
if r < volLen {
dest = vol
} else {
dest = dest[:r]
}
end = 0
}
}
return filepath.Clean(dest), "", nil
}
func volumeNameLen(s string) int {
return len(filepath.VolumeName(s))
}

View File

@@ -1,486 +0,0 @@
package bake
import (
"fmt"
"os"
"path/filepath"
"slices"
"testing"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/osutil"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/util/entitlements"
"github.com/stretchr/testify/require"
)
func TestEvaluateToExistingPath(t *testing.T) {
tempDir, err := osutil.GetLongPathName(t.TempDir())
require.NoError(t, err)
// Setup temporary directory structure for testing
existingFile := filepath.Join(tempDir, "existing_file")
require.NoError(t, os.WriteFile(existingFile, []byte("test"), 0644))
existingDir := filepath.Join(tempDir, "existing_dir")
require.NoError(t, os.Mkdir(existingDir, 0755))
symlinkToFile := filepath.Join(tempDir, "symlink_to_file")
require.NoError(t, os.Symlink(existingFile, symlinkToFile))
symlinkToDir := filepath.Join(tempDir, "symlink_to_dir")
require.NoError(t, os.Symlink(existingDir, symlinkToDir))
nonexistentPath := filepath.Join(tempDir, "nonexistent", "path", "file.txt")
tests := []struct {
name string
input string
expected string
expectErr bool
}{
{
name: "Existing file",
input: existingFile,
expected: existingFile,
expectErr: false,
},
{
name: "Existing directory",
input: existingDir,
expected: existingDir,
expectErr: false,
},
{
name: "Symlink to file",
input: symlinkToFile,
expected: existingFile,
expectErr: false,
},
{
name: "Symlink to directory",
input: symlinkToDir,
expected: existingDir,
expectErr: false,
},
{
name: "Non-existent path",
input: nonexistentPath,
expected: tempDir,
expectErr: false,
},
{
name: "Non-existent intermediate path",
input: filepath.Join(tempDir, "nonexistent", "file.txt"),
expected: tempDir,
expectErr: false,
},
{
name: "Root path",
input: "/",
expected: func() string {
root, _ := filepath.Abs("/")
return root
}(),
expectErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, _, err := evaluateToExistingPath(tt.input)
if tt.expectErr {
require.Error(t, err)
} else {
require.NoError(t, err)
require.Equal(t, tt.expected, result)
}
})
}
}
func TestDedupePaths(t *testing.T) {
wd := osutil.GetWd()
tcases := []struct {
in map[string]struct{}
out map[string]struct{}
}{
{
in: map[string]struct{}{
"/a/b/c": {},
"/a/b/d": {},
"/a/b/e": {},
},
out: map[string]struct{}{
"/a/b/c": {},
"/a/b/d": {},
"/a/b/e": {},
},
},
{
in: map[string]struct{}{
"/a/b/c": {},
"/a/b/c/d": {},
"/a/b/c/d/e": {},
"/a/b/../b/c": {},
},
out: map[string]struct{}{
"/a/b/c": {},
},
},
{
in: map[string]struct{}{
filepath.Join(wd, "a/b/c"): {},
filepath.Join(wd, "../aa"): {},
filepath.Join(wd, "a/b"): {},
filepath.Join(wd, "a/b/d"): {},
filepath.Join(wd, "../aa/b"): {},
filepath.Join(wd, "../../bb"): {},
},
out: map[string]struct{}{
"a/b": {},
"../aa": {},
filepath.Join(wd, "../../bb"): {},
},
},
}
for i, tc := range tcases {
t.Run(fmt.Sprintf("case%d", i), func(t *testing.T) {
out, err := dedupPaths(tc.in)
if err != nil {
require.NoError(t, err)
}
// convert to relative paths as that is shown to user
arr := make([]string, 0, len(out))
for k := range out {
arr = append(arr, k)
}
require.NoError(t, err)
arr = toRelativePaths(arr, wd)
m := make(map[string]struct{})
for _, v := range arr {
m[filepath.ToSlash(v)] = struct{}{}
}
o := make(map[string]struct{}, len(tc.out))
for k := range tc.out {
o[filepath.ToSlash(k)] = struct{}{}
}
require.Equal(t, o, m)
})
}
}
func TestValidateEntitlements(t *testing.T) {
dir1 := t.TempDir()
dir2 := t.TempDir()
// the paths returned by entitlements validation will have symlinks resolved
expDir1, err := filepath.EvalSymlinks(dir1)
require.NoError(t, err)
expDir2, err := filepath.EvalSymlinks(dir2)
require.NoError(t, err)
escapeLink := filepath.Join(dir1, "escape_link")
require.NoError(t, os.Symlink("../../aa", escapeLink))
wd, err := os.Getwd()
require.NoError(t, err)
expWd, err := filepath.EvalSymlinks(wd)
require.NoError(t, err)
tcases := []struct {
name string
conf EntitlementConf
opt build.Options
expected EntitlementConf
}{
{
name: "No entitlements",
opt: build.Options{
Inputs: build.Inputs{
ContextState: &llb.State{},
},
},
},
{
name: "NetworkHostMissing",
opt: build.Options{
Allow: []string{
entitlements.EntitlementNetworkHost.String(),
},
},
expected: EntitlementConf{
NetworkHost: true,
FSRead: []string{expWd},
},
},
{
name: "NetworkHostSet",
conf: EntitlementConf{
NetworkHost: true,
},
opt: build.Options{
Allow: []string{
entitlements.EntitlementNetworkHost.String(),
},
},
expected: EntitlementConf{
FSRead: []string{expWd},
},
},
{
name: "SecurityAndNetworkHostMissing",
opt: build.Options{
Allow: []string{
entitlements.EntitlementNetworkHost.String(),
entitlements.EntitlementSecurityInsecure.String(),
},
},
expected: EntitlementConf{
NetworkHost: true,
SecurityInsecure: true,
FSRead: []string{expWd},
},
},
{
name: "SecurityMissingAndNetworkHostSet",
conf: EntitlementConf{
NetworkHost: true,
},
opt: build.Options{
Allow: []string{
entitlements.EntitlementNetworkHost.String(),
entitlements.EntitlementSecurityInsecure.String(),
},
},
expected: EntitlementConf{
SecurityInsecure: true,
FSRead: []string{expWd},
},
},
{
name: "SSHMissing",
opt: build.Options{
SSHSpecs: []*pb.SSH{
{
ID: "test",
},
},
},
expected: EntitlementConf{
SSH: true,
FSRead: []string{expWd},
},
},
{
name: "ExportLocal",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
dir1,
filepath.Join(dir1, "subdir"),
dir2,
},
},
expected: EntitlementConf{
FSWrite: func() []string {
exp := []string{expDir1, expDir2}
slices.Sort(exp)
return exp
}(),
FSRead: []string{expWd},
},
},
{
name: "SecretFromSubFile",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: filepath.Join(dir1, "subfile"),
},
},
},
conf: EntitlementConf{
FSRead: []string{wd, dir1},
},
},
{
name: "SecretFromEscapeLink",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{wd, dir1},
},
expected: EntitlementConf{
FSRead: []string{filepath.Join(expDir1, "../..")},
},
},
{
name: "SecretFromEscapeLinkAllowRoot",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{"/"},
},
expected: EntitlementConf{
FSRead: func() []string {
// on windows root (/) is only allowed if it is the same volume as wd
if filepath.VolumeName(wd) == filepath.VolumeName(escapeLink) {
return nil
}
// if not, then escapeLink is not allowed
exp, _, err := evaluateToExistingPath(escapeLink)
require.NoError(t, err)
exp, err = filepath.EvalSymlinks(exp)
require.NoError(t, err)
return []string{exp}
}(),
},
},
{
name: "SecretFromEscapeLinkAllowAny",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{"*"},
},
expected: EntitlementConf{},
},
{
name: "NonExistingAllowedPathSubpath",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
dir1,
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{filepath.Join(dir1, "not/exists")},
},
expected: EntitlementConf{
FSWrite: []string{expDir1}, // dir1 is still needed as only subpath was allowed
},
},
{
name: "NonExistingAllowedPathMatches",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
filepath.Join(dir1, "not/exists"),
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{filepath.Join(dir1, "not/exists")},
},
expected: EntitlementConf{
FSWrite: []string{expDir1}, // dir1 is still needed as build also needs to write not/exists directory
},
},
{
name: "NonExistingBuildPath",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
filepath.Join(dir1, "not/exists"),
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{dir1},
},
},
}
for _, tc := range tcases {
t.Run(tc.name, func(t *testing.T) {
expected, err := tc.conf.Validate(map[string]build.Options{"test": tc.opt})
require.NoError(t, err)
require.Equal(t, tc.expected, expected)
})
}
}
func TestGroupSamePaths(t *testing.T) {
tests := []struct {
name string
in1 []string
in2 []string
expected1 []string
expected2 []string
expectedC []string
}{
{
name: "All common paths",
in1: []string{"/path/a", "/path/b", "/path/c"},
in2: []string{"/path/a", "/path/b", "/path/c"},
expected1: []string{},
expected2: []string{},
expectedC: []string{"/path/a", "/path/b", "/path/c"},
},
{
name: "No common paths",
in1: []string{"/path/a", "/path/b"},
in2: []string{"/path/c", "/path/d"},
expected1: []string{"/path/a", "/path/b"},
expected2: []string{"/path/c", "/path/d"},
expectedC: []string{},
},
{
name: "Some common paths",
in1: []string{"/path/a", "/path/b", "/path/c"},
in2: []string{"/path/b", "/path/c", "/path/d"},
expected1: []string{"/path/a"},
expected2: []string{"/path/d"},
expectedC: []string{"/path/b", "/path/c"},
},
{
name: "Empty inputs",
in1: []string{},
in2: []string{},
expected1: []string{},
expected2: []string{},
expectedC: []string{},
},
{
name: "One empty input",
in1: []string{"/path/a", "/path/b"},
in2: []string{},
expected1: []string{"/path/a", "/path/b"},
expected2: []string{},
expectedC: []string{},
},
{
name: "Unsorted inputs with common paths",
in1: []string{"/path/c", "/path/a", "/path/b"},
in2: []string{"/path/b", "/path/c", "/path/a"},
expected1: []string{},
expected2: []string{},
expectedC: []string{"/path/a", "/path/b", "/path/c"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
out1, out2, common := groupSamePaths(tt.in1, tt.in2)
require.Equal(t, tt.expected1, out1, "in1 should match expected1")
require.Equal(t, tt.expected2, out2, "in2 should match expected2")
require.Equal(t, tt.expectedC, common, "common should match expectedC")
})
}
}

View File

@@ -56,7 +56,7 @@ func formatHCLError(err error, files []File) error {
break break
} }
} }
src := &errdefs.Source{ src := errdefs.Source{
Info: &pb.SourceInfo{ Info: &pb.SourceInfo{
Filename: d.Subject.Filename, Filename: d.Subject.Filename,
Data: dt, Data: dt,
@@ -72,7 +72,7 @@ func formatHCLError(err error, files []File) error {
func toErrRange(in *hcl.Range) *pb.Range { func toErrRange(in *hcl.Range) *pb.Range {
return &pb.Range{ return &pb.Range{
Start: &pb.Position{Line: int32(in.Start.Line), Character: int32(in.Start.Column)}, Start: pb.Position{Line: int32(in.Start.Line), Character: int32(in.Start.Column)},
End: &pb.Position{Line: int32(in.End.Line), Character: int32(in.End.Column)}, End: pb.Position{Line: int32(in.End.Line), Character: int32(in.End.Column)},
} }
} }

View File

@@ -2,10 +2,8 @@ package bake
import ( import (
"reflect" "reflect"
"regexp"
"testing" "testing"
hcl "github.com/hashicorp/hcl/v2"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
) )
@@ -19,7 +17,6 @@ func TestHCLBasic(t *testing.T) {
target "db" { target "db" {
context = "./db" context = "./db"
tags = ["docker.io/tonistiigi/db"] tags = ["docker.io/tonistiigi/db"]
output = ["type=image"]
} }
target "webapp" { target "webapp" {
@@ -28,9 +25,6 @@ func TestHCLBasic(t *testing.T) {
args = { args = {
buildno = "123" buildno = "123"
} }
output = [
{ type = "image" }
]
} }
target "cross" { target "cross" {
@@ -55,18 +49,18 @@ func TestHCLBasic(t *testing.T) {
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, 4, len(c.Targets)) require.Equal(t, 4, len(c.Targets))
require.Equal(t, "db", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "db")
require.Equal(t, "./db", *c.Targets[0].Context) require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, "webapp", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args)) require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"]) require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, "cross", c.Targets[2].Name) require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms)) require.Equal(t, 2, len(c.Targets[2].Platforms))
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms) require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms)
require.Equal(t, "webapp-plus", c.Targets[3].Name) require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args)) require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args) require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
} }
@@ -115,18 +109,18 @@ func TestHCLBasicInJSON(t *testing.T) {
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, 4, len(c.Targets)) require.Equal(t, 4, len(c.Targets))
require.Equal(t, "db", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "db")
require.Equal(t, "./db", *c.Targets[0].Context) require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, "webapp", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args)) require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"]) require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, "cross", c.Targets[2].Name) require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms)) require.Equal(t, 2, len(c.Targets[2].Platforms))
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms) require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms)
require.Equal(t, "webapp-plus", c.Targets[3].Name) require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args)) require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args) require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
} }
@@ -152,7 +146,7 @@ func TestHCLWithFunctions(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"]) require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
} }
@@ -182,7 +176,7 @@ func TestHCLWithUserDefinedFunctions(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"]) require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
} }
@@ -211,7 +205,7 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("123"), c.Targets[0].Args["buildno"]) require.Equal(t, ptrstr("123"), c.Targets[0].Args["buildno"])
t.Setenv("BUILD_NUMBER", "456") t.Setenv("BUILD_NUMBER", "456")
@@ -224,7 +218,7 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets) require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("456"), c.Targets[0].Args["buildno"]) require.Equal(t, ptrstr("456"), c.Targets[0].Args["buildno"])
} }
@@ -247,7 +241,7 @@ func TestHCLWithVariablesInFunctions(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, []string{"user/repo:v1"}, c.Targets[0].Tags) require.Equal(t, []string{"user/repo:v1"}, c.Targets[0].Tags)
t.Setenv("REPO", "docker/buildx") t.Setenv("REPO", "docker/buildx")
@@ -256,7 +250,7 @@ func TestHCLWithVariablesInFunctions(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, []string{"docker/buildx:v1"}, c.Targets[0].Tags) require.Equal(t, []string{"docker/buildx:v1"}, c.Targets[0].Tags)
} }
@@ -285,7 +279,7 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
}, nil) }, nil)
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-abc"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-abc"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("abc-post"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("abc-post"), c.Targets[0].Args["v2"])
@@ -298,7 +292,7 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("def-post"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("def-post"), c.Targets[0].Args["v2"])
} }
@@ -334,7 +328,7 @@ func TestHCLVarsWithVars(t *testing.T) {
}, nil) }, nil)
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre--ABCDEF-"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre--ABCDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("ABCDEF-post"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("ABCDEF-post"), c.Targets[0].Args["v2"])
@@ -347,7 +341,7 @@ func TestHCLVarsWithVars(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre--NEWDEF-"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre--NEWDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("NEWDEF-post"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("NEWDEF-post"), c.Targets[0].Args["v2"])
} }
@@ -372,7 +366,7 @@ func TestHCLTypedVariables(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("lower"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("lower"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("yes"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("yes"), c.Targets[0].Args["v2"])
@@ -383,7 +377,7 @@ func TestHCLTypedVariables(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("higher"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("higher"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("no"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("no"), c.Targets[0].Args["v2"])
@@ -481,7 +475,7 @@ func TestHCLAttrs(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// env does not apply if no variable // env does not apply if no variable
@@ -490,7 +484,7 @@ func TestHCLAttrs(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// attr-multifile // attr-multifile
} }
@@ -598,172 +592,11 @@ func TestHCLAttrsCustomType(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, []string{"linux/arm64", "linux/amd64"}, c.Targets[0].Platforms) require.Equal(t, []string{"linux/arm64", "linux/amd64"}, c.Targets[0].Platforms)
require.Equal(t, ptrstr("linux/arm64"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("linux/arm64"), c.Targets[0].Args["v1"])
} }
func TestHCLAttrsCapsuleType(t *testing.T) {
dt := []byte(`
target "app" {
attest = [
{ type = "provenance", mode = "max" },
"type=sbom,disabled=true,generator=foo,\"ENV1=bar,baz\",ENV2=hello",
]
cache-from = [
{ type = "registry", ref = "user/app:cache" },
"type=local,src=path/to/cache",
]
cache-to = [
{ type = "local", dest = "path/to/cache" },
]
output = [
{ type = "oci", dest = "../out.tar" },
"type=local,dest=../out",
]
secret = [
{ id = "mysecret", src = "/local/secret" },
{ id = "mysecret2", env = "TOKEN" },
]
ssh = [
{ id = "default" },
{ id = "key", paths = ["path/to/key"] },
]
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"type=provenance,mode=max", "type=sbom,disabled=true,\"ENV1=bar,baz\",ENV2=hello,generator=foo"}, stringify(c.Targets[0].Attest))
require.Equal(t, []string{"type=local,dest=../out", "type=oci,dest=../out.tar"}, stringify(c.Targets[0].Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom))
require.Equal(t, []string{"type=local,dest=path/to/cache"}, stringify(c.Targets[0].CacheTo))
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,env=TOKEN"}, stringify(c.Targets[0].Secrets))
require.Equal(t, []string{"default", "key=path/to/key"}, stringify(c.Targets[0].SSH))
}
func TestHCLAttrsCapsuleType_ObjectVars(t *testing.T) {
dt := []byte(`
variable "foo" {
default = "bar"
}
target "app" {
cache-from = [
{ type = "registry", ref = "user/app:cache" },
"type=local,src=path/to/cache",
]
cache-to = [ target.app.cache-from[0] ]
output = [
{ type = "oci", dest = "../out.tar" },
"type=local,dest=../out",
]
secret = [
{ id = "mysecret", src = "/local/secret" },
]
ssh = [
{ id = "default" },
{ id = "key", paths = ["path/to/${target.app.output[0].type}"] },
]
}
target "web" {
cache-from = target.app.cache-from
output = [ "type=oci,dest=../${foo}.tar" ]
secret = [
{ id = target.app.output[0].type, src = "/${target.app.cache-from[1].type}/secret" },
]
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
findTarget := func(t *testing.T, name string) *Target {
t.Helper()
for _, tgt := range c.Targets {
if tgt.Name == name {
return tgt
}
}
t.Fatalf("could not find target %q", name)
return nil
}
app := findTarget(t, "app")
require.Equal(t, []string{"type=local,dest=../out", "type=oci,dest=../out.tar"}, stringify(app.Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(app.CacheFrom))
require.Equal(t, []string{"user/app:cache"}, stringify(app.CacheTo))
require.Equal(t, []string{"id=mysecret,src=/local/secret"}, stringify(app.Secrets))
require.Equal(t, []string{"default", "key=path/to/oci"}, stringify(app.SSH))
web := findTarget(t, "web")
require.Equal(t, []string{"type=oci,dest=../bar.tar"}, stringify(web.Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(web.CacheFrom))
require.Equal(t, []string{"id=oci,src=/local/secret"}, stringify(web.Secrets))
}
func TestHCLAttrsCapsuleType_MissingVars(t *testing.T) {
dt := []byte(`
target "app" {
attest = [
"type=sbom,disabled=${SBOM}",
]
cache-from = [
{ type = "registry", ref = "user/app:${FOO1}" },
"type=local,src=path/to/cache:${FOO2}",
]
cache-to = [
{ type = "local", dest = "path/to/${BAR}" },
]
output = [
{ type = "oci", dest = "../${OUTPUT}.tar" },
]
secret = [
{ id = "mysecret", src = "/local/${SECRET}" },
]
ssh = [
{ id = "key", paths = ["path/to/${SSH_KEY}"] },
]
}
`)
var diags hcl.Diagnostics
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorAs(t, err, &diags)
re := regexp.MustCompile(`There is no variable named "([\w\d_]+)"`)
var actual []string
for _, diag := range diags {
if m := re.FindStringSubmatch(diag.Error()); m != nil {
actual = append(actual, m[1])
}
}
require.ElementsMatch(t,
[]string{"SBOM", "FOO1", "FOO2", "BAR", "OUTPUT", "SECRET", "SSH_KEY"},
actual)
}
func TestHCLMultiFileAttrs(t *testing.T) { func TestHCLMultiFileAttrs(t *testing.T) {
dt := []byte(` dt := []byte(`
variable "FOO" { variable "FOO" {
@@ -785,7 +618,7 @@ func TestHCLMultiFileAttrs(t *testing.T) {
}, nil) }, nil)
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
t.Setenv("FOO", "ghi") t.Setenv("FOO", "ghi")
@@ -797,7 +630,7 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"])
} }
@@ -820,7 +653,7 @@ func TestHCLMultiFileGlobalAttrs(t *testing.T) {
}, nil) }, nil)
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", *c.Targets[0].Args["v1"]) require.Equal(t, "pre-def", *c.Targets[0].Args["v1"])
} }
@@ -1006,12 +839,12 @@ func TestHCLRenameMultiFile(t *testing.T) {
require.Equal(t, 2, len(c.Targets)) require.Equal(t, 2, len(c.Targets))
require.Equal(t, "bar", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "bar")
require.Equal(t, "x", *c.Targets[0].Dockerfile) require.Equal(t, *c.Targets[0].Dockerfile, "x")
require.Equal(t, "z", *c.Targets[0].Target) require.Equal(t, *c.Targets[0].Target, "z")
require.Equal(t, "foo", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "foo")
require.Equal(t, "y", *c.Targets[1].Context) require.Equal(t, *c.Targets[1].Context, "y")
} }
func TestHCLMatrixBasic(t *testing.T) { func TestHCLMatrixBasic(t *testing.T) {
@@ -1029,10 +862,10 @@ func TestHCLMatrixBasic(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 2, len(c.Targets)) require.Equal(t, 2, len(c.Targets))
require.Equal(t, "x", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "x")
require.Equal(t, "y", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "y")
require.Equal(t, "x.Dockerfile", *c.Targets[0].Dockerfile) require.Equal(t, *c.Targets[0].Dockerfile, "x.Dockerfile")
require.Equal(t, "y.Dockerfile", *c.Targets[1].Dockerfile) require.Equal(t, *c.Targets[1].Dockerfile, "y.Dockerfile")
require.Equal(t, 1, len(c.Groups)) require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name) require.Equal(t, "default", c.Groups[0].Name)
@@ -1115,9 +948,9 @@ func TestHCLMatrixMaps(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 2, len(c.Targets)) require.Equal(t, 2, len(c.Targets))
require.Equal(t, "aa", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "aa")
require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb")) require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb"))
require.Equal(t, "cc", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "cc")
require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd")) require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd"))
} }
@@ -1308,7 +1141,7 @@ func TestJSONAttributes(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-abc-def"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-abc-def"), c.Targets[0].Args["v1"])
} }
@@ -1333,7 +1166,7 @@ func TestJSONFunctions(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
} }
@@ -1351,7 +1184,7 @@ func TestJSONInvalidFunctions(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
} }
@@ -1379,7 +1212,7 @@ func TestHCLFunctionInAttr(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("FOO <> [baz]"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("FOO <> [baz]"), c.Targets[0].Args["v1"])
} }
@@ -1410,7 +1243,7 @@ services:
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("foo"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("foo"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["v2"]) require.Equal(t, ptrstr("bar"), c.Targets[0].Args["v2"])
require.Equal(t, "dir", *c.Targets[0].Context) require.Equal(t, "dir", *c.Targets[0].Context)
@@ -1433,7 +1266,7 @@ func TestHCLBuiltinVars(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 1, len(c.Targets)) require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "foo", *c.Targets[0].Context) require.Equal(t, "foo", *c.Targets[0].Context)
require.Equal(t, "test", *c.Targets[0].Dockerfile) require.Equal(t, "test", *c.Targets[0].Dockerfile)
} }
@@ -1499,17 +1332,17 @@ target "b" {
require.Equal(t, 4, len(c.Targets)) require.Equal(t, 4, len(c.Targets))
require.Equal(t, "metadata-a", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "metadata-a")
require.Equal(t, []string{"app/a:1.0.0", "app/a:latest"}, c.Targets[0].Tags) require.Equal(t, []string{"app/a:1.0.0", "app/a:latest"}, c.Targets[0].Tags)
require.Equal(t, "metadata-b", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "metadata-b")
require.Equal(t, []string{"app/b:1.0.0", "app/b:latest"}, c.Targets[1].Tags) require.Equal(t, []string{"app/b:1.0.0", "app/b:latest"}, c.Targets[1].Tags)
require.Equal(t, "a", c.Targets[2].Name) require.Equal(t, c.Targets[2].Name, "a")
require.Equal(t, ".", *c.Targets[2].Context) require.Equal(t, ".", *c.Targets[2].Context)
require.Equal(t, "a", *c.Targets[2].Target) require.Equal(t, "a", *c.Targets[2].Target)
require.Equal(t, "b", c.Targets[3].Name) require.Equal(t, c.Targets[3].Name, "b")
require.Equal(t, ".", *c.Targets[3].Context) require.Equal(t, ".", *c.Targets[3].Context)
require.Equal(t, "b", *c.Targets[3].Target) require.Equal(t, "b", *c.Targets[3].Target)
} }
@@ -1556,10 +1389,10 @@ target "two" {
require.Equal(t, 2, len(c.Targets)) require.Equal(t, 2, len(c.Targets))
require.Equal(t, "one", c.Targets[0].Name) require.Equal(t, c.Targets[0].Name, "one")
require.Equal(t, map[string]*string{"a": ptrstr("pre-ghi-jkl")}, c.Targets[0].Args) require.Equal(t, map[string]*string{"a": ptrstr("pre-ghi-jkl")}, c.Targets[0].Args)
require.Equal(t, "two", c.Targets[1].Name) require.Equal(t, c.Targets[1].Name, "two")
require.Equal(t, map[string]*string{"b": ptrstr("pre-jkl")}, c.Targets[1].Args) require.Equal(t, map[string]*string{"b": ptrstr("pre-jkl")}, c.Targets[1].Args)
} }
@@ -1645,7 +1478,7 @@ func TestHCLIndexOfFunc(t *testing.T) {
require.Empty(t, c.Targets[1].Tags[1]) require.Empty(t, c.Targets[1].Tags[1])
} }
func ptrstr(s any) *string { func ptrstr(s interface{}) *string {
var n *string var n *string
if reflect.ValueOf(s).Kind() == reflect.String { if reflect.ValueOf(s).Kind() == reflect.String {
ss := s.(string) ss := s.(string)

View File

@@ -1,355 +0,0 @@
Copyright (c) 2014 HashiCorp, Inc.
Mozilla Public License, version 2.0
1. Definitions
1.1. “Contributor”
means each individual or legal entity that creates, contributes to the
creation of, or owns Covered Software.
1.2. “Contributor Version”
means the combination of the Contributions of others (if any) used by a
Contributor and that particular Contributors Contribution.
1.3. “Contribution”
means Covered Software of a particular Contributor.
1.4. “Covered Software”
means Source Code Form to which the initial Contributor has attached the
notice in Exhibit A, the Executable Form of such Source Code Form, and
Modifications of such Source Code Form, in each case including portions
thereof.
1.5. “Incompatible With Secondary Licenses”
means
a. that the initial Contributor has attached the notice described in
Exhibit B to the Covered Software; or
b. that the Covered Software was made available under the terms of version
1.1 or earlier of the License, but not also under the terms of a
Secondary License.
1.6. “Executable Form”
means any form of the work other than Source Code Form.
1.7. “Larger Work”
means a work that combines Covered Software with other material, in a separate
file or files, that is not Covered Software.
1.8. “License”
means this document.
1.9. “Licensable”
means having the right to grant, to the maximum extent possible, whether at the
time of the initial grant or subsequently, any and all of the rights conveyed by
this License.
1.10. “Modifications”
means any of the following:
a. any file in Source Code Form that results from an addition to, deletion
from, or modification of the contents of Covered Software; or
b. any new file in Source Code Form that contains any Covered Software.
1.11. “Patent Claims” of a Contributor
means any patent claim(s), including without limitation, method, process,
and apparatus claims, in any patent Licensable by such Contributor that
would be infringed, but for the grant of the License, by the making,
using, selling, offering for sale, having made, import, or transfer of
either its Contributions or its Contributor Version.
1.12. “Secondary License”
means either the GNU General Public License, Version 2.0, the GNU Lesser
General Public License, Version 2.1, the GNU Affero General Public
License, Version 3.0, or any later versions of those licenses.
1.13. “Source Code Form”
means the form of the work preferred for making modifications.
1.14. “You” (or “Your”)
means an individual or a legal entity exercising rights under this
License. For legal entities, “You” includes any entity that controls, is
controlled by, or is under common control with You. For purposes of this
definition, “control” means (a) the power, direct or indirect, to cause
the direction or management of such entity, whether by contract or
otherwise, or (b) ownership of more than fifty percent (50%) of the
outstanding shares or beneficial ownership of such entity.
2. License Grants and Conditions
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
a. under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or as
part of a Larger Work; and
b. under Patent Claims of such Contributor to make, use, sell, offer for
sale, have made, import, and otherwise transfer either its Contributions
or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution become
effective for each Contribution on the date the Contributor first distributes
such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under this
License. No additional rights or licenses will be implied from the distribution
or licensing of Covered Software under this License. Notwithstanding Section
2.1(b) above, no patent license is granted by a Contributor:
a. for any code that a Contributor has removed from Covered Software; or
b. for infringements caused by: (i) Your and any other third partys
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
c. under Patent Claims infringed by Covered Software in the absence of its
Contributions.
This License does not grant any rights in the trademarks, service marks, or
logos of any Contributor (except as may be necessary to comply with the
notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this License
(see Section 10.2) or under the terms of a Secondary License (if permitted
under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its Contributions
are its original creation(s) or it has sufficient rights to grant the
rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under applicable
copyright doctrines of fair use, fair dealing, or other equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
Section 2.1.
3. Responsibilities
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under the
terms of this License. You must inform recipients that the Source Code Form
of the Covered Software is governed by the terms of this License, and how
they can obtain a copy of this License. You may not attempt to alter or
restrict the recipients rights in the Source Code Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
a. such Covered Software must also be made available in Source Code Form,
as described in Section 3.1, and You must inform recipients of the
Executable Form how they can obtain a copy of such Source Code Form by
reasonable means in a timely manner, at a charge no more than the cost
of distribution to the recipient; and
b. You may distribute such Executable Form under the terms of this License,
or sublicense it under different terms, provided that the license for
the Executable Form does not attempt to limit or alter the recipients
rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for the
Covered Software. If the Larger Work is a combination of Covered Software
with a work governed by one or more Secondary Licenses, and the Covered
Software is not Incompatible With Secondary Licenses, this License permits
You to additionally distribute such Covered Software under the terms of
such Secondary License(s), so that the recipient of the Larger Work may, at
their option, further distribute the Covered Software under the terms of
either this License or such Secondary License(s).
3.4. Notices
You may not remove or alter the substance of any license notices (including
copyright notices, patent notices, disclaimers of warranty, or limitations
of liability) contained within the Source Code Form of the Covered
Software, except that You may alter any license notices to the extent
required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on behalf
of any Contributor. You must make it absolutely clear that any such
warranty, support, indemnity, or liability obligation is offered by You
alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
If it is impossible for You to comply with any of the terms of this License
with respect to some or all of the Covered Software due to statute, judicial
order, or regulation then You must: (a) comply with the terms of this License
to the maximum extent possible; and (b) describe the limitations and the code
they affect. Such description must be placed in a text file included with all
distributions of the Covered Software under this License. Except to the
extent prohibited by statute or regulation, such description must be
sufficiently detailed for a recipient of ordinary skill to be able to
understand it.
5. Termination
5.1. The rights granted under this License will terminate automatically if You
fail to comply with any of its terms. However, if You become compliant,
then the rights granted under this License from a particular Contributor
are reinstated (a) provisionally, unless and until such Contributor
explicitly and finally terminates Your grants, and (b) on an ongoing basis,
if such Contributor fails to notify You of the non-compliance by some
reasonable means prior to 60 days after You have come back into compliance.
Moreover, Your grants from a particular Contributor are reinstated on an
ongoing basis if such Contributor notifies You of the non-compliance by
some reasonable means, this is the first time You have received notice of
non-compliance with this License from such Contributor, and You become
compliant prior to 30 days after Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions, counter-claims,
and cross-claims) alleging that a Contributor Version directly or
indirectly infringes any patent, then the rights granted to You by any and
all Contributors for the Covered Software under Section 2.1 of this License
shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
license agreements (excluding distributors and resellers) which have been
validly granted by You or Your distributors under this License prior to
termination shall survive termination.
6. Disclaimer of Warranty
Covered Software is provided under this License on an “as is” basis, without
warranty of any kind, either expressed, implied, or statutory, including,
without limitation, warranties that the Covered Software is free of defects,
merchantable, fit for a particular purpose or non-infringing. The entire
risk as to the quality and performance of the Covered Software is with You.
Should any Covered Software prove defective in any respect, You (not any
Contributor) assume the cost of any necessary servicing, repair, or
correction. This disclaimer of warranty constitutes an essential part of this
License. No use of any Covered Software is authorized under this License
except under this disclaimer.
7. Limitation of Liability
Under no circumstances and under no legal theory, whether tort (including
negligence), contract, or otherwise, shall any Contributor, or anyone who
distributes Covered Software as permitted above, be liable to You for any
direct, indirect, special, incidental, or consequential damages of any
character including, without limitation, damages for lost profits, loss of
goodwill, work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses, even if such party shall have been
informed of the possibility of such damages. This limitation of liability
shall not apply to liability for death or personal injury resulting from such
partys negligence to the extent applicable law prohibits such limitation.
Some jurisdictions do not allow the exclusion or limitation of incidental or
consequential damages, so this exclusion and limitation may not apply to You.
8. Litigation
Any litigation relating to this License may be brought only in the courts of
a jurisdiction where the defendant maintains its principal place of business
and such litigation shall be governed by laws of that jurisdiction, without
reference to its conflict-of-law provisions. Nothing in this Section shall
prevent a partys ability to bring cross-claims or counter-claims.
9. Miscellaneous
This License represents the complete agreement concerning the subject matter
hereof. If any provision of this License is held to be unenforceable, such
provision shall be reformed only to the extent necessary to make it
enforceable. Any law or regulation which provides that the language of a
contract shall be construed against the drafter shall not be used to construe
this License against a Contributor.
10. Versions of the License
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version of
the License under which You originally received the Covered Software, or
under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a modified
version of this License if you rename the license and remove any
references to the name of the license steward (except to note that such
modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular file, then
You may include the notice in a location (such as a LICENSE file in a relevant
directory) where a recipient would be likely to look for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - “Incompatible With Secondary Licenses” Notice
This Source Code Form is “Incompatible
With Secondary Licenses”, as defined by
the Mozilla Public License, v. 2.0.

View File

@@ -1,348 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"github.com/hashicorp/hcl/v2"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/gocty"
)
// DecodeOptions allows customizing sections of the decoding process.
type DecodeOptions struct {
ImpliedType func(gv any) (cty.Type, error)
Convert func(in cty.Value, want cty.Type) (cty.Value, error)
}
func (o DecodeOptions) DecodeBody(body hcl.Body, ctx *hcl.EvalContext, val any) hcl.Diagnostics {
o = o.withDefaults()
rv := reflect.ValueOf(val)
if rv.Kind() != reflect.Ptr {
panic(fmt.Sprintf("target value must be a pointer, not %s", rv.Type().String()))
}
return o.decodeBodyToValue(body, ctx, rv.Elem())
}
// DecodeBody extracts the configuration within the given body into the given
// value. This value must be a non-nil pointer to either a struct or
// a map, where in the former case the configuration will be decoded using
// struct tags and in the latter case only attributes are allowed and their
// values are decoded into the map.
//
// The given EvalContext is used to resolve any variables or functions in
// expressions encountered while decoding. This may be nil to require only
// constant values, for simple applications that do not support variables or
// functions.
//
// The returned diagnostics should be inspected with its HasErrors method to
// determine if the populated value is valid and complete. If error diagnostics
// are returned then the given value may have been partially-populated but
// may still be accessed by a careful caller for static analysis and editor
// integration use-cases.
func DecodeBody(body hcl.Body, ctx *hcl.EvalContext, val any) hcl.Diagnostics {
return DecodeOptions{}.DecodeBody(body, ctx, val)
}
func (o DecodeOptions) decodeBodyToValue(body hcl.Body, ctx *hcl.EvalContext, val reflect.Value) hcl.Diagnostics {
et := val.Type()
switch et.Kind() {
case reflect.Struct:
return o.decodeBodyToStruct(body, ctx, val)
case reflect.Map:
return o.decodeBodyToMap(body, ctx, val)
default:
panic(fmt.Sprintf("target value must be pointer to struct or map, not %s", et.String()))
}
}
func (o DecodeOptions) decodeBodyToStruct(body hcl.Body, ctx *hcl.EvalContext, val reflect.Value) hcl.Diagnostics {
schema, partial := ImpliedBodySchema(val.Interface())
var content *hcl.BodyContent
var leftovers hcl.Body
var diags hcl.Diagnostics
if partial {
content, leftovers, diags = body.PartialContent(schema)
} else {
content, diags = body.Content(schema)
}
if content == nil {
return diags
}
tags := getFieldTags(val.Type())
if tags.Body != nil {
fieldIdx := *tags.Body
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
switch {
case bodyType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(body))
default:
diags = append(diags, o.decodeBodyToValue(body, ctx, fieldV)...)
}
}
if tags.Remain != nil {
fieldIdx := *tags.Remain
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
switch {
case bodyType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(leftovers))
case attrsType.AssignableTo(field.Type):
attrs, attrsDiags := leftovers.JustAttributes()
if len(attrsDiags) > 0 {
diags = append(diags, attrsDiags...)
}
fieldV.Set(reflect.ValueOf(attrs))
default:
diags = append(diags, o.decodeBodyToValue(leftovers, ctx, fieldV)...)
}
}
for name, fieldIdx := range tags.Attributes {
attr := content.Attributes[name]
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
if attr == nil {
if !exprType.AssignableTo(field.Type) {
continue
}
// As a special case, if the target is of type hcl.Expression then
// we'll assign an actual expression that evalues to a cty null,
// so the caller can deal with it within the cty realm rather
// than within the Go realm.
synthExpr := hcl.StaticExpr(cty.NullVal(cty.DynamicPseudoType), body.MissingItemRange())
fieldV.Set(reflect.ValueOf(synthExpr))
continue
}
switch {
case attrType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(attr))
case exprType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(attr.Expr))
default:
diags = append(diags, o.DecodeExpression(
attr.Expr, ctx, fieldV.Addr().Interface(),
)...)
}
}
blocksByType := content.Blocks.ByType()
for typeName, fieldIdx := range tags.Blocks {
blocks := blocksByType[typeName]
field := val.Type().Field(fieldIdx)
ty := field.Type
isSlice := false
isPtr := false
if ty.Kind() == reflect.Slice {
isSlice = true
ty = ty.Elem()
}
if ty.Kind() == reflect.Ptr {
isPtr = true
ty = ty.Elem()
}
if len(blocks) > 1 && !isSlice {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Duplicate %s block", typeName),
Detail: fmt.Sprintf(
"Only one %s block is allowed. Another was defined at %s.",
typeName, blocks[0].DefRange.String(),
),
Subject: &blocks[1].DefRange,
})
continue
}
if len(blocks) == 0 {
if isSlice || isPtr {
if val.Field(fieldIdx).IsNil() {
val.Field(fieldIdx).Set(reflect.Zero(field.Type))
}
} else {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Missing %s block", typeName),
Detail: fmt.Sprintf("A %s block is required.", typeName),
Subject: body.MissingItemRange().Ptr(),
})
}
continue
}
switch {
case isSlice:
elemType := ty
if isPtr {
elemType = reflect.PointerTo(ty)
}
sli := val.Field(fieldIdx)
if sli.IsNil() {
sli = reflect.MakeSlice(reflect.SliceOf(elemType), len(blocks), len(blocks))
}
for i, block := range blocks {
if isPtr {
if i >= sli.Len() {
sli = reflect.Append(sli, reflect.New(ty))
}
v := sli.Index(i)
if v.IsNil() {
v = reflect.New(ty)
}
diags = append(diags, o.decodeBlockToValue(block, ctx, v.Elem())...)
sli.Index(i).Set(v)
} else {
if i >= sli.Len() {
sli = reflect.Append(sli, reflect.Indirect(reflect.New(ty)))
}
diags = append(diags, o.decodeBlockToValue(block, ctx, sli.Index(i))...)
}
}
if sli.Len() > len(blocks) {
sli.SetLen(len(blocks))
}
val.Field(fieldIdx).Set(sli)
default:
block := blocks[0]
if isPtr {
v := val.Field(fieldIdx)
if v.IsNil() {
v = reflect.New(ty)
}
diags = append(diags, o.decodeBlockToValue(block, ctx, v.Elem())...)
val.Field(fieldIdx).Set(v)
} else {
diags = append(diags, o.decodeBlockToValue(block, ctx, val.Field(fieldIdx))...)
}
}
}
return diags
}
func (o DecodeOptions) decodeBodyToMap(body hcl.Body, ctx *hcl.EvalContext, v reflect.Value) hcl.Diagnostics {
attrs, diags := body.JustAttributes()
if attrs == nil {
return diags
}
mv := reflect.MakeMap(v.Type())
for k, attr := range attrs {
switch {
case attrType.AssignableTo(v.Type().Elem()):
mv.SetMapIndex(reflect.ValueOf(k), reflect.ValueOf(attr))
case exprType.AssignableTo(v.Type().Elem()):
mv.SetMapIndex(reflect.ValueOf(k), reflect.ValueOf(attr.Expr))
default:
ev := reflect.New(v.Type().Elem())
diags = append(diags, o.DecodeExpression(attr.Expr, ctx, ev.Interface())...)
mv.SetMapIndex(reflect.ValueOf(k), ev.Elem())
}
}
v.Set(mv)
return diags
}
func (o DecodeOptions) decodeBlockToValue(block *hcl.Block, ctx *hcl.EvalContext, v reflect.Value) hcl.Diagnostics {
diags := o.decodeBodyToValue(block.Body, ctx, v)
if len(block.Labels) > 0 {
blockTags := getFieldTags(v.Type())
for li, lv := range block.Labels {
lfieldIdx := blockTags.Labels[li].FieldIndex
v.Field(lfieldIdx).Set(reflect.ValueOf(lv))
}
}
return diags
}
func (o DecodeOptions) DecodeExpression(expr hcl.Expression, ctx *hcl.EvalContext, val any) hcl.Diagnostics {
o = o.withDefaults()
srcVal, diags := expr.Value(ctx)
convTy, err := o.ImpliedType(val)
if err != nil {
panic(fmt.Sprintf("unsuitable DecodeExpression target: %s", err))
}
srcVal, err = o.Convert(srcVal, convTy)
if err != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Unsuitable value type",
Detail: fmt.Sprintf("Unsuitable value: %s", err.Error()),
Subject: expr.StartRange().Ptr(),
Context: expr.Range().Ptr(),
})
return diags
}
err = gocty.FromCtyValue(srcVal, val)
if err != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Unsuitable value type",
Detail: fmt.Sprintf("Unsuitable value: %s", err.Error()),
Subject: expr.StartRange().Ptr(),
Context: expr.Range().Ptr(),
})
}
return diags
}
// DecodeExpression extracts the value of the given expression into the given
// value. This value must be something that gocty is able to decode into,
// since the final decoding is delegated to that package.
//
// The given EvalContext is used to resolve any variables or functions in
// expressions encountered while decoding. This may be nil to require only
// constant values, for simple applications that do not support variables or
// functions.
//
// The returned diagnostics should be inspected with its HasErrors method to
// determine if the populated value is valid and complete. If error diagnostics
// are returned then the given value may have been partially-populated but
// may still be accessed by a careful caller for static analysis and editor
// integration use-cases.
func DecodeExpression(expr hcl.Expression, ctx *hcl.EvalContext, val any) hcl.Diagnostics {
return DecodeOptions{}.DecodeExpression(expr, ctx, val)
}
func (o DecodeOptions) withDefaults() DecodeOptions {
if o.ImpliedType == nil {
o.ImpliedType = gocty.ImpliedType
}
if o.Convert == nil {
o.Convert = convert.Convert
}
return o
}

View File

@@ -1,806 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"encoding/json"
"fmt"
"reflect"
"testing"
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/hcl/v2"
hclJSON "github.com/hashicorp/hcl/v2/json"
"github.com/zclconf/go-cty/cty"
)
func TestDecodeBody(t *testing.T) {
deepEquals := func(other any) func(v any) bool {
return func(v any) bool {
return reflect.DeepEqual(v, other)
}
}
type withNameExpression struct {
Name hcl.Expression `hcl:"name"`
}
type withTwoAttributes struct {
A string `hcl:"a,optional"`
B string `hcl:"b,optional"`
}
type withNestedBlock struct {
Plain string `hcl:"plain,optional"`
Nested *withTwoAttributes `hcl:"nested,block"`
}
type withListofNestedBlocks struct {
Nested []*withTwoAttributes `hcl:"nested,block"`
}
type withListofNestedBlocksNoPointers struct {
Nested []withTwoAttributes `hcl:"nested,block"`
}
tests := []struct {
Body map[string]any
Target func() any
Check func(v any) bool
DiagCount int
}{
{
map[string]any{},
makeInstantiateType(struct{}{}),
deepEquals(struct{}{}),
0,
},
{
map[string]any{},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{}),
1, // name is required
},
{
map[string]any{},
makeInstantiateType(struct {
Name *string `hcl:"name"`
}{}),
deepEquals(struct {
Name *string `hcl:"name"`
}{}),
0,
}, // name nil
{
map[string]any{},
makeInstantiateType(struct {
Name string `hcl:"name,optional"`
}{}),
deepEquals(struct {
Name string `hcl:"name,optional"`
}{}),
0,
}, // name optional
{
map[string]any{},
makeInstantiateType(withNameExpression{}),
func(v any) bool {
if v == nil {
return false
}
wne, valid := v.(withNameExpression)
if !valid {
return false
}
if wne.Name == nil {
return false
}
nameVal, _ := wne.Name.Value(nil)
return nameVal.IsNull()
},
0,
},
{
map[string]any{
"name": "Ermintrude",
},
makeInstantiateType(withNameExpression{}),
func(v any) bool {
if v == nil {
return false
}
wne, valid := v.(withNameExpression)
if !valid {
return false
}
if wne.Name == nil {
return false
}
nameVal, _ := wne.Name.Value(nil)
return nameVal.Equals(cty.StringVal("Ermintrude")).True()
},
0,
},
{
map[string]any{
"name": "Ermintrude",
},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{"Ermintrude"}),
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 23,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{"Ermintrude"}),
1, // Extraneous "age" property
},
{
map[string]any{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Attrs hcl.Attributes `hcl:",remain"`
}{}),
func(gotI any) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Attrs hcl.Attributes `hcl:",remain"`
})
return got.Name == "Ermintrude" && len(got.Attrs) == 1 && got.Attrs["age"] != nil
},
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Remain hcl.Body `hcl:",remain"`
}{}),
func(gotI any) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Remain hcl.Body `hcl:",remain"`
})
attrs, _ := got.Remain.JustAttributes()
return got.Name == "Ermintrude" && len(attrs) == 1 && attrs["age"] != nil
},
0,
},
{
map[string]any{
"name": "Ermintrude",
"living": true,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Remain map[string]cty.Value `hcl:",remain"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
Remain map[string]cty.Value `hcl:",remain"`
}{
Name: "Ermintrude",
Remain: map[string]cty.Value{
"living": cty.True,
},
}),
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Body hcl.Body `hcl:",body"`
Remain hcl.Body `hcl:",remain"`
}{}),
func(gotI any) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Body hcl.Body `hcl:",body"`
Remain hcl.Body `hcl:",remain"`
})
attrs, _ := got.Body.JustAttributes()
return got.Name == "Ermintrude" && len(attrs) == 2 &&
attrs["name"] != nil && attrs["age"] != nil
},
0,
},
{
map[string]any{
"noodle": map[string]any{},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating no diagnostics is good enough for this one.
return true
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating no diagnostics is good enough for this one.
return true
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}, {}},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]any{},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]any{
"noodle": []map[string]any{},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]any{
"noodle": map[string]any{},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle != nil
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle != nil
},
0,
},
{
map[string]any{
"noodle": []map[string]any{},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle == nil
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}, {}},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]any{
"noodle": []map[string]any{},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 0
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 1
},
0,
},
{
map[string]any{
"noodle": []map[string]any{{}, {}},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 2
},
0,
},
{
map[string]any{
"noodle": map[string]any{},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
//nolint:misspell
// Generating two diagnostics is good enough for this one.
// (one for the missing noodle block and the other for
// the JSON serialization detecting the missing level of
// heirarchy for the label.)
return true
},
2,
},
{
map[string]any{
"noodle": map[string]any{
"foo_foo": map[string]any{},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodle := gotI.(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}).Noodle
return noodle.Name == "foo_foo"
},
0,
},
{
map[string]any{
"noodle": map[string]any{
"foo_foo": map[string]any{},
"bar_baz": map[string]any{},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
// One diagnostic is enough for this one.
return true
},
1,
},
{
map[string]any{
"noodle": map[string]any{
"foo_foo": map[string]any{},
"bar_baz": map[string]any{},
},
},
makeInstantiateType(struct {
Noodles []struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodles := gotI.(struct {
Noodles []struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}).Noodles
return len(noodles) == 2 && (noodles[0].Name == "foo_foo" || noodles[0].Name == "bar_baz") && (noodles[1].Name == "foo_foo" || noodles[1].Name == "bar_baz") && noodles[0].Name != noodles[1].Name
},
0,
},
{
map[string]any{
"noodle": map[string]any{
"foo_foo": map[string]any{
"type": "rice",
},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
Type string `hcl:"type"`
} `hcl:"noodle,block"`
}{}),
func(gotI any) bool {
noodle := gotI.(struct {
Noodle struct {
Name string `hcl:"name,label"`
Type string `hcl:"type"`
} `hcl:"noodle,block"`
}).Noodle
return noodle.Name == "foo_foo" && noodle.Type == "rice"
},
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 34,
},
makeInstantiateType(map[string]string(nil)),
deepEquals(map[string]string{
"name": "Ermintrude",
"age": "34",
}),
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 89,
},
makeInstantiateType(map[string]*hcl.Attribute(nil)),
func(gotI any) bool {
got := gotI.(map[string]*hcl.Attribute)
return len(got) == 2 && got["name"] != nil && got["age"] != nil
},
0,
},
{
map[string]any{
"name": "Ermintrude",
"age": 13,
},
makeInstantiateType(map[string]hcl.Expression(nil)),
func(gotI any) bool {
got := gotI.(map[string]hcl.Expression)
return len(got) == 2 && got["name"] != nil && got["age"] != nil
},
0,
},
{
map[string]any{
"name": "Ermintrude",
"living": true,
},
makeInstantiateType(map[string]cty.Value(nil)),
deepEquals(map[string]cty.Value{
"name": cty.StringVal("Ermintrude"),
"living": cty.True,
}),
0,
},
{
// Retain "nested" block while decoding
map[string]any{
"plain": "foo",
},
func() any {
return &withNestedBlock{
Plain: "bar",
Nested: &withTwoAttributes{
A: "bar",
},
}
},
func(gotI any) bool {
foo := gotI.(withNestedBlock)
return foo.Plain == "foo" && foo.Nested != nil && foo.Nested.A == "bar"
},
0,
},
{
// Retain values in "nested" block while decoding
map[string]any{
"nested": map[string]any{
"a": "foo",
},
},
func() any {
return &withNestedBlock{
Nested: &withTwoAttributes{
B: "bar",
},
}
},
func(gotI any) bool {
foo := gotI.(withNestedBlock)
return foo.Nested.A == "foo" && foo.Nested.B == "bar"
},
0,
},
{
// Retain values in "nested" block list while decoding
map[string]any{
"nested": []map[string]any{
{
"a": "foo",
},
},
},
func() any {
return &withListofNestedBlocks{
Nested: []*withTwoAttributes{
{
B: "bar",
},
},
}
},
func(gotI any) bool {
n := gotI.(withListofNestedBlocks)
return n.Nested[0].A == "foo" && n.Nested[0].B == "bar"
},
0,
},
{
// Remove additional elements from the list while decoding nested blocks
map[string]any{
"nested": []map[string]any{
{
"a": "foo",
},
},
},
func() any {
return &withListofNestedBlocks{
Nested: []*withTwoAttributes{
{
B: "bar",
},
{
B: "bar",
},
},
}
},
func(gotI any) bool {
n := gotI.(withListofNestedBlocks)
return len(n.Nested) == 1
},
0,
},
{
// Make sure decoding value slices works the same as pointer slices.
map[string]any{
"nested": []map[string]any{
{
"b": "bar",
},
{
"b": "baz",
},
},
},
func() any {
return &withListofNestedBlocksNoPointers{
Nested: []withTwoAttributes{
{
B: "foo",
},
},
}
},
func(gotI any) bool {
n := gotI.(withListofNestedBlocksNoPointers)
return n.Nested[0].B == "bar" && len(n.Nested) == 2
},
0,
},
}
for i, test := range tests {
// For convenience here we're going to use the JSON parser
// to process the given body.
buf, err := json.Marshal(test.Body)
if err != nil {
t.Fatalf("error JSON-encoding body for test %d: %s", i, err)
}
t.Run(string(buf), func(t *testing.T) {
file, diags := hclJSON.Parse(buf, "test.json")
if len(diags) != 0 {
t.Fatalf("diagnostics while parsing: %s", diags.Error())
}
targetVal := reflect.ValueOf(test.Target())
diags = DecodeBody(file.Body, nil, targetVal.Interface())
if len(diags) != test.DiagCount {
t.Errorf("wrong number of diagnostics %d; want %d", len(diags), test.DiagCount)
for _, diag := range diags {
t.Logf(" - %s", diag.Error())
}
}
got := targetVal.Elem().Interface()
if !test.Check(got) {
t.Errorf("wrong result\ngot: %s", spew.Sdump(got))
}
})
}
}
func TestDecodeExpression(t *testing.T) {
tests := []struct {
Value cty.Value
Target any
Want any
DiagCount int
}{
{
cty.StringVal("hello"),
"",
"hello",
0,
},
{
cty.StringVal("hello"),
cty.NilVal,
cty.StringVal("hello"),
0,
},
{
cty.NumberIntVal(2),
"",
"2",
0,
},
{
cty.StringVal("true"),
false,
true,
0,
},
{
cty.NullVal(cty.String),
"",
"",
1, // null value is not allowed
},
{
cty.UnknownVal(cty.String),
"",
"",
1, // value must be known
},
{
cty.ListVal([]cty.Value{cty.True}),
false,
false,
1, // bool required
},
}
for i, test := range tests {
t.Run(fmt.Sprintf("%02d", i), func(t *testing.T) {
expr := &fixedExpression{test.Value}
targetVal := reflect.New(reflect.TypeOf(test.Target))
diags := DecodeExpression(expr, nil, targetVal.Interface())
if len(diags) != test.DiagCount {
t.Errorf("wrong number of diagnostics %d; want %d", len(diags), test.DiagCount)
for _, diag := range diags {
t.Logf(" - %s", diag.Error())
}
}
got := targetVal.Elem().Interface()
if !reflect.DeepEqual(got, test.Want) {
t.Errorf("wrong result\ngot: %#v\nwant: %#v", got, test.Want)
}
})
}
}
type fixedExpression struct {
val cty.Value
}
func (e *fixedExpression) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
return e.val, nil
}
func (e *fixedExpression) Range() (r hcl.Range) {
return
}
func (e *fixedExpression) StartRange() (r hcl.Range) {
return
}
func (e *fixedExpression) Variables() []hcl.Traversal {
return nil
}
func makeInstantiateType(target any) func() any {
return func() any {
return reflect.New(reflect.TypeOf(target)).Interface()
}
}

View File

@@ -1,65 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
// Package gohcl allows decoding HCL configurations into Go data structures.
//
// It provides a convenient and concise way of describing the schema for
// configuration and then accessing the resulting data via native Go
// types.
//
// A struct field tag scheme is used, similar to other decoding and
// unmarshalling libraries. The tags are formatted as in the following example:
//
// ThingType string `hcl:"thing_type,attr"`
//
// Within each tag there are two comma-separated tokens. The first is the
// name of the corresponding construct in configuration, while the second
// is a keyword giving the kind of construct expected. The following
// kind keywords are supported:
//
// attr (the default) indicates that the value is to be populated from an attribute
// block indicates that the value is to populated from a block
// label indicates that the value is to populated from a block label
// optional is the same as attr, but the field is optional
// remain indicates that the value is to be populated from the remaining body after populating other fields
//
// "attr" fields may either be of type *hcl.Expression, in which case the raw
// expression is assigned, or of any type accepted by gocty, in which case
// gocty will be used to assign the value to a native Go type.
//
// "block" fields may be a struct that recursively uses the same tags, or a
// slice of such structs, in which case multiple blocks of the corresponding
// type are decoded into the slice.
//
// "body" can be placed on a single field of type hcl.Body to capture
// the full hcl.Body that was decoded for a block. This does not allow leftover
// values like "remain", so a decoding error will still be returned if leftover
// fields are given. If you want to capture the decoding body PLUS leftover
// fields, you must specify a "remain" field as well to prevent errors. The
// body field and the remain field will both contain the leftover fields.
//
// "label" fields are considered only in a struct used as the type of a field
// marked as "block", and are used sequentially to capture the labels of
// the blocks being decoded. In this case, the name token is used only as
// an identifier for the label in diagnostic messages.
//
// "optional" fields behave like "attr" fields, but they are optional
// and will not give parsing errors if they are missing.
//
// "remain" can be placed on a single field that may be either of type
// hcl.Body or hcl.Attributes, in which case any remaining body content is
// placed into this field for delayed processing. If no "remain" field is
// present then any attributes or blocks not matched by another valid tag
// will cause an error diagnostic.
//
// Only a subset of this tagging/typing vocabulary is supported for the
// "Encode" family of functions. See the EncodeIntoBody docs for full details
// on the constraints there.
//
// Broadly-speaking this package deals with two types of error. The first is
// errors in the configuration itself, which are returned as diagnostics
// written with the configuration author as the target audience. The second
// is bugs in the calling program, such as invalid struct tags, which are
// surfaced via panics since there can be no useful runtime handling of such
// errors and they should certainly not be returned to the user as diagnostics.
package gohcl

View File

@@ -1,192 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"sort"
"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/zclconf/go-cty/cty/gocty"
)
// EncodeIntoBody replaces the contents of the given hclwrite Body with
// attributes and blocks derived from the given value, which must be a
// struct value or a pointer to a struct value with the struct tags defined
// in this package.
//
// This function can work only with fully-decoded data. It will ignore any
// fields tagged as "remain", any fields that decode attributes into either
// hcl.Attribute or hcl.Expression values, and any fields that decode blocks
// into hcl.Attributes values. This function does not have enough information
// to complete the decoding of these types.
//
// Any fields tagged as "label" are ignored by this function. Use EncodeAsBlock
// to produce a whole hclwrite.Block including block labels.
//
// As long as a suitable value is given to encode and the destination body
// is non-nil, this function will always complete. It will panic in case of
// any errors in the calling program, such as passing an inappropriate type
// or a nil body.
//
// The layout of the resulting HCL source is derived from the ordering of
// the struct fields, with blank lines around nested blocks of different types.
// Fields representing attributes should usually precede those representing
// blocks so that the attributes can group together in the result. For more
// control, use the hclwrite API directly.
func EncodeIntoBody(val any, dst *hclwrite.Body) {
rv := reflect.ValueOf(val)
ty := rv.Type()
if ty.Kind() == reflect.Ptr {
rv = rv.Elem()
ty = rv.Type()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("value is %s, not struct", ty.Kind()))
}
tags := getFieldTags(ty)
populateBody(rv, ty, tags, dst)
}
// EncodeAsBlock creates a new hclwrite.Block populated with the data from
// the given value, which must be a struct or pointer to struct with the
// struct tags defined in this package.
//
// If the given struct type has fields tagged with "label" tags then they
// will be used in order to annotate the created block with labels.
//
// This function has the same constraints as EncodeIntoBody and will panic
// if they are violated.
func EncodeAsBlock(val any, blockType string) *hclwrite.Block {
rv := reflect.ValueOf(val)
ty := rv.Type()
if ty.Kind() == reflect.Ptr {
rv = rv.Elem()
ty = rv.Type()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("value is %s, not struct", ty.Kind()))
}
tags := getFieldTags(ty)
labels := make([]string, len(tags.Labels))
for i, lf := range tags.Labels {
lv := rv.Field(lf.FieldIndex)
// We just stringify whatever we find. It should always be a string
// but if not then we'll still do something reasonable.
labels[i] = fmt.Sprintf("%s", lv.Interface())
}
block := hclwrite.NewBlock(blockType, labels)
populateBody(rv, ty, tags, block.Body())
return block
}
func populateBody(rv reflect.Value, ty reflect.Type, tags *fieldTags, dst *hclwrite.Body) {
nameIdxs := make(map[string]int, len(tags.Attributes)+len(tags.Blocks))
namesOrder := make([]string, 0, len(tags.Attributes)+len(tags.Blocks))
for n, i := range tags.Attributes {
nameIdxs[n] = i
namesOrder = append(namesOrder, n)
}
for n, i := range tags.Blocks {
nameIdxs[n] = i
namesOrder = append(namesOrder, n)
}
sort.SliceStable(namesOrder, func(i, j int) bool {
ni, nj := namesOrder[i], namesOrder[j]
return nameIdxs[ni] < nameIdxs[nj]
})
dst.Clear()
prevWasBlock := false
for _, name := range namesOrder {
fieldIdx := nameIdxs[name]
field := ty.Field(fieldIdx)
fieldTy := field.Type
fieldVal := rv.Field(fieldIdx)
if fieldTy.Kind() == reflect.Ptr {
fieldTy = fieldTy.Elem()
fieldVal = fieldVal.Elem()
}
if _, isAttr := tags.Attributes[name]; isAttr {
if exprType.AssignableTo(fieldTy) || attrType.AssignableTo(fieldTy) {
continue // ignore undecoded fields
}
if !fieldVal.IsValid() {
continue // ignore (field value is nil pointer)
}
if fieldTy.Kind() == reflect.Ptr && fieldVal.IsNil() {
continue // ignore
}
if prevWasBlock {
dst.AppendNewline()
prevWasBlock = false
}
valTy, err := gocty.ImpliedType(fieldVal.Interface())
if err != nil {
panic(fmt.Sprintf("cannot encode %T as HCL expression: %s", fieldVal.Interface(), err))
}
val, err := gocty.ToCtyValue(fieldVal.Interface(), valTy)
if err != nil {
// This should never happen, since we should always be able
// to decode into the implied type.
panic(fmt.Sprintf("failed to encode %T as %#v: %s", fieldVal.Interface(), valTy, err))
}
dst.SetAttributeValue(name, val)
} else { // must be a block, then
elemTy := fieldTy
isSeq := false
if elemTy.Kind() == reflect.Slice || elemTy.Kind() == reflect.Array {
isSeq = true
elemTy = elemTy.Elem()
}
if bodyType.AssignableTo(elemTy) || attrsType.AssignableTo(elemTy) {
continue // ignore undecoded fields
}
prevWasBlock = false
if isSeq {
l := fieldVal.Len()
for i := range l {
elemVal := fieldVal.Index(i)
if !elemVal.IsValid() {
continue // ignore (elem value is nil pointer)
}
if elemTy.Kind() == reflect.Ptr && elemVal.IsNil() {
continue // ignore
}
block := EncodeAsBlock(elemVal.Interface(), name)
if !prevWasBlock {
dst.AppendNewline()
prevWasBlock = true
}
dst.AppendBlock(block)
}
} else {
if !fieldVal.IsValid() {
continue // ignore (field value is nil pointer)
}
if elemTy.Kind() == reflect.Ptr && fieldVal.IsNil() {
continue // ignore
}
block := EncodeAsBlock(fieldVal.Interface(), name)
if !prevWasBlock {
dst.AppendNewline()
prevWasBlock = true
}
dst.AppendBlock(block)
}
}
}
}

View File

@@ -1,67 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl_test
import (
"fmt"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclwrite"
)
func ExampleEncodeIntoBody() {
type Service struct {
Name string `hcl:"name,label"`
Exe []string `hcl:"executable"`
}
type Constraints struct {
OS string `hcl:"os"`
Arch string `hcl:"arch"`
}
type App struct {
Name string `hcl:"name"`
Desc string `hcl:"description"`
Constraints *Constraints `hcl:"constraints,block"`
Services []Service `hcl:"service,block"`
}
app := App{
Name: "awesome-app",
Desc: "Such an awesome application",
Constraints: &Constraints{
OS: "linux",
Arch: "amd64",
},
Services: []Service{
{
Name: "web",
Exe: []string{"./web", "--listen=:8080"},
},
{
Name: "worker",
Exe: []string{"./worker"},
},
},
}
f := hclwrite.NewEmptyFile()
gohcl.EncodeIntoBody(&app, f.Body())
fmt.Printf("%s", f.Bytes())
// Output:
// name = "awesome-app"
// description = "Such an awesome application"
//
// constraints {
// os = "linux"
// arch = "amd64"
// }
//
// service "web" {
// executable = ["./web", "--listen=:8080"]
// }
// service "worker" {
// executable = ["./worker"]
// }
}

View File

@@ -1,185 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"sort"
"strings"
"github.com/hashicorp/hcl/v2"
)
// ImpliedBodySchema produces a hcl.BodySchema derived from the type of the
// given value, which must be a struct value or a pointer to one. If an
// inappropriate value is passed, this function will panic.
//
// The second return argument indicates whether the given struct includes
// a "remain" field, and thus the returned schema is non-exhaustive.
//
// This uses the tags on the fields of the struct to discover how each
// field's value should be expressed within configuration. If an invalid
// mapping is attempted, this function will panic.
func ImpliedBodySchema(val any) (schema *hcl.BodySchema, partial bool) {
ty := reflect.TypeOf(val)
if ty.Kind() == reflect.Ptr {
ty = ty.Elem()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("given value must be struct, not %T", val))
}
var attrSchemas []hcl.AttributeSchema
var blockSchemas []hcl.BlockHeaderSchema
tags := getFieldTags(ty)
attrNames := make([]string, 0, len(tags.Attributes))
for n := range tags.Attributes {
attrNames = append(attrNames, n)
}
sort.Strings(attrNames)
for _, n := range attrNames {
idx := tags.Attributes[n]
optional := tags.Optional[n]
field := ty.Field(idx)
var required bool
switch {
case field.Type.AssignableTo(exprType):
//nolint:misspell
// If we're decoding to hcl.Expression then absense can be
// indicated via a null value, so we don't specify that
// the field is required during decoding.
required = false
case field.Type.Kind() != reflect.Ptr && !optional:
required = true
default:
required = false
}
attrSchemas = append(attrSchemas, hcl.AttributeSchema{
Name: n,
Required: required,
})
}
blockNames := make([]string, 0, len(tags.Blocks))
for n := range tags.Blocks {
blockNames = append(blockNames, n)
}
sort.Strings(blockNames)
for _, n := range blockNames {
idx := tags.Blocks[n]
field := ty.Field(idx)
fty := field.Type
if fty.Kind() == reflect.Slice {
fty = fty.Elem()
}
if fty.Kind() == reflect.Ptr {
fty = fty.Elem()
}
if fty.Kind() != reflect.Struct {
panic(fmt.Sprintf(
"hcl 'block' tag kind cannot be applied to %s field %s: struct required", field.Type.String(), field.Name,
))
}
ftags := getFieldTags(fty)
var labelNames []string
if len(ftags.Labels) > 0 {
labelNames = make([]string, len(ftags.Labels))
for i, l := range ftags.Labels {
labelNames[i] = l.Name
}
}
blockSchemas = append(blockSchemas, hcl.BlockHeaderSchema{
Type: n,
LabelNames: labelNames,
})
}
partial = tags.Remain != nil
schema = &hcl.BodySchema{
Attributes: attrSchemas,
Blocks: blockSchemas,
}
return schema, partial
}
type fieldTags struct {
Attributes map[string]int
Blocks map[string]int
Labels []labelField
Remain *int
Body *int
Optional map[string]bool
}
type labelField struct {
FieldIndex int
Name string
}
func getFieldTags(ty reflect.Type) *fieldTags {
ret := &fieldTags{
Attributes: map[string]int{},
Blocks: map[string]int{},
Optional: map[string]bool{},
}
ct := ty.NumField()
for i := range ct {
field := ty.Field(i)
tag := field.Tag.Get("hcl")
if tag == "" {
continue
}
comma := strings.Index(tag, ",")
var name, kind string
if comma != -1 {
name = tag[:comma]
kind = tag[comma+1:]
} else {
name = tag
kind = "attr"
}
switch kind {
case "attr":
ret.Attributes[name] = i
case "block":
ret.Blocks[name] = i
case "label":
ret.Labels = append(ret.Labels, labelField{
FieldIndex: i,
Name: name,
})
case "remain":
if ret.Remain != nil {
panic("only one 'remain' tag is permitted")
}
idx := i // copy, because this loop will continue assigning to i
ret.Remain = &idx
case "body":
if ret.Body != nil {
panic("only one 'body' tag is permitted")
}
idx := i // copy, because this loop will continue assigning to i
ret.Body = &idx
case "optional":
ret.Attributes[name] = i
ret.Optional[name] = true
default:
panic(fmt.Sprintf("invalid hcl field tag kind %q on %s %q", kind, field.Type.String(), field.Name))
}
}
return ret
}

View File

@@ -1,233 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"testing"
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/hcl/v2"
)
func TestImpliedBodySchema(t *testing.T) {
tests := []struct {
val any
wantSchema *hcl.BodySchema
wantPartial bool
}{
{
struct{}{},
&hcl.BodySchema{},
false,
},
{
struct {
Ignored bool
}{},
&hcl.BodySchema{},
false,
},
{
struct {
Attr1 bool `hcl:"attr1"`
Attr2 bool `hcl:"attr2"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "attr1",
Required: true,
},
{
Name: "attr2",
Required: true,
},
},
},
false,
},
{
struct {
Attr *bool `hcl:"attr,attr"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "attr",
Required: false,
},
},
},
false,
},
{
struct {
Thing struct{} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
},
},
},
false,
},
{
struct {
Thing struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing []struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing *struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing struct {
Name string `hcl:"name,label"`
Something string `hcl:"something"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"name"},
},
},
},
false,
},
{
struct {
Doodad string `hcl:"doodad"`
Thing struct {
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "doodad",
Required: true,
},
},
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"name"},
},
},
},
false,
},
{
struct {
Doodad string `hcl:"doodad"`
Config string `hcl:",remain"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "doodad",
Required: true,
},
},
},
true,
},
{
struct {
Expr hcl.Expression `hcl:"expr"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "expr",
Required: false,
},
},
},
false,
},
{
struct {
Meh string `hcl:"meh,optional"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "meh",
Required: false,
},
},
},
false,
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%#v", test.val), func(t *testing.T) {
schema, partial := ImpliedBodySchema(test.val)
if !reflect.DeepEqual(schema, test.wantSchema) {
t.Errorf(
"wrong schema\ngot: %s\nwant: %s",
spew.Sdump(schema), spew.Sdump(test.wantSchema),
)
}
if partial != test.wantPartial {
t.Errorf(
"wrong partial flag\ngot: %#v\nwant: %#v",
partial, test.wantPartial,
)
}
})
}
}

View File

@@ -1,19 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"reflect"
"github.com/hashicorp/hcl/v2"
)
var victimExpr hcl.Expression
var victimBody hcl.Body
var exprType = reflect.TypeOf(&victimExpr).Elem()
var bodyType = reflect.TypeOf(&victimBody).Elem()
var blockType = reflect.TypeOf((*hcl.Block)(nil)) //nolint:unused
var attrType = reflect.TypeOf((*hcl.Attribute)(nil))
var attrsType = reflect.TypeOf(hcl.Attributes(nil))

View File

@@ -7,15 +7,15 @@ import (
"math" "math"
"math/big" "math/big"
"reflect" "reflect"
"slices"
"strconv" "strconv"
"strings" "strings"
"github.com/docker/buildx/bake/hclparser/gohcl"
"github.com/docker/buildx/util/userfunc" "github.com/docker/buildx/util/userfunc"
"github.com/hashicorp/hcl/v2" "github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/zclconf/go-cty/cty" "github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/gocty"
) )
type Opt struct { type Opt struct {
@@ -25,17 +25,11 @@ type Opt struct {
} }
type variable struct { type variable struct {
Name string `json:"-" hcl:"name,label"` Name string `json:"-" hcl:"name,label"`
Default *hcl.Attribute `json:"default,omitempty" hcl:"default,optional"` Default *hcl.Attribute `json:"default,omitempty" hcl:"default,optional"`
Description string `json:"description,omitempty" hcl:"description,optional"` Description string `json:"description,omitempty" hcl:"description,optional"`
Validations []*variableValidation `json:"validation,omitempty" hcl:"validation,block"` Body hcl.Body `json:"-" hcl:",body"`
Body hcl.Body `json:"-" hcl:",body"` Remain hcl.Body `json:"-" hcl:",remain"`
Remain hcl.Body `json:"-" hcl:",remain"`
}
type variableValidation struct {
Condition hcl.Expression `json:"condition" hcl:"condition"`
ErrorMessage hcl.Expression `json:"error_message" hcl:"error_message"`
} }
type functionDef struct { type functionDef struct {
@@ -454,7 +448,7 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
} }
// decode! // decode!
diag = decodeBody(body(), ectx, output.Interface()) diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() { if diag.HasErrors() {
return diag return diag
} }
@@ -476,11 +470,11 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
} }
// store the result into the evaluation context (so it can be referenced) // store the result into the evaluation context (so it can be referenced)
outputType, err := ImpliedType(output.Interface()) outputType, err := gocty.ImpliedType(output.Interface())
if err != nil { if err != nil {
return err return err
} }
outputValue, err := ToCtyValue(output.Interface(), outputType) outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil { if err != nil {
return err return err
} }
@@ -492,12 +486,7 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
m = map[string]cty.Value{} m = map[string]cty.Value{}
} }
m[name] = outputValue m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
// The logical contents of this structure is similar to a map,
// but it's possible for some attributes to be different in a way that's
// illegal for a map so we use an object here instead which is structurally
// equivalent but allows disparate types for different keys.
p.ectx.Variables[block.Type] = cty.ObjectVal(m)
} }
return nil return nil
@@ -552,37 +541,10 @@ func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
return names, nil return names, nil
} }
func (p *parser) validateVariables(vars map[string]*variable, ectx *hcl.EvalContext) hcl.Diagnostics {
var diags hcl.Diagnostics
for _, v := range vars {
for _, validation := range v.Validations {
condition, condDiags := validation.Condition.Value(ectx)
if condDiags.HasErrors() {
diags = append(diags, condDiags...)
continue
}
if !condition.True() {
message, msgDiags := validation.ErrorMessage.Value(ectx)
if msgDiags.HasErrors() {
diags = append(diags, msgDiags...)
continue
}
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Validation failed",
Detail: message.AsString(),
Subject: validation.Condition.Range().Ptr(),
})
}
}
}
return diags
}
type Variable struct { type Variable struct {
Name string `json:"name"` Name string
Description string `json:"description,omitempty"` Description string
Value *string `json:"value,omitempty"` Value *string
} }
type ParseMeta struct { type ParseMeta struct {
@@ -590,7 +552,7 @@ type ParseMeta struct {
AllVariables []*Variable AllVariables []*Variable
} }
func Parse(b hcl.Body, opt Opt, val any) (*ParseMeta, hcl.Diagnostics) { func Parse(b hcl.Body, opt Opt, val interface{}) (*ParseMeta, hcl.Diagnostics) {
reserved := map[string]struct{}{} reserved := map[string]struct{}{}
schema, _ := gohcl.ImpliedBodySchema(val) schema, _ := gohcl.ImpliedBodySchema(val)
@@ -724,9 +686,6 @@ func Parse(b hcl.Body, opt Opt, val any) (*ParseMeta, hcl.Diagnostics) {
} }
vars = append(vars, v) vars = append(vars, v)
} }
if diags := p.validateVariables(p.vars, p.ectx); diags.HasErrors() {
return nil, diags
}
for k := range p.funcs { for k := range p.funcs {
if err := p.resolveFunction(p.ectx, k); err != nil { if err := p.resolveFunction(p.ectx, k); err != nil {
@@ -764,7 +723,7 @@ func Parse(b hcl.Body, opt Opt, val any) (*ParseMeta, hcl.Diagnostics) {
types := map[string]field{} types := map[string]field{}
renamed := map[string]map[string][]string{} renamed := map[string]map[string][]string{}
vt := reflect.ValueOf(val).Elem().Type() vt := reflect.ValueOf(val).Elem().Type()
for i := range vt.NumField() { for i := 0; i < vt.NumField(); i++ {
tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",") tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",")
p.blockTypes[tags[0]] = vt.Field(i).Type.Elem().Elem() p.blockTypes[tags[0]] = vt.Field(i).Type.Elem().Elem()
@@ -832,7 +791,7 @@ func Parse(b hcl.Body, opt Opt, val any) (*ParseMeta, hcl.Diagnostics) {
oldValue, exists := t.values[lblName] oldValue, exists := t.values[lblName]
if !exists && lblExists { if !exists && lblExists {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice { if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := range v.Elem().Field(t.idx).Len() { for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() { if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i} oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
@@ -899,7 +858,7 @@ func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context
func setName(v reflect.Value, name string) { func setName(v reflect.Value, name string) {
numFields := v.Elem().Type().NumField() numFields := v.Elem().Type().NumField()
for i := range numFields { for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] { for _, t := range parts[1:] {
if t == "label" { if t == "label" {
@@ -911,10 +870,12 @@ func setName(v reflect.Value, name string) {
func getName(v reflect.Value) (string, bool) { func getName(v reflect.Value) (string, bool) {
numFields := v.Elem().Type().NumField() numFields := v.Elem().Type().NumField()
for i := range numFields { for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
if slices.Contains(parts[1:], "label") { for _, t := range parts[1:] {
return v.Elem().Field(i).String(), true if t == "label" {
return v.Elem().Field(i).String(), true
}
} }
} }
return "", false return "", false
@@ -922,10 +883,12 @@ func getName(v reflect.Value) (string, bool) {
func getNameIndex(v reflect.Value) (int, bool) { func getNameIndex(v reflect.Value) (int, bool) {
numFields := v.Elem().Type().NumField() numFields := v.Elem().Type().NumField()
for i := range numFields { for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
if slices.Contains(parts[1:], "label") { for _, t := range parts[1:] {
return i, true if t == "label" {
return i, true
}
} }
} }
return 0, false return 0, false
@@ -984,8 +947,3 @@ func key(ks ...any) uint64 {
} }
return hash.Sum64() return hash.Sum64()
} }
func decodeBody(body hcl.Body, ctx *hcl.EvalContext, val any) hcl.Diagnostics {
dec := gohcl.DecodeOptions{ImpliedType: ImpliedType}
return dec.DecodeBody(body, ctx, val)
}

View File

@@ -170,6 +170,7 @@ func indexOfFunc() function.Function {
} }
} }
return cty.NilVal, errors.New("item not found") return cty.NilVal, errors.New("item not found")
}, },
}) })
} }

View File

@@ -1,160 +0,0 @@
// MIT License
//
// Copyright (c) 2017-2018 Martin Atkins
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package hclparser
import (
"reflect"
"github.com/zclconf/go-cty/cty"
)
// ImpliedType takes an arbitrary Go value (as an interface{}) and attempts
// to find a suitable cty.Type instance that could be used for a conversion
// with ToCtyValue.
//
// This allows -- for simple situations at least -- types to be defined just
// once in Go and the cty types derived from the Go types, but in the process
// it makes some assumptions that may be undesirable so applications are
// encouraged to build their cty types directly if exacting control is
// required.
//
// Not all Go types can be represented as cty types, so an error may be
// returned which is usually considered to be a bug in the calling program.
// In particular, ImpliedType will never use capsule types in its returned
// type, because it cannot know the capsule types supported by the calling
// program.
func ImpliedType(gv any) (cty.Type, error) {
rt := reflect.TypeOf(gv)
var path cty.Path
return impliedType(rt, path)
}
func impliedType(rt reflect.Type, path cty.Path) (cty.Type, error) {
if ety, err := impliedTypeExt(rt, path); err == nil {
return ety, nil
}
switch rt.Kind() {
case reflect.Ptr:
return impliedType(rt.Elem(), path)
// Primitive types
case reflect.Bool:
return cty.Bool, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return cty.Number, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return cty.Number, nil
case reflect.Float32, reflect.Float64:
return cty.Number, nil
case reflect.String:
return cty.String, nil
// Collection types
case reflect.Slice:
path := append(path, cty.IndexStep{Key: cty.UnknownVal(cty.Number)})
ety, err := impliedType(rt.Elem(), path)
if err != nil {
return cty.NilType, err
}
return cty.List(ety), nil
case reflect.Map:
if !stringType.AssignableTo(rt.Key()) {
return cty.NilType, path.NewErrorf("no cty.Type for %s (must have string keys)", rt)
}
path := append(path, cty.IndexStep{Key: cty.UnknownVal(cty.String)})
ety, err := impliedType(rt.Elem(), path)
if err != nil {
return cty.NilType, err
}
return cty.Map(ety), nil
// Structural types
case reflect.Struct:
return impliedStructType(rt, path)
default:
return cty.NilType, path.NewErrorf("no cty.Type for %s", rt)
}
}
func impliedStructType(rt reflect.Type, path cty.Path) (cty.Type, error) {
if valueType.AssignableTo(rt) {
// Special case: cty.Value represents cty.DynamicPseudoType, for
// type conformance checking.
return cty.DynamicPseudoType, nil
}
fieldIdxs := structTagIndices(rt)
if len(fieldIdxs) == 0 {
return cty.NilType, path.NewErrorf("no cty.Type for %s (no cty field tags)", rt)
}
atys := make(map[string]cty.Type, len(fieldIdxs))
{
// Temporary extension of path for attributes
path := append(path, nil)
for k, fi := range fieldIdxs {
path[len(path)-1] = cty.GetAttrStep{Name: k}
ft := rt.Field(fi).Type
aty, err := impliedType(ft, path)
if err != nil {
return cty.NilType, err
}
atys[k] = aty
}
}
return cty.Object(atys), nil
}
var (
valueType = reflect.TypeOf(cty.Value{})
stringType = reflect.TypeOf("")
)
// structTagIndices interrogates the fields of the given type (which must
// be a struct type, or we'll panic) and returns a map from the cty
// attribute names declared via struct tags to the indices of the
// fields holding those tags.
//
// This function will panic if two fields within the struct are tagged with
// the same cty attribute name.
func structTagIndices(st reflect.Type) map[string]int {
ct := st.NumField()
ret := make(map[string]int, ct)
for i := range ct {
field := st.Field(i)
attrName := field.Tag.Get("cty")
if attrName != "" {
ret[attrName] = i
}
}
return ret
}

View File

@@ -1,166 +0,0 @@
package hclparser
import (
"reflect"
"sync"
"github.com/containerd/errdefs"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/gocty"
)
type ToCtyValueConverter interface {
// ToCtyValue will convert this capsule value into a native
// cty.Value. This should not return a capsule type.
ToCtyValue() cty.Value
}
type FromCtyValueConverter interface {
// FromCtyValue will initialize this value using a cty.Value.
FromCtyValue(in cty.Value, path cty.Path) error
}
type extensionType int
const (
unwrapCapsuleValueExtension extensionType = iota
)
func impliedTypeExt(rt reflect.Type, _ cty.Path) (cty.Type, error) {
if rt.Kind() != reflect.Pointer {
rt = reflect.PointerTo(rt)
}
if isCapsuleType(rt) {
return capsuleValueCapsuleType(rt), nil
}
return cty.NilType, errdefs.ErrNotImplemented
}
func isCapsuleType(rt reflect.Type) bool {
fromCtyValueType := reflect.TypeFor[FromCtyValueConverter]()
toCtyValueType := reflect.TypeFor[ToCtyValueConverter]()
return rt.Implements(fromCtyValueType) && rt.Implements(toCtyValueType)
}
var capsuleValueTypes sync.Map
func capsuleValueCapsuleType(rt reflect.Type) cty.Type {
if rt.Kind() != reflect.Pointer {
panic("capsule value must be a pointer")
}
elem := rt.Elem()
if val, loaded := capsuleValueTypes.Load(elem); loaded {
return val.(cty.Type)
}
toCtyValueType := reflect.TypeFor[ToCtyValueConverter]()
// First time used. Initialize new capsule ops.
ops := &cty.CapsuleOps{
ConversionTo: func(_ cty.Type) func(cty.Value, cty.Path) (any, error) {
return func(in cty.Value, p cty.Path) (any, error) {
rv := reflect.New(elem).Interface()
if err := rv.(FromCtyValueConverter).FromCtyValue(in, p); err != nil {
return nil, err
}
return rv, nil
}
},
ConversionFrom: func(want cty.Type) func(any, cty.Path) (cty.Value, error) {
return func(in any, _ cty.Path) (cty.Value, error) {
rv := reflect.ValueOf(in).Convert(toCtyValueType)
v := rv.Interface().(ToCtyValueConverter).ToCtyValue()
return convert.Convert(v, want)
}
},
ExtensionData: func(key any) any {
switch key {
case unwrapCapsuleValueExtension:
zero := reflect.Zero(elem).Interface()
if conv, ok := zero.(ToCtyValueConverter); ok {
return conv.ToCtyValue().Type()
}
zero = reflect.Zero(rt).Interface()
if conv, ok := zero.(ToCtyValueConverter); ok {
return conv.ToCtyValue().Type()
}
}
return nil
},
}
// Attempt to store the new type. Use whichever was loaded first in the case
// of a race condition.
ety := cty.CapsuleWithOps(elem.Name(), elem, ops)
val, _ := capsuleValueTypes.LoadOrStore(elem, ety)
return val.(cty.Type)
}
// UnwrapCtyValue will unwrap capsule type values into their native cty value
// equivalents if possible.
func UnwrapCtyValue(in cty.Value) cty.Value {
want := toCtyValueType(in.Type())
if in.Type().Equals(want) {
return in
} else if out, err := convert.Convert(in, want); err == nil {
return out
}
return cty.NullVal(want)
}
func toCtyValueType(in cty.Type) cty.Type {
if et := in.MapElementType(); et != nil {
return cty.Map(toCtyValueType(*et))
}
if et := in.SetElementType(); et != nil {
return cty.Set(toCtyValueType(*et))
}
if et := in.ListElementType(); et != nil {
return cty.List(toCtyValueType(*et))
}
if in.IsObjectType() {
var optional []string
inAttrTypes := in.AttributeTypes()
outAttrTypes := make(map[string]cty.Type, len(inAttrTypes))
for name, typ := range inAttrTypes {
outAttrTypes[name] = toCtyValueType(typ)
if in.AttributeOptional(name) {
optional = append(optional, name)
}
}
return cty.ObjectWithOptionalAttrs(outAttrTypes, optional)
}
if in.IsTupleType() {
inTypes := in.TupleElementTypes()
outTypes := make([]cty.Type, len(inTypes))
for i, typ := range inTypes {
outTypes[i] = toCtyValueType(typ)
}
return cty.Tuple(outTypes)
}
if in.IsCapsuleType() {
if out := in.CapsuleExtensionData(unwrapCapsuleValueExtension); out != nil {
return out.(cty.Type)
}
return cty.DynamicPseudoType
}
return in
}
func ToCtyValue(val any, ty cty.Type) (cty.Value, error) {
out, err := gocty.ToCtyValue(val, ty)
if err != nil {
return out, err
}
return UnwrapCtyValue(out), nil
}

View File

@@ -11,7 +11,6 @@ import (
controllerapi "github.com/docker/buildx/controller/pb" controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver" "github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/docker/go-units"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb" "github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui" "github.com/moby/buildkit/frontend/dockerui"
@@ -20,8 +19,6 @@ import (
"github.com/pkg/errors" "github.com/pkg/errors"
) )
const maxBakeDefinitionSize = 2 * 1024 * 1024 // 2 MB
type Input struct { type Input struct {
State *llb.State State *llb.State
URL string URL string
@@ -109,6 +106,7 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
} }
return nil, err return nil, err
}, ch) }, ch)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -180,9 +178,9 @@ func filesFromURLRef(ctx context.Context, c gwclient.Client, ref gwclient.Refere
name := inp.URL name := inp.URL
inp.URL = "" inp.URL = ""
if int64(len(dt)) > stat.Size { if len(dt) > stat.Size() {
if stat.Size > maxBakeDefinitionSize { if stat.Size() > 1024*512 {
return nil, errors.Errorf("non-archive definition URL bigger than maximum allowed size (%s)", units.HumanSize(maxBakeDefinitionSize)) return nil, errors.Errorf("non-archive definition URL bigger than maximum allowed size")
} }
dt, err = ref.ReadFile(ctx, gwclient.ReadRequest{ dt, err = ref.ReadFile(ctx, gwclient.ReadRequest{

View File

@@ -15,10 +15,9 @@ import (
"sync" "sync"
"time" "time"
"github.com/containerd/containerd/v2/core/images" "github.com/containerd/containerd/images"
"github.com/distribution/reference" "github.com/distribution/reference"
"github.com/docker/buildx/builder" "github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver" "github.com/docker/buildx/driver"
"github.com/docker/buildx/util/confutil" "github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop" "github.com/docker/buildx/util/desktop"
@@ -40,6 +39,7 @@ import (
"github.com/moby/buildkit/solver/errdefs" "github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb" "github.com/moby/buildkit/solver/pb"
spb "github.com/moby/buildkit/sourcepolicy/pb" spb "github.com/moby/buildkit/sourcepolicy/pb"
"github.com/moby/buildkit/util/entitlements"
"github.com/moby/buildkit/util/progress/progresswriter" "github.com/moby/buildkit/util/progress/progresswriter"
"github.com/moby/buildkit/util/tracing" "github.com/moby/buildkit/util/tracing"
"github.com/opencontainers/go-digest" "github.com/opencontainers/go-digest"
@@ -50,39 +50,35 @@ import (
fstypes "github.com/tonistiigi/fsutil/types" fstypes "github.com/tonistiigi/fsutil/types"
"go.opentelemetry.io/otel/trace" "go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
"google.golang.org/protobuf/proto"
) )
const ( const (
printFallbackImage = "docker/dockerfile:1.7.1@sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e" printFallbackImage = "docker/dockerfile:1.5@sha256:dbbd5e059e8a07ff7ea6233b213b36aa516b4c53c645f1817a4dd18b83cbea56"
printLintFallbackImage = "docker/dockerfile:1.8.1@sha256:e87caa74dcb7d46cd820352bfea12591f3dba3ddc4285e19c7dcd13359f7cefd" printLintFallbackImage = "docker.io/docker/dockerfile-upstream:1.8.1@sha256:e87caa74dcb7d46cd820352bfea12591f3dba3ddc4285e19c7dcd13359f7cefd"
) )
type Options struct { type Options struct {
Inputs Inputs Inputs Inputs
Ref string Ref string
Allow []string Allow []entitlements.Entitlement
Attests map[string]*string Attests map[string]*string
BuildArgs map[string]string BuildArgs map[string]string
CacheFrom []client.CacheOptionsEntry CacheFrom []client.CacheOptionsEntry
CacheTo []client.CacheOptionsEntry CacheTo []client.CacheOptionsEntry
CgroupParent string CgroupParent string
Exports []client.ExportEntry Exports []client.ExportEntry
ExportsLocalPathsTemporary []string // should be removed after client.ExportEntry update in buildkit v0.19.0 ExtraHosts []string
ExtraHosts []string Labels map[string]string
Labels map[string]string NetworkMode string
NetworkMode string NoCache bool
NoCache bool NoCacheFilter []string
NoCacheFilter []string Platforms []specs.Platform
Platforms []specs.Platform Pull bool
Pull bool ShmSize opts.MemBytes
SecretSpecs []*controllerapi.Secret Tags []string
SSHSpecs []*controllerapi.SSH Target string
ShmSize opts.MemBytes Ulimits *opts.UlimitOpt
Tags []string
Target string
Ulimits *opts.UlimitOpt
Session []session.Attachable Session []session.Attachable
Linked bool // Linked marks this target as exclusively linked (not requested by the user). Linked bool // Linked marks this target as exclusively linked (not requested by the user).
@@ -105,9 +101,6 @@ type Inputs struct {
ContextState *llb.State ContextState *llb.State
DockerfileInline string DockerfileInline string
NamedContexts map[string]NamedContext NamedContexts map[string]NamedContext
// DockerfileMappingSrc and DockerfileMappingDst are filled in by the builder.
DockerfileMappingSrc string
DockerfileMappingDst string
} }
type NamedContext struct { type NamedContext struct {
@@ -154,11 +147,11 @@ func toRepoOnly(in string) (string, error) {
return strings.Join(out, ","), nil return strings.Join(out, ","), nil
} }
func Build(ctx context.Context, nodes []builder.Node, opts map[string]Options, docker *dockerutil.Client, cfg *confutil.Config, w progress.Writer) (resp map[string]*client.SolveResponse, err error) { func Build(ctx context.Context, nodes []builder.Node, opt map[string]Options, docker *dockerutil.Client, configDir string, w progress.Writer) (resp map[string]*client.SolveResponse, err error) {
return BuildWithResultHandler(ctx, nodes, opts, docker, cfg, w, nil) return BuildWithResultHandler(ctx, nodes, opt, docker, configDir, w, nil)
} }
func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[string]Options, docker *dockerutil.Client, cfg *confutil.Config, w progress.Writer, resultHandleFunc func(driverIndex int, rCtx *ResultHandle)) (resp map[string]*client.SolveResponse, err error) { func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opt map[string]Options, docker *dockerutil.Client, configDir string, w progress.Writer, resultHandleFunc func(driverIndex int, rCtx *ResultHandle)) (resp map[string]*client.SolveResponse, err error) {
if len(nodes) == 0 { if len(nodes) == 0 {
return nil, errors.Errorf("driver required for build") return nil, errors.Errorf("driver required for build")
} }
@@ -176,9 +169,9 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
} }
} }
if noMobyDriver != nil && !noDefaultLoad() && noCallFunc(opts) { if noMobyDriver != nil && !noDefaultLoad() && noCallFunc(opt) {
var noOutputTargets []string var noOutputTargets []string
for name, opt := range opts { for name, opt := range opt {
if noMobyDriver.Features(ctx)[driver.DefaultLoad] { if noMobyDriver.Features(ctx)[driver.DefaultLoad] {
continue continue
} }
@@ -199,7 +192,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
} }
} }
drivers, err := resolveDrivers(ctx, nodes, opts, w) drivers, err := resolveDrivers(ctx, nodes, opt, w)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -216,7 +209,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
reqForNodes := make(map[string][]*reqForNode) reqForNodes := make(map[string][]*reqForNode)
eg, ctx := errgroup.WithContext(ctx) eg, ctx := errgroup.WithContext(ctx)
for k, opt := range opts { for k, opt := range opt {
multiDriver := len(drivers[k]) > 1 multiDriver := len(drivers[k]) > 1
hasMobyDriver := false hasMobyDriver := false
addGitAttrs, err := getGitAttributes(ctx, opt.Inputs.ContextPath, opt.Inputs.DockerfilePath) addGitAttrs, err := getGitAttributes(ctx, opt.Inputs.ContextPath, opt.Inputs.DockerfilePath)
@@ -236,13 +229,11 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
if err != nil { if err != nil {
return nil, err return nil, err
} }
localOpt := opt so, release, err := toSolveOpt(ctx, np.Node(), multiDriver, opt, gatewayOpts, configDir, w, docker)
so, release, err := toSolveOpt(ctx, np.Node(), multiDriver, &localOpt, gatewayOpts, cfg, w, docker)
opts[k] = localOpt
if err != nil { if err != nil {
return nil, err return nil, err
} }
if err := saveLocalState(so, k, opt, np.Node(), cfg); err != nil { if err := saveLocalState(so, k, opt, np.Node(), configDir); err != nil {
return nil, err return nil, err
} }
addGitAttrs(so) addGitAttrs(so)
@@ -278,7 +269,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
} }
// validate that all links between targets use same drivers // validate that all links between targets use same drivers
for name := range opts { for name := range opt {
dps := reqForNodes[name] dps := reqForNodes[name]
for i, dp := range dps { for i, dp := range dps {
so := reqForNodes[name][i].so so := reqForNodes[name][i].so
@@ -314,10 +305,10 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
var respMu sync.Mutex var respMu sync.Mutex
results := waitmap.New() results := waitmap.New()
multiTarget := len(opts) > 1 multiTarget := len(opt) > 1
childTargets := calculateChildTargets(reqForNodes, opts) childTargets := calculateChildTargets(reqForNodes, opt)
for k, opt := range opts { for k, opt := range opt {
err := func(k string) (err error) { err := func(k string) (err error) {
opt := opt opt := opt
dps := drivers[k] dps := drivers[k]
@@ -503,9 +494,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
resultHandle, rr, err = NewResultHandle(ctx, cc, *so, "buildx", buildFunc, ch) resultHandle, rr, err = NewResultHandle(ctx, cc, *so, "buildx", buildFunc, ch)
resultHandleFunc(dp.driverIndex, resultHandle) resultHandleFunc(dp.driverIndex, resultHandle)
} else { } else {
span, ctx := tracing.StartSpan(ctx, "build")
rr, err = c.Build(ctx, *so, "buildx", buildFunc, ch) rr, err = c.Build(ctx, *so, "buildx", buildFunc, ch)
tracing.FinishWithError(span, err)
} }
if !so.Internal && desktop.BuildBackendEnabled() && node.Driver.HistoryAPISupported(ctx) { if !so.Internal && desktop.BuildBackendEnabled() && node.Driver.HistoryAPISupported(ctx) {
if err != nil { if err != nil {
@@ -539,7 +528,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
node := dp.Node().Driver node := dp.Node().Driver
if node.IsMobyDriver() { if node.IsMobyDriver() {
for _, e := range so.Exports { for _, e := range so.Exports {
if e.Type == "moby" && e.Attrs["push"] != "" && !node.Features(ctx)[driver.DirectPush] { if e.Type == "moby" && e.Attrs["push"] != "" {
if ok, _ := strconv.ParseBool(e.Attrs["push"]); ok { if ok, _ := strconv.ParseBool(e.Attrs["push"]); ok {
pushNames = e.Attrs["name"] pushNames = e.Attrs["name"]
if pushNames == "" { if pushNames == "" {
@@ -622,7 +611,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
// This is fallback for some very old buildkit versions. // This is fallback for some very old buildkit versions.
// Note that the mediatype isn't really correct as most of the time it is image manifest and // Note that the mediatype isn't really correct as most of the time it is image manifest and
// not manifest list but actually both are handled because for Docker mediatypes the // not manifest list but actually both are handled because for Docker mediatypes the
// mediatype value in the Accept header does not seem to matter. // mediatype value in the Accpet header does not seem to matter.
s, ok = r.ExporterResponse[exptypes.ExporterImageDigestKey] s, ok = r.ExporterResponse[exptypes.ExporterImageDigestKey]
if ok { if ok {
descs = append(descs, specs.Descriptor{ descs = append(descs, specs.Descriptor{
@@ -834,7 +823,7 @@ func remoteDigestWithMoby(ctx context.Context, d *driver.DriverHandle, name stri
if err != nil { if err != nil {
return "", err return "", err
} }
img, err := api.ImageInspect(ctx, name) img, _, err := api.ImageInspectWithRaw(ctx, name)
if err != nil { if err != nil {
return "", err return "", err
} }
@@ -1142,7 +1131,7 @@ func ReadSourcePolicy() (*spb.Policy, error) {
var pol spb.Policy var pol spb.Policy
if err := json.Unmarshal(data, &pol); err != nil { if err := json.Unmarshal(data, &pol); err != nil {
// maybe it's in protobuf format? // maybe it's in protobuf format?
e2 := proto.Unmarshal(data, &pol) e2 := pol.Unmarshal(data)
if e2 != nil { if e2 != nil {
return nil, errors.Wrap(err, "failed to parse source policy") return nil, errors.Wrap(err, "failed to parse source policy")
} }

View File

@@ -4,7 +4,6 @@ import (
"context" "context"
stderrors "errors" stderrors "errors"
"net" "net"
"slices"
"github.com/containerd/platforms" "github.com/containerd/platforms"
"github.com/docker/buildx/builder" "github.com/docker/buildx/builder"
@@ -38,7 +37,15 @@ func Dial(ctx context.Context, nodes []builder.Node, pw progress.Writer, platfor
for _, ls := range resolved { for _, ls := range resolved {
for _, rn := range ls { for _, rn := range ls {
if platform != nil { if platform != nil {
if !slices.ContainsFunc(rn.platforms, platforms.Only(*platform).Match) { p := *platform
var found bool
for _, pp := range rn.platforms {
if platforms.Only(p).Match(pp) {
found = true
break
}
}
if !found {
continue continue
} }
} }

View File

@@ -3,7 +3,6 @@ package build
import ( import (
"context" "context"
"fmt" "fmt"
"slices"
"sync" "sync"
"github.com/containerd/platforms" "github.com/containerd/platforms"
@@ -222,7 +221,7 @@ func (r *nodeResolver) get(p specs.Platform, matcher matchMaker, additionalPlatf
for i, node := range r.nodes { for i, node := range r.nodes {
platforms := node.Platforms platforms := node.Platforms
if additionalPlatforms != nil { if additionalPlatforms != nil {
platforms = slices.Clone(platforms) platforms = append([]specs.Platform{}, platforms...)
platforms = append(platforms, additionalPlatforms(i, node)...) platforms = append(platforms, additionalPlatforms(i, node)...)
} }
for _, p2 := range platforms { for _, p2 := range platforms {

View File

@@ -23,7 +23,7 @@ func setupTest(tb testing.TB) {
gitutil.GitInit(c, tb) gitutil.GitInit(c, tb)
df := []byte("FROM alpine:latest\n") df := []byte("FROM alpine:latest\n")
require.NoError(tb, os.WriteFile("Dockerfile", df, 0644)) assert.NoError(tb, os.WriteFile("Dockerfile", df, 0644))
gitutil.GitAdd(c, tb, "Dockerfile") gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit") gitutil.GitCommit(c, tb, "initial commit")
@@ -32,7 +32,7 @@ func setupTest(tb testing.TB) {
func TestGetGitAttributesNotGitRepo(t *testing.T) { func TestGetGitAttributesNotGitRepo(t *testing.T) {
_, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile") _, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
require.NoError(t, err) assert.NoError(t, err)
} }
func TestGetGitAttributesBadGitRepo(t *testing.T) { func TestGetGitAttributesBadGitRepo(t *testing.T) {
@@ -47,7 +47,7 @@ func TestGetGitAttributesNoContext(t *testing.T) {
setupTest(t) setupTest(t)
addGitAttrs, err := getGitAttributes(context.Background(), "", "Dockerfile") addGitAttrs, err := getGitAttributes(context.Background(), "", "Dockerfile")
require.NoError(t, err) assert.NoError(t, err)
var so client.SolveOpt var so client.SolveOpt
addGitAttrs(&so) addGitAttrs(&so)
assert.Empty(t, so.FrontendAttrs) assert.Empty(t, so.FrontendAttrs)
@@ -195,8 +195,8 @@ func TestLocalDirsSub(t *testing.T) {
gitutil.GitInit(c, t) gitutil.GitInit(c, t)
df := []byte("FROM alpine:latest\n") df := []byte("FROM alpine:latest\n")
require.NoError(t, os.MkdirAll("app", 0755)) assert.NoError(t, os.MkdirAll("app", 0755))
require.NoError(t, os.WriteFile("app/Dockerfile", df, 0644)) assert.NoError(t, os.WriteFile("app/Dockerfile", df, 0644))
gitutil.GitAdd(c, t, "app/Dockerfile") gitutil.GitAdd(c, t, "app/Dockerfile")
gitutil.GitCommit(c, t, "initial commit") gitutil.GitCommit(c, t, "initial commit")

View File

@@ -16,7 +16,7 @@ import (
type Container struct { type Container struct {
cancelOnce sync.Once cancelOnce sync.Once
containerCancel func(error) containerCancel func()
isUnavailable atomic.Bool isUnavailable atomic.Bool
initStarted atomic.Bool initStarted atomic.Bool
container gateway.Container container gateway.Container
@@ -31,18 +31,18 @@ func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllera
errCh := make(chan error) errCh := make(chan error)
go func() { go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) { err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancelCause(ctx) ctx, cancel := context.WithCancel(ctx)
go func() { go func() {
<-mainCtx.Done() <-mainCtx.Done()
cancel(errors.WithStack(context.Canceled)) cancel()
}() }()
containerCfg, err := resultCtx.getContainerConfig(cfg) containerCfg, err := resultCtx.getContainerConfig(cfg)
if err != nil { if err != nil {
return nil, err return nil, err
} }
containerCtx, containerCancel := context.WithCancelCause(ctx) containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel(errors.WithStack(context.Canceled)) defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg) bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -83,7 +83,7 @@ func (c *Container) Cancel() {
c.markUnavailable() c.markUnavailable()
c.cancelOnce.Do(func() { c.cancelOnce.Do(func() {
if c.containerCancel != nil { if c.containerCancel != nil {
c.containerCancel(errors.WithStack(context.Canceled)) c.containerCancel()
} }
close(c.releaseCh) close(c.releaseCh)
}) })

View File

@@ -5,13 +5,12 @@ import (
"github.com/docker/buildx/builder" "github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate" "github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/confutil"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
) )
func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, cfg *confutil.Config) error { func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, configDir string) error {
var err error var err error
if so.Ref == "" || opts.CallFunc != nil { if so.Ref == "" {
return nil return nil
} }
lp := opts.Inputs.ContextPath lp := opts.Inputs.ContextPath
@@ -31,7 +30,7 @@ func saveLocalState(so *client.SolveOpt, target string, opts Options, node build
if lp == "" && dp == "" { if lp == "" && dp == "" {
return nil return nil
} }
l, err := localstate.New(cfg) l, err := localstate.New(configDir)
if err != nil { if err != nil {
return err return err
} }

View File

@@ -11,8 +11,8 @@ import (
"strings" "strings"
"syscall" "syscall"
"github.com/containerd/containerd/v2/core/content" "github.com/containerd/containerd/content"
"github.com/containerd/containerd/v2/plugins/content/local" "github.com/containerd/containerd/content/local"
"github.com/containerd/platforms" "github.com/containerd/platforms"
"github.com/distribution/reference" "github.com/distribution/reference"
"github.com/docker/buildx/builder" "github.com/docker/buildx/builder"
@@ -35,7 +35,7 @@ import (
"github.com/tonistiigi/fsutil" "github.com/tonistiigi/fsutil"
) )
func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *Options, bopts gateway.BuildOpts, cfg *confutil.Config, pw progress.Writer, docker *dockerutil.Client) (_ *client.SolveOpt, release func(), err error) { func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt Options, bopts gateway.BuildOpts, configDir string, pw progress.Writer, docker *dockerutil.Client) (_ *client.SolveOpt, release func(), err error) {
nodeDriver := node.Driver nodeDriver := node.Driver
defers := make([]func(), 0, 2) defers := make([]func(), 0, 2)
releaseF := func() { releaseF := func() {
@@ -263,7 +263,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
so.Exports = opt.Exports so.Exports = opt.Exports
so.Session = slices.Clone(opt.Session) so.Session = slices.Clone(opt.Session)
releaseLoad, err := loadInputs(ctx, nodeDriver, &opt.Inputs, pw, &so) releaseLoad, err := loadInputs(ctx, nodeDriver, opt.Inputs, pw, &so)
if err != nil { if err != nil {
return nil, nil, err return nil, nil, err
} }
@@ -271,7 +271,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
// add node identifier to shared key if one was specified // add node identifier to shared key if one was specified
if so.SharedKey != "" { if so.SharedKey != "" {
so.SharedKey += ":" + cfg.TryNodeIdentifier() so.SharedKey += ":" + confutil.TryNodeIdentifier(configDir)
} }
if opt.Pull { if opt.Pull {
@@ -318,7 +318,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
switch opt.NetworkMode { switch opt.NetworkMode {
case "host": case "host":
so.FrontendAttrs["force-network-mode"] = opt.NetworkMode so.FrontendAttrs["force-network-mode"] = opt.NetworkMode
so.AllowedEntitlements = append(so.AllowedEntitlements, entitlements.EntitlementNetworkHost.String()) so.AllowedEntitlements = append(so.AllowedEntitlements, entitlements.EntitlementNetworkHost)
case "none": case "none":
so.FrontendAttrs["force-network-mode"] = opt.NetworkMode so.FrontendAttrs["force-network-mode"] = opt.NetworkMode
case "", "default": case "", "default":
@@ -356,7 +356,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
return &so, releaseF, nil return &so, releaseF, nil
} }
func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw progress.Writer, target *client.SolveOpt) (func(), error) { func loadInputs(ctx context.Context, d *driver.DriverHandle, inp Inputs, pw progress.Writer, target *client.SolveOpt) (func(), error) {
if inp.ContextPath == "" { if inp.ContextPath == "" {
return nil, errors.New("please specify build context (e.g. \".\" for the current directory)") return nil, errors.New("please specify build context (e.g. \".\" for the current directory)")
} }
@@ -364,12 +364,11 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
// TODO: handle stdin, symlinks, remote contexts, check files exist // TODO: handle stdin, symlinks, remote contexts, check files exist
var ( var (
err error err error
dockerfileReader io.ReadCloser dockerfileReader io.ReadCloser
dockerfileDir string dockerfileDir string
dockerfileName = inp.DockerfilePath dockerfileName = inp.DockerfilePath
dockerfileSrcName = inp.DockerfilePath toRemove []string
toRemove []string
) )
switch { switch {
@@ -441,11 +440,6 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
if inp.DockerfileInline != "" { if inp.DockerfileInline != "" {
dockerfileReader = io.NopCloser(strings.NewReader(inp.DockerfileInline)) dockerfileReader = io.NopCloser(strings.NewReader(inp.DockerfileInline))
dockerfileSrcName = "inline"
} else if inp.DockerfilePath == "-" {
dockerfileSrcName = "stdin"
} else if inp.DockerfilePath == "" {
dockerfileSrcName = filepath.Join(inp.ContextPath, "Dockerfile")
} }
if dockerfileReader != nil { if dockerfileReader != nil {
@@ -546,9 +540,6 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
_ = os.RemoveAll(dir) _ = os.RemoveAll(dir)
} }
} }
inp.DockerfileMappingSrc = dockerfileSrcName
inp.DockerfileMappingDst = dockerfileName
return release, nil return release, nil
} }

View File

@@ -8,14 +8,13 @@ import (
"strings" "strings"
"sync" "sync"
"github.com/containerd/containerd/v2/core/content" "github.com/containerd/containerd/content"
"github.com/containerd/containerd/v2/core/content/proxy" "github.com/containerd/containerd/content/proxy"
"github.com/docker/buildx/util/confutil" "github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
controlapi "github.com/moby/buildkit/api/services/control" controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
provenancetypes "github.com/moby/buildkit/solver/llbsolver/provenance/types" provenancetypes "github.com/moby/buildkit/solver/llbsolver/provenance/types"
digest "github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1" ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors" "github.com/pkg/errors"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
@@ -125,8 +124,8 @@ func lookupProvenance(res *controlapi.BuildResultInfo) *ocispecs.Descriptor {
for _, a := range res.Attestations { for _, a := range res.Attestations {
if a.MediaType == "application/vnd.in-toto+json" && strings.HasPrefix(a.Annotations["in-toto.io/predicate-type"], "https://slsa.dev/provenance/") { if a.MediaType == "application/vnd.in-toto+json" && strings.HasPrefix(a.Annotations["in-toto.io/predicate-type"], "https://slsa.dev/provenance/") {
return &ocispecs.Descriptor{ return &ocispecs.Descriptor{
Digest: digest.Digest(a.Digest), Digest: a.Digest,
Size: a.Size, Size: a.Size_,
MediaType: a.MediaType, MediaType: a.MediaType,
Annotations: a.Annotations, Annotations: a.Annotations,
} }

View File

@@ -10,6 +10,7 @@ import (
"time" "time"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
) )
func generateRandomData(size int) []byte { func generateRandomData(size int) []byte {
@@ -28,11 +29,11 @@ func TestSyncMultiReaderParallel(t *testing.T) {
readers := make([]io.ReadCloser, numReaders) readers := make([]io.ReadCloser, numReaders)
for i := range numReaders { for i := 0; i < numReaders; i++ {
readers[i] = mr.NewReadCloser() readers[i] = mr.NewReadCloser()
} }
for i := range numReaders { for i := 0; i < numReaders; i++ {
wg.Add(1) wg.Add(1)
go func(readerId int) { go func(readerId int) {
defer wg.Done() defer wg.Done()
@@ -56,7 +57,7 @@ func TestSyncMultiReaderParallel(t *testing.T) {
return return
} }
assert.NoError(t, err, "Reader %d error", readerId) require.NoError(t, err, "Reader %d error", readerId)
if mathrand.Intn(1000) == 0 { //nolint:gosec if mathrand.Intn(1000) == 0 { //nolint:gosec
t.Logf("Reader %d closing", readerId) t.Logf("Reader %d closing", readerId)

View File

@@ -82,7 +82,7 @@ func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt
var respHandle *ResultHandle var respHandle *ResultHandle
go func() { go func() {
defer func() { cancel(errors.WithStack(context.Canceled)) }() // ensure no dangling processes defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result var res *gateway.Result
var err error var err error
@@ -181,7 +181,7 @@ func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt
case <-respHandle.done: case <-respHandle.done:
case <-ctx.Done(): case <-ctx.Done():
} }
return nil, context.Cause(ctx) return nil, ctx.Err()
}, nil) }, nil)
if respHandle != nil { if respHandle != nil {
return return
@@ -295,14 +295,14 @@ func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
func (r *ResultHandle) getContainerConfig(cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) { func (r *ResultHandle) getContainerConfig(cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil { if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build") logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(r.res, cfg) ccfg, err := containerConfigFromResult(r.res, *cfg)
if err != nil { if err != nil {
return containerCfg, err return containerCfg, err
} }
containerCfg = *ccfg containerCfg = *ccfg
} else { } else {
logrus.Debugf("creating container from failed build %+v", cfg) logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, cfg) ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil { if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available") return containerCfg, errors.Wrapf(err, "no result nor error is available")
} }
@@ -315,19 +315,19 @@ func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin i
processCfg := newStartRequest(stdin, stdout, stderr) processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil { if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build") logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, cfg); err != nil { if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err return processCfg, err
} }
} else { } else {
logrus.Debugf("creating container from failed build %+v", cfg) logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, cfg); err != nil { if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err return processCfg, err
} }
} }
return processCfg, nil return processCfg, nil
} }
func containerConfigFromResult(res *gateway.Result, cfg *controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) { func containerConfigFromResult(res *gateway.Result, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial { if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps") return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
} }
@@ -352,7 +352,7 @@ func containerConfigFromResult(res *gateway.Result, cfg *controllerapi.InvokeCon
}, nil }, nil
} }
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg *controllerapi.InvokeConfig) error { func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey] imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image var img *specs.Image
if len(imgData) > 0 { if len(imgData) > 0 {
@@ -403,7 +403,7 @@ func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Res
return nil return nil
} }
func containerConfigFromError(solveErr *errdefs.SolveError, cfg *controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) { func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr) exec, err := execOpFromError(solveErr)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -431,7 +431,7 @@ func containerConfigFromError(solveErr *errdefs.SolveError, cfg *controllerapi.I
}, nil }, nil
} }
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg *controllerapi.InvokeConfig) error { func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr) exec, err := execOpFromError(solveErr)
if err != nil { if err != nil {
return err return err

View File

@@ -7,15 +7,12 @@ import (
"github.com/docker/buildx/driver" "github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/docker/go-units"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb" "github.com/moby/buildkit/client/llb"
gwclient "github.com/moby/buildkit/frontend/gateway/client" gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors" "github.com/pkg/errors"
) )
const maxDockerfileSize = 2 * 1024 * 1024 // 2 MB
func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) { func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, ctx, d, pw) c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil { if err != nil {
@@ -46,8 +43,8 @@ func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, ur
if err != nil { if err != nil {
return nil, err return nil, err
} }
if stat.Size > maxDockerfileSize { if stat.Size() > 512*1024 {
return nil, errors.Errorf("Dockerfile %s bigger than allowed max size (%s)", url, units.HumanSize(maxDockerfileSize)) return nil, errors.Errorf("Dockerfile %s bigger than allowed max size", url)
} }
dt, err := ref.ReadFile(ctx, gwclient.ReadRequest{ dt, err := ref.ReadFile(ctx, gwclient.ReadRequest{
@@ -66,6 +63,7 @@ func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, ur
out = dir out = dir
return nil, nil return nil, nil
}, ch) }, ch)
if err != nil { if err != nil {
return "", err return "", err
} }

View File

@@ -138,7 +138,7 @@ func TestToBuildkitExtraHosts(t *testing.T) {
actualOut, actualErr := toBuildkitExtraHosts(context.TODO(), tc.input, nil) actualOut, actualErr := toBuildkitExtraHosts(context.TODO(), tc.input, nil)
if tc.expectedErr == "" { if tc.expectedErr == "" {
require.Equal(t, tc.expectedOut, actualOut) require.Equal(t, tc.expectedOut, actualOut)
require.NoError(t, actualErr) require.Nil(t, actualErr)
} else { } else {
require.Zero(t, actualOut) require.Zero(t, actualOut)
require.Error(t, actualErr, tc.expectedErr) require.Error(t, actualErr, tc.expectedErr)

View File

@@ -5,7 +5,6 @@ import (
"encoding/json" "encoding/json"
"net/url" "net/url"
"os" "os"
"slices"
"sort" "sort"
"strings" "strings"
"sync" "sync"
@@ -200,7 +199,7 @@ func (b *Builder) Boot(ctx context.Context) (bool, error) {
err = err1 err = err1
} }
if err == nil && len(errCh) > 0 { if err == nil && len(errCh) == len(toBoot) {
return false, <-errCh return false, <-errCh
} }
return true, err return true, err
@@ -289,15 +288,7 @@ func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
return nil, err return nil, err
} }
contexts, err := dockerCli.ContextStore().List() builders := make([]*Builder, len(storeng))
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
builders := make([]*Builder, len(storeng), len(storeng)+len(contexts))
seen := make(map[string]struct{}) seen := make(map[string]struct{})
for i, ng := range storeng { for i, ng := range storeng {
b, err := New(dockerCli, b, err := New(dockerCli,
@@ -312,6 +303,14 @@ func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
seen[b.NodeGroup.Name] = struct{}{} seen[b.NodeGroup.Name] = struct{}{}
} }
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
for _, c := range contexts { for _, c := range contexts {
// if a context has the same name as an instance from the store, do not // if a context has the same name as an instance from the store, do not
// add it to the builders list. An instance from the store takes // add it to the builders list. An instance from the store takes
@@ -440,7 +439,7 @@ func Create(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Cre
if buildkitdConfigFile == "" { if buildkitdConfigFile == "" {
// if buildkit daemon config is not provided, check if the default one // if buildkit daemon config is not provided, check if the default one
// is available and use it // is available and use it
if f, ok := confutil.NewConfig(dockerCli).BuildKitConfigFile(); ok { if f, ok := confutil.DefaultConfigFile(dockerCli); ok {
buildkitdConfigFile = f buildkitdConfigFile = f
} }
} }
@@ -523,9 +522,8 @@ func Create(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Cre
return nil, err return nil, err
} }
cancelCtx, cancel := context.WithCancelCause(ctx) timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
timeoutCtx, _ := context.WithTimeoutCause(cancelCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent defer cancel()
defer func() { cancel(errors.WithStack(context.Canceled)) }()
nodes, err := b.LoadNodes(timeoutCtx, WithData()) nodes, err := b.LoadNodes(timeoutCtx, WithData())
if err != nil { if err != nil {
@@ -586,7 +584,7 @@ func Leave(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Leav
return err return err
} }
ls, err := localstate.New(confutil.NewConfig(dockerCli)) ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil { if err != nil {
return err return err
} }
@@ -657,7 +655,13 @@ func parseBuildkitdFlags(inp string, driver string, driverOpts map[string]string
flags.StringArrayVar(&allowInsecureEntitlements, "allow-insecure-entitlement", nil, "") flags.StringArrayVar(&allowInsecureEntitlements, "allow-insecure-entitlement", nil, "")
_ = flags.Parse(res) _ = flags.Parse(res)
hasNetworkHostEntitlement := slices.Contains(allowInsecureEntitlements, "network.host") var hasNetworkHostEntitlement bool
for _, e := range allowInsecureEntitlements {
if e == "network.host" {
hasNetworkHostEntitlement = true
break
}
}
var hasNetworkHostEntitlementInConf bool var hasNetworkHostEntitlementInConf bool
if buildkitdConfigFile != "" { if buildkitdConfigFile != "" {
@@ -666,8 +670,11 @@ func parseBuildkitdFlags(inp string, driver string, driverOpts map[string]string
return nil, err return nil, err
} else if btoml != nil { } else if btoml != nil {
if ies := btoml.GetArray("insecure-entitlements"); ies != nil { if ies := btoml.GetArray("insecure-entitlements"); ies != nil {
if slices.Contains(ies.([]string), "network.host") { for _, e := range ies.([]string) {
hasNetworkHostEntitlementInConf = true if e == "network.host" {
hasNetworkHostEntitlementInConf = true
break
}
} }
} }
} }

View File

@@ -19,20 +19,17 @@ func TestCsvToMap(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Contains(t, r, "tolerations") require.Contains(t, r, "tolerations")
require.Equal(t, "key=foo,value=bar;key=foo2,value=bar2", r["tolerations"]) require.Equal(t, r["tolerations"], "key=foo,value=bar;key=foo2,value=bar2")
require.Contains(t, r, "replicas") require.Contains(t, r, "replicas")
require.Equal(t, "1", r["replicas"]) require.Equal(t, r["replicas"], "1")
require.Contains(t, r, "namespace") require.Contains(t, r, "namespace")
require.Equal(t, "default", r["namespace"]) require.Equal(t, r["namespace"], "default")
} }
func TestParseBuildkitdFlags(t *testing.T) { func TestParseBuildkitdFlags(t *testing.T) {
dirConf := t.TempDir() buildkitdConf := `
buildkitdConfPath := path.Join(dirConf, "buildkitd-conf.toml")
require.NoError(t, os.WriteFile(buildkitdConfPath, []byte(`
# debug enables additional debug logging # debug enables additional debug logging
debug = true debug = true
# insecure-entitlements allows insecure entitlements, disabled by default. # insecure-entitlements allows insecure entitlements, disabled by default.
@@ -40,18 +37,10 @@ insecure-entitlements = [ "network.host", "security.insecure" ]
[log] [log]
# log formatter: json or text # log formatter: json or text
format = "text" format = "text"
`), 0644)) `
dirConf := t.TempDir()
buildkitdConfBrokenPath := path.Join(dirConf, "buildkitd-conf-broken.toml") buildkitdConfPath := path.Join(dirConf, "buildkitd-conf.toml")
require.NoError(t, os.WriteFile(buildkitdConfBrokenPath, []byte(` require.NoError(t, os.WriteFile(buildkitdConfPath, []byte(buildkitdConf), 0644))
[worker.oci]
gc = "maybe"
`), 0644))
buildkitdConfUnknownFieldPath := path.Join(dirConf, "buildkitd-unknown-field.toml")
require.NoError(t, os.WriteFile(buildkitdConfUnknownFieldPath, []byte(`
foo = "bar"
`), 0644))
testCases := []struct { testCases := []struct {
name string name string
@@ -168,26 +157,6 @@ foo = "bar"
nil, nil,
true, true,
}, },
{
"error parsing buildkit config",
"",
"docker-container",
nil,
buildkitdConfBrokenPath,
nil,
true,
},
{
"unknown field in buildkit config",
"",
"docker-container",
nil,
buildkitdConfUnknownFieldPath,
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
} }
for _, tt := range testCases { for _, tt := range testCases {
tt := tt tt := tt

View File

@@ -32,11 +32,10 @@ type Node struct {
Err error Err error
// worker settings // worker settings
IDs []string IDs []string
Platforms []ocispecs.Platform Platforms []ocispecs.Platform
GCPolicy []client.PruneInfo GCPolicy []client.PruneInfo
Labels map[string]string Labels map[string]string
CDIDevices []client.CDIDevice
} }
// Nodes returns nodes for this builder. // Nodes returns nodes for this builder.
@@ -169,7 +168,7 @@ func (b *Builder) LoadNodes(ctx context.Context, opts ...LoadNodesOption) (_ []N
// dynamic nodes are used in Kubernetes driver. // dynamic nodes are used in Kubernetes driver.
// Kubernetes' pods are dynamically mapped to BuildKit Nodes. // Kubernetes' pods are dynamically mapped to BuildKit Nodes.
if di.DriverInfo != nil && len(di.DriverInfo.DynamicNodes) > 0 { if di.DriverInfo != nil && len(di.DriverInfo.DynamicNodes) > 0 {
for i := range di.DriverInfo.DynamicNodes { for i := 0; i < len(di.DriverInfo.DynamicNodes); i++ {
diClone := di diClone := di
if pl := di.DriverInfo.DynamicNodes[i].Platforms; len(pl) > 0 { if pl := di.DriverInfo.DynamicNodes[i].Platforms; len(pl) > 0 {
diClone.Platforms = pl diClone.Platforms = pl
@@ -260,7 +259,6 @@ func (n *Node) loadData(ctx context.Context, clientOpt ...client.ClientOpt) erro
n.GCPolicy = w.GCPolicy n.GCPolicy = w.GCPolicy
n.Labels = w.Labels n.Labels = w.Labels
} }
n.CDIDevices = w.CDIDevices
} }
sort.Strings(n.IDs) sort.Strings(n.IDs)
n.Platforms = platformutil.Dedupe(n.Platforms) n.Platforms = platformutil.Dedupe(n.Platforms)

View File

@@ -1,75 +0,0 @@
package main
import (
"context"
"os"
"runtime"
"runtime/pprof"
"github.com/moby/buildkit/util/bklog"
"github.com/sirupsen/logrus"
)
func setupDebugProfiles(ctx context.Context) (stop func()) {
var stopFuncs []func()
if fn := setupCPUProfile(ctx); fn != nil {
stopFuncs = append(stopFuncs, fn)
}
if fn := setupHeapProfile(ctx); fn != nil {
stopFuncs = append(stopFuncs, fn)
}
return func() {
for _, fn := range stopFuncs {
fn()
}
}
}
func setupCPUProfile(ctx context.Context) (stop func()) {
if cpuProfile := os.Getenv("BUILDX_CPU_PROFILE"); cpuProfile != "" {
f, err := os.Create(cpuProfile)
if err != nil {
bklog.G(ctx).Warn("could not create cpu profile", logrus.WithError(err))
return nil
}
if err := pprof.StartCPUProfile(f); err != nil {
bklog.G(ctx).Warn("could not start cpu profile", logrus.WithError(err))
_ = f.Close()
return nil
}
return func() {
pprof.StopCPUProfile()
if err := f.Close(); err != nil {
bklog.G(ctx).Warn("could not close file for cpu profile", logrus.WithError(err))
}
}
}
return nil
}
func setupHeapProfile(ctx context.Context) (stop func()) {
if heapProfile := os.Getenv("BUILDX_MEM_PROFILE"); heapProfile != "" {
// Memory profile is only created on stop.
return func() {
f, err := os.Create(heapProfile)
if err != nil {
bklog.G(ctx).Warn("could not create memory profile", logrus.WithError(err))
return
}
// get up-to-date statistics
runtime.GC()
if err := pprof.WriteHeapProfile(f); err != nil {
bklog.G(ctx).Warn("could not write memory profile", logrus.WithError(err))
}
if err := f.Close(); err != nil {
bklog.G(ctx).Warn("could not close file for memory profile", logrus.WithError(err))
}
}
}
return nil
}

View File

@@ -4,10 +4,8 @@ import (
"context" "context"
"fmt" "fmt"
"os" "os"
"path/filepath"
"github.com/docker/buildx/commands" "github.com/docker/buildx/commands"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/util/desktop" "github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version" "github.com/docker/buildx/version"
"github.com/docker/cli/cli" "github.com/docker/cli/cli"
@@ -18,21 +16,23 @@ import (
cliflags "github.com/docker/cli/cli/flags" cliflags "github.com/docker/cli/cli/flags"
"github.com/moby/buildkit/solver/errdefs" "github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack" "github.com/moby/buildkit/util/stack"
"github.com/pkg/errors"
"go.opentelemetry.io/otel" "go.opentelemetry.io/otel"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc" _ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "github.com/docker/buildx/driver/docker" _ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container" _ "github.com/docker/buildx/driver/docker-container"
_ "github.com/docker/buildx/driver/kubernetes" _ "github.com/docker/buildx/driver/kubernetes"
_ "github.com/docker/buildx/driver/remote" _ "github.com/docker/buildx/driver/remote"
// Use custom grpc codec to utilize vtprotobuf
_ "github.com/moby/buildkit/util/grpcutil/encoding/proto"
) )
func init() { func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision) stack.SetVersionInfo(version.Version, version.Revision)
} }
@@ -42,8 +42,7 @@ func runStandalone(cmd *command.DockerCli) error {
} }
defer flushMetrics(cmd) defer flushMetrics(cmd)
executable := os.Args[0] rootCmd := commands.NewRootCmd(os.Args[0], false, cmd)
rootCmd := commands.NewRootCmd(filepath.Base(executable), false, cmd)
return rootCmd.Execute() return rootCmd.Execute()
} }
@@ -71,16 +70,6 @@ func runPlugin(cmd *command.DockerCli) error {
}) })
} }
func run(cmd *command.DockerCli) error {
stopProfiles := setupDebugProfiles(context.TODO())
defer stopProfiles()
if plugin.RunningStandalone() {
return runStandalone(cmd)
}
return runPlugin(cmd)
}
func main() { func main() {
cmd, err := command.NewDockerCli() cmd, err := command.NewDockerCli()
if err != nil { if err != nil {
@@ -88,11 +77,15 @@ func main() {
os.Exit(1) os.Exit(1)
} }
if err = run(cmd); err == nil { if plugin.RunningStandalone() {
err = runStandalone(cmd)
} else {
err = runPlugin(cmd)
}
if err == nil {
return return
} }
// Check the error from the run function above.
if sterr, ok := err.(cli.StatusError); ok { if sterr, ok := err.(cli.StatusError); ok {
if sterr.Status != "" { if sterr.Status != "" {
fmt.Fprintln(cmd.Err(), sterr.Status) fmt.Fprintln(cmd.Err(), sterr.Status)
@@ -113,15 +106,8 @@ func main() {
} else { } else {
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err) fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
} }
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
var ebr *desktop.ErrorWithBuildRef
if errors.As(err, &ebr) {
ebr.Print(cmd.Err()) ebr.Print(cmd.Err())
} else {
var be *controllererrors.BuildError
if errors.As(err, &be) {
be.PrintBuildDetails(cmd.Err())
}
} }
os.Exit(1) os.Exit(1)

View File

@@ -25,6 +25,7 @@ import (
"github.com/docker/buildx/controller/pb" "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/localstate" "github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/buildflags" "github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion" "github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil" "github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop" "github.com/docker/buildx/util/desktop"
@@ -37,40 +38,30 @@ import (
"github.com/moby/buildkit/util/progress/progressui" "github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"github.com/tonistiigi/go-csvvalue"
"go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/attribute"
) )
type bakeOptions struct { type bakeOptions struct {
files []string files []string
overrides []string overrides []string
printOnly bool
sbom string listTargets bool
provenance string listVars bool
allow []string sbom string
provenance string
allow []string
builder string builder string
metadataFile string metadataFile string
exportPush bool exportPush bool
exportLoad bool exportLoad bool
callFunc string callFunc string
print bool
list string
// TODO: remove deprecated flags
listTargets bool
listVars bool
} }
func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) { func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
mp := dockerCli.MeterProvider() mp := dockerCli.MeterProvider()
ctx, end, err := tracing.TraceCurrentCommand(ctx, append([]string{"bake"}, targets...), ctx, end, err := tracing.TraceCurrentCommand(ctx, "bake")
attribute.String("builder", in.builder),
attribute.StringSlice("targets", targets),
attribute.StringSlice("files", in.files),
)
if err != nil { if err != nil {
return err return err
} }
@@ -116,27 +107,16 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err != nil { if err != nil {
return err return err
} }
wd, err := os.Getwd()
if err != nil {
return errors.Wrapf(err, "failed to get current working directory")
}
// filesystem access under the current working directory is allowed by default
ent.FSRead = append(ent.FSRead, wd)
ent.FSWrite = append(ent.FSWrite, wd)
ctx2, cancel := context.WithCancelCause(context.TODO()) ctx2, cancel := context.WithCancel(context.TODO())
defer cancel(errors.WithStack(context.Canceled)) defer cancel()
var nodes []builder.Node var nodes []builder.Node
var progressConsoleDesc, progressTextDesc string var progressConsoleDesc, progressTextDesc string
if in.print && in.list != "" {
return errors.New("--print and --list are mutually exclusive")
}
// instance only needed for reading remote bake files or building // instance only needed for reading remote bake files or building
var driverType string var driverType string
if url != "" || !(in.print || in.list != "") { if url != "" || !in.printOnly {
b, err := builder.New(dockerCli, b, err := builder.New(dockerCli,
builder.WithName(in.builder), builder.WithName(in.builder),
builder.WithContextPathHash(contextPathHash), builder.WithContextPathHash(contextPathHash),
@@ -197,7 +177,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
"BAKE_LOCAL_PLATFORM": platforms.Format(platforms.DefaultSpec()), "BAKE_LOCAL_PLATFORM": platforms.Format(platforms.DefaultSpec()),
} }
if in.list != "" { if in.listTargets || in.listVars {
cfg, pm, err := bake.ParseFiles(files, defaults) cfg, pm, err := bake.ParseFiles(files, defaults)
if err != nil { if err != nil {
return err return err
@@ -205,19 +185,14 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err = printer.Wait(); err != nil { if err = printer.Wait(); err != nil {
return err return err
} }
list, err := parseList(in.list) if in.listTargets {
if err != nil { return printTargetList(dockerCli.Out(), cfg)
return err } else if in.listVars {
} return printVars(dockerCli.Out(), pm.AllVariables)
switch list.Type {
case "targets":
return printTargetList(dockerCli.Out(), list.Format, cfg)
case "variables":
return printVars(dockerCli.Out(), list.Format, pm.AllVariables)
} }
} }
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, defaults, &ent) tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, defaults)
if err != nil { if err != nil {
return err return err
} }
@@ -249,7 +224,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
Target: tgts, Target: tgts,
} }
if in.print { if in.printOnly {
if err = printer.Wait(); err != nil { if err = printer.Wait(); err != nil {
return err return err
} }
@@ -275,10 +250,8 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err != nil { if err != nil {
return err return err
} }
if progressMode != progressui.RawJSONMode { if err := exp.Prompt(ctx, &syncWriter{w: dockerCli.Err(), wait: printer.Wait}); err != nil {
if err := exp.Prompt(ctx, url != "", &syncWriter{w: dockerCli.Err(), wait: printer.Wait}); err != nil { return err
return err
}
} }
if printer.IsDone() { if printer.IsDone() {
// init new printer as old one was stopped to show the prompt // init new printer as old one was stopped to show the prompt
@@ -287,12 +260,12 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
} }
} }
if err := saveLocalStateGroup(dockerCli, in, targets, bo); err != nil { if err := saveLocalStateGroup(dockerCli, in, targets, bo, overrides, def); err != nil {
return err return err
} }
done := timeBuildCommand(mp, attributes) done := timeBuildCommand(mp, attributes)
resp, retErr := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), printer) resp, retErr := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), printer)
if err := printer.Wait(); retErr == nil { if err := printer.Wait(); retErr == nil {
retErr = err retErr = err
} }
@@ -309,7 +282,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term) desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term)
} }
if len(in.metadataFile) > 0 { if len(in.metadataFile) > 0 {
dt := make(map[string]any) dt := make(map[string]interface{})
for t, r := range resp { for t, r := range resp {
dt[t] = decodeExporterResponse(r.ExporterResponse) dt[t] = decodeExporterResponse(r.ExporterResponse)
} }
@@ -362,7 +335,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if callFormatJSON { if callFormatJSON {
jsonResults[name] = map[string]any{} jsonResults[name] = map[string]any{}
buf := &bytes.Buffer{} buf := &bytes.Buffer{}
if code, err := printResult(buf, pf, res, name, &req.Inputs); err != nil { if code, err := printResult(buf, pf, res); err != nil {
jsonResults[name]["error"] = err.Error() jsonResults[name]["error"] = err.Error()
exitCode = 1 exitCode = 1
} else if code != 0 && exitCode == 0 { } else if code != 0 && exitCode == 0 {
@@ -388,7 +361,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
} }
fmt.Fprintln(dockerCli.Out()) fmt.Fprintln(dockerCli.Out())
if code, err := printResult(dockerCli.Out(), pf, res, name, &req.Inputs); err != nil { if code, err := printResult(dockerCli.Out(), pf, res); err != nil {
fmt.Fprintf(dockerCli.Out(), "error: %v\n", err) fmt.Fprintf(dockerCli.Out(), "error: %v\n", err)
exitCode = 1 exitCode = 1
} else if code != 0 && exitCode == 0 { } else if code != 0 && exitCode == 0 {
@@ -447,13 +420,6 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
if !cmd.Flags().Lookup("pull").Changed { if !cmd.Flags().Lookup("pull").Changed {
cFlags.pull = nil cFlags.pull = nil
} }
if options.list == "" {
if options.listTargets {
options.list = "targets"
} else if options.listVars {
options.list = "variables"
}
}
options.builder = rootOpts.builder options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function. // Other common flags (noCache, pull and progress) are processed in runBake function.
@@ -466,6 +432,7 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.StringArrayVarP(&options.files, "file", "f", []string{}, "Build definition file") flags.StringArrayVarP(&options.files, "file", "f", []string{}, "Build definition file")
flags.BoolVar(&options.exportLoad, "load", false, `Shorthand for "--set=*.output=type=docker"`) flags.BoolVar(&options.exportLoad, "load", false, `Shorthand for "--set=*.output=type=docker"`)
flags.BoolVar(&options.printOnly, "print", false, "Print the options without building")
flags.BoolVar(&options.exportPush, "push", false, `Shorthand for "--set=*.output=type=registry"`) flags.BoolVar(&options.exportPush, "push", false, `Shorthand for "--set=*.output=type=registry"`)
flags.StringVar(&options.sbom, "sbom", "", `Shorthand for "--set=*.attest=type=sbom"`) flags.StringVar(&options.sbom, "sbom", "", `Shorthand for "--set=*.attest=type=sbom"`)
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`) flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
@@ -476,30 +443,20 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.VarPF(callAlias(&options.callFunc, "check"), "check", "", `Shorthand for "--call=check"`) flags.VarPF(callAlias(&options.callFunc, "check"), "check", "", `Shorthand for "--call=check"`)
flags.Lookup("check").NoOptDefVal = "true" flags.Lookup("check").NoOptDefVal = "true"
flags.BoolVar(&options.print, "print", false, "Print the options without building")
flags.StringVar(&options.list, "list", "", "List targets or variables")
// TODO: remove deprecated flags
flags.BoolVar(&options.listTargets, "list-targets", false, "List available targets") flags.BoolVar(&options.listTargets, "list-targets", false, "List available targets")
cobrautil.MarkFlagsExperimental(flags, "list-targets")
flags.MarkHidden("list-targets") flags.MarkHidden("list-targets")
flags.MarkDeprecated("list-targets", "list-targets is deprecated, use list=targets instead")
flags.BoolVar(&options.listVars, "list-variables", false, "List defined variables") flags.BoolVar(&options.listVars, "list-variables", false, "List defined variables")
cobrautil.MarkFlagsExperimental(flags, "list-variables")
flags.MarkHidden("list-variables") flags.MarkHidden("list-variables")
flags.MarkDeprecated("list-variables", "list-variables is deprecated, use list=variables instead")
commonBuildFlags(&cFlags, flags) commonBuildFlags(&cFlags, flags)
return cmd return cmd
} }
func saveLocalStateGroup(dockerCli command.Cli, in bakeOptions, targets []string, bo map[string]build.Options) error { func saveLocalStateGroup(dockerCli command.Cli, in bakeOptions, targets []string, bo map[string]build.Options, overrides []string, def any) error {
l, err := localstate.New(confutil.NewConfig(dockerCli))
if err != nil {
return err
}
defer l.MigrateIfNeeded()
prm := confutil.MetadataProvenance() prm := confutil.MetadataProvenance()
if len(in.metadataFile) == 0 { if len(in.metadataFile) == 0 {
prm = confutil.MetadataProvenanceModeDisabled prm = confutil.MetadataProvenanceModeDisabled
@@ -507,22 +464,25 @@ func saveLocalStateGroup(dockerCli command.Cli, in bakeOptions, targets []string
groupRef := identity.NewID() groupRef := identity.NewID()
refs := make([]string, 0, len(bo)) refs := make([]string, 0, len(bo))
for k, b := range bo { for k, b := range bo {
if b.CallFunc != nil {
continue
}
b.Ref = identity.NewID() b.Ref = identity.NewID()
b.GroupRef = groupRef b.GroupRef = groupRef
b.ProvenanceResponseMode = prm b.ProvenanceResponseMode = prm
refs = append(refs, b.Ref) refs = append(refs, b.Ref)
bo[k] = b bo[k] = b
} }
if len(refs) == 0 { l, err := localstate.New(confutil.ConfigDir(dockerCli))
return nil if err != nil {
return err
}
dtdef, err := json.MarshalIndent(def, "", " ")
if err != nil {
return err
} }
return l.SaveGroup(groupRef, localstate.StateGroup{ return l.SaveGroup(groupRef, localstate.StateGroup{
Refs: refs, Definition: dtdef,
Targets: targets, Targets: targets,
Inputs: overrides,
Refs: refs,
}) })
} }
@@ -584,70 +544,10 @@ func readBakeFiles(ctx context.Context, nodes []builder.Node, url string, names
return return
} }
type listEntry struct { func printVars(w io.Writer, vars []*hclparser.Variable) error {
Type string
Format string
}
func parseList(input string) (listEntry, error) {
res := listEntry{}
fields, err := csvvalue.Fields(input, nil)
if err != nil {
return res, err
}
if len(fields) == 1 && fields[0] == input && !strings.HasPrefix(input, "type=") {
res.Type = input
}
if res.Type == "" {
for _, field := range fields {
key, value, ok := strings.Cut(field, "=")
if !ok {
return res, errors.Errorf("invalid value %s", field)
}
key = strings.TrimSpace(strings.ToLower(key))
switch key {
case "type":
res.Type = value
case "format":
res.Format = value
default:
return res, errors.Errorf("unexpected key '%s' in '%s'", key, field)
}
}
}
if res.Format == "" {
res.Format = "table"
}
switch res.Type {
case "targets", "variables":
default:
return res, errors.Errorf("invalid list type %q", res.Type)
}
switch res.Format {
case "table", "json":
default:
return res, errors.Errorf("invalid list format %q", res.Format)
}
return res, nil
}
func printVars(w io.Writer, format string, vars []*hclparser.Variable) error {
slices.SortFunc(vars, func(a, b *hclparser.Variable) int { slices.SortFunc(vars, func(a, b *hclparser.Variable) int {
return cmp.Compare(a.Name, b.Name) return cmp.Compare(a.Name, b.Name)
}) })
if format == "json" {
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
return enc.Encode(vars)
}
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0) tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush() defer tw.Flush()
@@ -665,7 +565,12 @@ func printVars(w io.Writer, format string, vars []*hclparser.Variable) error {
return nil return nil
} }
func printTargetList(w io.Writer, format string, cfg *bake.Config) error { func printTargetList(w io.Writer, cfg *bake.Config) error {
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush()
tw.Write([]byte("TARGET\tDESCRIPTION\n"))
type targetOrGroup struct { type targetOrGroup struct {
name string name string
target *bake.Target target *bake.Target
@@ -684,20 +589,6 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
return cmp.Compare(a.name, b.name) return cmp.Compare(a.name, b.name)
}) })
var tw *tabwriter.Writer
if format == "table" {
tw = tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush()
tw.Write([]byte("TARGET\tDESCRIPTION\n"))
}
type targetList struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Group bool `json:"group,omitempty"`
}
var targetsList []targetList
for _, tgt := range list { for _, tgt := range list {
if strings.HasPrefix(tgt.name, "_") { if strings.HasPrefix(tgt.name, "_") {
// convention for a private target // convention for a private target
@@ -706,9 +597,9 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
var descr string var descr string
if tgt.target != nil { if tgt.target != nil {
descr = tgt.target.Description descr = tgt.target.Description
targetsList = append(targetsList, targetList{Name: tgt.name, Description: descr})
} else if tgt.group != nil { } else if tgt.group != nil {
descr = tgt.group.Description descr = tgt.group.Description
if len(tgt.group.Targets) > 0 { if len(tgt.group.Targets) > 0 {
slices.Sort(tgt.group.Targets) slices.Sort(tgt.group.Targets)
names := strings.Join(tgt.group.Targets, ", ") names := strings.Join(tgt.group.Targets, ", ")
@@ -718,17 +609,8 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
descr = names descr = names
} }
} }
targetsList = append(targetsList, targetList{Name: tgt.name, Description: descr, Group: true})
} }
if format == "table" { fmt.Fprintf(tw, "%s\t%s\n", tgt.name, descr)
fmt.Fprintf(tw, "%s\t%s\n", tgt.name, descr)
}
}
if format == "json" {
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
return enc.Encode(targetsList)
} }
return nil return nil
@@ -739,7 +621,7 @@ func bakeMetricAttributes(dockerCli command.Cli, driverType, url, cmdContext str
commandNameAttribute.String("bake"), commandNameAttribute.String("bake"),
attribute.Stringer(string(commandOptionsHash), &bakeOptionsHash{ attribute.Stringer(string(commandOptionsHash), &bakeOptionsHash{
bakeOptions: options, bakeOptions: options,
cfg: confutil.NewConfig(dockerCli), configDir: confutil.ConfigDir(dockerCli),
url: url, url: url,
cmdContext: cmdContext, cmdContext: cmdContext,
targets: targets, targets: targets,
@@ -751,7 +633,7 @@ func bakeMetricAttributes(dockerCli command.Cli, driverType, url, cmdContext str
type bakeOptionsHash struct { type bakeOptionsHash struct {
*bakeOptions *bakeOptions
cfg *confutil.Config configDir string
url string url string
cmdContext string cmdContext string
targets []string targets []string
@@ -775,7 +657,7 @@ func (o *bakeOptionsHash) String() string {
joinedFiles := strings.Join(files, ",") joinedFiles := strings.Join(files, ",")
joinedTargets := strings.Join(targets, ",") joinedTargets := strings.Join(targets, ",")
salt := o.cfg.TryNodeIdentifier() salt := confutil.TryNodeIdentifier(o.configDir)
h := sha256.New() h := sha256.New()
for _, s := range []string{url, cmdContext, joinedFiles, joinedTargets, salt} { for _, s := range []string{url, cmdContext, joinedFiles, joinedTargets, salt} {

View File

@@ -11,7 +11,6 @@ import (
"io" "io"
"os" "os"
"path/filepath" "path/filepath"
"slices"
"strconv" "strconv"
"strings" "strings"
"sync" "sync"
@@ -42,7 +41,7 @@ import (
"github.com/docker/cli/cli/command" "github.com/docker/cli/cli/command"
dockeropts "github.com/docker/cli/opts" dockeropts "github.com/docker/cli/opts"
"github.com/docker/docker/api/types/versions" "github.com/docker/docker/api/types/versions"
"github.com/docker/docker/pkg/atomicwriter" "github.com/docker/docker/pkg/ioutils"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
"github.com/moby/buildkit/exporter/containerimage/exptypes" "github.com/moby/buildkit/exporter/containerimage/exptypes"
"github.com/moby/buildkit/frontend/subrequests" "github.com/moby/buildkit/frontend/subrequests"
@@ -50,7 +49,6 @@ import (
"github.com/moby/buildkit/frontend/subrequests/outline" "github.com/moby/buildkit/frontend/subrequests/outline"
"github.com/moby/buildkit/frontend/subrequests/targets" "github.com/moby/buildkit/frontend/subrequests/targets"
"github.com/moby/buildkit/solver/errdefs" "github.com/moby/buildkit/solver/errdefs"
solverpb "github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/grpcerrors" "github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/progress/progressui" "github.com/moby/buildkit/util/progress/progressui"
"github.com/morikuni/aec" "github.com/morikuni/aec"
@@ -62,7 +60,6 @@ import (
"go.opentelemetry.io/otel/attribute" "go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/metric" "go.opentelemetry.io/otel/metric"
"google.golang.org/grpc/codes" "google.golang.org/grpc/codes"
"google.golang.org/protobuf/proto"
) )
type buildOptions struct { type buildOptions struct {
@@ -157,7 +154,7 @@ func (o *buildOptions) toControllerOptions() (*controllerapi.BuildOptions, error
return nil, err return nil, err
} }
inAttests := slices.Clone(o.attests) inAttests := append([]string{}, o.attests...)
if o.provenance != "" { if o.provenance != "" {
inAttests = append(inAttests, buildflags.CanonicalizeAttest("provenance", o.provenance)) inAttests = append(inAttests, buildflags.CanonicalizeAttest("provenance", o.provenance))
} }
@@ -184,17 +181,14 @@ func (o *buildOptions) toControllerOptions() (*controllerapi.BuildOptions, error
} }
} }
cacheFrom, err := buildflags.ParseCacheEntry(o.cacheFrom) opts.CacheFrom, err = buildflags.ParseCacheEntry(o.cacheFrom)
if err != nil { if err != nil {
return nil, err return nil, err
} }
opts.CacheFrom = cacheFrom.ToPB() opts.CacheTo, err = buildflags.ParseCacheEntry(o.cacheTo)
cacheTo, err := buildflags.ParseCacheEntry(o.cacheTo)
if err != nil { if err != nil {
return nil, err return nil, err
} }
opts.CacheTo = cacheTo.ToPB()
opts.Secrets, err = buildflags.ParseSecretSpecs(o.secrets) opts.Secrets, err = buildflags.ParseSecretSpecs(o.secrets)
if err != nil { if err != nil {
@@ -242,7 +236,7 @@ func buildMetricAttributes(dockerCli command.Cli, driverType string, options *bu
commandNameAttribute.String("build"), commandNameAttribute.String("build"),
attribute.Stringer(string(commandOptionsHash), &buildOptionsHash{ attribute.Stringer(string(commandOptionsHash), &buildOptionsHash{
buildOptions: options, buildOptions: options,
cfg: confutil.NewConfig(dockerCli), configDir: confutil.ConfigDir(dockerCli),
}), }),
driverNameAttribute.String(options.builder), driverNameAttribute.String(options.builder),
driverTypeAttribute.String(driverType), driverTypeAttribute.String(driverType),
@@ -254,7 +248,7 @@ func buildMetricAttributes(dockerCli command.Cli, driverType string, options *bu
// the fmt.Stringer interface. // the fmt.Stringer interface.
type buildOptionsHash struct { type buildOptionsHash struct {
*buildOptions *buildOptions
cfg *confutil.Config configDir string
result string result string
resultOnce sync.Once resultOnce sync.Once
} }
@@ -271,7 +265,7 @@ func (o *buildOptionsHash) String() string {
if contextPath != "-" && osutil.IsLocalDir(contextPath) { if contextPath != "-" && osutil.IsLocalDir(contextPath) {
contextPath = osutil.ToAbs(contextPath) contextPath = osutil.ToAbs(contextPath)
} }
salt := o.cfg.TryNodeIdentifier() salt := confutil.TryNodeIdentifier(o.configDir)
h := sha256.New() h := sha256.New()
for _, s := range []string{target, contextPath, dockerfile, salt} { for _, s := range []string{target, contextPath, dockerfile, salt} {
@@ -286,11 +280,7 @@ func (o *buildOptionsHash) String() string {
func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions) (err error) { func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions) (err error) {
mp := dockerCli.MeterProvider() mp := dockerCli.MeterProvider()
ctx, end, err := tracing.TraceCurrentCommand(ctx, []string{"build", options.contextPath}, ctx, end, err := tracing.TraceCurrentCommand(ctx, "build")
attribute.String("builder", options.builder),
attribute.String("context", options.contextPath),
attribute.String("dockerfile", options.dockerfileName),
)
if err != nil { if err != nil {
return err return err
} }
@@ -333,8 +323,8 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
} }
attributes := buildMetricAttributes(dockerCli, driverType, &options) attributes := buildMetricAttributes(dockerCli, driverType, &options)
ctx2, cancel := context.WithCancelCause(context.TODO()) ctx2, cancel := context.WithCancel(context.TODO())
defer func() { cancel(errors.WithStack(context.Canceled)) }() defer cancel()
progressMode, err := options.toDisplayMode() progressMode, err := options.toDisplayMode()
if err != nil { if err != nil {
return err return err
@@ -356,12 +346,11 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
done := timeBuildCommand(mp, attributes) done := timeBuildCommand(mp, attributes)
var resp *client.SolveResponse var resp *client.SolveResponse
var inputs *build.Inputs
var retErr error var retErr error
if confutil.IsExperimental() { if confutil.IsExperimental() {
resp, inputs, retErr = runControllerBuild(ctx, dockerCli, opts, options, printer) resp, retErr = runControllerBuild(ctx, dockerCli, opts, options, printer)
} else { } else {
resp, inputs, retErr = runBasicBuild(ctx, dockerCli, opts, printer) resp, retErr = runBasicBuild(ctx, dockerCli, opts, printer)
} }
if err := printer.Wait(); retErr == nil { if err := printer.Wait(); retErr == nil {
@@ -398,7 +387,7 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
} }
} }
if opts.CallFunc != nil { if opts.CallFunc != nil {
if exitcode, err := printResult(dockerCli.Out(), opts.CallFunc, resp.ExporterResponse, options.target, inputs); err != nil { if exitcode, err := printResult(dockerCli.Out(), opts.CallFunc, resp.ExporterResponse); err != nil {
return err return err
} else if exitcode != 0 { } else if exitcode != 0 {
os.Exit(exitcode) os.Exit(exitcode)
@@ -416,22 +405,22 @@ func getImageID(resp map[string]string) string {
return dgst return dgst
} }
func runBasicBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, printer *progress.Printer) (*client.SolveResponse, *build.Inputs, error) { func runBasicBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, printer *progress.Printer) (*client.SolveResponse, error) {
resp, res, dfmap, err := cbuild.RunBuild(ctx, dockerCli, opts, dockerCli.In(), printer, false) resp, res, err := cbuild.RunBuild(ctx, dockerCli, *opts, dockerCli.In(), printer, false)
if res != nil { if res != nil {
res.Done() res.Done()
} }
return resp, dfmap, err return resp, err
} }
func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, options buildOptions, printer *progress.Printer) (*client.SolveResponse, *build.Inputs, error) { func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, options buildOptions, printer *progress.Printer) (*client.SolveResponse, error) {
if options.invokeConfig != nil && (options.dockerfileName == "-" || options.contextPath == "-") { if options.invokeConfig != nil && (options.dockerfileName == "-" || options.contextPath == "-") {
// stdin must be usable for monitor // stdin must be usable for monitor
return nil, nil, errors.Errorf("Dockerfile or context from stdin is not supported with invoke") return nil, errors.Errorf("Dockerfile or context from stdin is not supported with invoke")
} }
c, err := controller.NewController(ctx, options.ControlOptions, dockerCli, printer) c, err := controller.NewController(ctx, options.ControlOptions, dockerCli, printer)
if err != nil { if err != nil {
return nil, nil, err return nil, err
} }
defer func() { defer func() {
if err := c.Close(); err != nil { if err := c.Close(); err != nil {
@@ -443,13 +432,12 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
// so we need to resolve paths to abosolute ones in the client. // so we need to resolve paths to abosolute ones in the client.
opts, err = controllerapi.ResolveOptionPaths(opts) opts, err = controllerapi.ResolveOptionPaths(opts)
if err != nil { if err != nil {
return nil, nil, err return nil, err
} }
var ref string var ref string
var retErr error var retErr error
var resp *client.SolveResponse var resp *client.SolveResponse
var inputs *build.Inputs
var f *ioset.SingleForwarder var f *ioset.SingleForwarder
var pr io.ReadCloser var pr io.ReadCloser
@@ -467,15 +455,15 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
}) })
} }
ref, resp, inputs, err = c.Build(ctx, opts, pr, printer) ref, resp, err = c.Build(ctx, *opts, pr, printer)
if err != nil { if err != nil {
var be *controllererrors.BuildError var be *controllererrors.BuildError
if errors.As(err, &be) { if errors.As(err, &be) {
ref = be.SessionID ref = be.Ref
retErr = err retErr = err
// We can proceed to monitor // We can proceed to monitor
} else { } else {
return nil, nil, errors.Wrapf(err, "failed to build") return nil, errors.Wrapf(err, "failed to build")
} }
} }
@@ -516,7 +504,7 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
} }
} }
return resp, inputs, retErr return resp, retErr
} }
func printError(err error, printer *progress.Printer) error { func printError(err error, printer *progress.Printer) error {
@@ -598,7 +586,7 @@ func buildCmd(dockerCli command.Cli, rootOpts *rootOptions, debugConfig *debug.D
flags.StringSliceVar(&options.extraHosts, "add-host", []string{}, `Add a custom host-to-IP mapping (format: "host:ip")`) flags.StringSliceVar(&options.extraHosts, "add-host", []string{}, `Add a custom host-to-IP mapping (format: "host:ip")`)
flags.StringArrayVar(&options.allow, "allow", []string{}, `Allow extra privileged entitlement (e.g., "network.host", "security.insecure")`) flags.StringSliceVar(&options.allow, "allow", []string{}, `Allow extra privileged entitlement (e.g., "network.host", "security.insecure")`)
flags.StringArrayVarP(&options.annotations, "annotation", "", []string{}, "Add annotation to the image") flags.StringArrayVarP(&options.annotations, "annotation", "", []string{}, "Add annotation to the image")
@@ -728,7 +716,7 @@ type commonFlags struct {
func commonBuildFlags(options *commonFlags, flags *pflag.FlagSet) { func commonBuildFlags(options *commonFlags, flags *pflag.FlagSet) {
options.noCache = flags.Bool("no-cache", false, "Do not use cache when building the image") options.noCache = flags.Bool("no-cache", false, "Do not use cache when building the image")
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "quiet", "plain", "tty", "rawjson"). Use plain to show container output`) flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "plain", "tty", "rawjson"). Use plain to show container output`)
options.pull = flags.Bool("pull", false, "Always attempt to pull all referenced images") options.pull = flags.Bool("pull", false, "Always attempt to pull all referenced images")
flags.StringVar(&options.metadataFile, "metadata-file", "", "Write build result metadata to a file") flags.StringVar(&options.metadataFile, "metadata-file", "", "Write build result metadata to a file")
} }
@@ -745,15 +733,15 @@ func checkWarnedFlags(f *pflag.Flag) {
} }
} }
func writeMetadataFile(filename string, dt any) error { func writeMetadataFile(filename string, dt interface{}) error {
b, err := json.MarshalIndent(dt, "", " ") b, err := json.MarshalIndent(dt, "", " ")
if err != nil { if err != nil {
return err return err
} }
return atomicwriter.WriteFile(filename, b, 0644) return ioutils.AtomicWriteFile(filename, b, 0644)
} }
func decodeExporterResponse(exporterResponse map[string]string) map[string]any { func decodeExporterResponse(exporterResponse map[string]string) map[string]interface{} {
decFunc := func(k, v string) ([]byte, error) { decFunc := func(k, v string) ([]byte, error) {
if k == "result.json" { if k == "result.json" {
// result.json is part of metadata response for subrequests which // result.json is part of metadata response for subrequests which
@@ -762,20 +750,17 @@ func decodeExporterResponse(exporterResponse map[string]string) map[string]any {
} }
return base64.StdEncoding.DecodeString(v) return base64.StdEncoding.DecodeString(v)
} }
out := make(map[string]any) out := make(map[string]interface{})
for k, v := range exporterResponse { for k, v := range exporterResponse {
dt, err := decFunc(k, v) dt, err := decFunc(k, v)
if err != nil { if err != nil {
out[k] = v out[k] = v
continue continue
} }
var raw map[string]any var raw map[string]interface{}
if err = json.Unmarshal(dt, &raw); err != nil || len(raw) == 0 { if err = json.Unmarshal(dt, &raw); err != nil || len(raw) == 0 {
var rawList []map[string]any out[k] = v
if err = json.Unmarshal(dt, &rawList); err != nil || len(rawList) == 0 { continue
out[k] = v
continue
}
} }
out[k] = json.RawMessage(dt) out[k] = json.RawMessage(dt)
} }
@@ -893,10 +878,11 @@ func printWarnings(w io.Writer, warnings []client.VertexWarning, mode progressui
src.Print(w) src.Print(w)
} }
fmt.Fprintf(w, "\n") fmt.Fprintf(w, "\n")
} }
} }
func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string, target string, inp *build.Inputs) (int, error) { func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string) (int, error) {
switch f.Name { switch f.Name {
case "outline": case "outline":
return 0, printValue(w, outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res) return 0, printValue(w, outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res)
@@ -922,27 +908,8 @@ func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string,
} }
fmt.Fprintf(w, "Check complete, %s\n", warningCountMsg) fmt.Fprintf(w, "Check complete, %s\n", warningCountMsg)
} }
sourceInfoMap := func(sourceInfo *solverpb.SourceInfo) *solverpb.SourceInfo {
if sourceInfo == nil || inp == nil {
return sourceInfo
}
if target == "" {
target = "default"
}
if inp.DockerfileMappingSrc != "" { err := printValue(w, printLintViolationsWrapper, lint.SubrequestLintDefinition.Version, f.Format, res)
newSourceInfo := proto.Clone(sourceInfo).(*solverpb.SourceInfo)
newSourceInfo.Filename = inp.DockerfileMappingSrc
return newSourceInfo
}
return sourceInfo
}
printLintWarnings := func(dt []byte, w io.Writer) error {
return lintResults.PrintTo(w, sourceInfoMap)
}
err := printValue(w, printLintWarnings, lint.SubrequestLintDefinition.Version, f.Format, res)
if err != nil { if err != nil {
return 0, err return 0, err
} }
@@ -957,8 +924,13 @@ func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string,
if f.Format != "json" && len(lintResults.Warnings) > 0 { if f.Format != "json" && len(lintResults.Warnings) > 0 {
fmt.Fprintln(w) fmt.Fprintln(w)
} }
lintBuf := bytes.NewBuffer(nil) lintBuf := bytes.NewBuffer([]byte(lintResults.Error.Message + "\n"))
lintResults.PrintErrorTo(lintBuf, sourceInfoMap) sourceInfo := lintResults.Sources[lintResults.Error.Location.SourceIndex]
source := errdefs.Source{
Info: sourceInfo,
Ranges: lintResults.Error.Location.Ranges,
}
source.Print(lintBuf)
return 0, errors.New(lintBuf.String()) return 0, errors.New(lintBuf.String())
} else if len(lintResults.Warnings) == 0 && f.Format != "json" { } else if len(lintResults.Warnings) == 0 && f.Format != "json" {
fmt.Fprintln(w, "Check complete, no warnings found.") fmt.Fprintln(w, "Check complete, no warnings found.")
@@ -996,6 +968,11 @@ func printValue(w io.Writer, printer callFunc, version string, format string, re
return printer([]byte(res["result.json"]), w) return printer([]byte(res["result.json"]), w)
} }
// FIXME: remove once https://github.com/docker/buildx/pull/2672 is sorted
func printLintViolationsWrapper(dt []byte, w io.Writer) error {
return lint.PrintLintViolations(dt, w, nil)
}
type invokeConfig struct { type invokeConfig struct {
controllerapi.InvokeConfig controllerapi.InvokeConfig
onFlag string onFlag string
@@ -1023,7 +1000,7 @@ func (cfg *invokeConfig) runDebug(ctx context.Context, ref string, options *cont
return nil, errors.Errorf("failed to configure terminal: %v", err) return nil, errors.Errorf("failed to configure terminal: %v", err)
} }
defer con.Reset() defer con.Reset()
return monitor.RunMonitor(ctx, ref, options, &cfg.InvokeConfig, c, stdin, stdout, stderr, progress) return monitor.RunMonitor(ctx, ref, options, cfg.InvokeConfig, c, stdin, stdout, stderr, progress)
} }
func (cfg *invokeConfig) parseInvokeConfig(invoke, on string) error { func (cfg *invokeConfig) parseInvokeConfig(invoke, on string) error {

View File

@@ -64,7 +64,7 @@ func RootCmd(dockerCli command.Cli, children ...DebuggableCmd) *cobra.Command {
return errors.Errorf("failed to configure terminal: %v", err) return errors.Errorf("failed to configure terminal: %v", err)
} }
_, err = monitor.RunMonitor(ctx, "", nil, &controllerapi.InvokeConfig{ _, err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true, Tty: true,
}, c, dockerCli.In(), os.Stdout, os.Stderr, printer) }, c, dockerCli.In(), os.Stdout, os.Stderr, printer)
con.Reset() con.Reset()

View File

@@ -124,7 +124,7 @@ func duCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
return cmd return cmd
} }
func printKV(w io.Writer, k string, v any) { func printKV(w io.Writer, k string, v interface{}) {
fmt.Fprintf(w, "%s:\t%v\n", k, v) fmt.Fprintf(w, "%s:\t%v\n", k, v)
} }

View File

@@ -1,135 +0,0 @@
package history
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"os"
"strings"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli/command"
"github.com/pkg/browser"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type importOptions struct {
file []string
}
func runImport(ctx context.Context, dockerCli command.Cli, opts importOptions) error {
sock, err := desktop.BuildServerAddr()
if err != nil {
return err
}
tr := http.DefaultTransport.(*http.Transport).Clone()
tr.DialContext = func(ctx context.Context, _, _ string) (net.Conn, error) {
network, addr, ok := strings.Cut(sock, "://")
if !ok {
return nil, errors.Errorf("invalid endpoint address: %s", sock)
}
return remoteutil.DialContext(ctx, network, addr)
}
client := &http.Client{
Transport: tr,
}
var urls []string
if len(opts.file) == 0 {
u, err := importFrom(ctx, client, os.Stdin)
if err != nil {
return err
}
urls = append(urls, u...)
} else {
for _, fn := range opts.file {
var f *os.File
var rdr io.Reader = os.Stdin
if fn != "-" {
f, err = os.Open(fn)
if err != nil {
return errors.Wrapf(err, "failed to open file %s", fn)
}
rdr = f
}
u, err := importFrom(ctx, client, rdr)
if err != nil {
return err
}
urls = append(urls, u...)
if f != nil {
f.Close()
}
}
}
if len(urls) == 0 {
return errors.New("no build records found in the bundle")
}
for i, url := range urls {
fmt.Fprintln(dockerCli.Err(), url)
if i == 0 {
err = browser.OpenURL(url)
}
}
return err
}
func importFrom(ctx context.Context, c *http.Client, rdr io.Reader) ([]string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, "http://docker-desktop/upload", rdr)
if err != nil {
return nil, errors.Wrap(err, "failed to create request")
}
resp, err := c.Do(req)
if err != nil {
return nil, errors.Wrap(err, "failed to send request, check if Docker Desktop is running")
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(resp.Body)
return nil, errors.Errorf("failed to import build: %s", string(body))
}
var refs []string
dec := json.NewDecoder(resp.Body)
if err := dec.Decode(&refs); err != nil {
return nil, errors.Wrap(err, "failed to decode response")
}
var urls []string
for _, ref := range refs {
urls = append(urls, desktop.BuildURL(fmt.Sprintf(".imported/_/%s", ref)))
}
return urls, err
}
func importCmd(dockerCli command.Cli, _ RootOptions) *cobra.Command {
var options importOptions
cmd := &cobra.Command{
Use: "import [OPTIONS] < bundle.dockerbuild",
Short: "Import a build into Docker Desktop",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
return runImport(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringArrayVarP(&options.file, "file", "f", nil, "Import from a file path")
return cmd
}

View File

@@ -1,893 +0,0 @@
package history
import (
"bytes"
"cmp"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"text/tabwriter"
"text/template"
"time"
"github.com/containerd/containerd/v2/core/content"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/containerd/containerd/v2/core/images"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/formatter"
"github.com/docker/cli/cli/debug"
slsa "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/common"
slsa02 "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/v0.2"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/solver/errdefs"
provenancetypes "github.com/moby/buildkit/solver/llbsolver/provenance/types"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/stack"
"github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/tonistiigi/go-csvvalue"
spb "google.golang.org/genproto/googleapis/rpc/status"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
proto "google.golang.org/protobuf/proto"
)
type statusT string
const (
statusComplete statusT = "completed"
statusRunning statusT = "running"
statusError statusT = "failed"
statusCanceled statusT = "canceled"
)
type inspectOptions struct {
builder string
ref string
format string
}
type inspectOutput struct {
Name string `json:",omitempty"`
Ref string
Context string `json:",omitempty"`
Dockerfile string `json:",omitempty"`
VCSRepository string `json:",omitempty"`
VCSRevision string `json:",omitempty"`
Target string `json:",omitempty"`
Platform []string `json:",omitempty"`
KeepGitDir bool `json:",omitempty"`
NamedContexts []keyValueOutput `json:",omitempty"`
StartedAt *time.Time `json:",omitempty"`
CompletedAt *time.Time `json:",omitempty"`
Duration time.Duration `json:",omitempty"`
Status statusT `json:",omitempty"`
Error *errorOutput `json:",omitempty"`
NumCompletedSteps int32
NumTotalSteps int32
NumCachedSteps int32
BuildArgs []keyValueOutput `json:",omitempty"`
Labels []keyValueOutput `json:",omitempty"`
Config configOutput `json:",omitempty"`
Materials []materialOutput `json:",omitempty"`
Attachments []attachmentOutput `json:",omitempty"`
Errors []string `json:",omitempty"`
}
type configOutput struct {
Network string `json:",omitempty"`
ExtraHosts []string `json:",omitempty"`
Hostname string `json:",omitempty"`
CgroupParent string `json:",omitempty"`
ImageResolveMode string `json:",omitempty"`
MultiPlatform bool `json:",omitempty"`
NoCache bool `json:",omitempty"`
NoCacheFilter []string `json:",omitempty"`
ShmSize string `json:",omitempty"`
Ulimit string `json:",omitempty"`
CacheMountNS string `json:",omitempty"`
DockerfileCheckConfig string `json:",omitempty"`
SourceDateEpoch string `json:",omitempty"`
SandboxHostname string `json:",omitempty"`
RestRaw []keyValueOutput `json:",omitempty"`
}
type materialOutput struct {
URI string `json:",omitempty"`
Digests []string `json:",omitempty"`
}
type attachmentOutput struct {
Digest string `json:",omitempty"`
Platform string `json:",omitempty"`
Type string `json:",omitempty"`
}
type errorOutput struct {
Code int `json:",omitempty"`
Message string `json:",omitempty"`
Name string `json:",omitempty"`
Logs []string `json:",omitempty"`
Sources []byte `json:",omitempty"`
Stack []byte `json:",omitempty"`
}
type keyValueOutput struct {
Name string `json:",omitempty"`
Value string `json:",omitempty"`
}
func readAttr[T any](attrs map[string]string, k string, dest *T, f func(v string) (T, bool)) {
if sv, ok := attrs[k]; ok {
if f != nil {
v, ok := f(sv)
if ok {
*dest = v
}
}
if d, ok := any(dest).(*string); ok {
*d = sv
}
}
delete(attrs, k)
}
func runInspect(ctx context.Context, dockerCli command.Cli, opts inspectOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes, nil)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
rec := &recs[0]
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
store := proxy.NewContentStore(c.ContentClient())
var defaultPlatform string
workers, err := c.ListWorkers(ctx)
if err != nil {
return errors.Wrap(err, "failed to list workers")
}
workers0:
for _, w := range workers {
for _, p := range w.Platforms {
defaultPlatform = platforms.FormatAll(platforms.Normalize(p))
break workers0
}
}
ls, err := localstate.New(confutil.NewConfig(dockerCli))
if err != nil {
return err
}
st, _ := ls.ReadRef(rec.node.Builder, rec.node.Name, rec.Ref)
attrs := rec.FrontendAttrs
delete(attrs, "frontend.caps")
var out inspectOutput
var context string
var dockerfile string
if st != nil {
context = st.LocalPath
dockerfile = st.DockerfilePath
wd, _ := os.Getwd()
if dockerfile != "" && dockerfile != "-" {
if rel, err := filepath.Rel(context, dockerfile); err == nil {
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
dockerfile = rel
}
}
}
if context != "" {
if rel, err := filepath.Rel(wd, context); err == nil {
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
context = rel
}
}
}
}
if v, ok := attrs["context"]; ok && context == "" {
delete(attrs, "context")
context = v
}
if dockerfile == "" {
if v, ok := attrs["filename"]; ok {
dockerfile = v
if dfdir, ok := attrs["vcs:localdir:dockerfile"]; ok {
dockerfile = filepath.Join(dfdir, dockerfile)
}
}
}
delete(attrs, "filename")
out.Name = buildName(rec.FrontendAttrs, st)
out.Ref = rec.Ref
out.Context = context
out.Dockerfile = dockerfile
if _, ok := attrs["context"]; !ok {
if src, ok := attrs["vcs:source"]; ok {
out.VCSRepository = src
}
if rev, ok := attrs["vcs:revision"]; ok {
out.VCSRevision = rev
}
}
readAttr(attrs, "target", &out.Target, nil)
readAttr(attrs, "platform", &out.Platform, func(v string) ([]string, bool) {
return tryParseValue(v, &out.Errors, func(v string) ([]string, error) {
var pp []string
for _, v := range strings.Split(v, ",") {
p, err := platforms.Parse(v)
if err != nil {
return nil, err
}
pp = append(pp, platforms.FormatAll(platforms.Normalize(p)))
}
if len(pp) == 0 {
pp = append(pp, defaultPlatform)
}
return pp, nil
})
})
readAttr(attrs, "build-arg:BUILDKIT_CONTEXT_KEEP_GIT_DIR", &out.KeepGitDir, func(v string) (bool, bool) {
return tryParseValue(v, &out.Errors, strconv.ParseBool)
})
out.NamedContexts = readKeyValues(attrs, "context:")
if rec.CreatedAt != nil {
tm := rec.CreatedAt.AsTime().Local()
out.StartedAt = &tm
}
out.Status = statusRunning
if rec.CompletedAt != nil {
tm := rec.CompletedAt.AsTime().Local()
out.CompletedAt = &tm
out.Status = statusComplete
}
if rec.Error != nil || rec.ExternalError != nil {
out.Error = &errorOutput{}
if rec.Error != nil {
if codes.Code(rec.Error.Code) == codes.Canceled {
out.Status = statusCanceled
} else {
out.Status = statusError
}
out.Error.Code = int(codes.Code(rec.Error.Code))
out.Error.Message = rec.Error.Message
}
if rec.ExternalError != nil {
dt, err := content.ReadBlob(ctx, store, ociDesc(rec.ExternalError))
if err != nil {
return errors.Wrapf(err, "failed to read external error %s", rec.ExternalError.Digest)
}
var st spb.Status
if err := proto.Unmarshal(dt, &st); err != nil {
return errors.Wrapf(err, "failed to unmarshal external error %s", rec.ExternalError.Digest)
}
retErr := grpcerrors.FromGRPC(status.ErrorProto(&st))
var errsources bytes.Buffer
for _, s := range errdefs.Sources(retErr) {
s.Print(&errsources)
errsources.WriteString("\n")
}
out.Error.Sources = errsources.Bytes()
var ve *errdefs.VertexError
if errors.As(retErr, &ve) {
dgst, err := digest.Parse(ve.Vertex.Digest)
if err != nil {
return errors.Wrapf(err, "failed to parse vertex digest %s", ve.Vertex.Digest)
}
name, logs, err := loadVertexLogs(ctx, c, rec.Ref, dgst, 16)
if err != nil {
return errors.Wrapf(err, "failed to load vertex logs %s", dgst)
}
out.Error.Name = name
out.Error.Logs = logs
}
out.Error.Stack = fmt.Appendf(nil, "%+v", stack.Formatter(retErr))
}
}
if out.StartedAt != nil {
if out.CompletedAt != nil {
out.Duration = out.CompletedAt.Sub(*out.StartedAt)
} else {
out.Duration = rec.currentTimestamp.Sub(*out.StartedAt)
}
}
out.NumCompletedSteps = rec.NumCompletedSteps
out.NumTotalSteps = rec.NumTotalSteps
out.NumCachedSteps = rec.NumCachedSteps
out.BuildArgs = readKeyValues(attrs, "build-arg:")
out.Labels = readKeyValues(attrs, "label:")
readAttr(attrs, "force-network-mode", &out.Config.Network, nil)
readAttr(attrs, "hostname", &out.Config.Hostname, nil)
readAttr(attrs, "cgroup-parent", &out.Config.CgroupParent, nil)
readAttr(attrs, "image-resolve-mode", &out.Config.ImageResolveMode, nil)
readAttr(attrs, "build-arg:BUILDKIT_MULTI_PLATFORM", &out.Config.MultiPlatform, func(v string) (bool, bool) {
return tryParseValue(v, &out.Errors, strconv.ParseBool)
})
readAttr(attrs, "multi-platform", &out.Config.MultiPlatform, func(v string) (bool, bool) {
return tryParseValue(v, &out.Errors, strconv.ParseBool)
})
readAttr(attrs, "no-cache", &out.Config.NoCache, func(v string) (bool, bool) {
if v == "" {
return true, true
}
return false, false
})
readAttr(attrs, "no-cache", &out.Config.NoCacheFilter, func(v string) ([]string, bool) {
if v == "" {
return nil, false
}
return strings.Split(v, ","), true
})
readAttr(attrs, "add-hosts", &out.Config.ExtraHosts, func(v string) ([]string, bool) {
return tryParseValue(v, &out.Errors, func(v string) ([]string, error) {
fields, err := csvvalue.Fields(v, nil)
if err != nil {
return nil, err
}
return fields, nil
})
})
readAttr(attrs, "shm-size", &out.Config.ShmSize, nil)
readAttr(attrs, "ulimit", &out.Config.Ulimit, nil)
readAttr(attrs, "build-arg:BUILDKIT_CACHE_MOUNT_NS", &out.Config.CacheMountNS, nil)
readAttr(attrs, "build-arg:BUILDKIT_DOCKERFILE_CHECK", &out.Config.DockerfileCheckConfig, nil)
readAttr(attrs, "build-arg:SOURCE_DATE_EPOCH", &out.Config.SourceDateEpoch, nil)
readAttr(attrs, "build-arg:SANDBOX_HOSTNAME", &out.Config.SandboxHostname, nil)
var unusedAttrs []keyValueOutput
for k := range attrs {
if strings.HasPrefix(k, "vcs:") || strings.HasPrefix(k, "build-arg:") || strings.HasPrefix(k, "label:") || strings.HasPrefix(k, "context:") || strings.HasPrefix(k, "attest:") {
continue
}
unusedAttrs = append(unusedAttrs, keyValueOutput{
Name: k,
Value: attrs[k],
})
}
slices.SortFunc(unusedAttrs, func(a, b keyValueOutput) int {
return cmp.Compare(a.Name, b.Name)
})
out.Config.RestRaw = unusedAttrs
attachments, err := allAttachments(ctx, store, *rec)
if err != nil {
return err
}
provIndex := slices.IndexFunc(attachments, func(a attachment) bool {
return descrType(a.descr) == slsa02.PredicateSLSAProvenance
})
if provIndex != -1 {
prov := attachments[provIndex]
dt, err := content.ReadBlob(ctx, store, prov.descr)
if err != nil {
return errors.Errorf("failed to read provenance %s: %v", prov.descr.Digest, err)
}
var pred provenancetypes.ProvenancePredicate
if err := json.Unmarshal(dt, &pred); err != nil {
return errors.Errorf("failed to unmarshal provenance %s: %v", prov.descr.Digest, err)
}
for _, m := range pred.Materials {
out.Materials = append(out.Materials, materialOutput{
URI: m.URI,
Digests: digestSetToDigests(m.Digest),
})
}
}
if len(attachments) > 0 {
for _, a := range attachments {
p := ""
if a.platform != nil {
p = platforms.FormatAll(*a.platform)
}
out.Attachments = append(out.Attachments, attachmentOutput{
Digest: a.descr.Digest.String(),
Platform: p,
Type: descrType(a.descr),
})
}
}
if opts.format == formatter.JSONFormatKey {
enc := json.NewEncoder(dockerCli.Out())
enc.SetIndent("", " ")
return enc.Encode(out)
} else if opts.format != formatter.PrettyFormatKey {
tmpl, err := template.New("inspect").Parse(opts.format)
if err != nil {
return errors.Wrapf(err, "failed to parse format template")
}
var buf bytes.Buffer
if err := tmpl.Execute(&buf, out); err != nil {
return errors.Wrapf(err, "failed to execute format template")
}
fmt.Fprintln(dockerCli.Out(), buf.String())
return nil
}
tw := tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
if out.Name != "" {
fmt.Fprintf(tw, "Name:\t%s\n", out.Name)
}
if opts.ref == "" && out.Ref != "" {
fmt.Fprintf(tw, "Ref:\t%s\n", out.Ref)
}
if out.Context != "" {
fmt.Fprintf(tw, "Context:\t%s\n", out.Context)
}
if out.Dockerfile != "" {
fmt.Fprintf(tw, "Dockerfile:\t%s\n", out.Dockerfile)
}
if out.VCSRepository != "" {
fmt.Fprintf(tw, "VCS Repository:\t%s\n", out.VCSRepository)
}
if out.VCSRevision != "" {
fmt.Fprintf(tw, "VCS Revision:\t%s\n", out.VCSRevision)
}
if out.Target != "" {
fmt.Fprintf(tw, "Target:\t%s\n", out.Target)
}
if len(out.Platform) > 0 {
fmt.Fprintf(tw, "Platforms:\t%s\n", strings.Join(out.Platform, ", "))
}
if out.KeepGitDir {
fmt.Fprintf(tw, "Keep Git Dir:\t%s\n", strconv.FormatBool(out.KeepGitDir))
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
printTable(dockerCli.Out(), out.NamedContexts, "Named Context")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "Started:\t%s\n", out.StartedAt.Format("2006-01-02 15:04:05"))
var statusStr string
if out.Status == statusRunning {
statusStr = " (running)"
}
fmt.Fprintf(tw, "Duration:\t%s%s\n", formatDuration(out.Duration), statusStr)
if out.Status == statusError {
fmt.Fprintf(tw, "Error:\t%s %s\n", codes.Code(rec.Error.Code).String(), rec.Error.Message)
} else if out.Status == statusCanceled {
fmt.Fprintf(tw, "Status:\tCanceled\n")
}
fmt.Fprintf(tw, "Build Steps:\t%d/%d (%.0f%% cached)\n", out.NumCompletedSteps, out.NumTotalSteps, float64(out.NumCachedSteps)/float64(out.NumTotalSteps)*100)
tw.Flush()
fmt.Fprintln(dockerCli.Out())
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
if out.Config.Network != "" {
fmt.Fprintf(tw, "Network:\t%s\n", out.Config.Network)
}
if out.Config.Hostname != "" {
fmt.Fprintf(tw, "Hostname:\t%s\n", out.Config.Hostname)
}
if len(out.Config.ExtraHosts) > 0 {
fmt.Fprintf(tw, "Extra Hosts:\t%s\n", strings.Join(out.Config.ExtraHosts, ", "))
}
if out.Config.CgroupParent != "" {
fmt.Fprintf(tw, "Cgroup Parent:\t%s\n", out.Config.CgroupParent)
}
if out.Config.ImageResolveMode != "" {
fmt.Fprintf(tw, "Image Resolve Mode:\t%s\n", out.Config.ImageResolveMode)
}
if out.Config.MultiPlatform {
fmt.Fprintf(tw, "Multi-Platform:\t%s\n", strconv.FormatBool(out.Config.MultiPlatform))
}
if out.Config.NoCache {
fmt.Fprintf(tw, "No Cache:\t%s\n", strconv.FormatBool(out.Config.NoCache))
}
if len(out.Config.NoCacheFilter) > 0 {
fmt.Fprintf(tw, "No Cache Filter:\t%s\n", strings.Join(out.Config.NoCacheFilter, ", "))
}
if out.Config.ShmSize != "" {
fmt.Fprintf(tw, "Shm Size:\t%s\n", out.Config.ShmSize)
}
if out.Config.Ulimit != "" {
fmt.Fprintf(tw, "Resource Limits:\t%s\n", out.Config.Ulimit)
}
if out.Config.CacheMountNS != "" {
fmt.Fprintf(tw, "Cache Mount Namespace:\t%s\n", out.Config.CacheMountNS)
}
if out.Config.DockerfileCheckConfig != "" {
fmt.Fprintf(tw, "Dockerfile Check Config:\t%s\n", out.Config.DockerfileCheckConfig)
}
if out.Config.SourceDateEpoch != "" {
fmt.Fprintf(tw, "Source Date Epoch:\t%s\n", out.Config.SourceDateEpoch)
}
if out.Config.SandboxHostname != "" {
fmt.Fprintf(tw, "Sandbox Hostname:\t%s\n", out.Config.SandboxHostname)
}
for _, kv := range out.Config.RestRaw {
fmt.Fprintf(tw, "%s:\t%s\n", kv.Name, kv.Value)
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
printTable(dockerCli.Out(), out.BuildArgs, "Build Arg")
printTable(dockerCli.Out(), out.Labels, "Label")
if len(out.Materials) > 0 {
fmt.Fprintln(dockerCli.Out(), "Materials:")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "URI\tDIGEST\n")
for _, m := range out.Materials {
fmt.Fprintf(tw, "%s\t%s\n", m.URI, strings.Join(m.Digests, ", "))
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
}
if len(out.Attachments) > 0 {
fmt.Fprintf(tw, "Attachments:\n")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "DIGEST\tPLATFORM\tTYPE\n")
for _, a := range out.Attachments {
fmt.Fprintf(tw, "%s\t%s\t%s\n", a.Digest, a.Platform, a.Type)
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
}
if out.Error != nil {
if out.Error.Sources != nil {
fmt.Fprint(dockerCli.Out(), string(out.Error.Sources))
}
if len(out.Error.Logs) > 0 {
fmt.Fprintln(dockerCli.Out(), "Logs:")
fmt.Fprintf(dockerCli.Out(), "> => %s:\n", out.Error.Name)
for _, l := range out.Error.Logs {
fmt.Fprintln(dockerCli.Out(), "> "+l)
}
fmt.Fprintln(dockerCli.Out())
}
if len(out.Error.Stack) > 0 {
if debug.IsEnabled() {
fmt.Fprintf(dockerCli.Out(), "\n%s\n", out.Error.Stack)
} else {
fmt.Fprintf(dockerCli.Out(), "Enable --debug to see stack traces for error\n")
}
}
}
fmt.Fprintf(dockerCli.Out(), "Print build logs: docker buildx history logs %s\n", rec.Ref)
fmt.Fprintf(dockerCli.Out(), "View build in Docker Desktop: %s\n", desktop.BuildURL(fmt.Sprintf("%s/%s/%s", rec.node.Builder, rec.node.Name, rec.Ref)))
return nil
}
func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options inspectOptions
cmd := &cobra.Command{
Use: "inspect [OPTIONS] [REF]",
Short: "Inspect a build",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runInspect(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(
attachmentCmd(dockerCli, rootOpts),
)
flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.PrettyFormatKey, "Format the output")
return cmd
}
func loadVertexLogs(ctx context.Context, c *client.Client, ref string, dgst digest.Digest, limit int) (string, []string, error) {
st, err := c.ControlClient().Status(ctx, &controlapi.StatusRequest{
Ref: ref,
})
if err != nil {
return "", nil, err
}
var name string
var logs []string
lastState := map[int]int{}
loop0:
for {
select {
case <-ctx.Done():
st.CloseSend()
return "", nil, context.Cause(ctx)
default:
ev, err := st.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break loop0
}
return "", nil, err
}
ss := client.NewSolveStatus(ev)
for _, v := range ss.Vertexes {
if v.Digest == dgst {
name = v.Name
break
}
}
for _, l := range ss.Logs {
if l.Vertex == dgst {
parts := bytes.Split(l.Data, []byte("\n"))
for i, p := range parts {
var wrote bool
if i == 0 {
idx, ok := lastState[l.Stream]
if ok && idx != -1 {
logs[idx] = logs[idx] + string(p)
wrote = true
}
}
if !wrote {
if len(p) > 0 {
logs = append(logs, string(p))
}
lastState[l.Stream] = len(logs) - 1
}
if i == len(parts)-1 && len(p) == 0 {
lastState[l.Stream] = -1
}
}
}
}
}
}
if limit > 0 && len(logs) > limit {
logs = logs[len(logs)-limit:]
}
return name, logs, nil
}
type attachment struct {
platform *ocispecs.Platform
descr ocispecs.Descriptor
}
func allAttachments(ctx context.Context, store content.Store, rec historyRecord) ([]attachment, error) {
var attachments []attachment
if rec.Result != nil {
for _, a := range rec.Result.Attestations {
attachments = append(attachments, attachment{
descr: ociDesc(a),
})
}
for _, r := range rec.Result.Results {
attachments = append(attachments, walkAttachments(ctx, store, ociDesc(r), nil)...)
}
}
for key, ri := range rec.Results {
p, err := platforms.Parse(key)
if err != nil {
return nil, err
}
for _, a := range ri.Attestations {
attachments = append(attachments, attachment{
platform: &p,
descr: ociDesc(a),
})
}
for _, r := range ri.Results {
attachments = append(attachments, walkAttachments(ctx, store, ociDesc(r), &p)...)
}
}
slices.SortFunc(attachments, func(a, b attachment) int {
pCmp := 0
if a.platform == nil && b.platform != nil {
return -1
} else if a.platform != nil && b.platform == nil {
return 1
} else if a.platform != nil && b.platform != nil {
pCmp = cmp.Compare(platforms.FormatAll(*a.platform), platforms.FormatAll(*b.platform))
}
return cmp.Or(
pCmp,
cmp.Compare(descrType(a.descr), descrType(b.descr)),
)
})
return attachments, nil
}
func walkAttachments(ctx context.Context, store content.Store, desc ocispecs.Descriptor, platform *ocispecs.Platform) []attachment {
_, err := store.Info(ctx, desc.Digest)
if err != nil {
return nil
}
var out []attachment
if desc.Annotations["vnd.docker.reference.type"] != "attestation-manifest" {
out = append(out, attachment{platform: platform, descr: desc})
}
if desc.MediaType != ocispecs.MediaTypeImageIndex && desc.MediaType != images.MediaTypeDockerSchema2ManifestList {
return out
}
dt, err := content.ReadBlob(ctx, store, desc)
if err != nil {
return out
}
var idx ocispecs.Index
if err := json.Unmarshal(dt, &idx); err != nil {
return out
}
for _, d := range idx.Manifests {
p := platform
if d.Platform != nil {
p = d.Platform
}
out = append(out, walkAttachments(ctx, store, d, p)...)
}
return out
}
func ociDesc(in *controlapi.Descriptor) ocispecs.Descriptor {
return ocispecs.Descriptor{
MediaType: in.MediaType,
Digest: digest.Digest(in.Digest),
Size: in.Size,
Annotations: in.Annotations,
}
}
func descrType(desc ocispecs.Descriptor) string {
if typ, ok := desc.Annotations["in-toto.io/predicate-type"]; ok {
return typ
}
return desc.MediaType
}
func tryParseValue[T any](s string, errs *[]string, f func(string) (T, error)) (T, bool) {
v, err := f(s)
if err != nil {
errStr := fmt.Sprintf("failed to parse %s: (%v)", s, err)
*errs = append(*errs, errStr)
}
return v, true
}
func printTable(w io.Writer, kvs []keyValueOutput, title string) {
if len(kvs) == 0 {
return
}
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "%s\tVALUE\n", strings.ToUpper(title))
for _, k := range kvs {
fmt.Fprintf(tw, "%s\t%s\n", k.Name, k.Value)
}
tw.Flush()
fmt.Fprintln(w)
}
func readKeyValues(attrs map[string]string, prefix string) []keyValueOutput {
var out []keyValueOutput
for k, v := range attrs {
if strings.HasPrefix(k, prefix) {
out = append(out, keyValueOutput{
Name: strings.TrimPrefix(k, prefix),
Value: v,
})
}
}
if len(out) == 0 {
return nil
}
slices.SortFunc(out, func(a, b keyValueOutput) int {
return cmp.Compare(a.Name, b.Name)
})
return out
}
func digestSetToDigests(ds slsa.DigestSet) []string {
var out []string
for k, v := range ds {
out = append(out, fmt.Sprintf("%s:%s", k, v))
}
return out
}

View File

@@ -1,145 +0,0 @@
package history
import (
"context"
"io"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
intoto "github.com/in-toto/in-toto-golang/in_toto"
slsa02 "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/v0.2"
"github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type attachmentOptions struct {
builder string
typ string
platform string
ref string
digest digest.Digest
}
func runAttachment(ctx context.Context, dockerCli command.Cli, opts attachmentOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes, nil)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
rec := &recs[0]
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
store := proxy.NewContentStore(c.ContentClient())
if opts.digest != "" {
ra, err := store.ReaderAt(ctx, ocispecs.Descriptor{Digest: opts.digest})
if err != nil {
return err
}
_, err = io.Copy(dockerCli.Out(), io.NewSectionReader(ra, 0, ra.Size()))
return err
}
attachments, err := allAttachments(ctx, store, *rec)
if err != nil {
return err
}
typ := opts.typ
switch typ {
case "index":
typ = ocispecs.MediaTypeImageIndex
case "manifest":
typ = ocispecs.MediaTypeImageManifest
case "image":
typ = ocispecs.MediaTypeImageConfig
case "provenance":
typ = slsa02.PredicateSLSAProvenance
case "sbom":
typ = intoto.PredicateSPDX
}
for _, a := range attachments {
if opts.platform != "" && (a.platform == nil || platforms.FormatAll(*a.platform) != opts.platform) {
continue
}
if typ != "" && descrType(a.descr) != typ {
continue
}
ra, err := store.ReaderAt(ctx, a.descr)
if err != nil {
return err
}
_, err = io.Copy(dockerCli.Out(), io.NewSectionReader(ra, 0, ra.Size()))
return err
}
return errors.Errorf("no matching attachment found for ref %q", opts.ref)
}
func attachmentCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options attachmentOptions
cmd := &cobra.Command{
Use: "attachment [OPTIONS] REF [DIGEST]",
Short: "Inspect a build attachment",
Args: cobra.RangeArgs(1, 2),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
if len(args) > 1 {
dgst, err := digest.Parse(args[1])
if err != nil {
return errors.Wrapf(err, "invalid digest %q", args[1])
}
options.digest = dgst
}
if options.digest == "" && options.platform == "" && options.typ == "" {
return errors.New("at least one of --type, --platform or DIGEST must be specified")
}
options.builder = *rootOpts.Builder
return runAttachment(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.typ, "type", "", "Type of attachment")
flags.StringVar(&options.platform, "platform", "", "Platform of attachment")
return cmd
}

View File

@@ -1,117 +0,0 @@
package history
import (
"context"
"io"
"os"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type logsOptions struct {
builder string
ref string
progress string
}
func runLogs(ctx context.Context, dockerCli command.Cli, opts logsOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes, nil)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
rec := &recs[0]
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
cl, err := c.ControlClient().Status(ctx, &controlapi.StatusRequest{
Ref: rec.Ref,
})
if err != nil {
return err
}
var mode progressui.DisplayMode = progressui.DisplayMode(opts.progress)
if mode == progressui.AutoMode {
mode = progressui.PlainMode
}
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, mode)
if err != nil {
return err
}
loop0:
for {
select {
case <-ctx.Done():
cl.CloseSend()
return context.Cause(ctx)
default:
ev, err := cl.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break loop0
}
return err
}
printer.Write(client.NewSolveStatus(ev))
}
}
return printer.Wait()
}
func logsCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options logsOptions
cmd := &cobra.Command{
Use: "logs [OPTIONS] [REF]",
Short: "Print the logs of a build",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runLogs(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.progress, "progress", "plain", "Set type of progress output (plain, rawjson, tty)")
return cmd
}

View File

@@ -1,234 +0,0 @@
package history
import (
"context"
"encoding/json"
"fmt"
"os"
"slices"
"time"
"github.com/containerd/console"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/formatter"
"github.com/docker/go-units"
"github.com/spf13/cobra"
)
const (
lsHeaderBuildID = "BUILD ID"
lsHeaderName = "NAME"
lsHeaderStatus = "STATUS"
lsHeaderCreated = "CREATED AT"
lsHeaderDuration = "DURATION"
lsHeaderLink = ""
lsDefaultTableFormat = "table {{.Ref}}\t{{.Name}}\t{{.Status}}\t{{.CreatedAt}}\t{{.Duration}}\t{{.Link}}"
headerKeyTimestamp = "buildkit-current-timestamp"
)
type lsOptions struct {
builder string
format string
noTrunc bool
}
func runLs(ctx context.Context, dockerCli command.Cli, opts lsOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
out, err := queryRecords(ctx, "", nodes, nil)
if err != nil {
return err
}
ls, err := localstate.New(confutil.NewConfig(dockerCli))
if err != nil {
return err
}
for i, rec := range out {
st, _ := ls.ReadRef(rec.node.Builder, rec.node.Name, rec.Ref)
rec.name = buildName(rec.FrontendAttrs, st)
out[i] = rec
}
return lsPrint(dockerCli, out, opts)
}
func lsCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options lsOptions
cmd := &cobra.Command{
Use: "ls",
Short: "List build records",
Args: cli.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *rootOpts.Builder
return runLs(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output")
flags.BoolVar(&options.noTrunc, "no-trunc", false, "Don't truncate output")
return cmd
}
func lsPrint(dockerCli command.Cli, records []historyRecord, in lsOptions) error {
if in.format == formatter.TableFormatKey {
in.format = lsDefaultTableFormat
}
ctx := formatter.Context{
Output: dockerCli.Out(),
Format: formatter.Format(in.format),
Trunc: !in.noTrunc,
}
slices.SortFunc(records, func(a, b historyRecord) int {
if a.CompletedAt == nil && b.CompletedAt != nil {
return -1
}
if a.CompletedAt != nil && b.CompletedAt == nil {
return 1
}
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
var term bool
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
term = true
}
render := func(format func(subContext formatter.SubContext) error) error {
for _, r := range records {
if err := format(&lsContext{
format: formatter.Format(in.format),
isTerm: term,
trunc: !in.noTrunc,
record: &r,
}); err != nil {
return err
}
}
return nil
}
lsCtx := lsContext{
isTerm: term,
trunc: !in.noTrunc,
}
lsCtx.Header = formatter.SubHeaderContext{
"Ref": lsHeaderBuildID,
"Name": lsHeaderName,
"Status": lsHeaderStatus,
"CreatedAt": lsHeaderCreated,
"Duration": lsHeaderDuration,
"Link": lsHeaderLink,
}
return ctx.Write(&lsCtx, render)
}
type lsContext struct {
formatter.HeaderContext
isTerm bool
trunc bool
format formatter.Format
record *historyRecord
}
func (c *lsContext) MarshalJSON() ([]byte, error) {
m := map[string]any{
"ref": c.FullRef(),
"name": c.Name(),
"status": c.Status(),
"created_at": c.record.CreatedAt.AsTime().Format(time.RFC3339Nano),
"total_steps": c.record.NumTotalSteps,
"completed_steps": c.record.NumCompletedSteps,
"cached_steps": c.record.NumCachedSteps,
}
if c.record.CompletedAt != nil {
m["completed_at"] = c.record.CompletedAt.AsTime().Format(time.RFC3339Nano)
}
return json.Marshal(m)
}
func (c *lsContext) Ref() string {
return c.record.Ref
}
func (c *lsContext) FullRef() string {
return fmt.Sprintf("%s/%s/%s", c.record.node.Builder, c.record.node.Name, c.record.Ref)
}
func (c *lsContext) Name() string {
name := c.record.name
if c.trunc && c.format.IsTable() {
return trimBeginning(name, 36)
}
return name
}
func (c *lsContext) Status() string {
if c.record.CompletedAt != nil {
if c.record.Error != nil {
return "Error"
}
return "Completed"
}
return "Running"
}
func (c *lsContext) CreatedAt() string {
return units.HumanDuration(time.Since(c.record.CreatedAt.AsTime())) + " ago"
}
func (c *lsContext) Duration() string {
lastTime := c.record.currentTimestamp
if c.record.CompletedAt != nil {
tm := c.record.CompletedAt.AsTime()
lastTime = &tm
}
if lastTime == nil {
return ""
}
v := formatDuration(lastTime.Sub(c.record.CreatedAt.AsTime()))
if c.record.CompletedAt == nil {
v += "+"
}
return v
}
func (c *lsContext) Link() string {
url := desktop.BuildURL(c.FullRef())
if c.format.IsTable() {
if c.isTerm {
return desktop.ANSIHyperlink(url, "Open")
}
return ""
}
return url
}

View File

@@ -1,73 +0,0 @@
package history
import (
"context"
"fmt"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli/command"
"github.com/pkg/browser"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type openOptions struct {
builder string
ref string
}
func runOpen(ctx context.Context, dockerCli command.Cli, opts openOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes, nil)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
rec := &recs[0]
url := desktop.BuildURL(fmt.Sprintf("%s/%s/%s", rec.node.Builder, rec.node.Name, rec.Ref))
return browser.OpenURL(url)
}
func openCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options openOptions
cmd := &cobra.Command{
Use: "open [OPTIONS] [REF]",
Short: "Open a build in Docker Desktop",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runOpen(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
return cmd
}

View File

@@ -1,151 +0,0 @@
package history
import (
"context"
"io"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/hashicorp/go-multierror"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type rmOptions struct {
builder string
refs []string
all bool
}
func runRm(ctx context.Context, dockerCli command.Cli, opts rmOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
errs := make([][]error, len(opts.refs))
for i := range errs {
errs[i] = make([]error, len(nodes))
}
eg, ctx := errgroup.WithContext(ctx)
for i, node := range nodes {
node := node
eg.Go(func() error {
if node.Driver == nil {
return nil
}
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
refs := opts.refs
if opts.all {
serv, err := c.ControlClient().ListenBuildHistory(ctx, &controlapi.BuildHistoryRequest{
EarlyExit: true,
})
if err != nil {
return err
}
defer serv.CloseSend()
for {
resp, err := serv.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return err
}
if resp.Type == controlapi.BuildHistoryEventType_COMPLETE {
refs = append(refs, resp.Record.Ref)
}
}
}
for j, ref := range refs {
_, err = c.ControlClient().UpdateBuildHistory(ctx, &controlapi.UpdateBuildHistoryRequest{
Ref: ref,
Delete: true,
})
if opts.all {
if err != nil {
return err
}
} else {
errs[j][i] = err
}
}
return nil
})
}
if err := eg.Wait(); err != nil {
return err
}
var out []error
loop0:
for _, nodeErrs := range errs {
var nodeErr error
for _, err1 := range nodeErrs {
if err1 == nil {
continue loop0
}
if nodeErr == nil {
nodeErr = err1
} else {
nodeErr = multierror.Append(nodeErr, err1)
}
}
out = append(out, nodeErr)
}
if len(out) == 0 {
return nil
}
if len(out) == 1 {
return out[0]
}
return multierror.Append(out[0], out[1:]...)
}
func rmCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options rmOptions
cmd := &cobra.Command{
Use: "rm [OPTIONS] [REF...]",
Short: "Remove build records",
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 && !options.all {
return errors.New("rm requires at least one argument")
}
if len(args) > 0 && options.all {
return errors.New("rm requires either --all or at least one argument")
}
options.refs = args
options.builder = *rootOpts.Builder
return runRm(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.BoolVar(&options.all, "all", false, "Remove all build records")
return cmd
}

View File

@@ -1,32 +0,0 @@
package history
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
type RootOptions struct {
Builder *string
}
func RootCmd(rootcmd *cobra.Command, dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "history",
Short: "Commands to work on build records",
ValidArgsFunction: completion.Disable,
RunE: rootcmd.RunE,
}
cmd.AddCommand(
lsCmd(dockerCli, opts),
rmCmd(dockerCli, opts),
logsCmd(dockerCli, opts),
inspectCmd(dockerCli, opts),
openCmd(dockerCli, opts),
traceCmd(dockerCli, opts),
importCmd(dockerCli, opts),
)
return cmd
}

View File

@@ -1,228 +0,0 @@
package history
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net"
"os"
"time"
"github.com/containerd/console"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/otelutil"
"github.com/docker/buildx/util/otelutil/jaeger"
"github.com/docker/cli/cli/command"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/browser"
"github.com/pkg/errors"
"github.com/spf13/cobra"
jaegerui "github.com/tonistiigi/jaeger-ui-rest"
)
type traceOptions struct {
builder string
ref string
addr string
compare string
}
func loadTrace(ctx context.Context, ref string, nodes []builder.Node) (string, []byte, error) {
recs, err := queryRecords(ctx, ref, nodes, &queryOptions{
CompletedOnly: true,
})
if err != nil {
return "", nil, err
}
if len(recs) == 0 {
if ref == "" {
return "", nil, errors.New("no records found")
}
return "", nil, errors.Errorf("no record found for ref %q", ref)
}
rec := &recs[0]
if rec.CompletedAt == nil {
return "", nil, errors.Errorf("build %q is not completed, only completed builds can be traced", rec.Ref)
}
if rec.Trace == nil {
// build is complete but no trace yet. try to finalize the trace
time.Sleep(1 * time.Second) // give some extra time for last parts of trace to be written
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return "", nil, err
}
_, err = c.ControlClient().UpdateBuildHistory(ctx, &controlapi.UpdateBuildHistoryRequest{
Ref: rec.Ref,
Finalize: true,
})
if err != nil {
return "", nil, err
}
recs, err := queryRecords(ctx, rec.Ref, []builder.Node{*rec.node}, &queryOptions{
CompletedOnly: true,
})
if err != nil {
return "", nil, err
}
if len(recs) == 0 {
return "", nil, errors.Errorf("build record %q was deleted", rec.Ref)
}
rec = &recs[0]
if rec.Trace == nil {
return "", nil, errors.Errorf("build record %q is missing a trace", rec.Ref)
}
}
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return "", nil, err
}
store := proxy.NewContentStore(c.ContentClient())
ra, err := store.ReaderAt(ctx, ocispecs.Descriptor{
Digest: digest.Digest(rec.Trace.Digest),
MediaType: rec.Trace.MediaType,
Size: rec.Trace.Size,
})
if err != nil {
return "", nil, err
}
spans, err := otelutil.ParseSpanStubs(io.NewSectionReader(ra, 0, ra.Size()))
if err != nil {
return "", nil, err
}
wrapper := struct {
Data []jaeger.Trace `json:"data"`
}{
Data: spans.JaegerData().Data,
}
if len(wrapper.Data) == 0 {
return "", nil, errors.New("no trace data")
}
buf := &bytes.Buffer{}
enc := json.NewEncoder(buf)
enc.SetIndent("", " ")
if err := enc.Encode(wrapper); err != nil {
return "", nil, err
}
return string(wrapper.Data[0].TraceID), buf.Bytes(), nil
}
func runTrace(ctx context.Context, dockerCli command.Cli, opts traceOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
traceID, data, err := loadTrace(ctx, opts.ref, nodes)
if err != nil {
return err
}
srv := jaegerui.NewServer(jaegerui.Config{})
if err := srv.AddTrace(traceID, bytes.NewReader(data)); err != nil {
return err
}
url := "/trace/" + traceID
if opts.compare != "" {
traceIDcomp, data, err := loadTrace(ctx, opts.compare, nodes)
if err != nil {
return errors.Wrapf(err, "failed to load trace for %s", opts.compare)
}
if err := srv.AddTrace(traceIDcomp, bytes.NewReader(data)); err != nil {
return err
}
url = "/trace/" + traceIDcomp + "..." + traceID
}
var term bool
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
term = true
}
if !term && opts.compare == "" {
fmt.Fprintln(dockerCli.Out(), string(data))
return nil
}
ln, err := net.Listen("tcp", opts.addr)
if err != nil {
return err
}
go func() {
time.Sleep(100 * time.Millisecond)
browser.OpenURL(url)
}()
url = "http://" + ln.Addr().String() + url
fmt.Fprintf(dockerCli.Err(), "Trace available at %s\n", url)
go func() {
<-ctx.Done()
ln.Close()
}()
err = srv.Serve(ln)
if err != nil {
select {
case <-ctx.Done():
return nil
default:
}
}
return err
}
func traceCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options traceOptions
cmd := &cobra.Command{
Use: "trace [OPTIONS] [REF]",
Short: "Show the OpenTelemetry trace of a build record",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runTrace(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.addr, "addr", "127.0.0.1:0", "Address to bind the UI server")
flags.StringVar(&options.compare, "compare", "", "Compare with another build reference")
return cmd
}

View File

@@ -1,221 +0,0 @@
package history
import (
"context"
"fmt"
"io"
"path/filepath"
"slices"
"strconv"
"strings"
"sync"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
func buildName(fattrs map[string]string, ls *localstate.State) string {
var res string
var target, contextPath, dockerfilePath, vcsSource string
if v, ok := fattrs["target"]; ok {
target = v
}
if v, ok := fattrs["context"]; ok {
contextPath = filepath.ToSlash(v)
} else if v, ok := fattrs["vcs:localdir:context"]; ok && v != "." {
contextPath = filepath.ToSlash(v)
}
if v, ok := fattrs["vcs:source"]; ok {
vcsSource = v
}
if v, ok := fattrs["filename"]; ok && v != "Dockerfile" {
dockerfilePath = filepath.ToSlash(v)
}
if v, ok := fattrs["vcs:localdir:dockerfile"]; ok && v != "." {
dockerfilePath = filepath.ToSlash(filepath.Join(v, dockerfilePath))
}
var localPath string
if ls != nil && !build.IsRemoteURL(ls.LocalPath) {
if ls.LocalPath != "" && ls.LocalPath != "-" {
localPath = filepath.ToSlash(ls.LocalPath)
}
if ls.DockerfilePath != "" && ls.DockerfilePath != "-" && ls.DockerfilePath != "Dockerfile" {
dockerfilePath = filepath.ToSlash(ls.DockerfilePath)
}
}
// remove default dockerfile name
const defaultFilename = "/Dockerfile"
hasDefaultFileName := strings.HasSuffix(dockerfilePath, defaultFilename) || dockerfilePath == ""
dockerfilePath = strings.TrimSuffix(dockerfilePath, defaultFilename)
// dockerfile is a subpath of context
if strings.HasPrefix(dockerfilePath, localPath) && len(dockerfilePath) > len(localPath) {
res = dockerfilePath[strings.LastIndex(localPath, "/")+1:]
} else {
// Otherwise, use basename
bpath := localPath
if len(dockerfilePath) > 0 {
bpath = dockerfilePath
}
if len(bpath) > 0 {
lidx := strings.LastIndex(bpath, "/")
res = bpath[lidx+1:]
if !hasDefaultFileName {
if lidx != -1 {
res = filepath.ToSlash(filepath.Join(filepath.Base(bpath[:lidx]), res))
} else {
res = filepath.ToSlash(filepath.Join(filepath.Base(bpath), res))
}
}
}
}
if len(contextPath) > 0 {
res = contextPath
}
if len(target) > 0 {
if len(res) > 0 {
res = res + " (" + target + ")"
} else {
res = target
}
}
if res == "" && vcsSource != "" {
return vcsSource
}
return res
}
func trimBeginning(s string, n int) string {
if len(s) <= n {
return s
}
return ".." + s[len(s)-n+2:]
}
type historyRecord struct {
*controlapi.BuildHistoryRecord
currentTimestamp *time.Time
node *builder.Node
name string
}
type queryOptions struct {
CompletedOnly bool
}
func queryRecords(ctx context.Context, ref string, nodes []builder.Node, opts *queryOptions) ([]historyRecord, error) {
var mu sync.Mutex
var out []historyRecord
var offset *int
if strings.HasPrefix(ref, "^") {
off, err := strconv.Atoi(ref[1:])
if err != nil {
return nil, errors.Wrapf(err, "invalid offset %q", ref)
}
offset = &off
ref = ""
}
eg, ctx := errgroup.WithContext(ctx)
for _, node := range nodes {
node := node
eg.Go(func() error {
if node.Driver == nil {
return nil
}
var records []historyRecord
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
serv, err := c.ControlClient().ListenBuildHistory(ctx, &controlapi.BuildHistoryRequest{
EarlyExit: true,
Ref: ref,
})
if err != nil {
return err
}
md, err := serv.Header()
if err != nil {
return err
}
var ts *time.Time
if v, ok := md[headerKeyTimestamp]; ok {
t, err := time.Parse(time.RFC3339Nano, v[0])
if err != nil {
return err
}
ts = &t
}
defer serv.CloseSend()
for {
he, err := serv.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return err
}
if he.Type == controlapi.BuildHistoryEventType_DELETED || he.Record == nil {
continue
}
if opts != nil && opts.CompletedOnly && he.Type != controlapi.BuildHistoryEventType_COMPLETE {
continue
}
records = append(records, historyRecord{
BuildHistoryRecord: he.Record,
currentTimestamp: ts,
node: &node,
})
}
mu.Lock()
out = append(out, records...)
mu.Unlock()
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
slices.SortFunc(out, func(a, b historyRecord) int {
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
if offset != nil {
var filtered []historyRecord
for _, r := range out {
if *offset > 0 {
*offset--
continue
}
filtered = append(filtered, r)
break
}
if *offset > 0 {
return nil, errors.Errorf("no completed build found with offset %d", *offset)
}
out = filtered
}
return out, nil
}
func formatDuration(d time.Duration) string {
if d < time.Minute {
return fmt.Sprintf("%.1fs", d.Seconds())
}
return fmt.Sprintf("%dm %2ds", int(d.Minutes()), int(d.Seconds())%60)
}

View File

@@ -42,7 +42,7 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
return errors.Errorf("can't push with no tags specified, please set --tag or --dry-run") return errors.Errorf("can't push with no tags specified, please set --tag or --dry-run")
} }
fileArgs := make([]string, len(in.files), len(in.files)+len(args)) fileArgs := make([]string, len(in.files))
for i, f := range in.files { for i, f := range in.files {
dt, err := os.ReadFile(f) dt, err := os.ReadFile(f)
if err != nil { if err != nil {
@@ -173,8 +173,8 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
// new resolver cause need new auth // new resolver cause need new auth
r = imagetools.New(imageopt) r = imagetools.New(imageopt)
ctx2, cancel := context.WithCancelCause(context.TODO()) ctx2, cancel := context.WithCancel(context.TODO())
defer func() { cancel(errors.WithStack(context.Canceled)) }() defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressui.DisplayMode(in.progress)) printer, err := progress.NewPrinter(ctx2, os.Stderr, progressui.DisplayMode(in.progress))
if err != nil { if err != nil {
return err return err
@@ -194,7 +194,7 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
} }
s := s s := s
eg2.Go(func() error { eg2.Go(func() error {
sub.Log(1, fmt.Appendf(nil, "copying %s from %s to %s\n", s.Desc.Digest.String(), s.Ref.String(), t.String())) sub.Log(1, []byte(fmt.Sprintf("copying %s from %s to %s\n", s.Desc.Digest.String(), s.Ref.String(), t.String())))
return r.Copy(ctx, s, t) return r.Copy(ctx, s, t)
}) })
} }
@@ -202,7 +202,7 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
if err := eg2.Wait(); err != nil { if err := eg2.Wait(); err != nil {
return err return err
} }
sub.Log(1, fmt.Appendf(nil, "pushing %s to %s\n", desc.Digest.String(), t.String())) sub.Log(1, []byte(fmt.Sprintf("pushing %s to %s\n", desc.Digest.String(), t.String())))
return r.Push(ctx, t, desc, dt) return r.Push(ctx, t, desc, dt)
}) })
}) })

View File

@@ -10,12 +10,11 @@ type RootOptions struct {
Builder *string Builder *string
} }
func RootCmd(rootcmd *cobra.Command, dockerCli command.Cli, opts RootOptions) *cobra.Command { func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{ cmd := &cobra.Command{
Use: "imagetools", Use: "imagetools",
Short: "Commands to work on images in registry", Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable, ValidArgsFunction: completion.Disable,
RunE: rootcmd.RunE,
} }
cmd.AddCommand( cmd.AddCommand(

View File

@@ -17,7 +17,6 @@ import (
"github.com/docker/cli/cli/command" "github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug" "github.com/docker/cli/cli/debug"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/pkg/errors"
"github.com/spf13/cobra" "github.com/spf13/cobra"
) )
@@ -35,9 +34,8 @@ func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) e
return err return err
} }
timeoutCtx, cancel := context.WithCancelCause(ctx) timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent defer cancel()
defer func() { cancel(errors.WithStack(context.Canceled)) }()
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData()) nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if in.bootstrap { if in.bootstrap {
@@ -115,25 +113,6 @@ func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) e
fmt.Fprintf(w, "\t%s:\t%s\n", k, v) fmt.Fprintf(w, "\t%s:\t%s\n", k, v)
} }
} }
if len(nodes[i].CDIDevices) > 0 {
fmt.Fprintf(w, "Devices:\n")
for _, dev := range nodes[i].CDIDevices {
fmt.Fprintf(w, "\tName:\t%s\n", dev.Name)
if dev.OnDemand {
fmt.Fprintf(w, "\tOn-Demand:\t%v\n", dev.OnDemand)
} else {
fmt.Fprintf(w, "\tAutomatically allowed:\t%v\n", dev.AutoAllow)
}
if len(dev.Annotations) > 0 {
fmt.Fprintf(w, "\tAnnotations:\n")
for k, v := range dev.Annotations {
fmt.Fprintf(w, "\t\t%s:\t%s\n", k, v)
}
}
}
}
for ri, rule := range nodes[i].GCPolicy { for ri, rule := range nodes[i].GCPolicy {
fmt.Fprintf(w, "GC Policy rule#%d:\n", ri) fmt.Fprintf(w, "GC Policy rule#%d:\n", ri)
fmt.Fprintf(w, "\tAll:\t%v\n", rule.All) fmt.Fprintf(w, "\tAll:\t%v\n", rule.All)
@@ -143,20 +122,8 @@ func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) e
if rule.KeepDuration > 0 { if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String()) fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
} }
if rule.ReservedSpace > 0 { if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tReserved Space:\t%s\n", units.BytesSize(float64(rule.ReservedSpace))) fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
if rule.MaxUsedSpace > 0 {
fmt.Fprintf(w, "\tMax Used Space:\t%s\n", units.BytesSize(float64(rule.MaxUsedSpace)))
}
if rule.MinFreeSpace > 0 {
fmt.Fprintf(w, "\tMin Free Space:\t%s\n", units.BytesSize(float64(rule.MinFreeSpace)))
}
}
for f, dt := range nodes[i].Files {
fmt.Fprintf(w, "File#%s:\n", f)
for _, line := range strings.Split(string(dt), "\n") {
fmt.Fprintf(w, "\t> %s\n", line)
} }
} }
} }

View File

@@ -8,7 +8,6 @@ import (
"strings" "strings"
"time" "time"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder" "github.com/docker/buildx/builder"
"github.com/docker/buildx/store" "github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil" "github.com/docker/buildx/store/storeutil"
@@ -18,7 +17,6 @@ import (
"github.com/docker/cli/cli" "github.com/docker/cli/cli"
"github.com/docker/cli/cli/command" "github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/formatter" "github.com/docker/cli/cli/command/formatter"
"github.com/pkg/errors"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
) )
@@ -37,8 +35,7 @@ const (
) )
type lsOptions struct { type lsOptions struct {
format string format string
noTrunc bool
} }
func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error { func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
@@ -58,9 +55,8 @@ func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
return err return err
} }
timeoutCtx, cancel := context.WithCancelCause(ctx) timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent defer cancel()
defer func() { cancel(errors.WithStack(context.Canceled)) }()
eg, _ := errgroup.WithContext(timeoutCtx) eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders { for _, b := range builders {
@@ -76,7 +72,7 @@ func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
return err return err
} }
if hasErrors, err := lsPrint(dockerCli, current, builders, in); err != nil { if hasErrors, err := lsPrint(dockerCli, current, builders, in.format); err != nil {
return err return err
} else if hasErrors { } else if hasErrors {
_, _ = fmt.Fprintf(dockerCli.Err(), "\n") _, _ = fmt.Fprintf(dockerCli.Err(), "\n")
@@ -111,7 +107,6 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
flags := cmd.Flags() flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output") flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output")
flags.BoolVar(&options.noTrunc, "no-trunc", false, "Don't truncate output")
// hide builder persistent flag for this command // hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder") cobrautil.HideInheritedFlags(cmd, "builder")
@@ -119,15 +114,14 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
return cmd return cmd
} }
func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builder.Builder, in lsOptions) (hasErrors bool, _ error) { func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builder.Builder, format string) (hasErrors bool, _ error) {
if in.format == formatter.TableFormatKey { if format == formatter.TableFormatKey {
in.format = lsDefaultTableFormat format = lsDefaultTableFormat
} }
ctx := formatter.Context{ ctx := formatter.Context{
Output: dockerCli.Out(), Output: dockerCli.Out(),
Format: formatter.Format(in.format), Format: formatter.Format(format),
Trunc: !in.noTrunc,
} }
sort.SliceStable(builders, func(i, j int) bool { sort.SliceStable(builders, func(i, j int) bool {
@@ -144,12 +138,11 @@ func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builde
render := func(format func(subContext formatter.SubContext) error) error { render := func(format func(subContext formatter.SubContext) error) error {
for _, b := range builders { for _, b := range builders {
if err := format(&lsContext{ if err := format(&lsContext{
format: ctx.Format,
trunc: ctx.Trunc,
Builder: &lsBuilder{ Builder: &lsBuilder{
Builder: b, Builder: b,
Current: b.Name == current.Name, Current: b.Name == current.Name,
}, },
format: ctx.Format,
}); err != nil { }); err != nil {
return err return err
} }
@@ -159,9 +152,6 @@ func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builde
} }
continue continue
} }
if ctx.Format.IsJSON() {
continue
}
for _, n := range b.Nodes() { for _, n := range b.Nodes() {
if n.Err != nil { if n.Err != nil {
if ctx.Format.IsTable() { if ctx.Format.IsTable() {
@@ -170,7 +160,6 @@ func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builde
} }
if err := format(&lsContext{ if err := format(&lsContext{
format: ctx.Format, format: ctx.Format,
trunc: ctx.Trunc,
Builder: &lsBuilder{ Builder: &lsBuilder{
Builder: b, Builder: b,
Current: b.Name == current.Name, Current: b.Name == current.Name,
@@ -207,7 +196,6 @@ type lsContext struct {
Builder *lsBuilder Builder *lsBuilder
format formatter.Format format formatter.Format
trunc bool
node builder.Node node builder.Node
} }
@@ -273,11 +261,7 @@ func (c *lsContext) Platforms() string {
if c.node.Name == "" { if c.node.Name == "" {
return "" return ""
} }
pfs := platformutil.FormatInGroups(c.node.Node.Platforms, c.node.Platforms) return strings.Join(platformutil.FormatInGroups(c.node.Node.Platforms, c.node.Platforms), ", ")
if c.trunc && c.format.IsTable() {
return truncPlatforms(pfs, 4).String()
}
return strings.Join(pfs, ", ")
} }
func (c *lsContext) Error() string { func (c *lsContext) Error() string {
@@ -288,133 +272,3 @@ func (c *lsContext) Error() string {
} }
return "" return ""
} }
var truncMajorPlatforms = []string{
"linux/amd64",
"linux/arm64",
"linux/arm",
"linux/ppc64le",
"linux/s390x",
"linux/riscv64",
"linux/mips64",
}
type truncatedPlatforms struct {
res map[string][]string
input []string
max int
}
func (tp truncatedPlatforms) List() map[string][]string {
return tp.res
}
func (tp truncatedPlatforms) String() string {
var out []string
var count int
var keys []string
for k := range tp.res {
keys = append(keys, k)
}
sort.Strings(keys)
seen := make(map[string]struct{})
for _, mpf := range truncMajorPlatforms {
if tpf, ok := tp.res[mpf]; ok {
seen[mpf] = struct{}{}
if len(tpf) == 1 {
out = append(out, tpf[0])
count++
} else {
hasPreferredPlatform := false
for _, pf := range tpf {
if strings.HasSuffix(pf, "*") {
hasPreferredPlatform = true
break
}
}
mainpf := mpf
if hasPreferredPlatform {
mainpf += "*"
}
out = append(out, fmt.Sprintf("%s (+%d)", mainpf, len(tpf)))
count += len(tpf)
}
}
}
for _, mpf := range keys {
if len(out) >= tp.max {
break
}
if _, ok := seen[mpf]; ok {
continue
}
if len(tp.res[mpf]) == 1 {
out = append(out, tp.res[mpf][0])
count++
} else {
hasPreferredPlatform := false
for _, pf := range tp.res[mpf] {
if strings.HasSuffix(pf, "*") {
hasPreferredPlatform = true
break
}
}
mainpf := mpf
if hasPreferredPlatform {
mainpf += "*"
}
out = append(out, fmt.Sprintf("%s (+%d)", mainpf, len(tp.res[mpf])))
count += len(tp.res[mpf])
}
}
left := len(tp.input) - count
if left > 0 {
out = append(out, fmt.Sprintf("(%d more)", left))
}
return strings.Join(out, ", ")
}
func truncPlatforms(pfs []string, max int) truncatedPlatforms {
res := make(map[string][]string)
for _, mpf := range truncMajorPlatforms {
for _, pf := range pfs {
if len(res) >= max {
break
}
pp, err := platforms.Parse(strings.TrimSuffix(pf, "*"))
if err != nil {
continue
}
if pp.OS+"/"+pp.Architecture == mpf {
res[mpf] = append(res[mpf], pf)
}
}
}
left := make(map[string][]string)
for _, pf := range pfs {
if len(res) >= max {
break
}
pp, err := platforms.Parse(strings.TrimSuffix(pf, "*"))
if err != nil {
continue
}
ppf := strings.TrimSuffix(pp.OS+"/"+pp.Architecture, "*")
if _, ok := res[ppf]; !ok {
left[ppf] = append(left[ppf], pf)
}
}
for k, v := range left {
res[k] = v
}
return truncatedPlatforms{
res: res,
input: pfs,
max: max,
}
}

View File

@@ -1,174 +0,0 @@
package commands
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestTruncPlatforms(t *testing.T) {
tests := []struct {
name string
platforms []string
max int
expectedList map[string][]string
expectedOut string
}{
{
name: "arm64 preferred and emulated",
platforms: []string{"linux/arm64*", "linux/amd64", "linux/amd64/v2", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/386", "linux/mips64le", "linux/mips64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64",
"linux/amd64/v2",
},
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64*",
},
"linux/ppc64le": {
"linux/ppc64le",
},
},
expectedOut: "linux/amd64 (+2), linux/arm64*, linux/arm (+2), linux/ppc64le, (5 more)",
},
{
name: "riscv64 preferred only",
platforms: []string{"linux/riscv64*"},
max: 4,
expectedList: map[string][]string{
"linux/riscv64": {
"linux/riscv64*",
},
},
expectedOut: "linux/riscv64*",
},
{
name: "amd64 no preferred and emulated",
platforms: []string{"linux/amd64", "linux/amd64/v2", "linux/amd64/v3", "linux/386", "linux/arm64", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/mips64le", "linux/mips64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64",
"linux/amd64/v2",
"linux/amd64/v3",
},
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
"linux/ppc64le": {
"linux/ppc64le",
}},
expectedOut: "linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (5 more)",
},
{
name: "amd64 no preferred",
platforms: []string{"linux/amd64", "linux/386"},
max: 4,
expectedList: map[string][]string{
"linux/386": {
"linux/386",
},
"linux/amd64": {
"linux/amd64",
},
},
expectedOut: "linux/amd64, linux/386",
},
{
name: "arm64 no preferred",
platforms: []string{"linux/arm64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
},
expectedOut: "linux/arm64, linux/arm (+2)",
},
{
name: "all preferred",
platforms: []string{"darwin/arm64*", "linux/arm64*", "linux/arm/v5*", "linux/arm/v6*", "linux/arm/v7*", "windows/arm64*"},
max: 4,
expectedList: map[string][]string{
"darwin/arm64": {
"darwin/arm64*",
},
"linux/arm": {
"linux/arm/v5*",
"linux/arm/v6*",
"linux/arm/v7*",
},
"linux/arm64": {
"linux/arm64*",
},
"windows/arm64": {
"windows/arm64*",
},
},
expectedOut: "linux/arm64*, linux/arm* (+3), darwin/arm64*, windows/arm64*",
},
{
name: "no major preferred",
platforms: []string{"linux/amd64/v2*", "linux/arm/v6*", "linux/mips64le*", "linux/amd64", "linux/amd64/v3", "linux/386", "linux/arm64", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/mips64", "linux/arm/v7"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64/v2*",
"linux/amd64",
"linux/amd64/v3",
},
"linux/arm": {
"linux/arm/v6*",
"linux/arm/v7",
},
"linux/arm64": {
"linux/arm64",
},
"linux/ppc64le": {
"linux/ppc64le",
},
},
expectedOut: "linux/amd64* (+3), linux/arm64, linux/arm* (+2), linux/ppc64le, (5 more)",
},
{
name: "no major with multiple variants",
platforms: []string{"linux/arm64", "linux/arm/v7", "linux/arm/v6", "linux/mips64le/softfloat", "linux/mips64le/hardfloat"},
max: 4,
expectedList: map[string][]string{
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
"linux/mips64le": {
"linux/mips64le/softfloat",
"linux/mips64le/hardfloat",
},
},
expectedOut: "linux/arm64, linux/arm (+2), linux/mips64le (+2)",
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
tpfs := truncPlatforms(tt.platforms, tt.max)
assert.Equal(t, tt.expectedList, tpfs.List())
assert.Equal(t, tt.expectedOut, tpfs.String())
})
}
}

View File

@@ -16,23 +16,18 @@ import (
"github.com/docker/docker/api/types/filters" "github.com/docker/docker/api/types/filters"
"github.com/docker/go-units" "github.com/docker/go-units"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
gateway "github.com/moby/buildkit/frontend/gateway/client"
pb "github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/apicaps"
"github.com/pkg/errors" "github.com/pkg/errors"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
) )
type pruneOptions struct { type pruneOptions struct {
builder string builder string
all bool all bool
filter opts.FilterOpt filter opts.FilterOpt
reservedSpace opts.MemBytes keepStorage opts.MemBytes
maxUsedSpace opts.MemBytes force bool
minFreeSpace opts.MemBytes verbose bool
force bool
verbose bool
} }
const ( const (
@@ -110,19 +105,8 @@ func runPrune(ctx context.Context, dockerCli command.Cli, opts pruneOptions) err
if err != nil { if err != nil {
return err return err
} }
// check if the client supports newer prune options
if opts.maxUsedSpace.Value() != 0 || opts.minFreeSpace.Value() != 0 {
caps, err := loadLLBCaps(ctx, c)
if err != nil {
return errors.Wrap(err, "failed to load buildkit capabilities for prune")
}
if caps.Supports(pb.CapGCFreeSpaceFilter) != nil {
return errors.New("buildkit v0.17.0+ is required for max-used-space and min-free-space filters")
}
}
popts := []client.PruneOption{ popts := []client.PruneOption{
client.WithKeepOpt(pi.KeepDuration, opts.reservedSpace.Value(), opts.maxUsedSpace.Value(), opts.minFreeSpace.Value()), client.WithKeepOpt(pi.KeepDuration, opts.keepStorage.Value()),
client.WithFilter(pi.Filter), client.WithFilter(pi.Filter),
} }
if opts.all { if opts.all {
@@ -147,17 +131,6 @@ func runPrune(ctx context.Context, dockerCli command.Cli, opts pruneOptions) err
return nil return nil
} }
func loadLLBCaps(ctx context.Context, c *client.Client) (apicaps.CapSet, error) {
var caps apicaps.CapSet
_, err := c.Build(ctx, client.SolveOpt{
Internal: true,
}, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
caps = c.BuildOpts().LLBCaps
return nil, nil
}, nil)
return caps, err
}
func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command { func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options := pruneOptions{filter: opts.NewFilterOpt()} options := pruneOptions{filter: opts.NewFilterOpt()}
@@ -175,15 +148,10 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags := cmd.Flags() flags := cmd.Flags()
flags.BoolVarP(&options.all, "all", "a", false, "Include internal/frontend images") flags.BoolVarP(&options.all, "all", "a", false, "Include internal/frontend images")
flags.Var(&options.filter, "filter", `Provide filter values (e.g., "until=24h")`) flags.Var(&options.filter, "filter", `Provide filter values (e.g., "until=24h")`)
flags.Var(&options.reservedSpace, "reserved-space", "Amount of disk space always allowed to keep for cache") flags.Var(&options.keepStorage, "keep-storage", "Amount of disk space to keep for cache")
flags.Var(&options.minFreeSpace, "min-free-space", "Target amount of free disk space after pruning")
flags.Var(&options.maxUsedSpace, "max-used-space", "Maximum amount of disk space allowed to keep for cache")
flags.BoolVar(&options.verbose, "verbose", false, "Provide a more verbose output") flags.BoolVar(&options.verbose, "verbose", false, "Provide a more verbose output")
flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation") flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation")
flags.Var(&options.reservedSpace, "keep-storage", "Amount of disk space to keep for cache")
flags.MarkDeprecated("keep-storage", "keep-storage flag has been changed to max-storage")
return cmd return cmd
} }

View File

@@ -150,9 +150,8 @@ func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, i
return err return err
} }
timeoutCtx, cancel := context.WithCancelCause(ctx) timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent defer cancel()
defer func() { cancel(errors.WithStack(context.Canceled)) }()
eg, _ := errgroup.WithContext(timeoutCtx) eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders { for _, b := range builders {

View File

@@ -1,11 +1,9 @@
package commands package commands
import ( import (
"fmt"
"os" "os"
debugcmd "github.com/docker/buildx/commands/debug" debugcmd "github.com/docker/buildx/commands/debug"
historycmd "github.com/docker/buildx/commands/history"
imagetoolscmd "github.com/docker/buildx/commands/imagetools" imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote" "github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion" "github.com/docker/buildx/util/cobrautil/completion"
@@ -38,22 +36,13 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
if opt.debug { if opt.debug {
debug.Enable() debug.Enable()
} }
cmd.SetContext(appcontext.Context()) cmd.SetContext(appcontext.Context())
if !isPlugin { if !isPlugin {
return nil return nil
} }
return plugin.PersistentPreRunE(cmd, args) return plugin.PersistentPreRunE(cmd, args)
}, },
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
return cmd.Help()
}
_ = cmd.Help()
return cli.StatusError{
StatusCode: 1,
Status: fmt.Sprintf("ERROR: unknown command: %q", args[0]),
}
},
} }
if !isPlugin { if !isPlugin {
// match plugin behavior for standalone mode // match plugin behavior for standalone mode
@@ -106,8 +95,7 @@ func addCommands(cmd *cobra.Command, opts *rootOptions, dockerCli command.Cli) {
versionCmd(dockerCli), versionCmd(dockerCli),
pruneCmd(dockerCli, opts), pruneCmd(dockerCli, opts),
duCmd(dockerCli, opts), duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(cmd, dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}), imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
historycmd.RootCmd(cmd, dockerCli, historycmd.RootOptions{Builder: &opts.builder}),
) )
if confutil.IsExperimental() { if confutil.IsExperimental() {
cmd.AddCommand(debugcmd.RootCmd(dockerCli, cmd.AddCommand(debugcmd.RootCmd(dockerCli,

View File

@@ -46,6 +46,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Errorf("run `docker context use %s` to switch to context %s", in.builder, in.builder) return errors.Errorf("run `docker context use %s` to switch to context %s", in.builder, in.builder)
} }
} }
} }
return errors.Wrapf(err, "failed to find instance %q", in.builder) return errors.Wrapf(err, "failed to find instance %q", in.builder)
} }

View File

@@ -34,9 +34,9 @@ const defaultTargetName = "default"
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle, // NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can // this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error. // inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, *build.Inputs, error) { func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 { if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together") return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
} }
contexts := map[string]build.NamedContext{} contexts := map[string]build.NamedContext{}
@@ -70,18 +70,16 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
platforms, err := platformutil.Parse(in.Platforms) platforms, err := platformutil.Parse(in.Platforms)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
opts.Platforms = platforms opts.Platforms = platforms
dockerConfig := dockerCli.ConfigFile() dockerConfig := dockerCli.ConfigFile()
opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(authprovider.DockerAuthProviderConfig{ opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(dockerConfig, nil))
ConfigFile: dockerConfig,
}))
secrets, err := controllerapi.CreateSecrets(in.Secrets) secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
opts.Session = append(opts.Session, secrets) opts.Session = append(opts.Session, secrets)
@@ -91,13 +89,13 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
} }
ssh, err := controllerapi.CreateSSH(sshSpecs) ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
opts.Session = append(opts.Session, ssh) opts.Session = append(opts.Session, ssh)
outputs, _, err := controllerapi.CreateExports(in.Exports) outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
if in.ExportPush { if in.ExportPush {
var pushUsed bool var pushUsed bool
@@ -136,7 +134,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
annotations, err := buildflags.ParseAnnotations(in.Annotations) annotations, err := buildflags.ParseAnnotations(in.Annotations)
if err != nil { if err != nil {
return nil, nil, nil, errors.Wrap(err, "parse annotations") return nil, nil, errors.Wrap(err, "parse annotations")
} }
for _, o := range outputs { for _, o := range outputs {
@@ -156,7 +154,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
allow, err := buildflags.ParseEntitlements(in.Allow) allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
opts.Allow = allow opts.Allow = allow
@@ -180,28 +178,23 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
builder.WithContextPathHash(contextPathHash), builder.WithContextPathHash(contextPathHash),
) )
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil { if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, nil, errors.Wrapf(err, "failed to update builder last activity time") return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
} }
nodes, err := b.LoadNodes(ctx) nodes, err := b.LoadNodes(ctx)
if err != nil { if err != nil {
return nil, nil, nil, err return nil, nil, err
} }
var inputs *build.Inputs resp, res, err := buildTargets(ctx, dockerCli, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
buildOptions := map[string]build.Options{defaultTargetName: opts}
resp, res, err := buildTargets(ctx, dockerCli, nodes, buildOptions, progress, generateResult)
err = wrapBuildError(err, false) err = wrapBuildError(err, false)
if err != nil { if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error. // NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, nil, err return nil, res, err
} }
if i, ok := buildOptions[defaultTargetName]; ok { return resp, res, nil
inputs = &i.Inputs
}
return resp, res, inputs, nil
} }
// buildTargets runs the specified build and returns the result. // buildTargets runs the specified build and returns the result.
@@ -216,7 +209,7 @@ func buildTargets(ctx context.Context, dockerCli command.Cli, nodes []builder.No
if generateResult { if generateResult {
var mu sync.Mutex var mu sync.Mutex
var idx int var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) { resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock() mu.Lock()
defer mu.Unlock() defer mu.Unlock()
if res == nil || driverIndex < idx { if res == nil || driverIndex < idx {
@@ -224,7 +217,7 @@ func buildTargets(ctx context.Context, dockerCli command.Cli, nodes []builder.No
} }
}) })
} else { } else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), progress) resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
} }
if err != nil { if err != nil {
return nil, res, err return nil, res, err

View File

@@ -4,19 +4,18 @@ import (
"context" "context"
"io" "io"
"github.com/docker/buildx/build"
controllerapi "github.com/docker/buildx/controller/pb" controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
) )
type BuildxController interface { type BuildxController interface {
Build(ctx context.Context, options *controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, inputs *build.Inputs, err error) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process. // Invoke starts an IO session into the specified process.
// If pid doesn't match to any running processes, it starts a new process with the specified config. // If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is specified, the process will start in a newly created container. // If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach). // NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options *controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error Kill(ctx context.Context) error
Close() error Close() error
List(ctx context.Context) (refs []string, _ error) List(ctx context.Context) (refs []string, _ error)

View File

@@ -1,10 +1,7 @@
package errdefs package errdefs
import ( import (
"io"
"github.com/containerd/typeurl/v2" "github.com/containerd/typeurl/v2"
"github.com/docker/buildx/util/desktop"
"github.com/moby/buildkit/util/grpcerrors" "github.com/moby/buildkit/util/grpcerrors"
) )
@@ -13,7 +10,7 @@ func init() {
} }
type BuildError struct { type BuildError struct {
*Build Build
error error
} }
@@ -22,27 +19,16 @@ func (e *BuildError) Unwrap() error {
} }
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto { func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return e.Build return &e.Build
} }
func (e *BuildError) PrintBuildDetails(w io.Writer) error { func WrapBuild(err error, ref string) error {
if e.Ref == "" {
return nil
}
ebr := &desktop.ErrorWithBuildRef{
Ref: e.Ref,
Err: e.error,
}
return ebr.Print(w)
}
func WrapBuild(err error, sessionID string, ref string) error {
if err == nil { if err == nil {
return nil return nil
} }
return &BuildError{Build: &Build{SessionID: sessionID, Ref: ref}, error: err} return &BuildError{Build: Build{Ref: ref}, error: err}
} }
func (b *Build) WrapError(err error) error { func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: b} return &BuildError{error: err, Build: *b}
} }

View File

@@ -1,157 +1,77 @@
// Code generated by protoc-gen-go. DO NOT EDIT. // Code generated by protoc-gen-gogo. DO NOT EDIT.
// versions: // source: errdefs.proto
// protoc-gen-go v1.34.1
// protoc v3.11.4
// source: github.com/docker/buildx/controller/errdefs/errdefs.proto
package errdefs package errdefs
import ( import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect" fmt "fmt"
protoimpl "google.golang.org/protobuf/runtime/protoimpl" proto "github.com/gogo/protobuf/proto"
reflect "reflect" _ "github.com/moby/buildkit/solver/pb"
sync "sync" math "math"
) )
const ( // Reference imports to suppress errors if they are not otherwise used.
// Verify that this generated code is sufficiently up-to-date. var _ = proto.Marshal
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) var _ = fmt.Errorf
// Verify that runtime/protoimpl is sufficiently up-to-date. var _ = math.Inf
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
) // This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct { type Build struct {
state protoimpl.MessageState Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
sizeCache protoimpl.SizeCache XXX_NoUnkeyedLiteral struct{} `json:"-"`
unknownFields protoimpl.UnknownFields XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
SessionID string `protobuf:"bytes,1,opt,name=SessionID,proto3" json:"SessionID,omitempty"`
Ref string `protobuf:"bytes,2,opt,name=Ref,proto3" json:"Ref,omitempty"`
} }
func (x *Build) Reset() { func (m *Build) Reset() { *m = Build{} }
*x = Build{} func (m *Build) String() string { return proto.CompactTextString(m) }
if protoimpl.UnsafeEnabled { func (*Build) ProtoMessage() {}
mi := &file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Build) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Build) ProtoMessage() {}
func (x *Build) ProtoReflect() protoreflect.Message {
mi := &file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Build.ProtoReflect.Descriptor instead.
func (*Build) Descriptor() ([]byte, []int) { func (*Build) Descriptor() ([]byte, []int) {
return file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescGZIP(), []int{0} return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
} }
func (x *Build) GetSessionID() string { var xxx_messageInfo_Build proto.InternalMessageInfo
if x != nil {
return x.SessionID func (m *Build) GetRef() string {
if m != nil {
return m.Ref
} }
return "" return ""
} }
func (x *Build) GetRef() string { func init() {
if x != nil { proto.RegisterType((*Build)(nil), "errdefs.Build")
return x.Ref
}
return ""
} }
var File_github_com_docker_buildx_controller_errdefs_errdefs_proto protoreflect.FileDescriptor func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc = []byte{ var fileDescriptor_689dc58a5060aff5 = []byte{
0x0a, 0x39, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x6f, 0x63, // 111 bytes of a gzipped FileDescriptorProto
0x6b, 0x65, 0x72, 0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2f, 0x63, 0x6f, 0x6e, 0x74, 0x72, 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x6f, 0x6c, 0x6c, 0x65, 0x72, 0x2f, 0x65, 0x72, 0x72, 0x64, 0x65, 0x66, 0x73, 0x2f, 0x65, 0x72, 0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x72, 0x64, 0x65, 0x66, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x15, 0x64, 0x6f, 0x63, 0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x6b, 0x65, 0x72, 0x2e, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2e, 0x65, 0x72, 0x72, 0x64, 0x65, 0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x66, 0x73, 0x22, 0x37, 0x0a, 0x05, 0x42, 0x75, 0x69, 0x6c, 0x64, 0x12, 0x1c, 0x0a, 0x09, 0x53, 0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0x65, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09, 0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x53, 0x65, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x10, 0x0a, 0x03, 0x52, 0x65, 0x66, 0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x52, 0x65, 0x66, 0x42, 0x2d, 0x5a, 0x2b, 0x67,
0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x6f, 0x63, 0x6b, 0x65, 0x72,
0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2f, 0x63, 0x6f, 0x6e, 0x74, 0x72, 0x6f, 0x6c, 0x6c,
0x65, 0x72, 0x2f, 0x65, 0x72, 0x72, 0x64, 0x65, 0x66, 0x73, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x33,
}
var (
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescOnce sync.Once
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData = file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc
)
func file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescGZIP() []byte {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescOnce.Do(func() {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData = protoimpl.X.CompressGZIP(file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData)
})
return file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData
}
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes = []interface{}{
(*Build)(nil), // 0: docker.buildx.errdefs.Build
}
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs = []int32{
0, // [0:0] is the sub-list for method output_type
0, // [0:0] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
}
func init() { file_github_com_docker_buildx_controller_errdefs_errdefs_proto_init() }
func file_github_com_docker_buildx_controller_errdefs_errdefs_proto_init() {
if File_github_com_docker_buildx_controller_errdefs_errdefs_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Build); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes,
DependencyIndexes: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs,
MessageInfos: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes,
}.Build()
File_github_com_docker_buildx_controller_errdefs_errdefs_proto = out.File
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc = nil
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes = nil
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs = nil
} }

View File

@@ -1,10 +1,9 @@
syntax = "proto3"; syntax = "proto3";
package docker.buildx.errdefs; package errdefs;
option go_package = "github.com/docker/buildx/controller/errdefs"; import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build { message Build {
string SessionID = 1; string Ref = 1;
string Ref = 2; }
}

View File

@@ -1,241 +0,0 @@
// Code generated by protoc-gen-go-vtproto. DO NOT EDIT.
// protoc-gen-go-vtproto version: v0.6.1-0.20240319094008-0393e58bdf10
// source: github.com/docker/buildx/controller/errdefs/errdefs.proto
package errdefs
import (
fmt "fmt"
protohelpers "github.com/planetscale/vtprotobuf/protohelpers"
proto "google.golang.org/protobuf/proto"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
io "io"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
func (m *Build) CloneVT() *Build {
if m == nil {
return (*Build)(nil)
}
r := new(Build)
r.SessionID = m.SessionID
r.Ref = m.Ref
if len(m.unknownFields) > 0 {
r.unknownFields = make([]byte, len(m.unknownFields))
copy(r.unknownFields, m.unknownFields)
}
return r
}
func (m *Build) CloneMessageVT() proto.Message {
return m.CloneVT()
}
func (this *Build) EqualVT(that *Build) bool {
if this == that {
return true
} else if this == nil || that == nil {
return false
}
if this.SessionID != that.SessionID {
return false
}
if this.Ref != that.Ref {
return false
}
return string(this.unknownFields) == string(that.unknownFields)
}
func (this *Build) EqualMessageVT(thatMsg proto.Message) bool {
that, ok := thatMsg.(*Build)
if !ok {
return false
}
return this.EqualVT(that)
}
func (m *Build) MarshalVT() (dAtA []byte, err error) {
if m == nil {
return nil, nil
}
size := m.SizeVT()
dAtA = make([]byte, size)
n, err := m.MarshalToSizedBufferVT(dAtA[:size])
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
func (m *Build) MarshalToVT(dAtA []byte) (int, error) {
size := m.SizeVT()
return m.MarshalToSizedBufferVT(dAtA[:size])
}
func (m *Build) MarshalToSizedBufferVT(dAtA []byte) (int, error) {
if m == nil {
return 0, nil
}
i := len(dAtA)
_ = i
var l int
_ = l
if m.unknownFields != nil {
i -= len(m.unknownFields)
copy(dAtA[i:], m.unknownFields)
}
if len(m.Ref) > 0 {
i -= len(m.Ref)
copy(dAtA[i:], m.Ref)
i = protohelpers.EncodeVarint(dAtA, i, uint64(len(m.Ref)))
i--
dAtA[i] = 0x12
}
if len(m.SessionID) > 0 {
i -= len(m.SessionID)
copy(dAtA[i:], m.SessionID)
i = protohelpers.EncodeVarint(dAtA, i, uint64(len(m.SessionID)))
i--
dAtA[i] = 0xa
}
return len(dAtA) - i, nil
}
func (m *Build) SizeVT() (n int) {
if m == nil {
return 0
}
var l int
_ = l
l = len(m.SessionID)
if l > 0 {
n += 1 + l + protohelpers.SizeOfVarint(uint64(l))
}
l = len(m.Ref)
if l > 0 {
n += 1 + l + protohelpers.SizeOfVarint(uint64(l))
}
n += len(m.unknownFields)
return n
}
func (m *Build) UnmarshalVT(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: Build: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: Build: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field SessionID", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return protohelpers.ErrInvalidLength
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return protohelpers.ErrInvalidLength
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.SessionID = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
case 2:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Ref", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return protohelpers.ErrInvalidLength
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return protohelpers.ErrInvalidLength
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Ref = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := protohelpers.Skip(dAtA[iNdEx:])
if err != nil {
return err
}
if (skippy < 0) || (iNdEx+skippy) < 0 {
return protohelpers.ErrInvalidLength
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
m.unknownFields = append(m.unknownFields, dAtA[iNdEx:iNdEx+skippy]...)
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -11,7 +11,6 @@ import (
controllererrors "github.com/docker/buildx/controller/errdefs" controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb" controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes" "github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/ioset" "github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command" "github.com/docker/cli/cli/command"
@@ -22,7 +21,7 @@ import (
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController { func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{ return &localController{
dockerCli: dockerCli, dockerCli: dockerCli,
sessionID: "local", ref: "local",
processes: processes.NewManager(), processes: processes.NewManager(),
} }
} }
@@ -36,51 +35,46 @@ type buildConfig struct {
type localController struct { type localController struct {
dockerCli command.Cli dockerCli command.Cli
sessionID string ref string
buildConfig buildConfig buildConfig buildConfig
processes *processes.Manager processes *processes.Manager
buildOnGoing atomic.Bool buildOnGoing atomic.Bool
} }
func (b *localController) Build(ctx context.Context, options *controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, *build.Inputs, error) { func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) { if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, nil, errors.New("build ongoing") return "", nil, errors.New("build ongoing")
} }
defer b.buildOnGoing.Store(false) defer b.buildOnGoing.Store(false)
resp, res, dockerfileMappings, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true) resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error. // NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil { if res != nil {
b.buildConfig = buildConfig{ b.buildConfig = buildConfig{
resultCtx: res, resultCtx: res,
buildOptions: options, buildOptions: &options,
} }
if buildErr != nil { if buildErr != nil {
var ref string buildErr = controllererrors.WrapBuild(buildErr, b.ref)
var ebr *desktop.ErrorWithBuildRef
if errors.As(buildErr, &ebr) {
ref = ebr.Ref
}
buildErr = controllererrors.WrapBuild(buildErr, b.sessionID, ref)
} }
} }
if buildErr != nil { if buildErr != nil {
return "", nil, nil, buildErr return "", nil, buildErr
} }
return b.sessionID, resp, dockerfileMappings, nil return b.ref, resp, nil
} }
func (b *localController) ListProcesses(ctx context.Context, sessionID string) (infos []*controllerapi.ProcessInfo, retErr error) { func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if sessionID != b.sessionID { if ref != b.ref {
return nil, errors.Errorf("unknown session ID %q", sessionID) return nil, errors.Errorf("unknown ref %q", ref)
} }
return b.processes.ListProcesses(), nil return b.processes.ListProcesses(), nil
} }
func (b *localController) DisconnectProcess(ctx context.Context, sessionID, pid string) error { func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if sessionID != b.sessionID { if ref != b.ref {
return errors.Errorf("unknown session ID %q", sessionID) return errors.Errorf("unknown ref %q", ref)
} }
return b.processes.DeleteProcess(pid) return b.processes.DeleteProcess(pid)
} }
@@ -89,9 +83,9 @@ func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses() b.processes.CancelRunningProcesses()
} }
func (b *localController) Invoke(ctx context.Context, sessionID string, pid string, cfg *controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error { func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if sessionID != b.sessionID { if ref != b.ref {
return errors.Errorf("unknown session ID %q", sessionID) return errors.Errorf("unknown ref %q", ref)
} }
proc, ok := b.processes.Get(pid) proc, ok := b.processes.Get(pid)
@@ -101,7 +95,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
return errors.New("no build result is registered") return errors.New("no build result is registered")
} }
var err error var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, cfg) proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil { if err != nil {
return err return err
} }
@@ -109,7 +103,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
// Attach containerIn to this process // Attach containerIn to this process
ioCancelledCh := make(chan struct{}) ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func(error) { close(ioCancelledCh) }) proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select { select {
case <-ioCancelledCh: case <-ioCancelledCh:
@@ -117,7 +111,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
case err := <-proc.Done(): case err := <-proc.Done():
return err return err
case <-ctx.Done(): case <-ctx.Done():
return context.Cause(ctx) return ctx.Err()
} }
} }
@@ -136,7 +130,7 @@ func (b *localController) Close() error {
} }
func (b *localController) List(ctx context.Context) (res []string, _ error) { func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.sessionID}, nil return []string{b.ref}, nil
} }
func (b *localController) Disconnect(ctx context.Context, key string) error { func (b *localController) Disconnect(ctx context.Context, key string) error {
@@ -144,9 +138,9 @@ func (b *localController) Disconnect(ctx context.Context, key string) error {
return nil return nil
} }
func (b *localController) Inspect(ctx context.Context, sessionID string) (*controllerapi.InspectResponse, error) { func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if sessionID != b.sessionID { if ref != b.ref {
return nil, errors.Errorf("unknown session ID %q", sessionID) return nil, errors.Errorf("unknown ref %q", ref)
} }
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
} }

File diff suppressed because it is too large Load Diff

View File

@@ -5,7 +5,7 @@ package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto"; import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto"; import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "github.com/docker/buildx/controller/pb"; option go_package = "pb";
service Controller { service Controller {
rpc Build(BuildRequest) returns (BuildResponse); rpc Build(BuildRequest) returns (BuildResponse);
@@ -21,7 +21,7 @@ service Controller {
} }
message ListProcessesRequest { message ListProcessesRequest {
string SessionID = 1; string Ref = 1;
} }
message ListProcessesResponse { message ListProcessesResponse {
@@ -34,7 +34,7 @@ message ProcessInfo {
} }
message DisconnectProcessRequest { message DisconnectProcessRequest {
string SessionID = 1; string Ref = 1;
string ProcessID = 2; string ProcessID = 2;
} }
@@ -42,7 +42,7 @@ message DisconnectProcessResponse {
} }
message BuildRequest { message BuildRequest {
string SessionID = 1; string Ref = 1;
BuildOptions Options = 2; BuildOptions Options = 2;
} }
@@ -118,7 +118,7 @@ message CallFunc {
} }
message InspectRequest { message InspectRequest {
string SessionID = 1; string Ref = 1;
} }
message InspectResponse { message InspectResponse {
@@ -140,13 +140,13 @@ message BuildResponse {
} }
message DisconnectRequest { message DisconnectRequest {
string SessionID = 1; string Ref = 1;
} }
message DisconnectResponse {} message DisconnectResponse {}
message ListRequest { message ListRequest {
string SessionID = 1; string Ref = 1;
} }
message ListResponse { message ListResponse {
@@ -161,7 +161,7 @@ message InputMessage {
} }
message InputInitMessage { message InputInitMessage {
string SessionID = 1; string Ref = 1;
} }
message DataMessage { message DataMessage {
@@ -186,7 +186,7 @@ message Message {
} }
message InitMessage { message InitMessage {
string SessionID = 1; string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it // If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored. // instead of invoking the new one. In this case, InvokeConfig will be ignored.
@@ -227,7 +227,7 @@ message SignalMessage {
} }
message StatusRequest { message StatusRequest {
string SessionID = 1; string Ref = 1;
} }
message StatusResponse { message StatusResponse {

View File

@@ -1,452 +0,0 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.5.1
// - protoc v3.11.4
// source: github.com/docker/buildx/controller/pb/controller.proto
package pb
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
Controller_Build_FullMethodName = "/buildx.controller.v1.Controller/Build"
Controller_Inspect_FullMethodName = "/buildx.controller.v1.Controller/Inspect"
Controller_Status_FullMethodName = "/buildx.controller.v1.Controller/Status"
Controller_Input_FullMethodName = "/buildx.controller.v1.Controller/Input"
Controller_Invoke_FullMethodName = "/buildx.controller.v1.Controller/Invoke"
Controller_List_FullMethodName = "/buildx.controller.v1.Controller/List"
Controller_Disconnect_FullMethodName = "/buildx.controller.v1.Controller/Disconnect"
Controller_Info_FullMethodName = "/buildx.controller.v1.Controller/Info"
Controller_ListProcesses_FullMethodName = "/buildx.controller.v1.Controller/ListProcesses"
Controller_DisconnectProcess_FullMethodName = "/buildx.controller.v1.Controller/DisconnectProcess"
)
// ControllerClient is the client API for Controller service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type ControllerClient interface {
Build(ctx context.Context, in *BuildRequest, opts ...grpc.CallOption) (*BuildResponse, error)
Inspect(ctx context.Context, in *InspectRequest, opts ...grpc.CallOption) (*InspectResponse, error)
Status(ctx context.Context, in *StatusRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[StatusResponse], error)
Input(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[InputMessage, InputResponse], error)
Invoke(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[Message, Message], error)
List(ctx context.Context, in *ListRequest, opts ...grpc.CallOption) (*ListResponse, error)
Disconnect(ctx context.Context, in *DisconnectRequest, opts ...grpc.CallOption) (*DisconnectResponse, error)
Info(ctx context.Context, in *InfoRequest, opts ...grpc.CallOption) (*InfoResponse, error)
ListProcesses(ctx context.Context, in *ListProcessesRequest, opts ...grpc.CallOption) (*ListProcessesResponse, error)
DisconnectProcess(ctx context.Context, in *DisconnectProcessRequest, opts ...grpc.CallOption) (*DisconnectProcessResponse, error)
}
type controllerClient struct {
cc grpc.ClientConnInterface
}
func NewControllerClient(cc grpc.ClientConnInterface) ControllerClient {
return &controllerClient{cc}
}
func (c *controllerClient) Build(ctx context.Context, in *BuildRequest, opts ...grpc.CallOption) (*BuildResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(BuildResponse)
err := c.cc.Invoke(ctx, Controller_Build_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Inspect(ctx context.Context, in *InspectRequest, opts ...grpc.CallOption) (*InspectResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InspectResponse)
err := c.cc.Invoke(ctx, Controller_Inspect_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Status(ctx context.Context, in *StatusRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[StatusResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[0], Controller_Status_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[StatusRequest, StatusResponse]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_StatusClient = grpc.ServerStreamingClient[StatusResponse]
func (c *controllerClient) Input(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[InputMessage, InputResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[1], Controller_Input_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[InputMessage, InputResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InputClient = grpc.ClientStreamingClient[InputMessage, InputResponse]
func (c *controllerClient) Invoke(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[Message, Message], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[2], Controller_Invoke_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[Message, Message]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InvokeClient = grpc.BidiStreamingClient[Message, Message]
func (c *controllerClient) List(ctx context.Context, in *ListRequest, opts ...grpc.CallOption) (*ListResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListResponse)
err := c.cc.Invoke(ctx, Controller_List_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Disconnect(ctx context.Context, in *DisconnectRequest, opts ...grpc.CallOption) (*DisconnectResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DisconnectResponse)
err := c.cc.Invoke(ctx, Controller_Disconnect_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Info(ctx context.Context, in *InfoRequest, opts ...grpc.CallOption) (*InfoResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InfoResponse)
err := c.cc.Invoke(ctx, Controller_Info_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) ListProcesses(ctx context.Context, in *ListProcessesRequest, opts ...grpc.CallOption) (*ListProcessesResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListProcessesResponse)
err := c.cc.Invoke(ctx, Controller_ListProcesses_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) DisconnectProcess(ctx context.Context, in *DisconnectProcessRequest, opts ...grpc.CallOption) (*DisconnectProcessResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DisconnectProcessResponse)
err := c.cc.Invoke(ctx, Controller_DisconnectProcess_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// ControllerServer is the server API for Controller service.
// All implementations should embed UnimplementedControllerServer
// for forward compatibility.
type ControllerServer interface {
Build(context.Context, *BuildRequest) (*BuildResponse, error)
Inspect(context.Context, *InspectRequest) (*InspectResponse, error)
Status(*StatusRequest, grpc.ServerStreamingServer[StatusResponse]) error
Input(grpc.ClientStreamingServer[InputMessage, InputResponse]) error
Invoke(grpc.BidiStreamingServer[Message, Message]) error
List(context.Context, *ListRequest) (*ListResponse, error)
Disconnect(context.Context, *DisconnectRequest) (*DisconnectResponse, error)
Info(context.Context, *InfoRequest) (*InfoResponse, error)
ListProcesses(context.Context, *ListProcessesRequest) (*ListProcessesResponse, error)
DisconnectProcess(context.Context, *DisconnectProcessRequest) (*DisconnectProcessResponse, error)
}
// UnimplementedControllerServer should be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedControllerServer struct{}
func (UnimplementedControllerServer) Build(context.Context, *BuildRequest) (*BuildResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Build not implemented")
}
func (UnimplementedControllerServer) Inspect(context.Context, *InspectRequest) (*InspectResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Inspect not implemented")
}
func (UnimplementedControllerServer) Status(*StatusRequest, grpc.ServerStreamingServer[StatusResponse]) error {
return status.Errorf(codes.Unimplemented, "method Status not implemented")
}
func (UnimplementedControllerServer) Input(grpc.ClientStreamingServer[InputMessage, InputResponse]) error {
return status.Errorf(codes.Unimplemented, "method Input not implemented")
}
func (UnimplementedControllerServer) Invoke(grpc.BidiStreamingServer[Message, Message]) error {
return status.Errorf(codes.Unimplemented, "method Invoke not implemented")
}
func (UnimplementedControllerServer) List(context.Context, *ListRequest) (*ListResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method List not implemented")
}
func (UnimplementedControllerServer) Disconnect(context.Context, *DisconnectRequest) (*DisconnectResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Disconnect not implemented")
}
func (UnimplementedControllerServer) Info(context.Context, *InfoRequest) (*InfoResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Info not implemented")
}
func (UnimplementedControllerServer) ListProcesses(context.Context, *ListProcessesRequest) (*ListProcessesResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ListProcesses not implemented")
}
func (UnimplementedControllerServer) DisconnectProcess(context.Context, *DisconnectProcessRequest) (*DisconnectProcessResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method DisconnectProcess not implemented")
}
func (UnimplementedControllerServer) testEmbeddedByValue() {}
// UnsafeControllerServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to ControllerServer will
// result in compilation errors.
type UnsafeControllerServer interface {
mustEmbedUnimplementedControllerServer()
}
func RegisterControllerServer(s grpc.ServiceRegistrar, srv ControllerServer) {
// If the following call pancis, it indicates UnimplementedControllerServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&Controller_ServiceDesc, srv)
}
func _Controller_Build_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(BuildRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Build(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Build_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Build(ctx, req.(*BuildRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Inspect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InspectRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Inspect(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Inspect_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Inspect(ctx, req.(*InspectRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Status_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(StatusRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(ControllerServer).Status(m, &grpc.GenericServerStream[StatusRequest, StatusResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_StatusServer = grpc.ServerStreamingServer[StatusResponse]
func _Controller_Input_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(ControllerServer).Input(&grpc.GenericServerStream[InputMessage, InputResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InputServer = grpc.ClientStreamingServer[InputMessage, InputResponse]
func _Controller_Invoke_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(ControllerServer).Invoke(&grpc.GenericServerStream[Message, Message]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InvokeServer = grpc.BidiStreamingServer[Message, Message]
func _Controller_List_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).List(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_List_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).List(ctx, req.(*ListRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Disconnect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DisconnectRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Disconnect(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Disconnect_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Disconnect(ctx, req.(*DisconnectRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Info_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InfoRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Info(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Info_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Info(ctx, req.(*InfoRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_ListProcesses_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListProcessesRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).ListProcesses(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_ListProcesses_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).ListProcesses(ctx, req.(*ListProcessesRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_DisconnectProcess_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DisconnectProcessRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).DisconnectProcess(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_DisconnectProcess_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).DisconnectProcess(ctx, req.(*DisconnectProcessRequest))
}
return interceptor(ctx, in, info, handler)
}
// Controller_ServiceDesc is the grpc.ServiceDesc for Controller service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var Controller_ServiceDesc = grpc.ServiceDesc{
ServiceName: "buildx.controller.v1.Controller",
HandlerType: (*ControllerServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "Build",
Handler: _Controller_Build_Handler,
},
{
MethodName: "Inspect",
Handler: _Controller_Inspect_Handler,
},
{
MethodName: "List",
Handler: _Controller_List_Handler,
},
{
MethodName: "Disconnect",
Handler: _Controller_Disconnect_Handler,
},
{
MethodName: "Info",
Handler: _Controller_Info_Handler,
},
{
MethodName: "ListProcesses",
Handler: _Controller_ListProcesses_Handler,
},
{
MethodName: "DisconnectProcess",
Handler: _Controller_DisconnectProcess_Handler,
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "Status",
Handler: _Controller_Status_Handler,
ServerStreams: true,
},
{
StreamName: "Input",
Handler: _Controller_Input_Handler,
ClientStreams: true,
},
{
StreamName: "Invoke",
Handler: _Controller_Invoke_Handler,
ServerStreams: true,
ClientStreams: true,
},
},
Metadata: "github.com/docker/buildx/controller/pb/controller.proto",
}

File diff suppressed because it is too large Load Diff

View File

@@ -10,16 +10,15 @@ import (
"github.com/pkg/errors" "github.com/pkg/errors"
) )
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, error) { func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry var outs []client.ExportEntry
var localPaths []string
if len(entries) == 0 { if len(entries) == 0 {
return nil, nil, nil return nil, nil
} }
var stdoutUsed bool var stdoutUsed bool
for _, entry := range entries { for _, entry := range entries {
if entry.Type == "" { if entry.Type == "" {
return nil, nil, errors.Errorf("type is required for output") return nil, errors.Errorf("type is required for output")
} }
out := client.ExportEntry{ out := client.ExportEntry{
@@ -46,26 +45,24 @@ func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, erro
supportDir = !tar supportDir = !tar
case "registry": case "registry":
out.Type = client.ExporterImage out.Type = client.ExporterImage
out.Attrs["push"] = "true"
} }
if supportDir { if supportDir {
if entry.Destination == "" { if entry.Destination == "" {
return nil, nil, errors.Errorf("dest is required for %s exporter", out.Type) return nil, errors.Errorf("dest is required for %s exporter", out.Type)
} }
if entry.Destination == "-" { if entry.Destination == "-" {
return nil, nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type) return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
} }
fi, err := os.Stat(entry.Destination) fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) { if err != nil && !os.IsNotExist(err) {
return nil, nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination) return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
} }
if err == nil && !fi.IsDir() { if err == nil && !fi.IsDir() {
return nil, nil, errors.Errorf("destination directory %s is a file", entry.Destination) return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
} }
out.OutputDir = entry.Destination out.OutputDir = entry.Destination
localPaths = append(localPaths, entry.Destination)
} }
if supportFile { if supportFile {
if entry.Destination == "" && out.Type != client.ExporterDocker { if entry.Destination == "" && out.Type != client.ExporterDocker {
@@ -73,33 +70,32 @@ func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, erro
} }
if entry.Destination == "-" { if entry.Destination == "-" {
if stdoutUsed { if stdoutUsed {
return nil, nil, errors.Errorf("multiple outputs configured to write to stdout") return nil, errors.Errorf("multiple outputs configured to write to stdout")
} }
if _, err := console.ConsoleFromFile(os.Stdout); err == nil { if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type) return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
} }
out.Output = wrapWriteCloser(os.Stdout) out.Output = wrapWriteCloser(os.Stdout)
stdoutUsed = true stdoutUsed = true
} else if entry.Destination != "" { } else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination) fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) { if err != nil && !os.IsNotExist(err) {
return nil, nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination) return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
} }
if err == nil && fi.IsDir() { if err == nil && fi.IsDir() {
return nil, nil, errors.Errorf("destination file %s is a directory", entry.Destination) return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
} }
f, err := os.Create(entry.Destination) f, err := os.Create(entry.Destination)
if err != nil { if err != nil {
return nil, nil, errors.Errorf("failed to open %s", err) return nil, errors.Errorf("failed to open %s", err)
} }
out.Output = wrapWriteCloser(f) out.Output = wrapWriteCloser(f)
localPaths = append(localPaths, entry.Destination)
} }
} }
outs = append(outs, out) outs = append(outs, out)
} }
return outs, localPaths, nil return outs, nil
} }
func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) { func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) {

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

View File

@@ -153,6 +153,7 @@ func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
} }
} }
ps = append(ps, p) ps = append(ps, p)
} }
s.Paths = ps s.Paths = ps
ssh = append(ssh, s) ssh = append(ssh, s)

View File

@@ -3,10 +3,10 @@ package pb
import ( import (
"os" "os"
"path/filepath" "path/filepath"
"reflect"
"testing" "testing"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
"google.golang.org/protobuf/proto"
) )
func TestResolvePaths(t *testing.T) { func TestResolvePaths(t *testing.T) {
@@ -16,58 +16,54 @@ func TestResolvePaths(t *testing.T) {
require.NoError(t, os.Chdir(tmpwd)) require.NoError(t, os.Chdir(tmpwd))
tests := []struct { tests := []struct {
name string name string
options *BuildOptions options BuildOptions
want *BuildOptions want BuildOptions
}{ }{
{ {
name: "contextpath", name: "contextpath",
options: &BuildOptions{ContextPath: "test"}, options: BuildOptions{ContextPath: "test"},
want: &BuildOptions{ContextPath: filepath.Join(tmpwd, "test")}, want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
}, },
{ {
name: "contextpath-cwd", name: "contextpath-cwd",
options: &BuildOptions{ContextPath: "."}, options: BuildOptions{ContextPath: "."},
want: &BuildOptions{ContextPath: tmpwd}, want: BuildOptions{ContextPath: tmpwd},
}, },
{ {
name: "contextpath-dash", name: "contextpath-dash",
options: &BuildOptions{ContextPath: "-"}, options: BuildOptions{ContextPath: "-"},
want: &BuildOptions{ContextPath: "-"}, want: BuildOptions{ContextPath: "-"},
}, },
{ {
name: "contextpath-ssh", name: "contextpath-ssh",
options: &BuildOptions{ContextPath: "git@github.com:docker/buildx.git"}, options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: &BuildOptions{ContextPath: "git@github.com:docker/buildx.git"}, want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
}, },
{ {
name: "dockerfilename", name: "dockerfilename",
options: &BuildOptions{DockerfileName: "test", ContextPath: "."}, options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: &BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd}, want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
}, },
{ {
name: "dockerfilename-dash", name: "dockerfilename-dash",
options: &BuildOptions{DockerfileName: "-", ContextPath: "."}, options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: &BuildOptions{DockerfileName: "-", ContextPath: tmpwd}, want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
}, },
{ {
name: "dockerfilename-remote", name: "dockerfilename-remote",
options: &BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"}, options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: &BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"}, want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
}, },
{ {
name: "contexts", name: "contexts",
options: &BuildOptions{NamedContexts: map[string]string{ options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"a": "test1", "b": "test2", "alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git", want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
}}, "alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: &BuildOptions{NamedContexts: map[string]string{
"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git",
}},
}, },
{ {
name: "cache-from", name: "cache-from",
options: &BuildOptions{ options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{ CacheFrom: []*CacheOptionsEntry{
{ {
Type: "local", Type: "local",
@@ -79,7 +75,7 @@ func TestResolvePaths(t *testing.T) {
}, },
}, },
}, },
want: &BuildOptions{ want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{ CacheFrom: []*CacheOptionsEntry{
{ {
Type: "local", Type: "local",
@@ -94,7 +90,7 @@ func TestResolvePaths(t *testing.T) {
}, },
{ {
name: "cache-to", name: "cache-to",
options: &BuildOptions{ options: BuildOptions{
CacheTo: []*CacheOptionsEntry{ CacheTo: []*CacheOptionsEntry{
{ {
Type: "local", Type: "local",
@@ -106,7 +102,7 @@ func TestResolvePaths(t *testing.T) {
}, },
}, },
}, },
want: &BuildOptions{ want: BuildOptions{
CacheTo: []*CacheOptionsEntry{ CacheTo: []*CacheOptionsEntry{
{ {
Type: "local", Type: "local",
@@ -121,7 +117,7 @@ func TestResolvePaths(t *testing.T) {
}, },
{ {
name: "exports", name: "exports",
options: &BuildOptions{ options: BuildOptions{
Exports: []*ExportEntry{ Exports: []*ExportEntry{
{ {
Type: "local", Type: "local",
@@ -149,7 +145,7 @@ func TestResolvePaths(t *testing.T) {
}, },
}, },
}, },
want: &BuildOptions{ want: BuildOptions{
Exports: []*ExportEntry{ Exports: []*ExportEntry{
{ {
Type: "local", Type: "local",
@@ -180,7 +176,7 @@ func TestResolvePaths(t *testing.T) {
}, },
{ {
name: "secrets", name: "secrets",
options: &BuildOptions{ options: BuildOptions{
Secrets: []*Secret{ Secrets: []*Secret{
{ {
FilePath: "test1", FilePath: "test1",
@@ -195,7 +191,7 @@ func TestResolvePaths(t *testing.T) {
}, },
}, },
}, },
want: &BuildOptions{ want: BuildOptions{
Secrets: []*Secret{ Secrets: []*Secret{
{ {
FilePath: filepath.Join(tmpwd, "test1"), FilePath: filepath.Join(tmpwd, "test1"),
@@ -213,7 +209,7 @@ func TestResolvePaths(t *testing.T) {
}, },
{ {
name: "ssh", name: "ssh",
options: &BuildOptions{ options: BuildOptions{
SSH: []*SSH{ SSH: []*SSH{
{ {
ID: "default", ID: "default",
@@ -225,7 +221,7 @@ func TestResolvePaths(t *testing.T) {
}, },
}, },
}, },
want: &BuildOptions{ want: BuildOptions{
SSH: []*SSH{ SSH: []*SSH{
{ {
ID: "default", ID: "default",
@@ -242,10 +238,10 @@ func TestResolvePaths(t *testing.T) {
for _, tt := range tests { for _, tt := range tests {
tt := tt tt := tt
t.Run(tt.name, func(t *testing.T) { t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(tt.options) got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err) require.NoError(t, err)
if !proto.Equal(tt.want, got) { if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, got) t.Fatalf("expected %#v, got %#v", tt.want, *got)
} }
}) })
} }

View File

@@ -1,13 +1,10 @@
package pb package pb
import ( import (
"time"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control" control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest" "github.com/opencontainers/go-digest"
"google.golang.org/protobuf/types/known/timestamppb"
) )
type writer struct { type writer struct {
@@ -22,23 +19,25 @@ func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status) w.ch <- ToControlStatus(status)
} }
func (w *writer) WriteBuildRef(target string, ref string) {} func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, any) bool { func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true return true
} }
func (w *writer) ClearLogSource(any) {} func (w *writer) ClearLogSource(interface{}) {}
func ToControlStatus(s *client.SolveStatus) *StatusResponse { func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{} resp := StatusResponse{}
for _, v := range s.Vertexes { for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{ resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: string(v.Digest), Digest: v.Digest,
Inputs: digestSliceToPB(v.Inputs), Inputs: v.Inputs,
Name: v.Name, Name: v.Name,
Started: timestampToPB(v.Started), Started: v.Started,
Completed: timestampToPB(v.Completed), Completed: v.Completed,
Error: v.Error, Error: v.Error,
Cached: v.Cached, Cached: v.Cached,
ProgressGroup: v.ProgressGroup, ProgressGroup: v.ProgressGroup,
@@ -47,26 +46,26 @@ func ToControlStatus(s *client.SolveStatus) *StatusResponse {
for _, v := range s.Statuses { for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{ resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID, ID: v.ID,
Vertex: string(v.Vertex), Vertex: v.Vertex,
Name: v.Name, Name: v.Name,
Total: v.Total, Total: v.Total,
Current: v.Current, Current: v.Current,
Timestamp: timestamppb.New(v.Timestamp), Timestamp: v.Timestamp,
Started: timestampToPB(v.Started), Started: v.Started,
Completed: timestampToPB(v.Completed), Completed: v.Completed,
}) })
} }
for _, v := range s.Logs { for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{ resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: string(v.Vertex), Vertex: v.Vertex,
Stream: int64(v.Stream), Stream: int64(v.Stream),
Msg: v.Data, Msg: v.Data,
Timestamp: timestamppb.New(v.Timestamp), Timestamp: v.Timestamp,
}) })
} }
for _, v := range s.Warnings { for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{ resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: string(v.Vertex), Vertex: v.Vertex,
Level: int64(v.Level), Level: int64(v.Level),
Short: v.Short, Short: v.Short,
Detail: v.Detail, Detail: v.Detail,
@@ -82,11 +81,11 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{} s := client.SolveStatus{}
for _, v := range resp.Vertexes { for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{ s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: digest.Digest(v.Digest), Digest: v.Digest,
Inputs: digestSliceFromPB(v.Inputs), Inputs: v.Inputs,
Name: v.Name, Name: v.Name,
Started: timestampFromPB(v.Started), Started: v.Started,
Completed: timestampFromPB(v.Completed), Completed: v.Completed,
Error: v.Error, Error: v.Error,
Cached: v.Cached, Cached: v.Cached,
ProgressGroup: v.ProgressGroup, ProgressGroup: v.ProgressGroup,
@@ -95,26 +94,26 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
for _, v := range resp.Statuses { for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{ s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID, ID: v.ID,
Vertex: digest.Digest(v.Vertex), Vertex: v.Vertex,
Name: v.Name, Name: v.Name,
Total: v.Total, Total: v.Total,
Current: v.Current, Current: v.Current,
Timestamp: v.Timestamp.AsTime(), Timestamp: v.Timestamp,
Started: timestampFromPB(v.Started), Started: v.Started,
Completed: timestampFromPB(v.Completed), Completed: v.Completed,
}) })
} }
for _, v := range resp.Logs { for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{ s.Logs = append(s.Logs, &client.VertexLog{
Vertex: digest.Digest(v.Vertex), Vertex: v.Vertex,
Stream: int(v.Stream), Stream: int(v.Stream),
Data: v.Msg, Data: v.Msg,
Timestamp: v.Timestamp.AsTime(), Timestamp: v.Timestamp,
}) })
} }
for _, v := range resp.Warnings { for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{ s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: digest.Digest(v.Vertex), Vertex: v.Vertex,
Level: int(v.Level), Level: int(v.Level),
Short: v.Short, Short: v.Short,
Detail: v.Detail, Detail: v.Detail,
@@ -125,38 +124,3 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
} }
return &s return &s
} }
func timestampFromPB(ts *timestamppb.Timestamp) *time.Time {
if ts == nil {
return nil
}
t := ts.AsTime()
if t.IsZero() {
return nil
}
return &t
}
func timestampToPB(ts *time.Time) *timestamppb.Timestamp {
if ts == nil {
return nil
}
return timestamppb.New(*ts)
}
func digestSliceFromPB(elems []string) []digest.Digest {
clone := make([]digest.Digest, len(elems))
for i, e := range elems {
clone[i] = digest.Digest(e)
}
return clone
}
func digestSliceToPB(elems []digest.Digest) []string {
clone := make([]string, len(elems))
for i, e := range elems {
clone[i] = string(e)
}
return clone
}

View File

@@ -1,8 +1,6 @@
package pb package pb
import ( import (
"slices"
"github.com/moby/buildkit/session" "github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider" "github.com/moby/buildkit/session/sshforward/sshprovider"
) )
@@ -12,7 +10,7 @@ func CreateSSH(ssh []*SSH) (session.Attachable, error) {
for _, ssh := range ssh { for _, ssh := range ssh {
cfg := sshprovider.AgentConfig{ cfg := sshprovider.AgentConfig{
ID: ssh.ID, ID: ssh.ID,
Paths: slices.Clone(ssh.Paths), Paths: append([]string{}, ssh.Paths...),
} }
configs = append(configs, cfg) configs = append(configs, cfg)
} }

View File

@@ -18,16 +18,16 @@ type Process struct {
invokeConfig *pb.InvokeConfig invokeConfig *pb.InvokeConfig
errCh chan error errCh chan error
processCancel func() processCancel func()
serveIOCancel func(error) serveIOCancel func()
} }
// ForwardIO forwards process's io to the specified reader/writer. // ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when // Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup. // the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func(error)) { func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in) p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil { if f := p.serveIOCancel; f != nil {
f(errors.WithStack(context.Canceled)) f()
} }
p.serveIOCancel = ioCancelCallback p.serveIOCancel = ioCancelCallback
} }
@@ -39,7 +39,7 @@ func (p *Process) Done() <-chan error {
return p.errCh return p.errCh
} }
// Manager manages a set of processes. // Manager manages a set of proceses.
type Manager struct { type Manager struct {
container atomic.Value container atomic.Value
processes sync.Map processes sync.Map
@@ -124,16 +124,9 @@ func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *p
f.SetOut(&out) f.SetOut(&out)
// Register process // Register process
ctx, cancel := context.WithCancelCause(context.TODO()) ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once var cancelOnce sync.Once
processCancelFunc := func() { processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
cancelOnce.Do(func() {
cancel(errors.WithStack(context.Canceled))
f.Close()
in.Close()
out.Close()
})
}
p := &Process{ p := &Process{
inEnd: f, inEnd: f,
invokeConfig: cfg, invokeConfig: cfg,

View File

@@ -6,9 +6,8 @@ import (
"sync" "sync"
"time" "time"
"github.com/containerd/containerd/v2/defaults" "github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/v2/pkg/dialer" "github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb" "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client" "github.com/moby/buildkit/client"
@@ -28,7 +27,6 @@ func NewClient(ctx context.Context, addr string) (*Client, error) {
Backoff: backoffConfig, Backoff: backoffConfig,
} }
gopts := []grpc.DialOption{ gopts := []grpc.DialOption{
//nolint:staticcheck // ignore SA1019: WithBlock is deprecated and does not work with NewClient.
grpc.WithBlock(), grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()), grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams), grpc.WithConnectParams(connParams),
@@ -38,7 +36,6 @@ func NewClient(ctx context.Context, addr string) (*Client, error) {
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor), grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor), grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
} }
//nolint:staticcheck // ignore SA1019: Recommended NewClient has different behavior from DialContext.
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...) conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -75,36 +72,36 @@ func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
return res.Keys, nil return res.Keys, nil
} }
func (c *Client) Disconnect(ctx context.Context, sessionID string) error { func (c *Client) Disconnect(ctx context.Context, key string) error {
if sessionID == "" { if key == "" {
return nil return nil
} }
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{SessionID: sessionID}) _, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err return err
} }
func (c *Client) ListProcesses(ctx context.Context, sessionID string) (infos []*pb.ProcessInfo, retErr error) { func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{SessionID: sessionID}) res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil { if err != nil {
return nil, err return nil, err
} }
return res.Infos, nil return res.Infos, nil
} }
func (c *Client) DisconnectProcess(ctx context.Context, sessionID, pid string) error { func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{SessionID: sessionID, ProcessID: pid}) _, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err return err
} }
func (c *Client) Invoke(ctx context.Context, sessionID string, pid string, invokeConfig *pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error { func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if sessionID == "" || pid == "" { if ref == "" || pid == "" {
return errors.New("build session ID must be specified") return errors.New("build reference must be specified")
} }
stream, err := c.client().Invoke(ctx) stream, err := c.client().Invoke(ctx)
if err != nil { if err != nil {
return err return err
} }
return attachIO(ctx, stream, &pb.InitMessage{SessionID: sessionID, ProcessID: pid, InvokeConfig: invokeConfig}, ioAttachConfig{ return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in, stdin: in,
stdout: stdout, stdout: stdout,
stderr: stderr, stderr: stderr,
@@ -112,11 +109,11 @@ func (c *Client) Invoke(ctx context.Context, sessionID string, pid string, invok
}) })
} }
func (c *Client) Inspect(ctx context.Context, sessionID string) (*pb.InspectResponse, error) { func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{SessionID: sessionID}) return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
} }
func (c *Client) Build(ctx context.Context, options *pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, *build.Inputs, error) { func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID() ref := identity.NewID()
statusChan := make(chan *client.SolveStatus) statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx) eg, egCtx := errgroup.WithContext(ctx)
@@ -134,10 +131,10 @@ func (c *Client) Build(ctx context.Context, options *pb.BuildOptions, in io.Read
} }
return nil return nil
}) })
return ref, resp, nil, eg.Wait() return ref, resp, eg.Wait()
} }
func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) { func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx) eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{}) done := make(chan struct{})
@@ -146,8 +143,8 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
eg.Go(func() error { eg.Go(func() error {
defer close(done) defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{ pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
SessionID: sessionID, Ref: ref,
Options: options, Options: &options,
}) })
if err != nil { if err != nil {
return err return err
@@ -159,7 +156,7 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
}) })
eg.Go(func() error { eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{ stream, err := c.client().Status(egCtx, &pb.StatusRequest{
SessionID: sessionID, Ref: ref,
}) })
if err != nil { if err != nil {
return err return err
@@ -184,7 +181,7 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
if err := stream.Send(&pb.InputMessage{ if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{ Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{ Init: &pb.InputInitMessage{
SessionID: sessionID, Ref: ref,
}, },
}, },
}); err != nil { }); err != nil {

View File

@@ -62,10 +62,9 @@ func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts
serverRoot := filepath.Join(rootDir, "shared") serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running // connect to buildx server if it is already running
ctx2, cancel := context.WithCancelCause(ctx) ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
ctx2, _ = context.WithTimeoutCause(ctx2, 1*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename)) c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel(errors.WithStack(context.Canceled)) cancel()
if err != nil { if err != nil {
if !errors.Is(err, context.DeadlineExceeded) { if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server") return nil, errors.Wrap(err, "cannot connect to the buildx server")
@@ -91,10 +90,9 @@ func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts
go wait() go wait()
// wait for buildx server to be ready // wait for buildx server to be ready
ctx2, cancel = context.WithCancelCause(ctx) ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
ctx2, _ = context.WithTimeoutCause(ctx2, 10*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename)) c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel(errors.WithStack(context.Canceled)) cancel()
if err != nil { if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server") return errors.Wrap(err, "cannot connect to the buildx server")
} }
@@ -140,7 +138,7 @@ func serveCmd(dockerCli command.Cli) *cobra.Command {
return err return err
} }
pidF := filepath.Join(root, defaultPIDFilename) pidF := filepath.Join(root, defaultPIDFilename)
if err := os.WriteFile(pidF, fmt.Appendf(nil, "%d", os.Getpid()), 0600); err != nil { if err := os.WriteFile(pidF, []byte(fmt.Sprintf("%d", os.Getpid())), 0600); err != nil {
return err return err
} }
defer func() { defer func() {
@@ -150,8 +148,8 @@ func serveCmd(dockerCli command.Cli) *cobra.Command {
}() }()
// prepare server // prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, *build.Inputs, error) { b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, options, stdin, progress, true) return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
}) })
defer b.Close() defer b.Close()
@@ -260,7 +258,7 @@ func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error)
} }
func rootDataDir(dockerCli command.Cli) string { func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.NewConfig(dockerCli).Dir(), "controller") return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
} }
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) { func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {

View File

@@ -43,9 +43,9 @@ func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessa
if init == nil { if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput()) return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
} }
sessionID := init.SessionID ref := init.Ref
if sessionID == "" { if ref == "" {
return errors.New("no session ID is provided") return errors.New("no ref is provided")
} }
if err := initFn(init); err != nil { if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server") return errors.Wrap(err, "failed to initialize IO server")
@@ -302,6 +302,7 @@ func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage
out = cfg.stderr out = cfg.stderr
default: default:
return errors.Errorf("unsupported fd %d", file.Fd) return errors.Errorf("unsupported fd %d", file.Fd)
} }
if out == nil { if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd) logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
@@ -344,7 +345,7 @@ func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
case err := <-errCh: case err := <-errCh:
return nil, err return nil, err
case <-ctx.Done(): case <-ctx.Done():
return nil, context.Cause(ctx) return nil, ctx.Err()
} }
} }

View File

@@ -11,7 +11,6 @@ import (
controllererrors "github.com/docker/buildx/controller/errdefs" controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb" "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes" "github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/ioset" "github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress" "github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version" "github.com/docker/buildx/version"
@@ -20,7 +19,7 @@ import (
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
) )
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, inp *build.Inputs, err error) type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server { func NewServer(buildFunc BuildFunc) *Server {
return &Server{ return &Server{
@@ -37,7 +36,7 @@ type Server struct {
type session struct { type session struct {
buildOnGoing atomic.Bool buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse statusChan chan *pb.StatusResponse
cancelBuild func(error) cancelBuild func()
buildOptions *pb.BuildOptions buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter inputPipe *io.PipeWriter
@@ -53,9 +52,9 @@ func (s *session) cancelRunningProcesses() {
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) { func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock() m.sessionMu.Lock()
defer m.sessionMu.Unlock() defer m.sessionMu.Unlock()
s, ok := m.session[req.SessionID] s, ok := m.session[req.Ref]
if !ok { if !ok {
return nil, errors.Errorf("unknown session ID %q", req.SessionID) return nil, errors.Errorf("unknown ref %q", req.Ref)
} }
res = new(pb.ListProcessesResponse) res = new(pb.ListProcessesResponse)
res.Infos = append(res.Infos, s.processes.ListProcesses()...) res.Infos = append(res.Infos, s.processes.ListProcesses()...)
@@ -65,9 +64,9 @@ func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) { func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock() m.sessionMu.Lock()
defer m.sessionMu.Unlock() defer m.sessionMu.Unlock()
s, ok := m.session[req.SessionID] s, ok := m.session[req.Ref]
if !ok { if !ok {
return nil, errors.Errorf("unknown session ID %q", req.SessionID) return nil, errors.Errorf("unknown ref %q", req.Ref)
} }
return res, s.processes.DeleteProcess(req.ProcessID) return res, s.processes.DeleteProcess(req.ProcessID)
} }
@@ -101,22 +100,22 @@ func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListRes
} }
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) { func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
sessionID := req.SessionID key := req.Ref
if sessionID == "" { if key == "" {
return nil, errors.New("disconnect: empty session ID") return nil, errors.New("disconnect: empty key")
} }
m.sessionMu.Lock() m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok { if s, ok := m.session[key]; ok {
if s.cancelBuild != nil { if s.cancelBuild != nil {
s.cancelBuild(errors.WithStack(context.Canceled)) s.cancelBuild()
} }
s.cancelRunningProcesses() s.cancelRunningProcesses()
if s.result != nil { if s.result != nil {
s.result.Done() s.result.Done()
} }
} }
delete(m.session, sessionID) delete(m.session, key)
m.sessionMu.Unlock() m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil return &pb.DisconnectResponse{}, nil
@@ -127,7 +126,7 @@ func (m *Server) Close() error {
for k := range m.session { for k := range m.session {
if s, ok := m.session[k]; ok { if s, ok := m.session[k]; ok {
if s.cancelBuild != nil { if s.cancelBuild != nil {
s.cancelBuild(errors.WithStack(context.Canceled)) s.cancelBuild()
} }
s.cancelRunningProcesses() s.cancelRunningProcesses()
} }
@@ -137,26 +136,26 @@ func (m *Server) Close() error {
} }
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) { func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
sessionID := req.SessionID ref := req.Ref
if sessionID == "" { if ref == "" {
return nil, errors.New("inspect: empty session ID") return nil, errors.New("inspect: empty key")
} }
var bo *pb.BuildOptions var bo *pb.BuildOptions
m.sessionMu.Lock() m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok { if s, ok := m.session[ref]; ok {
bo = s.buildOptions bo = s.buildOptions
} else { } else {
m.sessionMu.Unlock() m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", sessionID) return nil, errors.Errorf("inspect: unknown key %v", ref)
} }
m.sessionMu.Unlock() m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil return &pb.InspectResponse{Options: bo}, nil
} }
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) { func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
sessionID := req.SessionID ref := req.Ref
if sessionID == "" { if ref == "" {
return nil, errors.New("build: empty session ID") return nil, errors.New("build: empty key")
} }
// Prepare status channel and session // Prepare status channel and session
@@ -164,7 +163,7 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
if m.session == nil { if m.session == nil {
m.session = make(map[string]*session) m.session = make(map[string]*session)
} }
s, ok := m.session[sessionID] s, ok := m.session[ref]
if ok { if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) { if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock() m.sessionMu.Unlock()
@@ -183,12 +182,12 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
inR, inW := io.Pipe() inR, inW := io.Pipe()
defer inR.Close() defer inR.Close()
s.inputPipe = inW s.inputPipe = inW
m.session[sessionID] = s m.session[ref] = s
m.sessionMu.Unlock() m.sessionMu.Unlock()
defer func() { defer func() {
close(statusChan) close(statusChan)
m.sessionMu.Lock() m.sessionMu.Lock()
s, ok := m.session[sessionID] s, ok := m.session[ref]
if ok { if ok {
s.statusChan = nil s.statusChan = nil
s.buildOnGoing.Store(false) s.buildOnGoing.Store(false)
@@ -199,29 +198,24 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
pw := pb.NewProgressWriter(statusChan) pw := pb.NewProgressWriter(statusChan)
// Build the specified request // Build the specified request
ctx, cancel := context.WithCancelCause(ctx) ctx, cancel := context.WithCancel(ctx)
defer func() { cancel(errors.WithStack(context.Canceled)) }() defer cancel()
resp, res, _, buildErr := m.buildFunc(ctx, req.Options, inR, pw) resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock() m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok { if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild). // NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil { if res != nil {
s.result = res s.result = res
s.cancelBuild = cancel s.cancelBuild = cancel
s.buildOptions = req.Options s.buildOptions = req.Options
m.session[sessionID] = s m.session[ref] = s
if buildErr != nil { if buildErr != nil {
var ref string buildErr = controllererrors.WrapBuild(buildErr, ref)
var ebr *desktop.ErrorWithBuildRef
if errors.As(buildErr, &ebr) {
ref = ebr.Ref
}
buildErr = controllererrors.WrapBuild(buildErr, sessionID, ref)
} }
} }
} else { } else {
m.sessionMu.Unlock() m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown session ID %v", sessionID) return nil, errors.Errorf("build: unknown key %v", ref)
} }
m.sessionMu.Unlock() m.sessionMu.Unlock()
@@ -238,9 +232,9 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
} }
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error { func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
sessionID := req.SessionID ref := req.Ref
if sessionID == "" { if ref == "" {
return errors.New("status: empty session ID") return errors.New("status: empty key")
} }
// Wait and get status channel prepared by Build() // Wait and get status channel prepared by Build()
@@ -248,12 +242,12 @@ func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer
for { for {
// TODO: timeout? // TODO: timeout?
m.sessionMu.Lock() m.sessionMu.Lock()
if _, ok := m.session[sessionID]; !ok || m.session[sessionID].statusChan == nil { if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock() m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue continue
} }
statusChan = m.session[sessionID].statusChan statusChan = m.session[ref].statusChan
m.sessionMu.Unlock() m.sessionMu.Unlock()
break break
} }
@@ -284,9 +278,9 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
if init == nil { if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit()) return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
} }
sessionID := init.SessionID ref := init.Ref
if sessionID == "" { if ref == "" {
return errors.New("input: no session ID is provided") return errors.New("input: no ref is provided")
} }
// Wait and get input stream pipe prepared by Build() // Wait and get input stream pipe prepared by Build()
@@ -294,12 +288,12 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
for { for {
// TODO: timeout? // TODO: timeout?
m.sessionMu.Lock() m.sessionMu.Lock()
if _, ok := m.session[sessionID]; !ok || m.session[sessionID].inputPipe == nil { if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock() m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue continue
} }
inputPipeW = m.session[sessionID].inputPipe inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock() m.sessionMu.Unlock()
break break
} }
@@ -341,7 +335,7 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
select { select {
case msg = <-msgCh: case msg = <-msgCh:
case <-ctx.Done(): case <-ctx.Done():
return context.Cause(ctx) return errors.Wrap(ctx.Err(), "canceled")
} }
if msg == nil { if msg == nil {
return nil return nil
@@ -370,23 +364,23 @@ func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
initDoneCh := make(chan *processes.Process) initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error) initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO()) eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancelCause(egCtx) srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error { eg.Go(func() error {
defer srvIOCancel(errors.WithStack(context.Canceled)) defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) { return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() { defer func() {
if retErr != nil { if retErr != nil {
initErrCh <- retErr initErrCh <- retErr
} }
}() }()
sessionID := initMessage.SessionID ref := initMessage.Ref
cfg := initMessage.InvokeConfig cfg := initMessage.InvokeConfig
m.sessionMu.Lock() m.sessionMu.Lock()
s, ok := m.session[sessionID] s, ok := m.session[ref]
if !ok { if !ok {
m.sessionMu.Unlock() m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown session ID %v", sessionID) return errors.Errorf("invoke: unknown key %v", ref)
} }
m.sessionMu.Unlock() m.sessionMu.Unlock()
@@ -418,7 +412,7 @@ func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
}) })
}) })
eg.Go(func() (rErr error) { eg.Go(func() (rErr error) {
defer srvIOCancel(errors.WithStack(context.Canceled)) defer srvIOCancel()
// Wait for init done // Wait for init done
var proc *processes.Process var proc *processes.Process
select { select {

View File

@@ -41,17 +41,11 @@ target "lint" {
platforms = GOLANGCI_LINT_MULTIPLATFORM != "" ? [ platforms = GOLANGCI_LINT_MULTIPLATFORM != "" ? [
"darwin/amd64", "darwin/amd64",
"darwin/arm64", "darwin/arm64",
"freebsd/amd64",
"freebsd/arm64",
"linux/amd64", "linux/amd64",
"linux/arm64", "linux/arm64",
"linux/s390x", "linux/s390x",
"linux/ppc64le", "linux/ppc64le",
"linux/riscv64", "linux/riscv64",
"netbsd/amd64",
"netbsd/arm64",
"openbsd/amd64",
"openbsd/arm64",
"windows/amd64", "windows/amd64",
"windows/arm64" "windows/arm64"
] : [] ] : []
@@ -160,8 +154,6 @@ target "binaries-cross" {
platforms = [ platforms = [
"darwin/amd64", "darwin/amd64",
"darwin/arm64", "darwin/arm64",
"freebsd/amd64",
"freebsd/arm64",
"linux/amd64", "linux/amd64",
"linux/arm/v6", "linux/arm/v6",
"linux/arm/v7", "linux/arm/v7",
@@ -169,10 +161,6 @@ target "binaries-cross" {
"linux/ppc64le", "linux/ppc64le",
"linux/riscv64", "linux/riscv64",
"linux/s390x", "linux/s390x",
"netbsd/amd64",
"netbsd/arm64",
"openbsd/amd64",
"openbsd/arm64",
"windows/amd64", "windows/amd64",
"windows/arm64" "windows/arm64"
] ]

View File

@@ -19,8 +19,8 @@ By default, Bake uses the following lookup order to find the configuration file:
3. `docker-compose.yml` 3. `docker-compose.yml`
4. `docker-compose.yaml` 4. `docker-compose.yaml`
5. `docker-bake.json` 5. `docker-bake.json`
6. `docker-bake.hcl` 6. `docker-bake.override.json`
7. `docker-bake.override.json` 7. `docker-bake.hcl`
8. `docker-bake.override.hcl` 8. `docker-bake.override.hcl`
You can specify the file location explicitly using the `--file` flag: You can specify the file location explicitly using the `--file` flag:
@@ -221,10 +221,8 @@ The following table shows the complete list of attributes that you can assign to
| [`attest`](#targetattest) | List | Build attestations | | [`attest`](#targetattest) | List | Build attestations |
| [`cache-from`](#targetcache-from) | List | External cache sources | | [`cache-from`](#targetcache-from) | List | External cache sources |
| [`cache-to`](#targetcache-to) | List | External cache destinations | | [`cache-to`](#targetcache-to) | List | External cache destinations |
| [`call`](#targetcall) | String | Specify the frontend method to call for the target. |
| [`context`](#targetcontext) | String | Set of files located in the specified path or URL | | [`context`](#targetcontext) | String | Set of files located in the specified path or URL |
| [`contexts`](#targetcontexts) | Map | Additional build contexts | | [`contexts`](#targetcontexts) | Map | Additional build contexts |
| [`description`](#targetdescription) | String | Description of a target |
| [`dockerfile-inline`](#targetdockerfile-inline) | String | Inline Dockerfile string | | [`dockerfile-inline`](#targetdockerfile-inline) | String | Inline Dockerfile string |
| [`dockerfile`](#targetdockerfile) | String | Dockerfile location | | [`dockerfile`](#targetdockerfile) | String | Dockerfile location |
| [`inherits`](#targetinherits) | List | Inherit attributes from other targets | | [`inherits`](#targetinherits) | List | Inherit attributes from other targets |
@@ -285,11 +283,19 @@ The key takes a list of annotations, in the format of `KEY=VALUE`.
```hcl ```hcl
target "default" { target "default" {
output = [{ type = "image", name = "foo" }] output = ["type=image,name=foo"]
annotations = ["org.opencontainers.image.authors=dvdksn"] annotations = ["org.opencontainers.image.authors=dvdksn"]
} }
``` ```
is the same as
```hcl
target "default" {
output = ["type=image,name=foo,annotation.org.opencontainers.image.authors=dvdksn"]
}
```
By default, the annotation is added to image manifests. You can configure the By default, the annotation is added to image manifests. You can configure the
level of the annotations by adding a prefix to the annotation, containing a level of the annotations by adding a prefix to the annotation, containing a
comma-separated list of all the levels that you want to annotate. The following comma-separated list of all the levels that you want to annotate. The following
@@ -297,7 +303,7 @@ example adds annotations to both the image index and manifests.
```hcl ```hcl
target "default" { target "default" {
output = [{ type = "image", name = "foo" }] output = ["type=image,name=foo"]
annotations = ["index,manifest:org.opencontainers.image.authors=dvdksn"] annotations = ["index,manifest:org.opencontainers.image.authors=dvdksn"]
} }
``` ```
@@ -313,13 +319,8 @@ This attribute accepts the long-form CSV version of attestation parameters.
```hcl ```hcl
target "default" { target "default" {
attest = [ attest = [
{ "type=provenance,mode=min",
type = "provenance", "type=sbom"
mode = "max",
},
{
type = "sbom",
}
] ]
} }
``` ```
@@ -335,15 +336,8 @@ This takes a list value, so you can specify multiple cache sources.
```hcl ```hcl
target "app" { target "app" {
cache-from = [ cache-from = [
{ "type=s3,region=eu-west-1,bucket=mybucket",
type = "s3", "user/repo:cache",
region = "eu-west-1",
bucket = "mybucket"
},
{
type = "registry",
ref = "user/repo:cache"
}
] ]
} }
``` ```
@@ -359,40 +353,12 @@ This takes a list value, so you can specify multiple cache export targets.
```hcl ```hcl
target "app" { target "app" {
cache-to = [ cache-to = [
{ "type=s3,region=eu-west-1,bucket=mybucket",
type = "s3", "type=inline"
region = "eu-west-1",
bucket = "mybucket"
},
{
type = "inline",
}
] ]
} }
``` ```
### `target.call`
Specifies the frontend method to use. Frontend methods let you, for example,
execute build checks only, instead of running a build. This is the same as the
`--call` flag.
```hcl
target "app" {
call = "check"
}
```
Supported values are:
- `build` builds the target (default)
- `check`: evaluates [build checks](https://docs.docker.com/build/checks/) for the target
- `outline`: displays the target's build arguments and their default values if available
- `targets`: lists all Bake targets in the loaded definition, along with its [description](#targetdescription).
For more information about frontend methods, refer to the CLI reference for
[`docker buildx build --call`](https://docs.docker.com/reference/cli/docker/buildx/build/#call).
### `target.context` ### `target.context`
Specifies the location of the build context to use for this target. Specifies the location of the build context to use for this target.
@@ -500,25 +466,6 @@ FROM baseapp
RUN echo "Hello world" RUN echo "Hello world"
``` ```
### `target.description`
Defines a human-readable description for the target, clarifying its purpose or
functionality.
```hcl
target "lint" {
description = "Runs golangci-lint to detect style errors"
args = {
GOLANGCI_LINT_VERSION = null
}
dockerfile = "lint.Dockerfile"
}
```
This attribute is useful when combined with the `docker buildx bake --list=targets`
option, providing a more informative output when listing the available build
targets in a Bake file.
### `target.dockerfile-inline` ### `target.dockerfile-inline`
Uses the string value as an inline Dockerfile for the build target. Uses the string value as an inline Dockerfile for the build target.
@@ -873,7 +820,7 @@ The following example configures the target to use a cache-only output,
```hcl ```hcl
target "default" { target "default" {
output = [{ type = "cacheonly" }] output = ["type=cacheonly"]
} }
``` ```
@@ -897,7 +844,7 @@ The following example forces the builder to always pull all images referenced in
```hcl ```hcl
target "default" { target "default" {
pull = true pull = "always"
} }
``` ```
@@ -913,8 +860,8 @@ variable "HOME" {
target "default" { target "default" {
secret = [ secret = [
{ type = "env", id = "KUBECONFIG" }, "type=env,id=KUBECONFIG",
{ type = "file", id = "aws", src = "${HOME}/.aws/credentials" }, "type=file,id=aws,src=${HOME}/.aws/credentials"
] ]
} }
``` ```
@@ -958,7 +905,7 @@ This can be useful if you need to access private repositories during a build.
```hcl ```hcl
target "default" { target "default" {
ssh = [{ id = "default" }] ssh = ["default"]
} }
``` ```

View File

@@ -17,7 +17,6 @@ Extended build capabilities with BuildKit
| [`debug`](buildx_debug.md) | Start debugger (EXPERIMENTAL) | | [`debug`](buildx_debug.md) | Start debugger (EXPERIMENTAL) |
| [`dial-stdio`](buildx_dial-stdio.md) | Proxy current stdio streams to builder instance | | [`dial-stdio`](buildx_dial-stdio.md) | Proxy current stdio streams to builder instance |
| [`du`](buildx_du.md) | Disk usage | | [`du`](buildx_du.md) | Disk usage |
| [`history`](buildx_history.md) | Commands to work on build records |
| [`imagetools`](buildx_imagetools.md) | Commands to work on images in registry | | [`imagetools`](buildx_imagetools.md) | Commands to work on images in registry |
| [`inspect`](buildx_inspect.md) | Inspect current builder instance | | [`inspect`](buildx_inspect.md) | Inspect current builder instance |
| [`ls`](buildx_ls.md) | List builder instances | | [`ls`](buildx_ls.md) | List builder instances |

Some files were not shown because too many files have changed in this diff Show More