Compare commits

..

8 Commits

Author SHA1 Message Date
Tõnis Tiigi
257815a6fb Merge pull request #2698 from tonistiigi/v0.17-cli-vendor
[v0.17] update docker/cli to 48a2cdff97
2024-09-13 08:03:56 -07:00
Tõnis Tiigi
dbccfa60a7 Merge pull request #2690 from crazy-max/v0.17.1_cherry-picks
[v0.17] cherry-picks for v0.17.1
2024-09-13 08:03:24 -07:00
Tonis Tiigi
59cb959195 [v0.17] update docker/cli to 48a2cdff97
Brings in fix for telemetry under WSL2

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-09-13 07:42:49 -07:00
CrazyMax
dd0d53efd5 ci: fix golvulncheck job permissions
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 120578091f)
2024-09-12 15:25:05 +02:00
CrazyMax
e8ceaad0a8 builder: do not set network.host entitlement flag if already set in buildkitd conf
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 617d59d70b)
2024-09-11 22:51:10 +02:00
CrazyMax
e5a6b8e140 bake: fix missing omitempty and optional tags for network field
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 9fb8b04b64)
2024-09-11 14:51:26 +02:00
CrazyMax
78c8c28cf5 Merge pull request #2681 from crazy-max/v0.17.0_update-buildkit
[v0.17 backport] vendor: update buildkit to v0.16.0
2024-09-10 18:40:29 +02:00
CrazyMax
4173281da3 vendor: update buildkit to v0.16.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
(cherry picked from commit 8201d301d5)
2024-09-10 18:19:31 +02:00
3084 changed files with 160380 additions and 247757 deletions

View File

@@ -188,89 +188,6 @@ To generate new vendored files with go modules run:
$ make vendor
```
### Generate profiling data
You can configure Buildx to generate [`pprof`](https://github.com/google/pprof)
memory and CPU profiles to analyze and optimize your builds. These profiles are
useful for identifying performance bottlenecks, detecting memory
inefficiencies, and ensuring the program (Buildx) runs efficiently.
The following environment variables control whether Buildx generates profiling
data for builds:
```console
$ export BUILDX_CPU_PROFILE=buildx_cpu.prof
$ export BUILDX_MEM_PROFILE=buildx_mem.prof
```
When set, Buildx emits profiling samples for the builds to the location
specified by the environment variable.
To analyze and visualize profiling samples, you need `pprof` from the Go
toolchain, and (optionally) GraphViz for visualization in a graphical format.
To inspect profiling data with `pprof`:
1. Build a local binary of Buildx from source.
```console
$ docker buildx bake
```
The binary gets exported to `./bin/build/buildx`.
2. Run a build and with the environment variables set to generate profiling data.
```console
$ export BUILDX_CPU_PROFILE=buildx_cpu.prof
$ export BUILDX_MEM_PROFILE=buildx_mem.prof
$ ./bin/build/buildx bake
```
This creates `buildx_cpu.prof` and `buildx_mem.prof` for the build.
3. Start `pprof` and specify the filename of the profile that you want to
analyze.
```console
$ go tool pprof buildx_cpu.prof
```
This opens the `pprof` interactive console. From here, you can inspect the
profiling sample using various commands. For example, use `top 10` command
to view the top 10 most time-consuming entries.
```plaintext
(pprof) top 10
Showing nodes accounting for 3.04s, 91.02% of 3.34s total
Dropped 123 nodes (cum <= 0.02s)
Showing top 10 nodes out of 159
flat flat% sum% cum cum%
1.14s 34.13% 34.13% 1.14s 34.13% syscall.syscall
0.91s 27.25% 61.38% 0.91s 27.25% runtime.kevent
0.35s 10.48% 71.86% 0.35s 10.48% runtime.pthread_cond_wait
0.22s 6.59% 78.44% 0.22s 6.59% runtime.pthread_cond_signal
0.15s 4.49% 82.93% 0.15s 4.49% runtime.usleep
0.10s 2.99% 85.93% 0.10s 2.99% runtime.memclrNoHeapPointers
0.10s 2.99% 88.92% 0.10s 2.99% runtime.memmove
0.03s 0.9% 89.82% 0.03s 0.9% runtime.madvise
0.02s 0.6% 90.42% 0.02s 0.6% runtime.(*mspan).typePointersOfUnchecked
0.02s 0.6% 91.02% 0.02s 0.6% runtime.pcvalue
```
To view the call graph in a GUI, run `go tool pprof -http=:8081 <sample>`.
> [!NOTE]
> Requires [GraphViz](https://www.graphviz.org/) to be installed.
```console
$ go tool pprof -http=:8081 buildx_cpu.prof
Serving web UI on http://127.0.0.1:8081
http://127.0.0.1:8081
```
For more information about using `pprof` and how to interpret the call graph,
refer to the [`pprof` README](https://github.com/google/pprof/blob/main/doc/README.md).
### Conventions
@@ -426,4 +343,4 @@ The rules:
If you are having trouble getting into the mood of idiomatic Go, we recommend
reading through [Effective Go](https://golang.org/doc/effective_go.html). The
[Go Blog](https://blog.golang.org) is also a great resource.
[Go Blog](https://blog.golang.org) is also a great resource.

50
.github/SECURITY.md vendored
View File

@@ -1,44 +1,12 @@
# Security Policy
# Reporting security issues
The maintainers of Docker Buildx take security seriously. If you discover
a security issue, please bring it to their attention right away!
The project maintainers take security seriously. If you discover a security
issue, please bring it to their attention right away!
## Reporting a Vulnerability
**Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Please **DO NOT** file a public issue, instead send your report privately
to [security@docker.com](mailto:security@docker.com).
Reporter(s) can expect a response within 72 hours, acknowledging the issue was
received.
## Review Process
After receiving the report, an initial triage and technical analysis is
performed to confirm the report and determine its scope. We may request
additional information in this stage of the process.
Once a reviewer has confirmed the relevance of the report, a draft security
advisory will be created on GitHub. The draft advisory will be used to discuss
the issue with maintainers, the reporter(s), and where applicable, other
affected parties under embargo.
If the vulnerability is accepted, a timeline for developing a patch, public
disclosure, and patch release will be determined. If there is an embargo period
on public disclosure before the patch release, the reporter(s) are expected to
participate in the discussion of the timeline and abide by agreed upon dates
for public disclosure.
## Accreditation
Security reports are greatly appreciated and we will publicly thank you,
although we will keep your name confidential if you request it. We also like to
send gifts - if you're into swag, make sure to let us know. We do not currently
offer a paid security bounty program at this time.
## Supported Versions
Once a new feature release is cut, support for the previous feature release is
discontinued. An exception may be made for urgent security releases that occur
shortly after a new feature release. Buildx does not offer LTS (Long-Term Support)
releases. Refer to the [Support Policy](https://github.com/docker/buildx/blob/master/PROJECT.md#support-policy)
for further details.
Security reports are greatly appreciated, and we will publicly thank you for it.
We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
ruling it out in the future.

5
.github/labeler.yml vendored
View File

@@ -96,11 +96,6 @@ area/hack:
- changed-files:
- any-glob-to-any-file: 'hack/**'
# Add 'area/history' label to changes in history command
area/history:
- changed-files:
- any-glob-to-any-file: 'commands/history/**'
# Add 'area/tests' label to changes in test files
area/tests:
- changed-files:

View File

@@ -1,14 +1,5 @@
name: build
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -28,15 +19,15 @@ on:
- 'docs/**'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:latest"
SCOUT_VERSION: "1.11.0"
REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test"
TESTFLAGS: "-v --parallel=6 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose"
GO_VERSION: "1.23"
GO_VERSION: "1.22"
GOTESTSUM_VERSION: "v1.9.0" # same as one in Dockerfile
jobs:
@@ -54,9 +45,9 @@ jobs:
- master
- latest
- buildx-stable-1
- v0.19.0-rc2
- v0.18.2
- v0.17.2
- v0.14.1
- v0.13.2
- v0.12.5
worker:
- docker-container
- remote
@@ -76,16 +67,6 @@ jobs:
- worker: docker+containerd # same as docker, but with containerd snapshotter
pkg: ./tests
mode: experimental
- worker: "docker@26.1"
pkg: ./tests
- worker: "docker+containerd@26.1" # same as docker, but with containerd snapshotter
pkg: ./tests
- worker: "docker@26.1"
pkg: ./tests
mode: experimental
- worker: "docker+containerd@26.1" # same as docker, but with containerd snapshotter
pkg: ./tests
mode: experimental
steps:
-
name: Prepare
@@ -96,7 +77,7 @@ jobs:
fi
testFlags="--run=//worker=$(echo "${{ matrix.worker }}" | sed 's/\+/\\+/g')$"
case "${{ matrix.worker }}" in
docker | docker+containerd | docker@* | docker+containerd@*)
docker | docker+containerd)
echo "TESTFLAGS=${{ env.TESTFLAGS_DOCKER }} $testFlags" >> $GITHUB_ENV
;;
*)
@@ -121,14 +102,13 @@ jobs:
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build test image
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
source: .
targets: integration-test
set: |
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
@@ -142,7 +122,7 @@ jobs:
-
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v4
with:
directory: ./bin/testreports
flags: integration
@@ -169,7 +149,7 @@ jobs:
matrix:
os:
- ubuntu-24.04
- macos-14
- macos-12
- windows-2022
env:
SKIP_INTEGRATION_TESTS: 1
@@ -214,7 +194,7 @@ jobs:
-
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v4
with:
directory: ${{ env.TESTREPORTS_DIR }}
env_vars: RUNNER_OS
@@ -235,81 +215,27 @@ jobs:
name: test-reports-${{ env.TESTREPORTS_NAME }}
path: ${{ env.TESTREPORTS_BASEDIR }}
test-bsd-unit:
runs-on: ubuntu-22.04
continue-on-error: true
strategy:
fail-fast: false
matrix:
os:
- freebsd
- openbsd
govulncheck:
runs-on: ubuntu-24.04
permissions:
# required to write sarif report
security-events: write
# required to check out the repository
contents: read
steps:
-
name: Prepare
run: |
echo "VAGRANT_FILE=hack/Vagrantfile.${{ matrix.os }}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v4
-
name: Cache Vagrant boxes
uses: actions/cache@v4
with:
path: ~/.vagrant.d/boxes
key: ${{ runner.os }}-vagrant-${{ matrix.os }}-${{ hashFiles(env.VAGRANT_FILE) }}
restore-keys: |
${{ runner.os }}-vagrant-${{ matrix.os }}-
-
name: Install vagrant
run: |
set -x
sudo apt-get update
sudo apt-get install -y libvirt-daemon libvirt-daemon-system vagrant vagrant-libvirt ruby-libvirt
sudo systemctl enable --now libvirtd
sudo chmod a+rw /var/run/libvirt/libvirt-sock
vagrant plugin install vagrant-libvirt
vagrant --version
-
name: Set up vagrant
run: |
ln -sf ${{ env.VAGRANT_FILE }} Vagrantfile
vagrant up --no-tty
-
name: Test
run: |
vagrant ssh -- "cd /vagrant; SKIP_INTEGRATION_TESTS=1 go test -mod=vendor -coverprofile=coverage.txt -covermode=atomic ${{ env.TESTFLAGS }} ./..."
vagrant ssh -c "sudo cat /vagrant/coverage.txt" > coverage.txt
-
name: Upload coverage
if: always()
uses: codecov/codecov-action@v5
with:
files: ./coverage.txt
env_vars: RUNNER_OS
flags: unit,${{ matrix.os }}
token: ${{ secrets.CODECOV_TOKEN }}
env:
RUNNER_OS: ${{ matrix.os }}
govulncheck:
runs-on: ubuntu-24.04
permissions:
# same as global permission
contents: read
# required to write sarif report
security-events: write
steps:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Run
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
targets: govulncheck
env:
@@ -363,8 +289,8 @@ jobs:
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
@@ -389,6 +315,9 @@ jobs:
- test-unit
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
@@ -396,8 +325,8 @@ jobs:
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Docker meta
@@ -420,11 +349,11 @@ jobs:
password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }}
-
name: Build and push image
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
files: |
./docker-bake.hcl
cwd://${{ steps.meta.outputs.bake-file }}
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
@@ -436,13 +365,14 @@ jobs:
runs-on: ubuntu-24.04
if: ${{ github.ref == 'refs/heads/master' && github.repository == 'docker/buildx' }}
permissions:
# same as global permission
contents: read
# required to write sarif report
security-events: write
needs:
- bin-image
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Login to DockerHub
uses: docker/login-action@v3
@@ -465,9 +395,6 @@ jobs:
release:
runs-on: ubuntu-24.04
permissions:
# required to create GitHub release
contents: write
needs:
- test-integration
- test-unit
@@ -497,7 +424,7 @@ jobs:
-
name: GitHub Release
if: startsWith(github.ref, 'refs/tags/v')
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2.2.1
uses: softprops/action-gh-release@c062e08bd532815e2082a85e87e3ef29c3e6d191 # v2.0.8
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:

View File

@@ -1,14 +1,5 @@
name: codeql
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
push:
branches:
@@ -16,16 +7,17 @@ on:
- 'v[0-9]*'
pull_request:
permissions:
actions: read
contents: read
security-events: write
env:
GO_VERSION: "1.23"
GO_VERSION: "1.22"
jobs:
codeql:
runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
security-events: write
steps:
-
name: Checkout

View File

@@ -1,14 +1,5 @@
name: docs-release
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
on:
workflow_dispatch:
inputs:
@@ -19,17 +10,10 @@ on:
types:
- released
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs:
open-pr:
runs-on: ubuntu-24.04
if: ${{ (github.event.release.prerelease != true || github.event.inputs.tag != '') && github.repository == 'docker/buildx' }}
permissions:
contents: write
pull-requests: write
steps:
-
name: Checkout docs repo
@@ -50,13 +34,9 @@ jobs:
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Generate yaml
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ env.RELEASE_NAME }}
targets: update-docs
@@ -77,7 +57,7 @@ jobs:
VENDOR_MODULE: github.com/docker/buildx@${{ env.RELEASE_NAME }}
-
name: Create PR on docs repo
uses: peter-evans/create-pull-request@67ccf781d68cd99b580ae25a5c18a1cc84ffff1f # v7.0.6
uses: peter-evans/create-pull-request@8867c4aba1b742c39f8d0ba35429c2dfa4b6cb20 # v7.0.1
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io

View File

@@ -3,15 +3,6 @@
# https://github.com/docker/docker.github.io/blob/98c7c9535063ae4cd2cd0a31478a21d16d2f07a3/docker-bake.hcl#L34-L36
name: docs-upstream
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -29,24 +20,21 @@ on:
- '.github/workflows/docs-upstream.yml'
- 'docs/**'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs:
docs-yaml:
runs-on: ubuntu-24.04
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
version: latest
-
name: Build reference YAML docs
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
targets: update-docs
provenance: false
@@ -65,7 +53,7 @@ jobs:
retention-days: 1
validate:
uses: docker/docs/.github/workflows/validate-upstream.yml@main
uses: docker/docs/.github/workflows/validate-upstream.yml@6b73b05acb21edf7995cc5b3c6672d8e314cee7a # pin for artifact v4 support: https://github.com/docker/docs/pull/19220
needs:
- docs-yaml
with:

View File

@@ -1,14 +1,5 @@
name: e2e
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -26,8 +17,6 @@ on:
- 'docs/**'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
DESTDIR: "./bin"
K3S_VERSION: "v1.21.2-k3s1"
@@ -35,16 +24,16 @@ jobs:
build:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
version: latest
-
name: Build
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
targets: binaries
set: |
@@ -177,75 +166,3 @@ jobs:
DRIVER_OPT: ${{ matrix.driver-opt }}
ENDPOINT: ${{ matrix.endpoint }}
PLATFORMS: ${{ matrix.platforms }}
bake:
runs-on: ubuntu-24.04
needs:
- build
env:
DOCKER_BUILD_CHECKS_ANNOTATIONS: false
DOCKER_BUILD_SUMMARY: false
strategy:
fail-fast: false
matrix:
include:
-
# https://github.com/docker/bake-action/blob/v5.11.0/.github/workflows/ci.yml#L227-L237
source: "https://github.com/docker/bake-action.git#v5.11.0:test/go"
overrides: |
*.output=/tmp/bake-build
-
# https://github.com/tonistiigi/xx/blob/2fc85604e7280bfb3f626569bd4c5413c43eb4af/.github/workflows/ld.yml#L90-L98
source: "https://github.com/tonistiigi/xx.git#2fc85604e7280bfb3f626569bd4c5413c43eb4af"
targets: |
ld64-static-tgz
overrides: |
ld64-static-tgz.output=type=local,dest=./dist
ld64-static-tgz.platform=linux/amd64
ld64-static-tgz.cache-from=type=gha,scope=xx-ld64-static-tgz
ld64-static-tgz.cache-to=type=gha,scope=xx-ld64-static-tgz
-
# https://github.com/moby/buildkit-bench/blob/54c194011c4fc99a94aa75d4b3d4f3ffd4c4ce27/docker-bake.hcl#L154-L160
source: "https://github.com/moby/buildkit-bench.git#54c194011c4fc99a94aa75d4b3d4f3ffd4c4ce27"
targets: |
tests-buildkit
envs: |
BUILDKIT_REFS=v0.18.2
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Environment variables
if: matrix.envs != ''
run: |
for l in "${{ matrix.envs }}"; do
echo "${l?}" >> $GITHUB_ENV
done
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Install buildx
uses: actions/download-artifact@v4
with:
name: binary
path: /home/runner/.docker/cli-plugins
-
name: Fix perms and check
run: |
chmod +x /home/runner/.docker/cli-plugins/docker-buildx
docker buildx version
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v6
with:
source: ${{ matrix.source }}
targets: ${{ matrix.targets }}
set: ${{ matrix.overrides }}

View File

@@ -1,14 +1,5 @@
name: labeler
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -18,12 +9,10 @@ on:
jobs:
labeler:
runs-on: ubuntu-latest
permissions:
# same as global permission
contents: read
# required for writing labels
pull-requests: write
runs-on: ubuntu-latest
steps:
-
name: Run

View File

@@ -1,14 +1,5 @@
name: validate
# Default to 'contents: read', which grants actions to read commits.
#
# If any permission is set, any permission not included in the list is
# implicitly set to "none".
#
# see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
@@ -25,10 +16,6 @@ on:
paths-ignore:
- '.github/releases.json'
env:
SETUP_BUILDX_VERSION: "edge"
SETUP_BUILDKIT_IMAGE: "moby/buildkit:latest"
jobs:
prepare:
runs-on: ubuntu-24.04
@@ -94,16 +81,17 @@ jobs:
if [ "$GITHUB_REPOSITORY" = "docker/buildx" ]; then
echo "GOLANGCI_LINT_MULTIPLATFORM=1" >> $GITHUB_ENV
fi
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.SETUP_BUILDX_VERSION }}
driver-opts: image=${{ env.SETUP_BUILDKIT_IMAGE }}
buildkitd-flags: --debug
version: latest
-
name: Validate
uses: docker/bake-action@v6
uses: docker/bake-action@v5
with:
targets: ${{ matrix.target }}
set: |

View File

@@ -1,52 +1,26 @@
run:
timeout: 30m
modules-download-mode: vendor
# default uses Go version from the go.mod file, fallback on the env var
# `GOVERSION`, fallback on 1.17: https://golangci-lint.run/usage/configuration/#run-configuration
go: "1.23"
linters:
enable:
- bodyclose
- depguard
- forbidigo
- gocritic
- gofmt
- goimports
- gosec
- gosimple
- govet
- depguard
- goimports
- ineffassign
- makezero
- misspell
- noctx
- nolintlint
- unused
- revive
- staticcheck
- testifylint
- typecheck
- unused
- whitespace
- nolintlint
- gosec
- forbidigo
disable-all: true
linters-settings:
gocritic:
disabled-checks:
- "ifElseChain"
- "assignOp"
- "appendAssign"
- "singleCaseSwitch"
- "exitAfterDefer" # FIXME
importas:
alias:
# Enforce alias to prevent it accidentally being used instead of
# buildkit errdefs package (or vice-versa).
- pkg: "github.com/containerd/errdefs"
alias: "cerrdefs"
- pkg: "github.com/opencontainers/image-spec/specs-go/v1"
alias: "ocispecs"
- pkg: "github.com/opencontainers/go-digest"
alias: "digest"
govet:
enable:
- nilness
@@ -69,27 +43,14 @@ linters-settings:
desc: The io/ioutil package has been deprecated.
forbidigo:
forbid:
- '^context\.WithCancel(# use context\.WithCancelCause instead)?$'
- '^context\.WithDeadline(# use context\.WithDeadline instead)?$'
- '^context\.WithTimeout(# use context\.WithTimeoutCause instead)?$'
- '^ctx\.Err(# use context\.Cause instead)?$'
- '^fmt\.Errorf(# use errors\.Errorf instead)?$'
- '^platforms\.DefaultString(# use platforms\.Format(platforms\.DefaultSpec()) instead\.)?$'
gosec:
excludes:
- G204 # Audit use of command execution
- G402 # TLS MinVersion too low
- G115 # integer overflow conversion (TODO: verify these)
config:
G306: "0644"
testifylint:
disable:
# disable rules that reduce the test condition
- "empty"
- "bool-compare"
- "len"
- "negative-positive"
issues:
exclude-files:

View File

@@ -1,24 +1,20 @@
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.23
ARG ALPINE_VERSION=3.21
ARG XX_VERSION=1.6.1
ARG GO_VERSION=1.22
ARG XX_VERSION=1.4.0
# for testing
ARG DOCKER_VERSION=27.5.0
ARG DOCKER_VERSION_ALT_26=26.1.3
ARG DOCKER_VERSION=27.1.1
ARG DOCKER_CLI_VERSION=${DOCKER_VERSION}
ARG GOTESTSUM_VERSION=v1.12.0
ARG REGISTRY_VERSION=2.8.3
ARG BUILDKIT_VERSION=v0.19.0-rc2
ARG UNDOCK_VERSION=0.9.0
ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.14.1
ARG UNDOCK_VERSION=0.7.0
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine${ALPINE_VERSION} AS golatest
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS golatest
FROM moby/moby-bin:$DOCKER_VERSION AS docker-engine
FROM dockereng/cli-bin:$DOCKER_CLI_VERSION AS docker-cli
FROM moby/moby-bin:$DOCKER_VERSION_ALT_26 AS docker-engine-alt
FROM dockereng/cli-bin:$DOCKER_VERSION_ALT_26 AS docker-cli-alt
FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM crazymax/undock:$UNDOCK_VERSION AS undock
@@ -81,7 +77,6 @@ RUN --mount=type=bind,target=. \
set -e
xx-go --wrap
DESTDIR=/usr/bin VERSION=$(cat /buildx-version/version) REVISION=$(cat /buildx-version/revision) GO_EXTRA_LDFLAGS="-s -w" ./hack/build
file /usr/bin/docker-buildx
xx-verify --static /usr/bin/docker-buildx
EOT
@@ -100,9 +95,7 @@ FROM scratch AS binaries-unix
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx
FROM binaries-unix AS binaries-darwin
FROM binaries-unix AS binaries-freebsd
FROM binaries-unix AS binaries-linux
FROM binaries-unix AS binaries-openbsd
FROM scratch AS binaries-windows
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx.exe
@@ -127,19 +120,16 @@ COPY --link --from=gotestsum /out /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=docker-engine / /usr/bin/
COPY --link --from=docker-cli / /usr/bin/
COPY --link --from=docker-engine-alt / /opt/docker-alt-26/
COPY --link --from=docker-cli-alt / /opt/docker-alt-26/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=undock /usr/local/bin/undock /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/
ENV TEST_DOCKER_EXTRA="docker@26.1=/opt/docker-alt-26"
FROM integration-test-base AS integration-test
COPY . .
# Release
FROM --platform=$BUILDPLATFORM alpine:${ALPINE_VERSION} AS releaser
FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work
ARG TARGETPLATFORM
RUN --mount=from=binaries \
@@ -154,7 +144,7 @@ COPY --from=releaser /out/ /
# Shell
FROM docker:$DOCKER_VERSION AS dockerd-release
FROM alpine:${ALPINE_VERSION} AS shell
FROM alpine AS shell
RUN apk add --no-cache iptables tmux git vim less openssh
RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx
COPY ./hack/demo-env/entrypoint.sh /usr/local/bin

View File

@@ -2,13 +2,11 @@ package bake
import (
"context"
"encoding"
"io"
"os"
"path"
"path/filepath"
"regexp"
"slices"
"sort"
"strconv"
"strings"
@@ -29,6 +27,7 @@ import (
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/entitlements"
"github.com/pkg/errors"
"github.com/tonistiigi/go-csvvalue"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
)
@@ -53,8 +52,8 @@ func defaultFilenames() []string {
names = append(names, composecli.DefaultFileNames...)
names = append(names, []string{
"docker-bake.json",
"docker-bake.hcl",
"docker-bake.override.json",
"docker-bake.hcl",
"docker-bake.override.hcl",
}...)
return names
@@ -193,7 +192,7 @@ func ListTargets(files []File) ([]string, error) {
return dedupSlice(targets), nil
}
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string, ent *EntitlementConf) (map[string]*Target, map[string]*Group, error) {
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, map[string]*Group, error) {
c, _, err := ParseFiles(files, defaults)
if err != nil {
return nil, nil, err
@@ -207,24 +206,23 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
if err != nil {
return nil, nil, err
}
targetsMap := map[string]*Target{}
groupsMap := map[string]*Group{}
m := map[string]*Target{}
n := map[string]*Group{}
for _, target := range targets {
ts, gs := c.ResolveGroup(target)
for _, tname := range ts {
t, err := c.ResolveTarget(tname, o, ent)
t, err := c.ResolveTarget(tname, o)
if err != nil {
return nil, nil, err
}
if t != nil {
targetsMap[tname] = t
m[tname] = t
}
}
for _, gname := range gs {
for _, group := range c.Groups {
if group.Name == gname {
groupsMap[gname] = group
n[gname] = group
break
}
}
@@ -232,26 +230,25 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
}
for _, target := range targets {
if _, ok := groupsMap["default"]; ok && target == "default" {
if target == "default" {
continue
}
if _, ok := groupsMap["default"]; !ok {
groupsMap["default"] = &Group{Name: "default"}
if _, ok := n["default"]; !ok {
n["default"] = &Group{Name: "default"}
}
groupsMap["default"].Targets = append(groupsMap["default"].Targets, target)
n["default"].Targets = append(n["default"].Targets, target)
}
if g, ok := groupsMap["default"]; ok {
if g, ok := n["default"]; ok {
g.Targets = dedupSlice(g.Targets)
sort.Strings(g.Targets)
}
for name, t := range targetsMap {
if err := c.loadLinks(name, t, targetsMap, o, nil, ent); err != nil {
for name, t := range m {
if err := c.loadLinks(name, t, m, o, nil); err != nil {
return nil, nil, err
}
}
return targetsMap, groupsMap, nil
return m, n, nil
}
func dedupSlice(s []string) []string {
@@ -478,12 +475,12 @@ func (c Config) expandTargets(pattern string) ([]string, error) {
return names, nil
}
func (c Config) loadLinks(name string, t *Target, m map[string]*Target, o map[string]map[string]Override, visited []string, ent *EntitlementConf) error {
func (c Config) loadLinks(name string, t *Target, m map[string]*Target, o map[string]map[string]Override, visited []string) error {
visited = append(visited, name)
for _, v := range t.Contexts {
if strings.HasPrefix(v, "target:") {
target := strings.TrimPrefix(v, "target:")
if target == name {
if target == t.Name {
return errors.Errorf("target %s cannot link to itself", target)
}
for _, v := range visited {
@@ -494,30 +491,20 @@ func (c Config) loadLinks(name string, t *Target, m map[string]*Target, o map[st
t2, ok := m[target]
if !ok {
var err error
t2, err = c.ResolveTarget(target, o, ent)
t2, err = c.ResolveTarget(target, o)
if err != nil {
return err
}
t2.Outputs = []*buildflags.ExportEntry{
{Type: "cacheonly"},
}
t2.Outputs = []string{"type=cacheonly"}
t2.linked = true
m[target] = t2
}
if err := c.loadLinks(target, t2, m, o, visited, ent); err != nil {
if err := c.loadLinks(target, t2, m, o, visited); err != nil {
return err
}
// entitlements are inherited from linked targets
for _, ent := range t2.Entitlements {
if !slices.Contains(t.Entitlements, ent) {
t.Entitlements = append(t.Entitlements, ent)
}
}
if len(t.Platforms) > 1 && len(t2.Platforms) > 1 {
if !isSubset(t.Platforms, t2.Platforms) {
return errors.Errorf("target %s can't be used by %s because its platforms %v are not a subset of %v", target, name, t.Platforms, t2.Platforms)
if !sliceEqual(t.Platforms, t2.Platforms) {
return errors.Errorf("target %s can't be used by %s because it is defined for different platforms %v and %v", target, name, t2.Platforms, t.Platforms)
}
}
}
@@ -555,8 +542,6 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
o := t[kk[1]]
// IMPORTANT: if you add more fields here, do not forget to update
// docs/bake-reference.md and https://docs.docker.com/build/bake/overrides/
switch keys[1] {
case "output", "cache-to", "cache-from", "tags", "platform", "secrets", "ssh", "attest", "entitlements", "network":
if len(parts) == 2 {
@@ -633,8 +618,8 @@ func (c Config) group(name string, visited map[string]visit) ([]string, []string
return targets, groups
}
func (c Config) ResolveTarget(name string, overrides map[string]map[string]Override, ent *EntitlementConf) (*Target, error) {
t, err := c.target(name, map[string]*Target{}, overrides, ent)
func (c Config) ResolveTarget(name string, overrides map[string]map[string]Override) (*Target, error) {
t, err := c.target(name, map[string]*Target{}, overrides)
if err != nil {
return nil, err
}
@@ -650,7 +635,7 @@ func (c Config) ResolveTarget(name string, overrides map[string]map[string]Overr
return t, nil
}
func (c Config) target(name string, visited map[string]*Target, overrides map[string]map[string]Override, ent *EntitlementConf) (*Target, error) {
func (c Config) target(name string, visited map[string]*Target, overrides map[string]map[string]Override) (*Target, error) {
if t, ok := visited[name]; ok {
return t, nil
}
@@ -667,7 +652,7 @@ func (c Config) target(name string, visited map[string]*Target, overrides map[st
}
tt := &Target{}
for _, name := range t.Inherits {
t, err := c.target(name, visited, overrides, ent)
t, err := c.target(name, visited, overrides)
if err != nil {
return nil, err
}
@@ -679,7 +664,7 @@ func (c Config) target(name string, visited map[string]*Target, overrides map[st
m.Merge(tt)
m.Merge(t)
tt = m
if err := tt.AddOverrides(overrides[name], ent); err != nil {
if err := tt.AddOverrides(overrides[name]); err != nil {
return nil, err
}
tt.normalize()
@@ -701,61 +686,59 @@ type Target struct {
// Inherits is the only field that cannot be overridden with --set
Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional" cty:"inherits"`
Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"`
Attest buildflags.Attests `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"`
DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional" cty:"dockerfile-inline"`
Args map[string]*string `json:"args,omitempty" hcl:"args,optional" cty:"args"`
Labels map[string]*string `json:"labels,omitempty" hcl:"labels,optional" cty:"labels"`
Tags []string `json:"tags,omitempty" hcl:"tags,optional" cty:"tags"`
CacheFrom buildflags.CacheOptions `json:"cache-from,omitempty" hcl:"cache-from,optional" cty:"cache-from"`
CacheTo buildflags.CacheOptions `json:"cache-to,omitempty" hcl:"cache-to,optional" cty:"cache-to"`
Target *string `json:"target,omitempty" hcl:"target,optional" cty:"target"`
Secrets buildflags.Secrets `json:"secret,omitempty" hcl:"secret,optional" cty:"secret"`
SSH buildflags.SSHKeys `json:"ssh,omitempty" hcl:"ssh,optional" cty:"ssh"`
Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional" cty:"platforms"`
Outputs buildflags.Exports `json:"output,omitempty" hcl:"output,optional" cty:"output"`
Pull *bool `json:"pull,omitempty" hcl:"pull,optional" cty:"pull"`
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"network,omitempty" hcl:"network,optional" cty:"network"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
ShmSize *string `json:"shm-size,omitempty" hcl:"shm-size,optional" cty:"shm-size"`
Ulimits []string `json:"ulimits,omitempty" hcl:"ulimits,optional" cty:"ulimits"`
Call *string `json:"call,omitempty" hcl:"call,optional" cty:"call"`
Entitlements []string `json:"entitlements,omitempty" hcl:"entitlements,optional" cty:"entitlements"`
Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"`
Attest []string `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"`
DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional" cty:"dockerfile-inline"`
Args map[string]*string `json:"args,omitempty" hcl:"args,optional" cty:"args"`
Labels map[string]*string `json:"labels,omitempty" hcl:"labels,optional" cty:"labels"`
Tags []string `json:"tags,omitempty" hcl:"tags,optional" cty:"tags"`
CacheFrom []string `json:"cache-from,omitempty" hcl:"cache-from,optional" cty:"cache-from"`
CacheTo []string `json:"cache-to,omitempty" hcl:"cache-to,optional" cty:"cache-to"`
Target *string `json:"target,omitempty" hcl:"target,optional" cty:"target"`
Secrets []string `json:"secret,omitempty" hcl:"secret,optional" cty:"secret"`
SSH []string `json:"ssh,omitempty" hcl:"ssh,optional" cty:"ssh"`
Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional" cty:"platforms"`
Outputs []string `json:"output,omitempty" hcl:"output,optional" cty:"output"`
Pull *bool `json:"pull,omitempty" hcl:"pull,optional" cty:"pull"`
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"network,omitempty" hcl:"network,optional" cty:"network"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
ShmSize *string `json:"shm-size,omitempty" hcl:"shm-size,optional"`
Ulimits []string `json:"ulimits,omitempty" hcl:"ulimits,optional"`
Call *string `json:"call,omitempty" hcl:"call,optional" cty:"call"`
Entitlements []string `json:"entitlements,omitempty" hcl:"entitlements,optional" cty:"entitlements"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides/AddOverrides and docs/bake-reference.md.
// linked is a private field to mark a target used as a linked one
linked bool
}
var (
_ hclparser.WithEvalContexts = &Target{}
_ hclparser.WithGetName = &Target{}
_ hclparser.WithEvalContexts = &Group{}
_ hclparser.WithGetName = &Group{}
)
var _ hclparser.WithEvalContexts = &Target{}
var _ hclparser.WithGetName = &Target{}
var _ hclparser.WithEvalContexts = &Group{}
var _ hclparser.WithGetName = &Group{}
func (t *Target) normalize() {
t.Annotations = removeDupesStr(t.Annotations)
t.Attest = t.Attest.Normalize()
t.Tags = removeDupesStr(t.Tags)
t.Secrets = t.Secrets.Normalize()
t.SSH = t.SSH.Normalize()
t.Platforms = removeDupesStr(t.Platforms)
t.CacheFrom = t.CacheFrom.Normalize()
t.CacheTo = t.CacheTo.Normalize()
t.Outputs = t.Outputs.Normalize()
t.NoCacheFilter = removeDupesStr(t.NoCacheFilter)
t.Ulimits = removeDupesStr(t.Ulimits)
t.Annotations = removeDupes(t.Annotations)
t.Attest = removeAttestDupes(t.Attest)
t.Tags = removeDupes(t.Tags)
t.Secrets = removeDupes(t.Secrets)
t.SSH = removeDupes(t.SSH)
t.Platforms = removeDupes(t.Platforms)
t.CacheFrom = removeDupes(t.CacheFrom)
t.CacheTo = removeDupes(t.CacheTo)
t.Outputs = removeDupes(t.Outputs)
t.NoCacheFilter = removeDupes(t.NoCacheFilter)
t.Ulimits = removeDupes(t.Ulimits)
if t.NetworkMode != nil && *t.NetworkMode == "host" {
t.Entitlements = append(t.Entitlements, "network.host")
}
t.Entitlements = removeDupesStr(t.Entitlements)
t.Entitlements = removeDupes(t.Entitlements)
for k, v := range t.Contexts {
if v == "" {
@@ -814,19 +797,20 @@ func (t *Target) Merge(t2 *Target) {
t.Annotations = append(t.Annotations, t2.Annotations...)
}
if t2.Attest != nil { // merge
t.Attest = t.Attest.Merge(t2.Attest)
t.Attest = append(t.Attest, t2.Attest...)
t.Attest = removeAttestDupes(t.Attest)
}
if t2.Secrets != nil { // merge
t.Secrets = t.Secrets.Merge(t2.Secrets)
t.Secrets = append(t.Secrets, t2.Secrets...)
}
if t2.SSH != nil { // merge
t.SSH = t.SSH.Merge(t2.SSH)
t.SSH = append(t.SSH, t2.SSH...)
}
if t2.Platforms != nil { // no merge
t.Platforms = t2.Platforms
}
if t2.CacheFrom != nil { // merge
t.CacheFrom = t.CacheFrom.Merge(t2.CacheFrom)
t.CacheFrom = append(t.CacheFrom, t2.CacheFrom...)
}
if t2.CacheTo != nil { // no merge
t.CacheTo = t2.CacheTo
@@ -861,9 +845,7 @@ func (t *Target) Merge(t2 *Target) {
t.Inherits = append(t.Inherits, t2.Inherits...)
}
func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementConf) error {
// IMPORTANT: if you add more fields here, do not forget to update
// docs/bake-reference.md and https://docs.docker.com/build/bake/overrides/
func (t *Target) AddOverrides(overrides map[string]Override) error {
for key, o := range overrides {
value := o.Value
keys := strings.SplitN(key, ".", 2)
@@ -874,7 +856,7 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Dockerfile = &value
case "args":
if len(keys) != 2 {
return errors.Errorf("invalid format for args, expecting args.<name>=<value>")
return errors.Errorf("args require name")
}
if t.Args == nil {
t.Args = map[string]*string{}
@@ -882,7 +864,7 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Args[keys[1]] = &value
case "contexts":
if len(keys) != 2 {
return errors.Errorf("invalid format for contexts, expecting contexts.<name>=<value>")
return errors.Errorf("contexts require name")
}
if t.Contexts == nil {
t.Contexts = map[string]string{}
@@ -890,7 +872,7 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
t.Contexts[keys[1]] = value
case "labels":
if len(keys) != 2 {
return errors.Errorf("invalid format for labels, expecting labels.<name>=<value>")
return errors.Errorf("labels require name")
}
if t.Labels == nil {
t.Labels = map[string]*string{}
@@ -899,85 +881,27 @@ func (t *Target) AddOverrides(overrides map[string]Override, ent *EntitlementCon
case "tags":
t.Tags = o.ArrValue
case "cache-from":
cacheFrom, err := buildflags.ParseCacheEntry(o.ArrValue)
if err != nil {
return err
}
t.CacheFrom = cacheFrom
for _, c := range t.CacheFrom {
if c.Type == "local" {
if v, ok := c.Attrs["src"]; ok {
ent.FSRead = append(ent.FSRead, v)
}
}
}
t.CacheFrom = o.ArrValue
case "cache-to":
cacheTo, err := buildflags.ParseCacheEntry(o.ArrValue)
if err != nil {
return err
}
t.CacheTo = cacheTo
for _, c := range t.CacheTo {
if c.Type == "local" {
if v, ok := c.Attrs["dest"]; ok {
ent.FSWrite = append(ent.FSWrite, v)
}
}
}
t.CacheTo = o.ArrValue
case "target":
t.Target = &value
case "call":
t.Call = &value
case "secrets":
secrets, err := parseArrValue[buildflags.Secret](o.ArrValue)
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
t.Secrets = secrets
for _, s := range t.Secrets {
if s.FilePath != "" {
ent.FSRead = append(ent.FSRead, s.FilePath)
}
}
t.Secrets = o.ArrValue
case "ssh":
ssh, err := parseArrValue[buildflags.SSH](o.ArrValue)
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
t.SSH = ssh
for _, s := range t.SSH {
ent.FSRead = append(ent.FSRead, s.Paths...)
}
t.SSH = o.ArrValue
case "platform":
t.Platforms = o.ArrValue
case "output":
outputs, err := parseArrValue[buildflags.ExportEntry](o.ArrValue)
if err != nil {
return errors.Wrap(err, "invalid value for outputs")
}
t.Outputs = outputs
for _, o := range t.Outputs {
if o.Destination != "" {
ent.FSWrite = append(ent.FSWrite, o.Destination)
}
}
t.Outputs = o.ArrValue
case "entitlements":
t.Entitlements = append(t.Entitlements, o.ArrValue...)
for _, v := range o.ArrValue {
if v == string(EntitlementKeyNetworkHost) {
ent.NetworkHost = true
} else if v == string(EntitlementKeySecurityInsecure) {
ent.SecurityInsecure = true
}
}
case "annotations":
t.Annotations = append(t.Annotations, o.ArrValue...)
case "attest":
attest, err := parseArrValue[buildflags.Attest](o.ArrValue)
if err != nil {
return errors.Wrap(err, "invalid value for attest")
}
t.Attest = t.Attest.Merge(attest)
t.Attest = append(t.Attest, o.ArrValue...)
case "no-cache":
noCache, err := strconv.ParseBool(value)
if err != nil {
@@ -1182,44 +1106,62 @@ func updateContext(t *build.Inputs, inp *Input) {
t.ContextState = &st
}
func isRemoteContext(t build.Inputs, inp *Input) bool {
if build.IsRemoteURL(t.ContextPath) {
return true
// validateContextsEntitlements is a basic check to ensure contexts do not
// escape local directories when loaded from remote sources. This is to be
// replaced with proper entitlements support in the future.
func validateContextsEntitlements(t build.Inputs, inp *Input) error {
if inp == nil || inp.State == nil {
return nil
}
if inp != nil && build.IsRemoteURL(inp.URL) && !strings.HasPrefix(t.ContextPath, "cwd://") {
return true
if v, ok := os.LookupEnv("BAKE_ALLOW_REMOTE_FS_ACCESS"); ok {
if vv, _ := strconv.ParseBool(v); vv {
return nil
}
}
return false
}
func collectLocalPaths(t build.Inputs) []string {
var out []string
if t.ContextState == nil {
if v, ok := isLocalPath(t.ContextPath); ok {
out = append(out, v)
if err := checkPath(t.ContextPath); err != nil {
return err
}
if v, ok := isLocalPath(t.DockerfilePath); ok {
out = append(out, v)
}
} else if strings.HasPrefix(t.ContextPath, "cwd://") {
out = append(out, strings.TrimPrefix(t.ContextPath, "cwd://"))
}
for _, v := range t.NamedContexts {
if v.State != nil {
continue
}
if v, ok := isLocalPath(v.Path); ok {
out = append(out, v)
if err := checkPath(v.Path); err != nil {
return err
}
}
return out
return nil
}
func isLocalPath(p string) (string, bool) {
func checkPath(p string) error {
if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
return "", false
return nil
}
return strings.TrimPrefix(p, "cwd://"), true
p, err := filepath.EvalSymlinks(p)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
p, err = filepath.Abs(p)
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return err
}
rel, err := filepath.Rel(wd, p)
if err != nil {
return err
}
parts := strings.Split(rel, string(os.PathSeparator))
if parts[0] == ".." {
return errors.Errorf("path %s is outside of the working directory, please set BAKE_ALLOW_REMOTE_FS_ACCESS=1", p)
}
return nil
}
func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
@@ -1259,6 +1201,9 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
// it's not outside the working directory and then resolve it to an
// absolute path.
bi.DockerfilePath = path.Clean(strings.TrimPrefix(bi.DockerfilePath, "cwd://"))
if err := checkPath(bi.DockerfilePath); err != nil {
return nil, err
}
var err error
bi.DockerfilePath, err = filepath.Abs(bi.DockerfilePath)
if err != nil {
@@ -1295,6 +1240,10 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
args := map[string]string{}
@@ -1351,35 +1300,24 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
}
bo.Platforms = platforms
secrets := t.Secrets
if isRemoteContext(bi, inp) {
if _, ok := os.LookupEnv("BUILDX_BAKE_GIT_AUTH_TOKEN"); ok {
secrets = append(secrets, &buildflags.Secret{
ID: llb.GitAuthTokenKey,
Env: "BUILDX_BAKE_GIT_AUTH_TOKEN",
})
}
if _, ok := os.LookupEnv("BUILDX_BAKE_GIT_AUTH_HEADER"); ok {
secrets = append(secrets, &buildflags.Secret{
ID: llb.GitAuthHeaderKey,
Env: "BUILDX_BAKE_GIT_AUTH_HEADER",
})
}
secrets, err := buildflags.ParseSecretSpecs(t.Secrets)
if err != nil {
return nil, err
}
secrets = secrets.Normalize()
bo.SecretSpecs = secrets.ToPB()
secretAttachment, err := controllerapi.CreateSecrets(bo.SecretSpecs)
secretAttachment, err := controllerapi.CreateSecrets(secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, secretAttachment)
bo.SSHSpecs = t.SSH.ToPB()
if len(bo.SSHSpecs) == 0 && buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL)) {
bo.SSHSpecs = []*controllerapi.SSH{{ID: "default"}}
sshSpecs, err := buildflags.ParseSSHSpecs(t.SSH)
if err != nil {
return nil, err
}
sshAttachment, err := controllerapi.CreateSSH(bo.SSHSpecs)
if len(sshSpecs) == 0 && (buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL))) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
sshAttachment, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, err
}
@@ -1395,14 +1333,23 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
}
}
if t.CacheFrom != nil {
bo.CacheFrom = controllerapi.CreateCaches(t.CacheFrom.ToPB())
}
if t.CacheTo != nil {
bo.CacheTo = controllerapi.CreateCaches(t.CacheTo.ToPB())
cacheImports, err := buildflags.ParseCacheEntry(t.CacheFrom)
if err != nil {
return nil, err
}
bo.CacheFrom = controllerapi.CreateCaches(cacheImports)
bo.Exports, bo.ExportsLocalPathsTemporary, err = controllerapi.CreateExports(t.Outputs.ToPB())
cacheExports, err := buildflags.ParseCacheEntry(t.CacheTo)
if err != nil {
return nil, err
}
bo.CacheTo = controllerapi.CreateCaches(cacheExports)
outputs, err := buildflags.ParseExports(t.Outputs)
if err != nil {
return nil, err
}
bo.Exports, err = controllerapi.CreateExports(outputs)
if err != nil {
return nil, err
}
@@ -1417,7 +1364,11 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
}
}
bo.Attests = controllerapi.CreateAttestations(t.Attest.ToPB())
attests, err := buildflags.ParseAttests(t.Attest)
if err != nil {
return nil, err
}
bo.Attests = controllerapi.CreateAttestations(attests)
bo.SourcePolicy, err = build.ReadSourcePolicy()
if err != nil {
@@ -1443,7 +1394,7 @@ func defaultTarget() *Target {
return &Target{}
}
func removeDupesStr(s []string) []string {
func removeDupes(s []string) []string {
i := 0
seen := make(map[string]struct{}, len(s))
for _, v := range s {
@@ -1460,76 +1411,106 @@ func removeDupesStr(s []string) []string {
return s[:i]
}
func setPushOverride(outputs []*buildflags.ExportEntry, push bool) []*buildflags.ExportEntry {
if !push {
// Disable push for any relevant export types
for i := 0; i < len(outputs); {
output := outputs[i]
switch output.Type {
case "registry":
// Filter out registry output type
outputs[i], outputs[len(outputs)-1] = outputs[len(outputs)-1], outputs[i]
outputs = outputs[:len(outputs)-1]
continue
case "image":
// Override push attribute
output.Attrs["push"] = "false"
}
i++
}
return outputs
}
// Force push to be enabled
setPush := true
for _, output := range outputs {
if output.Type != "docker" {
// If there is an output type that is not docker, don't set "push"
setPush = false
func removeAttestDupes(s []string) []string {
res := []string{}
m := map[string]int{}
for _, v := range s {
att, err := buildflags.ParseAttest(v)
if err != nil {
res = append(res, v)
continue
}
// Set push attribute for image
if output.Type == "image" {
output.Attrs["push"] = "true"
if i, ok := m[att.Type]; ok {
res[i] = v
} else {
m[att.Type] = len(res)
res = append(res, v)
}
}
if setPush {
// No existing output that pushes so add one
outputs = append(outputs, &buildflags.ExportEntry{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
})
}
return outputs
return res
}
func setLoadOverride(outputs []*buildflags.ExportEntry, load bool) []*buildflags.ExportEntry {
func parseOutput(str string) map[string]string {
fields, err := csvvalue.Fields(str, nil)
if err != nil {
return nil
}
res := map[string]string{}
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) == 2 {
res[parts[0]] = parts[1]
}
}
return res
}
func parseOutputType(str string) string {
if out := parseOutput(str); out != nil {
if v, ok := out["type"]; ok {
return v
}
}
return ""
}
func setPushOverride(outputs []string, push bool) []string {
var out []string
setPush := true
for _, output := range outputs {
typ := parseOutputType(output)
if typ == "image" || typ == "registry" {
// no need to set push if image or registry types already defined
setPush = false
if typ == "registry" {
if !push {
// don't set registry output if "push" is false
continue
}
// no need to set "push" attribute to true for registry
out = append(out, output)
continue
}
out = append(out, output+",push="+strconv.FormatBool(push))
} else {
if typ != "docker" {
// if there is any output that is not docker, don't set "push"
setPush = false
}
out = append(out, output)
}
}
if push && setPush {
out = append(out, "type=image,push=true")
}
return out
}
func setLoadOverride(outputs []string, load bool) []string {
if !load {
return outputs
}
setLoad := true
for _, output := range outputs {
switch output.Type {
case "docker":
// if dest is not set, we can reuse this entry and do not need to set load
if output.Destination == "" {
return outputs
if typ := parseOutputType(output); typ == "docker" {
if v := parseOutput(output); v != nil {
// dest set means we want to output as tar so don't set load
if _, ok := v["dest"]; !ok {
setLoad = false
break
}
}
case "image", "registry", "oci":
// Ignore
default:
} else if typ != "image" && typ != "registry" && typ != "oci" {
// if there is any output that is not an image, registry
// or oci, don't set "load" similar to push override
return outputs
setLoad = false
break
}
}
outputs = append(outputs, &buildflags.ExportEntry{
Type: "docker",
})
if setLoad {
outputs = append(outputs, "type=docker")
}
return outputs
}
@@ -1547,9 +1528,14 @@ func sanitizeTargetName(target string) string {
return strings.ReplaceAll(target, ".", "_")
}
func isSubset(s1, s2 []string) bool {
for _, item := range s1 {
if !slices.Contains(s2, item) {
func sliceEqual(s1, s2 []string) bool {
if len(s1) != len(s2) {
return false
}
sort.Strings(s1)
sort.Strings(s2)
for i := range s1 {
if s1[i] != s2[i] {
return false
}
}
@@ -1563,24 +1549,3 @@ func toNamedContexts(m map[string]string) map[string]build.NamedContext {
}
return m2
}
type arrValue[B any] interface {
encoding.TextUnmarshaler
*B
}
func parseArrValue[T any, PT arrValue[T]](s []string) ([]*T, error) {
outputs := make([]*T, 0, len(s))
for _, text := range s {
if text == "" {
continue
}
output := new(T)
if err := PT(output).UnmarshalText([]byte(text)); err != nil {
return nil, err
}
outputs = append(outputs, output)
}
return outputs, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,14 +5,13 @@ import (
"fmt"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"github.com/compose-spec/compose-go/v2/consts"
"github.com/compose-spec/compose-go/v2/dotenv"
"github.com/compose-spec/compose-go/v2/loader"
composetypes "github.com/compose-spec/compose-go/v2/types"
"github.com/docker/buildx/util/buildflags"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/pkg/errors"
@@ -103,12 +102,6 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
shmSize = &shmSizeStr
}
var networkModeP *string
if s.Build.Network != "" {
networkMode := s.Build.Network
networkModeP = &networkMode
}
var ulimits []string
if s.Build.Ulimits != nil {
for n, u := range s.Build.Ulimits {
@@ -120,16 +113,14 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
}
}
var ssh []*buildflags.SSH
var ssh []string
for _, bkey := range s.Build.SSH {
sshkey := composeToBuildkitSSH(bkey)
ssh = append(ssh, sshkey)
}
slices.SortFunc(ssh, func(a, b *buildflags.SSH) int {
return a.Less(b)
})
sort.Strings(ssh)
var secrets []*buildflags.Secret
var secrets []string
for _, bs := range s.Build.Secrets {
secret, err := composeToBuildkitSecret(bs, cfg.Secrets[bs.Source])
if err != nil {
@@ -145,16 +136,6 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
labels[k] = &v
}
cacheFrom, err := buildflags.ParseCacheEntry(s.Build.CacheFrom)
if err != nil {
return nil, err
}
cacheTo, err := buildflags.ParseCacheEntry(s.Build.CacheTo)
if err != nil {
return nil, err
}
g.Targets = append(g.Targets, targetName)
t := &Target{
Name: targetName,
@@ -171,9 +152,9 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
val, ok := cfg.Environment[val]
return val, ok
})),
CacheFrom: cacheFrom,
CacheTo: cacheTo,
NetworkMode: networkModeP,
CacheFrom: s.Build.CacheFrom,
CacheTo: s.Build.CacheTo,
NetworkMode: &s.Build.Network,
SSH: ssh,
Secrets: secrets,
ShmSize: shmSize,
@@ -192,6 +173,7 @@ func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Conf
c.Targets = append(c.Targets, t)
}
c.Groups = append(c.Groups, g)
}
return &c, nil
@@ -310,10 +292,8 @@ type xbake struct {
// https://github.com/docker/docs/blob/main/content/build/bake/compose-file.md#extension-field-with-x-bake
}
type (
stringMap map[string]string
stringArray []string
)
type stringMap map[string]string
type stringArray []string
func (sa *stringArray) UnmarshalYAML(unmarshal func(interface{}) error) error {
var multi []string
@@ -349,45 +329,23 @@ func (t *Target) composeExtTarget(exts map[string]interface{}) error {
t.Tags = dedupSlice(append(t.Tags, xb.Tags...))
}
if len(xb.CacheFrom) > 0 {
cacheFrom, err := buildflags.ParseCacheEntry(xb.CacheFrom)
if err != nil {
return err
}
t.CacheFrom = t.CacheFrom.Merge(cacheFrom)
t.CacheFrom = dedupSlice(append(t.CacheFrom, xb.CacheFrom...))
}
if len(xb.CacheTo) > 0 {
cacheTo, err := buildflags.ParseCacheEntry(xb.CacheTo)
if err != nil {
return err
}
t.CacheTo = t.CacheTo.Merge(cacheTo)
t.CacheTo = dedupSlice(append(t.CacheTo, xb.CacheTo...))
}
if len(xb.Secrets) > 0 {
secrets, err := parseArrValue[buildflags.Secret](xb.Secrets)
if err != nil {
return err
}
t.Secrets = t.Secrets.Merge(secrets)
t.Secrets = dedupSlice(append(t.Secrets, xb.Secrets...))
}
if len(xb.SSH) > 0 {
ssh, err := parseArrValue[buildflags.SSH](xb.SSH)
if err != nil {
return err
}
t.SSH = t.SSH.Merge(ssh)
slices.SortFunc(t.SSH, func(a, b *buildflags.SSH) int {
return a.Less(b)
})
t.SSH = dedupSlice(append(t.SSH, xb.SSH...))
sort.Strings(t.SSH)
}
if len(xb.Platforms) > 0 {
t.Platforms = dedupSlice(append(t.Platforms, xb.Platforms...))
}
if len(xb.Outputs) > 0 {
outputs, err := parseArrValue[buildflags.ExportEntry](xb.Outputs)
if err != nil {
return err
}
t.Outputs = t.Outputs.Merge(outputs)
t.Outputs = dedupSlice(append(t.Outputs, xb.Outputs...))
}
if xb.Pull != nil {
t.Pull = xb.Pull
@@ -407,30 +365,35 @@ func (t *Target) composeExtTarget(exts map[string]interface{}) error {
// composeToBuildkitSecret converts secret from compose format to buildkit's
// csv format.
func composeToBuildkitSecret(inp composetypes.ServiceSecretConfig, psecret composetypes.SecretConfig) (*buildflags.Secret, error) {
func composeToBuildkitSecret(inp composetypes.ServiceSecretConfig, psecret composetypes.SecretConfig) (string, error) {
if psecret.External {
return nil, errors.Errorf("unsupported external secret %s", psecret.Name)
return "", errors.Errorf("unsupported external secret %s", psecret.Name)
}
secret := &buildflags.Secret{}
var bkattrs []string
if inp.Source != "" {
secret.ID = inp.Source
bkattrs = append(bkattrs, "id="+inp.Source)
}
if psecret.File != "" {
secret.FilePath = psecret.File
bkattrs = append(bkattrs, "src="+psecret.File)
}
if psecret.Environment != "" {
secret.Env = psecret.Environment
bkattrs = append(bkattrs, "env="+psecret.Environment)
}
return secret, nil
return strings.Join(bkattrs, ","), nil
}
// composeToBuildkitSSH converts secret from compose format to buildkit's
// csv format.
func composeToBuildkitSSH(sshKey composetypes.SSHKey) *buildflags.SSH {
bkssh := &buildflags.SSH{ID: sshKey.ID}
func composeToBuildkitSSH(sshKey composetypes.SSHKey) string {
var bkattrs []string
bkattrs = append(bkattrs, sshKey.ID)
if sshKey.Path != "" {
bkssh.Paths = []string{sshKey.Path}
bkattrs = append(bkattrs, sshKey.Path)
}
return bkssh
return strings.Join(bkattrs, "=")
}

View File

@@ -12,7 +12,7 @@ import (
)
func TestParseCompose(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
db:
build: ./db
@@ -33,7 +33,7 @@ services:
cache_to:
- type=local,dest=path/to/cache
ssh:
- key=/path/to/key
- key=path/to/key
- default
secrets:
- token
@@ -74,14 +74,14 @@ secrets:
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, []string{"type=local,src=path/to/cache"}, stringify(c.Targets[1].CacheFrom))
require.Equal(t, []string{"type=local,dest=path/to/cache"}, stringify(c.Targets[1].CacheTo))
require.Equal(t, []string{"type=local,src=path/to/cache"}, c.Targets[1].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache"}, c.Targets[1].CacheTo)
require.Equal(t, "none", *c.Targets[1].NetworkMode)
require.Equal(t, []string{"default", "key=/path/to/key"}, stringify(c.Targets[1].SSH))
require.Equal(t, []string{"default", "key=path/to/key"}, c.Targets[1].SSH)
require.Equal(t, []string{
"id=aws,src=/root/.aws/credentials",
"id=token,env=ENV_TOKEN",
}, stringify(c.Targets[1].Secrets))
"id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, "dir", *c.Targets[2].Context)
@@ -89,7 +89,7 @@ secrets:
}
func TestNoBuildOutOfTreeService(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
external:
image: "verycooldb:1337"
@@ -103,7 +103,7 @@ services:
}
func TestParseComposeTarget(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
db:
build:
@@ -129,7 +129,7 @@ services:
}
func TestComposeBuildWithoutContext(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
db:
build:
@@ -153,7 +153,7 @@ services:
}
func TestBuildArgEnvCompose(t *testing.T) {
dt := []byte(`
var dt = []byte(`
version: "3.8"
services:
example:
@@ -179,7 +179,7 @@ services:
}
func TestInconsistentComposeFile(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
webapp:
entrypoint: echo 1
@@ -190,7 +190,7 @@ services:
}
func TestAdvancedNetwork(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
db:
networks:
@@ -215,7 +215,7 @@ networks:
}
func TestTags(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
example:
image: example
@@ -233,7 +233,7 @@ services:
}
func TestDependsOnList(t *testing.T) {
dt := []byte(`
var dt = []byte(`
version: "3.8"
services:
@@ -269,7 +269,7 @@ networks:
}
func TestComposeExt(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
addon:
image: ct-addon:bar
@@ -283,7 +283,7 @@ services:
tags:
- ct-addon:baz
ssh:
key: /path/to/key
key: path/to/key
args:
CT_ECR: foo
CT_TAG: bar
@@ -336,23 +336,23 @@ services:
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "CT_TAG": ptrstr("bar")}, c.Targets[0].Args)
require.Equal(t, []string{"ct-addon:baz", "ct-addon:foo", "ct-addon:alp"}, c.Targets[0].Tags)
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[0].Platforms)
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom))
require.Equal(t, []string{"type=local,dest=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheTo))
require.Equal(t, []string{"default", "key=/path/to/key", "other=path/to/otherkey"}, stringify(c.Targets[0].SSH))
require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
require.Equal(t, []string{"default", "key=path/to/key", "other=path/to/otherkey"}, c.Targets[0].SSH)
require.Equal(t, newBool(true), c.Targets[0].Pull)
require.Equal(t, map[string]string{"alpine": "docker-image://alpine:3.13"}, c.Targets[0].Contexts)
require.Equal(t, []string{"ct-fake-aws:bar"}, c.Targets[1].Tags)
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,src=/local/secret2"}, stringify(c.Targets[1].Secrets))
require.Equal(t, []string{"default"}, stringify(c.Targets[1].SSH))
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,src=/local/secret2"}, c.Targets[1].Secrets)
require.Equal(t, []string{"default"}, c.Targets[1].SSH)
require.Equal(t, []string{"linux/arm64"}, c.Targets[1].Platforms)
require.Equal(t, []string{"type=docker"}, stringify(c.Targets[1].Outputs))
require.Equal(t, []string{"type=docker"}, c.Targets[1].Outputs)
require.Equal(t, newBool(true), c.Targets[1].NoCache)
require.Equal(t, ptrstr("128MiB"), c.Targets[1].ShmSize)
require.Equal(t, []string{"nofile=1024:1024"}, c.Targets[1].Ulimits)
}
func TestComposeExtDedup(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
webapp:
image: app:bar
@@ -383,9 +383,9 @@ services:
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"ct-addon:foo", "ct-addon:baz"}, c.Targets[0].Tags)
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom))
require.Equal(t, []string{"type=local,dest=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheTo))
require.Equal(t, []string{"default", "key=path/to/key"}, stringify(c.Targets[0].SSH))
require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
require.Equal(t, []string{"default", "key=path/to/key"}, c.Targets[0].SSH)
}
func TestEnv(t *testing.T) {
@@ -396,7 +396,7 @@ func TestEnv(t *testing.T) {
_, err = envf.WriteString("FOO=bsdf -csdf\n")
require.NoError(t, err)
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:
@@ -424,7 +424,7 @@ func TestDotEnv(t *testing.T) {
err := os.WriteFile(filepath.Join(tmpdir, ".env"), []byte("FOO=bar"), 0644)
require.NoError(t, err)
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:
@@ -443,7 +443,7 @@ services:
}
func TestPorts(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
foo:
build:
@@ -664,7 +664,7 @@ target "default" {
}
func TestComposeNullArgs(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:
@@ -680,7 +680,7 @@ services:
}
func TestDependsOn(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
foo:
build:
@@ -711,7 +711,7 @@ services:
`), 0644)
require.NoError(t, err)
dt := []byte(`
var dt = []byte(`
include:
- compose-foo.yml
@@ -740,7 +740,7 @@ services:
}
func TestDevelop(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:
@@ -759,7 +759,7 @@ services:
}
func TestCgroup(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:
@@ -772,7 +772,7 @@ services:
}
func TestProjectName(t *testing.T) {
dt := []byte(`
var dt = []byte(`
services:
scratch:
build:

View File

@@ -2,24 +2,17 @@ package bake
import (
"bufio"
"cmp"
"context"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"syscall"
"github.com/containerd/console"
"github.com/docker/buildx/build"
"github.com/docker/buildx/util/osutil"
"github.com/moby/buildkit/util/entitlements"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type EntitlementKey string
@@ -74,8 +67,10 @@ func ParseEntitlements(in []string) (EntitlementConf, error) {
conf.ImagePush = append(conf.ImagePush, v)
conf.ImageLoad = append(conf.ImageLoad, v)
default:
return conf, errors.Errorf("unknown entitlement key %q", k)
return conf, errors.Errorf("uknown entitlement key %q", k)
}
// TODO: dedupe slices and parent paths
}
}
return conf, nil
@@ -106,66 +101,10 @@ func (c EntitlementConf) check(bo build.Options, expected *EntitlementConf) erro
}
}
}
rwPaths := map[string]struct{}{}
roPaths := map[string]struct{}{}
for _, p := range collectLocalPaths(bo.Inputs) {
roPaths[p] = struct{}{}
}
for _, p := range bo.ExportsLocalPathsTemporary {
rwPaths[p] = struct{}{}
}
for _, ce := range bo.CacheTo {
if ce.Type == "local" {
if dest, ok := ce.Attrs["dest"]; ok {
rwPaths[dest] = struct{}{}
}
}
}
for _, ci := range bo.CacheFrom {
if ci.Type == "local" {
if src, ok := ci.Attrs["src"]; ok {
roPaths[src] = struct{}{}
}
}
}
for _, secret := range bo.SecretSpecs {
if secret.FilePath != "" {
roPaths[secret.FilePath] = struct{}{}
}
}
for _, ssh := range bo.SSHSpecs {
for _, p := range ssh.Paths {
roPaths[p] = struct{}{}
}
if len(ssh.Paths) == 0 {
if !c.SSH {
expected.SSH = true
}
}
}
var err error
expected.FSRead, err = findMissingPaths(c.FSRead, roPaths)
if err != nil {
return err
}
expected.FSWrite, err = findMissingPaths(c.FSWrite, rwPaths)
if err != nil {
return err
}
return nil
}
func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Writer) error {
func (c EntitlementConf) Prompt(ctx context.Context, out io.Writer) error {
var term bool
if _, err := console.ConsoleFromFile(os.Stdin); err == nil {
term = true
@@ -174,78 +113,32 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
var msgs []string
var flags []string
// these warnings are currently disabled to give users time to update
var msgsFS []string
var flagsFS []string
if c.NetworkHost {
msgs = append(msgs, " - Running build containers that can access host network")
flags = append(flags, string(EntitlementKeyNetworkHost))
flags = append(flags, "network.host")
}
if c.SecurityInsecure {
msgs = append(msgs, " - Running privileged containers that can make system changes")
flags = append(flags, string(EntitlementKeySecurityInsecure))
flags = append(flags, "security.insecure")
}
if c.SSH {
msgsFS = append(msgsFS, " - Forwarding default SSH agent socket")
flagsFS = append(flagsFS, string(EntitlementKeySSH))
}
roPaths, rwPaths, commonPaths := groupSamePaths(c.FSRead, c.FSWrite)
wd, err := os.Getwd()
if err != nil {
return errors.Wrap(err, "failed to get current working directory")
}
wd, err = filepath.EvalSymlinks(wd)
if err != nil {
return errors.Wrap(err, "failed to evaluate working directory")
}
roPaths = toRelativePaths(roPaths, wd)
rwPaths = toRelativePaths(rwPaths, wd)
commonPaths = toRelativePaths(commonPaths, wd)
if len(commonPaths) > 0 {
for _, p := range commonPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Read and write access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFS)+"="+p)
}
}
if len(roPaths) > 0 {
for _, p := range roPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Read access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFSRead)+"="+p)
}
}
if len(rwPaths) > 0 {
for _, p := range rwPaths {
msgsFS = append(msgsFS, fmt.Sprintf(" - Write access to path %s", p))
flagsFS = append(flagsFS, string(EntitlementKeyFSWrite)+"="+p)
}
}
if len(msgs) == 0 && len(msgsFS) == 0 {
if len(msgs) == 0 {
return nil
}
fmt.Fprintf(out, "Your build is requesting privileges for following possibly insecure capabilities:\n\n")
for _, m := range slices.Concat(msgs, msgsFS) {
for _, m := range msgs {
fmt.Fprintf(out, "%s\n", m)
}
for i, f := range flags {
flags[i] = "--allow=" + f
}
for i, f := range flagsFS {
flagsFS[i] = "--allow=" + f
}
if term {
fmt.Fprintf(out, "\nIn order to not see this message in the future pass %q to grant requested privileges.\n", strings.Join(slices.Concat(flags, flagsFS), " "))
fmt.Fprintf(out, "\nIn order to not see this message in the future pass %q to grant requested privileges.\n", strings.Join(flags, " "))
} else {
fmt.Fprintf(out, "\nPass %q to grant requested privileges.\n", strings.Join(slices.Concat(flags, flagsFS), " "))
fmt.Fprintf(out, "\nPass %q to grant requested privileges.\n", strings.Join(flags, " "))
}
args := append([]string(nil), os.Args...)
@@ -256,33 +149,7 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
if idx != -1 {
fmt.Fprintf(out, "\nYour full command with requested privileges:\n\n")
fmt.Fprintf(out, "%s %s %s\n\n", strings.Join(args[:idx+1], " "), strings.Join(slices.Concat(flags, flagsFS), " "), strings.Join(args[idx+1:], " "))
}
fsEntitlementsEnabled := true
if isRemote {
if v, ok := os.LookupEnv("BAKE_ALLOW_REMOTE_FS_ACCESS"); ok {
vv, err := strconv.ParseBool(v)
if err != nil {
return errors.Wrapf(err, "failed to parse BAKE_ALLOW_REMOTE_FS_ACCESS value %q", v)
}
fsEntitlementsEnabled = !vv
}
}
v, fsEntitlementsSet := os.LookupEnv("BUILDX_BAKE_ENTITLEMENTS_FS")
if fsEntitlementsSet {
vv, err := strconv.ParseBool(v)
if err != nil {
return errors.Wrapf(err, "failed to parse BUILDX_BAKE_ENTITLEMENTS_FS value %q", v)
}
fsEntitlementsEnabled = vv
}
if !fsEntitlementsEnabled && len(msgs) == 0 {
return nil
}
if fsEntitlementsEnabled && !fsEntitlementsSet && len(msgsFS) != 0 {
fmt.Fprintf(out, "To disable filesystem entitlements checks, you can set BUILDX_BAKE_ENTITLEMENTS_FS=0 .\n\n")
fmt.Fprintf(out, "%s %s %s\n\n", strings.Join(args[:idx+1], " "), strings.Join(flags, " "), strings.Join(args[idx+1:], " "))
}
if term {
@@ -306,296 +173,3 @@ func (c EntitlementConf) Prompt(ctx context.Context, isRemote bool, out io.Write
return errors.Errorf("additional privileges requested")
}
func isParentOrEqualPath(p, parent string) bool {
if p == parent || parent == "/" {
return true
}
if strings.HasPrefix(p, filepath.Clean(parent+string(filepath.Separator))) {
return true
}
return false
}
func findMissingPaths(set []string, paths map[string]struct{}) ([]string, error) {
set, allowAny, err := evaluatePaths(set)
if err != nil {
return nil, err
} else if allowAny {
return nil, nil
}
paths, err = evaluateToExistingPaths(paths)
if err != nil {
return nil, err
}
paths, err = dedupPaths(paths)
if err != nil {
return nil, err
}
out := make([]string, 0, len(paths))
loop0:
for p := range paths {
for _, c := range set {
if isParentOrEqualPath(p, c) {
continue loop0
}
}
out = append(out, p)
}
if len(out) == 0 {
return nil, nil
}
slices.Sort(out)
return out, nil
}
func dedupPaths(in map[string]struct{}) (map[string]struct{}, error) {
arr := make([]string, 0, len(in))
for p := range in {
arr = append(arr, filepath.Clean(p))
}
slices.SortFunc(arr, func(a, b string) int {
return cmp.Compare(len(a), len(b))
})
m := make(map[string]struct{}, len(arr))
loop0:
for _, p := range arr {
for parent := range m {
if strings.HasPrefix(p, parent+string(filepath.Separator)) {
continue loop0
}
}
m[p] = struct{}{}
}
return m, nil
}
func toRelativePaths(in []string, wd string) []string {
out := make([]string, 0, len(in))
for _, p := range in {
rel, err := filepath.Rel(wd, p)
if err == nil {
// allow up to one level of ".." in the path
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)+"..") {
out = append(out, rel)
continue
}
}
out = append(out, p)
}
return out
}
func groupSamePaths(in1, in2 []string) ([]string, []string, []string) {
if in1 == nil || in2 == nil {
return in1, in2, nil
}
slices.Sort(in1)
slices.Sort(in2)
common := []string{}
i, j := 0, 0
for i < len(in1) && j < len(in2) {
switch {
case in1[i] == in2[j]:
common = append(common, in1[i])
i++
j++
case in1[i] < in2[j]:
i++
default:
j++
}
}
in1 = removeCommonPaths(in1, common)
in2 = removeCommonPaths(in2, common)
return in1, in2, common
}
func removeCommonPaths(in, common []string) []string {
filtered := make([]string, 0, len(in))
commonIndex := 0
for _, path := range in {
if commonIndex < len(common) && path == common[commonIndex] {
commonIndex++
continue
}
filtered = append(filtered, path)
}
return filtered
}
func evaluatePaths(in []string) ([]string, bool, error) {
out := make([]string, 0, len(in))
allowAny := false
for _, p := range in {
if p == "*" {
allowAny = true
continue
}
v, err := filepath.Abs(p)
if err != nil {
logrus.Warnf("failed to evaluate entitlement path %q: %v", p, err)
continue
}
v, rest, err := evaluateToExistingPath(v)
if err != nil {
return nil, false, errors.Wrapf(err, "failed to evaluate path %q", p)
}
v, err = osutil.GetLongPathName(v)
if err != nil {
return nil, false, errors.Wrapf(err, "failed to evaluate path %q", p)
}
if rest != "" {
v = filepath.Join(v, rest)
}
out = append(out, v)
}
return out, allowAny, nil
}
func evaluateToExistingPaths(in map[string]struct{}) (map[string]struct{}, error) {
m := make(map[string]struct{}, len(in))
for p := range in {
v, _, err := evaluateToExistingPath(p)
if err != nil {
return nil, errors.Wrapf(err, "failed to evaluate path %q", p)
}
v, err = osutil.GetLongPathName(v)
if err != nil {
return nil, errors.Wrapf(err, "failed to evaluate path %q", p)
}
m[v] = struct{}{}
}
return m, nil
}
func evaluateToExistingPath(in string) (string, string, error) {
in, err := filepath.Abs(in)
if err != nil {
return "", "", err
}
volLen := volumeNameLen(in)
pathSeparator := string(os.PathSeparator)
if volLen < len(in) && os.IsPathSeparator(in[volLen]) {
volLen++
}
vol := in[:volLen]
dest := vol
linksWalked := 0
var end int
for start := volLen; start < len(in); start = end {
for start < len(in) && os.IsPathSeparator(in[start]) {
start++
}
end = start
for end < len(in) && !os.IsPathSeparator(in[end]) {
end++
}
if end == start {
break
} else if in[start:end] == "." {
continue
} else if in[start:end] == ".." {
var r int
for r = len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
break
}
}
if r < volLen || dest[r+1:] == ".." {
if len(dest) > volLen {
dest += pathSeparator
}
dest += ".."
} else {
dest = dest[:r]
}
continue
}
if len(dest) > volumeNameLen(dest) && !os.IsPathSeparator(dest[len(dest)-1]) {
dest += pathSeparator
}
dest += in[start:end]
fi, err := os.Lstat(dest)
if err != nil {
// If the component doesn't exist, return the last valid path
if os.IsNotExist(err) {
for r := len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
return dest[:r], in[start:], nil
}
}
return vol, in[start:], nil
}
return "", "", err
}
if fi.Mode()&fs.ModeSymlink == 0 {
if !fi.Mode().IsDir() && end < len(in) {
return "", "", syscall.ENOTDIR
}
continue
}
linksWalked++
if linksWalked > 255 {
return "", "", errors.New("too many symlinks")
}
link, err := os.Readlink(dest)
if err != nil {
return "", "", err
}
in = link + in[end:]
v := volumeNameLen(link)
if v > 0 {
if v < len(link) && os.IsPathSeparator(link[v]) {
v++
}
vol = link[:v]
dest = vol
end = len(vol)
} else if len(link) > 0 && os.IsPathSeparator(link[0]) {
dest = link[:1]
end = 1
vol = link[:1]
volLen = 1
} else {
var r int
for r = len(dest) - 1; r >= volLen; r-- {
if os.IsPathSeparator(dest[r]) {
break
}
}
if r < volLen {
dest = vol
} else {
dest = dest[:r]
}
end = 0
}
}
return filepath.Clean(dest), "", nil
}
func volumeNameLen(s string) int {
return len(filepath.VolumeName(s))
}

View File

@@ -1,486 +0,0 @@
package bake
import (
"fmt"
"os"
"path/filepath"
"slices"
"testing"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/osutil"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/util/entitlements"
"github.com/stretchr/testify/require"
)
func TestEvaluateToExistingPath(t *testing.T) {
tempDir, err := osutil.GetLongPathName(t.TempDir())
require.NoError(t, err)
// Setup temporary directory structure for testing
existingFile := filepath.Join(tempDir, "existing_file")
require.NoError(t, os.WriteFile(existingFile, []byte("test"), 0644))
existingDir := filepath.Join(tempDir, "existing_dir")
require.NoError(t, os.Mkdir(existingDir, 0755))
symlinkToFile := filepath.Join(tempDir, "symlink_to_file")
require.NoError(t, os.Symlink(existingFile, symlinkToFile))
symlinkToDir := filepath.Join(tempDir, "symlink_to_dir")
require.NoError(t, os.Symlink(existingDir, symlinkToDir))
nonexistentPath := filepath.Join(tempDir, "nonexistent", "path", "file.txt")
tests := []struct {
name string
input string
expected string
expectErr bool
}{
{
name: "Existing file",
input: existingFile,
expected: existingFile,
expectErr: false,
},
{
name: "Existing directory",
input: existingDir,
expected: existingDir,
expectErr: false,
},
{
name: "Symlink to file",
input: symlinkToFile,
expected: existingFile,
expectErr: false,
},
{
name: "Symlink to directory",
input: symlinkToDir,
expected: existingDir,
expectErr: false,
},
{
name: "Non-existent path",
input: nonexistentPath,
expected: tempDir,
expectErr: false,
},
{
name: "Non-existent intermediate path",
input: filepath.Join(tempDir, "nonexistent", "file.txt"),
expected: tempDir,
expectErr: false,
},
{
name: "Root path",
input: "/",
expected: func() string {
root, _ := filepath.Abs("/")
return root
}(),
expectErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
result, _, err := evaluateToExistingPath(tt.input)
if tt.expectErr {
require.Error(t, err)
} else {
require.NoError(t, err)
require.Equal(t, tt.expected, result)
}
})
}
}
func TestDedupePaths(t *testing.T) {
wd := osutil.GetWd()
tcases := []struct {
in map[string]struct{}
out map[string]struct{}
}{
{
in: map[string]struct{}{
"/a/b/c": {},
"/a/b/d": {},
"/a/b/e": {},
},
out: map[string]struct{}{
"/a/b/c": {},
"/a/b/d": {},
"/a/b/e": {},
},
},
{
in: map[string]struct{}{
"/a/b/c": {},
"/a/b/c/d": {},
"/a/b/c/d/e": {},
"/a/b/../b/c": {},
},
out: map[string]struct{}{
"/a/b/c": {},
},
},
{
in: map[string]struct{}{
filepath.Join(wd, "a/b/c"): {},
filepath.Join(wd, "../aa"): {},
filepath.Join(wd, "a/b"): {},
filepath.Join(wd, "a/b/d"): {},
filepath.Join(wd, "../aa/b"): {},
filepath.Join(wd, "../../bb"): {},
},
out: map[string]struct{}{
"a/b": {},
"../aa": {},
filepath.Join(wd, "../../bb"): {},
},
},
}
for i, tc := range tcases {
t.Run(fmt.Sprintf("case%d", i), func(t *testing.T) {
out, err := dedupPaths(tc.in)
if err != nil {
require.NoError(t, err)
}
// convert to relative paths as that is shown to user
arr := make([]string, 0, len(out))
for k := range out {
arr = append(arr, k)
}
require.NoError(t, err)
arr = toRelativePaths(arr, wd)
m := make(map[string]struct{})
for _, v := range arr {
m[filepath.ToSlash(v)] = struct{}{}
}
o := make(map[string]struct{}, len(tc.out))
for k := range tc.out {
o[filepath.ToSlash(k)] = struct{}{}
}
require.Equal(t, o, m)
})
}
}
func TestValidateEntitlements(t *testing.T) {
dir1 := t.TempDir()
dir2 := t.TempDir()
// the paths returned by entitlements validation will have symlinks resolved
expDir1, err := filepath.EvalSymlinks(dir1)
require.NoError(t, err)
expDir2, err := filepath.EvalSymlinks(dir2)
require.NoError(t, err)
escapeLink := filepath.Join(dir1, "escape_link")
require.NoError(t, os.Symlink("../../aa", escapeLink))
wd, err := os.Getwd()
require.NoError(t, err)
expWd, err := filepath.EvalSymlinks(wd)
require.NoError(t, err)
tcases := []struct {
name string
conf EntitlementConf
opt build.Options
expected EntitlementConf
}{
{
name: "No entitlements",
opt: build.Options{
Inputs: build.Inputs{
ContextState: &llb.State{},
},
},
},
{
name: "NetworkHostMissing",
opt: build.Options{
Allow: []entitlements.Entitlement{
entitlements.EntitlementNetworkHost,
},
},
expected: EntitlementConf{
NetworkHost: true,
FSRead: []string{expWd},
},
},
{
name: "NetworkHostSet",
conf: EntitlementConf{
NetworkHost: true,
},
opt: build.Options{
Allow: []entitlements.Entitlement{
entitlements.EntitlementNetworkHost,
},
},
expected: EntitlementConf{
FSRead: []string{expWd},
},
},
{
name: "SecurityAndNetworkHostMissing",
opt: build.Options{
Allow: []entitlements.Entitlement{
entitlements.EntitlementNetworkHost,
entitlements.EntitlementSecurityInsecure,
},
},
expected: EntitlementConf{
NetworkHost: true,
SecurityInsecure: true,
FSRead: []string{expWd},
},
},
{
name: "SecurityMissingAndNetworkHostSet",
conf: EntitlementConf{
NetworkHost: true,
},
opt: build.Options{
Allow: []entitlements.Entitlement{
entitlements.EntitlementNetworkHost,
entitlements.EntitlementSecurityInsecure,
},
},
expected: EntitlementConf{
SecurityInsecure: true,
FSRead: []string{expWd},
},
},
{
name: "SSHMissing",
opt: build.Options{
SSHSpecs: []*pb.SSH{
{
ID: "test",
},
},
},
expected: EntitlementConf{
SSH: true,
FSRead: []string{expWd},
},
},
{
name: "ExportLocal",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
dir1,
filepath.Join(dir1, "subdir"),
dir2,
},
},
expected: EntitlementConf{
FSWrite: func() []string {
exp := []string{expDir1, expDir2}
slices.Sort(exp)
return exp
}(),
FSRead: []string{expWd},
},
},
{
name: "SecretFromSubFile",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: filepath.Join(dir1, "subfile"),
},
},
},
conf: EntitlementConf{
FSRead: []string{wd, dir1},
},
},
{
name: "SecretFromEscapeLink",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{wd, dir1},
},
expected: EntitlementConf{
FSRead: []string{filepath.Join(expDir1, "../..")},
},
},
{
name: "SecretFromEscapeLinkAllowRoot",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{"/"},
},
expected: EntitlementConf{
FSRead: func() []string {
// on windows root (/) is only allowed if it is the same volume as wd
if filepath.VolumeName(wd) == filepath.VolumeName(escapeLink) {
return nil
}
// if not, then escapeLink is not allowed
exp, _, err := evaluateToExistingPath(escapeLink)
require.NoError(t, err)
exp, err = filepath.EvalSymlinks(exp)
require.NoError(t, err)
return []string{exp}
}(),
},
},
{
name: "SecretFromEscapeLinkAllowAny",
opt: build.Options{
SecretSpecs: []*pb.Secret{
{
FilePath: escapeLink,
},
},
},
conf: EntitlementConf{
FSRead: []string{"*"},
},
expected: EntitlementConf{},
},
{
name: "NonExistingAllowedPathSubpath",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
dir1,
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{filepath.Join(dir1, "not/exists")},
},
expected: EntitlementConf{
FSWrite: []string{expDir1}, // dir1 is still needed as only subpath was allowed
},
},
{
name: "NonExistingAllowedPathMatches",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
filepath.Join(dir1, "not/exists"),
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{filepath.Join(dir1, "not/exists")},
},
expected: EntitlementConf{
FSWrite: []string{expDir1}, // dir1 is still needed as build also needs to write not/exists directory
},
},
{
name: "NonExistingBuildPath",
opt: build.Options{
ExportsLocalPathsTemporary: []string{
filepath.Join(dir1, "not/exists"),
},
},
conf: EntitlementConf{
FSRead: []string{wd},
FSWrite: []string{dir1},
},
},
}
for _, tc := range tcases {
t.Run(tc.name, func(t *testing.T) {
expected, err := tc.conf.Validate(map[string]build.Options{"test": tc.opt})
require.NoError(t, err)
require.Equal(t, tc.expected, expected)
})
}
}
func TestGroupSamePaths(t *testing.T) {
tests := []struct {
name string
in1 []string
in2 []string
expected1 []string
expected2 []string
expectedC []string
}{
{
name: "All common paths",
in1: []string{"/path/a", "/path/b", "/path/c"},
in2: []string{"/path/a", "/path/b", "/path/c"},
expected1: []string{},
expected2: []string{},
expectedC: []string{"/path/a", "/path/b", "/path/c"},
},
{
name: "No common paths",
in1: []string{"/path/a", "/path/b"},
in2: []string{"/path/c", "/path/d"},
expected1: []string{"/path/a", "/path/b"},
expected2: []string{"/path/c", "/path/d"},
expectedC: []string{},
},
{
name: "Some common paths",
in1: []string{"/path/a", "/path/b", "/path/c"},
in2: []string{"/path/b", "/path/c", "/path/d"},
expected1: []string{"/path/a"},
expected2: []string{"/path/d"},
expectedC: []string{"/path/b", "/path/c"},
},
{
name: "Empty inputs",
in1: []string{},
in2: []string{},
expected1: []string{},
expected2: []string{},
expectedC: []string{},
},
{
name: "One empty input",
in1: []string{"/path/a", "/path/b"},
in2: []string{},
expected1: []string{"/path/a", "/path/b"},
expected2: []string{},
expectedC: []string{},
},
{
name: "Unsorted inputs with common paths",
in1: []string{"/path/c", "/path/a", "/path/b"},
in2: []string{"/path/b", "/path/c", "/path/a"},
expected1: []string{},
expected2: []string{},
expectedC: []string{"/path/a", "/path/b", "/path/c"},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
out1, out2, common := groupSamePaths(tt.in1, tt.in2)
require.Equal(t, tt.expected1, out1, "in1 should match expected1")
require.Equal(t, tt.expected2, out2, "in2 should match expected2")
require.Equal(t, tt.expectedC, common, "common should match expectedC")
})
}
}

View File

@@ -56,7 +56,7 @@ func formatHCLError(err error, files []File) error {
break
}
}
src := &errdefs.Source{
src := errdefs.Source{
Info: &pb.SourceInfo{
Filename: d.Subject.Filename,
Data: dt,
@@ -72,7 +72,7 @@ func formatHCLError(err error, files []File) error {
func toErrRange(in *hcl.Range) *pb.Range {
return &pb.Range{
Start: &pb.Position{Line: int32(in.Start.Line), Character: int32(in.Start.Column)},
End: &pb.Position{Line: int32(in.End.Line), Character: int32(in.End.Column)},
Start: pb.Position{Line: int32(in.Start.Line), Character: int32(in.Start.Column)},
End: pb.Position{Line: int32(in.End.Line), Character: int32(in.End.Column)},
}
}

View File

@@ -17,7 +17,6 @@ func TestHCLBasic(t *testing.T) {
target "db" {
context = "./db"
tags = ["docker.io/tonistiigi/db"]
output = ["type=image"]
}
target "webapp" {
@@ -26,9 +25,6 @@ func TestHCLBasic(t *testing.T) {
args = {
buildno = "123"
}
output = [
{ type = "image" }
]
}
target "cross" {
@@ -53,18 +49,18 @@ func TestHCLBasic(t *testing.T) {
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, 4, len(c.Targets))
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "db")
require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, "cross", c.Targets[2].Name)
require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms))
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms)
require.Equal(t, "webapp-plus", c.Targets[3].Name)
require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
}
@@ -113,18 +109,18 @@ func TestHCLBasicInJSON(t *testing.T) {
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, 4, len(c.Targets))
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "db")
require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, "cross", c.Targets[2].Name)
require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms))
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[2].Platforms)
require.Equal(t, "webapp-plus", c.Targets[3].Name)
require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
}
@@ -150,7 +146,7 @@ func TestHCLWithFunctions(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
}
@@ -180,7 +176,7 @@ func TestHCLWithUserDefinedFunctions(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
}
@@ -209,7 +205,7 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("123"), c.Targets[0].Args["buildno"])
t.Setenv("BUILD_NUMBER", "456")
@@ -222,7 +218,7 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, []string{"webapp"}, c.Groups[0].Targets)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, ptrstr("456"), c.Targets[0].Args["buildno"])
}
@@ -245,7 +241,7 @@ func TestHCLWithVariablesInFunctions(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, []string{"user/repo:v1"}, c.Targets[0].Tags)
t.Setenv("REPO", "docker/buildx")
@@ -254,7 +250,7 @@ func TestHCLWithVariablesInFunctions(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "webapp", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, []string{"docker/buildx:v1"}, c.Targets[0].Tags)
}
@@ -283,7 +279,7 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-abc"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("abc-post"), c.Targets[0].Args["v2"])
@@ -296,7 +292,7 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("def-post"), c.Targets[0].Args["v2"])
}
@@ -332,7 +328,7 @@ func TestHCLVarsWithVars(t *testing.T) {
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre--ABCDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("ABCDEF-post"), c.Targets[0].Args["v2"])
@@ -345,7 +341,7 @@ func TestHCLVarsWithVars(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre--NEWDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("NEWDEF-post"), c.Targets[0].Args["v2"])
}
@@ -370,7 +366,7 @@ func TestHCLTypedVariables(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("lower"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("yes"), c.Targets[0].Args["v2"])
@@ -381,7 +377,7 @@ func TestHCLTypedVariables(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("higher"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("no"), c.Targets[0].Args["v2"])
@@ -479,7 +475,7 @@ func TestHCLAttrs(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// env does not apply if no variable
@@ -488,7 +484,7 @@ func TestHCLAttrs(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// attr-multifile
}
@@ -596,126 +592,11 @@ func TestHCLAttrsCustomType(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, []string{"linux/arm64", "linux/amd64"}, c.Targets[0].Platforms)
require.Equal(t, ptrstr("linux/arm64"), c.Targets[0].Args["v1"])
}
func TestHCLAttrsCapsuleType(t *testing.T) {
dt := []byte(`
target "app" {
attest = [
{ type = "provenance", mode = "max" },
"type=sbom,disabled=true",
]
cache-from = [
{ type = "registry", ref = "user/app:cache" },
"type=local,src=path/to/cache",
]
cache-to = [
{ type = "local", dest = "path/to/cache" },
]
output = [
{ type = "oci", dest = "../out.tar" },
"type=local,dest=../out",
]
secret = [
{ id = "mysecret", src = "/local/secret" },
{ id = "mysecret2", env = "TOKEN" },
]
ssh = [
{ id = "default" },
{ id = "key", paths = ["path/to/key"] },
]
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"type=provenance,mode=max", "type=sbom,disabled=true"}, stringify(c.Targets[0].Attest))
require.Equal(t, []string{"type=local,dest=../out", "type=oci,dest=../out.tar"}, stringify(c.Targets[0].Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(c.Targets[0].CacheFrom))
require.Equal(t, []string{"type=local,dest=path/to/cache"}, stringify(c.Targets[0].CacheTo))
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,env=TOKEN"}, stringify(c.Targets[0].Secrets))
require.Equal(t, []string{"default", "key=path/to/key"}, stringify(c.Targets[0].SSH))
}
func TestHCLAttrsCapsuleTypeVars(t *testing.T) {
dt := []byte(`
variable "foo" {
default = "bar"
}
target "app" {
cache-from = [
{ type = "registry", ref = "user/app:cache" },
"type=local,src=path/to/cache",
]
cache-to = [ target.app.cache-from[0] ]
output = [
{ type = "oci", dest = "../out.tar" },
"type=local,dest=../out",
]
secret = [
{ id = "mysecret", src = "/local/secret" },
]
ssh = [
{ id = "default" },
{ id = "key", paths = ["path/to/${target.app.output[0].type}"] },
]
}
target "web" {
cache-from = target.app.cache-from
output = [ "type=oci,dest=../${foo}.tar" ]
secret = [
{ id = target.app.output[0].type, src = "/${target.app.cache-from[1].type}/secret" },
]
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
findTarget := func(t *testing.T, name string) *Target {
t.Helper()
for _, tgt := range c.Targets {
if tgt.Name == name {
return tgt
}
}
t.Fatalf("could not find target %q", name)
return nil
}
app := findTarget(t, "app")
require.Equal(t, []string{"type=local,dest=../out", "type=oci,dest=../out.tar"}, stringify(app.Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(app.CacheFrom))
require.Equal(t, []string{"user/app:cache"}, stringify(app.CacheTo))
require.Equal(t, []string{"id=mysecret,src=/local/secret"}, stringify(app.Secrets))
require.Equal(t, []string{"default", "key=path/to/oci"}, stringify(app.SSH))
web := findTarget(t, "web")
require.Equal(t, []string{"type=oci,dest=../bar.tar"}, stringify(web.Outputs))
require.Equal(t, []string{"type=local,src=path/to/cache", "user/app:cache"}, stringify(web.CacheFrom))
require.Equal(t, []string{"id=oci,src=/local/secret"}, stringify(web.Secrets))
}
func TestHCLMultiFileAttrs(t *testing.T) {
dt := []byte(`
variable "FOO" {
@@ -737,7 +618,7 @@ func TestHCLMultiFileAttrs(t *testing.T) {
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
t.Setenv("FOO", "ghi")
@@ -749,7 +630,7 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"])
}
@@ -772,7 +653,7 @@ func TestHCLMultiFileGlobalAttrs(t *testing.T) {
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", *c.Targets[0].Args["v1"])
}
@@ -958,12 +839,12 @@ func TestHCLRenameMultiFile(t *testing.T) {
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "bar", c.Targets[0].Name)
require.Equal(t, "x", *c.Targets[0].Dockerfile)
require.Equal(t, "z", *c.Targets[0].Target)
require.Equal(t, c.Targets[0].Name, "bar")
require.Equal(t, *c.Targets[0].Dockerfile, "x")
require.Equal(t, *c.Targets[0].Target, "z")
require.Equal(t, "foo", c.Targets[1].Name)
require.Equal(t, "y", *c.Targets[1].Context)
require.Equal(t, c.Targets[1].Name, "foo")
require.Equal(t, *c.Targets[1].Context, "y")
}
func TestHCLMatrixBasic(t *testing.T) {
@@ -981,10 +862,10 @@ func TestHCLMatrixBasic(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "x", c.Targets[0].Name)
require.Equal(t, "y", c.Targets[1].Name)
require.Equal(t, "x.Dockerfile", *c.Targets[0].Dockerfile)
require.Equal(t, "y.Dockerfile", *c.Targets[1].Dockerfile)
require.Equal(t, c.Targets[0].Name, "x")
require.Equal(t, c.Targets[1].Name, "y")
require.Equal(t, *c.Targets[0].Dockerfile, "x.Dockerfile")
require.Equal(t, *c.Targets[1].Dockerfile, "y.Dockerfile")
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
@@ -1067,9 +948,9 @@ func TestHCLMatrixMaps(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "aa", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "aa")
require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb"))
require.Equal(t, "cc", c.Targets[1].Name)
require.Equal(t, c.Targets[1].Name, "cc")
require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd"))
}
@@ -1260,7 +1141,7 @@ func TestJSONAttributes(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-abc-def"), c.Targets[0].Args["v1"])
}
@@ -1285,7 +1166,7 @@ func TestJSONFunctions(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
}
@@ -1303,7 +1184,7 @@ func TestJSONInvalidFunctions(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
}
@@ -1331,7 +1212,7 @@ func TestHCLFunctionInAttr(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("FOO <> [baz]"), c.Targets[0].Args["v1"])
}
@@ -1362,7 +1243,7 @@ services:
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr("foo"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["v2"])
require.Equal(t, "dir", *c.Targets[0].Context)
@@ -1385,7 +1266,7 @@ func TestHCLBuiltinVars(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "foo", *c.Targets[0].Context)
require.Equal(t, "test", *c.Targets[0].Dockerfile)
}
@@ -1451,17 +1332,17 @@ target "b" {
require.Equal(t, 4, len(c.Targets))
require.Equal(t, "metadata-a", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "metadata-a")
require.Equal(t, []string{"app/a:1.0.0", "app/a:latest"}, c.Targets[0].Tags)
require.Equal(t, "metadata-b", c.Targets[1].Name)
require.Equal(t, c.Targets[1].Name, "metadata-b")
require.Equal(t, []string{"app/b:1.0.0", "app/b:latest"}, c.Targets[1].Tags)
require.Equal(t, "a", c.Targets[2].Name)
require.Equal(t, c.Targets[2].Name, "a")
require.Equal(t, ".", *c.Targets[2].Context)
require.Equal(t, "a", *c.Targets[2].Target)
require.Equal(t, "b", c.Targets[3].Name)
require.Equal(t, c.Targets[3].Name, "b")
require.Equal(t, ".", *c.Targets[3].Context)
require.Equal(t, "b", *c.Targets[3].Target)
}
@@ -1508,10 +1389,10 @@ target "two" {
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "one", c.Targets[0].Name)
require.Equal(t, c.Targets[0].Name, "one")
require.Equal(t, map[string]*string{"a": ptrstr("pre-ghi-jkl")}, c.Targets[0].Args)
require.Equal(t, "two", c.Targets[1].Name)
require.Equal(t, c.Targets[1].Name, "two")
require.Equal(t, map[string]*string{"b": ptrstr("pre-jkl")}, c.Targets[1].Args)
}

View File

@@ -1,355 +0,0 @@
Copyright (c) 2014 HashiCorp, Inc.
Mozilla Public License, version 2.0
1. Definitions
1.1. “Contributor”
means each individual or legal entity that creates, contributes to the
creation of, or owns Covered Software.
1.2. “Contributor Version”
means the combination of the Contributions of others (if any) used by a
Contributor and that particular Contributors Contribution.
1.3. “Contribution”
means Covered Software of a particular Contributor.
1.4. “Covered Software”
means Source Code Form to which the initial Contributor has attached the
notice in Exhibit A, the Executable Form of such Source Code Form, and
Modifications of such Source Code Form, in each case including portions
thereof.
1.5. “Incompatible With Secondary Licenses”
means
a. that the initial Contributor has attached the notice described in
Exhibit B to the Covered Software; or
b. that the Covered Software was made available under the terms of version
1.1 or earlier of the License, but not also under the terms of a
Secondary License.
1.6. “Executable Form”
means any form of the work other than Source Code Form.
1.7. “Larger Work”
means a work that combines Covered Software with other material, in a separate
file or files, that is not Covered Software.
1.8. “License”
means this document.
1.9. “Licensable”
means having the right to grant, to the maximum extent possible, whether at the
time of the initial grant or subsequently, any and all of the rights conveyed by
this License.
1.10. “Modifications”
means any of the following:
a. any file in Source Code Form that results from an addition to, deletion
from, or modification of the contents of Covered Software; or
b. any new file in Source Code Form that contains any Covered Software.
1.11. “Patent Claims” of a Contributor
means any patent claim(s), including without limitation, method, process,
and apparatus claims, in any patent Licensable by such Contributor that
would be infringed, but for the grant of the License, by the making,
using, selling, offering for sale, having made, import, or transfer of
either its Contributions or its Contributor Version.
1.12. “Secondary License”
means either the GNU General Public License, Version 2.0, the GNU Lesser
General Public License, Version 2.1, the GNU Affero General Public
License, Version 3.0, or any later versions of those licenses.
1.13. “Source Code Form”
means the form of the work preferred for making modifications.
1.14. “You” (or “Your”)
means an individual or a legal entity exercising rights under this
License. For legal entities, “You” includes any entity that controls, is
controlled by, or is under common control with You. For purposes of this
definition, “control” means (a) the power, direct or indirect, to cause
the direction or management of such entity, whether by contract or
otherwise, or (b) ownership of more than fifty percent (50%) of the
outstanding shares or beneficial ownership of such entity.
2. License Grants and Conditions
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
a. under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or as
part of a Larger Work; and
b. under Patent Claims of such Contributor to make, use, sell, offer for
sale, have made, import, and otherwise transfer either its Contributions
or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution become
effective for each Contribution on the date the Contributor first distributes
such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under this
License. No additional rights or licenses will be implied from the distribution
or licensing of Covered Software under this License. Notwithstanding Section
2.1(b) above, no patent license is granted by a Contributor:
a. for any code that a Contributor has removed from Covered Software; or
b. for infringements caused by: (i) Your and any other third partys
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
c. under Patent Claims infringed by Covered Software in the absence of its
Contributions.
This License does not grant any rights in the trademarks, service marks, or
logos of any Contributor (except as may be necessary to comply with the
notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this License
(see Section 10.2) or under the terms of a Secondary License (if permitted
under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its Contributions
are its original creation(s) or it has sufficient rights to grant the
rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under applicable
copyright doctrines of fair use, fair dealing, or other equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted in
Section 2.1.
3. Responsibilities
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under the
terms of this License. You must inform recipients that the Source Code Form
of the Covered Software is governed by the terms of this License, and how
they can obtain a copy of this License. You may not attempt to alter or
restrict the recipients rights in the Source Code Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
a. such Covered Software must also be made available in Source Code Form,
as described in Section 3.1, and You must inform recipients of the
Executable Form how they can obtain a copy of such Source Code Form by
reasonable means in a timely manner, at a charge no more than the cost
of distribution to the recipient; and
b. You may distribute such Executable Form under the terms of this License,
or sublicense it under different terms, provided that the license for
the Executable Form does not attempt to limit or alter the recipients
rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for the
Covered Software. If the Larger Work is a combination of Covered Software
with a work governed by one or more Secondary Licenses, and the Covered
Software is not Incompatible With Secondary Licenses, this License permits
You to additionally distribute such Covered Software under the terms of
such Secondary License(s), so that the recipient of the Larger Work may, at
their option, further distribute the Covered Software under the terms of
either this License or such Secondary License(s).
3.4. Notices
You may not remove or alter the substance of any license notices (including
copyright notices, patent notices, disclaimers of warranty, or limitations
of liability) contained within the Source Code Form of the Covered
Software, except that You may alter any license notices to the extent
required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on behalf
of any Contributor. You must make it absolutely clear that any such
warranty, support, indemnity, or liability obligation is offered by You
alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
If it is impossible for You to comply with any of the terms of this License
with respect to some or all of the Covered Software due to statute, judicial
order, or regulation then You must: (a) comply with the terms of this License
to the maximum extent possible; and (b) describe the limitations and the code
they affect. Such description must be placed in a text file included with all
distributions of the Covered Software under this License. Except to the
extent prohibited by statute or regulation, such description must be
sufficiently detailed for a recipient of ordinary skill to be able to
understand it.
5. Termination
5.1. The rights granted under this License will terminate automatically if You
fail to comply with any of its terms. However, if You become compliant,
then the rights granted under this License from a particular Contributor
are reinstated (a) provisionally, unless and until such Contributor
explicitly and finally terminates Your grants, and (b) on an ongoing basis,
if such Contributor fails to notify You of the non-compliance by some
reasonable means prior to 60 days after You have come back into compliance.
Moreover, Your grants from a particular Contributor are reinstated on an
ongoing basis if such Contributor notifies You of the non-compliance by
some reasonable means, this is the first time You have received notice of
non-compliance with this License from such Contributor, and You become
compliant prior to 30 days after Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions, counter-claims,
and cross-claims) alleging that a Contributor Version directly or
indirectly infringes any patent, then the rights granted to You by any and
all Contributors for the Covered Software under Section 2.1 of this License
shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all end user
license agreements (excluding distributors and resellers) which have been
validly granted by You or Your distributors under this License prior to
termination shall survive termination.
6. Disclaimer of Warranty
Covered Software is provided under this License on an “as is” basis, without
warranty of any kind, either expressed, implied, or statutory, including,
without limitation, warranties that the Covered Software is free of defects,
merchantable, fit for a particular purpose or non-infringing. The entire
risk as to the quality and performance of the Covered Software is with You.
Should any Covered Software prove defective in any respect, You (not any
Contributor) assume the cost of any necessary servicing, repair, or
correction. This disclaimer of warranty constitutes an essential part of this
License. No use of any Covered Software is authorized under this License
except under this disclaimer.
7. Limitation of Liability
Under no circumstances and under no legal theory, whether tort (including
negligence), contract, or otherwise, shall any Contributor, or anyone who
distributes Covered Software as permitted above, be liable to You for any
direct, indirect, special, incidental, or consequential damages of any
character including, without limitation, damages for lost profits, loss of
goodwill, work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses, even if such party shall have been
informed of the possibility of such damages. This limitation of liability
shall not apply to liability for death or personal injury resulting from such
partys negligence to the extent applicable law prohibits such limitation.
Some jurisdictions do not allow the exclusion or limitation of incidental or
consequential damages, so this exclusion and limitation may not apply to You.
8. Litigation
Any litigation relating to this License may be brought only in the courts of
a jurisdiction where the defendant maintains its principal place of business
and such litigation shall be governed by laws of that jurisdiction, without
reference to its conflict-of-law provisions. Nothing in this Section shall
prevent a partys ability to bring cross-claims or counter-claims.
9. Miscellaneous
This License represents the complete agreement concerning the subject matter
hereof. If any provision of this License is held to be unenforceable, such
provision shall be reformed only to the extent necessary to make it
enforceable. Any law or regulation which provides that the language of a
contract shall be construed against the drafter shall not be used to construe
this License against a Contributor.
10. Versions of the License
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version of
the License under which You originally received the Covered Software, or
under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a modified
version of this License if you rename the license and remove any
references to the name of the license steward (except to note that such
modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
This Source Code Form is subject to the
terms of the Mozilla Public License, v.
2.0. If a copy of the MPL was not
distributed with this file, You can
obtain one at
http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular file, then
You may include the notice in a location (such as a LICENSE file in a relevant
directory) where a recipient would be likely to look for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - “Incompatible With Secondary Licenses” Notice
This Source Code Form is “Incompatible
With Secondary Licenses”, as defined by
the Mozilla Public License, v. 2.0.

View File

@@ -1,348 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"github.com/hashicorp/hcl/v2"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/gocty"
)
// DecodeOptions allows customizing sections of the decoding process.
type DecodeOptions struct {
ImpliedType func(gv interface{}) (cty.Type, error)
Convert func(in cty.Value, want cty.Type) (cty.Value, error)
}
func (o DecodeOptions) DecodeBody(body hcl.Body, ctx *hcl.EvalContext, val interface{}) hcl.Diagnostics {
o = o.withDefaults()
rv := reflect.ValueOf(val)
if rv.Kind() != reflect.Ptr {
panic(fmt.Sprintf("target value must be a pointer, not %s", rv.Type().String()))
}
return o.decodeBodyToValue(body, ctx, rv.Elem())
}
// DecodeBody extracts the configuration within the given body into the given
// value. This value must be a non-nil pointer to either a struct or
// a map, where in the former case the configuration will be decoded using
// struct tags and in the latter case only attributes are allowed and their
// values are decoded into the map.
//
// The given EvalContext is used to resolve any variables or functions in
// expressions encountered while decoding. This may be nil to require only
// constant values, for simple applications that do not support variables or
// functions.
//
// The returned diagnostics should be inspected with its HasErrors method to
// determine if the populated value is valid and complete. If error diagnostics
// are returned then the given value may have been partially-populated but
// may still be accessed by a careful caller for static analysis and editor
// integration use-cases.
func DecodeBody(body hcl.Body, ctx *hcl.EvalContext, val interface{}) hcl.Diagnostics {
return DecodeOptions{}.DecodeBody(body, ctx, val)
}
func (o DecodeOptions) decodeBodyToValue(body hcl.Body, ctx *hcl.EvalContext, val reflect.Value) hcl.Diagnostics {
et := val.Type()
switch et.Kind() {
case reflect.Struct:
return o.decodeBodyToStruct(body, ctx, val)
case reflect.Map:
return o.decodeBodyToMap(body, ctx, val)
default:
panic(fmt.Sprintf("target value must be pointer to struct or map, not %s", et.String()))
}
}
func (o DecodeOptions) decodeBodyToStruct(body hcl.Body, ctx *hcl.EvalContext, val reflect.Value) hcl.Diagnostics {
schema, partial := ImpliedBodySchema(val.Interface())
var content *hcl.BodyContent
var leftovers hcl.Body
var diags hcl.Diagnostics
if partial {
content, leftovers, diags = body.PartialContent(schema)
} else {
content, diags = body.Content(schema)
}
if content == nil {
return diags
}
tags := getFieldTags(val.Type())
if tags.Body != nil {
fieldIdx := *tags.Body
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
switch {
case bodyType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(body))
default:
diags = append(diags, o.decodeBodyToValue(body, ctx, fieldV)...)
}
}
if tags.Remain != nil {
fieldIdx := *tags.Remain
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
switch {
case bodyType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(leftovers))
case attrsType.AssignableTo(field.Type):
attrs, attrsDiags := leftovers.JustAttributes()
if len(attrsDiags) > 0 {
diags = append(diags, attrsDiags...)
}
fieldV.Set(reflect.ValueOf(attrs))
default:
diags = append(diags, o.decodeBodyToValue(leftovers, ctx, fieldV)...)
}
}
for name, fieldIdx := range tags.Attributes {
attr := content.Attributes[name]
field := val.Type().Field(fieldIdx)
fieldV := val.Field(fieldIdx)
if attr == nil {
if !exprType.AssignableTo(field.Type) {
continue
}
// As a special case, if the target is of type hcl.Expression then
// we'll assign an actual expression that evalues to a cty null,
// so the caller can deal with it within the cty realm rather
// than within the Go realm.
synthExpr := hcl.StaticExpr(cty.NullVal(cty.DynamicPseudoType), body.MissingItemRange())
fieldV.Set(reflect.ValueOf(synthExpr))
continue
}
switch {
case attrType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(attr))
case exprType.AssignableTo(field.Type):
fieldV.Set(reflect.ValueOf(attr.Expr))
default:
diags = append(diags, o.DecodeExpression(
attr.Expr, ctx, fieldV.Addr().Interface(),
)...)
}
}
blocksByType := content.Blocks.ByType()
for typeName, fieldIdx := range tags.Blocks {
blocks := blocksByType[typeName]
field := val.Type().Field(fieldIdx)
ty := field.Type
isSlice := false
isPtr := false
if ty.Kind() == reflect.Slice {
isSlice = true
ty = ty.Elem()
}
if ty.Kind() == reflect.Ptr {
isPtr = true
ty = ty.Elem()
}
if len(blocks) > 1 && !isSlice {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Duplicate %s block", typeName),
Detail: fmt.Sprintf(
"Only one %s block is allowed. Another was defined at %s.",
typeName, blocks[0].DefRange.String(),
),
Subject: &blocks[1].DefRange,
})
continue
}
if len(blocks) == 0 {
if isSlice || isPtr {
if val.Field(fieldIdx).IsNil() {
val.Field(fieldIdx).Set(reflect.Zero(field.Type))
}
} else {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: fmt.Sprintf("Missing %s block", typeName),
Detail: fmt.Sprintf("A %s block is required.", typeName),
Subject: body.MissingItemRange().Ptr(),
})
}
continue
}
switch {
case isSlice:
elemType := ty
if isPtr {
elemType = reflect.PointerTo(ty)
}
sli := val.Field(fieldIdx)
if sli.IsNil() {
sli = reflect.MakeSlice(reflect.SliceOf(elemType), len(blocks), len(blocks))
}
for i, block := range blocks {
if isPtr {
if i >= sli.Len() {
sli = reflect.Append(sli, reflect.New(ty))
}
v := sli.Index(i)
if v.IsNil() {
v = reflect.New(ty)
}
diags = append(diags, o.decodeBlockToValue(block, ctx, v.Elem())...)
sli.Index(i).Set(v)
} else {
if i >= sli.Len() {
sli = reflect.Append(sli, reflect.Indirect(reflect.New(ty)))
}
diags = append(diags, o.decodeBlockToValue(block, ctx, sli.Index(i))...)
}
}
if sli.Len() > len(blocks) {
sli.SetLen(len(blocks))
}
val.Field(fieldIdx).Set(sli)
default:
block := blocks[0]
if isPtr {
v := val.Field(fieldIdx)
if v.IsNil() {
v = reflect.New(ty)
}
diags = append(diags, o.decodeBlockToValue(block, ctx, v.Elem())...)
val.Field(fieldIdx).Set(v)
} else {
diags = append(diags, o.decodeBlockToValue(block, ctx, val.Field(fieldIdx))...)
}
}
}
return diags
}
func (o DecodeOptions) decodeBodyToMap(body hcl.Body, ctx *hcl.EvalContext, v reflect.Value) hcl.Diagnostics {
attrs, diags := body.JustAttributes()
if attrs == nil {
return diags
}
mv := reflect.MakeMap(v.Type())
for k, attr := range attrs {
switch {
case attrType.AssignableTo(v.Type().Elem()):
mv.SetMapIndex(reflect.ValueOf(k), reflect.ValueOf(attr))
case exprType.AssignableTo(v.Type().Elem()):
mv.SetMapIndex(reflect.ValueOf(k), reflect.ValueOf(attr.Expr))
default:
ev := reflect.New(v.Type().Elem())
diags = append(diags, o.DecodeExpression(attr.Expr, ctx, ev.Interface())...)
mv.SetMapIndex(reflect.ValueOf(k), ev.Elem())
}
}
v.Set(mv)
return diags
}
func (o DecodeOptions) decodeBlockToValue(block *hcl.Block, ctx *hcl.EvalContext, v reflect.Value) hcl.Diagnostics {
diags := o.decodeBodyToValue(block.Body, ctx, v)
if len(block.Labels) > 0 {
blockTags := getFieldTags(v.Type())
for li, lv := range block.Labels {
lfieldIdx := blockTags.Labels[li].FieldIndex
v.Field(lfieldIdx).Set(reflect.ValueOf(lv))
}
}
return diags
}
func (o DecodeOptions) DecodeExpression(expr hcl.Expression, ctx *hcl.EvalContext, val interface{}) hcl.Diagnostics {
o = o.withDefaults()
srcVal, diags := expr.Value(ctx)
convTy, err := o.ImpliedType(val)
if err != nil {
panic(fmt.Sprintf("unsuitable DecodeExpression target: %s", err))
}
srcVal, err = o.Convert(srcVal, convTy)
if err != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Unsuitable value type",
Detail: fmt.Sprintf("Unsuitable value: %s", err.Error()),
Subject: expr.StartRange().Ptr(),
Context: expr.Range().Ptr(),
})
return diags
}
err = gocty.FromCtyValue(srcVal, val)
if err != nil {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Unsuitable value type",
Detail: fmt.Sprintf("Unsuitable value: %s", err.Error()),
Subject: expr.StartRange().Ptr(),
Context: expr.Range().Ptr(),
})
}
return diags
}
// DecodeExpression extracts the value of the given expression into the given
// value. This value must be something that gocty is able to decode into,
// since the final decoding is delegated to that package.
//
// The given EvalContext is used to resolve any variables or functions in
// expressions encountered while decoding. This may be nil to require only
// constant values, for simple applications that do not support variables or
// functions.
//
// The returned diagnostics should be inspected with its HasErrors method to
// determine if the populated value is valid and complete. If error diagnostics
// are returned then the given value may have been partially-populated but
// may still be accessed by a careful caller for static analysis and editor
// integration use-cases.
func DecodeExpression(expr hcl.Expression, ctx *hcl.EvalContext, val interface{}) hcl.Diagnostics {
return DecodeOptions{}.DecodeExpression(expr, ctx, val)
}
func (o DecodeOptions) withDefaults() DecodeOptions {
if o.ImpliedType == nil {
o.ImpliedType = gocty.ImpliedType
}
if o.Convert == nil {
o.Convert = convert.Convert
}
return o
}

View File

@@ -1,806 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"encoding/json"
"fmt"
"reflect"
"testing"
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/hcl/v2"
hclJSON "github.com/hashicorp/hcl/v2/json"
"github.com/zclconf/go-cty/cty"
)
func TestDecodeBody(t *testing.T) {
deepEquals := func(other interface{}) func(v interface{}) bool {
return func(v interface{}) bool {
return reflect.DeepEqual(v, other)
}
}
type withNameExpression struct {
Name hcl.Expression `hcl:"name"`
}
type withTwoAttributes struct {
A string `hcl:"a,optional"`
B string `hcl:"b,optional"`
}
type withNestedBlock struct {
Plain string `hcl:"plain,optional"`
Nested *withTwoAttributes `hcl:"nested,block"`
}
type withListofNestedBlocks struct {
Nested []*withTwoAttributes `hcl:"nested,block"`
}
type withListofNestedBlocksNoPointers struct {
Nested []withTwoAttributes `hcl:"nested,block"`
}
tests := []struct {
Body map[string]interface{}
Target func() interface{}
Check func(v interface{}) bool
DiagCount int
}{
{
map[string]interface{}{},
makeInstantiateType(struct{}{}),
deepEquals(struct{}{}),
0,
},
{
map[string]interface{}{},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{}),
1, // name is required
},
{
map[string]interface{}{},
makeInstantiateType(struct {
Name *string `hcl:"name"`
}{}),
deepEquals(struct {
Name *string `hcl:"name"`
}{}),
0,
}, // name nil
{
map[string]interface{}{},
makeInstantiateType(struct {
Name string `hcl:"name,optional"`
}{}),
deepEquals(struct {
Name string `hcl:"name,optional"`
}{}),
0,
}, // name optional
{
map[string]interface{}{},
makeInstantiateType(withNameExpression{}),
func(v interface{}) bool {
if v == nil {
return false
}
wne, valid := v.(withNameExpression)
if !valid {
return false
}
if wne.Name == nil {
return false
}
nameVal, _ := wne.Name.Value(nil)
return nameVal.IsNull()
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
},
makeInstantiateType(withNameExpression{}),
func(v interface{}) bool {
if v == nil {
return false
}
wne, valid := v.(withNameExpression)
if !valid {
return false
}
if wne.Name == nil {
return false
}
nameVal, _ := wne.Name.Value(nil)
return nameVal.Equals(cty.StringVal("Ermintrude")).True()
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{"Ermintrude"}),
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 23,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
}{"Ermintrude"}),
1, // Extraneous "age" property
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Attrs hcl.Attributes `hcl:",remain"`
}{}),
func(gotI interface{}) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Attrs hcl.Attributes `hcl:",remain"`
})
return got.Name == "Ermintrude" && len(got.Attrs) == 1 && got.Attrs["age"] != nil
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Remain hcl.Body `hcl:",remain"`
}{}),
func(gotI interface{}) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Remain hcl.Body `hcl:",remain"`
})
attrs, _ := got.Remain.JustAttributes()
return got.Name == "Ermintrude" && len(attrs) == 1 && attrs["age"] != nil
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"living": true,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Remain map[string]cty.Value `hcl:",remain"`
}{}),
deepEquals(struct {
Name string `hcl:"name"`
Remain map[string]cty.Value `hcl:",remain"`
}{
Name: "Ermintrude",
Remain: map[string]cty.Value{
"living": cty.True,
},
}),
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 50,
},
makeInstantiateType(struct {
Name string `hcl:"name"`
Body hcl.Body `hcl:",body"`
Remain hcl.Body `hcl:",remain"`
}{}),
func(gotI interface{}) bool {
got := gotI.(struct {
Name string `hcl:"name"`
Body hcl.Body `hcl:",body"`
Remain hcl.Body `hcl:",remain"`
})
attrs, _ := got.Body.JustAttributes()
return got.Name == "Ermintrude" && len(attrs) == 2 &&
attrs["name"] != nil && attrs["age"] != nil
},
0,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating no diagnostics is good enough for this one.
return true
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating no diagnostics is good enough for this one.
return true
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}, {}},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]interface{}{},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle != nil
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle != nil
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
return gotI.(struct {
Noodle *struct{} `hcl:"noodle,block"`
}).Noodle == nil
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}, {}},
},
makeInstantiateType(struct {
Noodle *struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// Generating one diagnostic is good enough for this one.
return true
},
1,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 0
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 1
},
0,
},
{
map[string]interface{}{
"noodle": []map[string]interface{}{{}, {}},
},
makeInstantiateType(struct {
Noodle []struct{} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodle := gotI.(struct {
Noodle []struct{} `hcl:"noodle,block"`
}).Noodle
return len(noodle) == 2
},
0,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
//nolint:misspell
// Generating two diagnostics is good enough for this one.
// (one for the missing noodle block and the other for
// the JSON serialization detecting the missing level of
// heirarchy for the label.)
return true
},
2,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{
"foo_foo": map[string]interface{}{},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodle := gotI.(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}).Noodle
return noodle.Name == "foo_foo"
},
0,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{
"foo_foo": map[string]interface{}{},
"bar_baz": map[string]interface{}{},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
// One diagnostic is enough for this one.
return true
},
1,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{
"foo_foo": map[string]interface{}{},
"bar_baz": map[string]interface{}{},
},
},
makeInstantiateType(struct {
Noodles []struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodles := gotI.(struct {
Noodles []struct {
Name string `hcl:"name,label"`
} `hcl:"noodle,block"`
}).Noodles
return len(noodles) == 2 && (noodles[0].Name == "foo_foo" || noodles[0].Name == "bar_baz") && (noodles[1].Name == "foo_foo" || noodles[1].Name == "bar_baz") && noodles[0].Name != noodles[1].Name
},
0,
},
{
map[string]interface{}{
"noodle": map[string]interface{}{
"foo_foo": map[string]interface{}{
"type": "rice",
},
},
},
makeInstantiateType(struct {
Noodle struct {
Name string `hcl:"name,label"`
Type string `hcl:"type"`
} `hcl:"noodle,block"`
}{}),
func(gotI interface{}) bool {
noodle := gotI.(struct {
Noodle struct {
Name string `hcl:"name,label"`
Type string `hcl:"type"`
} `hcl:"noodle,block"`
}).Noodle
return noodle.Name == "foo_foo" && noodle.Type == "rice"
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 34,
},
makeInstantiateType(map[string]string(nil)),
deepEquals(map[string]string{
"name": "Ermintrude",
"age": "34",
}),
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 89,
},
makeInstantiateType(map[string]*hcl.Attribute(nil)),
func(gotI interface{}) bool {
got := gotI.(map[string]*hcl.Attribute)
return len(got) == 2 && got["name"] != nil && got["age"] != nil
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"age": 13,
},
makeInstantiateType(map[string]hcl.Expression(nil)),
func(gotI interface{}) bool {
got := gotI.(map[string]hcl.Expression)
return len(got) == 2 && got["name"] != nil && got["age"] != nil
},
0,
},
{
map[string]interface{}{
"name": "Ermintrude",
"living": true,
},
makeInstantiateType(map[string]cty.Value(nil)),
deepEquals(map[string]cty.Value{
"name": cty.StringVal("Ermintrude"),
"living": cty.True,
}),
0,
},
{
// Retain "nested" block while decoding
map[string]interface{}{
"plain": "foo",
},
func() interface{} {
return &withNestedBlock{
Plain: "bar",
Nested: &withTwoAttributes{
A: "bar",
},
}
},
func(gotI interface{}) bool {
foo := gotI.(withNestedBlock)
return foo.Plain == "foo" && foo.Nested != nil && foo.Nested.A == "bar"
},
0,
},
{
// Retain values in "nested" block while decoding
map[string]interface{}{
"nested": map[string]interface{}{
"a": "foo",
},
},
func() interface{} {
return &withNestedBlock{
Nested: &withTwoAttributes{
B: "bar",
},
}
},
func(gotI interface{}) bool {
foo := gotI.(withNestedBlock)
return foo.Nested.A == "foo" && foo.Nested.B == "bar"
},
0,
},
{
// Retain values in "nested" block list while decoding
map[string]interface{}{
"nested": []map[string]interface{}{
{
"a": "foo",
},
},
},
func() interface{} {
return &withListofNestedBlocks{
Nested: []*withTwoAttributes{
{
B: "bar",
},
},
}
},
func(gotI interface{}) bool {
n := gotI.(withListofNestedBlocks)
return n.Nested[0].A == "foo" && n.Nested[0].B == "bar"
},
0,
},
{
// Remove additional elements from the list while decoding nested blocks
map[string]interface{}{
"nested": []map[string]interface{}{
{
"a": "foo",
},
},
},
func() interface{} {
return &withListofNestedBlocks{
Nested: []*withTwoAttributes{
{
B: "bar",
},
{
B: "bar",
},
},
}
},
func(gotI interface{}) bool {
n := gotI.(withListofNestedBlocks)
return len(n.Nested) == 1
},
0,
},
{
// Make sure decoding value slices works the same as pointer slices.
map[string]interface{}{
"nested": []map[string]interface{}{
{
"b": "bar",
},
{
"b": "baz",
},
},
},
func() interface{} {
return &withListofNestedBlocksNoPointers{
Nested: []withTwoAttributes{
{
B: "foo",
},
},
}
},
func(gotI interface{}) bool {
n := gotI.(withListofNestedBlocksNoPointers)
return n.Nested[0].B == "bar" && len(n.Nested) == 2
},
0,
},
}
for i, test := range tests {
// For convenience here we're going to use the JSON parser
// to process the given body.
buf, err := json.Marshal(test.Body)
if err != nil {
t.Fatalf("error JSON-encoding body for test %d: %s", i, err)
}
t.Run(string(buf), func(t *testing.T) {
file, diags := hclJSON.Parse(buf, "test.json")
if len(diags) != 0 {
t.Fatalf("diagnostics while parsing: %s", diags.Error())
}
targetVal := reflect.ValueOf(test.Target())
diags = DecodeBody(file.Body, nil, targetVal.Interface())
if len(diags) != test.DiagCount {
t.Errorf("wrong number of diagnostics %d; want %d", len(diags), test.DiagCount)
for _, diag := range diags {
t.Logf(" - %s", diag.Error())
}
}
got := targetVal.Elem().Interface()
if !test.Check(got) {
t.Errorf("wrong result\ngot: %s", spew.Sdump(got))
}
})
}
}
func TestDecodeExpression(t *testing.T) {
tests := []struct {
Value cty.Value
Target interface{}
Want interface{}
DiagCount int
}{
{
cty.StringVal("hello"),
"",
"hello",
0,
},
{
cty.StringVal("hello"),
cty.NilVal,
cty.StringVal("hello"),
0,
},
{
cty.NumberIntVal(2),
"",
"2",
0,
},
{
cty.StringVal("true"),
false,
true,
0,
},
{
cty.NullVal(cty.String),
"",
"",
1, // null value is not allowed
},
{
cty.UnknownVal(cty.String),
"",
"",
1, // value must be known
},
{
cty.ListVal([]cty.Value{cty.True}),
false,
false,
1, // bool required
},
}
for i, test := range tests {
t.Run(fmt.Sprintf("%02d", i), func(t *testing.T) {
expr := &fixedExpression{test.Value}
targetVal := reflect.New(reflect.TypeOf(test.Target))
diags := DecodeExpression(expr, nil, targetVal.Interface())
if len(diags) != test.DiagCount {
t.Errorf("wrong number of diagnostics %d; want %d", len(diags), test.DiagCount)
for _, diag := range diags {
t.Logf(" - %s", diag.Error())
}
}
got := targetVal.Elem().Interface()
if !reflect.DeepEqual(got, test.Want) {
t.Errorf("wrong result\ngot: %#v\nwant: %#v", got, test.Want)
}
})
}
}
type fixedExpression struct {
val cty.Value
}
func (e *fixedExpression) Value(ctx *hcl.EvalContext) (cty.Value, hcl.Diagnostics) {
return e.val, nil
}
func (e *fixedExpression) Range() (r hcl.Range) {
return
}
func (e *fixedExpression) StartRange() (r hcl.Range) {
return
}
func (e *fixedExpression) Variables() []hcl.Traversal {
return nil
}
func makeInstantiateType(target interface{}) func() interface{} {
return func() interface{} {
return reflect.New(reflect.TypeOf(target)).Interface()
}
}

View File

@@ -1,65 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
// Package gohcl allows decoding HCL configurations into Go data structures.
//
// It provides a convenient and concise way of describing the schema for
// configuration and then accessing the resulting data via native Go
// types.
//
// A struct field tag scheme is used, similar to other decoding and
// unmarshalling libraries. The tags are formatted as in the following example:
//
// ThingType string `hcl:"thing_type,attr"`
//
// Within each tag there are two comma-separated tokens. The first is the
// name of the corresponding construct in configuration, while the second
// is a keyword giving the kind of construct expected. The following
// kind keywords are supported:
//
// attr (the default) indicates that the value is to be populated from an attribute
// block indicates that the value is to populated from a block
// label indicates that the value is to populated from a block label
// optional is the same as attr, but the field is optional
// remain indicates that the value is to be populated from the remaining body after populating other fields
//
// "attr" fields may either be of type *hcl.Expression, in which case the raw
// expression is assigned, or of any type accepted by gocty, in which case
// gocty will be used to assign the value to a native Go type.
//
// "block" fields may be a struct that recursively uses the same tags, or a
// slice of such structs, in which case multiple blocks of the corresponding
// type are decoded into the slice.
//
// "body" can be placed on a single field of type hcl.Body to capture
// the full hcl.Body that was decoded for a block. This does not allow leftover
// values like "remain", so a decoding error will still be returned if leftover
// fields are given. If you want to capture the decoding body PLUS leftover
// fields, you must specify a "remain" field as well to prevent errors. The
// body field and the remain field will both contain the leftover fields.
//
// "label" fields are considered only in a struct used as the type of a field
// marked as "block", and are used sequentially to capture the labels of
// the blocks being decoded. In this case, the name token is used only as
// an identifier for the label in diagnostic messages.
//
// "optional" fields behave like "attr" fields, but they are optional
// and will not give parsing errors if they are missing.
//
// "remain" can be placed on a single field that may be either of type
// hcl.Body or hcl.Attributes, in which case any remaining body content is
// placed into this field for delayed processing. If no "remain" field is
// present then any attributes or blocks not matched by another valid tag
// will cause an error diagnostic.
//
// Only a subset of this tagging/typing vocabulary is supported for the
// "Encode" family of functions. See the EncodeIntoBody docs for full details
// on the constraints there.
//
// Broadly-speaking this package deals with two types of error. The first is
// errors in the configuration itself, which are returned as diagnostics
// written with the configuration author as the target audience. The second
// is bugs in the calling program, such as invalid struct tags, which are
// surfaced via panics since there can be no useful runtime handling of such
// errors and they should certainly not be returned to the user as diagnostics.
package gohcl

View File

@@ -1,192 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"sort"
"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/zclconf/go-cty/cty/gocty"
)
// EncodeIntoBody replaces the contents of the given hclwrite Body with
// attributes and blocks derived from the given value, which must be a
// struct value or a pointer to a struct value with the struct tags defined
// in this package.
//
// This function can work only with fully-decoded data. It will ignore any
// fields tagged as "remain", any fields that decode attributes into either
// hcl.Attribute or hcl.Expression values, and any fields that decode blocks
// into hcl.Attributes values. This function does not have enough information
// to complete the decoding of these types.
//
// Any fields tagged as "label" are ignored by this function. Use EncodeAsBlock
// to produce a whole hclwrite.Block including block labels.
//
// As long as a suitable value is given to encode and the destination body
// is non-nil, this function will always complete. It will panic in case of
// any errors in the calling program, such as passing an inappropriate type
// or a nil body.
//
// The layout of the resulting HCL source is derived from the ordering of
// the struct fields, with blank lines around nested blocks of different types.
// Fields representing attributes should usually precede those representing
// blocks so that the attributes can group togather in the result. For more
// control, use the hclwrite API directly.
func EncodeIntoBody(val interface{}, dst *hclwrite.Body) {
rv := reflect.ValueOf(val)
ty := rv.Type()
if ty.Kind() == reflect.Ptr {
rv = rv.Elem()
ty = rv.Type()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("value is %s, not struct", ty.Kind()))
}
tags := getFieldTags(ty)
populateBody(rv, ty, tags, dst)
}
// EncodeAsBlock creates a new hclwrite.Block populated with the data from
// the given value, which must be a struct or pointer to struct with the
// struct tags defined in this package.
//
// If the given struct type has fields tagged with "label" tags then they
// will be used in order to annotate the created block with labels.
//
// This function has the same constraints as EncodeIntoBody and will panic
// if they are violated.
func EncodeAsBlock(val interface{}, blockType string) *hclwrite.Block {
rv := reflect.ValueOf(val)
ty := rv.Type()
if ty.Kind() == reflect.Ptr {
rv = rv.Elem()
ty = rv.Type()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("value is %s, not struct", ty.Kind()))
}
tags := getFieldTags(ty)
labels := make([]string, len(tags.Labels))
for i, lf := range tags.Labels {
lv := rv.Field(lf.FieldIndex)
// We just stringify whatever we find. It should always be a string
// but if not then we'll still do something reasonable.
labels[i] = fmt.Sprintf("%s", lv.Interface())
}
block := hclwrite.NewBlock(blockType, labels)
populateBody(rv, ty, tags, block.Body())
return block
}
func populateBody(rv reflect.Value, ty reflect.Type, tags *fieldTags, dst *hclwrite.Body) {
nameIdxs := make(map[string]int, len(tags.Attributes)+len(tags.Blocks))
namesOrder := make([]string, 0, len(tags.Attributes)+len(tags.Blocks))
for n, i := range tags.Attributes {
nameIdxs[n] = i
namesOrder = append(namesOrder, n)
}
for n, i := range tags.Blocks {
nameIdxs[n] = i
namesOrder = append(namesOrder, n)
}
sort.SliceStable(namesOrder, func(i, j int) bool {
ni, nj := namesOrder[i], namesOrder[j]
return nameIdxs[ni] < nameIdxs[nj]
})
dst.Clear()
prevWasBlock := false
for _, name := range namesOrder {
fieldIdx := nameIdxs[name]
field := ty.Field(fieldIdx)
fieldTy := field.Type
fieldVal := rv.Field(fieldIdx)
if fieldTy.Kind() == reflect.Ptr {
fieldTy = fieldTy.Elem()
fieldVal = fieldVal.Elem()
}
if _, isAttr := tags.Attributes[name]; isAttr {
if exprType.AssignableTo(fieldTy) || attrType.AssignableTo(fieldTy) {
continue // ignore undecoded fields
}
if !fieldVal.IsValid() {
continue // ignore (field value is nil pointer)
}
if fieldTy.Kind() == reflect.Ptr && fieldVal.IsNil() {
continue // ignore
}
if prevWasBlock {
dst.AppendNewline()
prevWasBlock = false
}
valTy, err := gocty.ImpliedType(fieldVal.Interface())
if err != nil {
panic(fmt.Sprintf("cannot encode %T as HCL expression: %s", fieldVal.Interface(), err))
}
val, err := gocty.ToCtyValue(fieldVal.Interface(), valTy)
if err != nil {
// This should never happen, since we should always be able
// to decode into the implied type.
panic(fmt.Sprintf("failed to encode %T as %#v: %s", fieldVal.Interface(), valTy, err))
}
dst.SetAttributeValue(name, val)
} else { // must be a block, then
elemTy := fieldTy
isSeq := false
if elemTy.Kind() == reflect.Slice || elemTy.Kind() == reflect.Array {
isSeq = true
elemTy = elemTy.Elem()
}
if bodyType.AssignableTo(elemTy) || attrsType.AssignableTo(elemTy) {
continue // ignore undecoded fields
}
prevWasBlock = false
if isSeq {
l := fieldVal.Len()
for i := 0; i < l; i++ {
elemVal := fieldVal.Index(i)
if !elemVal.IsValid() {
continue // ignore (elem value is nil pointer)
}
if elemTy.Kind() == reflect.Ptr && elemVal.IsNil() {
continue // ignore
}
block := EncodeAsBlock(elemVal.Interface(), name)
if !prevWasBlock {
dst.AppendNewline()
prevWasBlock = true
}
dst.AppendBlock(block)
}
} else {
if !fieldVal.IsValid() {
continue // ignore (field value is nil pointer)
}
if elemTy.Kind() == reflect.Ptr && fieldVal.IsNil() {
continue // ignore
}
block := EncodeAsBlock(fieldVal.Interface(), name)
if !prevWasBlock {
dst.AppendNewline()
prevWasBlock = true
}
dst.AppendBlock(block)
}
}
}
}

View File

@@ -1,67 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl_test
import (
"fmt"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclwrite"
)
func ExampleEncodeIntoBody() {
type Service struct {
Name string `hcl:"name,label"`
Exe []string `hcl:"executable"`
}
type Constraints struct {
OS string `hcl:"os"`
Arch string `hcl:"arch"`
}
type App struct {
Name string `hcl:"name"`
Desc string `hcl:"description"`
Constraints *Constraints `hcl:"constraints,block"`
Services []Service `hcl:"service,block"`
}
app := App{
Name: "awesome-app",
Desc: "Such an awesome application",
Constraints: &Constraints{
OS: "linux",
Arch: "amd64",
},
Services: []Service{
{
Name: "web",
Exe: []string{"./web", "--listen=:8080"},
},
{
Name: "worker",
Exe: []string{"./worker"},
},
},
}
f := hclwrite.NewEmptyFile()
gohcl.EncodeIntoBody(&app, f.Body())
fmt.Printf("%s", f.Bytes())
// Output:
// name = "awesome-app"
// description = "Such an awesome application"
//
// constraints {
// os = "linux"
// arch = "amd64"
// }
//
// service "web" {
// executable = ["./web", "--listen=:8080"]
// }
// service "worker" {
// executable = ["./worker"]
// }
}

View File

@@ -1,185 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"sort"
"strings"
"github.com/hashicorp/hcl/v2"
)
// ImpliedBodySchema produces a hcl.BodySchema derived from the type of the
// given value, which must be a struct value or a pointer to one. If an
// inappropriate value is passed, this function will panic.
//
// The second return argument indicates whether the given struct includes
// a "remain" field, and thus the returned schema is non-exhaustive.
//
// This uses the tags on the fields of the struct to discover how each
// field's value should be expressed within configuration. If an invalid
// mapping is attempted, this function will panic.
func ImpliedBodySchema(val interface{}) (schema *hcl.BodySchema, partial bool) {
ty := reflect.TypeOf(val)
if ty.Kind() == reflect.Ptr {
ty = ty.Elem()
}
if ty.Kind() != reflect.Struct {
panic(fmt.Sprintf("given value must be struct, not %T", val))
}
var attrSchemas []hcl.AttributeSchema
var blockSchemas []hcl.BlockHeaderSchema
tags := getFieldTags(ty)
attrNames := make([]string, 0, len(tags.Attributes))
for n := range tags.Attributes {
attrNames = append(attrNames, n)
}
sort.Strings(attrNames)
for _, n := range attrNames {
idx := tags.Attributes[n]
optional := tags.Optional[n]
field := ty.Field(idx)
var required bool
switch {
case field.Type.AssignableTo(exprType):
//nolint:misspell
// If we're decoding to hcl.Expression then absense can be
// indicated via a null value, so we don't specify that
// the field is required during decoding.
required = false
case field.Type.Kind() != reflect.Ptr && !optional:
required = true
default:
required = false
}
attrSchemas = append(attrSchemas, hcl.AttributeSchema{
Name: n,
Required: required,
})
}
blockNames := make([]string, 0, len(tags.Blocks))
for n := range tags.Blocks {
blockNames = append(blockNames, n)
}
sort.Strings(blockNames)
for _, n := range blockNames {
idx := tags.Blocks[n]
field := ty.Field(idx)
fty := field.Type
if fty.Kind() == reflect.Slice {
fty = fty.Elem()
}
if fty.Kind() == reflect.Ptr {
fty = fty.Elem()
}
if fty.Kind() != reflect.Struct {
panic(fmt.Sprintf(
"hcl 'block' tag kind cannot be applied to %s field %s: struct required", field.Type.String(), field.Name,
))
}
ftags := getFieldTags(fty)
var labelNames []string
if len(ftags.Labels) > 0 {
labelNames = make([]string, len(ftags.Labels))
for i, l := range ftags.Labels {
labelNames[i] = l.Name
}
}
blockSchemas = append(blockSchemas, hcl.BlockHeaderSchema{
Type: n,
LabelNames: labelNames,
})
}
partial = tags.Remain != nil
schema = &hcl.BodySchema{
Attributes: attrSchemas,
Blocks: blockSchemas,
}
return schema, partial
}
type fieldTags struct {
Attributes map[string]int
Blocks map[string]int
Labels []labelField
Remain *int
Body *int
Optional map[string]bool
}
type labelField struct {
FieldIndex int
Name string
}
func getFieldTags(ty reflect.Type) *fieldTags {
ret := &fieldTags{
Attributes: map[string]int{},
Blocks: map[string]int{},
Optional: map[string]bool{},
}
ct := ty.NumField()
for i := 0; i < ct; i++ {
field := ty.Field(i)
tag := field.Tag.Get("hcl")
if tag == "" {
continue
}
comma := strings.Index(tag, ",")
var name, kind string
if comma != -1 {
name = tag[:comma]
kind = tag[comma+1:]
} else {
name = tag
kind = "attr"
}
switch kind {
case "attr":
ret.Attributes[name] = i
case "block":
ret.Blocks[name] = i
case "label":
ret.Labels = append(ret.Labels, labelField{
FieldIndex: i,
Name: name,
})
case "remain":
if ret.Remain != nil {
panic("only one 'remain' tag is permitted")
}
idx := i // copy, because this loop will continue assigning to i
ret.Remain = &idx
case "body":
if ret.Body != nil {
panic("only one 'body' tag is permitted")
}
idx := i // copy, because this loop will continue assigning to i
ret.Body = &idx
case "optional":
ret.Attributes[name] = i
ret.Optional[name] = true
default:
panic(fmt.Sprintf("invalid hcl field tag kind %q on %s %q", kind, field.Type.String(), field.Name))
}
}
return ret
}

View File

@@ -1,233 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"fmt"
"reflect"
"testing"
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/hcl/v2"
)
func TestImpliedBodySchema(t *testing.T) {
tests := []struct {
val interface{}
wantSchema *hcl.BodySchema
wantPartial bool
}{
{
struct{}{},
&hcl.BodySchema{},
false,
},
{
struct {
Ignored bool
}{},
&hcl.BodySchema{},
false,
},
{
struct {
Attr1 bool `hcl:"attr1"`
Attr2 bool `hcl:"attr2"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "attr1",
Required: true,
},
{
Name: "attr2",
Required: true,
},
},
},
false,
},
{
struct {
Attr *bool `hcl:"attr,attr"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "attr",
Required: false,
},
},
},
false,
},
{
struct {
Thing struct{} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
},
},
},
false,
},
{
struct {
Thing struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing []struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing *struct {
Type string `hcl:"type,label"`
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"type", "name"},
},
},
},
false,
},
{
struct {
Thing struct {
Name string `hcl:"name,label"`
Something string `hcl:"something"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"name"},
},
},
},
false,
},
{
struct {
Doodad string `hcl:"doodad"`
Thing struct {
Name string `hcl:"name,label"`
} `hcl:"thing,block"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "doodad",
Required: true,
},
},
Blocks: []hcl.BlockHeaderSchema{
{
Type: "thing",
LabelNames: []string{"name"},
},
},
},
false,
},
{
struct {
Doodad string `hcl:"doodad"`
Config string `hcl:",remain"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "doodad",
Required: true,
},
},
},
true,
},
{
struct {
Expr hcl.Expression `hcl:"expr"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "expr",
Required: false,
},
},
},
false,
},
{
struct {
Meh string `hcl:"meh,optional"`
}{},
&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{
{
Name: "meh",
Required: false,
},
},
},
false,
},
}
for _, test := range tests {
t.Run(fmt.Sprintf("%#v", test.val), func(t *testing.T) {
schema, partial := ImpliedBodySchema(test.val)
if !reflect.DeepEqual(schema, test.wantSchema) {
t.Errorf(
"wrong schema\ngot: %s\nwant: %s",
spew.Sdump(schema), spew.Sdump(test.wantSchema),
)
}
if partial != test.wantPartial {
t.Errorf(
"wrong partial flag\ngot: %#v\nwant: %#v",
partial, test.wantPartial,
)
}
})
}
}

View File

@@ -1,19 +0,0 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
package gohcl
import (
"reflect"
"github.com/hashicorp/hcl/v2"
)
var victimExpr hcl.Expression
var victimBody hcl.Body
var exprType = reflect.TypeOf(&victimExpr).Elem()
var bodyType = reflect.TypeOf(&victimBody).Elem()
var blockType = reflect.TypeOf((*hcl.Block)(nil)) //nolint:unused
var attrType = reflect.TypeOf((*hcl.Attribute)(nil))
var attrsType = reflect.TypeOf(hcl.Attributes(nil))

View File

@@ -10,11 +10,12 @@ import (
"strconv"
"strings"
"github.com/docker/buildx/bake/hclparser/gohcl"
"github.com/docker/buildx/util/userfunc"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/gocty"
)
type Opt struct {
@@ -24,17 +25,11 @@ type Opt struct {
}
type variable struct {
Name string `json:"-" hcl:"name,label"`
Default *hcl.Attribute `json:"default,omitempty" hcl:"default,optional"`
Description string `json:"description,omitempty" hcl:"description,optional"`
Validations []*variableValidation `json:"validation,omitempty" hcl:"validation,block"`
Body hcl.Body `json:"-" hcl:",body"`
Remain hcl.Body `json:"-" hcl:",remain"`
}
type variableValidation struct {
Condition hcl.Expression `json:"condition" hcl:"condition"`
ErrorMessage hcl.Expression `json:"error_message" hcl:"error_message"`
Name string `json:"-" hcl:"name,label"`
Default *hcl.Attribute `json:"default,omitempty" hcl:"default,optional"`
Description string `json:"description,omitempty" hcl:"description,optional"`
Body hcl.Body `json:"-" hcl:",body"`
Remain hcl.Body `json:"-" hcl:",remain"`
}
type functionDef struct {
@@ -453,7 +448,7 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
}
// decode!
diag = decodeBody(body(), ectx, output.Interface())
diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() {
return diag
}
@@ -475,11 +470,11 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
}
// store the result into the evaluation context (so it can be referenced)
outputType, err := ImpliedType(output.Interface())
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
outputValue, err := ToCtyValue(output.Interface(), outputType)
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
@@ -491,12 +486,7 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
m = map[string]cty.Value{}
}
m[name] = outputValue
// The logical contents of this structure is similar to a map,
// but it's possible for some attributes to be different in a way that's
// illegal for a map so we use an object here instead which is structurally
// equivalent but allows disparate types for different keys.
p.ectx.Variables[block.Type] = cty.ObjectVal(m)
p.ectx.Variables[block.Type] = cty.MapVal(m)
}
return nil
@@ -551,37 +541,10 @@ func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
return names, nil
}
func (p *parser) validateVariables(vars map[string]*variable, ectx *hcl.EvalContext) hcl.Diagnostics {
var diags hcl.Diagnostics
for _, v := range vars {
for _, validation := range v.Validations {
condition, condDiags := validation.Condition.Value(ectx)
if condDiags.HasErrors() {
diags = append(diags, condDiags...)
continue
}
if !condition.True() {
message, msgDiags := validation.ErrorMessage.Value(ectx)
if msgDiags.HasErrors() {
diags = append(diags, msgDiags...)
continue
}
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Validation failed",
Detail: message.AsString(),
Subject: validation.Condition.Range().Ptr(),
})
}
}
}
return diags
}
type Variable struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Value *string `json:"value,omitempty"`
Name string
Description string
Value *string
}
type ParseMeta struct {
@@ -723,9 +686,6 @@ func Parse(b hcl.Body, opt Opt, val interface{}) (*ParseMeta, hcl.Diagnostics) {
}
vars = append(vars, v)
}
if diags := p.validateVariables(p.vars, p.ectx); diags.HasErrors() {
return nil, diags
}
for k := range p.funcs {
if err := p.resolveFunction(p.ectx, k); err != nil {
@@ -987,8 +947,3 @@ func key(ks ...any) uint64 {
}
return hash.Sum64()
}
func decodeBody(body hcl.Body, ctx *hcl.EvalContext, val interface{}) hcl.Diagnostics {
dec := gohcl.DecodeOptions{ImpliedType: ImpliedType}
return dec.DecodeBody(body, ctx, val)
}

View File

@@ -170,6 +170,7 @@ func indexOfFunc() function.Function {
}
}
return cty.NilVal, errors.New("item not found")
},
})
}

View File

@@ -1,160 +0,0 @@
// MIT License
//
// Copyright (c) 2017-2018 Martin Atkins
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package hclparser
import (
"reflect"
"github.com/zclconf/go-cty/cty"
)
// ImpliedType takes an arbitrary Go value (as an interface{}) and attempts
// to find a suitable cty.Type instance that could be used for a conversion
// with ToCtyValue.
//
// This allows -- for simple situations at least -- types to be defined just
// once in Go and the cty types derived from the Go types, but in the process
// it makes some assumptions that may be undesirable so applications are
// encouraged to build their cty types directly if exacting control is
// required.
//
// Not all Go types can be represented as cty types, so an error may be
// returned which is usually considered to be a bug in the calling program.
// In particular, ImpliedType will never use capsule types in its returned
// type, because it cannot know the capsule types supported by the calling
// program.
func ImpliedType(gv interface{}) (cty.Type, error) {
rt := reflect.TypeOf(gv)
var path cty.Path
return impliedType(rt, path)
}
func impliedType(rt reflect.Type, path cty.Path) (cty.Type, error) {
if ety, err := impliedTypeExt(rt, path); err == nil {
return ety, nil
}
switch rt.Kind() {
case reflect.Ptr:
return impliedType(rt.Elem(), path)
// Primitive types
case reflect.Bool:
return cty.Bool, nil
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
return cty.Number, nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
return cty.Number, nil
case reflect.Float32, reflect.Float64:
return cty.Number, nil
case reflect.String:
return cty.String, nil
// Collection types
case reflect.Slice:
path := append(path, cty.IndexStep{Key: cty.UnknownVal(cty.Number)})
ety, err := impliedType(rt.Elem(), path)
if err != nil {
return cty.NilType, err
}
return cty.List(ety), nil
case reflect.Map:
if !stringType.AssignableTo(rt.Key()) {
return cty.NilType, path.NewErrorf("no cty.Type for %s (must have string keys)", rt)
}
path := append(path, cty.IndexStep{Key: cty.UnknownVal(cty.String)})
ety, err := impliedType(rt.Elem(), path)
if err != nil {
return cty.NilType, err
}
return cty.Map(ety), nil
// Structural types
case reflect.Struct:
return impliedStructType(rt, path)
default:
return cty.NilType, path.NewErrorf("no cty.Type for %s", rt)
}
}
func impliedStructType(rt reflect.Type, path cty.Path) (cty.Type, error) {
if valueType.AssignableTo(rt) {
// Special case: cty.Value represents cty.DynamicPseudoType, for
// type conformance checking.
return cty.DynamicPseudoType, nil
}
fieldIdxs := structTagIndices(rt)
if len(fieldIdxs) == 0 {
return cty.NilType, path.NewErrorf("no cty.Type for %s (no cty field tags)", rt)
}
atys := make(map[string]cty.Type, len(fieldIdxs))
{
// Temporary extension of path for attributes
path := append(path, nil)
for k, fi := range fieldIdxs {
path[len(path)-1] = cty.GetAttrStep{Name: k}
ft := rt.Field(fi).Type
aty, err := impliedType(ft, path)
if err != nil {
return cty.NilType, err
}
atys[k] = aty
}
}
return cty.Object(atys), nil
}
var (
valueType = reflect.TypeOf(cty.Value{})
stringType = reflect.TypeOf("")
)
// structTagIndices interrogates the fields of the given type (which must
// be a struct type, or we'll panic) and returns a map from the cty
// attribute names declared via struct tags to the indices of the
// fields holding those tags.
//
// This function will panic if two fields within the struct are tagged with
// the same cty attribute name.
func structTagIndices(st reflect.Type) map[string]int {
ct := st.NumField()
ret := make(map[string]int, ct)
for i := 0; i < ct; i++ {
field := st.Field(i)
attrName := field.Tag.Get("cty")
if attrName != "" {
ret[attrName] = i
}
}
return ret
}

View File

@@ -1,166 +0,0 @@
package hclparser
import (
"reflect"
"sync"
"github.com/containerd/errdefs"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
"github.com/zclconf/go-cty/cty/gocty"
)
type ToCtyValueConverter interface {
// ToCtyValue will convert this capsule value into a native
// cty.Value. This should not return a capsule type.
ToCtyValue() cty.Value
}
type FromCtyValueConverter interface {
// FromCtyValue will initialize this value using a cty.Value.
FromCtyValue(in cty.Value, path cty.Path) error
}
type extensionType int
const (
unwrapCapsuleValueExtension extensionType = iota
)
func impliedTypeExt(rt reflect.Type, _ cty.Path) (cty.Type, error) {
if rt.Kind() != reflect.Pointer {
rt = reflect.PointerTo(rt)
}
if isCapsuleType(rt) {
return capsuleValueCapsuleType(rt), nil
}
return cty.NilType, errdefs.ErrNotImplemented
}
func isCapsuleType(rt reflect.Type) bool {
fromCtyValueType := reflect.TypeFor[FromCtyValueConverter]()
toCtyValueType := reflect.TypeFor[ToCtyValueConverter]()
return rt.Implements(fromCtyValueType) && rt.Implements(toCtyValueType)
}
var capsuleValueTypes sync.Map
func capsuleValueCapsuleType(rt reflect.Type) cty.Type {
if rt.Kind() != reflect.Pointer {
panic("capsule value must be a pointer")
}
elem := rt.Elem()
if val, loaded := capsuleValueTypes.Load(elem); loaded {
return val.(cty.Type)
}
toCtyValueType := reflect.TypeFor[ToCtyValueConverter]()
// First time used. Initialize new capsule ops.
ops := &cty.CapsuleOps{
ConversionTo: func(_ cty.Type) func(cty.Value, cty.Path) (any, error) {
return func(in cty.Value, p cty.Path) (any, error) {
rv := reflect.New(elem).Interface()
if err := rv.(FromCtyValueConverter).FromCtyValue(in, p); err != nil {
return nil, err
}
return rv, nil
}
},
ConversionFrom: func(want cty.Type) func(any, cty.Path) (cty.Value, error) {
return func(in any, _ cty.Path) (cty.Value, error) {
rv := reflect.ValueOf(in).Convert(toCtyValueType)
v := rv.Interface().(ToCtyValueConverter).ToCtyValue()
return convert.Convert(v, want)
}
},
ExtensionData: func(key any) any {
switch key {
case unwrapCapsuleValueExtension:
zero := reflect.Zero(elem).Interface()
if conv, ok := zero.(ToCtyValueConverter); ok {
return conv.ToCtyValue().Type()
}
zero = reflect.Zero(rt).Interface()
if conv, ok := zero.(ToCtyValueConverter); ok {
return conv.ToCtyValue().Type()
}
}
return nil
},
}
// Attempt to store the new type. Use whichever was loaded first in the case
// of a race condition.
ety := cty.CapsuleWithOps(elem.Name(), elem, ops)
val, _ := capsuleValueTypes.LoadOrStore(elem, ety)
return val.(cty.Type)
}
// UnwrapCtyValue will unwrap capsule type values into their native cty value
// equivalents if possible.
func UnwrapCtyValue(in cty.Value) cty.Value {
want := toCtyValueType(in.Type())
if in.Type().Equals(want) {
return in
} else if out, err := convert.Convert(in, want); err == nil {
return out
}
return cty.NullVal(want)
}
func toCtyValueType(in cty.Type) cty.Type {
if et := in.MapElementType(); et != nil {
return cty.Map(toCtyValueType(*et))
}
if et := in.SetElementType(); et != nil {
return cty.Set(toCtyValueType(*et))
}
if et := in.ListElementType(); et != nil {
return cty.List(toCtyValueType(*et))
}
if in.IsObjectType() {
var optional []string
inAttrTypes := in.AttributeTypes()
outAttrTypes := make(map[string]cty.Type, len(inAttrTypes))
for name, typ := range inAttrTypes {
outAttrTypes[name] = toCtyValueType(typ)
if in.AttributeOptional(name) {
optional = append(optional, name)
}
}
return cty.ObjectWithOptionalAttrs(outAttrTypes, optional)
}
if in.IsTupleType() {
inTypes := in.TupleElementTypes()
outTypes := make([]cty.Type, len(inTypes))
for i, typ := range inTypes {
outTypes[i] = toCtyValueType(typ)
}
return cty.Tuple(outTypes)
}
if in.IsCapsuleType() {
if out := in.CapsuleExtensionData(unwrapCapsuleValueExtension); out != nil {
return out.(cty.Type)
}
return cty.DynamicPseudoType
}
return in
}
func ToCtyValue(val any, ty cty.Type) (cty.Value, error) {
out, err := gocty.ToCtyValue(val, ty)
if err != nil {
return out, err
}
return UnwrapCtyValue(out), nil
}

View File

@@ -11,7 +11,6 @@ import (
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui"
@@ -20,8 +19,6 @@ import (
"github.com/pkg/errors"
)
const maxBakeDefinitionSize = 2 * 1024 * 1024 // 2 MB
type Input struct {
State *llb.State
URL string
@@ -109,6 +106,7 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
}
return nil, err
}, ch)
if err != nil {
return nil, nil, err
}
@@ -180,9 +178,9 @@ func filesFromURLRef(ctx context.Context, c gwclient.Client, ref gwclient.Refere
name := inp.URL
inp.URL = ""
if int64(len(dt)) > stat.Size {
if stat.Size > maxBakeDefinitionSize {
return nil, errors.Errorf("non-archive definition URL bigger than maximum allowed size (%s)", units.HumanSize(maxBakeDefinitionSize))
if len(dt) > stat.Size() {
if stat.Size() > 1024*512 {
return nil, errors.Errorf("non-archive definition URL bigger than maximum allowed size")
}
dt, err = ref.ReadFile(ctx, gwclient.ReadRequest{

View File

@@ -15,10 +15,9 @@ import (
"sync"
"time"
"github.com/containerd/containerd/v2/core/images"
"github.com/containerd/containerd/images"
"github.com/distribution/reference"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
@@ -51,39 +50,35 @@ import (
fstypes "github.com/tonistiigi/fsutil/types"
"go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup"
"google.golang.org/protobuf/proto"
)
const (
printFallbackImage = "docker/dockerfile:1.7.1@sha256:a57df69d0ea827fb7266491f2813635de6f17269be881f696fbfdf2d83dda33e"
printLintFallbackImage = "docker/dockerfile:1.8.1@sha256:e87caa74dcb7d46cd820352bfea12591f3dba3ddc4285e19c7dcd13359f7cefd"
printFallbackImage = "docker/dockerfile:1.5@sha256:dbbd5e059e8a07ff7ea6233b213b36aa516b4c53c645f1817a4dd18b83cbea56"
printLintFallbackImage = "docker.io/docker/dockerfile-upstream:1.8.1@sha256:e87caa74dcb7d46cd820352bfea12591f3dba3ddc4285e19c7dcd13359f7cefd"
)
type Options struct {
Inputs Inputs
Ref string
Allow []entitlements.Entitlement
Attests map[string]*string
BuildArgs map[string]string
CacheFrom []client.CacheOptionsEntry
CacheTo []client.CacheOptionsEntry
CgroupParent string
Exports []client.ExportEntry
ExportsLocalPathsTemporary []string // should be removed after client.ExportEntry update in buildkit v0.19.0
ExtraHosts []string
Labels map[string]string
NetworkMode string
NoCache bool
NoCacheFilter []string
Platforms []specs.Platform
Pull bool
SecretSpecs []*controllerapi.Secret
SSHSpecs []*controllerapi.SSH
ShmSize opts.MemBytes
Tags []string
Target string
Ulimits *opts.UlimitOpt
Ref string
Allow []entitlements.Entitlement
Attests map[string]*string
BuildArgs map[string]string
CacheFrom []client.CacheOptionsEntry
CacheTo []client.CacheOptionsEntry
CgroupParent string
Exports []client.ExportEntry
ExtraHosts []string
Labels map[string]string
NetworkMode string
NoCache bool
NoCacheFilter []string
Platforms []specs.Platform
Pull bool
ShmSize opts.MemBytes
Tags []string
Target string
Ulimits *opts.UlimitOpt
Session []session.Attachable
Linked bool // Linked marks this target as exclusively linked (not requested by the user).
@@ -106,9 +101,6 @@ type Inputs struct {
ContextState *llb.State
DockerfileInline string
NamedContexts map[string]NamedContext
// DockerfileMappingSrc and DockerfileMappingDst are filled in by the builder.
DockerfileMappingSrc string
DockerfileMappingDst string
}
type NamedContext struct {
@@ -155,11 +147,11 @@ func toRepoOnly(in string) (string, error) {
return strings.Join(out, ","), nil
}
func Build(ctx context.Context, nodes []builder.Node, opts map[string]Options, docker *dockerutil.Client, cfg *confutil.Config, w progress.Writer) (resp map[string]*client.SolveResponse, err error) {
return BuildWithResultHandler(ctx, nodes, opts, docker, cfg, w, nil)
func Build(ctx context.Context, nodes []builder.Node, opt map[string]Options, docker *dockerutil.Client, configDir string, w progress.Writer) (resp map[string]*client.SolveResponse, err error) {
return BuildWithResultHandler(ctx, nodes, opt, docker, configDir, w, nil)
}
func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[string]Options, docker *dockerutil.Client, cfg *confutil.Config, w progress.Writer, resultHandleFunc func(driverIndex int, rCtx *ResultHandle)) (resp map[string]*client.SolveResponse, err error) {
func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opt map[string]Options, docker *dockerutil.Client, configDir string, w progress.Writer, resultHandleFunc func(driverIndex int, rCtx *ResultHandle)) (resp map[string]*client.SolveResponse, err error) {
if len(nodes) == 0 {
return nil, errors.Errorf("driver required for build")
}
@@ -177,9 +169,9 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
}
}
if noMobyDriver != nil && !noDefaultLoad() && noCallFunc(opts) {
if noMobyDriver != nil && !noDefaultLoad() && noCallFunc(opt) {
var noOutputTargets []string
for name, opt := range opts {
for name, opt := range opt {
if noMobyDriver.Features(ctx)[driver.DefaultLoad] {
continue
}
@@ -200,7 +192,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
}
}
drivers, err := resolveDrivers(ctx, nodes, opts, w)
drivers, err := resolveDrivers(ctx, nodes, opt, w)
if err != nil {
return nil, err
}
@@ -217,7 +209,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
reqForNodes := make(map[string][]*reqForNode)
eg, ctx := errgroup.WithContext(ctx)
for k, opt := range opts {
for k, opt := range opt {
multiDriver := len(drivers[k]) > 1
hasMobyDriver := false
addGitAttrs, err := getGitAttributes(ctx, opt.Inputs.ContextPath, opt.Inputs.DockerfilePath)
@@ -237,13 +229,11 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
if err != nil {
return nil, err
}
localOpt := opt
so, release, err := toSolveOpt(ctx, np.Node(), multiDriver, &localOpt, gatewayOpts, cfg, w, docker)
opts[k] = localOpt
so, release, err := toSolveOpt(ctx, np.Node(), multiDriver, opt, gatewayOpts, configDir, w, docker)
if err != nil {
return nil, err
}
if err := saveLocalState(so, k, opt, np.Node(), cfg); err != nil {
if err := saveLocalState(so, k, opt, np.Node(), configDir); err != nil {
return nil, err
}
addGitAttrs(so)
@@ -279,7 +269,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
}
// validate that all links between targets use same drivers
for name := range opts {
for name := range opt {
dps := reqForNodes[name]
for i, dp := range dps {
so := reqForNodes[name][i].so
@@ -315,10 +305,10 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
var respMu sync.Mutex
results := waitmap.New()
multiTarget := len(opts) > 1
childTargets := calculateChildTargets(reqForNodes, opts)
multiTarget := len(opt) > 1
childTargets := calculateChildTargets(reqForNodes, opt)
for k, opt := range opts {
for k, opt := range opt {
err := func(k string) (err error) {
opt := opt
dps := drivers[k]
@@ -504,9 +494,7 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opts map[
resultHandle, rr, err = NewResultHandle(ctx, cc, *so, "buildx", buildFunc, ch)
resultHandleFunc(dp.driverIndex, resultHandle)
} else {
span, ctx := tracing.StartSpan(ctx, "build")
rr, err = c.Build(ctx, *so, "buildx", buildFunc, ch)
tracing.FinishWithError(span, err)
}
if !so.Internal && desktop.BuildBackendEnabled() && node.Driver.HistoryAPISupported(ctx) {
if err != nil {
@@ -1143,7 +1131,7 @@ func ReadSourcePolicy() (*spb.Policy, error) {
var pol spb.Policy
if err := json.Unmarshal(data, &pol); err != nil {
// maybe it's in protobuf format?
e2 := proto.Unmarshal(data, &pol)
e2 := pol.Unmarshal(data)
if e2 != nil {
return nil, errors.Wrap(err, "failed to parse source policy")
}

View File

@@ -23,7 +23,7 @@ func setupTest(tb testing.TB) {
gitutil.GitInit(c, tb)
df := []byte("FROM alpine:latest\n")
require.NoError(tb, os.WriteFile("Dockerfile", df, 0644))
assert.NoError(tb, os.WriteFile("Dockerfile", df, 0644))
gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit")
@@ -32,7 +32,7 @@ func setupTest(tb testing.TB) {
func TestGetGitAttributesNotGitRepo(t *testing.T) {
_, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
require.NoError(t, err)
assert.NoError(t, err)
}
func TestGetGitAttributesBadGitRepo(t *testing.T) {
@@ -47,7 +47,7 @@ func TestGetGitAttributesNoContext(t *testing.T) {
setupTest(t)
addGitAttrs, err := getGitAttributes(context.Background(), "", "Dockerfile")
require.NoError(t, err)
assert.NoError(t, err)
var so client.SolveOpt
addGitAttrs(&so)
assert.Empty(t, so.FrontendAttrs)
@@ -195,8 +195,8 @@ func TestLocalDirsSub(t *testing.T) {
gitutil.GitInit(c, t)
df := []byte("FROM alpine:latest\n")
require.NoError(t, os.MkdirAll("app", 0755))
require.NoError(t, os.WriteFile("app/Dockerfile", df, 0644))
assert.NoError(t, os.MkdirAll("app", 0755))
assert.NoError(t, os.WriteFile("app/Dockerfile", df, 0644))
gitutil.GitAdd(c, t, "app/Dockerfile")
gitutil.GitCommit(c, t, "initial commit")

View File

@@ -16,7 +16,7 @@ import (
type Container struct {
cancelOnce sync.Once
containerCancel func(error)
containerCancel func()
isUnavailable atomic.Bool
initStarted atomic.Bool
container gateway.Container
@@ -31,18 +31,18 @@ func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllera
errCh := make(chan error)
go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancelCause(ctx)
ctx, cancel := context.WithCancel(ctx)
go func() {
<-mainCtx.Done()
cancel(errors.WithStack(context.Canceled))
cancel()
}()
containerCfg, err := resultCtx.getContainerConfig(cfg)
if err != nil {
return nil, err
}
containerCtx, containerCancel := context.WithCancelCause(ctx)
defer containerCancel(errors.WithStack(context.Canceled))
containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil {
return nil, err
@@ -83,7 +83,7 @@ func (c *Container) Cancel() {
c.markUnavailable()
c.cancelOnce.Do(func() {
if c.containerCancel != nil {
c.containerCancel(errors.WithStack(context.Canceled))
c.containerCancel()
}
close(c.releaseCh)
})

View File

@@ -5,13 +5,12 @@ import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/confutil"
"github.com/moby/buildkit/client"
)
func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, cfg *confutil.Config) error {
func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, configDir string) error {
var err error
if so.Ref == "" || opts.CallFunc != nil {
if so.Ref == "" {
return nil
}
lp := opts.Inputs.ContextPath
@@ -31,7 +30,7 @@ func saveLocalState(so *client.SolveOpt, target string, opts Options, node build
if lp == "" && dp == "" {
return nil
}
l, err := localstate.New(cfg)
l, err := localstate.New(configDir)
if err != nil {
return err
}

View File

@@ -11,8 +11,8 @@ import (
"strings"
"syscall"
"github.com/containerd/containerd/v2/core/content"
"github.com/containerd/containerd/v2/plugins/content/local"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/content/local"
"github.com/containerd/platforms"
"github.com/distribution/reference"
"github.com/docker/buildx/builder"
@@ -35,7 +35,7 @@ import (
"github.com/tonistiigi/fsutil"
)
func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *Options, bopts gateway.BuildOpts, cfg *confutil.Config, pw progress.Writer, docker *dockerutil.Client) (_ *client.SolveOpt, release func(), err error) {
func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt Options, bopts gateway.BuildOpts, configDir string, pw progress.Writer, docker *dockerutil.Client) (_ *client.SolveOpt, release func(), err error) {
nodeDriver := node.Driver
defers := make([]func(), 0, 2)
releaseF := func() {
@@ -263,7 +263,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
so.Exports = opt.Exports
so.Session = slices.Clone(opt.Session)
releaseLoad, err := loadInputs(ctx, nodeDriver, &opt.Inputs, pw, &so)
releaseLoad, err := loadInputs(ctx, nodeDriver, opt.Inputs, pw, &so)
if err != nil {
return nil, nil, err
}
@@ -271,7 +271,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
// add node identifier to shared key if one was specified
if so.SharedKey != "" {
so.SharedKey += ":" + cfg.TryNodeIdentifier()
so.SharedKey += ":" + confutil.TryNodeIdentifier(configDir)
}
if opt.Pull {
@@ -356,7 +356,7 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt *O
return &so, releaseF, nil
}
func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw progress.Writer, target *client.SolveOpt) (func(), error) {
func loadInputs(ctx context.Context, d *driver.DriverHandle, inp Inputs, pw progress.Writer, target *client.SolveOpt) (func(), error) {
if inp.ContextPath == "" {
return nil, errors.New("please specify build context (e.g. \".\" for the current directory)")
}
@@ -364,12 +364,11 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
// TODO: handle stdin, symlinks, remote contexts, check files exist
var (
err error
dockerfileReader io.ReadCloser
dockerfileDir string
dockerfileName = inp.DockerfilePath
dockerfileSrcName = inp.DockerfilePath
toRemove []string
err error
dockerfileReader io.ReadCloser
dockerfileDir string
dockerfileName = inp.DockerfilePath
toRemove []string
)
switch {
@@ -441,11 +440,6 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
if inp.DockerfileInline != "" {
dockerfileReader = io.NopCloser(strings.NewReader(inp.DockerfileInline))
dockerfileSrcName = "inline"
} else if inp.DockerfilePath == "-" {
dockerfileSrcName = "stdin"
} else if inp.DockerfilePath == "" {
dockerfileSrcName = filepath.Join(inp.ContextPath, "Dockerfile")
}
if dockerfileReader != nil {
@@ -546,9 +540,6 @@ func loadInputs(ctx context.Context, d *driver.DriverHandle, inp *Inputs, pw pro
_ = os.RemoveAll(dir)
}
}
inp.DockerfileMappingSrc = dockerfileSrcName
inp.DockerfileMappingDst = dockerfileName
return release, nil
}

View File

@@ -8,14 +8,13 @@ import (
"strings"
"sync"
"github.com/containerd/containerd/v2/core/content"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/content/proxy"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
provenancetypes "github.com/moby/buildkit/solver/llbsolver/provenance/types"
digest "github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
@@ -125,8 +124,8 @@ func lookupProvenance(res *controlapi.BuildResultInfo) *ocispecs.Descriptor {
for _, a := range res.Attestations {
if a.MediaType == "application/vnd.in-toto+json" && strings.HasPrefix(a.Annotations["in-toto.io/predicate-type"], "https://slsa.dev/provenance/") {
return &ocispecs.Descriptor{
Digest: digest.Digest(a.Digest),
Size: a.Size,
Digest: a.Digest,
Size: a.Size_,
MediaType: a.MediaType,
Annotations: a.Annotations,
}

View File

@@ -10,6 +10,7 @@ import (
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func generateRandomData(size int) []byte {
@@ -56,7 +57,7 @@ func TestSyncMultiReaderParallel(t *testing.T) {
return
}
assert.NoError(t, err, "Reader %d error", readerId)
require.NoError(t, err, "Reader %d error", readerId)
if mathrand.Intn(1000) == 0 { //nolint:gosec
t.Logf("Reader %d closing", readerId)

View File

@@ -82,7 +82,7 @@ func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt
var respHandle *ResultHandle
go func() {
defer func() { cancel(errors.WithStack(context.Canceled)) }() // ensure no dangling processes
defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result
var err error
@@ -181,7 +181,7 @@ func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt
case <-respHandle.done:
case <-ctx.Done():
}
return nil, context.Cause(ctx)
return nil, ctx.Err()
}, nil)
if respHandle != nil {
return
@@ -295,14 +295,14 @@ func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
func (r *ResultHandle) getContainerConfig(cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(r.res, cfg)
ccfg, err := containerConfigFromResult(r.res, *cfg)
if err != nil {
return containerCfg, err
}
containerCfg = *ccfg
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, cfg)
ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available")
}
@@ -315,19 +315,19 @@ func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin i
processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, cfg); err != nil {
if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err
}
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, cfg); err != nil {
if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err
}
}
return processCfg, nil
}
func containerConfigFromResult(res *gateway.Result, cfg *controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
func containerConfigFromResult(res *gateway.Result, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
}
@@ -352,7 +352,7 @@ func containerConfigFromResult(res *gateway.Result, cfg *controllerapi.InvokeCon
}, nil
}
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg *controllerapi.InvokeConfig) error {
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image
if len(imgData) > 0 {
@@ -403,7 +403,7 @@ func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Res
return nil
}
func containerConfigFromError(solveErr *errdefs.SolveError, cfg *controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr)
if err != nil {
return nil, err
@@ -431,7 +431,7 @@ func containerConfigFromError(solveErr *errdefs.SolveError, cfg *controllerapi.I
}, nil
}
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg *controllerapi.InvokeConfig) error {
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr)
if err != nil {
return err

View File

@@ -7,15 +7,12 @@ import (
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors"
)
const maxDockerfileSize = 2 * 1024 * 1024 // 2 MB
func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil {
@@ -46,8 +43,8 @@ func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, ur
if err != nil {
return nil, err
}
if stat.Size > maxDockerfileSize {
return nil, errors.Errorf("Dockerfile %s bigger than allowed max size (%s)", url, units.HumanSize(maxDockerfileSize))
if stat.Size() > 512*1024 {
return nil, errors.Errorf("Dockerfile %s bigger than allowed max size", url)
}
dt, err := ref.ReadFile(ctx, gwclient.ReadRequest{
@@ -66,6 +63,7 @@ func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, ur
out = dir
return nil, nil
}, ch)
if err != nil {
return "", err
}

View File

@@ -138,7 +138,7 @@ func TestToBuildkitExtraHosts(t *testing.T) {
actualOut, actualErr := toBuildkitExtraHosts(context.TODO(), tc.input, nil)
if tc.expectedErr == "" {
require.Equal(t, tc.expectedOut, actualOut)
require.NoError(t, actualErr)
require.Nil(t, actualErr)
} else {
require.Zero(t, actualOut)
require.Error(t, actualErr, tc.expectedErr)

View File

@@ -288,15 +288,7 @@ func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
return nil, err
}
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
builders := make([]*Builder, len(storeng), len(storeng)+len(contexts))
builders := make([]*Builder, len(storeng))
seen := make(map[string]struct{})
for i, ng := range storeng {
b, err := New(dockerCli,
@@ -311,6 +303,14 @@ func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
seen[b.NodeGroup.Name] = struct{}{}
}
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
for _, c := range contexts {
// if a context has the same name as an instance from the store, do not
// add it to the builders list. An instance from the store takes
@@ -439,7 +439,7 @@ func Create(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Cre
if buildkitdConfigFile == "" {
// if buildkit daemon config is not provided, check if the default one
// is available and use it
if f, ok := confutil.NewConfig(dockerCli).BuildKitConfigFile(); ok {
if f, ok := confutil.DefaultConfigFile(dockerCli); ok {
buildkitdConfigFile = f
}
}
@@ -522,9 +522,8 @@ func Create(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Cre
return nil, err
}
cancelCtx, cancel := context.WithCancelCause(ctx)
timeoutCtx, _ := context.WithTimeoutCause(cancelCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
defer func() { cancel(errors.WithStack(context.Canceled)) }()
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, WithData())
if err != nil {
@@ -585,7 +584,7 @@ func Leave(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts Leav
return err
}
ls, err := localstate.New(confutil.NewConfig(dockerCli))
ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}

View File

@@ -19,20 +19,17 @@ func TestCsvToMap(t *testing.T) {
require.NoError(t, err)
require.Contains(t, r, "tolerations")
require.Equal(t, "key=foo,value=bar;key=foo2,value=bar2", r["tolerations"])
require.Equal(t, r["tolerations"], "key=foo,value=bar;key=foo2,value=bar2")
require.Contains(t, r, "replicas")
require.Equal(t, "1", r["replicas"])
require.Equal(t, r["replicas"], "1")
require.Contains(t, r, "namespace")
require.Equal(t, "default", r["namespace"])
require.Equal(t, r["namespace"], "default")
}
func TestParseBuildkitdFlags(t *testing.T) {
dirConf := t.TempDir()
buildkitdConfPath := path.Join(dirConf, "buildkitd-conf.toml")
require.NoError(t, os.WriteFile(buildkitdConfPath, []byte(`
buildkitdConf := `
# debug enables additional debug logging
debug = true
# insecure-entitlements allows insecure entitlements, disabled by default.
@@ -40,18 +37,10 @@ insecure-entitlements = [ "network.host", "security.insecure" ]
[log]
# log formatter: json or text
format = "text"
`), 0644))
buildkitdConfBrokenPath := path.Join(dirConf, "buildkitd-conf-broken.toml")
require.NoError(t, os.WriteFile(buildkitdConfBrokenPath, []byte(`
[worker.oci]
gc = "maybe"
`), 0644))
buildkitdConfUnknownFieldPath := path.Join(dirConf, "buildkitd-unknown-field.toml")
require.NoError(t, os.WriteFile(buildkitdConfUnknownFieldPath, []byte(`
foo = "bar"
`), 0644))
`
dirConf := t.TempDir()
buildkitdConfPath := path.Join(dirConf, "buildkitd-conf.toml")
require.NoError(t, os.WriteFile(buildkitdConfPath, []byte(buildkitdConf), 0644))
testCases := []struct {
name string
@@ -168,26 +157,6 @@ foo = "bar"
nil,
true,
},
{
"error parsing buildkit config",
"",
"docker-container",
nil,
buildkitdConfBrokenPath,
nil,
true,
},
{
"unknown field in buildkit config",
"",
"docker-container",
nil,
buildkitdConfUnknownFieldPath,
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
}
for _, tt := range testCases {
tt := tt

View File

@@ -1,75 +0,0 @@
package main
import (
"context"
"os"
"runtime"
"runtime/pprof"
"github.com/moby/buildkit/util/bklog"
"github.com/sirupsen/logrus"
)
func setupDebugProfiles(ctx context.Context) (stop func()) {
var stopFuncs []func()
if fn := setupCPUProfile(ctx); fn != nil {
stopFuncs = append(stopFuncs, fn)
}
if fn := setupHeapProfile(ctx); fn != nil {
stopFuncs = append(stopFuncs, fn)
}
return func() {
for _, fn := range stopFuncs {
fn()
}
}
}
func setupCPUProfile(ctx context.Context) (stop func()) {
if cpuProfile := os.Getenv("BUILDX_CPU_PROFILE"); cpuProfile != "" {
f, err := os.Create(cpuProfile)
if err != nil {
bklog.G(ctx).Warn("could not create cpu profile", logrus.WithError(err))
return nil
}
if err := pprof.StartCPUProfile(f); err != nil {
bklog.G(ctx).Warn("could not start cpu profile", logrus.WithError(err))
_ = f.Close()
return nil
}
return func() {
pprof.StopCPUProfile()
if err := f.Close(); err != nil {
bklog.G(ctx).Warn("could not close file for cpu profile", logrus.WithError(err))
}
}
}
return nil
}
func setupHeapProfile(ctx context.Context) (stop func()) {
if heapProfile := os.Getenv("BUILDX_MEM_PROFILE"); heapProfile != "" {
// Memory profile is only created on stop.
return func() {
f, err := os.Create(heapProfile)
if err != nil {
bklog.G(ctx).Warn("could not create memory profile", logrus.WithError(err))
return
}
// get up-to-date statistics
runtime.GC()
if err := pprof.WriteHeapProfile(f); err != nil {
bklog.G(ctx).Warn("could not write memory profile", logrus.WithError(err))
}
if err := f.Close(); err != nil {
bklog.G(ctx).Warn("could not close file for memory profile", logrus.WithError(err))
}
}
}
return nil
}

View File

@@ -6,7 +6,6 @@ import (
"os"
"github.com/docker/buildx/commands"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
@@ -17,21 +16,23 @@ import (
cliflags "github.com/docker/cli/cli/flags"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack"
"github.com/pkg/errors"
"go.opentelemetry.io/otel"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
_ "github.com/docker/buildx/driver/kubernetes"
_ "github.com/docker/buildx/driver/remote"
// Use custom grpc codec to utilize vtprotobuf
_ "github.com/moby/buildkit/util/grpcutil/encoding/proto"
)
func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision)
}
@@ -69,16 +70,6 @@ func runPlugin(cmd *command.DockerCli) error {
})
}
func run(cmd *command.DockerCli) error {
stopProfiles := setupDebugProfiles(context.TODO())
defer stopProfiles()
if plugin.RunningStandalone() {
return runStandalone(cmd)
}
return runPlugin(cmd)
}
func main() {
cmd, err := command.NewDockerCli()
if err != nil {
@@ -86,11 +77,15 @@ func main() {
os.Exit(1)
}
if err = run(cmd); err == nil {
if plugin.RunningStandalone() {
err = runStandalone(cmd)
} else {
err = runPlugin(cmd)
}
if err == nil {
return
}
// Check the error from the run function above.
if sterr, ok := err.(cli.StatusError); ok {
if sterr.Status != "" {
fmt.Fprintln(cmd.Err(), sterr.Status)
@@ -111,15 +106,8 @@ func main() {
} else {
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
}
var ebr *desktop.ErrorWithBuildRef
if errors.As(err, &ebr) {
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
ebr.Print(cmd.Err())
} else {
var be *controllererrors.BuildError
if errors.As(err, &be) {
be.PrintBuildDetails(cmd.Err())
}
}
os.Exit(1)

View File

@@ -25,6 +25,7 @@ import (
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
@@ -37,30 +38,24 @@ import (
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/tonistiigi/go-csvvalue"
"go.opentelemetry.io/otel/attribute"
)
type bakeOptions struct {
files []string
overrides []string
sbom string
provenance string
allow []string
files []string
overrides []string
printOnly bool
listTargets bool
listVars bool
sbom string
provenance string
allow []string
builder string
metadataFile string
exportPush bool
exportLoad bool
callFunc string
print bool
list string
// TODO: remove deprecated flags
listTargets bool
listVars bool
}
func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
@@ -112,27 +107,16 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return errors.Wrapf(err, "failed to get current working directory")
}
// filesystem access under the current working directory is allowed by default
ent.FSRead = append(ent.FSRead, wd)
ent.FSWrite = append(ent.FSWrite, wd)
ctx2, cancel := context.WithCancelCause(context.TODO())
defer cancel(errors.WithStack(context.Canceled))
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
var nodes []builder.Node
var progressConsoleDesc, progressTextDesc string
if in.print && in.list != "" {
return errors.New("--print and --list are mutually exclusive")
}
// instance only needed for reading remote bake files or building
var driverType string
if url != "" || !(in.print || in.list != "") {
if url != "" || !in.printOnly {
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithContextPathHash(contextPathHash),
@@ -193,7 +177,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
"BAKE_LOCAL_PLATFORM": platforms.Format(platforms.DefaultSpec()),
}
if in.list != "" {
if in.listTargets || in.listVars {
cfg, pm, err := bake.ParseFiles(files, defaults)
if err != nil {
return err
@@ -201,19 +185,14 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err = printer.Wait(); err != nil {
return err
}
list, err := parseList(in.list)
if err != nil {
return err
}
switch list.Type {
case "targets":
return printTargetList(dockerCli.Out(), list.Format, cfg)
case "variables":
return printVars(dockerCli.Out(), list.Format, pm.AllVariables)
if in.listTargets {
return printTargetList(dockerCli.Out(), cfg)
} else if in.listVars {
return printVars(dockerCli.Out(), pm.AllVariables)
}
}
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, defaults, &ent)
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, defaults)
if err != nil {
return err
}
@@ -245,7 +224,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
Target: tgts,
}
if in.print {
if in.printOnly {
if err = printer.Wait(); err != nil {
return err
}
@@ -271,7 +250,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if err != nil {
return err
}
if err := exp.Prompt(ctx, url != "", &syncWriter{w: dockerCli.Err(), wait: printer.Wait}); err != nil {
if err := exp.Prompt(ctx, &syncWriter{w: dockerCli.Err(), wait: printer.Wait}); err != nil {
return err
}
if printer.IsDone() {
@@ -286,7 +265,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
}
done := timeBuildCommand(mp, attributes)
resp, retErr := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), printer)
resp, retErr := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), printer)
if err := printer.Wait(); retErr == nil {
retErr = err
}
@@ -356,7 +335,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
if callFormatJSON {
jsonResults[name] = map[string]any{}
buf := &bytes.Buffer{}
if code, err := printResult(buf, pf, res, name, &req.Inputs); err != nil {
if code, err := printResult(buf, pf, res); err != nil {
jsonResults[name]["error"] = err.Error()
exitCode = 1
} else if code != 0 && exitCode == 0 {
@@ -382,7 +361,7 @@ func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in ba
}
fmt.Fprintln(dockerCli.Out())
if code, err := printResult(dockerCli.Out(), pf, res, name, &req.Inputs); err != nil {
if code, err := printResult(dockerCli.Out(), pf, res); err != nil {
fmt.Fprintf(dockerCli.Out(), "error: %v\n", err)
exitCode = 1
} else if code != 0 && exitCode == 0 {
@@ -441,13 +420,6 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
if !cmd.Flags().Lookup("pull").Changed {
cFlags.pull = nil
}
if options.list == "" {
if options.listTargets {
options.list = "targets"
} else if options.listVars {
options.list = "variables"
}
}
options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function.
@@ -460,6 +432,7 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.StringArrayVarP(&options.files, "file", "f", []string{}, "Build definition file")
flags.BoolVar(&options.exportLoad, "load", false, `Shorthand for "--set=*.output=type=docker"`)
flags.BoolVar(&options.printOnly, "print", false, "Print the options without building")
flags.BoolVar(&options.exportPush, "push", false, `Shorthand for "--set=*.output=type=registry"`)
flags.StringVar(&options.sbom, "sbom", "", `Shorthand for "--set=*.attest=type=sbom"`)
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
@@ -470,16 +443,13 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.VarPF(callAlias(&options.callFunc, "check"), "check", "", `Shorthand for "--call=check"`)
flags.Lookup("check").NoOptDefVal = "true"
flags.BoolVar(&options.print, "print", false, "Print the options without building")
flags.StringVar(&options.list, "list", "", "List targets or variables")
// TODO: remove deprecated flags
flags.BoolVar(&options.listTargets, "list-targets", false, "List available targets")
cobrautil.MarkFlagsExperimental(flags, "list-targets")
flags.MarkHidden("list-targets")
flags.MarkDeprecated("list-targets", "list-targets is deprecated, use list=targets instead")
flags.BoolVar(&options.listVars, "list-variables", false, "List defined variables")
cobrautil.MarkFlagsExperimental(flags, "list-variables")
flags.MarkHidden("list-variables")
flags.MarkDeprecated("list-variables", "list-variables is deprecated, use list=variables instead")
commonBuildFlags(&cFlags, flags)
@@ -494,19 +464,13 @@ func saveLocalStateGroup(dockerCli command.Cli, in bakeOptions, targets []string
groupRef := identity.NewID()
refs := make([]string, 0, len(bo))
for k, b := range bo {
if b.CallFunc != nil {
continue
}
b.Ref = identity.NewID()
b.GroupRef = groupRef
b.ProvenanceResponseMode = prm
refs = append(refs, b.Ref)
bo[k] = b
}
if len(refs) == 0 {
return nil
}
l, err := localstate.New(confutil.NewConfig(dockerCli))
l, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
@@ -580,70 +544,10 @@ func readBakeFiles(ctx context.Context, nodes []builder.Node, url string, names
return
}
type listEntry struct {
Type string
Format string
}
func parseList(input string) (listEntry, error) {
res := listEntry{}
fields, err := csvvalue.Fields(input, nil)
if err != nil {
return res, err
}
if len(fields) == 1 && fields[0] == input && !strings.HasPrefix(input, "type=") {
res.Type = input
}
if res.Type == "" {
for _, field := range fields {
key, value, ok := strings.Cut(field, "=")
if !ok {
return res, errors.Errorf("invalid value %s", field)
}
key = strings.TrimSpace(strings.ToLower(key))
switch key {
case "type":
res.Type = value
case "format":
res.Format = value
default:
return res, errors.Errorf("unexpected key '%s' in '%s'", key, field)
}
}
}
if res.Format == "" {
res.Format = "table"
}
switch res.Type {
case "targets", "variables":
default:
return res, errors.Errorf("invalid list type %q", res.Type)
}
switch res.Format {
case "table", "json":
default:
return res, errors.Errorf("invalid list format %q", res.Format)
}
return res, nil
}
func printVars(w io.Writer, format string, vars []*hclparser.Variable) error {
func printVars(w io.Writer, vars []*hclparser.Variable) error {
slices.SortFunc(vars, func(a, b *hclparser.Variable) int {
return cmp.Compare(a.Name, b.Name)
})
if format == "json" {
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
return enc.Encode(vars)
}
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush()
@@ -661,7 +565,12 @@ func printVars(w io.Writer, format string, vars []*hclparser.Variable) error {
return nil
}
func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
func printTargetList(w io.Writer, cfg *bake.Config) error {
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush()
tw.Write([]byte("TARGET\tDESCRIPTION\n"))
type targetOrGroup struct {
name string
target *bake.Target
@@ -680,20 +589,6 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
return cmp.Compare(a.name, b.name)
})
var tw *tabwriter.Writer
if format == "table" {
tw = tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
defer tw.Flush()
tw.Write([]byte("TARGET\tDESCRIPTION\n"))
}
type targetList struct {
Name string `json:"name"`
Description string `json:"description,omitempty"`
Group bool `json:"group,omitempty"`
}
var targetsList []targetList
for _, tgt := range list {
if strings.HasPrefix(tgt.name, "_") {
// convention for a private target
@@ -702,9 +597,9 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
var descr string
if tgt.target != nil {
descr = tgt.target.Description
targetsList = append(targetsList, targetList{Name: tgt.name, Description: descr})
} else if tgt.group != nil {
descr = tgt.group.Description
if len(tgt.group.Targets) > 0 {
slices.Sort(tgt.group.Targets)
names := strings.Join(tgt.group.Targets, ", ")
@@ -714,17 +609,8 @@ func printTargetList(w io.Writer, format string, cfg *bake.Config) error {
descr = names
}
}
targetsList = append(targetsList, targetList{Name: tgt.name, Description: descr, Group: true})
}
if format == "table" {
fmt.Fprintf(tw, "%s\t%s\n", tgt.name, descr)
}
}
if format == "json" {
enc := json.NewEncoder(w)
enc.SetIndent("", " ")
return enc.Encode(targetsList)
fmt.Fprintf(tw, "%s\t%s\n", tgt.name, descr)
}
return nil
@@ -735,7 +621,7 @@ func bakeMetricAttributes(dockerCli command.Cli, driverType, url, cmdContext str
commandNameAttribute.String("bake"),
attribute.Stringer(string(commandOptionsHash), &bakeOptionsHash{
bakeOptions: options,
cfg: confutil.NewConfig(dockerCli),
configDir: confutil.ConfigDir(dockerCli),
url: url,
cmdContext: cmdContext,
targets: targets,
@@ -747,7 +633,7 @@ func bakeMetricAttributes(dockerCli command.Cli, driverType, url, cmdContext str
type bakeOptionsHash struct {
*bakeOptions
cfg *confutil.Config
configDir string
url string
cmdContext string
targets []string
@@ -771,7 +657,7 @@ func (o *bakeOptionsHash) String() string {
joinedFiles := strings.Join(files, ",")
joinedTargets := strings.Join(targets, ",")
salt := o.cfg.TryNodeIdentifier()
salt := confutil.TryNodeIdentifier(o.configDir)
h := sha256.New()
for _, s := range []string{url, cmdContext, joinedFiles, joinedTargets, salt} {

View File

@@ -49,7 +49,6 @@ import (
"github.com/moby/buildkit/frontend/subrequests/outline"
"github.com/moby/buildkit/frontend/subrequests/targets"
"github.com/moby/buildkit/solver/errdefs"
solverpb "github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/morikuni/aec"
@@ -61,7 +60,6 @@ import (
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/metric"
"google.golang.org/grpc/codes"
"google.golang.org/protobuf/proto"
)
type buildOptions struct {
@@ -183,17 +181,14 @@ func (o *buildOptions) toControllerOptions() (*controllerapi.BuildOptions, error
}
}
cacheFrom, err := buildflags.ParseCacheEntry(o.cacheFrom)
opts.CacheFrom, err = buildflags.ParseCacheEntry(o.cacheFrom)
if err != nil {
return nil, err
}
opts.CacheFrom = cacheFrom.ToPB()
cacheTo, err := buildflags.ParseCacheEntry(o.cacheTo)
opts.CacheTo, err = buildflags.ParseCacheEntry(o.cacheTo)
if err != nil {
return nil, err
}
opts.CacheTo = cacheTo.ToPB()
opts.Secrets, err = buildflags.ParseSecretSpecs(o.secrets)
if err != nil {
@@ -241,7 +236,7 @@ func buildMetricAttributes(dockerCli command.Cli, driverType string, options *bu
commandNameAttribute.String("build"),
attribute.Stringer(string(commandOptionsHash), &buildOptionsHash{
buildOptions: options,
cfg: confutil.NewConfig(dockerCli),
configDir: confutil.ConfigDir(dockerCli),
}),
driverNameAttribute.String(options.builder),
driverTypeAttribute.String(driverType),
@@ -253,7 +248,7 @@ func buildMetricAttributes(dockerCli command.Cli, driverType string, options *bu
// the fmt.Stringer interface.
type buildOptionsHash struct {
*buildOptions
cfg *confutil.Config
configDir string
result string
resultOnce sync.Once
}
@@ -270,7 +265,7 @@ func (o *buildOptionsHash) String() string {
if contextPath != "-" && osutil.IsLocalDir(contextPath) {
contextPath = osutil.ToAbs(contextPath)
}
salt := o.cfg.TryNodeIdentifier()
salt := confutil.TryNodeIdentifier(o.configDir)
h := sha256.New()
for _, s := range []string{target, contextPath, dockerfile, salt} {
@@ -328,8 +323,8 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
}
attributes := buildMetricAttributes(dockerCli, driverType, &options)
ctx2, cancel := context.WithCancelCause(context.TODO())
defer func() { cancel(errors.WithStack(context.Canceled)) }()
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
progressMode, err := options.toDisplayMode()
if err != nil {
return err
@@ -351,12 +346,11 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
done := timeBuildCommand(mp, attributes)
var resp *client.SolveResponse
var inputs *build.Inputs
var retErr error
if confutil.IsExperimental() {
resp, inputs, retErr = runControllerBuild(ctx, dockerCli, opts, options, printer)
resp, retErr = runControllerBuild(ctx, dockerCli, opts, options, printer)
} else {
resp, inputs, retErr = runBasicBuild(ctx, dockerCli, opts, printer)
resp, retErr = runBasicBuild(ctx, dockerCli, opts, printer)
}
if err := printer.Wait(); retErr == nil {
@@ -393,7 +387,7 @@ func runBuild(ctx context.Context, dockerCli command.Cli, options buildOptions)
}
}
if opts.CallFunc != nil {
if exitcode, err := printResult(dockerCli.Out(), opts.CallFunc, resp.ExporterResponse, options.target, inputs); err != nil {
if exitcode, err := printResult(dockerCli.Out(), opts.CallFunc, resp.ExporterResponse); err != nil {
return err
} else if exitcode != 0 {
os.Exit(exitcode)
@@ -411,22 +405,22 @@ func getImageID(resp map[string]string) string {
return dgst
}
func runBasicBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, printer *progress.Printer) (*client.SolveResponse, *build.Inputs, error) {
resp, res, dfmap, err := cbuild.RunBuild(ctx, dockerCli, opts, dockerCli.In(), printer, false)
func runBasicBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, printer *progress.Printer) (*client.SolveResponse, error) {
resp, res, err := cbuild.RunBuild(ctx, dockerCli, *opts, dockerCli.In(), printer, false)
if res != nil {
res.Done()
}
return resp, dfmap, err
return resp, err
}
func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, options buildOptions, printer *progress.Printer) (*client.SolveResponse, *build.Inputs, error) {
func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *controllerapi.BuildOptions, options buildOptions, printer *progress.Printer) (*client.SolveResponse, error) {
if options.invokeConfig != nil && (options.dockerfileName == "-" || options.contextPath == "-") {
// stdin must be usable for monitor
return nil, nil, errors.Errorf("Dockerfile or context from stdin is not supported with invoke")
return nil, errors.Errorf("Dockerfile or context from stdin is not supported with invoke")
}
c, err := controller.NewController(ctx, options.ControlOptions, dockerCli, printer)
if err != nil {
return nil, nil, err
return nil, err
}
defer func() {
if err := c.Close(); err != nil {
@@ -438,13 +432,12 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
// so we need to resolve paths to abosolute ones in the client.
opts, err = controllerapi.ResolveOptionPaths(opts)
if err != nil {
return nil, nil, err
return nil, err
}
var ref string
var retErr error
var resp *client.SolveResponse
var inputs *build.Inputs
var f *ioset.SingleForwarder
var pr io.ReadCloser
@@ -462,7 +455,7 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
})
}
ref, resp, inputs, err = c.Build(ctx, opts, pr, printer)
ref, resp, err = c.Build(ctx, *opts, pr, printer)
if err != nil {
var be *controllererrors.BuildError
if errors.As(err, &be) {
@@ -470,7 +463,7 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
retErr = err
// We can proceed to monitor
} else {
return nil, nil, errors.Wrapf(err, "failed to build")
return nil, errors.Wrapf(err, "failed to build")
}
}
@@ -511,7 +504,7 @@ func runControllerBuild(ctx context.Context, dockerCli command.Cli, opts *contro
}
}
return resp, inputs, retErr
return resp, retErr
}
func printError(err error, printer *progress.Printer) error {
@@ -723,7 +716,7 @@ type commonFlags struct {
func commonBuildFlags(options *commonFlags, flags *pflag.FlagSet) {
options.noCache = flags.Bool("no-cache", false, "Do not use cache when building the image")
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "quiet", "plain", "tty", "rawjson"). Use plain to show container output`)
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "plain", "tty", "rawjson"). Use plain to show container output`)
options.pull = flags.Bool("pull", false, "Always attempt to pull all referenced images")
flags.StringVar(&options.metadataFile, "metadata-file", "", "Write build result metadata to a file")
}
@@ -766,11 +759,8 @@ func decodeExporterResponse(exporterResponse map[string]string) map[string]inter
}
var raw map[string]interface{}
if err = json.Unmarshal(dt, &raw); err != nil || len(raw) == 0 {
var rawList []map[string]interface{}
if err = json.Unmarshal(dt, &rawList); err != nil || len(rawList) == 0 {
out[k] = v
continue
}
out[k] = v
continue
}
out[k] = json.RawMessage(dt)
}
@@ -888,10 +878,11 @@ func printWarnings(w io.Writer, warnings []client.VertexWarning, mode progressui
src.Print(w)
}
fmt.Fprintf(w, "\n")
}
}
func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string, target string, inp *build.Inputs) (int, error) {
func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string) (int, error) {
switch f.Name {
case "outline":
return 0, printValue(w, outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res)
@@ -917,27 +908,8 @@ func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string,
}
fmt.Fprintf(w, "Check complete, %s\n", warningCountMsg)
}
sourceInfoMap := func(sourceInfo *solverpb.SourceInfo) *solverpb.SourceInfo {
if sourceInfo == nil || inp == nil {
return sourceInfo
}
if target == "" {
target = "default"
}
if inp.DockerfileMappingSrc != "" {
newSourceInfo := proto.Clone(sourceInfo).(*solverpb.SourceInfo)
newSourceInfo.Filename = inp.DockerfileMappingSrc
return newSourceInfo
}
return sourceInfo
}
printLintWarnings := func(dt []byte, w io.Writer) error {
return lintResults.PrintTo(w, sourceInfoMap)
}
err := printValue(w, printLintWarnings, lint.SubrequestLintDefinition.Version, f.Format, res)
err := printValue(w, printLintViolationsWrapper, lint.SubrequestLintDefinition.Version, f.Format, res)
if err != nil {
return 0, err
}
@@ -952,8 +924,13 @@ func printResult(w io.Writer, f *controllerapi.CallFunc, res map[string]string,
if f.Format != "json" && len(lintResults.Warnings) > 0 {
fmt.Fprintln(w)
}
lintBuf := bytes.NewBuffer(nil)
lintResults.PrintErrorTo(lintBuf, sourceInfoMap)
lintBuf := bytes.NewBuffer([]byte(lintResults.Error.Message + "\n"))
sourceInfo := lintResults.Sources[lintResults.Error.Location.SourceIndex]
source := errdefs.Source{
Info: sourceInfo,
Ranges: lintResults.Error.Location.Ranges,
}
source.Print(lintBuf)
return 0, errors.New(lintBuf.String())
} else if len(lintResults.Warnings) == 0 && f.Format != "json" {
fmt.Fprintln(w, "Check complete, no warnings found.")
@@ -991,6 +968,11 @@ func printValue(w io.Writer, printer callFunc, version string, format string, re
return printer([]byte(res["result.json"]), w)
}
// FIXME: remove once https://github.com/docker/buildx/pull/2672 is sorted
func printLintViolationsWrapper(dt []byte, w io.Writer) error {
return lint.PrintLintViolations(dt, w, nil)
}
type invokeConfig struct {
controllerapi.InvokeConfig
onFlag string
@@ -1018,7 +1000,7 @@ func (cfg *invokeConfig) runDebug(ctx context.Context, ref string, options *cont
return nil, errors.Errorf("failed to configure terminal: %v", err)
}
defer con.Reset()
return monitor.RunMonitor(ctx, ref, options, &cfg.InvokeConfig, c, stdin, stdout, stderr, progress)
return monitor.RunMonitor(ctx, ref, options, cfg.InvokeConfig, c, stdin, stdout, stderr, progress)
}
func (cfg *invokeConfig) parseInvokeConfig(invoke, on string) error {

View File

@@ -64,7 +64,7 @@ func RootCmd(dockerCli command.Cli, children ...DebuggableCmd) *cobra.Command {
return errors.Errorf("failed to configure terminal: %v", err)
}
_, err = monitor.RunMonitor(ctx, "", nil, &controllerapi.InvokeConfig{
_, err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true,
}, c, dockerCli.In(), os.Stdout, os.Stderr, printer)
con.Reset()

View File

@@ -1,604 +0,0 @@
package history
import (
"bytes"
"cmp"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"text/tabwriter"
"time"
"github.com/containerd/containerd/v2/core/content"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/containerd/containerd/v2/core/images"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
slsa "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/common"
slsa02 "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/v0.2"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/solver/errdefs"
provenancetypes "github.com/moby/buildkit/solver/llbsolver/provenance/types"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/moby/buildkit/util/stack"
"github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/tonistiigi/go-csvvalue"
spb "google.golang.org/genproto/googleapis/rpc/status"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
proto "google.golang.org/protobuf/proto"
)
type inspectOptions struct {
builder string
ref string
}
func runInspect(ctx context.Context, dockerCli command.Cli, opts inspectOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
if opts.ref == "" {
slices.SortFunc(recs, func(a, b historyRecord) int {
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
}
rec := &recs[0]
ls, err := localstate.New(confutil.NewConfig(dockerCli))
if err != nil {
return err
}
st, _ := ls.ReadRef(rec.node.Builder, rec.node.Name, rec.Ref)
tw := tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
attrs := rec.FrontendAttrs
delete(attrs, "frontend.caps")
writeAttr := func(k, name string, f func(v string) (string, bool)) {
if v, ok := attrs[k]; ok {
if f != nil {
v, ok = f(v)
}
if ok {
fmt.Fprintf(tw, "%s:\t%s\n", name, v)
}
}
delete(attrs, k)
}
var context string
var dockerfile string
if st != nil {
context = st.LocalPath
dockerfile = st.DockerfilePath
wd, _ := os.Getwd()
if dockerfile != "" && dockerfile != "-" {
if rel, err := filepath.Rel(context, dockerfile); err == nil {
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
dockerfile = rel
}
}
}
if context != "" {
if rel, err := filepath.Rel(wd, context); err == nil {
if !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
context = rel
}
}
}
}
if v, ok := attrs["context"]; ok && context == "" {
delete(attrs, "context")
context = v
}
if dockerfile == "" {
if v, ok := attrs["filename"]; ok {
dockerfile = v
if dfdir, ok := attrs["vcs:localdir:dockerfile"]; ok {
dockerfile = filepath.Join(dfdir, dockerfile)
}
}
}
delete(attrs, "filename")
if context != "" {
fmt.Fprintf(tw, "Context:\t%s\n", context)
}
if dockerfile != "" {
fmt.Fprintf(tw, "Dockerfile:\t%s\n", dockerfile)
}
if _, ok := attrs["context"]; !ok {
if src, ok := attrs["vcs:source"]; ok {
fmt.Fprintf(tw, "VCS Repository:\t%s\n", src)
}
if rev, ok := attrs["vcs:revision"]; ok {
fmt.Fprintf(tw, "VCS Revision:\t%s\n", rev)
}
}
writeAttr("target", "Target", nil)
writeAttr("platform", "Platform", func(v string) (string, bool) {
return tryParseValue(v, func(v string) (string, error) {
var pp []string
for _, v := range strings.Split(v, ",") {
p, err := platforms.Parse(v)
if err != nil {
return "", err
}
pp = append(pp, platforms.FormatAll(platforms.Normalize(p)))
}
return strings.Join(pp, ", "), nil
}), true
})
writeAttr("build-arg:BUILDKIT_CONTEXT_KEEP_GIT_DIR", "Keep Git Dir", func(v string) (string, bool) {
return tryParseValue(v, func(v string) (string, error) {
b, err := strconv.ParseBool(v)
if err != nil {
return "", err
}
return strconv.FormatBool(b), nil
}), true
})
tw.Flush()
fmt.Fprintln(dockerCli.Out())
printTable(dockerCli.Out(), attrs, "context:", "Named Context")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "Started:\t%s\n", rec.CreatedAt.AsTime().Local().Format("2006-01-02 15:04:05"))
var duration time.Duration
var statusStr string
if rec.CompletedAt != nil {
duration = rec.CompletedAt.AsTime().Sub(rec.CreatedAt.AsTime())
} else {
duration = rec.currentTimestamp.Sub(rec.CreatedAt.AsTime())
statusStr = " (running)"
}
fmt.Fprintf(tw, "Duration:\t%s%s\n", formatDuration(duration), statusStr)
if rec.Error != nil {
if codes.Code(rec.Error.Code) == codes.Canceled {
fmt.Fprintf(tw, "Status:\tCanceled\n")
} else {
fmt.Fprintf(tw, "Error:\t%s %s\n", codes.Code(rec.Error.Code).String(), rec.Error.Message)
}
}
fmt.Fprintf(tw, "Build Steps:\t%d/%d (%.0f%% cached)\n", rec.NumCompletedSteps, rec.NumTotalSteps, float64(rec.NumCachedSteps)/float64(rec.NumTotalSteps)*100)
tw.Flush()
fmt.Fprintln(dockerCli.Out())
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
writeAttr("force-network-mode", "Network", nil)
writeAttr("hostname", "Hostname", nil)
writeAttr("add-hosts", "Extra Hosts", func(v string) (string, bool) {
return tryParseValue(v, func(v string) (string, error) {
fields, err := csvvalue.Fields(v, nil)
if err != nil {
return "", err
}
return strings.Join(fields, ", "), nil
}), true
})
writeAttr("cgroup-parent", "Cgroup Parent", nil)
writeAttr("image-resolve-mode", "Image Resolve Mode", nil)
writeAttr("multi-platform", "Force Multi-Platform", nil)
writeAttr("build-arg:BUILDKIT_MULTI_PLATFORM", "Force Multi-Platform", nil)
writeAttr("no-cache", "Disable Cache", func(v string) (string, bool) {
if v == "" {
return "true", true
}
return v, true
})
writeAttr("shm-size", "Shm Size", nil)
writeAttr("ulimit", "Resource Limits", nil)
writeAttr("build-arg:BUILDKIT_CACHE_MOUNT_NS", "Cache Mount Namespace", nil)
writeAttr("build-arg:BUILDKIT_DOCKERFILE_CHECK", "Dockerfile Check Config", nil)
writeAttr("build-arg:SOURCE_DATE_EPOCH", "Source Date Epoch", nil)
writeAttr("build-arg:SANDBOX_HOSTNAME", "Sandbox Hostname", nil)
var unusedAttrs []string
for k := range attrs {
if strings.HasPrefix(k, "vcs:") || strings.HasPrefix(k, "build-arg:") || strings.HasPrefix(k, "label:") || strings.HasPrefix(k, "context:") || strings.HasPrefix(k, "attest:") {
continue
}
unusedAttrs = append(unusedAttrs, k)
}
slices.Sort(unusedAttrs)
for _, k := range unusedAttrs {
fmt.Fprintf(tw, "%s:\t%s\n", k, attrs[k])
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
printTable(dockerCli.Out(), attrs, "build-arg:", "Build Arg")
printTable(dockerCli.Out(), attrs, "label:", "Label")
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
store := proxy.NewContentStore(c.ContentClient())
attachments, err := allAttachments(ctx, store, *rec)
if err != nil {
return err
}
provIndex := slices.IndexFunc(attachments, func(a attachment) bool {
return descrType(a.descr) == slsa02.PredicateSLSAProvenance
})
if provIndex != -1 {
prov := attachments[provIndex]
dt, err := content.ReadBlob(ctx, store, prov.descr)
if err != nil {
return errors.Errorf("failed to read provenance %s: %v", prov.descr.Digest, err)
}
var pred provenancetypes.ProvenancePredicate
if err := json.Unmarshal(dt, &pred); err != nil {
return errors.Errorf("failed to unmarshal provenance %s: %v", prov.descr.Digest, err)
}
fmt.Fprintln(dockerCli.Out(), "Materials:")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "URI\tDIGEST\n")
for _, m := range pred.Materials {
fmt.Fprintf(tw, "%s\t%s\n", m.URI, strings.Join(digestSetToDigests(m.Digest), ", "))
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
}
if len(attachments) > 0 {
fmt.Fprintf(tw, "Attachments:\n")
tw = tabwriter.NewWriter(dockerCli.Out(), 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "DIGEST\tPLATFORM\tTYPE\n")
for _, a := range attachments {
p := ""
if a.platform != nil {
p = platforms.FormatAll(*a.platform)
}
fmt.Fprintf(tw, "%s\t%s\t%s\n", a.descr.Digest, p, descrType(a.descr))
}
tw.Flush()
fmt.Fprintln(dockerCli.Out())
}
if rec.ExternalError != nil {
dt, err := content.ReadBlob(ctx, store, ociDesc(rec.ExternalError))
if err != nil {
return errors.Wrapf(err, "failed to read external error %s", rec.ExternalError.Digest)
}
var st spb.Status
if err := proto.Unmarshal(dt, &st); err != nil {
return errors.Wrapf(err, "failed to unmarshal external error %s", rec.ExternalError.Digest)
}
retErr := grpcerrors.FromGRPC(status.ErrorProto(&st))
for _, s := range errdefs.Sources(retErr) {
s.Print(dockerCli.Out())
}
fmt.Fprintln(dockerCli.Out())
var ve *errdefs.VertexError
if errors.As(retErr, &ve) {
dgst, err := digest.Parse(ve.Vertex.Digest)
if err != nil {
return errors.Wrapf(err, "failed to parse vertex digest %s", ve.Vertex.Digest)
}
name, logs, err := loadVertexLogs(ctx, c, rec.Ref, dgst, 16)
if err != nil {
return errors.Wrapf(err, "failed to load vertex logs %s", dgst)
}
if len(logs) > 0 {
fmt.Fprintln(dockerCli.Out(), "Logs:")
fmt.Fprintf(dockerCli.Out(), "> => %s:\n", name)
for _, l := range logs {
fmt.Fprintln(dockerCli.Out(), "> "+l)
}
fmt.Fprintln(dockerCli.Out())
}
}
if debug.IsEnabled() {
fmt.Fprintf(dockerCli.Out(), "\n%+v\n", stack.Formatter(retErr))
} else if len(stack.Traces(retErr)) > 0 {
fmt.Fprintf(dockerCli.Out(), "Enable --debug to see stack traces for error\n")
}
}
fmt.Fprintf(dockerCli.Out(), "Print build logs: docker buildx history logs %s\n", rec.Ref)
fmt.Fprintf(dockerCli.Out(), "View build in Docker Desktop: %s\n", desktop.BuildURL(fmt.Sprintf("%s/%s/%s", rec.node.Builder, rec.node.Name, rec.Ref)))
return nil
}
func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options inspectOptions
cmd := &cobra.Command{
Use: "inspect [OPTIONS] [REF]",
Short: "Inspect a build",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runInspect(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(
attachmentCmd(dockerCli, rootOpts),
)
// flags := cmd.Flags()
return cmd
}
func loadVertexLogs(ctx context.Context, c *client.Client, ref string, dgst digest.Digest, limit int) (string, []string, error) {
st, err := c.ControlClient().Status(ctx, &controlapi.StatusRequest{
Ref: ref,
})
if err != nil {
return "", nil, err
}
var name string
var logs []string
lastState := map[int]int{}
loop0:
for {
select {
case <-ctx.Done():
st.CloseSend()
return "", nil, context.Cause(ctx)
default:
ev, err := st.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break loop0
}
return "", nil, err
}
ss := client.NewSolveStatus(ev)
for _, v := range ss.Vertexes {
if v.Digest == dgst {
name = v.Name
break
}
}
for _, l := range ss.Logs {
if l.Vertex == dgst {
parts := bytes.Split(l.Data, []byte("\n"))
for i, p := range parts {
var wrote bool
if i == 0 {
idx, ok := lastState[l.Stream]
if ok && idx != -1 {
logs[idx] = logs[idx] + string(p)
wrote = true
}
}
if !wrote {
if len(p) > 0 {
logs = append(logs, string(p))
}
lastState[l.Stream] = len(logs) - 1
}
if i == len(parts)-1 && len(p) == 0 {
lastState[l.Stream] = -1
}
}
}
}
}
}
if limit > 0 && len(logs) > limit {
logs = logs[len(logs)-limit:]
}
return name, logs, nil
}
type attachment struct {
platform *ocispecs.Platform
descr ocispecs.Descriptor
}
func allAttachments(ctx context.Context, store content.Store, rec historyRecord) ([]attachment, error) {
var attachments []attachment
if rec.Result != nil {
for _, a := range rec.Result.Attestations {
attachments = append(attachments, attachment{
descr: ociDesc(a),
})
}
for _, r := range rec.Result.Results {
attachments = append(attachments, walkAttachments(ctx, store, ociDesc(r), nil)...)
}
}
for key, ri := range rec.Results {
p, err := platforms.Parse(key)
if err != nil {
return nil, err
}
for _, a := range ri.Attestations {
attachments = append(attachments, attachment{
platform: &p,
descr: ociDesc(a),
})
}
for _, r := range ri.Results {
attachments = append(attachments, walkAttachments(ctx, store, ociDesc(r), &p)...)
}
}
slices.SortFunc(attachments, func(a, b attachment) int {
pCmp := 0
if a.platform == nil && b.platform != nil {
return -1
} else if a.platform != nil && b.platform == nil {
return 1
} else if a.platform != nil && b.platform != nil {
pCmp = cmp.Compare(platforms.FormatAll(*a.platform), platforms.FormatAll(*b.platform))
}
return cmp.Or(
pCmp,
cmp.Compare(descrType(a.descr), descrType(b.descr)),
)
})
return attachments, nil
}
func walkAttachments(ctx context.Context, store content.Store, desc ocispecs.Descriptor, platform *ocispecs.Platform) []attachment {
_, err := store.Info(ctx, desc.Digest)
if err != nil {
return nil
}
var out []attachment
if desc.Annotations["vnd.docker.reference.type"] != "attestation-manifest" {
out = append(out, attachment{platform: platform, descr: desc})
}
if desc.MediaType != ocispecs.MediaTypeImageIndex && desc.MediaType != images.MediaTypeDockerSchema2ManifestList {
return out
}
dt, err := content.ReadBlob(ctx, store, desc)
if err != nil {
return out
}
var idx ocispecs.Index
if err := json.Unmarshal(dt, &idx); err != nil {
return out
}
for _, d := range idx.Manifests {
p := platform
if d.Platform != nil {
p = d.Platform
}
out = append(out, walkAttachments(ctx, store, d, p)...)
}
return out
}
func ociDesc(in *controlapi.Descriptor) ocispecs.Descriptor {
return ocispecs.Descriptor{
MediaType: in.MediaType,
Digest: digest.Digest(in.Digest),
Size: in.Size,
Annotations: in.Annotations,
}
}
func descrType(desc ocispecs.Descriptor) string {
if typ, ok := desc.Annotations["in-toto.io/predicate-type"]; ok {
return typ
}
return desc.MediaType
}
func tryParseValue(s string, f func(string) (string, error)) string {
v, err := f(s)
if err != nil {
return fmt.Sprintf("%s (%v)", s, err)
}
return v
}
func printTable(w io.Writer, attrs map[string]string, prefix, title string) {
var keys []string
for k := range attrs {
if strings.HasPrefix(k, prefix) {
keys = append(keys, strings.TrimPrefix(k, prefix))
}
}
slices.Sort(keys)
if len(keys) == 0 {
return
}
tw := tabwriter.NewWriter(w, 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "%s\tVALUE\n", strings.ToUpper(title))
for _, k := range keys {
fmt.Fprintf(tw, "%s\t%s\n", k, attrs[prefix+k])
}
tw.Flush()
fmt.Fprintln(w)
}
func digestSetToDigests(ds slsa.DigestSet) []string {
var out []string
for k, v := range ds {
out = append(out, fmt.Sprintf("%s:%s", k, v))
}
return out
}

View File

@@ -1,152 +0,0 @@
package history
import (
"context"
"io"
"slices"
"github.com/containerd/containerd/v2/core/content/proxy"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
intoto "github.com/in-toto/in-toto-golang/in_toto"
slsa02 "github.com/in-toto/in-toto-golang/in_toto/slsa_provenance/v0.2"
"github.com/opencontainers/go-digest"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type attachmentOptions struct {
builder string
typ string
platform string
ref string
digest digest.Digest
}
func runAttachment(ctx context.Context, dockerCli command.Cli, opts attachmentOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
if opts.ref == "" {
slices.SortFunc(recs, func(a, b historyRecord) int {
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
}
rec := &recs[0]
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
store := proxy.NewContentStore(c.ContentClient())
if opts.digest != "" {
ra, err := store.ReaderAt(ctx, ocispecs.Descriptor{Digest: opts.digest})
if err != nil {
return err
}
_, err = io.Copy(dockerCli.Out(), io.NewSectionReader(ra, 0, ra.Size()))
return err
}
attachments, err := allAttachments(ctx, store, *rec)
if err != nil {
return err
}
typ := opts.typ
switch typ {
case "index":
typ = ocispecs.MediaTypeImageIndex
case "manifest":
typ = ocispecs.MediaTypeImageManifest
case "image":
typ = ocispecs.MediaTypeImageConfig
case "provenance":
typ = slsa02.PredicateSLSAProvenance
case "sbom":
typ = intoto.PredicateSPDX
}
for _, a := range attachments {
if opts.platform != "" && (a.platform == nil || platforms.FormatAll(*a.platform) != opts.platform) {
continue
}
if typ != "" && descrType(a.descr) != typ {
continue
}
ra, err := store.ReaderAt(ctx, a.descr)
if err != nil {
return err
}
_, err = io.Copy(dockerCli.Out(), io.NewSectionReader(ra, 0, ra.Size()))
return err
}
return errors.Errorf("no matching attachment found for ref %q", opts.ref)
}
func attachmentCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options attachmentOptions
cmd := &cobra.Command{
Use: "attachment [OPTIONS] REF [DIGEST]",
Short: "Inspect a build attachment",
Args: cobra.RangeArgs(1, 2),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
if len(args) > 1 {
dgst, err := digest.Parse(args[1])
if err != nil {
return errors.Wrapf(err, "invalid digest %q", args[1])
}
options.digest = dgst
}
if options.digest == "" && options.platform == "" && options.typ == "" {
return errors.New("at least one of --type, --platform or DIGEST must be specified")
}
options.builder = *rootOpts.Builder
return runAttachment(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.typ, "type", "", "Type of attachment")
flags.StringVar(&options.platform, "platform", "", "Platform of attachment")
return cmd
}

View File

@@ -1,124 +0,0 @@
package history
import (
"context"
"io"
"os"
"slices"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type logsOptions struct {
builder string
ref string
progress string
}
func runLogs(ctx context.Context, dockerCli command.Cli, opts logsOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
if opts.ref == "" {
slices.SortFunc(recs, func(a, b historyRecord) int {
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
}
rec := &recs[0]
c, err := rec.node.Driver.Client(ctx)
if err != nil {
return err
}
cl, err := c.ControlClient().Status(ctx, &controlapi.StatusRequest{
Ref: rec.Ref,
})
if err != nil {
return err
}
var mode progressui.DisplayMode = progressui.DisplayMode(opts.progress)
if mode == progressui.AutoMode {
mode = progressui.PlainMode
}
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, mode)
if err != nil {
return err
}
loop0:
for {
select {
case <-ctx.Done():
cl.CloseSend()
return context.Cause(ctx)
default:
ev, err := cl.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break loop0
}
return err
}
printer.Write(client.NewSolveStatus(ev))
}
}
return printer.Wait()
}
func logsCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options logsOptions
cmd := &cobra.Command{
Use: "logs [OPTIONS] [REF]",
Short: "Print the logs of a build",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runLogs(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.progress, "progress", "plain", "Set type of progress output (plain, rawjson, tty)")
return cmd
}

View File

@@ -1,234 +0,0 @@
package history
import (
"context"
"encoding/json"
"fmt"
"os"
"time"
"github.com/containerd/console"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/formatter"
"github.com/docker/go-units"
"github.com/spf13/cobra"
"golang.org/x/exp/slices"
)
const (
lsHeaderBuildID = "BUILD ID"
lsHeaderName = "NAME"
lsHeaderStatus = "STATUS"
lsHeaderCreated = "CREATED AT"
lsHeaderDuration = "DURATION"
lsHeaderLink = ""
lsDefaultTableFormat = "table {{.Ref}}\t{{.Name}}\t{{.Status}}\t{{.CreatedAt}}\t{{.Duration}}\t{{.Link}}"
headerKeyTimestamp = "buildkit-current-timestamp"
)
type lsOptions struct {
builder string
format string
noTrunc bool
}
func runLs(ctx context.Context, dockerCli command.Cli, opts lsOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
out, err := queryRecords(ctx, "", nodes)
if err != nil {
return err
}
ls, err := localstate.New(confutil.NewConfig(dockerCli))
if err != nil {
return err
}
for i, rec := range out {
st, _ := ls.ReadRef(rec.node.Builder, rec.node.Name, rec.Ref)
rec.name = buildName(rec.FrontendAttrs, st)
out[i] = rec
}
return lsPrint(dockerCli, out, opts)
}
func lsCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options lsOptions
cmd := &cobra.Command{
Use: "ls",
Short: "List build records",
Args: cli.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *rootOpts.Builder
return runLs(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output")
flags.BoolVar(&options.noTrunc, "no-trunc", false, "Don't truncate output")
return cmd
}
func lsPrint(dockerCli command.Cli, records []historyRecord, in lsOptions) error {
if in.format == formatter.TableFormatKey {
in.format = lsDefaultTableFormat
}
ctx := formatter.Context{
Output: dockerCli.Out(),
Format: formatter.Format(in.format),
Trunc: !in.noTrunc,
}
slices.SortFunc(records, func(a, b historyRecord) int {
if a.CompletedAt == nil && b.CompletedAt != nil {
return -1
}
if a.CompletedAt != nil && b.CompletedAt == nil {
return 1
}
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
var term bool
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
term = true
}
render := func(format func(subContext formatter.SubContext) error) error {
for _, r := range records {
if err := format(&lsContext{
format: formatter.Format(in.format),
isTerm: term,
trunc: !in.noTrunc,
record: &r,
}); err != nil {
return err
}
}
return nil
}
lsCtx := lsContext{
isTerm: term,
trunc: !in.noTrunc,
}
lsCtx.Header = formatter.SubHeaderContext{
"Ref": lsHeaderBuildID,
"Name": lsHeaderName,
"Status": lsHeaderStatus,
"CreatedAt": lsHeaderCreated,
"Duration": lsHeaderDuration,
"Link": lsHeaderLink,
}
return ctx.Write(&lsCtx, render)
}
type lsContext struct {
formatter.HeaderContext
isTerm bool
trunc bool
format formatter.Format
record *historyRecord
}
func (c *lsContext) MarshalJSON() ([]byte, error) {
m := map[string]interface{}{
"ref": c.FullRef(),
"name": c.Name(),
"status": c.Status(),
"created_at": c.record.CreatedAt.AsTime().Format(time.RFC3339Nano),
"total_steps": c.record.NumTotalSteps,
"completed_steps": c.record.NumCompletedSteps,
"cached_steps": c.record.NumCachedSteps,
}
if c.record.CompletedAt != nil {
m["completed_at"] = c.record.CompletedAt.AsTime().Format(time.RFC3339Nano)
}
return json.Marshal(m)
}
func (c *lsContext) Ref() string {
return c.record.Ref
}
func (c *lsContext) FullRef() string {
return fmt.Sprintf("%s/%s/%s", c.record.node.Builder, c.record.node.Name, c.record.Ref)
}
func (c *lsContext) Name() string {
name := c.record.name
if c.trunc && c.format.IsTable() {
return trimBeginning(name, 36)
}
return name
}
func (c *lsContext) Status() string {
if c.record.CompletedAt != nil {
if c.record.Error != nil {
return "Error"
}
return "Completed"
}
return "Running"
}
func (c *lsContext) CreatedAt() string {
return units.HumanDuration(time.Since(c.record.CreatedAt.AsTime())) + " ago"
}
func (c *lsContext) Duration() string {
lastTime := c.record.currentTimestamp
if c.record.CompletedAt != nil {
tm := c.record.CompletedAt.AsTime()
lastTime = &tm
}
if lastTime == nil {
return ""
}
v := formatDuration(lastTime.Sub(c.record.CreatedAt.AsTime()))
if c.record.CompletedAt == nil {
v += "+"
}
return v
}
func (c *lsContext) Link() string {
url := desktop.BuildURL(c.FullRef())
if c.format.IsTable() {
if c.isTerm {
return desktop.ANSIHyperlink(url, "Open")
}
return ""
}
return url
}

View File

@@ -1,80 +0,0 @@
package history
import (
"context"
"fmt"
"slices"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/desktop"
"github.com/docker/cli/cli/command"
"github.com/pkg/browser"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type openOptions struct {
builder string
ref string
}
func runOpen(ctx context.Context, dockerCli command.Cli, opts openOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
recs, err := queryRecords(ctx, opts.ref, nodes)
if err != nil {
return err
}
if len(recs) == 0 {
if opts.ref == "" {
return errors.New("no records found")
}
return errors.Errorf("no record found for ref %q", opts.ref)
}
if opts.ref == "" {
slices.SortFunc(recs, func(a, b historyRecord) int {
return b.CreatedAt.AsTime().Compare(a.CreatedAt.AsTime())
})
}
rec := &recs[0]
url := desktop.BuildURL(fmt.Sprintf("%s/%s/%s", rec.node.Builder, rec.node.Name, rec.Ref))
return browser.OpenURL(url)
}
func openCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options openOptions
cmd := &cobra.Command{
Use: "open [OPTIONS] [REF]",
Short: "Open a build in Docker Desktop",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) > 0 {
options.ref = args[0]
}
options.builder = *rootOpts.Builder
return runOpen(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
return cmd
}

View File

@@ -1,151 +0,0 @@
package history
import (
"context"
"io"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/hashicorp/go-multierror"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type rmOptions struct {
builder string
refs []string
all bool
}
func runRm(ctx context.Context, dockerCli command.Cli, opts rmOptions) error {
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
errs := make([][]error, len(opts.refs))
for i := range errs {
errs[i] = make([]error, len(nodes))
}
eg, ctx := errgroup.WithContext(ctx)
for i, node := range nodes {
node := node
eg.Go(func() error {
if node.Driver == nil {
return nil
}
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
refs := opts.refs
if opts.all {
serv, err := c.ControlClient().ListenBuildHistory(ctx, &controlapi.BuildHistoryRequest{
EarlyExit: true,
})
if err != nil {
return err
}
defer serv.CloseSend()
for {
resp, err := serv.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return err
}
if resp.Type == controlapi.BuildHistoryEventType_COMPLETE {
refs = append(refs, resp.Record.Ref)
}
}
}
for j, ref := range refs {
_, err = c.ControlClient().UpdateBuildHistory(ctx, &controlapi.UpdateBuildHistoryRequest{
Ref: ref,
Delete: true,
})
if opts.all {
if err != nil {
return err
}
} else {
errs[j][i] = err
}
}
return nil
})
}
if err := eg.Wait(); err != nil {
return err
}
var out []error
loop0:
for _, nodeErrs := range errs {
var nodeErr error
for _, err1 := range nodeErrs {
if err1 == nil {
continue loop0
}
if nodeErr == nil {
nodeErr = err1
} else {
nodeErr = multierror.Append(nodeErr, err1)
}
}
out = append(out, nodeErr)
}
if len(out) == 0 {
return nil
}
if len(out) == 1 {
return out[0]
}
return multierror.Append(out[0], out[1:]...)
}
func rmCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options rmOptions
cmd := &cobra.Command{
Use: "rm [OPTIONS] [REF...]",
Short: "Remove build records",
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 && !options.all {
return errors.New("rm requires at least one argument")
}
if len(args) > 0 && options.all {
return errors.New("rm requires either --all or at least one argument")
}
options.refs = args
options.builder = *rootOpts.Builder
return runRm(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.BoolVar(&options.all, "all", false, "Remove all build records")
return cmd
}

View File

@@ -1,30 +0,0 @@
package history
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
type RootOptions struct {
Builder *string
}
func RootCmd(rootcmd *cobra.Command, dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "history",
Short: "Commands to work on build records",
ValidArgsFunction: completion.Disable,
RunE: rootcmd.RunE,
}
cmd.AddCommand(
lsCmd(dockerCli, opts),
rmCmd(dockerCli, opts),
logsCmd(dockerCli, opts),
inspectCmd(dockerCli, opts),
openCmd(dockerCli, opts),
)
return cmd
}

View File

@@ -1,180 +0,0 @@
package history
import (
"context"
"fmt"
"io"
"path/filepath"
"strings"
"sync"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
controlapi "github.com/moby/buildkit/api/services/control"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
func buildName(fattrs map[string]string, ls *localstate.State) string {
var res string
var target, contextPath, dockerfilePath, vcsSource string
if v, ok := fattrs["target"]; ok {
target = v
}
if v, ok := fattrs["context"]; ok {
contextPath = filepath.ToSlash(v)
} else if v, ok := fattrs["vcs:localdir:context"]; ok && v != "." {
contextPath = filepath.ToSlash(v)
}
if v, ok := fattrs["vcs:source"]; ok {
vcsSource = v
}
if v, ok := fattrs["filename"]; ok && v != "Dockerfile" {
dockerfilePath = filepath.ToSlash(v)
}
if v, ok := fattrs["vcs:localdir:dockerfile"]; ok && v != "." {
dockerfilePath = filepath.ToSlash(filepath.Join(v, dockerfilePath))
}
var localPath string
if ls != nil && !build.IsRemoteURL(ls.LocalPath) {
if ls.LocalPath != "" && ls.LocalPath != "-" {
localPath = filepath.ToSlash(ls.LocalPath)
}
if ls.DockerfilePath != "" && ls.DockerfilePath != "-" && ls.DockerfilePath != "Dockerfile" {
dockerfilePath = filepath.ToSlash(ls.DockerfilePath)
}
}
// remove default dockerfile name
const defaultFilename = "/Dockerfile"
hasDefaultFileName := strings.HasSuffix(dockerfilePath, defaultFilename) || dockerfilePath == ""
dockerfilePath = strings.TrimSuffix(dockerfilePath, defaultFilename)
// dockerfile is a subpath of context
if strings.HasPrefix(dockerfilePath, localPath) && len(dockerfilePath) > len(localPath) {
res = dockerfilePath[strings.LastIndex(localPath, "/")+1:]
} else {
// Otherwise, use basename
bpath := localPath
if len(dockerfilePath) > 0 {
bpath = dockerfilePath
}
if len(bpath) > 0 {
lidx := strings.LastIndex(bpath, "/")
res = bpath[lidx+1:]
if !hasDefaultFileName {
if lidx != -1 {
res = filepath.ToSlash(filepath.Join(filepath.Base(bpath[:lidx]), res))
} else {
res = filepath.ToSlash(filepath.Join(filepath.Base(bpath), res))
}
}
}
}
if len(contextPath) > 0 {
res = contextPath
}
if len(target) > 0 {
if len(res) > 0 {
res = res + " (" + target + ")"
} else {
res = target
}
}
if res == "" && vcsSource != "" {
return vcsSource
}
return res
}
func trimBeginning(s string, n int) string {
if len(s) <= n {
return s
}
return ".." + s[len(s)-n+2:]
}
type historyRecord struct {
*controlapi.BuildHistoryRecord
currentTimestamp *time.Time
node *builder.Node
name string
}
func queryRecords(ctx context.Context, ref string, nodes []builder.Node) ([]historyRecord, error) {
var mu sync.Mutex
var out []historyRecord
eg, ctx := errgroup.WithContext(ctx)
for _, node := range nodes {
node := node
eg.Go(func() error {
if node.Driver == nil {
return nil
}
var records []historyRecord
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
serv, err := c.ControlClient().ListenBuildHistory(ctx, &controlapi.BuildHistoryRequest{
EarlyExit: true,
Ref: ref,
})
if err != nil {
return err
}
md, err := serv.Header()
if err != nil {
return err
}
var ts *time.Time
if v, ok := md[headerKeyTimestamp]; ok {
t, err := time.Parse(time.RFC3339Nano, v[0])
if err != nil {
return err
}
ts = &t
}
defer serv.CloseSend()
for {
he, err := serv.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
break
}
return err
}
if he.Type == controlapi.BuildHistoryEventType_DELETED || he.Record == nil {
continue
}
records = append(records, historyRecord{
BuildHistoryRecord: he.Record,
currentTimestamp: ts,
node: &node,
})
}
mu.Lock()
out = append(out, records...)
mu.Unlock()
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
return out, nil
}
func formatDuration(d time.Duration) string {
if d < time.Minute {
return fmt.Sprintf("%.1fs", d.Seconds())
}
return fmt.Sprintf("%dm %2ds", int(d.Minutes()), int(d.Seconds())%60)
}

View File

@@ -42,7 +42,7 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
return errors.Errorf("can't push with no tags specified, please set --tag or --dry-run")
}
fileArgs := make([]string, len(in.files), len(in.files)+len(args))
fileArgs := make([]string, len(in.files))
for i, f := range in.files {
dt, err := os.ReadFile(f)
if err != nil {
@@ -173,8 +173,8 @@ func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, arg
// new resolver cause need new auth
r = imagetools.New(imageopt)
ctx2, cancel := context.WithCancelCause(context.TODO())
defer func() { cancel(errors.WithStack(context.Canceled)) }()
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressui.DisplayMode(in.progress))
if err != nil {
return err

View File

@@ -10,12 +10,11 @@ type RootOptions struct {
Builder *string
}
func RootCmd(rootcmd *cobra.Command, dockerCli command.Cli, opts RootOptions) *cobra.Command {
func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "imagetools",
Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable,
RunE: rootcmd.RunE,
}
cmd.AddCommand(

View File

@@ -17,7 +17,6 @@ import (
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/docker/go-units"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
@@ -35,9 +34,8 @@ func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) e
return err
}
timeoutCtx, cancel := context.WithCancelCause(ctx)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
defer func() { cancel(errors.WithStack(context.Canceled)) }()
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if in.bootstrap {
@@ -124,20 +122,8 @@ func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) e
if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
}
if rule.ReservedSpace > 0 {
fmt.Fprintf(w, "\tReserved Space:\t%s\n", units.BytesSize(float64(rule.ReservedSpace)))
}
if rule.MaxUsedSpace > 0 {
fmt.Fprintf(w, "\tMax Used Space:\t%s\n", units.BytesSize(float64(rule.MaxUsedSpace)))
}
if rule.MinFreeSpace > 0 {
fmt.Fprintf(w, "\tMin Free Space:\t%s\n", units.BytesSize(float64(rule.MinFreeSpace)))
}
}
for f, dt := range nodes[i].Files {
fmt.Fprintf(w, "File#%s:\n", f)
for _, line := range strings.Split(string(dt), "\n") {
fmt.Fprintf(w, "\t> %s\n", line)
if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
}
}

View File

@@ -8,7 +8,6 @@ import (
"strings"
"time"
"github.com/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
@@ -18,7 +17,6 @@ import (
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/command/formatter"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
@@ -37,8 +35,7 @@ const (
)
type lsOptions struct {
format string
noTrunc bool
format string
}
func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
@@ -58,9 +55,8 @@ func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
return err
}
timeoutCtx, cancel := context.WithCancelCause(ctx)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
defer func() { cancel(errors.WithStack(context.Canceled)) }()
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {
@@ -76,7 +72,7 @@ func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
return err
}
if hasErrors, err := lsPrint(dockerCli, current, builders, in); err != nil {
if hasErrors, err := lsPrint(dockerCli, current, builders, in.format); err != nil {
return err
} else if hasErrors {
_, _ = fmt.Fprintf(dockerCli.Err(), "\n")
@@ -111,7 +107,6 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output")
flags.BoolVar(&options.noTrunc, "no-trunc", false, "Don't truncate output")
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
@@ -119,15 +114,14 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
return cmd
}
func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builder.Builder, in lsOptions) (hasErrors bool, _ error) {
if in.format == formatter.TableFormatKey {
in.format = lsDefaultTableFormat
func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builder.Builder, format string) (hasErrors bool, _ error) {
if format == formatter.TableFormatKey {
format = lsDefaultTableFormat
}
ctx := formatter.Context{
Output: dockerCli.Out(),
Format: formatter.Format(in.format),
Trunc: !in.noTrunc,
Format: formatter.Format(format),
}
sort.SliceStable(builders, func(i, j int) bool {
@@ -144,12 +138,11 @@ func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builde
render := func(format func(subContext formatter.SubContext) error) error {
for _, b := range builders {
if err := format(&lsContext{
format: ctx.Format,
trunc: ctx.Trunc,
Builder: &lsBuilder{
Builder: b,
Current: b.Name == current.Name,
},
format: ctx.Format,
}); err != nil {
return err
}
@@ -167,7 +160,6 @@ func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builde
}
if err := format(&lsContext{
format: ctx.Format,
trunc: ctx.Trunc,
Builder: &lsBuilder{
Builder: b,
Current: b.Name == current.Name,
@@ -204,7 +196,6 @@ type lsContext struct {
Builder *lsBuilder
format formatter.Format
trunc bool
node builder.Node
}
@@ -270,11 +261,7 @@ func (c *lsContext) Platforms() string {
if c.node.Name == "" {
return ""
}
pfs := platformutil.FormatInGroups(c.node.Node.Platforms, c.node.Platforms)
if c.trunc && c.format.IsTable() {
return truncPlatforms(pfs, 4).String()
}
return strings.Join(pfs, ", ")
return strings.Join(platformutil.FormatInGroups(c.node.Node.Platforms, c.node.Platforms), ", ")
}
func (c *lsContext) Error() string {
@@ -285,133 +272,3 @@ func (c *lsContext) Error() string {
}
return ""
}
var truncMajorPlatforms = []string{
"linux/amd64",
"linux/arm64",
"linux/arm",
"linux/ppc64le",
"linux/s390x",
"linux/riscv64",
"linux/mips64",
}
type truncatedPlatforms struct {
res map[string][]string
input []string
max int
}
func (tp truncatedPlatforms) List() map[string][]string {
return tp.res
}
func (tp truncatedPlatforms) String() string {
var out []string
var count int
var keys []string
for k := range tp.res {
keys = append(keys, k)
}
sort.Strings(keys)
seen := make(map[string]struct{})
for _, mpf := range truncMajorPlatforms {
if tpf, ok := tp.res[mpf]; ok {
seen[mpf] = struct{}{}
if len(tpf) == 1 {
out = append(out, tpf[0])
count++
} else {
hasPreferredPlatform := false
for _, pf := range tpf {
if strings.HasSuffix(pf, "*") {
hasPreferredPlatform = true
break
}
}
mainpf := mpf
if hasPreferredPlatform {
mainpf += "*"
}
out = append(out, fmt.Sprintf("%s (+%d)", mainpf, len(tpf)))
count += len(tpf)
}
}
}
for _, mpf := range keys {
if len(out) >= tp.max {
break
}
if _, ok := seen[mpf]; ok {
continue
}
if len(tp.res[mpf]) == 1 {
out = append(out, tp.res[mpf][0])
count++
} else {
hasPreferredPlatform := false
for _, pf := range tp.res[mpf] {
if strings.HasSuffix(pf, "*") {
hasPreferredPlatform = true
break
}
}
mainpf := mpf
if hasPreferredPlatform {
mainpf += "*"
}
out = append(out, fmt.Sprintf("%s (+%d)", mainpf, len(tp.res[mpf])))
count += len(tp.res[mpf])
}
}
left := len(tp.input) - count
if left > 0 {
out = append(out, fmt.Sprintf("(%d more)", left))
}
return strings.Join(out, ", ")
}
func truncPlatforms(pfs []string, max int) truncatedPlatforms {
res := make(map[string][]string)
for _, mpf := range truncMajorPlatforms {
for _, pf := range pfs {
if len(res) >= max {
break
}
pp, err := platforms.Parse(strings.TrimSuffix(pf, "*"))
if err != nil {
continue
}
if pp.OS+"/"+pp.Architecture == mpf {
res[mpf] = append(res[mpf], pf)
}
}
}
left := make(map[string][]string)
for _, pf := range pfs {
if len(res) >= max {
break
}
pp, err := platforms.Parse(strings.TrimSuffix(pf, "*"))
if err != nil {
continue
}
ppf := strings.TrimSuffix(pp.OS+"/"+pp.Architecture, "*")
if _, ok := res[ppf]; !ok {
left[ppf] = append(left[ppf], pf)
}
}
for k, v := range left {
res[k] = v
}
return truncatedPlatforms{
res: res,
input: pfs,
max: max,
}
}

View File

@@ -1,174 +0,0 @@
package commands
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestTruncPlatforms(t *testing.T) {
tests := []struct {
name string
platforms []string
max int
expectedList map[string][]string
expectedOut string
}{
{
name: "arm64 preferred and emulated",
platforms: []string{"linux/arm64*", "linux/amd64", "linux/amd64/v2", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/386", "linux/mips64le", "linux/mips64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64",
"linux/amd64/v2",
},
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64*",
},
"linux/ppc64le": {
"linux/ppc64le",
},
},
expectedOut: "linux/amd64 (+2), linux/arm64*, linux/arm (+2), linux/ppc64le, (5 more)",
},
{
name: "riscv64 preferred only",
platforms: []string{"linux/riscv64*"},
max: 4,
expectedList: map[string][]string{
"linux/riscv64": {
"linux/riscv64*",
},
},
expectedOut: "linux/riscv64*",
},
{
name: "amd64 no preferred and emulated",
platforms: []string{"linux/amd64", "linux/amd64/v2", "linux/amd64/v3", "linux/386", "linux/arm64", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/mips64le", "linux/mips64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64",
"linux/amd64/v2",
"linux/amd64/v3",
},
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
"linux/ppc64le": {
"linux/ppc64le",
}},
expectedOut: "linux/amd64 (+3), linux/arm64, linux/arm (+2), linux/ppc64le, (5 more)",
},
{
name: "amd64 no preferred",
platforms: []string{"linux/amd64", "linux/386"},
max: 4,
expectedList: map[string][]string{
"linux/386": {
"linux/386",
},
"linux/amd64": {
"linux/amd64",
},
},
expectedOut: "linux/amd64, linux/386",
},
{
name: "arm64 no preferred",
platforms: []string{"linux/arm64", "linux/arm/v7", "linux/arm/v6"},
max: 4,
expectedList: map[string][]string{
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
},
expectedOut: "linux/arm64, linux/arm (+2)",
},
{
name: "all preferred",
platforms: []string{"darwin/arm64*", "linux/arm64*", "linux/arm/v5*", "linux/arm/v6*", "linux/arm/v7*", "windows/arm64*"},
max: 4,
expectedList: map[string][]string{
"darwin/arm64": {
"darwin/arm64*",
},
"linux/arm": {
"linux/arm/v5*",
"linux/arm/v6*",
"linux/arm/v7*",
},
"linux/arm64": {
"linux/arm64*",
},
"windows/arm64": {
"windows/arm64*",
},
},
expectedOut: "linux/arm64*, linux/arm* (+3), darwin/arm64*, windows/arm64*",
},
{
name: "no major preferred",
platforms: []string{"linux/amd64/v2*", "linux/arm/v6*", "linux/mips64le*", "linux/amd64", "linux/amd64/v3", "linux/386", "linux/arm64", "linux/riscv64", "linux/ppc64le", "linux/s390x", "linux/mips64", "linux/arm/v7"},
max: 4,
expectedList: map[string][]string{
"linux/amd64": {
"linux/amd64/v2*",
"linux/amd64",
"linux/amd64/v3",
},
"linux/arm": {
"linux/arm/v6*",
"linux/arm/v7",
},
"linux/arm64": {
"linux/arm64",
},
"linux/ppc64le": {
"linux/ppc64le",
},
},
expectedOut: "linux/amd64* (+3), linux/arm64, linux/arm* (+2), linux/ppc64le, (5 more)",
},
{
name: "no major with multiple variants",
platforms: []string{"linux/arm64", "linux/arm/v7", "linux/arm/v6", "linux/mips64le/softfloat", "linux/mips64le/hardfloat"},
max: 4,
expectedList: map[string][]string{
"linux/arm": {
"linux/arm/v7",
"linux/arm/v6",
},
"linux/arm64": {
"linux/arm64",
},
"linux/mips64le": {
"linux/mips64le/softfloat",
"linux/mips64le/hardfloat",
},
},
expectedOut: "linux/arm64, linux/arm (+2), linux/mips64le (+2)",
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
tpfs := truncPlatforms(tt.platforms, tt.max)
assert.Equal(t, tt.expectedList, tpfs.List())
assert.Equal(t, tt.expectedOut, tpfs.String())
})
}
}

View File

@@ -16,23 +16,18 @@ import (
"github.com/docker/docker/api/types/filters"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
gateway "github.com/moby/buildkit/frontend/gateway/client"
pb "github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/util/apicaps"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type pruneOptions struct {
builder string
all bool
filter opts.FilterOpt
reservedSpace opts.MemBytes
maxUsedSpace opts.MemBytes
minFreeSpace opts.MemBytes
force bool
verbose bool
builder string
all bool
filter opts.FilterOpt
keepStorage opts.MemBytes
force bool
verbose bool
}
const (
@@ -110,19 +105,8 @@ func runPrune(ctx context.Context, dockerCli command.Cli, opts pruneOptions) err
if err != nil {
return err
}
// check if the client supports newer prune options
if opts.maxUsedSpace.Value() != 0 || opts.minFreeSpace.Value() != 0 {
caps, err := loadLLBCaps(ctx, c)
if err != nil {
return errors.Wrap(err, "failed to load buildkit capabilities for prune")
}
if caps.Supports(pb.CapGCFreeSpaceFilter) != nil {
return errors.New("buildkit v0.17.0+ is required for max-used-space and min-free-space filters")
}
}
popts := []client.PruneOption{
client.WithKeepOpt(pi.KeepDuration, opts.reservedSpace.Value(), opts.maxUsedSpace.Value(), opts.minFreeSpace.Value()),
client.WithKeepOpt(pi.KeepDuration, opts.keepStorage.Value()),
client.WithFilter(pi.Filter),
}
if opts.all {
@@ -147,17 +131,6 @@ func runPrune(ctx context.Context, dockerCli command.Cli, opts pruneOptions) err
return nil
}
func loadLLBCaps(ctx context.Context, c *client.Client) (apicaps.CapSet, error) {
var caps apicaps.CapSet
_, err := c.Build(ctx, client.SolveOpt{
Internal: true,
}, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
caps = c.BuildOpts().LLBCaps
return nil, nil
}, nil)
return caps, err
}
func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options := pruneOptions{filter: opts.NewFilterOpt()}
@@ -175,15 +148,10 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags := cmd.Flags()
flags.BoolVarP(&options.all, "all", "a", false, "Include internal/frontend images")
flags.Var(&options.filter, "filter", `Provide filter values (e.g., "until=24h")`)
flags.Var(&options.reservedSpace, "reserved-space", "Amount of disk space always allowed to keep for cache")
flags.Var(&options.minFreeSpace, "min-free-space", "Target amount of free disk space after pruning")
flags.Var(&options.maxUsedSpace, "max-used-space", "Maximum amount of disk space allowed to keep for cache")
flags.Var(&options.keepStorage, "keep-storage", "Amount of disk space to keep for cache")
flags.BoolVar(&options.verbose, "verbose", false, "Provide a more verbose output")
flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation")
flags.Var(&options.reservedSpace, "keep-storage", "Amount of disk space to keep for cache")
flags.MarkDeprecated("keep-storage", "keep-storage flag has been changed to max-storage")
return cmd
}

View File

@@ -150,9 +150,8 @@ func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, i
return err
}
timeoutCtx, cancel := context.WithCancelCause(ctx)
timeoutCtx, _ = context.WithTimeoutCause(timeoutCtx, 20*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
defer func() { cancel(errors.WithStack(context.Canceled)) }()
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {

View File

@@ -1,11 +1,9 @@
package commands
import (
"fmt"
"os"
debugcmd "github.com/docker/buildx/commands/debug"
historycmd "github.com/docker/buildx/commands/history"
imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion"
@@ -38,22 +36,13 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
if opt.debug {
debug.Enable()
}
cmd.SetContext(appcontext.Context())
if !isPlugin {
return nil
}
return plugin.PersistentPreRunE(cmd, args)
},
RunE: func(cmd *cobra.Command, args []string) error {
if len(args) == 0 {
return cmd.Help()
}
_ = cmd.Help()
return cli.StatusError{
StatusCode: 1,
Status: fmt.Sprintf("ERROR: unknown command: %q", args[0]),
}
},
}
if !isPlugin {
// match plugin behavior for standalone mode
@@ -106,8 +95,7 @@ func addCommands(cmd *cobra.Command, opts *rootOptions, dockerCli command.Cli) {
versionCmd(dockerCli),
pruneCmd(dockerCli, opts),
duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(cmd, dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
historycmd.RootCmd(cmd, dockerCli, historycmd.RootOptions{Builder: &opts.builder}),
imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
)
if confutil.IsExperimental() {
cmd.AddCommand(debugcmd.RootCmd(dockerCli,

View File

@@ -46,6 +46,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Errorf("run `docker context use %s` to switch to context %s", in.builder, in.builder)
}
}
}
return errors.Wrapf(err, "failed to find instance %q", in.builder)
}

View File

@@ -34,9 +34,9 @@ const defaultTargetName = "default"
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, *build.Inputs, error) {
func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
}
contexts := map[string]build.NamedContext{}
@@ -70,7 +70,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
platforms, err := platformutil.Parse(in.Platforms)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
opts.Platforms = platforms
@@ -79,7 +79,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
opts.Session = append(opts.Session, secrets)
@@ -89,13 +89,13 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
}
ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
opts.Session = append(opts.Session, ssh)
outputs, _, err := controllerapi.CreateExports(in.Exports)
outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
if in.ExportPush {
var pushUsed bool
@@ -134,7 +134,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
annotations, err := buildflags.ParseAnnotations(in.Annotations)
if err != nil {
return nil, nil, nil, errors.Wrap(err, "parse annotations")
return nil, nil, errors.Wrap(err, "parse annotations")
}
for _, o := range outputs {
@@ -154,7 +154,7 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
opts.Allow = allow
@@ -178,28 +178,23 @@ func RunBuild(ctx context.Context, dockerCli command.Cli, in *controllerapi.Buil
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return nil, nil, nil, err
return nil, nil, err
}
var inputs *build.Inputs
buildOptions := map[string]build.Options{defaultTargetName: opts}
resp, res, err := buildTargets(ctx, dockerCli, nodes, buildOptions, progress, generateResult)
resp, res, err := buildTargets(ctx, dockerCli, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
err = wrapBuildError(err, false)
if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, nil, err
return nil, res, err
}
if i, ok := buildOptions[defaultTargetName]; ok {
inputs = &i.Inputs
}
return resp, res, inputs, nil
return resp, res, nil
}
// buildTargets runs the specified build and returns the result.
@@ -214,7 +209,7 @@ func buildTargets(ctx context.Context, dockerCli command.Cli, nodes []builder.No
if generateResult {
var mu sync.Mutex
var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock()
defer mu.Unlock()
if res == nil || driverIndex < idx {
@@ -222,7 +217,7 @@ func buildTargets(ctx context.Context, dockerCli command.Cli, nodes []builder.No
}
})
} else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.NewConfig(dockerCli), progress)
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
}
if err != nil {
return nil, res, err

View File

@@ -4,19 +4,18 @@ import (
"context"
"io"
"github.com/docker/buildx/build"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
)
type BuildxController interface {
Build(ctx context.Context, options *controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, inputs *build.Inputs, err error)
Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process.
// If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options *controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error
Close() error
List(ctx context.Context) (refs []string, _ error)

View File

@@ -1,10 +1,7 @@
package errdefs
import (
"io"
"github.com/containerd/typeurl/v2"
"github.com/docker/buildx/util/desktop"
"github.com/moby/buildkit/util/grpcerrors"
)
@@ -13,7 +10,7 @@ func init() {
}
type BuildError struct {
*Build
Build
error
}
@@ -22,27 +19,16 @@ func (e *BuildError) Unwrap() error {
}
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return e.Build
return &e.Build
}
func (e *BuildError) PrintBuildDetails(w io.Writer) error {
if e.Ref == "" {
return nil
}
ebr := &desktop.ErrorWithBuildRef{
Ref: e.Ref,
Err: e.error,
}
return ebr.Print(w)
}
func WrapBuild(err error, sessionID string, ref string) error {
func WrapBuild(err error, ref string) error {
if err == nil {
return nil
}
return &BuildError{Build: &Build{SessionID: sessionID, Ref: ref}, error: err}
return &BuildError{Build: Build{Ref: ref}, error: err}
}
func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: b}
return &BuildError{error: err, Build: *b}
}

View File

@@ -1,157 +1,77 @@
// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// protoc-gen-go v1.34.1
// protoc v3.11.4
// source: github.com/docker/buildx/controller/errdefs/errdefs.proto
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: errdefs.proto
package errdefs
import (
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
fmt "fmt"
proto "github.com/gogo/protobuf/proto"
_ "github.com/moby/buildkit/solver/pb"
math "math"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
SessionID string `protobuf:"bytes,1,opt,name=SessionID,proto3" json:"SessionID,omitempty"`
Ref string `protobuf:"bytes,2,opt,name=Ref,proto3" json:"Ref,omitempty"`
Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (x *Build) Reset() {
*x = Build{}
if protoimpl.UnsafeEnabled {
mi := &file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0]
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
ms.StoreMessageInfo(mi)
}
}
func (x *Build) String() string {
return protoimpl.X.MessageStringOf(x)
}
func (*Build) ProtoMessage() {}
func (x *Build) ProtoReflect() protoreflect.Message {
mi := &file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0]
if protoimpl.UnsafeEnabled && x != nil {
ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
if ms.LoadMessageInfo() == nil {
ms.StoreMessageInfo(mi)
}
return ms
}
return mi.MessageOf(x)
}
// Deprecated: Use Build.ProtoReflect.Descriptor instead.
func (m *Build) Reset() { *m = Build{} }
func (m *Build) String() string { return proto.CompactTextString(m) }
func (*Build) ProtoMessage() {}
func (*Build) Descriptor() ([]byte, []int) {
return file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescGZIP(), []int{0}
return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
}
func (x *Build) GetSessionID() string {
if x != nil {
return x.SessionID
var xxx_messageInfo_Build proto.InternalMessageInfo
func (m *Build) GetRef() string {
if m != nil {
return m.Ref
}
return ""
}
func (x *Build) GetRef() string {
if x != nil {
return x.Ref
}
return ""
func init() {
proto.RegisterType((*Build)(nil), "errdefs.Build")
}
var File_github_com_docker_buildx_controller_errdefs_errdefs_proto protoreflect.FileDescriptor
func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc = []byte{
0x0a, 0x39, 0x67, 0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x6f, 0x63,
0x6b, 0x65, 0x72, 0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2f, 0x63, 0x6f, 0x6e, 0x74, 0x72,
0x6f, 0x6c, 0x6c, 0x65, 0x72, 0x2f, 0x65, 0x72, 0x72, 0x64, 0x65, 0x66, 0x73, 0x2f, 0x65, 0x72,
0x72, 0x64, 0x65, 0x66, 0x73, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x15, 0x64, 0x6f, 0x63,
0x6b, 0x65, 0x72, 0x2e, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2e, 0x65, 0x72, 0x72, 0x64, 0x65,
0x66, 0x73, 0x22, 0x37, 0x0a, 0x05, 0x42, 0x75, 0x69, 0x6c, 0x64, 0x12, 0x1c, 0x0a, 0x09, 0x53,
0x65, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x09,
0x53, 0x65, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x44, 0x12, 0x10, 0x0a, 0x03, 0x52, 0x65, 0x66,
0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x52, 0x65, 0x66, 0x42, 0x2d, 0x5a, 0x2b, 0x67,
0x69, 0x74, 0x68, 0x75, 0x62, 0x2e, 0x63, 0x6f, 0x6d, 0x2f, 0x64, 0x6f, 0x63, 0x6b, 0x65, 0x72,
0x2f, 0x62, 0x75, 0x69, 0x6c, 0x64, 0x78, 0x2f, 0x63, 0x6f, 0x6e, 0x74, 0x72, 0x6f, 0x6c, 0x6c,
0x65, 0x72, 0x2f, 0x65, 0x72, 0x72, 0x64, 0x65, 0x66, 0x73, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74,
0x6f, 0x33,
}
var (
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescOnce sync.Once
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData = file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc
)
func file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescGZIP() []byte {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescOnce.Do(func() {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData = protoimpl.X.CompressGZIP(file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData)
})
return file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDescData
}
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes = make([]protoimpl.MessageInfo, 1)
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes = []interface{}{
(*Build)(nil), // 0: docker.buildx.errdefs.Build
}
var file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs = []int32{
0, // [0:0] is the sub-list for method output_type
0, // [0:0] is the sub-list for method input_type
0, // [0:0] is the sub-list for extension type_name
0, // [0:0] is the sub-list for extension extendee
0, // [0:0] is the sub-list for field type_name
}
func init() { file_github_com_docker_buildx_controller_errdefs_errdefs_proto_init() }
func file_github_com_docker_buildx_controller_errdefs_errdefs_proto_init() {
if File_github_com_docker_buildx_controller_errdefs_errdefs_proto != nil {
return
}
if !protoimpl.UnsafeEnabled {
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} {
switch v := v.(*Build); i {
case 0:
return &v.state
case 1:
return &v.sizeCache
case 2:
return &v.unknownFields
default:
return nil
}
}
}
type x struct{}
out := protoimpl.TypeBuilder{
File: protoimpl.DescBuilder{
GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
RawDescriptor: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc,
NumEnums: 0,
NumMessages: 1,
NumExtensions: 0,
NumServices: 0,
},
GoTypes: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes,
DependencyIndexes: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs,
MessageInfos: file_github_com_docker_buildx_controller_errdefs_errdefs_proto_msgTypes,
}.Build()
File_github_com_docker_buildx_controller_errdefs_errdefs_proto = out.File
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_rawDesc = nil
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_goTypes = nil
file_github_com_docker_buildx_controller_errdefs_errdefs_proto_depIdxs = nil
var fileDescriptor_689dc58a5060aff5 = []byte{
// 111 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
}

View File

@@ -1,10 +1,9 @@
syntax = "proto3";
package docker.buildx.errdefs;
package errdefs;
option go_package = "github.com/docker/buildx/controller/errdefs";
import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build {
string SessionID = 1;
string Ref = 2;
}
string Ref = 1;
}

View File

@@ -1,241 +0,0 @@
// Code generated by protoc-gen-go-vtproto. DO NOT EDIT.
// protoc-gen-go-vtproto version: v0.6.1-0.20240319094008-0393e58bdf10
// source: github.com/docker/buildx/controller/errdefs/errdefs.proto
package errdefs
import (
fmt "fmt"
protohelpers "github.com/planetscale/vtprotobuf/protohelpers"
proto "google.golang.org/protobuf/proto"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
io "io"
)
const (
// Verify that this generated code is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion)
// Verify that runtime/protoimpl is sufficiently up-to-date.
_ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20)
)
func (m *Build) CloneVT() *Build {
if m == nil {
return (*Build)(nil)
}
r := new(Build)
r.SessionID = m.SessionID
r.Ref = m.Ref
if len(m.unknownFields) > 0 {
r.unknownFields = make([]byte, len(m.unknownFields))
copy(r.unknownFields, m.unknownFields)
}
return r
}
func (m *Build) CloneMessageVT() proto.Message {
return m.CloneVT()
}
func (this *Build) EqualVT(that *Build) bool {
if this == that {
return true
} else if this == nil || that == nil {
return false
}
if this.SessionID != that.SessionID {
return false
}
if this.Ref != that.Ref {
return false
}
return string(this.unknownFields) == string(that.unknownFields)
}
func (this *Build) EqualMessageVT(thatMsg proto.Message) bool {
that, ok := thatMsg.(*Build)
if !ok {
return false
}
return this.EqualVT(that)
}
func (m *Build) MarshalVT() (dAtA []byte, err error) {
if m == nil {
return nil, nil
}
size := m.SizeVT()
dAtA = make([]byte, size)
n, err := m.MarshalToSizedBufferVT(dAtA[:size])
if err != nil {
return nil, err
}
return dAtA[:n], nil
}
func (m *Build) MarshalToVT(dAtA []byte) (int, error) {
size := m.SizeVT()
return m.MarshalToSizedBufferVT(dAtA[:size])
}
func (m *Build) MarshalToSizedBufferVT(dAtA []byte) (int, error) {
if m == nil {
return 0, nil
}
i := len(dAtA)
_ = i
var l int
_ = l
if m.unknownFields != nil {
i -= len(m.unknownFields)
copy(dAtA[i:], m.unknownFields)
}
if len(m.Ref) > 0 {
i -= len(m.Ref)
copy(dAtA[i:], m.Ref)
i = protohelpers.EncodeVarint(dAtA, i, uint64(len(m.Ref)))
i--
dAtA[i] = 0x12
}
if len(m.SessionID) > 0 {
i -= len(m.SessionID)
copy(dAtA[i:], m.SessionID)
i = protohelpers.EncodeVarint(dAtA, i, uint64(len(m.SessionID)))
i--
dAtA[i] = 0xa
}
return len(dAtA) - i, nil
}
func (m *Build) SizeVT() (n int) {
if m == nil {
return 0
}
var l int
_ = l
l = len(m.SessionID)
if l > 0 {
n += 1 + l + protohelpers.SizeOfVarint(uint64(l))
}
l = len(m.Ref)
if l > 0 {
n += 1 + l + protohelpers.SizeOfVarint(uint64(l))
}
n += len(m.unknownFields)
return n
}
func (m *Build) UnmarshalVT(dAtA []byte) error {
l := len(dAtA)
iNdEx := 0
for iNdEx < l {
preIndex := iNdEx
var wire uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
wire |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
fieldNum := int32(wire >> 3)
wireType := int(wire & 0x7)
if wireType == 4 {
return fmt.Errorf("proto: Build: wiretype end group for non-group")
}
if fieldNum <= 0 {
return fmt.Errorf("proto: Build: illegal tag %d (wire type %d)", fieldNum, wire)
}
switch fieldNum {
case 1:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field SessionID", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return protohelpers.ErrInvalidLength
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return protohelpers.ErrInvalidLength
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.SessionID = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
case 2:
if wireType != 2 {
return fmt.Errorf("proto: wrong wireType = %d for field Ref", wireType)
}
var stringLen uint64
for shift := uint(0); ; shift += 7 {
if shift >= 64 {
return protohelpers.ErrIntOverflow
}
if iNdEx >= l {
return io.ErrUnexpectedEOF
}
b := dAtA[iNdEx]
iNdEx++
stringLen |= uint64(b&0x7F) << shift
if b < 0x80 {
break
}
}
intStringLen := int(stringLen)
if intStringLen < 0 {
return protohelpers.ErrInvalidLength
}
postIndex := iNdEx + intStringLen
if postIndex < 0 {
return protohelpers.ErrInvalidLength
}
if postIndex > l {
return io.ErrUnexpectedEOF
}
m.Ref = string(dAtA[iNdEx:postIndex])
iNdEx = postIndex
default:
iNdEx = preIndex
skippy, err := protohelpers.Skip(dAtA[iNdEx:])
if err != nil {
return err
}
if (skippy < 0) || (iNdEx+skippy) < 0 {
return protohelpers.ErrInvalidLength
}
if (iNdEx + skippy) > l {
return io.ErrUnexpectedEOF
}
m.unknownFields = append(m.unknownFields, dAtA[iNdEx:iNdEx+skippy]...)
iNdEx += skippy
}
}
if iNdEx > l {
return io.ErrUnexpectedEOF
}
return nil
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -11,7 +11,6 @@ import (
controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
@@ -22,7 +21,7 @@ import (
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{
dockerCli: dockerCli,
sessionID: "local",
ref: "local",
processes: processes.NewManager(),
}
}
@@ -36,51 +35,46 @@ type buildConfig struct {
type localController struct {
dockerCli command.Cli
sessionID string
ref string
buildConfig buildConfig
processes *processes.Manager
buildOnGoing atomic.Bool
}
func (b *localController) Build(ctx context.Context, options *controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, *build.Inputs, error) {
func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, nil, errors.New("build ongoing")
return "", nil, errors.New("build ongoing")
}
defer b.buildOnGoing.Store(false)
resp, res, dockerfileMappings, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil {
b.buildConfig = buildConfig{
resultCtx: res,
buildOptions: options,
buildOptions: &options,
}
if buildErr != nil {
var ref string
var ebr *desktop.ErrorWithBuildRef
if errors.As(buildErr, &ebr) {
ref = ebr.Ref
}
buildErr = controllererrors.WrapBuild(buildErr, b.sessionID, ref)
buildErr = controllererrors.WrapBuild(buildErr, b.ref)
}
}
if buildErr != nil {
return "", nil, nil, buildErr
return "", nil, buildErr
}
return b.sessionID, resp, dockerfileMappings, nil
return b.ref, resp, nil
}
func (b *localController) ListProcesses(ctx context.Context, sessionID string) (infos []*controllerapi.ProcessInfo, retErr error) {
if sessionID != b.sessionID {
return nil, errors.Errorf("unknown session ID %q", sessionID)
func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return b.processes.ListProcesses(), nil
}
func (b *localController) DisconnectProcess(ctx context.Context, sessionID, pid string) error {
if sessionID != b.sessionID {
return errors.Errorf("unknown session ID %q", sessionID)
func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
return b.processes.DeleteProcess(pid)
}
@@ -89,9 +83,9 @@ func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses()
}
func (b *localController) Invoke(ctx context.Context, sessionID string, pid string, cfg *controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if sessionID != b.sessionID {
return errors.Errorf("unknown session ID %q", sessionID)
func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
proc, ok := b.processes.Get(pid)
@@ -101,7 +95,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
return errors.New("no build result is registered")
}
var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, cfg)
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil {
return err
}
@@ -109,7 +103,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
// Attach containerIn to this process
ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func(error) { close(ioCancelledCh) })
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select {
case <-ioCancelledCh:
@@ -117,7 +111,7 @@ func (b *localController) Invoke(ctx context.Context, sessionID string, pid stri
case err := <-proc.Done():
return err
case <-ctx.Done():
return context.Cause(ctx)
return ctx.Err()
}
}
@@ -136,7 +130,7 @@ func (b *localController) Close() error {
}
func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.sessionID}, nil
return []string{b.ref}, nil
}
func (b *localController) Disconnect(ctx context.Context, key string) error {
@@ -144,9 +138,9 @@ func (b *localController) Disconnect(ctx context.Context, key string) error {
return nil
}
func (b *localController) Inspect(ctx context.Context, sessionID string) (*controllerapi.InspectResponse, error) {
if sessionID != b.sessionID {
return nil, errors.Errorf("unknown session ID %q", sessionID)
func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,7 +5,7 @@ package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "github.com/docker/buildx/controller/pb";
option go_package = "pb";
service Controller {
rpc Build(BuildRequest) returns (BuildResponse);
@@ -21,7 +21,7 @@ service Controller {
}
message ListProcessesRequest {
string SessionID = 1;
string Ref = 1;
}
message ListProcessesResponse {
@@ -34,7 +34,7 @@ message ProcessInfo {
}
message DisconnectProcessRequest {
string SessionID = 1;
string Ref = 1;
string ProcessID = 2;
}
@@ -42,7 +42,7 @@ message DisconnectProcessResponse {
}
message BuildRequest {
string SessionID = 1;
string Ref = 1;
BuildOptions Options = 2;
}
@@ -118,7 +118,7 @@ message CallFunc {
}
message InspectRequest {
string SessionID = 1;
string Ref = 1;
}
message InspectResponse {
@@ -140,13 +140,13 @@ message BuildResponse {
}
message DisconnectRequest {
string SessionID = 1;
string Ref = 1;
}
message DisconnectResponse {}
message ListRequest {
string SessionID = 1;
string Ref = 1;
}
message ListResponse {
@@ -161,7 +161,7 @@ message InputMessage {
}
message InputInitMessage {
string SessionID = 1;
string Ref = 1;
}
message DataMessage {
@@ -186,7 +186,7 @@ message Message {
}
message InitMessage {
string SessionID = 1;
string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored.
@@ -227,7 +227,7 @@ message SignalMessage {
}
message StatusRequest {
string SessionID = 1;
string Ref = 1;
}
message StatusResponse {

View File

@@ -1,452 +0,0 @@
// Code generated by protoc-gen-go-grpc. DO NOT EDIT.
// versions:
// - protoc-gen-go-grpc v1.5.1
// - protoc v3.11.4
// source: github.com/docker/buildx/controller/pb/controller.proto
package pb
import (
context "context"
grpc "google.golang.org/grpc"
codes "google.golang.org/grpc/codes"
status "google.golang.org/grpc/status"
)
// This is a compile-time assertion to ensure that this generated file
// is compatible with the grpc package it is being compiled against.
// Requires gRPC-Go v1.64.0 or later.
const _ = grpc.SupportPackageIsVersion9
const (
Controller_Build_FullMethodName = "/buildx.controller.v1.Controller/Build"
Controller_Inspect_FullMethodName = "/buildx.controller.v1.Controller/Inspect"
Controller_Status_FullMethodName = "/buildx.controller.v1.Controller/Status"
Controller_Input_FullMethodName = "/buildx.controller.v1.Controller/Input"
Controller_Invoke_FullMethodName = "/buildx.controller.v1.Controller/Invoke"
Controller_List_FullMethodName = "/buildx.controller.v1.Controller/List"
Controller_Disconnect_FullMethodName = "/buildx.controller.v1.Controller/Disconnect"
Controller_Info_FullMethodName = "/buildx.controller.v1.Controller/Info"
Controller_ListProcesses_FullMethodName = "/buildx.controller.v1.Controller/ListProcesses"
Controller_DisconnectProcess_FullMethodName = "/buildx.controller.v1.Controller/DisconnectProcess"
)
// ControllerClient is the client API for Controller service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type ControllerClient interface {
Build(ctx context.Context, in *BuildRequest, opts ...grpc.CallOption) (*BuildResponse, error)
Inspect(ctx context.Context, in *InspectRequest, opts ...grpc.CallOption) (*InspectResponse, error)
Status(ctx context.Context, in *StatusRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[StatusResponse], error)
Input(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[InputMessage, InputResponse], error)
Invoke(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[Message, Message], error)
List(ctx context.Context, in *ListRequest, opts ...grpc.CallOption) (*ListResponse, error)
Disconnect(ctx context.Context, in *DisconnectRequest, opts ...grpc.CallOption) (*DisconnectResponse, error)
Info(ctx context.Context, in *InfoRequest, opts ...grpc.CallOption) (*InfoResponse, error)
ListProcesses(ctx context.Context, in *ListProcessesRequest, opts ...grpc.CallOption) (*ListProcessesResponse, error)
DisconnectProcess(ctx context.Context, in *DisconnectProcessRequest, opts ...grpc.CallOption) (*DisconnectProcessResponse, error)
}
type controllerClient struct {
cc grpc.ClientConnInterface
}
func NewControllerClient(cc grpc.ClientConnInterface) ControllerClient {
return &controllerClient{cc}
}
func (c *controllerClient) Build(ctx context.Context, in *BuildRequest, opts ...grpc.CallOption) (*BuildResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(BuildResponse)
err := c.cc.Invoke(ctx, Controller_Build_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Inspect(ctx context.Context, in *InspectRequest, opts ...grpc.CallOption) (*InspectResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InspectResponse)
err := c.cc.Invoke(ctx, Controller_Inspect_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Status(ctx context.Context, in *StatusRequest, opts ...grpc.CallOption) (grpc.ServerStreamingClient[StatusResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[0], Controller_Status_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[StatusRequest, StatusResponse]{ClientStream: stream}
if err := x.ClientStream.SendMsg(in); err != nil {
return nil, err
}
if err := x.ClientStream.CloseSend(); err != nil {
return nil, err
}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_StatusClient = grpc.ServerStreamingClient[StatusResponse]
func (c *controllerClient) Input(ctx context.Context, opts ...grpc.CallOption) (grpc.ClientStreamingClient[InputMessage, InputResponse], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[1], Controller_Input_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[InputMessage, InputResponse]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InputClient = grpc.ClientStreamingClient[InputMessage, InputResponse]
func (c *controllerClient) Invoke(ctx context.Context, opts ...grpc.CallOption) (grpc.BidiStreamingClient[Message, Message], error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
stream, err := c.cc.NewStream(ctx, &Controller_ServiceDesc.Streams[2], Controller_Invoke_FullMethodName, cOpts...)
if err != nil {
return nil, err
}
x := &grpc.GenericClientStream[Message, Message]{ClientStream: stream}
return x, nil
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InvokeClient = grpc.BidiStreamingClient[Message, Message]
func (c *controllerClient) List(ctx context.Context, in *ListRequest, opts ...grpc.CallOption) (*ListResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListResponse)
err := c.cc.Invoke(ctx, Controller_List_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Disconnect(ctx context.Context, in *DisconnectRequest, opts ...grpc.CallOption) (*DisconnectResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DisconnectResponse)
err := c.cc.Invoke(ctx, Controller_Disconnect_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) Info(ctx context.Context, in *InfoRequest, opts ...grpc.CallOption) (*InfoResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(InfoResponse)
err := c.cc.Invoke(ctx, Controller_Info_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) ListProcesses(ctx context.Context, in *ListProcessesRequest, opts ...grpc.CallOption) (*ListProcessesResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(ListProcessesResponse)
err := c.cc.Invoke(ctx, Controller_ListProcesses_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
func (c *controllerClient) DisconnectProcess(ctx context.Context, in *DisconnectProcessRequest, opts ...grpc.CallOption) (*DisconnectProcessResponse, error) {
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
out := new(DisconnectProcessResponse)
err := c.cc.Invoke(ctx, Controller_DisconnectProcess_FullMethodName, in, out, cOpts...)
if err != nil {
return nil, err
}
return out, nil
}
// ControllerServer is the server API for Controller service.
// All implementations should embed UnimplementedControllerServer
// for forward compatibility.
type ControllerServer interface {
Build(context.Context, *BuildRequest) (*BuildResponse, error)
Inspect(context.Context, *InspectRequest) (*InspectResponse, error)
Status(*StatusRequest, grpc.ServerStreamingServer[StatusResponse]) error
Input(grpc.ClientStreamingServer[InputMessage, InputResponse]) error
Invoke(grpc.BidiStreamingServer[Message, Message]) error
List(context.Context, *ListRequest) (*ListResponse, error)
Disconnect(context.Context, *DisconnectRequest) (*DisconnectResponse, error)
Info(context.Context, *InfoRequest) (*InfoResponse, error)
ListProcesses(context.Context, *ListProcessesRequest) (*ListProcessesResponse, error)
DisconnectProcess(context.Context, *DisconnectProcessRequest) (*DisconnectProcessResponse, error)
}
// UnimplementedControllerServer should be embedded to have
// forward compatible implementations.
//
// NOTE: this should be embedded by value instead of pointer to avoid a nil
// pointer dereference when methods are called.
type UnimplementedControllerServer struct{}
func (UnimplementedControllerServer) Build(context.Context, *BuildRequest) (*BuildResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Build not implemented")
}
func (UnimplementedControllerServer) Inspect(context.Context, *InspectRequest) (*InspectResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Inspect not implemented")
}
func (UnimplementedControllerServer) Status(*StatusRequest, grpc.ServerStreamingServer[StatusResponse]) error {
return status.Errorf(codes.Unimplemented, "method Status not implemented")
}
func (UnimplementedControllerServer) Input(grpc.ClientStreamingServer[InputMessage, InputResponse]) error {
return status.Errorf(codes.Unimplemented, "method Input not implemented")
}
func (UnimplementedControllerServer) Invoke(grpc.BidiStreamingServer[Message, Message]) error {
return status.Errorf(codes.Unimplemented, "method Invoke not implemented")
}
func (UnimplementedControllerServer) List(context.Context, *ListRequest) (*ListResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method List not implemented")
}
func (UnimplementedControllerServer) Disconnect(context.Context, *DisconnectRequest) (*DisconnectResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Disconnect not implemented")
}
func (UnimplementedControllerServer) Info(context.Context, *InfoRequest) (*InfoResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method Info not implemented")
}
func (UnimplementedControllerServer) ListProcesses(context.Context, *ListProcessesRequest) (*ListProcessesResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method ListProcesses not implemented")
}
func (UnimplementedControllerServer) DisconnectProcess(context.Context, *DisconnectProcessRequest) (*DisconnectProcessResponse, error) {
return nil, status.Errorf(codes.Unimplemented, "method DisconnectProcess not implemented")
}
func (UnimplementedControllerServer) testEmbeddedByValue() {}
// UnsafeControllerServer may be embedded to opt out of forward compatibility for this service.
// Use of this interface is not recommended, as added methods to ControllerServer will
// result in compilation errors.
type UnsafeControllerServer interface {
mustEmbedUnimplementedControllerServer()
}
func RegisterControllerServer(s grpc.ServiceRegistrar, srv ControllerServer) {
// If the following call pancis, it indicates UnimplementedControllerServer was
// embedded by pointer and is nil. This will cause panics if an
// unimplemented method is ever invoked, so we test this at initialization
// time to prevent it from happening at runtime later due to I/O.
if t, ok := srv.(interface{ testEmbeddedByValue() }); ok {
t.testEmbeddedByValue()
}
s.RegisterService(&Controller_ServiceDesc, srv)
}
func _Controller_Build_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(BuildRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Build(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Build_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Build(ctx, req.(*BuildRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Inspect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InspectRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Inspect(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Inspect_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Inspect(ctx, req.(*InspectRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Status_Handler(srv interface{}, stream grpc.ServerStream) error {
m := new(StatusRequest)
if err := stream.RecvMsg(m); err != nil {
return err
}
return srv.(ControllerServer).Status(m, &grpc.GenericServerStream[StatusRequest, StatusResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_StatusServer = grpc.ServerStreamingServer[StatusResponse]
func _Controller_Input_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(ControllerServer).Input(&grpc.GenericServerStream[InputMessage, InputResponse]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InputServer = grpc.ClientStreamingServer[InputMessage, InputResponse]
func _Controller_Invoke_Handler(srv interface{}, stream grpc.ServerStream) error {
return srv.(ControllerServer).Invoke(&grpc.GenericServerStream[Message, Message]{ServerStream: stream})
}
// This type alias is provided for backwards compatibility with existing code that references the prior non-generic stream type by name.
type Controller_InvokeServer = grpc.BidiStreamingServer[Message, Message]
func _Controller_List_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).List(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_List_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).List(ctx, req.(*ListRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Disconnect_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DisconnectRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Disconnect(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Disconnect_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Disconnect(ctx, req.(*DisconnectRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_Info_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(InfoRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).Info(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_Info_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).Info(ctx, req.(*InfoRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_ListProcesses_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(ListProcessesRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).ListProcesses(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_ListProcesses_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).ListProcesses(ctx, req.(*ListProcessesRequest))
}
return interceptor(ctx, in, info, handler)
}
func _Controller_DisconnectProcess_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
in := new(DisconnectProcessRequest)
if err := dec(in); err != nil {
return nil, err
}
if interceptor == nil {
return srv.(ControllerServer).DisconnectProcess(ctx, in)
}
info := &grpc.UnaryServerInfo{
Server: srv,
FullMethod: Controller_DisconnectProcess_FullMethodName,
}
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
return srv.(ControllerServer).DisconnectProcess(ctx, req.(*DisconnectProcessRequest))
}
return interceptor(ctx, in, info, handler)
}
// Controller_ServiceDesc is the grpc.ServiceDesc for Controller service.
// It's only intended for direct use with grpc.RegisterService,
// and not to be introspected or modified (even as a copy)
var Controller_ServiceDesc = grpc.ServiceDesc{
ServiceName: "buildx.controller.v1.Controller",
HandlerType: (*ControllerServer)(nil),
Methods: []grpc.MethodDesc{
{
MethodName: "Build",
Handler: _Controller_Build_Handler,
},
{
MethodName: "Inspect",
Handler: _Controller_Inspect_Handler,
},
{
MethodName: "List",
Handler: _Controller_List_Handler,
},
{
MethodName: "Disconnect",
Handler: _Controller_Disconnect_Handler,
},
{
MethodName: "Info",
Handler: _Controller_Info_Handler,
},
{
MethodName: "ListProcesses",
Handler: _Controller_ListProcesses_Handler,
},
{
MethodName: "DisconnectProcess",
Handler: _Controller_DisconnectProcess_Handler,
},
},
Streams: []grpc.StreamDesc{
{
StreamName: "Status",
Handler: _Controller_Status_Handler,
ServerStreams: true,
},
{
StreamName: "Input",
Handler: _Controller_Input_Handler,
ClientStreams: true,
},
{
StreamName: "Invoke",
Handler: _Controller_Invoke_Handler,
ServerStreams: true,
ClientStreams: true,
},
},
Metadata: "github.com/docker/buildx/controller/pb/controller.proto",
}

File diff suppressed because it is too large Load Diff

View File

@@ -10,16 +10,15 @@ import (
"github.com/pkg/errors"
)
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, error) {
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
var localPaths []string
if len(entries) == 0 {
return nil, nil, nil
return nil, nil
}
var stdoutUsed bool
for _, entry := range entries {
if entry.Type == "" {
return nil, nil, errors.Errorf("type is required for output")
return nil, errors.Errorf("type is required for output")
}
out := client.ExportEntry{
@@ -46,26 +45,24 @@ func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, erro
supportDir = !tar
case "registry":
out.Type = client.ExporterImage
out.Attrs["push"] = "true"
}
if supportDir {
if entry.Destination == "" {
return nil, nil, errors.Errorf("dest is required for %s exporter", out.Type)
return nil, errors.Errorf("dest is required for %s exporter", out.Type)
}
if entry.Destination == "-" {
return nil, nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
}
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
}
if err == nil && !fi.IsDir() {
return nil, nil, errors.Errorf("destination directory %s is a file", entry.Destination)
return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
}
out.OutputDir = entry.Destination
localPaths = append(localPaths, entry.Destination)
}
if supportFile {
if entry.Destination == "" && out.Type != client.ExporterDocker {
@@ -73,33 +70,32 @@ func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, []string, erro
}
if entry.Destination == "-" {
if stdoutUsed {
return nil, nil, errors.Errorf("multiple outputs configured to write to stdout")
return nil, errors.Errorf("multiple outputs configured to write to stdout")
}
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
stdoutUsed = true
} else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
}
if err == nil && fi.IsDir() {
return nil, nil, errors.Errorf("destination file %s is a directory", entry.Destination)
return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
}
f, err := os.Create(entry.Destination)
if err != nil {
return nil, nil, errors.Errorf("failed to open %s", err)
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
localPaths = append(localPaths, entry.Destination)
}
}
outs = append(outs, out)
}
return outs, localPaths, nil
return outs, nil
}
func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) {

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

View File

@@ -153,6 +153,7 @@ func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
}
}
ps = append(ps, p)
}
s.Paths = ps
ssh = append(ssh, s)

View File

@@ -3,10 +3,10 @@ package pb
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/stretchr/testify/require"
"google.golang.org/protobuf/proto"
)
func TestResolvePaths(t *testing.T) {
@@ -16,58 +16,54 @@ func TestResolvePaths(t *testing.T) {
require.NoError(t, os.Chdir(tmpwd))
tests := []struct {
name string
options *BuildOptions
want *BuildOptions
options BuildOptions
want BuildOptions
}{
{
name: "contextpath",
options: &BuildOptions{ContextPath: "test"},
want: &BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
options: BuildOptions{ContextPath: "test"},
want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
},
{
name: "contextpath-cwd",
options: &BuildOptions{ContextPath: "."},
want: &BuildOptions{ContextPath: tmpwd},
options: BuildOptions{ContextPath: "."},
want: BuildOptions{ContextPath: tmpwd},
},
{
name: "contextpath-dash",
options: &BuildOptions{ContextPath: "-"},
want: &BuildOptions{ContextPath: "-"},
options: BuildOptions{ContextPath: "-"},
want: BuildOptions{ContextPath: "-"},
},
{
name: "contextpath-ssh",
options: &BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: &BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "dockerfilename",
options: &BuildOptions{DockerfileName: "test", ContextPath: "."},
want: &BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
},
{
name: "dockerfilename-dash",
options: &BuildOptions{DockerfileName: "-", ContextPath: "."},
want: &BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
},
{
name: "dockerfilename-remote",
options: &BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: &BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "contexts",
options: &BuildOptions{NamedContexts: map[string]string{
"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git",
}},
want: &BuildOptions{NamedContexts: map[string]string{
"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git",
}},
options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
},
{
name: "cache-from",
options: &BuildOptions{
options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
@@ -79,7 +75,7 @@ func TestResolvePaths(t *testing.T) {
},
},
},
want: &BuildOptions{
want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
@@ -94,7 +90,7 @@ func TestResolvePaths(t *testing.T) {
},
{
name: "cache-to",
options: &BuildOptions{
options: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
@@ -106,7 +102,7 @@ func TestResolvePaths(t *testing.T) {
},
},
},
want: &BuildOptions{
want: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
@@ -121,7 +117,7 @@ func TestResolvePaths(t *testing.T) {
},
{
name: "exports",
options: &BuildOptions{
options: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
@@ -149,7 +145,7 @@ func TestResolvePaths(t *testing.T) {
},
},
},
want: &BuildOptions{
want: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
@@ -180,7 +176,7 @@ func TestResolvePaths(t *testing.T) {
},
{
name: "secrets",
options: &BuildOptions{
options: BuildOptions{
Secrets: []*Secret{
{
FilePath: "test1",
@@ -195,7 +191,7 @@ func TestResolvePaths(t *testing.T) {
},
},
},
want: &BuildOptions{
want: BuildOptions{
Secrets: []*Secret{
{
FilePath: filepath.Join(tmpwd, "test1"),
@@ -213,7 +209,7 @@ func TestResolvePaths(t *testing.T) {
},
{
name: "ssh",
options: &BuildOptions{
options: BuildOptions{
SSH: []*SSH{
{
ID: "default",
@@ -225,7 +221,7 @@ func TestResolvePaths(t *testing.T) {
},
},
},
want: &BuildOptions{
want: BuildOptions{
SSH: []*SSH{
{
ID: "default",
@@ -242,10 +238,10 @@ func TestResolvePaths(t *testing.T) {
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(tt.options)
got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err)
if !proto.Equal(tt.want, got) {
t.Fatalf("expected %#v, got %#v", tt.want, got)
if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, *got)
}
})
}

View File

@@ -1,13 +1,10 @@
package pb
import (
"time"
"github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest"
"google.golang.org/protobuf/types/known/timestamppb"
)
type writer struct {
@@ -22,7 +19,9 @@ func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status)
}
func (w *writer) WriteBuildRef(target string, ref string) {}
func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true
@@ -34,11 +33,11 @@ func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{}
for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: string(v.Digest),
Inputs: digestSliceToPB(v.Inputs),
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: timestampToPB(v.Started),
Completed: timestampToPB(v.Completed),
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
@@ -47,26 +46,26 @@ func ToControlStatus(s *client.SolveStatus) *StatusResponse {
for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID,
Vertex: string(v.Vertex),
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: timestamppb.New(v.Timestamp),
Started: timestampToPB(v.Started),
Completed: timestampToPB(v.Completed),
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: string(v.Vertex),
Vertex: v.Vertex,
Stream: int64(v.Stream),
Msg: v.Data,
Timestamp: timestamppb.New(v.Timestamp),
Timestamp: v.Timestamp,
})
}
for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: string(v.Vertex),
Vertex: v.Vertex,
Level: int64(v.Level),
Short: v.Short,
Detail: v.Detail,
@@ -82,11 +81,11 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{}
for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: digest.Digest(v.Digest),
Inputs: digestSliceFromPB(v.Inputs),
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: timestampFromPB(v.Started),
Completed: timestampFromPB(v.Completed),
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
@@ -95,26 +94,26 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID,
Vertex: digest.Digest(v.Vertex),
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp.AsTime(),
Started: timestampFromPB(v.Started),
Completed: timestampFromPB(v.Completed),
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{
Vertex: digest.Digest(v.Vertex),
Vertex: v.Vertex,
Stream: int(v.Stream),
Data: v.Msg,
Timestamp: v.Timestamp.AsTime(),
Timestamp: v.Timestamp,
})
}
for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: digest.Digest(v.Vertex),
Vertex: v.Vertex,
Level: int(v.Level),
Short: v.Short,
Detail: v.Detail,
@@ -125,38 +124,3 @@ func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
}
return &s
}
func timestampFromPB(ts *timestamppb.Timestamp) *time.Time {
if ts == nil {
return nil
}
t := ts.AsTime()
if t.IsZero() {
return nil
}
return &t
}
func timestampToPB(ts *time.Time) *timestamppb.Timestamp {
if ts == nil {
return nil
}
return timestamppb.New(*ts)
}
func digestSliceFromPB(elems []string) []digest.Digest {
clone := make([]digest.Digest, len(elems))
for i, e := range elems {
clone[i] = digest.Digest(e)
}
return clone
}
func digestSliceToPB(elems []digest.Digest) []string {
clone := make([]string, len(elems))
for i, e := range elems {
clone[i] = string(e)
}
return clone
}

View File

@@ -18,16 +18,16 @@ type Process struct {
invokeConfig *pb.InvokeConfig
errCh chan error
processCancel func()
serveIOCancel func(error)
serveIOCancel func()
}
// ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func(error)) {
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil {
f(errors.WithStack(context.Canceled))
f()
}
p.serveIOCancel = ioCancelCallback
}
@@ -124,16 +124,9 @@ func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *p
f.SetOut(&out)
// Register process
ctx, cancel := context.WithCancelCause(context.TODO())
ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once
processCancelFunc := func() {
cancelOnce.Do(func() {
cancel(errors.WithStack(context.Canceled))
f.Close()
in.Close()
out.Close()
})
}
processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
p := &Process{
inEnd: f,
invokeConfig: cfg,

View File

@@ -6,9 +6,8 @@ import (
"sync"
"time"
"github.com/containerd/containerd/v2/defaults"
"github.com/containerd/containerd/v2/pkg/dialer"
"github.com/docker/buildx/build"
"github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
@@ -28,7 +27,6 @@ func NewClient(ctx context.Context, addr string) (*Client, error) {
Backoff: backoffConfig,
}
gopts := []grpc.DialOption{
//nolint:staticcheck // ignore SA1019: WithBlock is deprecated and does not work with NewClient.
grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams),
@@ -38,7 +36,6 @@ func NewClient(ctx context.Context, addr string) (*Client, error) {
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
}
//nolint:staticcheck // ignore SA1019: Recommended NewClient has different behavior from DialContext.
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil {
return nil, err
@@ -75,36 +72,36 @@ func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
return res.Keys, nil
}
func (c *Client) Disconnect(ctx context.Context, sessionID string) error {
if sessionID == "" {
func (c *Client) Disconnect(ctx context.Context, key string) error {
if key == "" {
return nil
}
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{SessionID: sessionID})
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err
}
func (c *Client) ListProcesses(ctx context.Context, sessionID string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{SessionID: sessionID})
func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil {
return nil, err
}
return res.Infos, nil
}
func (c *Client) DisconnectProcess(ctx context.Context, sessionID, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{SessionID: sessionID, ProcessID: pid})
func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err
}
func (c *Client) Invoke(ctx context.Context, sessionID string, pid string, invokeConfig *pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if sessionID == "" || pid == "" {
return errors.New("build session ID must be specified")
func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if ref == "" || pid == "" {
return errors.New("build reference must be specified")
}
stream, err := c.client().Invoke(ctx)
if err != nil {
return err
}
return attachIO(ctx, stream, &pb.InitMessage{SessionID: sessionID, ProcessID: pid, InvokeConfig: invokeConfig}, ioAttachConfig{
return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in,
stdout: stdout,
stderr: stderr,
@@ -112,11 +109,11 @@ func (c *Client) Invoke(ctx context.Context, sessionID string, pid string, invok
})
}
func (c *Client) Inspect(ctx context.Context, sessionID string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{SessionID: sessionID})
func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
}
func (c *Client) Build(ctx context.Context, options *pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, *build.Inputs, error) {
func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID()
statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx)
@@ -134,10 +131,10 @@ func (c *Client) Build(ctx context.Context, options *pb.BuildOptions, in io.Read
}
return nil
})
return ref, resp, nil, eg.Wait()
return ref, resp, eg.Wait()
}
func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{})
@@ -146,8 +143,8 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
eg.Go(func() error {
defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
SessionID: sessionID,
Options: options,
Ref: ref,
Options: &options,
})
if err != nil {
return err
@@ -159,7 +156,7 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
})
eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{
SessionID: sessionID,
Ref: ref,
})
if err != nil {
return err
@@ -184,7 +181,7 @@ func (c *Client) build(ctx context.Context, sessionID string, options *pb.BuildO
if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{
SessionID: sessionID,
Ref: ref,
},
},
}); err != nil {

View File

@@ -62,10 +62,9 @@ func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts
serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running
ctx2, cancel := context.WithCancelCause(ctx)
ctx2, _ = context.WithTimeoutCause(ctx2, 1*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel(errors.WithStack(context.Canceled))
cancel()
if err != nil {
if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server")
@@ -91,10 +90,9 @@ func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts
go wait()
// wait for buildx server to be ready
ctx2, cancel = context.WithCancelCause(ctx)
ctx2, _ = context.WithTimeoutCause(ctx2, 10*time.Second, errors.WithStack(context.DeadlineExceeded)) //nolint:govet,lostcancel // no need to manually cancel this context as we already rely on parent
ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel(errors.WithStack(context.Canceled))
cancel()
if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server")
}
@@ -150,8 +148,8 @@ func serveCmd(dockerCli command.Cli) *cobra.Command {
}()
// prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, *build.Inputs, error) {
return cbuild.RunBuild(ctx, dockerCli, options, stdin, progress, true)
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
})
defer b.Close()
@@ -260,7 +258,7 @@ func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error)
}
func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.NewConfig(dockerCli).Dir(), "controller")
return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
}
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {

View File

@@ -43,9 +43,9 @@ func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessa
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
}
sessionID := init.SessionID
if sessionID == "" {
return errors.New("no session ID is provided")
ref := init.Ref
if ref == "" {
return errors.New("no ref is provided")
}
if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server")
@@ -302,6 +302,7 @@ func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage
out = cfg.stderr
default:
return errors.Errorf("unsupported fd %d", file.Fd)
}
if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
@@ -344,7 +345,7 @@ func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
case err := <-errCh:
return nil, err
case <-ctx.Done():
return nil, context.Cause(ctx)
return nil, ctx.Err()
}
}

View File

@@ -11,7 +11,6 @@ import (
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
@@ -20,7 +19,7 @@ import (
"golang.org/x/sync/errgroup"
)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, inp *build.Inputs, err error)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server {
return &Server{
@@ -37,7 +36,7 @@ type Server struct {
type session struct {
buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse
cancelBuild func(error)
cancelBuild func()
buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter
@@ -53,9 +52,9 @@ func (s *session) cancelRunningProcesses() {
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.SessionID]
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown session ID %q", req.SessionID)
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
res = new(pb.ListProcessesResponse)
res.Infos = append(res.Infos, s.processes.ListProcesses()...)
@@ -65,9 +64,9 @@ func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.SessionID]
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown session ID %q", req.SessionID)
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
return res, s.processes.DeleteProcess(req.ProcessID)
}
@@ -101,22 +100,22 @@ func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListRes
}
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
sessionID := req.SessionID
if sessionID == "" {
return nil, errors.New("disconnect: empty session ID")
key := req.Ref
if key == "" {
return nil, errors.New("disconnect: empty key")
}
m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok {
if s, ok := m.session[key]; ok {
if s.cancelBuild != nil {
s.cancelBuild(errors.WithStack(context.Canceled))
s.cancelBuild()
}
s.cancelRunningProcesses()
if s.result != nil {
s.result.Done()
}
}
delete(m.session, sessionID)
delete(m.session, key)
m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil
@@ -127,7 +126,7 @@ func (m *Server) Close() error {
for k := range m.session {
if s, ok := m.session[k]; ok {
if s.cancelBuild != nil {
s.cancelBuild(errors.WithStack(context.Canceled))
s.cancelBuild()
}
s.cancelRunningProcesses()
}
@@ -137,26 +136,26 @@ func (m *Server) Close() error {
}
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
sessionID := req.SessionID
if sessionID == "" {
return nil, errors.New("inspect: empty session ID")
ref := req.Ref
if ref == "" {
return nil, errors.New("inspect: empty key")
}
var bo *pb.BuildOptions
m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok {
if s, ok := m.session[ref]; ok {
bo = s.buildOptions
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", sessionID)
return nil, errors.Errorf("inspect: unknown key %v", ref)
}
m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil
}
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
sessionID := req.SessionID
if sessionID == "" {
return nil, errors.New("build: empty session ID")
ref := req.Ref
if ref == "" {
return nil, errors.New("build: empty key")
}
// Prepare status channel and session
@@ -164,7 +163,7 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
if m.session == nil {
m.session = make(map[string]*session)
}
s, ok := m.session[sessionID]
s, ok := m.session[ref]
if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock()
@@ -183,12 +182,12 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
inR, inW := io.Pipe()
defer inR.Close()
s.inputPipe = inW
m.session[sessionID] = s
m.session[ref] = s
m.sessionMu.Unlock()
defer func() {
close(statusChan)
m.sessionMu.Lock()
s, ok := m.session[sessionID]
s, ok := m.session[ref]
if ok {
s.statusChan = nil
s.buildOnGoing.Store(false)
@@ -199,29 +198,24 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
pw := pb.NewProgressWriter(statusChan)
// Build the specified request
ctx, cancel := context.WithCancelCause(ctx)
defer func() { cancel(errors.WithStack(context.Canceled)) }()
resp, res, _, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock()
if s, ok := m.session[sessionID]; ok {
if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil {
s.result = res
s.cancelBuild = cancel
s.buildOptions = req.Options
m.session[sessionID] = s
m.session[ref] = s
if buildErr != nil {
var ref string
var ebr *desktop.ErrorWithBuildRef
if errors.As(buildErr, &ebr) {
ref = ebr.Ref
}
buildErr = controllererrors.WrapBuild(buildErr, sessionID, ref)
buildErr = controllererrors.WrapBuild(buildErr, ref)
}
}
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown session ID %v", sessionID)
return nil, errors.Errorf("build: unknown key %v", ref)
}
m.sessionMu.Unlock()
@@ -238,9 +232,9 @@ func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResp
}
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
sessionID := req.SessionID
if sessionID == "" {
return errors.New("status: empty session ID")
ref := req.Ref
if ref == "" {
return errors.New("status: empty key")
}
// Wait and get status channel prepared by Build()
@@ -248,12 +242,12 @@ func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[sessionID]; !ok || m.session[sessionID].statusChan == nil {
if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
statusChan = m.session[sessionID].statusChan
statusChan = m.session[ref].statusChan
m.sessionMu.Unlock()
break
}
@@ -284,9 +278,9 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
}
sessionID := init.SessionID
if sessionID == "" {
return errors.New("input: no session ID is provided")
ref := init.Ref
if ref == "" {
return errors.New("input: no ref is provided")
}
// Wait and get input stream pipe prepared by Build()
@@ -294,12 +288,12 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[sessionID]; !ok || m.session[sessionID].inputPipe == nil {
if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
inputPipeW = m.session[sessionID].inputPipe
inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock()
break
}
@@ -341,7 +335,7 @@ func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
select {
case msg = <-msgCh:
case <-ctx.Done():
return context.Cause(ctx)
return errors.Wrap(ctx.Err(), "canceled")
}
if msg == nil {
return nil
@@ -370,23 +364,23 @@ func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancelCause(egCtx)
srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error {
defer srvIOCancel(errors.WithStack(context.Canceled))
defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() {
if retErr != nil {
initErrCh <- retErr
}
}()
sessionID := initMessage.SessionID
ref := initMessage.Ref
cfg := initMessage.InvokeConfig
m.sessionMu.Lock()
s, ok := m.session[sessionID]
s, ok := m.session[ref]
if !ok {
m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown session ID %v", sessionID)
return errors.Errorf("invoke: unknown key %v", ref)
}
m.sessionMu.Unlock()
@@ -418,7 +412,7 @@ func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
})
})
eg.Go(func() (rErr error) {
defer srvIOCancel(errors.WithStack(context.Canceled))
defer srvIOCancel()
// Wait for init done
var proc *processes.Process
select {

View File

@@ -41,15 +41,11 @@ target "lint" {
platforms = GOLANGCI_LINT_MULTIPLATFORM != "" ? [
"darwin/amd64",
"darwin/arm64",
"freebsd/amd64",
"freebsd/arm64",
"linux/amd64",
"linux/arm64",
"linux/s390x",
"linux/ppc64le",
"linux/riscv64",
"openbsd/amd64",
"openbsd/arm64",
"windows/amd64",
"windows/arm64"
] : []
@@ -158,8 +154,6 @@ target "binaries-cross" {
platforms = [
"darwin/amd64",
"darwin/arm64",
"freebsd/amd64",
"freebsd/arm64",
"linux/amd64",
"linux/arm/v6",
"linux/arm/v7",
@@ -167,8 +161,6 @@ target "binaries-cross" {
"linux/ppc64le",
"linux/riscv64",
"linux/s390x",
"openbsd/amd64",
"openbsd/arm64",
"windows/amd64",
"windows/arm64"
]

View File

@@ -19,8 +19,8 @@ By default, Bake uses the following lookup order to find the configuration file:
3. `docker-compose.yml`
4. `docker-compose.yaml`
5. `docker-bake.json`
6. `docker-bake.hcl`
7. `docker-bake.override.json`
6. `docker-bake.override.json`
7. `docker-bake.hcl`
8. `docker-bake.override.hcl`
You can specify the file location explicitly using the `--file` flag:
@@ -221,10 +221,8 @@ The following table shows the complete list of attributes that you can assign to
| [`attest`](#targetattest) | List | Build attestations |
| [`cache-from`](#targetcache-from) | List | External cache sources |
| [`cache-to`](#targetcache-to) | List | External cache destinations |
| [`call`](#targetcall) | String | Specify the frontend method to call for the target. |
| [`context`](#targetcontext) | String | Set of files located in the specified path or URL |
| [`contexts`](#targetcontexts) | Map | Additional build contexts |
| [`description`](#targetdescription) | String | Description of a target |
| [`dockerfile-inline`](#targetdockerfile-inline) | String | Inline Dockerfile string |
| [`dockerfile`](#targetdockerfile) | String | Dockerfile location |
| [`inherits`](#targetinherits) | List | Inherit attributes from other targets |
@@ -361,28 +359,6 @@ target "app" {
}
```
### `target.call`
Specifies the frontend method to use. Frontend methods let you, for example,
execute build checks only, instead of running a build. This is the same as the
`--call` flag.
```hcl
target "app" {
call = "check"
}
```
Supported values are:
- `build` builds the target (default)
- `check`: evaluates [build checks](https://docs.docker.com/build/checks/) for the target
- `outline`: displays the target's build arguments and their default values if available
- `targets`: lists all Bake targets in the loaded definition, along with its [description](#targetdescription).
For more information about frontend methods, refer to the CLI reference for
[`docker buildx build --call`](https://docs.docker.com/reference/cli/docker/buildx/build/#call).
### `target.context`
Specifies the location of the build context to use for this target.
@@ -490,25 +466,6 @@ FROM baseapp
RUN echo "Hello world"
```
### `target.description`
Defines a human-readable description for the target, clarifying its purpose or
functionality.
```hcl
target "lint" {
description = "Runs golangci-lint to detect style errors"
args = {
GOLANGCI_LINT_VERSION = null
}
dockerfile = "lint.Dockerfile"
}
```
This attribute is useful when combined with the `docker buildx bake --list=targets`
option, providing a more informative output when listing the available build
targets in a Bake file.
### `target.dockerfile-inline`
Uses the string value as an inline Dockerfile for the build target.
@@ -887,7 +844,7 @@ The following example forces the builder to always pull all images referenced in
```hcl
target "default" {
pull = true
pull = "always"
}
```

View File

@@ -17,7 +17,6 @@ Extended build capabilities with BuildKit
| [`debug`](buildx_debug.md) | Start debugger (EXPERIMENTAL) |
| [`dial-stdio`](buildx_dial-stdio.md) | Proxy current stdio streams to builder instance |
| [`du`](buildx_du.md) | Disk usage |
| [`history`](buildx_history.md) | Commands to work on build records |
| [`imagetools`](buildx_imagetools.md) | Commands to work on images in registry |
| [`inspect`](buildx_inspect.md) | Inspect current builder instance |
| [`ls`](buildx_ls.md) | List builder instances |

View File

@@ -13,25 +13,24 @@ Build from a file
### Options
| Name | Type | Default | Description |
|:------------------------------------|:--------------|:--------|:-------------------------------------------------------------------------------------------------------------|
| [`--allow`](#allow) | `stringArray` | | Allow build to access specified resources |
| [`--builder`](#builder) | `string` | | Override the configured builder instance |
| [`--call`](#call) | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| [`--check`](#check) | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| [`-f`](#file), [`--file`](#file) | `stringArray` | | Build definition file |
| [`--list`](#list) | `string` | | List targets or variables |
| `--load` | `bool` | | Shorthand for `--set=*.output=type=docker` |
| [`--metadata-file`](#metadata-file) | `string` | | Write build result metadata to a file |
| [`--no-cache`](#no-cache) | `bool` | | Do not use cache when building the image |
| [`--print`](#print) | `bool` | | Print the options without building |
| [`--progress`](#progress) | `string` | `auto` | Set type of progress output (`auto`, `quiet`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| [`--provenance`](#provenance) | `string` | | Shorthand for `--set=*.attest=type=provenance` |
| [`--pull`](#pull) | `bool` | | Always attempt to pull all referenced images |
| `--push` | `bool` | | Shorthand for `--set=*.output=type=registry` |
| [`--sbom`](#sbom) | `string` | | Shorthand for `--set=*.attest=type=sbom` |
| [`--set`](#set) | `stringArray` | | Override target value (e.g., `targetpattern.key=value`) |
| Name | Type | Default | Description |
|:------------------------------------|:--------------|:--------|:----------------------------------------------------------------------------------------------------|
| `--allow` | `stringArray` | | Allow build to access specified resources |
| [`--builder`](#builder) | `string` | | Override the configured builder instance |
| [`--call`](#call) | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| [`--check`](#check) | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| [`-f`](#file), [`--file`](#file) | `stringArray` | | Build definition file |
| `--load` | `bool` | | Shorthand for `--set=*.output=type=docker` |
| [`--metadata-file`](#metadata-file) | `string` | | Write build result metadata to a file |
| [`--no-cache`](#no-cache) | `bool` | | Do not use cache when building the image |
| [`--print`](#print) | `bool` | | Print the options without building |
| [`--progress`](#progress) | `string` | `auto` | Set type of progress output (`auto`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| [`--provenance`](#provenance) | `string` | | Shorthand for `--set=*.attest=type=provenance` |
| [`--pull`](#pull) | `bool` | | Always attempt to pull all referenced images |
| `--push` | `bool` | | Shorthand for `--set=*.output=type=registry` |
| [`--sbom`](#sbom) | `string` | | Shorthand for `--set=*.attest=type=sbom` |
| [`--set`](#set) | `stringArray` | | Override target value (e.g., `targetpattern.key=value`) |
<!---MARKER_GEN_END-->
@@ -51,80 +50,6 @@ guide for introduction to writing bake files.
## Examples
### <a name="allow"></a> Allow extra privileged entitlement (--allow)
```text
--allow=ENTITLEMENT[=VALUE]
```
Entitlements are designed to provide controlled access to privileged
operations. By default, Buildx and BuildKit operates with restricted
permissions to protect users and their systems from unintended side effects or
security risks. The `--allow` flag explicitly grants access to additional
entitlements, making it clear when a build or bake operation requires elevated
privileges.
In addition to BuildKit's `network.host` and `security.insecure` entitlements
(see [`docker buildx build --allow`](https://docs.docker.com/reference/cli/docker/buildx/build/#allow),
Bake supports file system entitlements that grant granular control over file
system access. These are particularly useful when working with builds that need
access to files outside the default working directory.
Bake supports the following filesystem entitlements:
- `--allow fs=<path|*>` - Grant read and write access to files outside of the
working directory.
- `--allow fs.read=<path|*>` - Grant read access to files outside of the
working directory.
- `--allow fs.write=<path|*>` - Grant write access to files outside of the
working directory.
The `fs` entitlements take a path value (relative or absolute) to a directory
on the filesystem. Alternatively, you can pass a wildcard (`*`) to allow Bake
to access the entire filesystem.
### Example: fs.read
Given the following Bake configuration, Bake would need to access the parent
directory, relative to the Bake file.
```hcl
target "app" {
context = "../src"
}
```
Assuming `docker buildx bake app` is executed in the same directory as the
`docker-bake.hcl` file, you would need to explicitly allow Bake to read from
the `../src` directory. In this case, the following invocations all work:
```console
$ docker buildx bake --allow fs.read=* app
$ docker buildx bake --allow fs.read=../src app
$ docker buildx bake --allow fs=* app
```
### Example: fs.write
The following `docker-bake.hcl` file requires write access to the `/tmp`
directory.
```hcl
target "app" {
output = "/tmp"
}
```
Assuming `docker buildx bake app` is executed outside of the `/tmp` directory,
you would need to allow the `fs.write` entitlement, either by specifying the
path or using a wildcard:
```console
$ docker buildx bake --allow fs=/tmp app
$ docker buildx bake --allow fs.write=/tmp app
$ docker buildx bake --allow fs.write=* app
```
### <a name="builder"></a> Override the configured builder instance (--builder)
Same as [`buildx --builder`](buildx.md#builder).
@@ -176,42 +101,6 @@ $ docker buildx bake -f docker-bake.dev.hcl db webapp-release
See the [Bake file reference](https://docs.docker.com/build/bake/reference/)
for more details.
### <a name="list"></a> List targets and variables (--list)
The `--list` flag displays all available targets or variables in the Bake
configuration, along with a description (if set using the `description`
property in the Bake file).
To list all targets:
```console {title="List targets"}
$ docker buildx bake --list=targets
TARGET DESCRIPTION
binaries
default binaries
update-docs
validate
validate-golangci Validate .golangci.yml schema (does not run Go linter)
```
To list variables:
```console
$ docker buildx bake --list=variables
VARIABLE VALUE DESCRIPTION
REGISTRY docker.io/username Registry and namespace
IMAGE_NAME my-app Image name
GO_VERSION <null>
```
By default, the output of `docker buildx bake --list` is presented in a table
format. Alternatively, you can use a long-form CSV syntax and specify a
`format` attribute to output the list in JSON.
```console
$ docker buildx bake --list=type=targets,format=json
```
### <a name="metadata-file"></a> Write build results metadata to a file (--metadata-file)
Similar to [`buildx build --metadata-file`](buildx_build.md#metadata-file) but

View File

@@ -13,46 +13,46 @@ Start a build
### Options
| Name | Type | Default | Description |
|:----------------------------------------|:--------------|:----------|:-------------------------------------------------------------------------------------------------------------|
| [`--add-host`](#add-host) | `stringSlice` | | Add a custom host-to-IP mapping (format: `host:ip`) |
| [`--allow`](#allow) | `stringSlice` | | Allow extra privileged entitlement (e.g., `network.host`, `security.insecure`) |
| [`--annotation`](#annotation) | `stringArray` | | Add annotation to the image |
| [`--attest`](#attest) | `stringArray` | | Attestation parameters (format: `type=sbom,generator=image`) |
| [`--build-arg`](#build-arg) | `stringArray` | | Set build-time variables |
| [`--build-context`](#build-context) | `stringArray` | | Additional build contexts (e.g., name=path) |
| [`--builder`](#builder) | `string` | | Override the configured builder instance |
| [`--cache-from`](#cache-from) | `stringArray` | | External cache sources (e.g., `user/app:cache`, `type=local,src=path/to/dir`) |
| [`--cache-to`](#cache-to) | `stringArray` | | Cache export destinations (e.g., `user/app:cache`, `type=local,dest=path/to/dir`) |
| [`--call`](#call) | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| [`--cgroup-parent`](#cgroup-parent) | `string` | | Set the parent cgroup for the `RUN` instructions during build |
| [`--check`](#check) | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--detach` | `bool` | | Detach buildx server (supported only on linux) (EXPERIMENTAL) |
| [`-f`](#file), [`--file`](#file) | `string` | | Name of the Dockerfile (default: `PATH/Dockerfile`) |
| `--iidfile` | `string` | | Write the image ID to a file |
| `--label` | `stringArray` | | Set metadata for an image |
| [`--load`](#load) | `bool` | | Shorthand for `--output=type=docker` |
| [`--metadata-file`](#metadata-file) | `string` | | Write build result metadata to a file |
| [`--network`](#network) | `string` | `default` | Set the networking mode for the `RUN` instructions during build |
| `--no-cache` | `bool` | | Do not use cache when building the image |
| [`--no-cache-filter`](#no-cache-filter) | `stringArray` | | Do not cache specified stages |
| [`-o`](#output), [`--output`](#output) | `stringArray` | | Output destination (format: `type=local,dest=path`) |
| [`--platform`](#platform) | `stringArray` | | Set target platform for build |
| [`--progress`](#progress) | `string` | `auto` | Set type of progress output (`auto`, `quiet`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| [`--provenance`](#provenance) | `string` | | Shorthand for `--attest=type=provenance` |
| `--pull` | `bool` | | Always attempt to pull all referenced images |
| [`--push`](#push) | `bool` | | Shorthand for `--output=type=registry` |
| `-q`, `--quiet` | `bool` | | Suppress the build output and print image ID on success |
| `--root` | `string` | | Specify root directory of server to connect (EXPERIMENTAL) |
| [`--sbom`](#sbom) | `string` | | Shorthand for `--attest=type=sbom` |
| [`--secret`](#secret) | `stringArray` | | Secret to expose to the build (format: `id=mysecret[,src=/local/secret]`) |
| `--server-config` | `string` | | Specify buildx server config file (used only when launching new server) (EXPERIMENTAL) |
| [`--shm-size`](#shm-size) | `bytes` | `0` | Shared memory size for build containers |
| [`--ssh`](#ssh) | `stringArray` | | SSH agent socket or keys to expose to the build (format: `default\|<id>[=<socket>\|<key>[,<key>]]`) |
| [`-t`](#tag), [`--tag`](#tag) | `stringArray` | | Name and optionally a tag (format: `name:tag`) |
| [`--target`](#target) | `string` | | Set the target build stage to build |
| [`--ulimit`](#ulimit) | `ulimit` | | Ulimit options |
| Name | Type | Default | Description |
|:----------------------------------------|:--------------|:----------|:----------------------------------------------------------------------------------------------------|
| [`--add-host`](#add-host) | `stringSlice` | | Add a custom host-to-IP mapping (format: `host:ip`) |
| [`--allow`](#allow) | `stringSlice` | | Allow extra privileged entitlement (e.g., `network.host`, `security.insecure`) |
| [`--annotation`](#annotation) | `stringArray` | | Add annotation to the image |
| [`--attest`](#attest) | `stringArray` | | Attestation parameters (format: `type=sbom,generator=image`) |
| [`--build-arg`](#build-arg) | `stringArray` | | Set build-time variables |
| [`--build-context`](#build-context) | `stringArray` | | Additional build contexts (e.g., name=path) |
| [`--builder`](#builder) | `string` | | Override the configured builder instance |
| [`--cache-from`](#cache-from) | `stringArray` | | External cache sources (e.g., `user/app:cache`, `type=local,src=path/to/dir`) |
| [`--cache-to`](#cache-to) | `stringArray` | | Cache export destinations (e.g., `user/app:cache`, `type=local,dest=path/to/dir`) |
| [`--call`](#call) | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| [`--cgroup-parent`](#cgroup-parent) | `string` | | Set the parent cgroup for the `RUN` instructions during build |
| [`--check`](#check) | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--detach` | `bool` | | Detach buildx server (supported only on linux) (EXPERIMENTAL) |
| [`-f`](#file), [`--file`](#file) | `string` | | Name of the Dockerfile (default: `PATH/Dockerfile`) |
| `--iidfile` | `string` | | Write the image ID to a file |
| `--label` | `stringArray` | | Set metadata for an image |
| [`--load`](#load) | `bool` | | Shorthand for `--output=type=docker` |
| [`--metadata-file`](#metadata-file) | `string` | | Write build result metadata to a file |
| [`--network`](#network) | `string` | `default` | Set the networking mode for the `RUN` instructions during build |
| `--no-cache` | `bool` | | Do not use cache when building the image |
| [`--no-cache-filter`](#no-cache-filter) | `stringArray` | | Do not cache specified stages |
| [`-o`](#output), [`--output`](#output) | `stringArray` | | Output destination (format: `type=local,dest=path`) |
| [`--platform`](#platform) | `stringArray` | | Set target platform for build |
| [`--progress`](#progress) | `string` | `auto` | Set type of progress output (`auto`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| [`--provenance`](#provenance) | `string` | | Shorthand for `--attest=type=provenance` |
| `--pull` | `bool` | | Always attempt to pull all referenced images |
| [`--push`](#push) | `bool` | | Shorthand for `--output=type=registry` |
| `-q`, `--quiet` | `bool` | | Suppress the build output and print image ID on success |
| `--root` | `string` | | Specify root directory of server to connect (EXPERIMENTAL) |
| [`--sbom`](#sbom) | `string` | | Shorthand for `--attest=type=sbom` |
| [`--secret`](#secret) | `stringArray` | | Secret to expose to the build (format: `id=mysecret[,src=/local/secret]`) |
| `--server-config` | `string` | | Specify buildx server config file (used only when launching new server) (EXPERIMENTAL) |
| [`--shm-size`](#shm-size) | `bytes` | `0` | Shared memory size for build containers |
| [`--ssh`](#ssh) | `stringArray` | | SSH agent socket or keys to expose to the build (format: `default\|<id>[=<socket>\|<key>[,<key>]]`) |
| [`-t`](#tag), [`--tag`](#tag) | `stringArray` | | Name and optionally a tag (format: `name:tag`) |
| [`--target`](#target) | `string` | | Set the target build stage to build |
| [`--ulimit`](#ulimit) | `ulimit` | | Ulimit options |
<!---MARKER_GEN_END-->
@@ -626,7 +626,7 @@ For example, the following Dockerfile contains four stages:
```dockerfile
# syntax=docker/dockerfile:1
FROM oven/bun:1 AS base
FROM oven/bun:1 as base
WORKDIR /app
FROM base AS install
@@ -828,12 +828,8 @@ $ docker buildx build --platform=darwin .
--progress=VALUE
```
Set type of progress output. Supported values are:
- `auto` (default): Uses the `tty` mode if the client is a TTY, or `plain` otherwise
- `tty`: An interactive stream of the output with color and redrawing
- `plain`: Prints the raw build progress in a plaintext format
- `quiet`: Suppress the build output and print image ID on success (same as `--quiet`)
- `rawjson`: Prints the raw build progress as JSON lines
Set type of progress output (`auto`, `plain`, `tty`, `rawjson`). Use `plain` to show container
output (default `auto`).
> [!NOTE]
> You can also use the `BUILDKIT_PROGRESS` environment variable to set its value.
@@ -916,39 +912,17 @@ For more information about how to use build secrets, see
Supported types are:
- [`type=file`](#typefile)
- [`type=env`](#typeenv)
- [`file`](#file)
- [`env`](#env)
Buildx attempts to detect the `type` automatically if unset. If an environment
variable with the same key as `id` is set, then Buildx uses `type=env` and the
variable value becomes the secret. If no such environment variable is set, and
`type` is not set, then Buildx falls back to `type=file`.
Buildx attempts to detect the `type` automatically if unset.
#### `type=file`
#### `file`
Source a build secret from a file.
Attribute keys:
##### `type=file` synopsis
```console
$ docker buildx build --secret [type=file,]id=<ID>[,src=<FILEPATH>] .
```
##### `type=file` attributes
| Key | Description | Default |
| --------------- | ----------------------------------------------------------------------------------------------------- | -------------------------- |
| `id` | ID of the secret. | N/A (this key is required) |
| `src`, `source` | Filepath of the file containing the secret value (absolute or relative to current working directory). | `id` if unset. |
###### `type=file` usage
In the following example, `type=file` is automatically detected because no
environment variable mathing `aws` (the ID) is set.
```console
$ docker buildx build --secret id=aws,src=$HOME/.aws/credentials .
```
- `id` - ID of the secret. Defaults to base name of the `src` path.
- `src`, `source` - Secret filename. `id` used if unset.
```dockerfile
# syntax=docker/dockerfile:1
@@ -958,31 +932,16 @@ RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
aws s3 cp s3://... ...
```
#### `type=env`
Source a build secret from an environment variable.
##### `type=env` synopsis
```console
$ docker buildx build --secret [type=env,]id=<ID>[,env=<VARIABLE>] .
$ docker buildx build --secret id=aws,src=$HOME/.aws/credentials .
```
##### `type=env` attributes
#### `env`
| Key | Description | Default |
| ---------------------- | ----------------------------------------------- | -------------------------- |
| `id` | ID of the secret. | N/A (this key is required) |
| `env`, `src`, `source` | Environment variable to source the secret from. | `id` if unset. |
Attribute keys:
##### `type=env` usage
In the following example, `type=env` is automatically detected because an
environment variable matching `id` is set.
```console
$ SECRET_TOKEN=token docker buildx build --secret id=SECRET_TOKEN .
```
- `id` - ID of the secret. Defaults to `env` name.
- `env` - Secret environment variable. `id` used if unset, otherwise will look for `src`, `source` if `id` unset.
```dockerfile
# syntax=docker/dockerfile:1
@@ -992,26 +951,10 @@ RUN --mount=type=bind,target=. \
yarn run test
```
In the following example, the build argument `SECRET_TOKEN` is set to contain
the value of the environment variable `API_KEY`.
```console
$ API_KEY=token docker buildx build --secret id=SECRET_TOKEN,env=API_KEY .
$ SECRET_TOKEN=token docker buildx build --secret id=SECRET_TOKEN .
```
You can also specify the name of the environment variable with `src` or `source`:
```console
$ API_KEY=token docker buildx build --secret type=env,id=SECRET_TOKEN,src=API_KEY .
```
> [!NOTE]
> Specifying the environment variable name with `src` or `source`, you are
> required to set `type=env` explicitly, or else Buildx assumes that the secret
> is `type=file`, and looks for a file with the name of `src` or `source` (in
> this case, a file named `API_KEY` relative to the location where the `docker
> buildx build` command was executed.
### <a name="shm-size"></a> Shared memory size for build containers (--shm-size)
Sets the size of the shared memory allocated for build containers when using

View File

@@ -9,46 +9,46 @@ Start a build
### Options
| Name | Type | Default | Description |
|:--------------------|:--------------|:----------|:-------------------------------------------------------------------------------------------------------------|
| `--add-host` | `stringSlice` | | Add a custom host-to-IP mapping (format: `host:ip`) |
| `--allow` | `stringSlice` | | Allow extra privileged entitlement (e.g., `network.host`, `security.insecure`) |
| `--annotation` | `stringArray` | | Add annotation to the image |
| `--attest` | `stringArray` | | Attestation parameters (format: `type=sbom,generator=image`) |
| `--build-arg` | `stringArray` | | Set build-time variables |
| `--build-context` | `stringArray` | | Additional build contexts (e.g., name=path) |
| `--builder` | `string` | | Override the configured builder instance |
| `--cache-from` | `stringArray` | | External cache sources (e.g., `user/app:cache`, `type=local,src=path/to/dir`) |
| `--cache-to` | `stringArray` | | Cache export destinations (e.g., `user/app:cache`, `type=local,dest=path/to/dir`) |
| `--call` | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| `--cgroup-parent` | `string` | | Set the parent cgroup for the `RUN` instructions during build |
| `--check` | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--detach` | `bool` | | Detach buildx server (supported only on linux) (EXPERIMENTAL) |
| `-f`, `--file` | `string` | | Name of the Dockerfile (default: `PATH/Dockerfile`) |
| `--iidfile` | `string` | | Write the image ID to a file |
| `--label` | `stringArray` | | Set metadata for an image |
| `--load` | `bool` | | Shorthand for `--output=type=docker` |
| `--metadata-file` | `string` | | Write build result metadata to a file |
| `--network` | `string` | `default` | Set the networking mode for the `RUN` instructions during build |
| `--no-cache` | `bool` | | Do not use cache when building the image |
| `--no-cache-filter` | `stringArray` | | Do not cache specified stages |
| `-o`, `--output` | `stringArray` | | Output destination (format: `type=local,dest=path`) |
| `--platform` | `stringArray` | | Set target platform for build |
| `--progress` | `string` | `auto` | Set type of progress output (`auto`, `quiet`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| `--provenance` | `string` | | Shorthand for `--attest=type=provenance` |
| `--pull` | `bool` | | Always attempt to pull all referenced images |
| `--push` | `bool` | | Shorthand for `--output=type=registry` |
| `-q`, `--quiet` | `bool` | | Suppress the build output and print image ID on success |
| `--root` | `string` | | Specify root directory of server to connect (EXPERIMENTAL) |
| `--sbom` | `string` | | Shorthand for `--attest=type=sbom` |
| `--secret` | `stringArray` | | Secret to expose to the build (format: `id=mysecret[,src=/local/secret]`) |
| `--server-config` | `string` | | Specify buildx server config file (used only when launching new server) (EXPERIMENTAL) |
| `--shm-size` | `bytes` | `0` | Shared memory size for build containers |
| `--ssh` | `stringArray` | | SSH agent socket or keys to expose to the build (format: `default\|<id>[=<socket>\|<key>[,<key>]]`) |
| `-t`, `--tag` | `stringArray` | | Name and optionally a tag (format: `name:tag`) |
| `--target` | `string` | | Set the target build stage to build |
| `--ulimit` | `ulimit` | | Ulimit options |
| Name | Type | Default | Description |
|:--------------------|:--------------|:----------|:----------------------------------------------------------------------------------------------------|
| `--add-host` | `stringSlice` | | Add a custom host-to-IP mapping (format: `host:ip`) |
| `--allow` | `stringSlice` | | Allow extra privileged entitlement (e.g., `network.host`, `security.insecure`) |
| `--annotation` | `stringArray` | | Add annotation to the image |
| `--attest` | `stringArray` | | Attestation parameters (format: `type=sbom,generator=image`) |
| `--build-arg` | `stringArray` | | Set build-time variables |
| `--build-context` | `stringArray` | | Additional build contexts (e.g., name=path) |
| `--builder` | `string` | | Override the configured builder instance |
| `--cache-from` | `stringArray` | | External cache sources (e.g., `user/app:cache`, `type=local,src=path/to/dir`) |
| `--cache-to` | `stringArray` | | Cache export destinations (e.g., `user/app:cache`, `type=local,dest=path/to/dir`) |
| `--call` | `string` | `build` | Set method for evaluating build (`check`, `outline`, `targets`) |
| `--cgroup-parent` | `string` | | Set the parent cgroup for the `RUN` instructions during build |
| `--check` | `bool` | | Shorthand for `--call=check` |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--detach` | `bool` | | Detach buildx server (supported only on linux) (EXPERIMENTAL) |
| `-f`, `--file` | `string` | | Name of the Dockerfile (default: `PATH/Dockerfile`) |
| `--iidfile` | `string` | | Write the image ID to a file |
| `--label` | `stringArray` | | Set metadata for an image |
| `--load` | `bool` | | Shorthand for `--output=type=docker` |
| `--metadata-file` | `string` | | Write build result metadata to a file |
| `--network` | `string` | `default` | Set the networking mode for the `RUN` instructions during build |
| `--no-cache` | `bool` | | Do not use cache when building the image |
| `--no-cache-filter` | `stringArray` | | Do not cache specified stages |
| `-o`, `--output` | `stringArray` | | Output destination (format: `type=local,dest=path`) |
| `--platform` | `stringArray` | | Set target platform for build |
| `--progress` | `string` | `auto` | Set type of progress output (`auto`, `plain`, `tty`, `rawjson`). Use plain to show container output |
| `--provenance` | `string` | | Shorthand for `--attest=type=provenance` |
| `--pull` | `bool` | | Always attempt to pull all referenced images |
| `--push` | `bool` | | Shorthand for `--output=type=registry` |
| `-q`, `--quiet` | `bool` | | Suppress the build output and print image ID on success |
| `--root` | `string` | | Specify root directory of server to connect (EXPERIMENTAL) |
| `--sbom` | `string` | | Shorthand for `--attest=type=sbom` |
| `--secret` | `stringArray` | | Secret to expose to the build (format: `id=mysecret[,src=/local/secret]`) |
| `--server-config` | `string` | | Specify buildx server config file (used only when launching new server) (EXPERIMENTAL) |
| `--shm-size` | `bytes` | `0` | Shared memory size for build containers |
| `--ssh` | `stringArray` | | SSH agent socket or keys to expose to the build (format: `default\|<id>[=<socket>\|<key>[,<key>]]`) |
| `-t`, `--tag` | `stringArray` | | Name and optionally a tag (format: `name:tag`) |
| `--target` | `string` | | Set the target build stage to build |
| `--ulimit` | `ulimit` | | Ulimit options |
<!---MARKER_GEN_END-->

View File

@@ -1,26 +0,0 @@
# docker buildx history
<!---MARKER_GEN_START-->
Commands to work on build records
### Subcommands
| Name | Description |
|:---------------------------------------|:-------------------------------|
| [`inspect`](buildx_history_inspect.md) | Inspect a build |
| [`logs`](buildx_history_logs.md) | Print the logs of a build |
| [`ls`](buildx_history_ls.md) | List build records |
| [`open`](buildx_history_open.md) | Open a build in Docker Desktop |
| [`rm`](buildx_history_rm.md) | Remove build records |
### Options
| Name | Type | Default | Description |
|:----------------|:---------|:--------|:-----------------------------------------|
| `--builder` | `string` | | Override the configured builder instance |
| `-D`, `--debug` | `bool` | | Enable debug logging |
<!---MARKER_GEN_END-->

View File

@@ -1,22 +0,0 @@
# docker buildx history inspect
<!---MARKER_GEN_START-->
Inspect a build
### Subcommands
| Name | Description |
|:-----------------------------------------------------|:---------------------------|
| [`attachment`](buildx_history_inspect_attachment.md) | Inspect a build attachment |
### Options
| Name | Type | Default | Description |
|:----------------|:---------|:--------|:-----------------------------------------|
| `--builder` | `string` | | Override the configured builder instance |
| `-D`, `--debug` | `bool` | | Enable debug logging |
<!---MARKER_GEN_END-->

View File

@@ -1,17 +0,0 @@
# docker buildx history inspect attachment
<!---MARKER_GEN_START-->
Inspect a build attachment
### Options
| Name | Type | Default | Description |
|:----------------|:---------|:--------|:-----------------------------------------|
| `--builder` | `string` | | Override the configured builder instance |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--platform` | `string` | | Platform of attachment |
| `--type` | `string` | | Type of attachment |
<!---MARKER_GEN_END-->

View File

@@ -1,16 +0,0 @@
# docker buildx history logs
<!---MARKER_GEN_START-->
Print the logs of a build
### Options
| Name | Type | Default | Description |
|:----------------|:---------|:--------|:--------------------------------------------------|
| `--builder` | `string` | | Override the configured builder instance |
| `-D`, `--debug` | `bool` | | Enable debug logging |
| `--progress` | `string` | `plain` | Set type of progress output (plain, rawjson, tty) |
<!---MARKER_GEN_END-->

Some files were not shown because too many files have changed in this diff Show More