Compare commits

...

930 Commits

Author SHA1 Message Date
Tõnis Tiigi
d537b9e418 Merge pull request #2115 from crazy-max/rework-driver-resolution2
build: rework node resolution
2023-11-14 08:43:03 -08:00
Justin Chadwell
616fb3e55c build: rework node resolution
This patch reworks and updates the node resolution logic for selecting a
node from a builder.

The new implementation reworks the logic to make use of containerd's
platforms.Matcher interface instead of manually associated strings, and
additionally provides a few behavioural changes over the original
implementation, namely platforms can be matched with non-strict
semantics. e.g. i386 builds can be scheduled on an amd64 node, arm/v6
builds can be scheduled on an arm/v7 node.

We also add a new collection of tests for tracking regressions and
making the intended behaviour clearer.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-14 15:08:30 +01:00
Tõnis Tiigi
80aa28f75c Merge pull request #2113 from tonistiigi/wait-child-release2
build: wait from child targets to complete before session release
2023-11-13 18:24:48 -08:00
Tõnis Tiigi
0408f3ac45 Merge pull request #2093 from jsternberg/rootless-init-config
driver: docker-container driver uses --config correctly in rootless mode
2023-11-13 09:10:07 -08:00
Tonis Tiigi
7683ef9137 build: wait from child targets to complete before session release
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-11-07 14:41:33 -08:00
Jonathan A. Sternberg
3f423468df driver: docker-container driver uses --config correctly in rootless mode
The `docker-container` driver relies on the default config file location
for buildkit when writing the configuration file. When run in a rootless
version of docker (dind), the default location is different.

Instead of trying to figure out where the appropriate default location
is, this just writes the files to the same location and sets the
`--config` parameter explicitly. This flag is placed first so a
user-specified config option in `--buildkitd-flags` will take precedence
over the implicit config parameter.

This also fixes the `--config` option with the rootless image.
Previously, the config directory was being copied in a way that rendered
`/etc` unreadable and the configuration file wasn't readable either. It
also wasn't copied to the correct place. Now, `--config` is used to
specify the directory, `/etc` isn't included in the copied archive (so
the permissions aren't overwritten), and the directory is set as world
readable to be readable from the rootless buildkit process`.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-11-07 16:20:07 -06:00
CrazyMax
ff8bca206b Merge pull request #2100 from dvdksn/bake-empty-definition
bake: improve error message for empty definition
2023-11-06 03:17:12 -08:00
CrazyMax
08a70ecdcc Merge pull request #2106 from jedevc/fix-bake-matrix-with-target-access
bake: fix global target access when using a matrix
2023-11-06 03:16:09 -08:00
Justin Chadwell
d83da63320 bake: fix global target access when using a matrix
Previously, we would fail while trying to use the global "target" field
when using a matrix. The contents of the matrix really don't matter for
this.

What was happening was that we would copy the "target" property into the
child evaluation context, so that when it was updated on the parent, it
wouldn't propagate to the child. The correct behavior here is to avoid
copying variables from the target evaluation context if it is the
root.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-02 17:49:58 +00:00
CrazyMax
639e0bc5ed Merge pull request #1838 from c-ameron/feat/local-remote-files
command(bake): Specify local and remote bake files
2023-11-02 05:19:40 -07:00
David Karlsson
d0a9a81e2e bake: improve error message for empty definition
Closes #2095

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-31 15:36:50 +01:00
CrazyMax
de1a560f07 Merge pull request #2099 from crazy-max/ci-codeql
ci: enable SAST scanning with CodeQL
2023-10-30 02:23:39 -07:00
CrazyMax
e168fd826c ci: enable SAST scanning with CodeQL
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-28 08:01:20 +02:00
Tõnis Tiigi
2f1b7a0131 Merge pull request #2098 from crazy-max/annotations-push
build: make annotations work with push flag
2023-10-26 09:14:23 -07:00
CrazyMax
f3871b158f build: make annotations work with push flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-26 12:01:10 +02:00
Tõnis Tiigi
deb9dbe9bb Merge pull request #1062 from crazy-max/bake-fix-attrs
bake: set attribute even if diagnosed as duplicated
2023-10-25 09:31:59 -07:00
Tõnis Tiigi
6f71ea8904 Merge pull request #2086 from ktock/debug-errormessage
debug: Following-up patches for error messages
2023-10-25 09:27:44 -07:00
CrazyMax
e437f7ba04 Merge pull request #1869 from crazy-max/fix-boot
builder: return error if all nodes fail to boot
2023-10-25 08:12:05 -07:00
Cameron Adams
abfc04f621 command(bake): Specify local and remote bake files
This adds the ability to source additional local build definition files when
sourcing Bake files via a remote url.
Prefixing a file with 'cwd://' will source a bake file on the local
machine, instead of the remote location.
Local files will be read/have precedence before remote files.

Usage:
```
docker buildx bake https://github.com/example/upstream.git --file cwd://docker-bake.override.hcl --print
```
This will source a default file from the example/upstream repository,
and also source a build definition from the local machine.

Also moves remote and local files reading logic to a func

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Cameron Adams <pnzreba@gmail.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-24 19:37:04 +02:00
CrazyMax
612dfdd813 builder: return error if all nodes fail to boot
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-24 10:59:04 +02:00
CrazyMax
ee19ce5ef2 Merge pull request #2076 from crazy-max/bake-display-files
bake: display read definition files in build output
2023-10-24 01:56:05 -07:00
CrazyMax
23c2498dee bake: display read definition files in build output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-23 16:26:18 +02:00
CrazyMax
390eedc50b Merge pull request #2096 from jedevc/vendor-buildkit
vendor: update buildkit to master@d5c1d785b042
2023-10-23 07:14:42 -07:00
Justin Chadwell
adc839aa40 vendor: update buildkit to master@d5c1d785b042
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 14:48:50 +01:00
CrazyMax
7838ade9f3 Merge pull request #2094 from crazy-max/update-compose
vendor: update compose-go to v1.20.0
2023-10-23 01:40:04 -07:00
CrazyMax
c043c9229e compose: fix deprecated Load func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-21 08:30:51 +02:00
CrazyMax
05a0fdf744 vendor: update compose-go to v1.20.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-21 08:29:01 +02:00
CrazyMax
dfb557b34f chore: fix lint issues
G601: Implicit memory aliasing in for loop. (gosec)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:39 +02:00
CrazyMax
d0d8bfbca4 bake: set attribute even if diagnosed as duplicated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:38 +02:00
CrazyMax
21e4eb89b2 bake: fork merged bodies interface logic from hcl repo and use it
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:38 +02:00
Kohei Tokunaga
14834e6085 debug: print error message on monitor instead of processes pkg
This can catch and print more errors on monitor side not only process-related
errors.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-20 14:09:32 +09:00
Kohei Tokunaga
267e30a19c debug: monitor: return last build result
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-20 14:09:28 +09:00
Tõnis Tiigi
be4fd7110d Merge pull request #2089 from dvdksn/docs-bakefile-lookup-order
docs: clarify bake file lookup order and merge
2023-10-19 09:22:29 -07:00
Tõnis Tiigi
24668122d9 Merge pull request #2088 from crazy-max/update-hcl
vendor: update hcl dependencies
2023-10-19 09:19:07 -07:00
Tõnis Tiigi
31d021a9ca Merge pull request #2015 from crazy-max/fix-bake-cwd-dockerfile
bake: local dockerfile support for remote definition
2023-10-19 09:18:20 -07:00
CrazyMax
7497e6481e bake: deny access to local dockerfile for remote invocation with local context
we don't currently support reading a remote Dockerfile with a local
context when doing a remote invocation because we automatically derive
the dockerfile from the context atm. To avoid mistakenly reading a local
Dockerfile, we check if the Dockerfile exists locally and if so, we
error out.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 17:41:05 +02:00
CrazyMax
de9d253f45 bake: local dockerfile support for remote definition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 17:41:05 +02:00
David Karlsson
f4f511201b docs: clarify bake file lookup order and merge
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-19 16:05:30 +02:00
CrazyMax
beca8b6adf vendor: update github.com/zclconf/go-cty to v1.14.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:52:01 +02:00
CrazyMax
457dc402d3 vendor: update github.com/hashicorp/go-cty-funcs to a090f58aa992
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:50:44 +02:00
CrazyMax
34b9a629a0 vendor: update github.com/hashicorp/hcl/v2 to v2.19.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:49:10 +02:00
CrazyMax
ad674e2666 Merge pull request #2084 from crazy-max/go-1.21
update to go 1.21.3
2023-10-19 03:55:10 -07:00
CrazyMax
503d483731 fix lint issues after update
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 12:31:24 +02:00
CrazyMax
6e5aefbb98 update golangci-lint to 1.54.2
also update config to show all issues at once and
set same exlucde rules as buildkit ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 12:24:16 +02:00
Tõnis Tiigi
7d2c9d5ef5 Merge pull request #2078 from crazy-max/fix-rm-state-group
build: fix racy localstate group removal
2023-10-18 16:54:21 -07:00
CrazyMax
1734abbb76 build: fix racy localstate group removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-18 19:12:46 +02:00
Kohei Tokunaga
b06a55cf53 debug: do not print error when error is nil
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-18 12:20:17 +09:00
Kohei Tokunaga
38137b29dd debug: reload: print error message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-18 11:25:24 +09:00
CrazyMax
fc7144f61d update to go 1.21.3
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-18 03:21:44 +02:00
Tõnis Tiigi
ac93a7fbfb Merge pull request #2006 from ktock/debug-command
debug: Add `buildx debug` command
2023-10-17 18:20:33 -07:00
CrazyMax
48f9b86b9a Merge pull request #2083 from crazy-max/test-host-gateway
test: docker host-gateway
2023-10-17 04:59:55 -07:00
CrazyMax
6c32a8c4c1 test: docker host-gateway
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 12:57:46 +02:00
CrazyMax
7a08248c4e Merge pull request #2082 from dvdksn/fix-docs-upstream-ci
ci: update docs upstream validation workflow
2023-10-16 16:39:05 -07:00
CrazyMax
05af608774 Merge pull request #2018 from crazy-max/driver-client-meta
driver(docker): opt to set additional dial meta to the client
2023-10-16 16:14:34 -07:00
CrazyMax
511e41386f driver(docker): opt to set additional dial meta to the client
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 01:01:17 +02:00
CrazyMax
fd251d2a7b builder: use variadic options in LoadNodes func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
CrazyMax
5836c24e7d driver: move Features and HostGatewayIP to specific driver
Adds a new HostGatewayIP entry in the Driver interface so we can
move Features and HostGatewayIP handling with cache to the very
specific driver (docker).

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
David Karlsson
c8f8a106ed ci: update docs upstream validation workflow
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-16 20:42:47 +02:00
Kohei Tokunaga
198764f116 debug: update docs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:36 +09:00
Kohei Tokunaga
0dd89f6029 monitor: print error information before launching monitor
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:35 +09:00
Kohei Tokunaga
8da8ee2aea controller: return original error to preserve stacktrace
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:35 +09:00
Kohei Tokunaga
6db8569f09 process: Do not print error log when process is canceled
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:30 +09:00
Kohei Tokunaga
5a0e4c1023 debug: set on=error by default
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:27 +09:00
Kohei Tokunaga
ded91da575 exec, rollback: return error when no session found
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:26 +09:00
Kohei Tokunaga
508b2ef0c6 debug: Add buildx debug command
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:21 +09:00
Tõnis Tiigi
05b8821625 Merge pull request #2048 from doreamon-design/feat/support-memory-opt-for-docker-container
feat: support memory/cpu driver options for docker-container
2023-10-12 10:05:54 -07:00
Tõnis Tiigi
01245e72ab Merge pull request #2072 from silvin-lubecki/undo-client-interface
Revert "Introduce a client interface, to abstract from buildkit client."
2023-10-11 16:40:28 -07:00
Silvin Lubecki
22e9e3342b Revert "Introduce a client interface, to abstract from buildkit client."
This reverts commit 2aa6d52b06.

Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
2023-10-11 22:13:23 +02:00
Silvin Lubecki
0e3911147a Merge pull request #2071 from silvin-lubecki/client-interface
(drivers): Introduce a client interface
2023-10-11 22:08:35 +02:00
Silvin Lubecki
2aa6d52b06 Introduce a client interface, to abstract from buildkit client.
This will allow further enhancement like caching remote driver responses.

Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
2023-10-06 14:24:04 +02:00
CrazyMax
561a4330cf Merge pull request #2065 from dvdksn/update-compose-go
vendor: update compose-go to v1.19.0
2023-10-03 22:26:17 +02:00
CrazyMax
7b4bc4f00a Merge pull request #2063 from crazy-max/node-workerid
builder: set node worker ids
2023-10-02 09:11:27 +02:00
CrazyMax
a012e0043b Merge pull request #2066 from jsternberg/early-flock-release
commands: release the file lock before attempting to boot during create
2023-09-30 12:40:02 +02:00
Justin Chadwell
2c2294fa43 Merge pull request #2068 from crazy-max/inspect-fix-platforms 2023-09-30 09:25:02 +01:00
CrazyMax
197824c6f2 builder: set node worker ids
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 07:10:02 +02:00
CrazyMax
22e61ef06f inspect: do not display platforms field if empty
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 07:08:47 +02:00
CrazyMax
159eac42f3 compose: test with develop section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 06:46:51 +02:00
CrazyMax
6c77b76b7b Merge pull request #2038 from crazy-max/localstate-group
build: support local state group
2023-09-30 06:01:14 +02:00
CrazyMax
130e9fe093 localstate: tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
e9fb769c60 build: handle local state group file removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
3dcb03452c build: support local state group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
9b7d30c9a0 build: opt to set build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:03:00 +02:00
CrazyMax
2134a1e104 build: set target to local state
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:03:00 +02:00
CrazyMax
cc6957d1cc build: move localstate logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:02:45 +02:00
Jonathan A. Sternberg
0878d5b22b commands: release the file lock before attempting to boot during create
If the boot command hung or took a long time, it blocked any read
operations (such as `buildx ls`).

When the boot happens, we no longer need to hold the file lock so we can
release it. Releasing multiple times is legal and causes whichever
release that is second to be a no-op so the defer is kept to ensure the
lock is released even when an error condition happens.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-09-29 15:53:38 -05:00
David Karlsson
c8002e58a4 vendor: update compose-go to v1.19.0
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-09-29 20:59:14 +02:00
Zero
cfcd1d9420 feat: support memory and cpu driver options for docker-container
Signed-off-by: Zero <tobewhatwewant@outlook.com>
2023-09-29 23:57:19 +08:00
CrazyMax
e6756d951a Merge pull request #2035 from jedevc/update-buildkit
vendor: update buildkit to master
2023-09-28 15:14:00 +02:00
Justin Chadwell
b9aad03e7a progress: create new display object after unpause
Without this, we attempt to overwrite the previous display in the lines
above which have been filled with logs from the monitor (see when used
with `--invoke`).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell
0bd6f3c7f5 remote: remove explicit backoff config from client opts
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell
e2ebab5f26 vendor: update buildkit to master@cbfd4023383d
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
CrazyMax
e018f8b6fb Merge pull request #1994 from crazy-max/load-progress
build: sublogger to show docker load progress output
2023-09-13 08:21:44 +02:00
CrazyMax
03bedfb3c3 build: sublogger to show docker load progress output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-12 22:24:29 +02:00
CrazyMax
bdaaca40a2 Merge pull request #2045 from docker/dependabot/github_actions/docker/bake-action-4
build(deps): bump docker/bake-action from 3 to 4
2023-09-12 21:23:04 +02:00
dependabot[bot]
bc021c89a8 build(deps): bump docker/bake-action from 3 to 4
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 3 to 4.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:55:11 +00:00
CrazyMax
798402314c Merge pull request #2046 from docker/dependabot/github_actions/docker/login-action-3
build(deps): bump docker/login-action from 2 to 3
2023-09-12 20:54:59 +02:00
CrazyMax
7cfb440136 Merge pull request #2044 from docker/dependabot/github_actions/docker/metadata-action-5
build(deps): bump docker/metadata-action from 4 to 5
2023-09-12 20:54:30 +02:00
CrazyMax
80358842c4 Merge pull request #2043 from docker/dependabot/github_actions/docker/setup-buildx-action-3
build(deps): bump docker/setup-buildx-action from 2 to 3
2023-09-12 20:54:16 +02:00
CrazyMax
77aedb751e Merge pull request #2042 from docker/dependabot/github_actions/docker/setup-qemu-action-3
build(deps): bump docker/setup-qemu-action from 2 to 3
2023-09-12 20:54:01 +02:00
dependabot[bot]
739ec964db build(deps): bump docker/login-action from 2 to 3
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:26 +00:00
dependabot[bot]
320a3109f3 build(deps): bump docker/metadata-action from 4 to 5
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:13 +00:00
dependabot[bot]
2c986bc184 build(deps): bump docker/setup-buildx-action from 2 to 3
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:07 +00:00
dependabot[bot]
6c31f43cc9 build(deps): bump docker/setup-qemu-action from 2 to 3
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:01 +00:00
CrazyMax
7b049b99c5 Merge pull request #1998 from dvdksn/build/docker-driver-errmsgs
build: improve error messages for docker driver
2023-09-12 11:32:32 +02:00
David Karlsson
bf5a70023c build: improve error messages for docker driver
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-09-12 09:39:51 +02:00
Justin Chadwell
8d001e338f Merge pull request #2040 from crazy-max/validate-build-args 2023-09-11 16:51:20 +01:00
CrazyMax
73ea0826ca build: validate build args and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-11 16:56:50 +02:00
Justin Chadwell
66e6dab26b Merge pull request #2020 from jedevc/build-annotation-flags 2023-09-11 15:55:01 +01:00
Justin Chadwell
0138f2a00f bake: add annotations field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:07 +01:00
Justin Chadwell
a59058e8a5 build: add --annotation shortcut flag
This extracts the same logic for parsing annotations from the imagetools
create command, and allows the same flags to be attached to the build
command.

These annotations are then merged into all provided exporters.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:04 +01:00
CrazyMax
f6b7a3c522 Merge pull request #2039 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-11 16:30:46 +02:00
Justin Chadwell
8fe2070d10 imagetools: make annotation parser more generic
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Justin Chadwell
54bb799d15 imagetools: simplify return type of annotation parser
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Sebastiaan van Stijn
957044825f update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:53:03 +02:00
CrazyMax
42a0f3d504 Merge pull request #2037 from kd-collective/fix_broken_link
docs: remove broken link
2023-09-09 09:03:15 +02:00
Kijima Daigo
84ad208985 docs: remove broken link
Signed-off-by: Kijima Daigo <norimaking777@gmail.com>
2023-09-09 11:51:56 +09:00
CrazyMax
3631dc17c9 Merge pull request #2027 from thaJeztah/swap_reference
migrate reference github.com/distribution/reference
2023-09-08 15:49:08 +02:00
Sebastiaan van Stijn
bafdc63b8c migrate to github.com/distribution/reference v0.5.0
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Also update compose-go, which also switched to distribution/reference;

full diff: https://github.com/compose-spec/compose-go/compare/v1.18.3...v1.18.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:41:35 +02:00
Sebastiaan van Stijn
51c94cd2a6 vendor: github.com/compose-spec/compose-go v1.18.3
- Parse service device count to int if possible
- introduce ResourceResolver to accept remote resources
- use include.env_file to resolve variables in included compose.yaml file
- remove potential dependencies to disabled services in ForServices
- ability to convert a mapping (back) to KEY=VALUE strings
- load: include details about included files on Project
- include disabled services
- local environment to override included .env
- load: move env var profile detection to option
- add support for multi-document yaml

full diff: https://github.com/compose-spec/compose-go/compare/v1.17.0...v1.18.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:41:35 +02:00
Tõnis Tiigi
31d88398bc Merge pull request #2031 from thaJeztah/bump_notary
vendor: github.com/theupdateframework/notary v0.7.0
2023-09-06 08:37:17 -07:00
Justin Chadwell
fbf6594758 Merge pull request #2034 from tonistiigi/trace-delegate-driver 2023-09-06 11:20:56 +01:00
Tonis Tiigi
f54a67de6d enable trace delegation for docker/remote driver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-09-05 22:04:57 -07:00
CrazyMax
f35b2b7cab Merge pull request #2032 from docker/dependabot/github_actions/actions/checkout-4
build(deps): bump actions/checkout from 3 to 4
2023-09-05 08:36:03 +02:00
dependabot[bot]
29ba5ecef6 build(deps): bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 18:51:00 +00:00
Sebastiaan van Stijn
fb50d82fd8 vendor: github.com/theupdateframework/notary v0.7.0
update the dependency to v0.7.0 to be closer to what docker/cli uses;
https://github.com/theupdateframework/notary/compare/v0.6.1...v0.7.0

Note that docker/cli is slightly ahead of v0.7.0, and uses bf96a202a09a;
https://github.com/theupdateframework/notary/compare/v0.7.0...bf96a202a09a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 15:37:45 +02:00
Justin Chadwell
87e8e4b847 Merge pull request #2029 from testwill/loop
chore: slice loop replace
2023-08-31 07:10:10 +02:00
guoguangwu
a71a24c0f4 chore: slice loop replace
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-08-31 12:24:02 +08:00
Tõnis Tiigi
76119b0f61 Merge pull request #2026 from ktock/fix-invoke-flag
debug: fix short-form custom command name on `--invoke` isn't used
2023-08-30 10:05:49 -07:00
Kohei Tokunaga
7843b5f417 debug: fix short-form custom command name on --invoke isn't used
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-08-30 18:41:16 +09:00
Justin Chadwell
da6662975f Merge pull request #2019 from jedevc/chore-bake-field-reorder 2023-08-24 14:17:55 +01:00
Justin Chadwell
de4dbb7d00 chore: reorder target fields
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-24 14:01:14 +01:00
Justin Chadwell
3bd4bca994 Merge pull request #2013 from jedevc/tests-add-imagetools-create
tests: add imagetools tests for copying manifests and indexes
2023-08-23 13:31:45 +01:00
CrazyMax
296832c90e Merge pull request #2014 from crazy-max/fix-compose-test
test: fix non-deterministic compose context path
2023-08-23 11:27:47 +02:00
CrazyMax
56d55a4137 test: fix non-deterministic compose context path
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-23 11:15:50 +02:00
CrazyMax
626e6f8fa3 Merge pull request #1905 from thaJeztah/cgroup_parent_description
update flag-description for --cgroup-parent
2023-08-23 10:15:47 +02:00
Justin Chadwell
5941bf0494 tests: add imagetools tests for copying manifests and indexes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 17:38:36 +01:00
CrazyMax
29a496cdab Merge pull request #2012 from jedevc/git-propogate-errors
git: propogate failure to locate git binary
2023-08-22 13:30:36 +02:00
Justin Chadwell
a43d9a67c7 git: fix error wrapping to ensure internal errors are propogated
Also, tidy up the error printing, so that now we always print out the
"current commit information was not captured by the build" message,
instead of just for not locating the git binary.

Before:

	WARNING: buildx: git was not found in the system. Current commit information was not captured by the build

After:

	WARNING: current commit information was not captured by the build: git was not found in the system: <error message>

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:17:17 +01:00
Justin Chadwell
c47eb3bf5a git: propogate failure to locate git binary
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:16:19 +01:00
CrazyMax
a97e1641a4 Merge pull request #2000 from jedevc/fix-race-container-creation
docker-container: avoid fail if container conflict
2023-08-21 16:11:02 +02:00
Akihiro Suda
86ae8ea854 Merge pull request #1999 from crazy-max/update-k8s
vendor: bump k8s to v0.26.7
2023-08-10 23:26:24 +09:00
Justin Chadwell
d37d483097 docker-container: avoid fail if container conflict
Fixes the race condition where two boots are executed simultaneously
across multiple processes.

We initially check to see if the container exists, but if during
container creation we get a name conflict, we don't treat this error as
a hard failure, and instead move immediately into waiting for the node
to boot.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-10 14:29:36 +01:00
CrazyMax
4e96faa201 vendor: bump k8s to v0.26.7
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-10 15:22:24 +02:00
Tõnis Tiigi
e5419ef6d7 Merge pull request #1927 from crazy-max/fix-load-status
build: read body response to check for erroneous image export to docker
2023-08-10 12:29:03 +03:00
CrazyMax
14747a490a Merge pull request #1971 from glours/bump-compose-go-v1.17.0
bump compose-go version to v1.17.0 to fix issue with depends_on
2023-08-08 21:03:59 +02:00
Justin Chadwell
e5cee892ed Merge pull request #1965 from mqasimsarfraz/qasim/oci-annotations 2023-08-08 14:36:26 +01:00
CrazyMax
ef4b984df4 build: read body response to check for erroneous image export to docker
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-08 06:34:24 +02:00
CrazyMax
a8f402e28d Merge pull request #1993 from kenyon/typo-fix
README: fix typo
2023-08-08 06:32:44 +02:00
Kenyon Ralph
2eba99b40b README: fix whitespace
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-07 11:34:13 -07:00
Kenyon Ralph
7686fa1f16 README: fix typo
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-07 11:33:54 -07:00
Justin Chadwell
51b9bab245 Merge pull request #1987 from jedevc/vendor-buildkit-master-tests 2023-08-07 17:08:16 +01:00
CrazyMax
6b5758f4cd Merge pull request #1821 from jedevc/allow-debug-env
commands: consume DEBUG environment variable
2023-08-07 15:06:47 +02:00
CrazyMax
bd375a14a8 Merge pull request #1940 from kenyon/patch-1
README: clarify the Linux package install instructions
2023-08-07 14:43:29 +02:00
CrazyMax
b01693f63e bake: test compose include
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-04 17:24:30 +02:00
Guillaume Lours
4a059d5144 adapt compose unit tests, build context is now transformed to absolute paths by compose-go
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-08-04 17:23:12 +02:00
Guillaume Lours
f3775c0046 bump compose-go version to v1.17.0 to fix issue with depends_on
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-08-04 17:23:04 +02:00
Justin Chadwell
50fbdd86f9 Merge pull request #1985 from jsternberg/integration-test/version 2023-08-04 16:14:42 +01:00
Justin Chadwell
1f61de0fcc Merge pull request #1988 from jedevc/fix-attests-on-docker-driver 2023-08-04 16:07:21 +01:00
Justin Chadwell
e206c585bb build: error on attests on non-multiplatform driver
On drivers that do not support multi-platform builds (the default
`docker` driver), we do not support building attestations (unless using
the containerd store).

We need to check this feature before attempting to build using
attestations.

Also adds a test to ensure that attestations can be pushed to registries
at all, and that it adequately fails on the docker driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 15:28:32 +01:00
Justin Chadwell
5e46d8057d tests: add unsupported features detection skeleton
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell
4e7709e54c vendor: update buildkit to master@b49a8873179b
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell
5ed8f1b7d9 tests: avoid hardcoded driver check in testImageIDOutput
To detect if there is a docker daemon available, we can use the sandbox
.DockerAddress() function.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 11:22:10 +01:00
Jonathan A. Sternberg
1d12c1f5b3 Integration test for docker buildx version
An integration test for `docker buildx version` has been created. The
integration test checks that there is one line output, the output is
composed of three sections, and that these sections could feasibly be
the package path, version, and revision information.

The intention of the checks is to find obvious errors in the output like
the package path not existing or the version and revision being swapped.
It is not intended to assert that these values must be certain values
because it is assumed these values may vary depending on the build
process for buildx.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-08-03 09:51:03 -05:00
Qasim Sarfraz
3ef93e081c tests: add testImagetoolsAnnotation integration test
Signed-off-by: Qasim Sarfraz <qasimsarfraz@microsoft.com>
2023-08-03 12:16:40 +02:00
Qasim Sarfraz
18894a8e3a allow annotations for OCI image index
Signed-off-by: Qasim Sarfraz <qasimsarfraz@microsoft.com>
2023-08-03 12:08:04 +02:00
Justin Chadwell
13ec635988 Merge pull request #1914 from ktock/updatedebugdocs 2023-08-03 09:52:13 +01:00
Tõnis Tiigi
f804b8fa4b Merge pull request #1982 from thaJeztah/update_go1.20.7
update to go1.20.7
2023-08-03 10:59:18 +03:00
Kenyon Ralph
21a55ff9a1 README: clarify the Linux package install instructions
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-02 13:55:22 -07:00
Sebastiaan van Stijn
dd350284df update to go1.20.7
Includes a fix for CVE-2023-29409

go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7

From the mailing list announcement:

[security] Go 1.20.7 and Go 1.19.12 are released

Hello gophers,

We have just released Go versions 1.20.7 and 1.19.12, minor point releases.

These minor releases include 1 security fixes following the security policy:

- crypto/tls: restrict RSA keys in certificates to <= 8192 bits

  Extremely large RSA keys in certificate chains can cause a client/server
  to expend significant CPU time verifying signatures. Limit this by
  restricting the size of RSA keys transmitted during handshakes to <=
  8192 bits.

  Based on a survey of publicly trusted RSA keys, there are currently only
  three certificates in circulation with keys larger than this, and all
  three appear to be test certificates that are not actively deployed. It
  is possible there are larger keys in use in private PKIs, but we target
  the web PKI, so causing breakage here in the interests of increasing the
  default safety of users of crypto/tls seems reasonable.

  Thanks to Mateusz Poliwczak for reporting this issue.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 00:06:05 +02:00
Tõnis Tiigi
c010d3de8d Merge pull request #1967 from thaJeztah/update_cli
vendor: github.com/docker/docker, github.com/docker/cli v24.0.5
2023-08-01 11:51:44 +03:00
Justin Chadwell
d11dbbf9f7 Merge pull request #1978 from tonistiigi/imagetools-test 2023-07-31 15:39:42 +01:00
Tonis Tiigi
75cdceb9f1 tests: add integration test for imagetools create
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-31 17:17:11 +03:00
Tõnis Tiigi
10ff93f190 Merge pull request #1968 from jedevc/reset-dont-modify-input
progress: don't modify ResetTime inputs
2023-07-31 16:02:23 +03:00
CrazyMax
bf00185809 Merge pull request #1681 from crazy-max/fup-buildinfo
docs: update since buildinfo removal
2023-07-31 10:55:30 +02:00
CrazyMax
90f03e57c2 Merge pull request #1972 from crazy-max/docs-fix-create
docs: fix platform example for create command
2023-07-31 10:54:26 +02:00
Justin Chadwell
a59fd3ebfe Merge pull request #1970 from ktock/entrypointconfig 2023-07-27 15:21:37 +01:00
Kohei Tokunaga
3eb490153d remote controller: Fix entrypoint interaction bugs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-07-27 09:12:52 +09:00
CrazyMax
d957d8b987 docs: fix platform example for create command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-26 17:50:58 +02:00
Justin Chadwell
5a1f252bd9 progress: don't modify ResetTime inputs
No other parts of the progress rendering modify the inputs, so we should
avoid this as well.

This actually fixes an edge case in pushWithMoby which writes the same
VertexStatus multiple times, modifying the timestamps and similar.
However, if the operation takes long enough the small time difference
can accumulate, and move the Start time far into the past.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-25 09:35:43 +01:00
Sebastiaan van Stijn
ab4585f38c vendor: github.com/docker/cli v24.0.5
Fix a panic when `auths: null` is found in the CLI config file.

full diff: https://github.com/docker/cli/compare/v24.0.4...v24.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:09:50 +02:00
Sebastiaan van Stijn
3003045c0b vendor: github.com/docker/docker v24.0.5
- client: Client.postHijacked: use Client.buildRequest

full diff: https://github.com/moby/moby/compare/36e9e796c6fc...v24.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:09:02 +02:00
Sebastiaan van Stijn
a6f3f290b4 vendor: golang.org/x/net v0.10.0
- http2: properly discard data received after request/response body is closed
- http2: don't reuse connections that are experiencing errors
- internal/socks: permit authenticating with an empty password

full diff: https://github.com/golang/net/compare/v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:05:20 +02:00
Sebastiaan van Stijn
27d072a099 vendor: golang.org/x/text v0.9.0
no changes in vendored files

full diff: https://github.com/golang/text/compare/v0.8.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:04:17 +02:00
Sebastiaan van Stijn
8e3df1943c vendor: golang.org/x/term v0.8.0
no changes in vendored files

full diff: https://github.com/golang/term/compare/v0.6.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:03:14 +02:00
Sebastiaan van Stijn
8c54de66ce vendor: golang.org/x/sys v0.8.0
full diff: https://github.com/golang/sys/compare/v0.7.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:02:03 +02:00
Sebastiaan van Stijn
06b9ac2dc4 vendor: github.com/sirupsen/logrus v1.9.3
Fix a potential denial of service in logrus.Writer() that could be triggered
by logging text longer than 64kb without newlines.

full diff: https://github.com/sirupsen/logrus/compare/v1.9.0...v1.9.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 08:59:00 +02:00
Justin Chadwell
b8739d7441 Merge pull request #1934 from jedevc/use-buildkit-client-wait 2023-07-19 16:23:54 +01:00
Justin Chadwell
23fe02993b Merge pull request #1963 from jedevc/split-to-driver-pairs-fix-type 2023-07-19 15:57:43 +01:00
Justin Chadwell
1d177f00d2 chore: tidy splitToDriverPairs to avoid unneccessary int
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 14:43:50 +01:00
CrazyMax
ceaba7011f Merge pull request #1938 from jonapich/feature/custom-annotations
kubernetes driver // allow custom annotations and labels
2023-07-19 14:21:12 +02:00
Jonathan Piché
9c06f383ba allow custom annotations and labels into kubernetes manifests
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>

Signed-off-by: Jonathan Piché <jpiche@coveo.com>
2023-07-19 07:48:26 -04:00
Justin Chadwell
e11c5e3e96 remote: use buildkit's client.Wait method to bootstrap
This native implementation uses GRPC level waiting, instead of starting
a busy loop. We also a manual max backoff of one second to improve
responsiveness.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 12:39:19 +01:00
CrazyMax
f5719f3017 Merge pull request #1959 from thaJeztah/update_cli
vendor: github.com/docker/cli v24.0.4
2023-07-19 11:57:02 +02:00
CrazyMax
163babdca7 Merge pull request #1961 from crazy-max/fix-internal-build
build: set remote bake def and remote dockerfile as internal solve
2023-07-19 11:47:52 +02:00
Sebastiaan van Stijn
094d1aded8 commands: NewRootCmd: remove obsolete logrus filter hook
This hook was added in 278f94a8b6 and
72758fef22 to suppress spurious warnings
printed by the CLI's cli/connhelper/commandconn package;
3fb4fb83df/cli/connhelper/commandconn/commandconn.go (L203-L214)

Those logs were removed in a5ebe2282a
so we can remove the hook.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 11:04:45 +02:00
CrazyMax
05ef20b434 build: set remote bake def and remote dockerfile as internal solve
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-19 04:18:28 +02:00
Sebastiaan van Stijn
cc718b3444 vendor: github.com/docker/cli v24.0.4
full diff: https://github.com/docker/cli/compare/v24.0.2...v24.0.4

notable changes:

- ssh: fix error on commandconn close, add ping and default
- commandconn: return original error while closing

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 17:13:42 +02:00
Justin Chadwell
e98e8f6ac9 Merge pull request #1948 from thaJeztah/buildkit_0.12 2023-07-18 15:05:18 +01:00
CrazyMax
36541ed9d5 Merge pull request #1954 from crazy-max/result-handle-internal
build: mark result handle build as internal
2023-07-17 17:05:34 +02:00
CrazyMax
418ea82d3a build: mark result handle build as internal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 16:53:12 +02:00
Sebastiaan van Stijn
130bbda00e vendor: github.com/moby/buildkit v0.12.1-0.20230717122532-faa0cc7da353
full diff:

- https://github.com/moby/buildkit/compare/20230620112432...v0.12.0
- https://github.com/moby/buildkit/compare/v0.12.0...faa0cc7da3536923d85b74b2bb2d13c12a6ecc99

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:37:27 +02:00
Sebastiaan van Stijn
2666bd6996 vendor: github.com/tonistiigi/vt100 v0.0.0-20230623042737-f9a4f7ef6531
full diff: 8066bb9726...f9a4f7ef65

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:43 +02:00
Sebastiaan van Stijn
ff2c8da803 vendor: github.com/tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
full diff: 9e7a6df485...36ef4d8c0d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:43 +02:00
Sebastiaan van Stijn
e094296f37 vendor: github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb
full diff: 4e3ac2762d...02993c407b

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:40 +02:00
Justin Chadwell
7c3b77fb36 Merge pull request #1950 from thaJeztah/remove_imageutil_dead_code 2023-07-17 12:26:05 +01:00
CrazyMax
fb4c4f07ca Merge pull request #1941 from crazy-max/fix-kube-config
k8s: fix missing kubeconfig check from endpoint
2023-07-17 12:36:27 +02:00
Sebastiaan van Stijn
b9e25e82cf util/imagetools: remove unused Resolver.ImageConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 12:35:42 +02:00
CrazyMax
089036da29 Merge pull request #1946 from crazy-max/update-go
update go to 1.20.6
2023-07-17 12:01:26 +02:00
CrazyMax
1123bfed10 hack(generated-files): bump golang image to bookworm
#7 [internal] load metadata for docker.io/library/golang:1.20.6-buster
#7 ERROR: docker.io/library/golang:1.20.6-buster: not found

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 11:18:37 +02:00
CrazyMax
7f2293308b update go to 1.20.6
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 10:30:00 +02:00
CrazyMax
a65131f9d3 Merge pull request #1945 from crazy-max/bump-docker
vendor: github.com/docker/docker@24.0 36e9e79
2023-07-17 10:29:40 +02:00
CrazyMax
8a3a646c61 vendor: github.com/docker/docker@24.0 36e9e79
client: define a "dummy" hostname to use for local connections
fixes "http: invalid Host header" errors when compiling with go1.20.6
or go1.19.11

full diff: https://github.com/docker/docker/compare/v24.0.2...36e9e796c6fc

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 10:06:04 +02:00
CrazyMax
4384947be1 k8s: fix missing kubeconfig check from endpoint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-15 21:51:56 +02:00
Tõnis Tiigi
69421182ca Merge pull request #1891 from droopy4096/auth-token
Add Bearer token support
2023-07-12 11:57:47 -07:00
Justin Chadwell
068382f5df Merge pull request #1936 from jedevc/hack-set-go-version-to-1.20.5 2023-07-12 14:49:59 +01:00
Justin Chadwell
c4bec05466 hack: force go version to 1.20.5
A temporary workaround for "http: invalid Host header" introduced in
go 1.20.6.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-12 14:31:23 +01:00
CrazyMax
89e1ac0a6e Merge pull request #1894 from crazy-max/fix-host-gateway
build: fix host-gateway handling
2023-07-03 21:53:27 +02:00
Justin Chadwell
b84e0e11b4 Merge pull request #1918 from crazy-max/docs-hidden-fix 2023-07-03 11:50:04 +01:00
Justin Chadwell
d95f5f8f3b Merge pull request #1925 from dvdksn/test/build-progress 2023-07-03 10:48:52 +01:00
David Karlsson
b4c0941683 tests: add basic build progress test
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-07-02 21:09:33 +02:00
Justin Chadwell
cf9798cede Merge pull request #1919 from crazy-max/fix-build-details-link 2023-06-30 13:51:23 +01:00
CrazyMax
20d2501edc test: build details output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-30 13:48:02 +02:00
CrazyMax
d45601fdc6 build: missing newline when printing build details on error
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-30 13:48:02 +02:00
CrazyMax
c81a9a89cf Merge pull request #1913 from yastanotheruser/master
Controller: Include CgroupParent in build.Options
2023-06-30 00:07:15 +02:00
CrazyMax
87b9f9ecfb docs: update generated content
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-27 17:21:05 +02:00
CrazyMax
cbc473359a vendor: update cli-docs-tool to 0.6.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-27 17:21:04 +02:00
Justin Chadwell
2eba60db75 Merge pull request #1916 from jedevc/add-local-bake-test 2023-06-27 11:28:47 +01:00
Justin Chadwell
0dcbed3f53 tests: add simple local bake test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-27 11:16:44 +01:00
Kohei Tokunaga
ca08eb65e2 docs: debug: update the output of help command to the latest message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-27 15:17:01 +09:00
Kohei Tokunaga
6f37d9bee7 monitor: attach: fix typo in long help message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-27 15:16:32 +09:00
Jhan S. Álvarez
e65f6b8c8b controller: include CgroupParent in build.Options
Signed-off-by: Jhan S. Álvarez <alvarezpcuser@gmail.com>
2023-06-26 23:18:53 -05:00
CrazyMax
707dc43d55 Merge pull request #1903 from crazy-max/fix-bake-compose-profiles
bake: ignore profiles in compose definitions
2023-06-26 15:02:53 +02:00
CrazyMax
8cbb7a9319 build: fix host-gateway handling
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-24 17:47:48 +02:00
CrazyMax
4f5a56aadb Merge pull request #1904 from thaJeztah/cleanup_ParseEntitlements
utils/buildflags: ParseEntitlements(): use BuildKit's parsing
2023-06-23 14:26:44 +02:00
Sebastiaan van Stijn
399beb53d9 utils/buildflags: ParseEntitlements(): use BuildKit's parsing
Use buildkit's parsing of entitlements to make sure that accepted
values match what's accepted by BuildKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-23 10:44:30 +02:00
Sebastiaan van Stijn
7dec9fd6e7 update flag-description for --cgroup-parent
This attempts to make it clearer that the --cgroup-parent option is only used
for the containers used during build. Instead of mentioning "build container",
I opted for using "RUN instructions" (to match the --network description),
although this may not be ideal (as it assumes the "Dockerfile" front-end, which
of course may not be the case).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-23 10:28:50 +02:00
CrazyMax
120f3a8918 bake: ignore profiles in compose definitions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-22 17:44:35 +02:00
CrazyMax
bd672eaf5b Merge pull request #1886 from crazy-max/docker-local
build: prefer local image resolution for docker driver
2023-06-21 11:20:24 +02:00
CrazyMax
c2500ea2d8 build: prefer local image resolution for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 11:05:06 +02:00
Justin Chadwell
a4663b4b2e Merge pull request #1900 from cyphar/build-go_extra_flags 2023-06-21 09:59:19 +01:00
Aleksa Sarai
57c618b83a build: add GO_EXTRA_FLAGS argument
This is useful for setting things like -buildmode=pie when packaging
docker-buildx for distributions.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2023-06-21 18:48:05 +10:00
CrazyMax
b3a4f95110 Merge pull request #1897 from crazy-max/test-containerd-snap
test: register docker worker with containerd snapshotter
2023-06-21 00:11:31 +02:00
CrazyMax
28a1eb3527 test: fix testImageIDOutput
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
75ecc15958 test: fix inspect and ls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
2235ebce2f test: register docker worker with containerd snapshotter
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
7147463418 dockerfile: update docker to 24.0.2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 16:50:38 +02:00
Justin Chadwell
010e4c8d54 Merge pull request #1890 from jedevc/tests-share-docker-container-backend 2023-06-20 14:24:09 +01:00
Justin Chadwell
6f394a0691 tests: set a dedicated buildx config dir for each worker
This should help reduce any unexpected config conflict between workers.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:40:45 +01:00
Justin Chadwell
efd7279118 ci: run docker-container tests in parallel
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:51 +01:00
Justin Chadwell
601056f3a7 tests: share single docker between docker-container backends
This means that we can run our docker-container tests in parallel again,
which can help speed up our test runs by a *significant* amount.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:49 +01:00
Justin Chadwell
0a7f96cbfb vendor: update buildkit to master@2d91ddcceedc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:27:46 +01:00
CrazyMax
1c530c2fe0 Merge pull request #1896 from dvdksn/docs/add-experimental-debugmonitor
docs: add experimental annotation for debug-shell command
2023-06-20 12:46:26 +02:00
David Karlsson
1e576dd7c6 chore: make docs
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-06-20 12:05:30 +02:00
CrazyMax
7a5472153b docs: set experimental annotation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 11:42:02 +02:00
Dmitry Makovey
b986ce566b Add Bearer token
Signed-off-by: Dmitry Makovey <dmakovey@gitlab.com>
2023-06-16 14:55:09 -06:00
CrazyMax
daba16f4be Merge pull request #1879 from crazy-max/fix-ctx-validation
builder: skip name validation for docker context
2023-06-15 14:05:37 +02:00
CrazyMax
ee36e2264e Merge pull request #1880 from jedevc/fix-dockerfile-cwd-join
bake: fix incorrect dockerfile resolution against `cwd://` context
2023-06-15 14:05:13 +02:00
CrazyMax
329e98d9f0 Merge pull request #1883 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.2
build(deps): Bump peter-evans/create-pull-request from 5.0.1 to 5.0.2
2023-06-15 13:55:26 +02:00
dependabot[bot]
f4513f7028 build(deps): Bump peter-evans/create-pull-request from 5.0.1 to 5.0.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.1 to 5.0.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](284f54f989...153407881e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-14 18:59:43 +00:00
CrazyMax
b1c5449428 builder: skip name validation for docker context
Although a builder from the store cannot be created unless
it has a valid name, this is not the case for a Docker context.

We should skip name validation when checking a node from the
store and fall back to finding one from Docker context instead.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-14 14:19:30 +02:00
Justin Chadwell
431732f5d1 bake: fix incorrect dockerfile resolution against cwd:// context
We need to resolve the strip the cwd:// prefix before attempting to
resolve the dockerfile. Otherwise, we'll get the cwd:// prefix in the
dockerfile name, which isn't stripped out later.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-14 12:18:53 +01:00
CrazyMax
687feca9e8 Merge pull request #1877 from jedevc/cache-dockerutil-features
dockerutil: cache feature collection
2023-06-13 11:42:22 +02:00
Justin Chadwell
d4a2c8d0c3 dockerutil: cache feature collection
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-13 09:47:38 +01:00
Justin Chadwell
bef42b2441 Merge pull request #1876 from tonistiigi/remove-history-request 2023-06-13 09:38:05 +01:00
CrazyMax
2de333fdd3 check history api support once
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-13 10:29:22 +02:00
Tonis Tiigi
1138789f20 avoid extra client for history API detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-12 17:39:09 -07:00
Justin Chadwell
1f4ac09ffb Merge pull request #1874 from thaJeztah/bump_engine_24.0.2 2023-06-09 11:39:13 +01:00
CrazyMax
26a8ffb393 Merge pull request #1873 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@67a08623b95a
2023-06-09 12:14:11 +02:00
CrazyMax
9b7aada99b Merge pull request #1716 from distorhead/use-docker-cli-in-stream
build(cli): allow passing in-stream using command.Cli
2023-06-09 12:11:47 +02:00
Sebastiaan van Stijn
fd6207695b vendor: github.com/docker/cli v24.0.2
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:44 +02:00
Sebastiaan van Stijn
def96d2bf4 vendor: github.com/docker/docker v24.0.2
no changes in vendored files

full diff: https://github.com/docker/docker/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:01 +02:00
Justin Chadwell
f5f00e68ef bake(cli): allow passing in-stream using command.Cli
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:53 +01:00
Justin Chadwell
14aebe713e debug-shell(cli): allow passing in-stream using command.Cli
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:35 +01:00
Justin Chadwell
9d2388e6f5 vendor: update buildkit to master@67a08623b95a
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:36:52 +01:00
Timofey Kirillov
75e2c46295 build(cli): allow passing in-stream using command.Cli
Use command.Cli::In() which is always initialized either to os.Stdin, or to user-specified stream:
* 5be21394cb/cli/command/cli.go (L494)
* https://github.com/docker/cli/blob/master/cli/command/cli_options.go#L16C1-L26

Signed-off-by: Timofey Kirillov <timofey.kirillov@flant.com>
2023-06-09 12:19:02 +03:00
Justin Chadwell
2c02db8db4 Merge pull request #1835 from ktock/long-form-json 2023-06-08 17:10:37 +01:00
Kohei Tokunaga
e304a05d2a docs: monitor: mention about long form flag
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:03 +01:00
Kohei Tokunaga
14c1ea0e11 invoke: Allow JSON array for long form flags
This commit allows specifying a JSON array to the long-form arg, entrypoint and
envvars.
Non-JSON-array value can still be specified. Buildx treats the value as a JSON
array only when it can be parsed as a JSON array.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:02 +01:00
Justin Chadwell
c30bcade2c Merge pull request #1870 from tianon/invoke-entrypoint-fixes 2023-06-08 16:03:24 +01:00
Tianon Gravi
62bfb19db4 Fix a couple --invoke entrypoint interaction bugs
When running `--invoke` against images that have `Cmd` set, the interactions with `Entrypoint` start to cause issues like the following:

    /usr/local/bin/bash: /usr/local/bin/bash: cannot execute binary file

Or:

    sh: can't open 'bash': No such file or directory

This patch fixes those by explicitly setting `Cmd` to be empty if it is unspecified and `Entrypoint` is being set, which matches `docker`'s behavior:

    $ docker image inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' bash
    ["docker-entrypoint.sh"] + ["bash"]
    $ docker create --name foo --entrypoint bash bash
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["bash"] + null
    $ docker rm foo
    $ docker create --name foo bash ls
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["docker-entrypoint.sh"] + ["ls"]

(There are still some weird edge cases in the interaction between the `InvokeConfig` and the original image config, but this fixes the most irritating for me and the rest are going to be deeper changes that are possibly less acceptable. 😅)

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-07 16:33:28 -07:00
Justin Chadwell
47e34f2684 Merge pull request #1813 from jedevc/oci-export-on-containerd 2023-06-07 15:19:54 +02:00
Justin Chadwell
3d981be4ad Merge pull request #1850 from ktock/monitor-longhelp 2023-06-07 15:09:35 +02:00
CrazyMax
5d94b0fcc7 Merge pull request #1866 from crazy-max/bump-compose
vendor: github.com/compose-spec/compose-go v1.14.0
2023-06-07 14:55:55 +02:00
CrazyMax
569c66fb62 Merge pull request #1867 from crazy-max/bump-testify
vendor: github.com/stretchr/testify v1.8.4
2023-06-07 12:19:33 +02:00
CrazyMax
93f7fbdd78 Merge pull request #1865 from crazy-max/bump-semver
vendor: github.com/Masterminds/semver/v3 v3.2.1
2023-06-07 12:19:17 +02:00
Tõnis Tiigi
ea06685c11 Merge pull request #1858 from potherca-contrib/fix/typo-Shortand
Fix typo "Shortand" -> "Shorthand"
2023-06-06 14:03:40 -07:00
Ben Peachey
eaba4fa9e6 Update docs with result from make docs command.
Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-06-06 21:43:41 +02:00
CrazyMax
99e3882e2a Merge pull request #1841 from crazy-max/desktop
build: display build details link
2023-06-06 19:06:19 +02:00
CrazyMax
0a2f35970c build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
CrazyMax
ab5f5e4169 build: fix quiet mode
We are using the quiet flag option and we are not taking
progress quiet mode into account

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:47:55 +02:00
Justin Chadwell
696770d29c Merge pull request #1840 from jedevc/fix-check-path-for-bake-cmd-context 2023-06-06 17:49:04 +02:00
CrazyMax
b47b4e5957 vendor: github.com/stretchr/testify v1.8.4
full diff https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:44:58 +02:00
CrazyMax
9a125afba0 vendor: github.com/compose-spec/compose-go v1.14.0
https: //github.com/compose-spec/compose-go/compare/v1.13.4...v1.14.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:34:34 +02:00
Justin Chadwell
d34103b0d9 bake: fix potential context entitlements escape
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
c820350b5e tests: refactor cmd helpers to allow configuring cwd, etc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
61a7854659 tests: add bake test for remote subdir context
Fixed in 12b6a3ad9a, but now we have
regression tests! So we can add a check that we don't break this
behavior again.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
e859ebc12e tests: add bake test for remote cmd context override
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
ef997fd6d0 bake: fix BAKE_CMD_CONTEXT relative path resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
76c96347ff tests: add basic remote bake context test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
48d7dafbd5 git: update gitutil test utilities
- Adds a new GitServeHTTP function to start an http server to serve a
  target git repository.
- Adds a new GitDir helper method to get the path to the .git
  directory
- Updates the GitAdd method to take a variable number of files

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
d03e93f6f1 test: tmpdir can be a test helper
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
fcb7810a38 Merge pull request #1854 from crazy-max/inspect-features 2023-06-06 16:59:58 +02:00
CrazyMax
459d94bdf1 vendor: github.com/Masterminds/semver/v3 v3.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 16:55:59 +02:00
Justin Chadwell
7cef021a8a Merge pull request #1804 from jedevc/fixup-solve 2023-06-06 09:12:10 +02:00
Justin Chadwell
c6db4cf342 build: clarify NewResultHandle comment
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 09:08:30 +02:00
CrazyMax
6c9436fbd5 inspect: display builder features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-05 23:11:58 +02:00
Justin Chadwell
a906149930 Merge pull request #1862 from jedevc/update-contributing-tests
docs: update CONTRIBUTING.md with specific test notes
2023-06-01 21:55:12 +01:00
Justin Chadwell
af328fe413 docs: update CONTRIBUTING.md with specific test notes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-01 11:34:20 +01:00
Justin Chadwell
183a73abae build: docker exporter should instead use oci with containerd
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:59:55 +01:00
Justin Chadwell
b7f0b3d763 build: clear exports for secondary solve request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:56 +01:00
Justin Chadwell
5b27d5a9f6 build: cleanup res if returned in basic build
In practice, this shouldn't happen, but the check is good to include
anyways.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:03 +01:00
Ben Peachey
8f24c58f4d Fix typo in commands/build.go and docs/reference/buildx_build.md
`Shortand` -> `Shorthand`

Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-05-31 13:40:58 +02:00
Justin Chadwell
cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell
8d822fb06c build: move main solve request into main gateway call
Now, we always perform the full solve request in the main gateway call.
This ensures that progress works properly, and makes the lifetime
semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to
always return a single ResultContext, even though the details of how a
gateway is created is different:
- For a failed build, we can just keep the gateway open.
- For a successful build, we immediately open another gateway and
  re-evaluate the build definition in that gateway. This should give an
  instant cache hit (since the build was just successful).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:46:46 +01:00
CrazyMax
0758a9b75d Merge pull request #1856 from crazy-max/history-feature-update
driver: update history api check
2023-05-31 10:27:54 +02:00
CrazyMax
f8fa526678 driver: update history api check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-31 10:10:54 +02:00
Tõnis Tiigi
4abff3ce12 Merge pull request #1852 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@c36941f4a10e
2023-05-30 16:14:26 -07:00
CrazyMax
e7034f66bc Merge pull request #1846 from crazy-max/history-capability
driver: check history capability
2023-05-30 20:25:31 +02:00
CrazyMax
8c65e4fc1d driver: add context to Features interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
d196ac347e driver: cache features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
9b723ece46 driver: check history capability
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:17 +02:00
Justin Chadwell
5e2f8bd64a Merge pull request #1853 from jedevc/logrus-show-error 2023-05-30 10:29:49 +01:00
Justin Chadwell
5788ab33d2 logutil: print error from WithError if found
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 10:00:24 +01:00
CrazyMax
f1788002e1 Merge pull request #1851 from crazy-max/test-flow
test: update workflow
2023-05-30 10:58:17 +02:00
Justin Chadwell
6c62225d1b vendor: update buildkit to master@c36941f4a10e
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 09:10:52 +01:00
CrazyMax
38b4eef5c6 ci: make release and bin-image depend on test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
a4db138c5e docs: update CONTRIBUTING.md to run tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
55377b2b0f make: test-unit and test-integration targets
Also switch test target to run the test script.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
98dedd3225 ci: cache integration tests base and build in dedicated step
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
74b121be66 test: opt to avoid building the integration tests image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
b9cf46785b test: use trap to remove cache volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
ecf8dd0a26 test: type integration not needed
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
73c17ef4d2 ci: send test reports to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
e762e46b4b ci: merge test workflow to build one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
Kohei Tokunaga
cafeedba79 monitor: add long help for commands
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-29 23:23:56 +09:00
CrazyMax
17bdbbd3c3 Merge pull request #1831 from crazy-max/fix-ctn-certs-path
driver(docker-container): fix incorrect path when writing certs
2023-05-26 16:15:21 +02:00
Justin Chadwell
2dae553d18 build: update outline fallback image
Subrequests have been included in docker/dockerfile:1.5 labs, so we can
update the fallback to point to this release.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin Chadwell
91c17f25fb build: tidy up print func
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
CrazyMax
63fc01e08a Merge pull request #1842 from crazy-max/update-xx
update xx to 1.2.1
2023-05-26 11:22:33 +02:00
Justin Chadwell
354ccc9469 Merge pull request #1844 from tonistiigi/imageid-fixes 2023-05-26 10:21:53 +01:00
Tonis Tiigi
68ae67720a build: fix writing correct image ID with -q
Container driver wrote manifest digest that had a
mismatch with --iidfile output.

When --iidfile was set the --metadata-file was not
written.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:20:29 -07:00
Tonis Tiigi
b273db20c3 remote: allow docker exporter on remote driver
There is no reason why one can't make a docker tarball or
load to Docker instance from remote driver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:18:57 -07:00
CrazyMax
0ae88ecc4d update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-25 15:26:56 +02:00
Justin Chadwell
341fb65f6f Merge pull request #1830 from jedevc/fix-missing-driver-in-build 2023-05-25 11:35:01 +01:00
Justin Chadwell
69a9c6609a Merge pull request #1699 from jedevc/bake-attestation-override 2023-05-25 09:54:34 +01:00
CrazyMax
1c96fdaf03 driver(docker-container): fix incorrect path when writing certs
Replace invalid characters in the name of the base directory of
certificates using the name of the registry which can contain
for example a colon when the port is specified.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-23 15:36:38 +02:00
Justin Chadwell
c77bd8a578 build: fix missing "docker" driver name in build progress
This was missing, since the driver property can only be fully populated
after loading nodes from disk. So we add logic to load the nodes, and
check for an error, which ensures that the "docker" driver is always
correctly present in the progress description.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:37:22 +01:00
Justin Chadwell
e5f701351c Merge pull request #1829 from jedevc/vendor-buildkit-master 2023-05-23 10:23:24 +01:00
Justin Chadwell
09798cdebd vendor: update buildkit to master@212ab16a39b1
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:16:09 +01:00
CrazyMax
0dfc35d558 Merge pull request #1822 from thaJeztah/vendor_engine_24.0.1
vendor: github.com/docker/docker and github.com/docker/cli v24.0.1
2023-05-23 11:11:34 +02:00
CrazyMax
8085f57a3a Merge pull request #1825 from thaJeztah/remove_k8s_replace
go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
2023-05-23 11:10:31 +02:00
Sebastiaan van Stijn
d582a21acd go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
Replace rules are not inherited by consumers of buildx as a module, and as
such would default to use the v0.26.2 version. Removing the replace rules
also removes various (indirect) dependencies (although brings in some new
packages from k8s itself).

The "azure" and "gcp" authentication packages in k8s.io/go-client are now
no longer functional, so removing those imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 16:10:49 +02:00
CrazyMax
580820a4de Merge pull request #1824 from thaJeztah/k8s_remove_openstack
cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
2023-05-22 16:09:30 +02:00
Sebastiaan van Stijn
b7e8afc61b cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
Kubernetes v1.26 removed the in-tree openstack cloud provider. While buildx
has a replace rule to pin the k8s dependencies to an older version, such
replace rules are non-transitive, which means that consumers of buildx as
a module could potentially end up with non-resolvable dependencies.

The used code was already non-functional since c41b006be1
(buildx v0.5.0-rc1);

a6b0959276/vendor/k8s.io/client-go/plugin/pkg/client/auth/openstack/openstack_stub.go (L26-L36)

See:

- 9bbf01bae9
- 98e81a7784

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 13:01:54 +02:00
Sebastiaan van Stijn
a8a637e19d vendor: github.com/docker/cli v24.0.1
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:16:37 +02:00
Sebastiaan van Stijn
79632a4c4c vendor: github.com/docker/docker v24.0.1
no changes in vendored files

full diff: https://github.com/moby/moby/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:08:30 +02:00
CrazyMax
a6b0959276 Merge pull request #1440 from crazy-max/gc-policies
builder: add worker gc policies and labels
2023-05-22 11:46:46 +02:00
CrazyMax
6d7142b057 Merge pull request #1820 from jedevc/use-docker-distribution
vendor: use docker/distribution instead of distribution/distribution
2023-05-22 11:17:01 +02:00
Justin Chadwell
d0bff18cee commands: consume DEBUG environment variable
When running in standalone mode, the --debug flag passed to docker
cannot be passed. The docker cli also supports a DEBUG env var, however,
in standalone mode this won't be consumed.

This patch reads the contents of the DEBUG environment variable, and
enables debugging logs when it's been set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:52:23 +01:00
Justin Chadwell
7e39644f69 Merge pull request #1816 from bpaquet/fix_session_token 2023-05-22 09:34:07 +01:00
Bertrand Paquet
adc6349b28 Fix AWS Authentication when mixing static creds and IAM profile
When the user supply static creds, we must not enrich them with a
session token which is unrelated.

Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
2023-05-22 09:26:28 +01:00
Justin Chadwell
f558fd8b22 vendor: use docker/distribution instead of distribution/distribution
This doesn't require pulling in a new direct dependency, we already use
docker/distribution throughout imagetools and build.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:24:41 +01:00
Justin Chadwell
432e16ef70 Merge pull request #1806 from thaJeztah/update_engine2 2023-05-22 09:19:35 +01:00
Sebastiaan van Stijn
8c86c2242a vendor: github.com/docker/cli v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:47:38 +02:00
Sebastiaan van Stijn
75ad5d732b vendor: github.com/docker/docker v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:23 +02:00
Sebastiaan van Stijn
9bd0202312 vendor: github.com/spf13/cobra v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:20 +02:00
Sebastiaan van Stijn
367f114cc7 vendor: github.com/opencontainers/runc v1.1.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:43:16 +02:00
Sebastiaan van Stijn
2959ce205e vendor: github.com/klauspost/compress v1.16.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:41:29 +02:00
Sebastiaan van Stijn
75b5c6560f vendor: github.com/moby/term v0.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:39:47 +02:00
Justin Chadwell
4429ccbcc2 Merge pull request #1817 from jedevc/docs-monitor-invoke-parts 2023-05-19 16:40:47 +01:00
Justin Chadwell
c59fc18325 Merge pull request #1770 from jedevc/integration-tests 2023-05-19 16:18:31 +01:00
Justin Chadwell
4ce80856b3 docs: add info for on-error and debug-shell values for invoke
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:51 +01:00
Justin Chadwell
af3feec4ea docs: fixup typo in buildkit console output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:30 +01:00
Justin Chadwell
90c849f5ef bake: merge attest entries
This ensures that `target.attest=["type=sbom,<value>"]` can be
appropriately merged when `--sbom=true` or `--set
target.attest=type=sbom`.

To merge, we simply naively take the last valid value.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:32:58 +01:00
Justin Chadwell
6024212ac8 ci: add integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
2d124e0ce9 test: add basic integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
e61a8cf637 vendor: update buildkit to master@31c870e82a48
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:25:31 +01:00
Justin Chadwell
167cd16acb Merge pull request #1797 from crazy-max/fix-image-create 2023-05-18 09:06:45 +01:00
Tõnis Tiigi
1dd31fefcb Merge pull request #1810 from nicks/patch-1
fix two bugs in bake-reference.md
2023-05-17 21:12:19 -07:00
Nick Santos
5a12b25bab Update bake-reference.md
Before this change, there were two bugs:
- the HCL was not valid. in hcl, argument names can't be quoted
- the target argument should be a real target

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-05-17 19:09:42 -04:00
CrazyMax
b702188b65 imagetools(create): set correct media type when combining manifests
When using imagetools create and combining multiple sources
we should check the media type of each manifest and set
the right media type for the manifest list.

If there is a mismatch we set OCI index as best effort.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-16 11:46:58 +02:00
Justin Chadwell
060ac842bb Merge pull request #1794 from jedevc/fix-printer-start-race 2023-05-12 09:43:58 +01:00
Justin Chadwell
31d1b778ff Merge pull request #1795 from thaJeztah/vendor_distribution_v2.8.2 2023-05-12 09:43:07 +01:00
Sebastiaan van Stijn
1cd4b54810 vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2-beta.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 19:08:16 +02:00
CrazyMax
c54926c5b2 Merge pull request #1767 from jedevc/issue-template-no-bug-feature
github: don't auto-add kind/ labels
2023-05-11 17:22:23 +02:00
Justin Chadwell
10aea8e970 Merge pull request #1791 from jedevc/docs/bake-reference-matrix
Document new bake matrix builds
2023-05-11 15:43:49 +01:00
Justin Chadwell
be6542911f progress: make printer ready only after pausing logrus
This fixes a possible race where messages printed directly after calls
to NewPrinter may appear before the printer starts. With this change, we
delay all of the logs until after.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 15:23:41 +01:00
CrazyMax
9b07f6510a Merge pull request #1793 from jedevc/move-path-resolution-to-controller
controller: move path resolution into controller package
2023-05-11 15:01:21 +02:00
Justin Chadwell
9ee19520dd controller: move path resolution into controller package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:56:05 +01:00
Justin Chadwell
878faae332 docs: add matrix docs for bake
Co-authored-by: David Karlsson <david.karlsson@docker.com
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:44:28 +01:00
Justin Chadwell
eaf38570e7 github: don't auto-add kind/bug labels
kind labels should only be manually added by maintainers after
confirming the issue. Otherwise, it can be tricky to sort through all
the issues when filtering for legitimate bugs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:43:25 +01:00
CrazyMax
167340df17 Merge pull request #1780 from techvelocity/elranshefer/add-support-for-proxy-url
kubernetes driver: add support for proxy-url
2023-05-11 14:40:23 +02:00
Justin Chadwell
e61a1da7fc Merge pull request #1748 from dvdksn/docs/bake-reference
docs: move and rewrite bake reference
2023-05-11 13:00:21 +01:00
David Karlsson
f8483d7243 docs: refactor bake file reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-05-11 12:54:08 +01:00
Justin Chadwell
2c8a9aad76 Merge pull request #1782 from jedevc/docs-monitor
docs: add initial guide about debugging monitor mode
2023-05-11 11:29:21 +01:00
Justin Chadwell
32009a701c Merge pull request #1786 from jedevc/debug-multi-platform
debug: evaluate all refs to allow multi-platform debugging
2023-05-11 09:54:13 +01:00
Justin Chadwell
0cbc316f76 docs: add initial guide about debugging monitor mode
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:53:21 +01:00
Justin Chadwell
45fccef3f3 debug: evaluate all refs to allow multi-platform debugging
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:44:50 +01:00
Justin Chadwell
fdcb4e2fb9 Merge pull request #1784 from jedevc/support-new-compose-features
Support new compose features in bake
2023-05-11 09:44:02 +01:00
Justin Chadwell
4a0a67d7a2 Merge pull request #1783 from jedevc/fix-bake-sub-dir-context
bake: copy remote context contents to root
2023-05-11 09:40:25 +01:00
CrazyMax
855d49ff58 Merge pull request #1788 from crazy-max/update-docker
vendor: github.com/docker/docker v23.0.6
2023-05-10 17:22:14 +02:00
CrazyMax
384e873db0 Merge pull request #1790 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-05-10 16:17:53 +02:00
CrazyMax
60e72ba989 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:43:24 +02:00
CrazyMax
45a2ae6762 vendor: github.com/docker/docker v23.0.6
full diff:
- https://github.com/docker/docker/compare/v23.0.1...v23.0.6
- https://github.com/docker/cli/compare/v23.0.1...v23.0.6

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:26:20 +02:00
Justin Chadwell
2eeef180ea Merge pull request #1746 from ktock/resultcleanup
controller: avoid "context canceled" error on cleanup
2023-05-10 13:24:12 +01:00
Kohei Tokunaga
8fd81f5cfd controller: avoid "context canceled" error on cleanup
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-10 12:56:33 +01:00
Justin Chadwell
1eb9ad979e Merge pull request #1785 from jedevc/default-to-local-controller
controller: default to using local controller
2023-05-10 11:18:55 +01:00
Justin Chadwell
77e0e860f8 controller: default to using local controller
--detach shouldn't be the default yet, since it still has the potential
to leak some sessions in odd edge cases, and is slightly more painful to
debug.

For now, we should set the local controller as the default, with the
idea that we can change it back in the future.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:51:58 +01:00
Justin Chadwell
e228c398f4 bake: support compose additional contexts
compose-go v1.13.0 supports the new additional_contexts to allow passing
additional build context during build, so we should map this to bake's
contexts property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:41:37 +01:00
Justin Chadwell
5d06406f26 bake: support compose inline dockerfile
compose-go v1.13.0 supports the new dockerfile_inline to allow including
a dockerfile inline in the compose definition, so we should map this to
bake's dockerfile-inline property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:39:00 +01:00
Justin Chadwell
cb061b684c Merge pull request #1753 from nicksieger/compose-go-v1.13.4
compose go v1.13.4
2023-05-10 09:25:10 +01:00
Elran Shefer
29b427ce13 kubernetes driver: add support for proxy-url
Signed-off-by: Elran Shefer <elran.shefer@velocity.tech>
2023-05-10 08:36:39 +03:00
Tõnis Tiigi
4fa7cd1fc2 Merge pull request #1772 from tonistiigi/container-hang-fix
container-driver: avoid hanging on container panic
2023-05-09 10:27:11 -07:00
Justin Chadwell
12b6a3ad9a bake: copy remote context contents to root
When resolving remote contexts locally in bake, then we need to ensure
that we properly unpack the contents of that context to the root
directory, instead of leaving it in the subdirectory.

Otherwise, any files will be found in the wrong location. Along with
this change, we also need a change to the dockerfile location lookup to
ensure that it is found at the root instead of in the subdirectory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:19:26 +01:00
Justin Chadwell
22e1901581 bake: reorganize input creation together
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:15:44 +01:00
Justin Chadwell
e23c37fa96 Merge pull request #1771 from jedevc/unset-other-build-client-opt
debug: secondary client build should not reuse solve opts
2023-05-09 14:57:23 +01:00
Justin Chadwell
e5a0ed1149 debug: secondary client build should not reuse solve opt ref
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 14:23:41 +01:00
Justin Chadwell
c9c1303e31 Merge pull request #1777 from ktock/fixerrormsg
monitor: improve error messages
2023-05-09 13:49:07 +01:00
Justin Chadwell
ae3299d9d4 Merge pull request #1628 from cpuguy83/policy_file
Support for passing through raw buildkit policies
2023-05-09 11:50:10 +01:00
CrazyMax
a948cc14c5 Merge pull request #1769 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.1
build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
2023-05-09 11:41:08 +02:00
CrazyMax
621b07c799 Merge pull request #1727 from crazy-max/cmd-completion
cli: add shell completion
2023-05-09 10:59:32 +02:00
Justin Chadwell
7ad970f93a Merge pull request #1776 from tonistiigi/fix-conn-leak
docker: fix connection leak on features check
2023-05-09 09:52:18 +01:00
Kohei Tokunaga
437fe55104 monitor: improve error messages
Print more understandable messages on error:

- When ps fails because the monitor doesn't attach to any session, print "no
  attaching session" instead of "unknown ref".
- Avoid disconnect silently fails when the monitor doesn't attach to any
  session. Print "no attaching session" error instead.
- Fix error message of "attach"'s arguments. ("server name must be passed" ->
  "ID of session or process must be passed")

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-08 15:42:37 +09:00
Tonis Tiigi
bebd244e33 docker: fix connection leak on features check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-07 21:58:19 -07:00
Tonis Tiigi
9f2143e3df container-driver: avoid hanging on container panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-03 17:11:08 -07:00
Brian Goff
98efe7af10 Support for passing through raw buildkit policies
This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-03 23:07:44 +00:00
Justin Chadwell
c7c37c3591 Merge pull request #1760 from ktock/monitor-commands
monitor: Move commands to a separated package
2023-05-03 14:33:08 +01:00
Justin Chadwell
a43837d26c monitor: extend monitor interface from controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-03 22:08:30 +09:00
dependabot[bot]
f115abb509 build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.0 to 5.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](5b4a9f6a9e...284f54f989)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-02 18:59:56 +00:00
Justin Chadwell
43a07f3997 Merge pull request #1597 from robertlestak/k8s-sa
kubernetes driver: add serviceaccount opt
2023-05-02 12:56:07 +01:00
CrazyMax
41e1693be0 Merge pull request #1756 from crazy-max/fix-bin-image
ci: fix bin-image job
2023-04-27 12:20:16 +02:00
Justin Chadwell
9d5af461b2 Merge pull request #1755 from jedevc/move-across-controller-boundary
Move metadata and subrequest output outside of controller
2023-04-27 11:15:54 +01:00
CrazyMax
b38c9c7db4 Merge pull request #1759 from crazy-max/bump-gomod
bump go.mod to 1.20
2023-04-27 09:00:31 +02:00
Kohei Tokunaga
9f884edbbf monitor: use shlex
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:38 +09:00
Kohei Tokunaga
0a7a2b1882 monitor: Move commands to a separated package
This commit moves monitor commands to `monior/commands` package.
Commands still need access to the `monitor` object and buildx controller so this
commit enables this via `Monitor` interface stored in `monitor/types`.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:35 +09:00
Justin Chadwell
6bec8f6e00 docs: add serviceaccount driver-opt for k8s driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:44:03 +01:00
robertlestak
65037e4611 kubernetes driver: add serviceaccount opt
Signed-off-by: robertlestak <robert.lestak@umusic.com>
2023-04-26 16:40:07 +01:00
Justin Chadwell
ba92989a94 controller: print result outside of controller
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2bf996d9ad controller: don't write metadata file in controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
75ed3e296b Merge pull request #1737 from jedevc/improved-controller-progress
controller: refactor progress api
2023-04-26 10:17:00 +01:00
CrazyMax
e14e0521cf Merge pull request #1732 from milas/builder-init-panic
builder: extra init error handling
2023-04-26 09:23:01 +02:00
Tõnis Tiigi
28e6995f7c Merge pull request #1735 from tonistiigi/localstate
build: support for saving local state by build ref
2023-04-25 15:17:08 -07:00
CrazyMax
8f72fb353c bump go.mod to 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 21:45:40 +02:00
Milas Bowman
14f5d490ef builder: extra init error handling
* Return errors from creating the `NodeGroup`
* Ensure that `b.NodeGroup != nil` before reading from
  it during validation

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 11:32:54 -04:00
CrazyMax
c9095e8eab ci: fix bin-image job
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 13:39:00 +02:00
CrazyMax
0589f69206 build: null values support with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
b724a173a9 build: fix JSON formatting for --print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
e5ccb64617 fix: changed attest type from Map to List
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
08d114195f chore: format with prettier
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
caf7d2ec9b build: added attest field
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
David Karlsson
2dffed3f3a build: sort bake fields alphabetically
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
Justin Chadwell
784dc2223d lint: add empty line between console and json code blocks
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 12:05:03 +02:00
CrazyMax
c3fd1e8b79 build: fix links and formatting
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
6f0c550ee9 build: move bake under the build section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
5d551dbbc1 build: customizing builds section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
043cb3a0db build: add bake section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:02 +02:00
Justin Chadwell
16d5b38f2b debug: display build warnings after each build
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 09:58:02 +01:00
Nick Sieger
956a1be656 vendor: update to compose-go 1.13.4
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:29:20 -05:00
Justin Chadwell
afcaa8df5f Merge pull request #1752 from nicksieger/compose-go-default-filenames
bake: use default filenames from compose-go
2023-04-21 17:28:32 +01:00
Nick Sieger
12885c01ad bake: use default filenames from compose-go
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:11:37 -05:00
Justin Chadwell
2ab8749052 controller: replace logrus status messages with progress messages
logrus info messages aren't particularly in-theme with the rest of the
progress output (and are also frustratingly racy). The progress output
is a lot neater, so we refactor it into that.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 12:33:04 +01:00
Justin Chadwell
e826141af4 controller: refactor progress api
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.

This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
  they can be displayed correctly from the client side.

Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.

To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:17:43 +01:00
Justin Chadwell
0c1fd31226 build: refactor out common build command components
We had some duplicated code between the basic runBuild and
launchControllerAndRunBuild.

This patch refactors out the common logic (since it's only really like
to keep growing), and has runBuild call into either the controller or
directly start the build depending on whether BUILDX_EXPERIMENTAL is
set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:13:58 +01:00
Justin Chadwell
0e9804901b controller: refactor status message conversion to pb package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:59:16 +01:00
Justin Chadwell
2402607846 build: use gateway's solve context to allow cancelling getResultAt
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:52:03 +01:00
CrazyMax
3d49bbd23a Merge pull request #1749 from docker/dependabot/github_actions/docker/bake-action-3
build(deps): Bump docker/bake-action from 2 to 3
2023-04-19 11:07:06 +02:00
dependabot[bot]
33b1fdbf39 build(deps): Bump docker/bake-action from 2 to 3
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 2 to 3.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-18 18:10:42 +00:00
Justin Chadwell
de4cdab411 Merge pull request #1747 from jedevc/only-capture-result-for-controller
controller: only capture the result when invoked through a controller
2023-04-18 18:28:08 +01:00
Justin Chadwell
a7e471b7b3 controller: only capture the result when invoked through a controller
This ensures that the code used to capture and evaluated a result is
only executed when built through the controller. Otherwise, no build
result should be recorded.

This ensures that new code added to capture and store the build result
for debugging isn't used when BUILDX_EXPERIMENTAL is not set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 16:35:42 +01:00
Justin Chadwell
ba6e5cddb0 Merge pull request #1745 from jedevc/remote-unneccessary-exec-server-name
monitor: update check for exec args
2023-04-18 11:29:01 +01:00
Justin Chadwell
e4ff82f864 monitor: update exec failure message when no args are passed
"exec" does not take a server name, so we can rename the error check
here.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 11:16:12 +01:00
Justin Chadwell
48b733d6da Merge pull request #1740 from jedevc/resolve-paths-context-dockerfile
build: avoid resolution of dockerfile if context is remote
2023-04-18 10:54:41 +01:00
Justin Chadwell
0b432cc5f2 Merge pull request #1640 from ktock/monitor-invoke-mode-restore
monitor: add `debug-shell` and `on-error`
2023-04-18 10:41:04 +01:00
Justin Chadwell
f6cccefffc build: avoid resolution of dockerfile if context is remote
In 566f41b598, we added a check to ensure
that we avoid resolving http URLs for Dockerfile. However, we have
another circumstance we should not resolve the path in - if the context
is a remote context, the dockerfile is resolved in that context (see
build.go#LoadInputs for more information).

Therefore, we should only resolve the dockerfile to a local directory if
the context is also resolved to a local directory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-17 11:18:49 +01:00
Kohei Tokunaga
fd5d90c699 remove unused fields from ResultContext
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
06399630a2 remove ResultContextError
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
20693aa808 Fix monitor to be aware of Inspect API and invokations from errors
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
f373b91cc3 Add flags and subcommand
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
ce48b1ae84 Enable to restore build options from the server
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:05 +09:00
Kohei Tokunaga
b3340cc7ba Enable to create container from error
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:36:33 +09:00
Kohei Tokunaga
1303715aba Allow passing ResultContext from server to the client through grpcerror
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:11:32 +09:00
Justin Chadwell
b716e48926 Merge pull request #1734 from jedevc/more-ssh-context-fixes
Various context fixups
2023-04-14 09:27:36 +01:00
Justin Chadwell
7d35a3b8d8 Merge pull request #1733 from jedevc/use-dockerui-context-detection
Use dockerui context detection
2023-04-14 09:25:07 +01:00
CrazyMax
200058b505 build: support for saving local state by build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-12 21:37:03 -07:00
Justin Chadwell
566f41b598 build: avoid file resolution of dockerfile urls
Dockerfiles can be HTTP URLs as well as local paths 🤦

We just copy the same logic we use for resolving context paths, and
apply it here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
6c0547e7e6 bake: ensure remote files behind ssh expose agent
The updateContext function may make modifications to the build inputs,
creating either an SSH URL, or an SSH llb.State. In these cases, we need
to ensure that we appropriately expose the client's default agent.

Previously, we would only expose it if the remote context was a git URL,
however, we need to also ensure that if the input was used to override
the context (in the case of ReadRemoteFiles), that we expose the agent
here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
871f865ac8 bake: update ReadRemoteFiles to use buildkit api
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 11:15:25 +01:00
Justin Chadwell
62a21520ea vendor: update buildkit to master@333ee9158128
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 10:11:46 +01:00
CrazyMax
a597266a52 cli: add completion to list builder names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:45:59 +02:00
CrazyMax
14b66817fb build: add completion to list context folders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:35:50 +02:00
CrazyMax
af011d6ca3 bake: add completion to list targets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:34:40 +02:00
CrazyMax
8a02cf8717 cli: set default completion
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:32:48 +02:00
CrazyMax
672eeed9a6 Merge pull request #1731 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-04-11 11:26:32 +02:00
CrazyMax
1b816ff838 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:10:01 +02:00
CrazyMax
10365ddf22 Merge pull request #1723 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.0
build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
2023-04-11 09:39:17 +02:00
dependabot[bot]
a28cb1491d build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.4 to 5.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](38e0b6e68b...5b4a9f6a9e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-05 18:59:48 +00:00
Justin Chadwell
1e149bb84f Merge pull request #1722 from jedevc/bake-matrix-check-map-type
bake: fix map type checking to detect matrix property set to list
2023-04-05 16:03:50 +01:00
Justin Chadwell
9827abbf76 bake: fix map type checking to detect matrix property set to list
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 13:41:03 +01:00
Justin Chadwell
a3293cdaaa Merge pull request #1720 from jedevc/bake-replace-func
Add `replace` function to bake stdlib
2023-04-05 11:59:40 +01:00
Justin Chadwell
f7d8bd2055 Merge pull request #1690 from jedevc/bake-matrix
Implement matrix for bake targets
2023-04-05 11:58:59 +01:00
Justin Chadwell
5d33a3af22 bake: add replace func to stdlib
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
Justin Chadwell
87f900ce77 chore: sort bake stdlib map
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
CrazyMax
bb5c93cafc Merge pull request #1719 from dvdksn/docs/update-readme-links
docs: prune links in README toc, add external reference
2023-04-05 11:30:29 +02:00
David Karlsson
c6ce0964b9 docs: prune links in README toc, add external reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-04-05 11:00:06 +02:00
Justin Chadwell
5c21e80a83 Merge pull request #1711 from jedevc/bake-remote-file-git
bake: forward default ssh agent for remote bake file
2023-04-04 11:50:42 +01:00
CrazyMax
498cc9ba0a Merge pull request #1715 from crazy-max/ci-restrict-repo
ci: check repo origin on push tag event
2023-04-03 21:16:51 +02:00
CrazyMax
805f3a199d ci: check repo origin on push tag event
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-03 20:58:37 +02:00
Justin Chadwell
91fdb0423d Merge pull request #1713 from ktock/monitor-buildapi-options
controller: Extract nested CommonOptions on controller API
2023-04-03 14:48:06 +01:00
Kohei Tokunaga
8ba8659496 controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Justin Chadwell
16e41ba297 Merge pull request #1710 from jedevc/use-buildkit-gitutil-parsegitref
Use buildkit's gitutil package to detect remote git files
2023-04-03 11:14:08 +01:00
Justin Chadwell
387ce5be7c bake: forward default ssh agent for remote bake file
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:50:43 +01:00
Justin Chadwell
87a120e8e3 bake: use build.IsRemoteURL
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:48:31 +01:00
Justin Chadwell
589d4e4cf5 build: use buildkit's gitutil package to detect remote files
BuildKit's gitutil package behaves slightly differently than moby's
urlutil, so we should rely on BuildKit's gitutil when detecting URLs to
avoid cases of accidentally producing invalid build requests that can
confuse users.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:23:59 +01:00
CrazyMax
6535f16aec Merge pull request #1705 from docker/dependabot/go_modules/github.com/opencontainers/runc-1.1.5
build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
2023-04-03 10:53:26 +02:00
Justin Chadwell
a1520ea1b2 bake: additional validation for matrixes
This adds the following constraints to the new features:
- Explicit renaming with the `name` property is *only* permitted when
  used with the `matrix` property.
- Group does not support either `name` or `matrix` (we may choose to
  relax this constraint over time).
- All generated names must be unique.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 09:23:28 +01:00
Tõnis Tiigi
0844213897 Merge pull request #1707 from crazy-max/fix-ls-docs
docs: update ls command output sample
2023-03-30 19:33:55 -07:00
CrazyMax
989ba55d9a docs: update ls command output sample
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 17:54:23 +02:00
CrazyMax
33388d6ede Merge pull request #1552 from crazy-max/moby-buildkit-version
driver: resolve BuildKit version for docker driver
2023-03-30 17:31:47 +02:00
dependabot[bot]
bfadbecb96 build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 15:25:26 +00:00
CrazyMax
f815f4acf7 Merge pull request #1703 from jedevc/vendor-buildkit-master
Bump BuildKit to latest master
2023-03-30 17:23:38 +02:00
CrazyMax
81d7decd13 driver: resolve BuildKit version for docker driver
Buildx currently returns the Docker Engine version for the docker driver
and it can be confusing. This is because BuildKit before 0.11 version does
not support the Info API that returns the BuildKit version.

This change resolves the BuildKit version from the Docker engine one.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 15:52:29 +02:00
Justin Chadwell
d699d08399 bake: add additional tests for matrix behavior
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:39:38 +01:00
Justin Chadwell
9541457c54 vendor: update buildkit to master@8b7bcb900d3c
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:35:49 +01:00
Justin Chadwell
c6cdcb02cf build: replace removed buildkit BUILDINFO field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:24:46 +01:00
Justin Chadwell
799715ea24 Merge pull request #1701 from jedevc/fixup-resolve-paths-context-path
build: fixup resolvePaths for remote context path
2023-03-29 13:59:36 +01:00
Justin Chadwell
b5c6b3f10b build: fixup resolvePaths for remote context path
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-29 10:10:20 +01:00
Tõnis Tiigi
3f59b27cf4 Merge pull request #1698 from crazy-max/fix-issue-template
chore: fix issue template
2023-03-27 09:10:56 -07:00
CrazyMax
00b18558dd chore: fix issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-27 13:30:33 +02:00
Tõnis Tiigi
948414e1b2 Merge pull request #1693 from jedevc/remote-tls-missing-key-cert
remote: avoid tls error if both key and cert are not set
2023-03-23 10:58:12 -07:00
Justin Chadwell
56876ab825 remote: avoid tls error if both key and cert are not set
Previously, we would explicitly error if all TLS parameters were not
available. However, it is a perfectly valid use case to connect to a
buildkit server that only provides TLS in one direction to verify the
server (which is possible today with buildctl).

To support this use case, we only need to error if only one of key or
cert is set, and the other is not - if both are unspecified, the client
will not present a certificate to the server.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-23 10:43:10 +00:00
Justin Chadwell
0806870261 bake: generate implicit groups for matrixes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-22 17:07:04 +00:00
Tõnis Tiigi
fd8eaab2df Merge pull request #1675 from jedevc/source-date-epoch-fixes
SOURCE_DATE_EPOCH build arg injection fixes
2023-03-21 22:29:24 -07:00
Justin Chadwell
77252f161c bake: add matrix to target block
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-21 12:05:14 +00:00
Justin Chadwell
4437802e63 bake: allow overriding name property
Previously, the name property could not be set in the body of a bake
target and could only be set for a label. This patch allows the body to
override the values of label fields, though the default is still the
label.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 17:55:28 +00:00
Justin Chadwell
1613fde55c bake: allow interception to create derived contexts
This patch allows high level clients to define an EvalContext method
which can derive a new context given a block and the base parent
context.

This allows users of the package to intercept evaluation before it
begins, and define additional variables and functions that are bound to
a single block.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 12:18:36 +00:00
CrazyMax
624bc064d8 Merge pull request #1677 from crazy-max/bump-k8s
vendor: bump k8s to v0.25.4
2023-03-19 17:05:55 +00:00
CrazyMax
0c4a68555e Merge pull request #1682 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.4
build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
2023-03-16 09:50:44 +00:00
dependabot[bot]
476ac18d2c build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](2b011faafd...38e0b6e68b)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-15 19:00:53 +00:00
CrazyMax
8ad30d0a35 docs: update since buildinfo removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-15 17:00:54 +01:00
Justin Chadwell
780531425b bake: move SOURCE_DATE_EPOCH parsing to overrides
Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-15 10:06:53 +00:00
CrazyMax
92d2dc8263 k8s: corev1.Handler has been replaced by corev1.ProbeHandler
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
CrazyMax
cfa6b4f7c8 vendor: bump k8s to v0.25.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
Justin Chadwell
5d4223e4f8 build: move SOURCE_DATE_EPOCH parsing into option generation
This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-14 10:21:32 +00:00
Tõnis Tiigi
4a73abfd64 Merge pull request #1673 from crazy-max/fix-k8s-deploy-name
k8s: generate node name if not provided
2023-03-13 17:20:51 -07:00
Justin Chadwell
6f722da04d Merge pull request #1670 from jedevc/version-v-prefix
version: add "v" prefix to version for tagging convention consistency
2023-03-13 08:00:46 +00:00
CrazyMax
527d57540e k8s: generate node name if not provided
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-11 21:22:10 +01:00
Justin Chadwell
b65f49622e version: add "v" prefix to version for tagging convention consistency
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-08 16:25:00 +00:00
Tõnis Tiigi
c5ce08bf3c Merge pull request #1626 from ktock/monitor-exec
monitor: Enable to exec into the container
2023-03-07 17:46:22 +00:00
Tõnis Tiigi
71b35ae42e Merge pull request #1663 from crazy-max/stripcreds
build: strip credentials from remote url on collecting Git provenance info
2023-03-06 16:26:09 +00:00
CrazyMax
15eb6418e8 build: strip credentials from remote url on collecting Git provenance info
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 17:00:05 +01:00
Justin Chadwell
2a83723d57 Merge pull request #1581 from ktock/resolvepath
monitor: resolve paths arguments in client
2023-03-02 11:09:34 +00:00
Kohei Tokunaga
e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:30:29 +09:00
Justin Chadwell
b5ea989eee monitor: resolve oci layout path in client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-02 10:28:56 +00:00
Kohei Tokunaga
17105bfc50 monitor: resolve paths arguments in client
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:13:45 +09:00
CrazyMax
eefe27ff42 Merge pull request #1658 from crazy-max/remove-guides
docs: guides moved to docs website
2023-03-01 10:40:12 +01:00
CrazyMax
1ea71e358a docs: guides moved to docs website
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-01 10:21:02 +01:00
Tõnis Tiigi
14d8f95ec9 Merge pull request #1657 from ktock/argpath
controller: use os.Executable() for getting the exceutable path
2023-03-01 00:12:36 -08:00
Kohei Tokunaga
b0728c96d3 controller: use os.Executable() for getting the exceutable path
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-01 15:49:07 +09:00
Tõnis Tiigi
5e685c0e04 Merge pull request #1650 from crazy-max/dirty-optin
build: makes git dirty check opt-in
2023-02-28 10:49:17 -08:00
CrazyMax
f2ac30f431 Merge pull request #1651 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-02-24 15:43:32 +01:00
CrazyMax
6808c0e585 ci: update docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:13:10 +01:00
CrazyMax
9de12bb9c8 docs: do not generate ref for internal serve command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:12:15 +01:00
CrazyMax
0645acfd79 build: makes git dirty check opt-in
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 13:19:33 +01:00
CrazyMax
439d58ddbd Merge pull request #1037 from crazy-max/bug-template
chore: github issue template
2023-02-24 13:06:55 +01:00
CrazyMax
c0a9274d64 chore: github issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 12:33:24 +01:00
Justin Chadwell
f3a4cd5176 Merge pull request #1614 from jedevc/typed-controller-api
controller: strongly type the controller api
2023-02-24 09:44:45 +00:00
Justin Chadwell
c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:43:15 +00:00
Justin Chadwell
c4ad930e2a Merge pull request #1644 from crazy-max/update-frontend
Dockerfile: align frontend version
2023-02-23 11:00:13 +00:00
CrazyMax
3d0c88695e Merge pull request #1646 from docker/dependabot/go_modules/golang.org/x/net-0.7.0
build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
2023-02-23 11:36:36 +01:00
dependabot[bot]
7332140fdf build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-22 19:50:31 +00:00
CrazyMax
132fababb0 Dockerfile: align frontend version
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 17:47:48 +01:00
CrazyMax
71507c0b58 Merge pull request #1643 from crazy-max/go-1.20
update to go 1.20
2023-02-22 17:42:59 +01:00
CrazyMax
7888fdee58 fix lint issue
should drop = nil from declaration of var n; it is the zero value

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:57:29 +01:00
CrazyMax
fb61fde581 fix cannot set environment variables in parallel tests
> FAIL: TestReadTargets/ArgsOverrides/leaf (0.00s)
> panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:56:46 +01:00
CrazyMax
5258e44030 remove deprecated golangci-lint linters
Linters 'deadcode' and 'varcheck' are deprecated since v1.49.0
and had been replaced by 'unused'

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:40:14 +01:00
CrazyMax
e16c1b289b update golangci-lint to v1.51.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:38:17 +01:00
CrazyMax
376b73f078 update to go 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:37:50 +01:00
Justin Chadwell
1c6060f27d Merge pull request #1612 from cpuguy83/env_no_provenance
Add env var to disable default attestations
2023-02-22 11:36:35 +00:00
Justin Chadwell
ed4fd965ff Merge pull request #1620 from jedevc/remote-controller-fixes
Improvements for remote controller code
2023-02-21 10:52:58 +00:00
Brian Goff
bc9cb2c66a Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-02-16 17:25:40 +00:00
CrazyMax
aa05f4c207 Merge pull request #1584 from dvdksn/docs/build-platform-local
docs: added --platform=local example
2023-02-16 18:08:25 +01:00
CrazyMax
62fbef22d0 Merge pull request #1177 from crazy-max/build-display-builder
build: print instance being used
2023-02-16 18:07:30 +01:00
David Karlsson
2563685d27 docs: added --platform=local example
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-02-16 17:24:45 +01:00
CrazyMax
598f1f0a62 build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
8311b0963a vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
b1949b7388 Merge pull request #1634 from dgageot/fix-1633
build: fix git ambiguous argument
2023-02-16 14:05:32 +01:00
CrazyMax
3341bd1740 Merge pull request #1610 from thaJeztah/update_docker
vendor: github.com/docker/docker, github.com/docker/cli v23.0.0
2023-02-16 13:43:12 +01:00
Justin Chadwell
74f64f88a7 Merge pull request #1617 from jedevc/process-multiple-attestation-annotations
imagetools: process `com.docker.reference.*` annotations
2023-02-16 12:05:20 +00:00
David Gageot
d4a4aaf509 Remove git warning: buildx/1633
Signed-off-by: David Gageot <david.gageot@docker.com>
2023-02-16 07:36:24 +01:00
Justin Chadwell
1f73f4fd5d Merge pull request #1630 from jedevc/debug-help-message-typo
monitor: fix typo in help message
2023-02-16 00:06:06 +00:00
Justin Chadwell
77f83d4171 monitor: fix typo in help message
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-15 14:48:56 +00:00
Justin Chadwell
642f28f439 imagetools: process com.docker.reference.* annotations
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.

Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:24:40 +00:00
Justin Chadwell
54f4dc8f6e controller: set absolute path of server binary before execution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:21:18 +00:00
CrazyMax
89d99b1694 inspect: display labels and gc policies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:56 +01:00
CrazyMax
9753f63f57 builder: add worker gc policies and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:15 +01:00
CrazyMax
04804ff355 Merge pull request #1557 from crazy-max/gomod-1.19
go.mod: bump to Go 1.19
2023-02-11 22:47:51 +01:00
Justin Chadwell
ed9ea2476d controller: use unique files per buildx version
This ensures that we should never accidentally connect to a server with
a mismatched version, while also allowing us to run multiple buildx
servers at a time.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:23 +00:00
Justin Chadwell
d0d29168a5 controller: use grpc with contexts for improved timeouts
This patch improves timeout logic for testing and creating a buildx
server. Instead of needing to have a custom loop, we can just use the
primitives provided by go and grpc for easier handling.

Additionally, without the explicit time.Sleeps, we defer to GRPCs
exponential backoff algorithm which should provide substantially better
results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:19 +00:00
Justin Chadwell
abda257763 controller: exit cleanly on SIGTERM
This signal may be sent using an external tool such as pkill, and since
we can handle it neatly, we should.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:15 +00:00
Justin Chadwell
1b91bc2e02 controller: add more informative server exit messages
When exiting, we should ideally always print a message, and give details
as to exactly what error we received.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:26 +00:00
Justin Chadwell
56b9e785e5 build: don't kill remote controller after build
We don't know if other builds might be running, etc, so we should allow
the server to decide when to exit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:25 +00:00
Sebastiaan van Stijn
081447c9b1 vendor: github.com/docker/cli v23.0.0
full diff: https://github.com/docker/cli/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:32:57 +01:00
Sebastiaan van Stijn
260117289b vendor: github.com/docker/docker v23.0.0
full diff: https://github.com/docker/docker/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:30:17 +01:00
Sebastiaan van Stijn
73dca749ca vendor: golang.org/x/oauth2 v0.1.0
no changes in vendored code

full diff: https://github.com/golang/oauth2/compare/6fdb5e3db783...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:26:17 +01:00
Sebastiaan van Stijn
8ac380bfb3 vendor: golang.org/x/net v0.5.0
full diff: https://github.com/golang/net/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:21:05 +01:00
Sebastiaan van Stijn
aeac7e08f9 vendor: golang.org/x/term v0.4.0
no changes in vendored code

full diff: https://github.com/golang/term/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:20:00 +01:00
Sebastiaan van Stijn
7c9cdc4353 vendor: golang.org/x/text v0.6.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:19:53 +01:00
Sebastiaan van Stijn
67572785cf vendor: golang.org/x/sys v0.4.0
full diff: https://github.com/golang/sys/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:12:32 +01:00
Sebastiaan van Stijn
8a70e7634d vendor: github.com/google/certificate-transparency-go v1.1.4
full diff:

- https://github.com/google/certificate-transparency-go/compare/v1.0.21...v1.1.4
- https://github.com/kubernetes/klog/compare/v2.30.0...v2.80.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:03:26 +01:00
Sebastiaan van Stijn
6dd5589a9c vendor: github.com/containerd/containerd v1.6.16
full diff: https://github.com/containerd/containerd/compare/1709cfe273d9...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 14:55:44 +01:00
CrazyMax
78058ce5f3 Merge pull request #1592 from crazy-max/fix-build-git
build: check reachable git commits
2023-02-08 14:23:52 +01:00
CrazyMax
fd5884189c build: check reachable git commits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-08 12:46:04 +01:00
Justin Chadwell
ab7a9f008d Merge pull request #1607 from jedevc/bake-fix-error-traces
bake: avoid nesting error diagnostics
2023-02-07 13:20:07 +00:00
CrazyMax
a8eb2a7fbe gitutil: override the locale to ensure consistent output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-07 13:26:28 +01:00
Justin Chadwell
fbb4f4dec8 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-07 11:40:20 +00:00
Justin Chadwell
46fd0a61ba Merge pull request #1605 from jedevc/bake-fix-errors
bake: avoid early-exit for resolution failures
2023-02-07 10:36:05 +00:00
Justin Chadwell
6444c813dc Merge pull request #1601 from jedevc/buildx-document-attestation-booleans
build: add docs for boolean attestation flags
2023-02-06 15:15:01 +00:00
Justin Chadwell
dc8a2b0398 bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 15:05:45 +00:00
CrazyMax
d9780e27cd Merge pull request #1602 from crazy-max/rm-releases-json
ci: remove releases-json workflow
2023-02-06 13:44:57 +01:00
CrazyMax
ab44d03771 ci: remove releases-json workflow
This is now directly handled by our actions-toolkit. We keep
`.github/releases.json` file for now but will be removed on
next v0.11 release.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
crazy-max
b53cb256e5 github: update .github/releases.json
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
CrazyMax
c3075923f4 Merge pull request #1591 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.9.0
2023-02-06 13:28:24 +01:00
CrazyMax
a32881313b Merge pull request #1596 from crazy-max/builder-fix-validation
builder: fix docker context not validated
2023-02-06 12:31:04 +01:00
Justin Chadwell
07548bc898 build: add docs for boolean attestation flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 10:53:49 +00:00
CrazyMax
0e544fe835 builder: fix docker context not validated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 21:23:29 +01:00
CrazyMax
21ac4c34fb update github.com/compose-spec/compose-go to v1.9.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 03:19:59 +01:00
Justin Chadwell
d2fa4a5724 Merge pull request #1582 from jedevc/lint-fixups
Linting tidy-ups
2023-02-01 16:08:07 +00:00
Justin Chadwell
4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:31:01 +00:00
Justin Chadwell
5da09f0c23 lint: enable gosec
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:50 +00:00
Justin Chadwell
48357ee0c6 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:43 +00:00
Justin Chadwell
6506166f02 Merge pull request #1579 from jedevc/refactor-controllers
Refactor controllers, moving them into a separate package
2023-02-01 09:57:32 +00:00
Justin Chadwell
5f130b25ad hack: move syntax directive to top of generated-files
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
cb94298a02 Merge pull request #1296 from ktock/monitor-list
monitor: Enable to run build and invoke in background
2023-01-31 14:14:57 +00:00
CrazyMax
046084c0b8 Merge pull request #1574 from crazy-max/ci-fix-workflow
ci: fix pr branch name in releases-json workflow
2023-01-31 15:00:01 +01:00
CrazyMax
18760253b9 ci: fix pr branch name in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 13:51:30 +01:00
CrazyMax
ded6376ece Merge pull request #1573 from crazy-max/fix-releases-json
ci: add workflow_dispatch in releases-json workflow
2023-01-31 13:48:33 +01:00
CrazyMax
a4d60a451d ci: add workflow_dispatch in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 11:38:46 +01:00
Justin Chadwell
0f4030de5d Merge pull request #1572 from crazy-max/ci-update-ver
ci: update buildx and buildkit to latest
2023-01-31 10:22:58 +00:00
CrazyMax
f1a5a3ec50 ci: update buildx and buildkit to latest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 22:13:27 +01:00
CrazyMax
87beaefbb8 Merge pull request #1571 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-01-30 21:18:54 +01:00
CrazyMax
451847183d ci: fix docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 21:01:27 +01:00
CrazyMax
7625a3a4b0 Merge pull request #1568 from crazy-max/silent-git-no-upstream
build: silently fail if git remote not found
2023-01-30 20:32:12 +01:00
CrazyMax
6db696748b build: better message output for git provenance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 20:24:09 +01:00
Tõnis Tiigi
14f9ae679d Merge pull request #1566 from tonistiigi/fix-result-descriptor
build: fix multi-node merge to read descriptor from result
2023-01-30 09:04:40 -08:00
CrazyMax
4789d2219c build: silently fail if git remote not found
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 14:33:16 +01:00
CrazyMax
eacecf657c Merge pull request #1561 from crazy-max/fix-preferred-platform
build: fix preferred platform not taken into account
2023-01-30 12:22:08 +01:00
CrazyMax
1de0be240f Merge pull request #1563 from crazy-max/releases-json
ci: generate releases.json on release event
2023-01-30 11:51:08 +01:00
CrazyMax
ea4bec2bad github: update .github/releases.json
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
CrazyMax
36d95bd3b9 ci: generate releases.json on release event
We are currently using the GitHub API in our setup-buildx-action
to check for latest and tagged releases to make sure they exist
before download. But this requires using a token to avoid
rate-limit. It's fine for public runners but GHES runners don't
have the `github.token` populated automatically. They need to
create a PAT.

This PR will solve this issue by generating and pushing a
`releases.json` file in this repo when we publish a GitHub Release
that will then be fetched through `raw.githubusercontent.com`
endpoint on `setup-buildx-action` repo. This endpoint is better
served for our purpose with 5000 requests per hour compared to the
GitHub API endpoint that is limited to 60 requests per hour (unauth)
and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an
action in a workflow run can't trigger a new workflow run anyway.
See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
Tonis Tiigi
c33b310b48 build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-29 22:52:41 -08:00
Justin Chadwell
8af76c68a8 Merge pull request #1564 from AkihiroSuda/fix-1562
bake: SOURCE_DATE_EPOCH: fix `panic: assignment to entry in nil map`
2023-01-29 22:07:20 +00:00
Akihiro Suda
1f56f51740 bake: SOURCE_DATE_EPOCH: fix panic: assignment to entry in nil map
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-29 08:43:06 +09:00
CrazyMax
49b3c0dba5 build: fix preferred platform not taken account
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-27 20:02:13 +01:00
Tõnis Tiigi
a718d07f64 Merge pull request #1559 from tonistiigi/update-buildkit-v0.11.2
vendor: update buildkit to v0.11.2
2023-01-26 10:57:21 -08:00
Tonis Tiigi
f6da7ee135 vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-26 10:33:20 -08:00
CrazyMax
7eb266de69 go.mod: bump to Go 1.19
Go 1.20 will be there soon, I think it's time to move our go.mod to
latest stable. We can then remove the compat in our vendor.Dockerfile

Downstream projects like compose or github.com/linuxkit/linuxkit should
not be affected.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-26 16:08:09 +01:00
CrazyMax
9f821dabeb Merge pull request #1548 from crazy-max/git-ls-remote
build: set remote origin url
2023-01-26 11:03:18 +01:00
Kohei Tokunaga
a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-26 17:17:07 +09:00
Tõnis Tiigi
b1b4e64c97 Merge pull request #1504 from jedevc/fix-1497
Refactor BuildWithResultHandler to simplify concurrency usage
2023-01-25 10:52:17 -08:00
CrazyMax
c1058c17aa build: set remote origin url
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-25 18:15:05 +01:00
Tõnis Tiigi
059c347fc2 Merge pull request #1549 from jedevc/silence-intoto-warnings
imagetools: silence intoto warnings
2023-01-24 17:03:49 -08:00
Justin Chadwell
7145e021f9 imagetools: silence intoto warnings
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 16:49:28 +00:00
CrazyMax
9723f4f76c Merge pull request #1547 from crazy-max/fix-rm
rm: do not check for context builders when removing inactive
2023-01-24 13:39:36 +01:00
Justin Chadwell
db72d0cc05 Merge pull request #1543 from jedevc/bump-ci-buildkit
ci: update buildkit release version in build pipeline
2023-01-24 12:17:34 +00:00
Justin Chadwell
00b7d5b858 Merge pull request #1505 from jedevc/inspect-lazy-attestations
Lazily load attestation data in `imagetools inspect`
2023-01-24 12:10:11 +00:00
CrazyMax
6cd0c11ab1 rm: do not check for context builders when removing inactive
This change has been introduced in e7b5ee7518
but we should not check context builders when removing inactive
ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-24 13:08:18 +01:00
Justin Chadwell
c1ab55a3f2 ci: update buildkit release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 11:47:58 +00:00
Justin Chadwell
c756e3ba96 Merge pull request #1534 from jedevc/readme-minimum-docker-version
docs: add note about minimum docker version
2023-01-24 11:02:21 +00:00
Justin Chadwell
566f37b65b docs: add note about minimum docker version
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 10:25:58 +00:00
CrazyMax
6d1ff27410 Merge pull request #1538 from tonistiigi/caps-req-internal
build: mark capabilities request as internal
2023-01-24 11:12:37 +01:00
Tonis Tiigi
be55b41427 build: mark capabilities request as internal
So it doesn't show up in the History API.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-24 00:48:35 -08:00
Tõnis Tiigi
a4f01b41a4 Merge pull request #1529 from thaJeztah/update_buildkit
vendor: github.com/moby/buildkit v0.11.1
2023-01-18 12:56:26 -08:00
Sebastiaan van Stijn
01e1c28dd9 vendor: github.com/moby/buildkit v0.11.1
full diff: https://github.com/moby/buildkit/compare/v0.11.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 20:56:54 +01:00
Tõnis Tiigi
51e41b16db Merge pull request #1511 from jedevc/error-on-attestations-docker
build: error when using docker exporter and attestations
2023-01-17 09:57:48 -08:00
Justin Chadwell
9e9cdc2e6d Merge pull request #1525 from jedevc/ci-bump-buildx
ci: update buildx release version in build pipeline
2023-01-17 16:39:39 +00:00
Justin Chadwell
bc1d590ca7 ci: update buildx release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 15:21:52 +00:00
Justin Chadwell
900d9c294d Merge pull request #1523 from jedevc/docs-reference-attest
docs: add reference for new attest family of flags
2023-01-17 13:47:10 +00:00
CrazyMax
65aac16139 Merge pull request #1512 from nicks/patch-1
docs: small tweak to opentelemetry
2023-01-17 13:30:33 +01:00
Justin Chadwell
4903f462f6 docs: add reference for new attest family of flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 11:45:38 +00:00
Justin Chadwell
44b5a19c13 Merge pull request #1516 from dvdksn/docs/fix-broken-bake-link-in-cli-reference
docs: fix broken link in buildx_bake CLI reference
2023-01-17 11:11:54 +00:00
David Karlsson
ba8fa6c403 docs: fix broken link in buildx_bake CLI reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-01-13 15:33:31 +01:00
CrazyMax
5b3083e9e1 Merge pull request #1515 from crazy-max/fix-docs-release
ci: fix typo in docs-release workflow
2023-01-13 15:16:16 +01:00
CrazyMax
523a16aa35 ci: fix typo in docs-release workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-13 15:10:03 +01:00
Justin Chadwell
43a748fd15 build: error when using docker exporter and attestations
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:41:53 +00:00
Justin Chadwell
15a80b56b5 build: avoid compatability error when attestations disabled
We should avoid erroring with attestations support compatability errors
when a user has specified --provenance=false.

A user may wish to enable --provenance=false that works across buildkit
versions, but currently it will fail on old versions - this patch fixes
this, to silently ignore the provenance flag for this check if it's set
to disabled.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:36:55 +00:00
Nick Santos
b14bfb9fa2 docs: small tweak to opentelemetry
The buildx builder persists across Desktop restarts. So when we start up Jaeger, we should recommend that people configure it to persist across restarts as well.

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-01-11 12:09:05 -05:00
Justin Chadwell
56950ece69 inspect: lazily load attestation data
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.

With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 13:00:42 +00:00
Justin Chadwell
1d2ac78443 inspect: move attestation loading to struct methods
This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 12:40:42 +00:00
Justin Chadwell
8b7aa1a168 build: create error group per opt
Using the syncronization primitive, we can avoid needing to create a
separate wait group.

This allows us to sidestep the issue where the wait group could be
completed, but the build invocation functions had not terminated - if
one of the functions was to terminate with an error, then it was
possible to encounter a race condition, where the result handling code
would begin executing, despite an error.

The refactor to use a separate error group which more elegantly handles
the concept of function returns and errors, ensures that we can't
encounter this issue.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 11:02:27 +00:00
Justin Chadwell
1180d919f5 build: reorder error group funcs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
347417ee12 build: use copy for BuildWithResultHandler loop vars
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
fb27e3f919 Merge pull request #1502 from developer-guy/fix/readme
fix the directory of the buildx binary
2023-01-10 10:28:04 +00:00
Batuhan Apaydın
edb16f8aab fix the directory of the buildx binary
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2023-01-10 11:26:11 +03:00
Tõnis Tiigi
5c56e947fe Merge pull request #1500 from tonistiigi/update-buildkit-v0.11.0
github: update CI to buildkit v0.11
2023-01-09 15:59:25 -08:00
Tonis Tiigi
571871b084 github: update CI to buildkit v0.11
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-09 15:50:56 -08:00
Tõnis Tiigi
8340c40647 Merge pull request #1498 from jedevc/attestation-printing
Improved attestation inspect
2023-01-09 11:22:20 -08:00
Justin Chadwell
9818055b0e docs: update with new inspect output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:07:10 +00:00
Justin Chadwell
484823c97d inspect: change additional spdxs to not have duplicates
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:03:23 +00:00
Justin Chadwell
3ce17b01dc inspect: provide access to multiple spdx documents
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:42:26 +00:00
Justin Chadwell
e68c566c1c inspect: parse sbom and provenance into json structs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:43 +00:00
Justin Chadwell
19d16aa941 inspect: break after first matching attestation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:06 +00:00
CrazyMax
6852713121 Merge pull request #1494 from thaJeztah/docs_update_docs_tools
go.mod: update cli-docs-tool v0.5.1
2023-01-09 13:14:01 +01:00
Sebastiaan van Stijn
c97500b117 go.mod: update cli-docs-tool v0.5.1 and re-generate docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:03:01 +01:00
CrazyMax
85040a9067 Merge pull request #1493 from thaJeztah/docs_fix_anchors
docs: update anchor links
2023-01-09 12:19:58 +01:00
Sebastiaan van Stijn
b8285c17e6 docs: update anchor links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-07 15:12:08 +01:00
Tõnis Tiigi
332dfb4b92 Merge pull request #1444 from crazy-max/inspect-attest
imagetools inspect: handle provenance and sboms
2023-01-06 16:29:33 -08:00
Tõnis Tiigi
cb279bb14b Merge pull request #1491 from jedevc/ociindex-refactor
vendor: update buildkit to v0.11.0-rc4
2023-01-06 16:29:18 -08:00
Justin Chadwell
60c9cf74ce vendor: update buildkit to v0.11.0-rc4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-06 15:51:17 -08:00
Tõnis Tiigi
ff6754eb04 Merge pull request #1456 from jedevc/oci-layout-reference-parsing
build: refactor reference parsing for oci image layouts
2023-01-05 23:50:25 -08:00
CrazyMax
e6b9aba997 imagetools inspect: handle provenance and sbom
use stub structs for SLSA/SBOM while waiting for
go-imageinspect library to be public.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 17:34:30 +01:00
CrazyMax
0302894bfb Merge pull request #1463 from crazy-max/e2e-k3s
e2e: use native k3s installation script
2023-01-05 16:50:45 +01:00
CrazyMax
e46394c3be e2e: use native k3s installation script
debianmaster/actions-k3s action gives some warnings in our e2e
workflow. This commit brings https://github.com/debianmaster/actions-k3s/blob/master/index.js
directly in the workflow through actions/github-script with
some changes to properly wait for nodes to be up.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 15:37:19 +01:00
Justin Chadwell
1885e41789 docs: update oci layout with tag resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-05 13:47:42 +00:00
CrazyMax
2fb9db994b imagetools inspect: missing annotations key
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-04 15:52:16 +01:00
Tõnis Tiigi
287aaf1696 Merge pull request #1482 from AkihiroSuda/auto-propagate-source-date-epoch
Propagate SOURCE_DATE_EPOCH from the client env
2022-12-27 17:32:08 -08:00
Akihiro Suda
0e6f5a155e Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-24 01:32:15 +09:00
Tõnis Tiigi
88852e2330 Merge pull request #1480 from crazy-max/fix-badge
disable buildx experimental in pipeline
2022-12-16 10:28:20 -08:00
CrazyMax
6369c50614 disable buildx experimental in pipeline
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-16 18:53:46 +01:00
CrazyMax
a22d0a35a4 readme: fix status badge
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-16 17:52:30 +01:00
CrazyMax
c93c02df85 Merge pull request #1479 from jedevc/fixup-git-err-check-order
build: check error from toSolveOpt before adding FrontendAttrs
2022-12-16 16:28:17 +01:00
Justin Chadwell
e584c6e1a7 build: check error from toSolveOpt before adding FrontendAttrs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-16 12:19:33 +00:00
Tõnis Tiigi
64e4c19971 Merge pull request #1477 from crazy-max/git-wsl
build: lookup the right git binary on WSL
2022-12-15 18:00:52 -08:00
Tonis Tiigi
551b8f6785 git: do not show warnings if project does not use git
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-15 17:51:46 -08:00
Tõnis Tiigi
fbbe1c1b91 Merge pull request #1472 from crazy-max/ci-attest
ci: opt-in sbom and provenance
2022-12-15 17:38:13 -08:00
Tonis Tiigi
1a85745bf1 github: update buildkit image to v0.11-rc3
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-15 16:54:32 -08:00
CrazyMax
0d1fea8134 build: warn if git operation fails
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 23:40:19 +01:00
CrazyMax
19417e76e7 build: lookup the right git binary on WSL
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 21:16:37 +01:00
Tõnis Tiigi
53d88a79ef Merge pull request #1475 from jedevc/attest-warn-duplicate
buildflags: error on duplicate attest field
2022-12-15 11:07:47 -08:00
Justin Chadwell
4c21b7e680 Merge pull request #1476 from jedevc/dont-filter-attestation-opts
build: forward all build opts everywhere
2022-12-15 19:05:39 +00:00
Justin Chadwell
a8f689c223 build: forward all build opts everywhere
All build options should be passed everywhere - the frontend and the
backend of buildkit should both be able to see all attestations, as well
as all other opts: e.g. epoch settings, and no-cache.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-15 18:19:47 +00:00
CrazyMax
ba8e3f9bc5 ci: generate provenance and sbom for bin image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 19:12:26 +01:00
CrazyMax
477200d1f9 ci: generate provenance and sbom for release binaries
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 19:12:25 +01:00
Tõnis Tiigi
662738a7e5 Merge pull request #1474 from crazy-max/fix-ci
ci: fix warnings
2022-12-15 09:29:25 -08:00
Justin Chadwell
f992b77535 buildflags: warn on duplicate attest field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-15 15:39:19 +00:00
CrazyMax
21b2f135b5 ci: update to ubuntu 22.04
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:34:03 +01:00
CrazyMax
71e6be5d99 ci: fix deprecated set-output syntax
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:24:02 +01:00
CrazyMax
df8e7d0a9a Merge pull request #1473 from crazy-max/fix-docs-prerelease
ci: do not publish docs on prerelease
2022-12-15 14:22:07 +01:00
CrazyMax
64422a48d9 Merge pull request #1471 from crazy-max/fix-builder-factory
builder: check endpoint from store when loading factory
2022-12-15 14:19:20 +01:00
CrazyMax
04f9c62772 ci: do not publish docs on prerelease
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:16:45 +01:00
CrazyMax
2185d07f05 builder: check endpoint from store when loading factory
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 13:32:54 +01:00
CrazyMax
a49d28e00e Merge pull request #1460 from crazy-max/docker-load
e2e: load to docker store
2022-12-15 10:35:29 +01:00
Tõnis Tiigi
629128c497 Merge pull request #1449 from crazy-max/bake-var-null
bake: support null arg and label value
2022-12-14 21:03:05 -08:00
Justin Chadwell
70682b043e build: refactor reference parsing for image layouts
We allow any valid image reference format for the oci-layout, not just
limiting to name@digest, we additionally allow images of the form
name:tag@digest now.

The name of the reference is used to find the local directory to lookup
the store in, while the tag and digest are attached to a random identity
to generate the dummy reference sent to the oci-layout context.

This separation of the target to replace and the value to replace it
with ensures that any tag or digest set in the client is properly sent
across to the server. The tag is used when a digest was not specified,
and it is resolved in the context of the local directory before being
sent, using the same helpers as we use for the local cache expoter.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-14 16:53:10 +00:00
Justin Chadwell
b741d26eb5 Merge pull request #1468 from crazy-max/fix-git-vcs-check
build: fix env vars check for vcs details
2022-12-14 12:51:29 +00:00
CrazyMax
cf8fa4a404 build: fix env vars check for vcs details
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 09:48:55 +01:00
CrazyMax
fe76a1b179 bake: support null label value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 08:30:58 +01:00
CrazyMax
df4957307f bake: support null arg value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 08:29:27 +01:00
Tõnis Tiigi
e21f56e801 Merge pull request #1434 from jedevc/resource-interpolation
Resource interpolation support
2022-12-13 18:06:51 -08:00
Justin Chadwell
e51b55e03c bake: add tests for block interpolation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:39 -08:00
Justin Chadwell
296b8249cb bake: support block-based interpolation
This patch adds support for block-based interpolation, so that
properties of blocks can be referenced in the current block and across
other blocks.

Previously, order-of-evaluation did not matter for blocks, and could be
evaluated in any order. However, now that blocks can refer to each
other, we split out this dynamic evaluation order into a separate
resolveBlock function.

Additionally, we need to support partial block evaluations - if block A
refers to property X of block B, when we should only evaluate property
X, and not the entire block. This ensures that we can safely evaluate
blocks that refer to other properties within themselves, and allows
sequences that would otherwise be co-recursive. We take special care in
this logic to ensure that each property is evaluated once *and only*
once - this could otherwise present inconsistencies with stateful
functions, and could risk inconsistent results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:39 -08:00
Justin Chadwell
7c6b840199 bake: add cty tags to hcl structures
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:36 -08:00
Tõnis Tiigi
2a6ff4cbfc Merge pull request #1462 from crazy-max/attest-vcs
build: set provenance vcs details
2022-12-13 17:23:20 -08:00
CrazyMax
6ad5e2fcf3 build: set provenance vcs details
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 00:09:38 +01:00
Tõnis Tiigi
37811320ef Merge pull request #1439 from crazy-max/last-activity
store: set nodegroup last activity
2022-12-13 15:00:47 -08:00
CrazyMax
99ac7f5f9e e2e: load to docker store
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-13 23:32:58 +01:00
Tõnis Tiigi
96aca741a2 Merge pull request #1467 from tonistiigi/update-buildkit-862b22
vendor: update buildkit to 862b22d7
2022-12-13 12:07:42 -08:00
Tonis Tiigi
12ec931237 vendor: update buildkit to 862b22d7
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-13 12:00:06 -08:00
Justin Chadwell
0e293a4ec9 Merge pull request #1464 from jedevc/vendor-buildkit-93b40706a007
vendor: update buildkit to 93b40706a007
2022-12-13 11:12:54 +00:00
Justin Chadwell
163712a23b vendor: update buildkit to 93b40706a007
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 11:03:53 +00:00
Tõnis Tiigi
5f4d463780 Merge pull request #1435 from thaJeztah/bump_buildkit_v0.11
vendor: github.com/moby/buildkit v0.11.0-rc1.0.20221207183946-5993b526de65
2022-12-08 19:51:10 -08:00
Tõnis Tiigi
abc8121aa8 Merge pull request #1457 from jedevc/add-hosts-length-check
build: don't set add-hosts option if empty
2022-12-08 19:48:09 -08:00
CrazyMax
8c47277141 store: set nodegroup last activity
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-08 19:49:51 +01:00
CrazyMax
36b5cd18e8 store: use constants for directory names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-08 19:35:55 +01:00
Justin Chadwell
1e72e32ec3 Merge pull request #1412 from jedevc/attestations-cli
Attestations from buildx
2022-12-08 17:30:31 +00:00
Sebastiaan van Stijn
8e5e5a563d vendor: github.com/moby/buildkit v0.11.0-rc1.0.20221207183946-5993b526de65
- drops the replace-rule for github.com/aws/aws-sdk-go-v2/config (as it no longer was replacing anything)
- drops the replace-rules for docker/cli and docker/docker (at least as long as we continue using tagged releases)
- removes the github.com/docker/docker/pkg/stringid package (which was redundant)

full diff: 9624ab4710..5993b526de

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-07 23:27:29 +01:00
Justin Chadwell
98049e7eda build: don't set add-hosts option if empty
This looks like an oversight, all of the other options have similar
checks. This can interfere with generated provenance where "add-hosts"
will be marked as an argument to the build, even though it's not
actually being utilized.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:16:01 +00:00
Justin Chadwell
25aa893bad bake: add attests field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:44:21 +00:00
Justin Chadwell
b270a20274 build: add attests flag and sbom/provenance shorthands
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:44:11 +00:00
Justin Chadwell
f0262dd10e build: add attestations to build options
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:43:36 +00:00
Justin Chadwell
f8b673eccd build: pass attestation attributes to build request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 16:10:54 +00:00
Justin Chadwell
0c0c9a0030 chore: sort buildOptions alphabetically
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 16:10:54 +00:00
CrazyMax
d1f79317cf Merge pull request #1455 from crazy-max/nodegroup-exclude-field
store: skip DockerContext field from being saved
2022-12-07 16:07:30 +01:00
CrazyMax
fa58522242 store: skip DockerContext field from being saved
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-07 13:15:18 +01:00
CrazyMax
aa6fd3d888 Merge pull request #1454 from crazy-max/builder-fix-default
builder: fix default docker context behavior
2022-12-07 11:03:06 +01:00
CrazyMax
ebdd8834a9 builder: fix default docker context behavior
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-07 08:52:12 +01:00
CrazyMax
fe8d5627e0 Merge pull request #1433 from crazy-max/makefile-build-opts
hack: mutualize build opts in Makefile and Dockerfile
2022-12-07 05:04:44 +01:00
CrazyMax
b242e3280b Merge pull request #1430 from crazy-max/builder-pkg
Refactor builder and drivers info logic
2022-12-06 12:41:10 +01:00
CrazyMax
cc01caaecb builder: enhance driver factory logic when loading drivers
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
e7b5ee7518 mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
63073b65c0 dockerutil pkg to manage docker api client and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
47cf72b8ba Merge pull request #1451 from crazy-max/update-buildkit
vendor: update buildkit to master@9624ab4
2022-12-05 17:30:39 +01:00
CrazyMax
af24d72dd8 kubernetes: fix context load test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 17:13:03 +01:00
CrazyMax
f451b455c4 vendor: update buildkit to master@9624ab4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 17:03:47 +01:00
Justin Chadwell
16f4dfafb1 Merge pull request #1450 from crazy-max/fix-hclparser-jsonfunc
hcl: SrcRange not checked when solving JSON func calls
2022-12-05 15:17:23 +00:00
CrazyMax
5b4e8b9d71 hcl: SrcRange not checked when solving JSON func calls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 15:05:17 +01:00
CrazyMax
b06eaffeeb Merge pull request #1442 from crazy-max/hcl-fix-panic
bake: fix panic for unsupported hcl variable type
2022-12-05 14:30:04 +01:00
CrazyMax
3d55540db1 Merge pull request #1445 from dvdksn/docs/move-manuals
docs: moved manual pages to docs repo, added link
2022-12-03 15:41:01 +01:00
Tõnis Tiigi
3c2b9aab96 Merge pull request #1446 from crazy-max/moby-host-gateway
build: skip "host-gateway" validation with moby driver
2022-12-02 20:24:40 -08:00
CrazyMax
49d46e71de build: skip "host-gateway" validation with moby driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-02 13:53:13 +01:00
David Karlsson
6c5168e1ec docs: moved manual pages to docs repo, added link
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-12-02 09:49:54 +01:00
Tõnis Tiigi
e91d5326fe Merge pull request #1441 from crazy-max/fix-nil-ref
imagetools: set default repo ref on creation if nil
2022-12-01 09:51:09 -05:00
CrazyMax
48b573e835 bake: fix panic for unsupported hcl variable type
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-01 00:52:05 +01:00
CrazyMax
4788eb24ab imagetools: set default repo ref on creation if nil
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-30 23:18:43 +01:00
CrazyMax
3ed2783f34 Merge pull request #1436 from thaJeztah/protobuf_extensions_fix
go.mod: golang_protobuf_extensions v1.0.4 - prevent incompat versions
2022-11-30 14:11:33 +01:00
Sebastiaan van Stijn
c0e8a41a6f go.mod: golang_protobuf_extensions v1.0.4 - prevent incompat versions
This module made a whoopsie, and updated to `google.golang.org/protobuf`
in a patch release, but `google.golang.org/protobuf` is not backward
compatible with `github.com/golang/protobuf`.

Updating the minimum version to v1.0.4 which corrects this, to prevent
users of buildx as a module from accidentally pulling in the wrong
version:

- v1.0.3 switched to use `google.golang.org/protobuf`; https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.2..v1.0.3
- This was reverted in v1.0.4 (which is the same as v1.0.2); https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.3..v1.0.4
- And a `v2` was created instead; https://github.com/matttproud/golang_protobuf_extensions/releases/tag/v2.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 13:31:42 +01:00
CrazyMax
23b217af24 hack: mutualize build opts in Makefile and Dockerfile
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-29 17:05:25 +01:00
CrazyMax
3dab19f933 Merge pull request #1432 from crazy-max/hack-hash-files
hack: improve hash-files script
2022-11-29 13:28:56 +01:00
CrazyMax
05efb6291f hack: improve hash-files script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-29 12:36:34 +01:00
CrazyMax
eba49fdefd Merge pull request #1255 from crazy-max/docker-api
docker api: use helper to parse context docker endpoint metadata
2022-11-29 12:16:52 +01:00
CrazyMax
29f2c49374 Merge pull request #1268 from crazy-max/hack-output
hack: use single output dir
2022-11-29 12:16:38 +01:00
CrazyMax
2245371696 Merge pull request #1420 from jedevc/oci-tar
Support new `tar` option for oci+docker exporters
2022-11-29 11:48:10 +01:00
CrazyMax
74631d5808 Merge pull request #1431 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.3
build(deps): Bump peter-evans/create-pull-request from 4.2.2 to 4.2.3
2022-11-28 20:11:01 +01:00
dependabot[bot]
9264b0ca09 build(deps): Bump peter-evans/create-pull-request from 4.2.2 to 4.2.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.2 to 4.2.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](331d02c7e2...2b011faafd)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-28 18:13:29 +00:00
CrazyMax
a96fb92939 Merge pull request #1429 from crazy-max/update-gh-release
ci: update softprops/action-gh-release to v0.1.15
2022-11-28 14:04:02 +01:00
CrazyMax
ae59e1f72e Merge pull request #1305 from jedevc/progress-group-prefixed-writer
progress: add prefix to vertex progress group
2022-11-28 13:32:27 +01:00
CrazyMax
47167a4e6f ci: update softprops/action-gh-release to v0.1.15
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-28 13:29:13 +01:00
CrazyMax
23cabd67fb Merge pull request #1427 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.2
build(deps): Bump peter-evans/create-pull-request from 4.2.0 to 4.2.2
2022-11-25 11:27:43 +01:00
CrazyMax
e66410b932 Merge pull request #1313 from jedevc/bake-group-recurse
bake: recursively resolve groups
2022-11-25 11:27:09 +01:00
dependabot[bot]
c3bba05770 build(deps): Bump peter-evans/create-pull-request from 4.2.0 to 4.2.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.0 to 4.2.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](b4d51739f9...331d02c7e2)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-24 18:07:15 +00:00
Justin Chadwell
69b91f2760 docs: add tar flag to oci+docker exporters
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
e6b09580b4 build: support tar flag for oci+docker exporters
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
36e663edda vendor: update buildkit to master@ae9d0f5
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
60e2029e70 Merge pull request #1419 from jedevc/docs-clarify-support
docs: clarify support for various sets of shared options
2022-11-23 10:25:14 +00:00
Justin Chadwell
5e1db43e34 docs: clarify support for various sets of shared options
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-22 11:10:50 +00:00
CrazyMax
6e9b743296 Merge pull request #1417 from accetto/patch-2
Correction in Synopsis.
2022-11-21 09:40:21 +01:00
accetto
ef9710d8e2 Correction in Synopsis.
Correction in Synopsis.

Signed-off-by: accetto <34798830+accetto@users.noreply.github.com>
2022-11-20 13:34:52 +00:00
CrazyMax
468b3b9c8c Merge pull request #1407 from AkihiroSuda/x-crypto-ssh
go.mod: golang.org/x/crypto v0.1.0 (Fix `ssh: parse error in message type 27` with OpenSSH >= 8.9)
2022-11-17 13:27:05 +01:00
CrazyMax
0d8c853917 Merge pull request #1394 from thaJeztah/update_cobra
go.mod: github.com/spf13/cobra v1.6.1
2022-11-17 13:22:29 +01:00
CrazyMax
df3b868fe7 Merge pull request #1414 from crazy-max/move-k8s-config
kubernetes: move context config logic to its own pkg
2022-11-17 11:22:40 +01:00
CrazyMax
3f6a5ab6ba kubernetes: move context config logic to its own pkg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-17 00:34:42 +01:00
Tõnis Tiigi
aa1f4389b1 Merge pull request #1396 from crazy-max/fix-indent
use double spaces with json marshal indent
2022-11-16 14:54:13 -08:00
Justin Chadwell
246cd2aee9 Merge pull request #1411 from jedevc/docker-container-volume-docs
docs: add cache persistence notes for docker-container driver
2022-11-16 12:29:01 +00:00
Justin Chadwell
0b6f8149d1 docs: add cache persistence notes for docker-container driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-16 11:56:42 +00:00
Akihiro Suda
4dda2ad58b go.mod: golang.org/x/crypto v0.1.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-16 07:43:29 +09:00
Justin Chadwell
15bb14fcf9 Merge pull request #1406 from felixdesouza/fds/fix-concurrent-map-write
Synchronise access to the map when printing.
2022-11-15 15:46:54 +00:00
Felix de Souza
b68114375f Address feedback
Signed-off-by: Felix de Souza <fdesouza@palantir.com>
2022-11-14 18:40:05 +00:00
Felix de Souza
83a09b3cf2 Synchronise access to the map when printing.
Signed-off-by: Felix de Souza <fdesouza@palantir.com>
2022-11-14 15:47:32 +00:00
Justin Chadwell
3690cb12e6 Merge pull request #1397 from markhildreth-gravity/patch-1
Correct documentation on gha scopes
2022-11-10 13:39:11 +00:00
Justin Chadwell
b4de4826c4 Merge pull request #1403 from jedevc/docs-fixups
Docs consistency fixes
2022-11-10 13:29:41 +00:00
Justin Chadwell
b06df637c7 docs: use consistent "Type" header across parameter tables
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-10 10:44:17 +00:00
Justin Chadwell
9bb9ae43f9 docs: add env parameter to docker-container driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-10 10:44:17 +00:00
Mark Hildreth
35e7172b89 Correct documentation on gha scopes
I believe the documentation is misleading in saying that specifying a scope in a gha cache declaration will `ensure each branch gets its own cache`. I believe this should say that each build or each image will get its own cache.

Signed-off-by: Mark Hildreth <mark.hildreth@gravityclimate.com>
2022-11-07 11:06:07 -05:00
CrazyMax
abebf4d955 use double spaces with json marshal indent
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-07 16:36:24 +01:00
Sebastiaan van Stijn
1c826d253b go.mod: github.com/spf13/cobra v1.6.1
full diff: https://github.com/spf13/cobra/compare/v1.5.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 19:03:52 +01:00
CrazyMax
d1b454232d Merge pull request #1392 from dvdksn/refactor-docs
docs: refactored file and directory structure
2022-11-05 10:07:10 +01:00
David Karlsson
be3b41acc6 docs: refactored file and directory structure
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-11-04 19:57:50 +01:00
Justin Chadwell
2a3e51ebfe Merge pull request #1390 from dvdksn/update-docs-default-branch
docs: update git ref to docs repo
2022-11-04 16:54:44 +00:00
David Karlsson
1382fda1c9 docs: update git ref to docs repo
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-11-04 16:37:46 +01:00
Justin Chadwell
c658096c17 Merge pull request #1385 from jedevc/exporter-docs
Add exporter docs
2022-11-04 15:16:30 +00:00
Justin Chadwell
6097919958 docs: change cache storage links to exporter to docs site
Temporary fix while we update docs upstream.

Co-Authored-By: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-04 14:57:17 +00:00
Tõnis Tiigi
330bdde0a3 Merge pull request #1383 from jedevc/with-tracer-delegate-panic
driver: don't create tracer delegate opt if tracer is nil
2022-11-02 10:58:58 -07:00
CrazyMax
a55404fa2e Merge pull request #1388 from crazy-max/fix-docs-validate
ci(docs-upstream): fix ref on pull request event
2022-11-02 15:22:29 +01:00
CrazyMax
c8c7c9f376 Merge pull request #1387 from crazy-max/update-compose-go
update github.com/compose-spec/compose-go to v1.6.0
2022-11-02 15:22:12 +01:00
CrazyMax
df34c1ce45 ci(docs-upstream): switch to reusable workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-01 19:45:00 +01:00
CrazyMax
da1d66c938 update github.com/compose-spec/compose-go to v1.6.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-01 16:14:56 +01:00
Justin Chadwell
d32926a7e5 docs: add exporter docs
This patch adds more complete documentation for the various exporters
available to buildx.

Co-Authored-By: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-01 12:16:45 +00:00
Justin Chadwell
7f008a7d1e driver: don't create tracer delegate opt if tracer is nil
The error handling for the cast to client.TracerDelegate was incorrect,
and previously, a client would unconditionally append an opt.

This results in the scenario that while the ClientOpt was not nil, the
tracer delegate in the ClientOpt was, which isn't an error case
explicitly handled by buildkit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-31 11:19:47 +00:00
Tõnis Tiigi
eab3f704f5 Merge pull request #1371 from jedevc/progress-tty-fail
progress: explicitly fail if tty requested but not available
2022-10-25 11:07:50 -07:00
Justin Chadwell
a50e89c38e progress: explicitly fail if tty requested but not available
The NewPrinter function is mostly borrowed from buildkit. However, at
some point, it seems that the implementations drifted.

This patch updates buildx to be more similar in behavior to it's
buildkit counterpart, specifically, it will explicitly fail if a TTY
output is requested using "--progress=tty", but the output is not
available.

To gracefully fallback to plain progress in this scenario,
"--progress=plain" is required.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-25 10:55:36 +01:00
CrazyMax
85723a138f hack: lint scripts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-24 08:30:35 +02:00
CrazyMax
9c69ba6f6f hack: use single output dir
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-24 08:30:35 +02:00
CrazyMax
e84ed65525 Merge pull request #1356 from thaJeztah/protobuf_extensions_tag
go.mod: matttproud/golang_protobuf_extensions v1.0.2 (use tag)
2022-10-24 08:15:35 +02:00
CrazyMax
4060abd3aa Merge pull request #1363 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.0
build(deps): Bump peter-evans/create-pull-request from 4.1.4 to 4.2.0
2022-10-24 08:15:17 +02:00
CrazyMax
c924a0428d Merge pull request #1368 from jedevc/userns-to-driver
Userns to driver
2022-10-24 08:15:01 +02:00
Justin Chadwell
33ef1b3a30 docker-container: move userns detection into driver
This moves the detection of the docker daemon's security options into
the driver from the factory, handling them in a similar way to how we do
cgroups.

Because of recent changes that modify error detection in driver
creation, this attempt to contact the docker daemon during builder
creation meant that a docker-container builder could not be created
without access to the docker socket. This patch resolves this, by
defering the Info call to the driver, when the container is actually
created.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:44:44 +01:00
Justin Chadwell
a6caf4b948 chore: tidy up duplicate dockertypes import
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:40:17 +01:00
Justin Chadwell
cc7e11da99 Merge pull request #1366 from jedevc/remove-structcheck
lint: remove structcheck
2022-10-20 10:22:50 +01:00
Justin Chadwell
a4c3efe783 lint: add nolintlint and fix violations
We should be able to detect nolint comments that point to linters that
are disabled (such as with the removed structcheck).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-20 09:49:51 +01:00
Justin Chadwell
4e22846e95 lint: remove structcheck
structcheck is abandoned, and has been replaced by the unused linter.
See https://golangci-lint.run/usage/linters/#structcheck for more
information.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-19 14:19:06 +01:00
dependabot[bot]
ddbd0cd095 build(deps): Bump peter-evans/create-pull-request from 4.1.4 to 4.2.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.4 to 4.2.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](ad43dccb4d...b4d51739f9)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-18 18:56:21 +00:00
Justin Chadwell
255a3ec82c Merge pull request #1361 from dvdksn/docs-fix-cache-codeblocks
docs: add user sign for console blocks
2022-10-18 11:57:53 +01:00
David Karlsson
167c77baec docs: add user sign for console blocks
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-18 12:50:14 +02:00
CrazyMax
ca2718366e Merge pull request #1000 from tonistiigi/docs-named-context
docs: make sure syntax with latest stable dockerfile image is defined
2022-10-18 10:08:06 +02:00
CrazyMax
58d3a643b9 Merge pull request #1345 from crazy-max/bake-print
bake: do not fail printing definition if instance unavailable
2022-10-18 10:05:14 +02:00
CrazyMax
718b8085fa Merge pull request #1354 from dvdksn/fix-cache-cmds
docs: fix cache command examples
2022-10-17 22:32:29 +02:00
CrazyMax
64930d7440 Merge pull request #1359 from jedevc/cache-inline-typo
docs: fix typo in cache docs
2022-10-17 22:31:27 +02:00
CrazyMax
4d2f948869 Merge pull request #1360 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.4
build(deps): Bump peter-evans/create-pull-request from 4.1.3 to 4.1.4
2022-10-17 22:30:54 +02:00
dependabot[bot]
19c224cbe1 build(deps): Bump peter-evans/create-pull-request from 4.1.3 to 4.1.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](671dc9c9e0...ad43dccb4d)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-17 18:34:07 +00:00
Justin Chadwell
efd1581c01 docs: fix typo in cache docs
We should keep consistency with the rest of the list, and call it
"inline" instead of "inline-cache".

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-17 17:06:41 +01:00
Tõnis Tiigi
ac85f590ba Merge pull request #1324 from jedevc/invoke-defaults
invoke: load defaults from image config
2022-10-13 09:01:29 -07:00
Sebastiaan van Stijn
b0d3162875 go.mod: matttproud/golang_protobuf_extensions v1.0.2 (use tag)
it's the same commit: https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.2...c182affec369

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 16:12:19 +02:00
David Karlsson
4715a7e9e1 docs: fix cache command examples
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-12 17:15:55 +02:00
CrazyMax
c5aec243c9 Merge pull request #1353 from dvdksn/absolute-links-part2
docs: replaced broken relative links
2022-10-11 23:16:10 +02:00
David Karlsson
c76f3d3dba docs: replaced broken relative links
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-11 21:35:52 +02:00
Justin Chadwell
7add6e48b6 Merge pull request #1352 from dvdksn/remove-cache-opt-link
remove/update links to satisfy htmlproofer
2022-10-11 13:17:26 +01:00
David Karlsson
1267e0c076 changed from relative to absolute links
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-11 13:53:06 +02:00
Justin Chadwell
361c093a35 Merge pull request #1332 from dvdksn/refactor-cache-docs
added tables for cache parameters
2022-10-11 12:19:35 +01:00
CrazyMax
9ad39a29f7 Merge pull request #1334 from dvdksn/driver-docs-review
Driver docs update
2022-10-07 16:17:18 +02:00
David Karlsson
f5a1d8bff9 moved context arg to last
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-06 14:04:54 +02:00
CrazyMax
8c86afbd57 Merge pull request #1310 from AkihiroSuda/gcos-rootless
kubernetes: rootless: support Google Container-Optimized OS  (Fix ` Options:[rbind ro]}]: operation not permitted` errors)
2022-10-06 11:05:19 +02:00
David Karlsson
4d6e36df99 review comments, align style with driver docs
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:33:57 +02:00
David Karlsson
f51884e893 address review comments
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:18:42 +02:00
David Karlsson
4afd9ecf16 fix incorrect heading text
Co-authored-by: Justin Chadwell <github@jedevc.com>
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:55 +02:00
David Karlsson
ed3b311de4 improve wording on default buildkit image
Co-authored-by: Justin Chadwell <github@jedevc.com>
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:50 +02:00
David Karlsson
d030fcc076 updated prose and structure for driver docs
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:27 +02:00
CrazyMax
398da1f916 Merge pull request #1343 from dgageot/fix-1342
[1342] Fix assignment to nil map
2022-10-04 17:31:06 +02:00
CrazyMax
3a5741f534 Merge pull request #1341 from dgageot/fix-1340
[1340] Disable git labels if `BUILDX_GIT_LABELS` is not `1` or `full`
2022-10-04 17:04:35 +02:00
CrazyMax
c53b0b8a12 bake: do not fail printing definition if instance unavailable
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-04 15:38:45 +02:00
David Gageot
8fd34669ed Fix assignment to nil map
Signed-off-by: David Gageot <david.gageot@docker.com>
2022-10-04 08:30:47 +02:00
David Gageot
be7e91899b Support empty env var when it can't be unset
Signed-off-by: David Gageot <david.gageot@docker.com>
2022-10-03 18:36:06 +02:00
David Karlsson
74a822568e added tables for cache parameters
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-09-29 21:40:47 +02:00
CrazyMax
105c214d15 Merge pull request #1333 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.3
build(deps): Bump peter-evans/create-pull-request from 4.1.2 to 4.1.3
2022-09-29 17:25:43 +02:00
dependabot[bot]
2b6a51ed34 build(deps): Bump peter-evans/create-pull-request from 4.1.2 to 4.1.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](171dd555b9...671dc9c9e0)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-28 18:46:49 +00:00
CrazyMax
e98c252490 Merge pull request #1327 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.2
build(deps): Bump peter-evans/create-pull-request from 4.0.4 to 4.1.2
2022-09-22 14:08:32 +02:00
CrazyMax
17f5d6309f Merge pull request #1329 from jedevc/ignore-workflows-on-docs-changes
ci: don't trigger build and e2e workflows on docs-only pull requests
2022-09-22 14:08:09 +02:00
CrazyMax
6a46ea04ab Merge pull request #1316 from jedevc/cache-docs
docs: add cache storage backend docs
2022-09-22 14:07:06 +02:00
CrazyMax
7bd97f6717 Merge pull request #1318 from crazy-max/new-docs-links
docs: update links to new docs
2022-09-22 14:06:47 +02:00
CrazyMax
2a9c98ae40 remove unrelated comment after upgrade
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-22 13:53:33 +02:00
Justin Chadwell
1adf80c613 ci: don't trigger build and e2e workflows on docs-only pull requests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 11:00:55 +01:00
Justin Chadwell
f823d3c73c docs: final cache storage fixups
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:52:13 +01:00
David Karlsson
91f0ed3fc3 docs: additions from editorial review
- editorial review
- address review comments, rework param sections
- added a common section for parameters
- remove liquid tags for notes

Signed-off-by: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:51:16 +01:00
Justin Chadwell
04b56c7331 docs: add cache storage backend docs
This fleshes out the documentation to include information along with
examples and detailed use cases for the cache backends. The general
format and style follows from the style of the build driver docs.

Eventually, these docs will be included on docs.docker.com along with
the rest of the build docs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:51:16 +01:00
dependabot[bot]
3c1a20097f build(deps): Bump peter-evans/create-pull-request from 4.0.4 to 4.1.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.4 to 4.1.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](923ad837f1...171dd555b9)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-21 18:48:36 +00:00
Justin Chadwell
966c4d4e14 invoke: load defaults from image config
If user does not specify image certain container parameters, we can load
them from the exporter metadata.

Additionally, we introduce a new "default" value for the --invoke flag,
that keeps all of the default parameters (since cobra does not have an
easy way of accepting an optional flag argument).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-21 11:43:52 +01:00
CrazyMax
6b8289d68e docs: update links to new docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-14 13:36:51 +02:00
CrazyMax
294421db9c Merge pull request #1317 from crazy-max/fix-link
docs: fix link in driver guide
2022-09-14 11:33:20 +02:00
CrazyMax
9fdf991c27 docs: update links in driver guide
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-13 22:11:47 +02:00
Justin Chadwell
77b33260f8 bake: recursively resolve groups
Groups that contained other groups were not recursively resolved by
ReadTargets, which prevented output from --print from being useable as a
self-contained bake file.

This patch ensures that all groups that are referenced inside the bake
file are actually defined under the groups field. This has required a
substantial refactor, as previously only a single group was returned
from ReadTargets, notably, returning a map of groups, instead of a
slice.

This does introduce a small behavior change to the behavior of --print -
while previously, passing a group name to bake would return all the
targets of that group back as the default group, now only the name of
that group will be inserted into the default group, keeping the original
group intact. The impact of this can be observed in some of the changes
to the bake_test.go file.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-12 13:51:34 +01:00
Akihiro Suda
33e5f47c6c kubernetes: rootless: support Google Container-Optimized OS
Tested with GKE Autopilot 1.24.3-gke.200 (kernel 5.10.123+, containerd 1.6.6).

ref: moby/buildkit PR 3097

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-09-09 17:16:35 +09:00
CrazyMax
25ceb90678 Merge pull request #1294 from crazy-max/s3-cache
s3 cache client-side support
2022-09-08 14:26:19 +02:00
CrazyMax
27e29055cb Merge pull request #1307 from zhyon404/master
return di.Err when it's not nil
2022-09-08 14:20:14 +02:00
Tõnis Tiigi
810ce31f4b Merge pull request #1297 from cdupuis/git-revision
Add git provenance labels
2022-09-06 10:23:50 -07:00
Christian Dupuis
e3c91c9d29 Add git provenance labels
as per #1290

Signed-off-by: Christian Dupuis <cd@atomist.com>
2022-09-06 19:11:55 +02:00
zhyon404
2f47838ea1 return di.Err when it's not nil
Signed-off-by: zhyon404 <zhyong4@gmail.com>
2022-09-05 18:05:03 +08:00
Justin Chadwell
0566e62995 progress: add prefix to vertex progress group
As buildkit now uses progress groups for the COPY --link instruction
we need to ensure that we additionally prefix the progress group name,
or the target name will be left off in bake commands with more than one
target.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-02 16:45:51 +01:00
CrazyMax
aeac42be47 Merge pull request #1299 from thaJeztah/store_cleanup
store: move regex to where it's used
2022-08-31 19:35:05 +02:00
Sebastiaan van Stijn
aa21ff7efd store: move regex to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 19:19:51 +02:00
CrazyMax
57d22a7bd1 s3 cache client-side support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-30 17:00:15 +02:00
CrazyMax
6804bcbf12 Merge pull request #1298 from nicksieger/compose-go-1.5.0
vendor: bump compose-go to v1.5.0
2022-08-30 15:31:19 +02:00
Nick Sieger
6d34cc0b60 vendor: bump compose-go to v1.5.0
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-08-29 10:22:31 -05:00
Tõnis Tiigi
1bb375fe5c Merge pull request #1265 from crazy-max/go-1.19
Dockerfile: update to go 1.19
2022-08-24 21:19:51 -07:00
CrazyMax
1378c616d6 docs: missing syntax directive
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-18 12:01:20 +02:00
Tonis Tiigi
3b5dfb3fb4 docs: make sure all named context examples use 1.4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-18 11:57:22 +02:00
CrazyMax
2a13491919 Dockerfile: update golangci-lint to 1.48.0 (go 1.19 support)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-11 02:10:33 +02:00
CrazyMax
3509a1a7ff Dockerfile: update to go 1.19
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-11 02:10:19 +02:00
CrazyMax
0defb614a4 docker api: use helper to parse context docker endpoint metadata
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-04 15:38:04 +02:00
3792 changed files with 402129 additions and 111330 deletions

View File

@@ -1,3 +1 @@
bin/
cross-out/
release-out/

View File

@@ -116,6 +116,60 @@ commit automatically with `git commit -s`.
### Run the unit- and integration-tests
Running tests:
```bash
make test
```
This runs all unit and integration tests, in a containerized environment.
Locally, every package can be tested separately with standard Go tools, but
integration tests are skipped if local user doesn't have enough permissions or
worker binaries are not installed.
```bash
# run unit tests only
make test-unit
# run integration tests only
make test-integration
# test a specific package
TESTPKGS=./bake make test
# run all integration tests with a specific worker
TESTFLAGS="--run=//worker=remote -v" make test-integration
# run a specific integration test
TESTFLAGS="--run /TestBuild/worker=remote/ -v" make test-integration
# run a selection of integration tests using a regexp
TESTFLAGS="--run /TestBuild.*/worker=remote/ -v" make test-integration
```
> **Note**
>
> Set `TEST_KEEP_CACHE=1` for the test framework to keep external dependant
> images in a docker volume if you are repeatedly calling `make test`. This
> helps to avoid rate limiting on the remote registry side.
> **Note**
>
> Set `TEST_DOCKERD=1` for the test framework to enable the docker workers,
> specifically the `docker` and `docker-container` drivers.
>
> The docker tests cannot be run in parallel, so require passing `--parallel=1`
> in `TESTFLAGS`.
> **Note**
>
> If you are working behind a proxy, you can set some of or all
> `HTTP_PROXY=http://ip:port`, `HTTPS_PROXY=http://ip:port`, `NO_PROXY=http://ip:port`
> for the test framework to specify the proxy build args.
### Run the helper commands
To enter a demo container environment and experiment, you may run:
```

124
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,124 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Bug Report
description: Report a bug
labels:
- status/triage
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to report a bug!
If this is a security issue please report it to the [Docker Security team](mailto:security@docker.com).
- type: checkboxes
attributes:
label: Contributing guidelines
description: |
Please read the contributing guidelines before proceeding.
options:
- label: I've read the [contributing guidelines](https://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree
required: true
- type: checkboxes
attributes:
label: I've found a bug and checked that ...
description: |
Make sure that your request fulfills all of the following requirements.
If one requirement cannot be satisfied, explain in detail why.
options:
- label: ... the documentation does not mention anything about my problem
- label: ... there are no open or closed issues that are related to my problem
- type: textarea
attributes:
label: Description
description: |
Please provide a brief description of the bug in 1-2 sentences.
validations:
required: true
- type: textarea
attributes:
label: Expected behaviour
description: |
Please describe precisely what you'd expect to happen.
validations:
required: true
- type: textarea
attributes:
label: Actual behaviour
description: |
Please describe precisely what is actually happening.
validations:
required: true
- type: input
attributes:
label: Buildx version
description: |
Output of `docker buildx version` command.
Example: `github.com/docker/buildx v0.8.1 5fac64c2c49dae1320f2b51f1a899ca451935554`
validations:
required: true
- type: textarea
attributes:
label: Docker info
description: |
Output of `docker info` command.
render: text
- type: textarea
attributes:
label: Builders list
description: |
Output of `docker buildx ls` command.
render: text
validations:
required: true
- type: textarea
attributes:
label: Configuration
description: >
Please provide a minimal Dockerfile, bake definition (if applicable) and
invoked commands to help reproducing your issue.
placeholder: |
```dockerfile
FROM alpine
echo hello
```
```hcl
group "default" {
targets = ["app"]
}
target "app" {
dockerfile = "Dockerfile"
target = "build"
}
```
```console
$ docker buildx build .
$ docker buildx bake
```
validations:
required: true
- type: textarea
attributes:
label: Build logs
description: |
Please provide logs output (and/or BuildKit logs if applicable).
render: text
validations:
required: false
- type: textarea
attributes:
label: Additional info
description: |
Please provide any additional information that could be useful.

12
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
blank_issues_enabled: true
contact_links:
- name: Questions and Discussions
url: https://github.com/docker/buildx/discussions/new
about: Use Github Discussions to ask questions and/or open discussion topics.
- name: Command line reference
url: https://docs.docker.com/engine/reference/commandline/buildx/
about: Read the command line reference.
- name: Documentation
url: https://docs.docker.com/build/
about: Read the documentation.

15
.github/ISSUE_TEMPLATE/feature.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Feature request
description: Missing functionality? Come tell us about it!
labels:
- kind/enhancement
- status/triage
body:
- type: textarea
id: description
attributes:
label: Description
description: What is the feature you want to see?
validations:
required: true

12
.github/SECURITY.md vendored Normal file
View File

@@ -0,0 +1,12 @@
# Reporting security issues
The project maintainers take security seriously. If you discover a security
issue, please bring it to their attention right away!
**Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Security reports are greatly appreciated, and we will publicly thank you for it.
We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
ruling it out in the future.

735
.github/releases.json vendored Normal file
View File

@@ -0,0 +1,735 @@
{
"latest": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.2": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.1": {
"id": 90346950,
"tag_name": "v0.10.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/checksums.txt"
]
},
"v0.10.0": {
"id": 88388110,
"tag_name": "v0.10.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/checksums.txt"
]
},
"v0.10.0-rc3": {
"id": 88191592,
"tag_name": "v0.10.0-rc3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/checksums.txt"
]
},
"v0.10.0-rc2": {
"id": 86248476,
"tag_name": "v0.10.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/checksums.txt"
]
},
"v0.10.0-rc1": {
"id": 85963900,
"tag_name": "v0.10.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/checksums.txt"
]
},
"v0.9.1": {
"id": 74760068,
"tag_name": "v0.9.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/checksums.txt"
]
},
"v0.9.0": {
"id": 74546589,
"tag_name": "v0.9.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/checksums.txt"
]
},
"v0.9.0-rc2": {
"id": 74052235,
"tag_name": "v0.9.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/checksums.txt"
]
},
"v0.9.0-rc1": {
"id": 73389692,
"tag_name": "v0.9.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/checksums.txt"
]
},
"v0.8.2": {
"id": 63479740,
"tag_name": "v0.8.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/checksums.txt"
]
},
"v0.8.1": {
"id": 62289050,
"tag_name": "v0.8.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/checksums.txt"
]
},
"v0.8.0": {
"id": 61423774,
"tag_name": "v0.8.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/checksums.txt"
]
},
"v0.8.0-rc1": {
"id": 60513568,
"tag_name": "v0.8.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/checksums.txt"
]
},
"v0.7.1": {
"id": 54098347,
"tag_name": "v0.7.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/checksums.txt"
]
},
"v0.7.0": {
"id": 53109422,
"tag_name": "v0.7.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/checksums.txt"
]
},
"v0.7.0-rc1": {
"id": 52726324,
"tag_name": "v0.7.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/checksums.txt"
]
},
"v0.6.3": {
"id": 48691641,
"tag_name": "v0.6.3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-arm64.exe"
]
},
"v0.6.2": {
"id": 48207405,
"tag_name": "v0.6.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-arm64.exe"
]
},
"v0.6.1": {
"id": 47064772,
"tag_name": "v0.6.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-arm64.exe"
]
},
"v0.6.0": {
"id": 46343260,
"tag_name": "v0.6.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-arm64.exe"
]
},
"v0.6.0-rc1": {
"id": 46230351,
"tag_name": "v0.6.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-arm64.exe"
]
},
"v0.5.1": {
"id": 35276550,
"tag_name": "v0.5.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.windows-amd64.exe"
]
},
"v0.5.0": {
"id": 35268960,
"tag_name": "v0.5.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.windows-amd64.exe"
]
},
"v0.5.0-rc1": {
"id": 35015334,
"tag_name": "v0.5.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.windows-amd64.exe"
]
},
"v0.4.2": {
"id": 30007794,
"tag_name": "v0.4.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.windows-amd64.exe"
]
},
"v0.4.1": {
"id": 26067509,
"tag_name": "v0.4.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.windows-amd64.exe"
]
},
"v0.4.0": {
"id": 26028174,
"tag_name": "v0.4.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.windows-amd64.exe"
]
},
"v0.3.1": {
"id": 20316235,
"tag_name": "v0.3.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.windows-amd64.exe"
]
},
"v0.3.0": {
"id": 19029664,
"tag_name": "v0.3.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.windows-amd64.exe"
]
},
"v0.2.2": {
"id": 17671545,
"tag_name": "v0.2.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.windows-amd64.exe"
]
},
"v0.2.1": {
"id": 17582885,
"tag_name": "v0.2.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.windows-amd64.exe"
]
},
"v0.2.0": {
"id": 16965310,
"tag_name": "v0.2.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.windows-amd64.exe"
]
}
}

View File

@@ -13,66 +13,147 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:latest"
REPO_SLUG: "docker/buildx-bin"
RELEASE_OUT: "./release-out"
DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test"
jobs:
test:
runs-on: ubuntu-latest
prepare-test:
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v4
with:
targets: integration-test-base
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.cache-to=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
test:
runs-on: ubuntu-22.04
needs:
- prepare-test
env:
TESTFLAGS: "-v --parallel=6 --timeout=30m"
TESTFLAGS_DOCKER: "-v --parallel=1 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose"
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildx-tests"
strategy:
fail-fast: false
matrix:
worker:
- docker
- docker\+containerd # same as docker, but with containerd snapshotter
- docker-container
- remote
pkg:
- ./tests
include:
- pkg: ./...
skip-integration-tests: 1
steps:
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build test image
uses: docker/bake-action@v4
with:
targets: integration-test
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
uses: docker/bake-action@v2
with:
targets: test
set: |
*.cache-from=type=gha,scope=test
*.cache-to=type=gha,scope=test
run: |
export TEST_REPORT_SUFFIX=-${{ github.job }}-$(echo "${{ matrix.pkg }}-${{ matrix.skip-integration-tests }}-${{ matrix.worker }}" | tr -dc '[:alnum:]-\n\r' | tr '[:upper:]' '[:lower:]')
./hack/test
env:
TEST_DOCKERD: "${{ startsWith(matrix.worker, 'docker') && '1' || '0' }}"
TESTFLAGS: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker\\+containerd') && env.TESTFLAGS_DOCKER || env.TESTFLAGS }} --run=//worker=${{ matrix.worker }}$"
TESTPKGS: "${{ matrix.pkg }}"
SKIP_INTEGRATION_TESTS: "${{ matrix.skip-integration-tests }}"
-
name: Upload coverage
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v3
with:
file: ./coverage/coverage.txt
directory: ./bin/testreports
-
name: Generate annotations
if: always()
uses: crazy-max/.github/.github/actions/gotest-annotations@1a64ea6d01db9a48aa61954cb20e265782c167d9
with:
directory: ./bin/testreports
-
name: Upload test reports
if: always()
uses: actions/upload-artifact@v3
with:
name: test-reports
path: ./bin/testreports
prepare:
runs-on: ubuntu-latest
prepare-binaries:
runs-on: ubuntu-22.04
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
run: |
echo ::set-output name=matrix::$(docker buildx bake binaries-cross --print | jq -cr '.target."binaries-cross".platforms')
echo "matrix=$(docker buildx bake binaries-cross --print | jq -cr '.target."binaries-cross".platforms')" >>${GITHUB_OUTPUT}
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
binaries:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
needs:
- prepare
- prepare-binaries
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.matrix) }}
platform: ${{ fromJson(needs.prepare-binaries.outputs.matrix) }}
steps:
-
name: Prepare
@@ -81,51 +162,56 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
uses: docker/bake-action@v2
with:
targets: release
set: |
*.platform=${{ matrix.platform }}
*.cache-from=type=gha,scope=binaries-${{ env.PLATFORM_PAIR }}
*.cache-to=type=gha,scope=binaries-${{ env.PLATFORM_PAIR }},mode=max
run: |
make release
env:
PLATFORMS: ${{ matrix.platform }}
CACHE_FROM: type=gha,scope=binaries-${{ env.PLATFORM_PAIR }}
CACHE_TO: type=gha,scope=binaries-${{ env.PLATFORM_PAIR }},mode=max
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: buildx
path: ${{ env.RELEASE_OUT }}/*
path: ${{ env.DESTDIR }}/*
if-no-files-found: error
bin-image:
runs-on: ubuntu-latest
if: ${{ github.event_name != 'pull_request' }}
runs-on: ubuntu-22.04
needs:
- test
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Docker meta
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REPO_SLUG }}
@@ -137,78 +223,80 @@ jobs:
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push image
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max
release:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
needs:
- test
- binaries
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Download binaries
uses: actions/download-artifact@v3
with:
name: buildx
path: ${{ env.RELEASE_OUT }}
path: ${{ env.DESTDIR }}
-
name: Create checksums
run: ./hack/hash-files
-
name: List artifacts
run: |
tree -nh ${{ env.RELEASE_OUT }}
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.RELEASE_OUT }} -type f -exec file -e ascii -- {} +
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: GitHub Release
if: startsWith(github.ref, 'refs/tags/v')
uses: softprops/action-gh-release@1e07f4398721186383de40550babbdf2b84acfc5
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v0.1.15
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
draft: true
files: ${{ env.RELEASE_OUT }}/*
files: ${{ env.DESTDIR }}/*
buildkit-edge:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
continue-on-error: true
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=moby/buildkit:master
buildkitd-flags: --debug
-
# Just run a bake target to check eveything runs fine
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: binaries

42
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: codeql
on:
push:
branches:
- 'master'
- 'v[0-9]*'
pull_request:
permissions:
actions: read
contents: read
security-events: write
env:
GO_VERSION: 1.21.3
jobs:
codeql:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v4
with:
go-version: ${{ env.GO_VERSION }}
-
name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: go
-
name: Autobuild
uses: github/codeql-action/autobuild@v2
-
name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:go"

View File

@@ -2,29 +2,31 @@ name: docs-release
on:
release:
types: [ released ]
types:
- released
jobs:
open-pr:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
if: ${{ github.event.release.prerelease != true && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout docs repo
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
repository: docker/docker.github.io
ref: master
repository: docker/docs
ref: main
-
name: Prepare
run: |
rm -rf ./_data/buildx/*
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
-
name: Build docs
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ github.event.release.name }}
targets: update-docs
@@ -42,7 +44,7 @@ jobs:
git add -A .
-
name: Create PR on docs repo
uses: peter-evans/create-pull-request@923ad837f191474af6b1721408744feb989a4c27 # v4.0.4
uses: peter-evans/create-pull-request@153407881ec5c347639a548ade7d8ad1d6740e38
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io

View File

@@ -22,19 +22,19 @@ on:
jobs:
docs-yaml:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-
name: Build reference YAML docs
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: update-docs
set: |
@@ -52,67 +52,11 @@ jobs:
retention-days: 1
validate:
runs-on: ubuntu-latest
uses: docker/docs/.github/workflows/validate-upstream.yml@main
needs:
- docs-yaml
steps:
-
name: Checkout
uses: actions/checkout@v3
with:
repository: docker/docker.github.io
-
name: Install js-yaml
run: npm install js-yaml
-
# use the actual buildx ref that triggers this workflow, so we make
# sure pages fetched by docs repo are still valid
# https://github.com/docker/docker.github.io/blob/98c7c9535063ae4cd2cd0a31478a21d16d2f07a3/_config.yml#L164-L173
name: Set correct ref to fetch remote resources
uses: actions/github-script@v6
with:
script: |
const fs = require('fs');
const yaml = require('js-yaml');
const configFile = '_config.yml'
const config = yaml.load(fs.readFileSync(configFile, 'utf8'));
for (const remote of config['fetch-remote']) {
if (remote['repo'] != 'https://github.com/docker/buildx') {
continue;
}
remote['ref'] = "${{ github.ref }}";
}
try {
fs.writeFileSync(configFile, yaml.dump(config), 'utf8')
} catch (err) {
console.error(err.message)
process.exit(1)
}
-
name: Prepare
run: |
# print docs jekyll config updated in previous step
yq _config.yml
# cleanup reference yaml docs and js-yaml module
rm -rf ./_data/buildx/* ./node_modules
-
name: Download built reference YAML docs
uses: actions/download-artifact@v3
with:
name: docs-yaml
path: ./_data/buildx/
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: latest
-
name: Validate
uses: docker/bake-action@v2
with:
targets: validate
set: |
*.cache-from=type=gha,scope=docs-upstream
*.cache-to=type=gha,scope=docs-upstream,mode=max
with:
module-name: docker/buildx
data-files-id: docs-yaml
data-files-folder: buildx
data-files-placeholder-folder: engine/reference/commandline

View File

@@ -11,26 +11,29 @@ on:
- 'master'
- 'v[0-9]*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
DESTDIR: "./bin"
K3S_VERSION: "v1.21.2-k3s1"
jobs:
build:
runs-on: ubuntu-20.04
env:
BIN_OUT: ./bin
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: binaries
set: |
@@ -40,13 +43,13 @@ jobs:
-
name: Rename binary
run: |
mv ${{ env.BIN_OUT }}/buildx ${{ env.BIN_OUT }}/docker-buildx
mv ${{ env.DESTDIR }}/build/buildx ${{ env.DESTDIR }}/build/docker-buildx
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: binary
path: ${{ env.BIN_OUT }}
path: ${{ env.DESTDIR }}/build
if-no-files-found: error
retention-days: 7
@@ -93,10 +96,10 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
if: matrix.driver == 'docker' || matrix.driver == 'docker-container'
-
name: Install buildx
@@ -129,20 +132,67 @@ jobs:
-
name: Install k3s
if: matrix.driver == 'kubernetes'
uses: debianmaster/actions-k3s@b9cf3f599fd118699a3c8a0d18a2f2bda6cf4ce4
id: k3s
uses: actions/github-script@v6
with:
version: v1.21.2-k3s1
script: |
const fs = require('fs');
let wait = function(milliseconds) {
return new Promise((resolve, reject) => {
if (typeof(milliseconds) !== 'number') {
throw new Error('milleseconds not a number');
}
setTimeout(() => resolve("done!"), milliseconds)
});
}
try {
const kubeconfig="/tmp/buildkit-k3s/kubeconfig.yaml";
core.info(`storing kubeconfig in ${kubeconfig}`);
await exec.exec('docker', ["run", "-d",
"--privileged",
"--name=buildkit-k3s",
"-e", "K3S_KUBECONFIG_OUTPUT="+kubeconfig,
"-e", "K3S_KUBECONFIG_MODE=666",
"-v", "/tmp/buildkit-k3s:/tmp/buildkit-k3s",
"-p", "6443:6443",
"-p", "80:80",
"-p", "443:443",
"-p", "8080:8080",
"rancher/k3s:${{ env.K3S_VERSION }}", "server"
]);
await wait(10000);
core.exportVariable('KUBECONFIG', kubeconfig);
let nodeName;
for (let count = 1; count <= 5; count++) {
try {
const nodeNameOutput = await exec.getExecOutput("kubectl get nodes --no-headers -oname");
nodeName = nodeNameOutput.stdout
} catch (error) {
core.info(`Unable to resolve node name (${error.message}). Attempt ${count} of 5.`)
} finally {
if (nodeName) {
break;
}
await wait(5000);
}
}
if (!nodeName) {
throw new Error(`Unable to resolve node name after 5 attempts.`);
}
await exec.exec(`kubectl wait --for=condition=Ready ${nodeName}`);
} catch (error) {
core.setFailed(error.message);
}
-
name: Config k3s
name: Print KUBECONFIG
if: matrix.driver == 'kubernetes'
run: |
(set -x ; cat ${{ steps.k3s.outputs.kubeconfig }})
-
name: Check k3s nodes
if: matrix.driver == 'kubernetes'
run: |
kubectl get nodes
yq ${{ env.KUBECONFIG }}
-
name: Launch remote buildkitd
if: matrix.driver == 'remote'

View File

@@ -13,13 +13,12 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
jobs:
validate:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
@@ -27,13 +26,14 @@ jobs:
- lint
- validate-vendor
- validate-docs
- validate-generated-files
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-

5
.gitignore vendored
View File

@@ -1,4 +1 @@
bin
coverage
cross-out
release-out
/bin

View File

@@ -11,30 +11,59 @@ linters:
enable:
- gofmt
- govet
- deadcode
- depguard
- goimports
- ineffassign
- misspell
- unused
- varcheck
- revive
- staticcheck
- typecheck
- structcheck
- nolintlint
- gosec
- forbidigo
disable-all: true
linters-settings:
depguard:
list-type: blacklist
include-go-root: true
packages:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- io/ioutil
rules:
main:
deny:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- pkg: "io/ioutil"
desc: The io/ioutil package has been deprecated.
forbidigo:
forbid:
- '^fmt\.Errorf(# use errors\.Errorf instead)?$'
gosec:
excludes:
- G204 # Audit use of command execution
- G402 # TLS MinVersion too low
config:
G306: "0644"
issues:
exclude-rules:
- linters:
- revive
text: "stutters"
- linters:
- revive
text: "empty-block"
- linters:
- revive
text: "superfluous-else"
- linters:
- revive
text: "unused-parameter"
- linters:
- revive
text: "redefines-builtin-id"
- linters:
- revive
text: "if-return"
# show all
max-issues-per-linter: 0
max-same-issues: 0

View File

@@ -1,10 +1,12 @@
# syntax=docker/dockerfile:1.4
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.18
ARG XX_VERSION=1.1.2
ARG DOCKERD_VERSION=20.10.14
ARG GO_VERSION=1.21.3
ARG XX_VERSION=1.2.1
FROM docker:$DOCKERD_VERSION AS dockerd-release
ARG DOCKER_VERSION=24.0.6
ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.11.6
# xx is a helper for cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
@@ -18,23 +20,55 @@ ENV GOFLAGS=-mod=vendor
ENV CGO_ENABLED=0
WORKDIR /src
FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM gobase AS docker
ARG TARGETPLATFORM
ARG DOCKER_VERSION
WORKDIR /opt/docker
RUN DOCKER_ARCH=$(case ${TARGETPLATFORM:-linux/amd64} in \
"linux/amd64") echo "x86_64" ;; \
"linux/arm/v6") echo "armel" ;; \
"linux/arm/v7") echo "armhf" ;; \
"linux/arm64") echo "aarch64" ;; \
"linux/ppc64le") echo "ppc64le" ;; \
"linux/s390x") echo "s390x" ;; \
*) echo "" ;; esac) \
&& echo "DOCKER_ARCH=$DOCKER_ARCH" \
&& wget -qO- "https://download.docker.com/linux/static/stable/${DOCKER_ARCH}/docker-${DOCKER_VERSION}.tgz" | tar xvz --strip 1
RUN ./dockerd --version && ./containerd --version && ./ctr --version && ./runc --version
FROM gobase AS gotestsum
ARG GOTESTSUM_VERSION
ENV GOFLAGS=
RUN --mount=target=/root/.cache,type=cache \
GOBIN=/out/ go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" && \
/out/gotestsum --version
FROM gobase AS buildx-version
RUN --mount=target=. \
PKG=github.com/docker/buildx VERSION=$(git describe --match 'v[0-9]*' --dirty='.m' --always --tags) REVISION=$(git rev-parse HEAD)$(if ! git diff --no-ext-diff --quiet --exit-code; then echo .m; fi); \
echo "-X ${PKG}/version.Version=${VERSION} -X ${PKG}/version.Revision=${REVISION} -X ${PKG}/version.Package=${PKG}" | tee /tmp/.ldflags; \
echo -n "${VERSION}" | tee /tmp/.version;
RUN --mount=type=bind,target=. <<EOT
set -e
mkdir /buildx-version
echo -n "$(./hack/git-meta version)" | tee /buildx-version/version
echo -n "$(./hack/git-meta revision)" | tee /buildx-version/revision
EOT
FROM gobase AS buildx-build
ARG LDFLAGS="-w -s"
ARG TARGETPLATFORM
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,source=/tmp/.ldflags,target=/tmp/.ldflags,from=buildx-version \
set -x; xx-go build -ldflags "$(cat /tmp/.ldflags) ${LDFLAGS}" -o /usr/bin/buildx ./cmd/buildx && \
xx-verify --static /usr/bin/buildx
--mount=type=bind,from=buildx-version,source=/buildx-version,target=/buildx-version <<EOT
set -e
xx-go --wrap
DESTDIR=/usr/bin VERSION=$(cat /buildx-version/version) REVISION=$(cat /buildx-version/revision) GO_EXTRA_LDFLAGS="-s -w" ./hack/build
xx-verify --static /usr/bin/docker-buildx
EOT
FROM gobase AS test
ENV SKIP_INTEGRATION_TESTS=1
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
@@ -45,29 +79,56 @@ FROM scratch AS test-coverage
COPY --from=test /tmp/coverage.txt /coverage.txt
FROM scratch AS binaries-unix
COPY --link --from=buildx-build /usr/bin/buildx /
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx
FROM binaries-unix AS binaries-darwin
FROM binaries-unix AS binaries-linux
FROM scratch AS binaries-windows
COPY --link --from=buildx-build /usr/bin/buildx /buildx.exe
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx.exe
FROM binaries-$TARGETOS AS binaries
# enable scanning for this stage
ARG BUILDKIT_SBOM_SCAN_STAGE=true
FROM gobase AS integration-test-base
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN apk add --no-cache \
btrfs-progs \
e2fsprogs \
e2fsprogs-extra \
ip6tables \
iptables \
openssl \
shadow-uidmap \
xfsprogs \
xz
COPY --link --from=gotestsum /out/gotestsum /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=docker /opt/docker/* /usr/bin/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/
FROM integration-test-base AS integration-test
COPY . .
# Release
FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work
ARG TARGETPLATFORM
RUN --mount=from=binaries \
--mount=type=bind,source=/tmp/.version,target=/tmp/.version,from=buildx-version \
mkdir -p /out && cp buildx* "/out/buildx-$(cat /tmp/.version).$(echo $TARGETPLATFORM | sed 's/\//-/g')$(ls buildx* | sed -e 's/^buildx//')"
--mount=type=bind,from=buildx-version,source=/buildx-version,target=/buildx-version <<EOT
set -e
mkdir -p /out
cp buildx* "/out/buildx-$(cat /buildx-version/version).$(echo $TARGETPLATFORM | sed 's/\//-/g')$(ls buildx* | sed -e 's/^buildx//')"
EOT
FROM scratch AS release
COPY --from=releaser /out/ /
# Shell
FROM docker:$DOCKERD_VERSION AS dockerd-release
FROM docker:$DOCKER_VERSION AS dockerd-release
FROM alpine AS shell
RUN apk add --no-cache iptables tmux git vim less openssh
RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx

View File

@@ -4,59 +4,93 @@ else ifneq (, $(shell docker buildx version))
export BUILDX_CMD = docker buildx
else ifneq (, $(shell which buildx))
export BUILDX_CMD = $(which buildx)
else
$(error "Buildx is required: https://github.com/docker/buildx#installing")
endif
export BIN_OUT = ./bin
export RELEASE_OUT = ./release-out
export BUILDX_CMD ?= docker buildx
.PHONY: all
all: binaries
.PHONY: build
build:
./hack/build
.PHONY: shell
shell:
./hack/shell
.PHONY: binaries
binaries:
$(BUILDX_CMD) bake binaries
.PHONY: binaries-cross
binaries-cross:
$(BUILDX_CMD) bake binaries-cross
.PHONY: install
install: binaries
mkdir -p ~/.docker/cli-plugins
install bin/buildx ~/.docker/cli-plugins/docker-buildx
install bin/build/buildx ~/.docker/cli-plugins/docker-buildx
.PHONY: release
release:
./hack/release
validate-all: lint test validate-vendor validate-docs
.PHONY: validate-all
validate-all: lint test validate-vendor validate-docs validate-generated-files
.PHONY: lint
lint:
$(BUILDX_CMD) bake lint
.PHONY: test
test:
$(BUILDX_CMD) bake test
./hack/test
.PHONY: test-unit
test-unit:
TESTPKGS=./... SKIP_INTEGRATION_TESTS=1 ./hack/test
.PHONY: test
test-integration:
TESTPKGS=./tests ./hack/test
.PHONY: validate-vendor
validate-vendor:
$(BUILDX_CMD) bake validate-vendor
.PHONY: validate-docs
validate-docs:
$(BUILDX_CMD) bake validate-docs
.PHONY: validate-authors
validate-authors:
$(BUILDX_CMD) bake validate-authors
.PHONY: validate-generated-files
validate-generated-files:
$(BUILDX_CMD) bake validate-generated-files
.PHONY: test-driver
test-driver:
./hack/test-driver
.PHONY: vendor
vendor:
./hack/update-vendor
.PHONY: docs
docs:
./hack/update-docs
.PHONY: authors
authors:
$(BUILDX_CMD) bake update-authors
.PHONY: mod-outdated
mod-outdated:
$(BUILDX_CMD) bake mod-outdated
.PHONY: shell binaries binaries-cross install release validate-all lint validate-vendor validate-docs validate-authors vendor docs authors
.PHONY: generated-files
generated-files:
$(BUILDX_CMD) bake update-generated-files

View File

@@ -2,7 +2,7 @@
[![GitHub release](https://img.shields.io/github/release/docker/buildx.svg?style=flat-square)](https://github.com/docker/buildx/releases/latest)
[![PkgGoDev](https://img.shields.io/badge/go.dev-docs-007d9c?style=flat-square&logo=go&logoColor=white)](https://pkg.go.dev/github.com/docker/buildx)
[![Build Status](https://img.shields.io/github/workflow/status/docker/buildx/build?label=build&logo=github&style=flat-square)](https://github.com/docker/buildx/actions?query=workflow%3Abuild)
[![Build Status](https://img.shields.io/github/actions/workflow/status/docker/buildx/build.yml?branch=master&label=build&logo=github&style=flat-square)](https://github.com/docker/buildx/actions?query=workflow%3Abuild)
[![Go Report Card](https://goreportcard.com/badge/github.com/docker/buildx?style=flat-square)](https://goreportcard.com/report/github.com/docker/buildx)
[![codecov](https://img.shields.io/codecov/c/github/docker/buildx?logo=codecov&style=flat-square)](https://codecov.io/gh/docker/buildx)
@@ -32,16 +32,6 @@ Key features:
- [Building with buildx](#building-with-buildx)
- [Working with builder instances](#working-with-builder-instances)
- [Building multi-platform images](#building-multi-platform-images)
- [Guides](docs/guides)
- [High-level build options with Bake](docs/guides/bake/index.md)
- [CI/CD](docs/guides/cicd.md)
- [CNI networking](docs/guides/cni-networking.md)
- [Using a custom network](docs/guides/custom-network.md)
- [Using a custom registry configuration](docs/guides/custom-registry-config.md)
- [OpenTelemetry support](docs/guides/opentelemetry.md)
- [Registry mirror](docs/guides/registry-mirror.md)
- [Drivers](docs/guides/drivers/index.md)
- [Resource limiting](docs/guides/resource-limiting.md)
- [Reference](docs/reference/buildx.md)
- [`buildx bake`](docs/reference/buildx_bake.md)
- [`buildx build`](docs/reference/buildx_build.md)
@@ -51,21 +41,26 @@ Key features:
- [`buildx imagetools create`](docs/reference/buildx_imagetools_create.md)
- [`buildx imagetools inspect`](docs/reference/buildx_imagetools_inspect.md)
- [`buildx inspect`](docs/reference/buildx_inspect.md)
- [`buildx install`](docs/reference/buildx_install.md)
- [`buildx ls`](docs/reference/buildx_ls.md)
- [`buildx prune`](docs/reference/buildx_prune.md)
- [`buildx rm`](docs/reference/buildx_rm.md)
- [`buildx stop`](docs/reference/buildx_stop.md)
- [`buildx uninstall`](docs/reference/buildx_uninstall.md)
- [`buildx use`](docs/reference/buildx_use.md)
- [`buildx version`](docs/reference/buildx_version.md)
- [Contributing](#contributing)
For more information on how to use Buildx, see
[Docker Build docs](https://docs.docker.com/build/).
# Installing
Using `buildx` as a docker CLI plugin requires using Docker 19.03 or newer.
A limited set of functionality works with older versions of Docker when
invoking the binary directly.
Using `buildx` with Docker requires Docker engine 19.03 or newer.
> **Warning**
>
> Using an incompatible version of Docker may result in unexpected behavior,
> and will likely cause issues, especially when using Buildx builders with more
> recent versions of BuildKit.
## Windows and macOS
@@ -74,8 +69,9 @@ for Windows and macOS.
## Linux packages
Docker Linux packages also include Docker Buildx when installed using the
[DEB or RPM packages](https://docs.docker.com/engine/install/).
Docker Engine package repositories contain Docker Buildx packages when installed according to the
[Docker Engine install documentation](https://docs.docker.com/engine/install/). Install the
`docker-buildx-plugin` package to install the Buildx plugin.
## Manual download
@@ -123,7 +119,8 @@ On Windows:
Here is how to install and use Buildx inside a Dockerfile through the
[`docker/buildx-bin`](https://hub.docker.com/r/docker/buildx-bin) image:
```Dockerfile
```dockerfile
# syntax=docker/dockerfile:1
FROM docker
COPY --from=docker/buildx-bin /buildx /usr/libexec/docker/cli-plugins/docker-buildx
RUN docker buildx version
@@ -143,14 +140,14 @@ To remove this alias, run [`docker buildx uninstall`](docs/reference/buildx_unin
# Buildx 0.6+
$ docker buildx bake "https://github.com/docker/buildx.git"
$ mkdir -p ~/.docker/cli-plugins
$ mv ./bin/buildx ~/.docker/cli-plugins/docker-buildx
$ mv ./bin/build/buildx ~/.docker/cli-plugins/docker-buildx
# Docker 19.03+
$ DOCKER_BUILDKIT=1 docker build --platform=local -o . "https://github.com/docker/buildx.git"
$ mkdir -p ~/.docker/cli-plugins
$ mv buildx ~/.docker/cli-plugins/docker-buildx
# Local
# Local
$ git clone https://github.com/docker/buildx.git && cd buildx
$ make install
```
@@ -190,12 +187,12 @@ through various "drivers". Each driver defines how and where a build should
run, and have different feature sets.
We currently support the following drivers:
- The `docker` driver ([guide](docs/guides/drivers/docker.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `docker-container` driver ([guide](docs/guides/drivers/docker-container.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `kubernetes` driver ([guide](docs/guides/drivers/kubernetes.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `remote` driver ([guide](docs/guides/drivers/remote.md))
- The `docker` driver ([guide](docs/manuals/drivers/docker.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `docker-container` driver ([guide](docs/manuals/drivers/docker-container.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `kubernetes` driver ([guide](docs/manuals/drivers/kubernetes.md), [reference](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver))
- The `remote` driver ([guide](docs/manuals/drivers/remote.md))
For more information on drivers, see the [drivers guide](docs/guides/drivers/index.md).
For more information on drivers, see the [drivers guide](docs/manuals/drivers/index.md).
## Working with builder instances
@@ -242,7 +239,7 @@ When you invoke a build, you can set the `--platform` flag to specify the target
platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or
`darwin/amd64`).
When the current builder instance is backed by the `docker-container` or
When the current builder instance is backed by the `docker-container` or
`kubernetes` driver, you can specify multiple platforms together. In this case,
it builds a manifest list which contains images for all specified architectures.
When you use this image in [`docker run`](https://docs.docker.com/engine/reference/commandline/run/)
@@ -298,6 +295,7 @@ inside your Dockerfile and can be leveraged by the processes running as part
of your build.
```dockerfile
# syntax=docker/dockerfile:1
FROM --platform=$BUILDPLATFORM golang:alpine AS build
ARG TARGETPLATFORM
ARG BUILDPLATFORM
@@ -311,7 +309,7 @@ cross-compilation helpers for more advanced use-cases.
## High-level build options
See [`docs/guides/bake/index.md`](docs/guides/bake/index.md) for more details.
See [High-level builds with Bake](https://docs.docker.com/build/bake/) for more details.
# Contributing

View File

@@ -3,7 +3,6 @@ package bake
import (
"context"
"encoding/csv"
"fmt"
"io"
"os"
"path"
@@ -12,23 +11,26 @@ import (
"sort"
"strconv"
"strings"
"time"
composecli "github.com/compose-spec/compose-go/cli"
"github.com/docker/buildx/bake/hclparser"
"github.com/docker/buildx/build"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/config"
"github.com/docker/docker/builder/remotecontext/urlutil"
hcl "github.com/hashicorp/hcl/v2"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
)
var (
httpPrefix = regexp.MustCompile(`^https?://`)
gitURLPathWithFragmentSuffix = regexp.MustCompile(`\.git(?:#.+)?$`)
validTargetNameChars = `[a-zA-Z0-9_-]+`
targetNamePattern = regexp.MustCompile(`^` + validTargetNameChars + `$`)
)
@@ -44,17 +46,18 @@ type Override struct {
}
func defaultFilenames() []string {
return []string{
"docker-compose.yml", // support app
"docker-compose.yaml", // support app
names := []string{}
names = append(names, composecli.DefaultFileNames...)
names = append(names, []string{
"docker-bake.json",
"docker-bake.override.json",
"docker-bake.hcl",
"docker-bake.override.hcl",
}
}...)
return names
}
func ReadLocalFiles(names []string) ([]File, error) {
func ReadLocalFiles(names []string, stdin io.Reader, l progress.SubLogger) ([]File, error) {
isDefault := false
if len(names) == 0 {
isDefault = true
@@ -62,20 +65,26 @@ func ReadLocalFiles(names []string) ([]File, error) {
}
out := make([]File, 0, len(names))
setStatus := func(st *client.VertexStatus) {
if l != nil {
l.SetStatus(st)
}
}
for _, n := range names {
var dt []byte
var err error
if n == "-" {
dt, err = io.ReadAll(os.Stdin)
dt, err = readWithProgress(stdin, setStatus)
if err != nil {
return nil, err
}
} else {
dt, err = os.ReadFile(n)
dt, err = readFileWithProgress(n, isDefault, setStatus)
if dt == nil && err == nil {
continue
}
if err != nil {
if isDefault && errors.Is(err, os.ErrNotExist) {
continue
}
return nil, err
}
}
@@ -84,7 +93,104 @@ func ReadLocalFiles(names []string) ([]File, error) {
return out, nil
}
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, []*Group, error) {
func readFileWithProgress(fname string, isDefault bool, setStatus func(st *client.VertexStatus)) (dt []byte, err error) {
st := &client.VertexStatus{
ID: "reading " + fname,
}
defer func() {
now := time.Now()
st.Completed = &now
if dt != nil || err != nil {
setStatus(st)
}
}()
now := time.Now()
st.Started = &now
f, err := os.Open(fname)
if err != nil {
if isDefault && errors.Is(err, os.ErrNotExist) {
return nil, nil
}
return nil, err
}
defer f.Close()
setStatus(st)
info, err := f.Stat()
if err != nil {
return nil, err
}
st.Total = info.Size()
setStatus(st)
buf := make([]byte, 1024)
for {
n, err := f.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
dt = append(dt, buf[:n]...)
st.Current += int64(n)
setStatus(st)
}
return dt, nil
}
func readWithProgress(r io.Reader, setStatus func(st *client.VertexStatus)) (dt []byte, err error) {
st := &client.VertexStatus{
ID: "reading from stdin",
}
defer func() {
now := time.Now()
st.Completed = &now
setStatus(st)
}()
now := time.Now()
st.Started = &now
setStatus(st)
buf := make([]byte, 1024)
for {
n, err := r.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
dt = append(dt, buf[:n]...)
st.Current += int64(n)
setStatus(st)
}
return dt, nil
}
func ListTargets(files []File) ([]string, error) {
c, err := ParseFiles(files, nil)
if err != nil {
return nil, err
}
var targets []string
for _, g := range c.Groups {
targets = append(targets, g.Name)
}
for _, t := range c.Targets {
targets = append(targets, t.Name)
}
return dedupSlice(targets), nil
}
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, map[string]*Group, error) {
c, err := ParseFiles(files, defaults)
if err != nil {
return nil, nil, err
@@ -99,42 +205,39 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
return nil, nil, err
}
m := map[string]*Target{}
for _, n := range targets {
for _, n := range c.ResolveGroup(n) {
t, err := c.ResolveTarget(n, o)
n := map[string]*Group{}
for _, target := range targets {
ts, gs := c.ResolveGroup(target)
for _, tname := range ts {
t, err := c.ResolveTarget(tname, o)
if err != nil {
return nil, nil, err
}
if t != nil {
m[n] = t
m[tname] = t
}
}
for _, gname := range gs {
for _, group := range c.Groups {
if group.Name == gname {
n[gname] = group
break
}
}
}
}
var g []*Group
if len(targets) == 0 || (len(targets) == 1 && targets[0] == "default") {
for _, group := range c.Groups {
if group.Name != "default" {
continue
}
g = []*Group{{Targets: group.Targets}}
for _, target := range targets {
if target == "default" {
continue
}
} else {
var gt []string
for _, target := range targets {
isGroup := false
for _, group := range c.Groups {
if target == group.Name {
gt = append(gt, group.Targets...)
isGroup = true
break
}
}
if !isGroup {
gt = append(gt, target)
}
if _, ok := n["default"]; !ok {
n["default"] = &Group{Name: "default"}
}
g = []*Group{{Targets: dedupSlice(gt)}}
n["default"].Targets = append(n["default"].Targets, target)
}
if g, ok := n["default"]; ok {
g.Targets = dedupSlice(g.Targets)
}
for name, t := range m {
@@ -143,7 +246,7 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
}
}
return m, g, nil
return m, n, nil
}
func dedupSlice(s []string) []string {
@@ -218,7 +321,7 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
hclFiles = append(hclFiles, hf)
} else if composeErr != nil {
return nil, fmt.Errorf("failed to parse %s: parsing yaml: %v, parsing hcl: %w", f.Name, composeErr, err)
return nil, errors.Wrapf(err, "failed to parse %s: parsing yaml: %v, parsing hcl", f.Name, composeErr)
} else {
return nil, err
}
@@ -235,13 +338,28 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
if len(hclFiles) > 0 {
if err := hclparser.Parse(hcl.MergeFiles(hclFiles), hclparser.Opt{
renamed, err := hclparser.Parse(hclparser.MergeFiles(hclFiles), hclparser.Opt{
LookupVar: os.LookupEnv,
Vars: defaults,
ValidateLabel: validateTargetName,
}, &c); err.HasErrors() {
}, &c)
if err.HasErrors() {
return nil, err
}
for _, renamed := range renamed {
for oldName, newNames := range renamed {
newNames = dedupSlice(newNames)
if len(newNames) == 1 && oldName == newNames[0] {
continue
}
c.Groups = append(c.Groups, &Group{
Name: oldName,
Targets: newNames,
})
}
}
c = dedupeConfig(c)
}
return &c, nil
@@ -273,8 +391,8 @@ func ParseFile(dt []byte, fn string) (*Config, error) {
}
type Config struct {
Groups []*Group `json:"group" hcl:"group,block"`
Targets []*Target `json:"target" hcl:"target,block"`
Groups []*Group `json:"group" hcl:"group,block" cty:"group"`
Targets []*Target `json:"target" hcl:"target,block" cty:"target"`
}
func mergeConfig(c1, c2 Config) Config {
@@ -420,7 +538,7 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
o := t[kk[1]]
switch keys[1] {
case "output", "cache-to", "cache-from", "tags", "platform", "secrets", "ssh":
case "output", "cache-to", "cache-from", "tags", "platform", "secrets", "ssh", "attest":
if len(parts) == 2 {
o.ArrValue = append(o.ArrValue, parts[1])
}
@@ -453,13 +571,19 @@ func (c Config) newOverrides(v []string) (map[string]map[string]Override, error)
return m, nil
}
func (c Config) ResolveGroup(name string) []string {
return dedupSlice(c.group(name, map[string][]string{}))
func (c Config) ResolveGroup(name string) ([]string, []string) {
targets, groups := c.group(name, map[string]visit{})
return dedupSlice(targets), dedupSlice(groups)
}
func (c Config) group(name string, visited map[string][]string) []string {
if _, ok := visited[name]; ok {
return visited[name]
type visit struct {
target []string
group []string
}
func (c Config) group(name string, visited map[string]visit) ([]string, []string) {
if v, ok := visited[name]; ok {
return v.target, v.group
}
var g *Group
for _, group := range c.Groups {
@@ -469,20 +593,24 @@ func (c Config) group(name string, visited map[string][]string) []string {
}
}
if g == nil {
return []string{name}
return []string{name}, nil
}
visited[name] = []string{}
visited[name] = visit{}
targets := make([]string, 0, len(g.Targets))
groups := []string{name}
for _, t := range g.Targets {
tgroup := c.group(t, visited)
if len(tgroup) > 0 {
targets = append(targets, tgroup...)
ttarget, tgroup := c.group(t, visited)
if len(ttarget) > 0 {
targets = append(targets, ttarget...)
} else {
targets = append(targets, t)
}
if len(tgroup) > 0 {
groups = append(groups, tgroup...)
}
}
visited[name] = targets
return targets
visited[name] = visit{target: targets, group: groups}
return targets, groups
}
func (c Config) ResolveTarget(name string, overrides map[string]map[string]Override) (*Target, error) {
@@ -540,42 +668,51 @@ func (c Config) target(name string, visited map[string]*Target, overrides map[st
}
type Group struct {
Name string `json:"-" hcl:"name,label"`
Targets []string `json:"targets" hcl:"targets"`
Name string `json:"-" hcl:"name,label" cty:"name"`
Targets []string `json:"targets" hcl:"targets" cty:"targets"`
// Target // TODO?
}
type Target struct {
Name string `json:"-" hcl:"name,label"`
Name string `json:"-" hcl:"name,label" cty:"name"`
// Inherits is the only field that cannot be overridden with --set
Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional"`
Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional" cty:"inherits"`
Context *string `json:"context,omitempty" hcl:"context,optional"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional"`
DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional"`
Args map[string]string `json:"args,omitempty" hcl:"args,optional"`
Labels map[string]string `json:"labels,omitempty" hcl:"labels,optional"`
Tags []string `json:"tags,omitempty" hcl:"tags,optional"`
CacheFrom []string `json:"cache-from,omitempty" hcl:"cache-from,optional"`
CacheTo []string `json:"cache-to,omitempty" hcl:"cache-to,optional"`
Target *string `json:"target,omitempty" hcl:"target,optional"`
Secrets []string `json:"secret,omitempty" hcl:"secret,optional"`
SSH []string `json:"ssh,omitempty" hcl:"ssh,optional"`
Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional"`
Outputs []string `json:"output,omitempty" hcl:"output,optional"`
Pull *bool `json:"pull,omitempty" hcl:"pull,optional"`
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional"`
NetworkMode *string `json:"-" hcl:"-"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/guides/bake/file-definition.md.
Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"`
Attest []string `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"`
DockerfileInline *string `json:"dockerfile-inline,omitempty" hcl:"dockerfile-inline,optional" cty:"dockerfile-inline"`
Args map[string]*string `json:"args,omitempty" hcl:"args,optional" cty:"args"`
Labels map[string]*string `json:"labels,omitempty" hcl:"labels,optional" cty:"labels"`
Tags []string `json:"tags,omitempty" hcl:"tags,optional" cty:"tags"`
CacheFrom []string `json:"cache-from,omitempty" hcl:"cache-from,optional" cty:"cache-from"`
CacheTo []string `json:"cache-to,omitempty" hcl:"cache-to,optional" cty:"cache-to"`
Target *string `json:"target,omitempty" hcl:"target,optional" cty:"target"`
Secrets []string `json:"secret,omitempty" hcl:"secret,optional" cty:"secret"`
SSH []string `json:"ssh,omitempty" hcl:"ssh,optional" cty:"ssh"`
Platforms []string `json:"platforms,omitempty" hcl:"platforms,optional" cty:"platforms"`
Outputs []string `json:"output,omitempty" hcl:"output,optional" cty:"output"`
Pull *bool `json:"pull,omitempty" hcl:"pull,optional" cty:"pull"`
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"-" hcl:"-" cty:"-"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/bake-reference.md.
// linked is a private field to mark a target used as a linked one
linked bool
}
var _ hclparser.WithEvalContexts = &Target{}
var _ hclparser.WithGetName = &Target{}
var _ hclparser.WithEvalContexts = &Group{}
var _ hclparser.WithGetName = &Group{}
func (t *Target) normalize() {
t.Annotations = removeDupes(t.Annotations)
t.Attest = removeAttestDupes(t.Attest)
t.Tags = removeDupes(t.Tags)
t.Secrets = removeDupes(t.Secrets)
t.SSH = removeDupes(t.SSH)
@@ -606,8 +743,11 @@ func (t *Target) Merge(t2 *Target) {
t.DockerfileInline = t2.DockerfileInline
}
for k, v := range t2.Args {
if v == nil {
continue
}
if t.Args == nil {
t.Args = map[string]string{}
t.Args = map[string]*string{}
}
t.Args[k] = v
}
@@ -618,8 +758,11 @@ func (t *Target) Merge(t2 *Target) {
t.Contexts[k] = v
}
for k, v := range t2.Labels {
if v == nil {
continue
}
if t.Labels == nil {
t.Labels = map[string]string{}
t.Labels = map[string]*string{}
}
t.Labels[k] = v
}
@@ -629,6 +772,13 @@ func (t *Target) Merge(t2 *Target) {
if t2.Target != nil {
t.Target = t2.Target
}
if t2.Annotations != nil { // merge
t.Annotations = append(t.Annotations, t2.Annotations...)
}
if t2.Attest != nil { // merge
t.Attest = append(t.Attest, t2.Attest...)
t.Attest = removeAttestDupes(t.Attest)
}
if t2.Secrets != nil { // merge
t.Secrets = append(t.Secrets, t2.Secrets...)
}
@@ -676,9 +826,9 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
return errors.Errorf("args require name")
}
if t.Args == nil {
t.Args = map[string]string{}
t.Args = map[string]*string{}
}
t.Args[keys[1]] = value
t.Args[keys[1]] = &value
case "contexts":
if len(keys) != 2 {
return errors.Errorf("contexts require name")
@@ -692,9 +842,9 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
return errors.Errorf("labels require name")
}
if t.Labels == nil {
t.Labels = map[string]string{}
t.Labels = map[string]*string{}
}
t.Labels[keys[1]] = value
t.Labels[keys[1]] = &value
case "tags":
t.Tags = o.ArrValue
case "cache-from":
@@ -711,6 +861,10 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
t.Platforms = o.ArrValue
case "output":
t.Outputs = o.ArrValue
case "annotations":
t.Annotations = append(t.Annotations, o.ArrValue...)
case "attest":
t.Attest = append(t.Attest, o.ArrValue...)
case "no-cache":
noCache, err := strconv.ParseBool(value)
if err != nil {
@@ -746,6 +900,116 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
return nil
}
func (g *Group) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
if _, ok := content.Attributes["matrix"]; ok {
return nil, errors.Errorf("matrix is not supported for groups")
}
return []*hcl.EvalContext{ectx}, nil
}
func (t *Target) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
attr, ok := content.Attributes["matrix"]
if !ok {
return []*hcl.EvalContext{ectx}, nil
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return nil, diags
}
value, err := attr.Expr.Value(ectx)
if err != nil {
return nil, err
}
if !value.Type().IsMapType() && !value.Type().IsObjectType() {
return nil, errors.Errorf("matrix must be a map")
}
matrix := value.AsValueMap()
ectxs := []*hcl.EvalContext{ectx}
for k, expr := range matrix {
if !expr.CanIterateElements() {
return nil, errors.Errorf("matrix values must be a list")
}
ectxs2 := []*hcl.EvalContext{}
for _, v := range expr.AsValueSlice() {
for _, e := range ectxs {
e2 := ectx.NewChild()
e2.Variables = make(map[string]cty.Value)
if e != ectx {
for k, v := range e.Variables {
e2.Variables[k] = v
}
}
e2.Variables[k] = v
ectxs2 = append(ectxs2, e2)
}
}
ectxs = ectxs2
}
return ectxs, nil
}
func (g *Group) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
if _, ok := content.Attributes["name"]; ok {
return "", errors.Errorf("name is not supported for groups")
}
if _, ok := content.Attributes["matrix"]; ok {
return "", errors.Errorf("matrix is not supported for groups")
}
return block.Labels[0], nil
}
func (t *Target) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
attr, ok := content.Attributes["name"]
if !ok {
return block.Labels[0], nil
}
if _, ok := content.Attributes["matrix"]; !ok {
return "", errors.Errorf("name requires matrix")
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return "", diags
}
value, diags := attr.Expr.Value(ectx)
if diags != nil {
return "", diags
}
value, err := convert.Convert(value, cty.String)
if err != nil {
return "", err
}
return value.AsString(), nil
}
func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) {
m2 := make(map[string]build.Options, len(m))
for k, v := range m {
@@ -770,7 +1034,7 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(v.Path, "cwd://") || strings.HasPrefix(v.Path, "target:") || strings.HasPrefix(v.Path, "docker-image:") {
continue
}
if IsRemoteURL(v.Path) {
if build.IsRemoteURL(v.Path) {
continue
}
st := llb.Scratch().File(llb.Copy(*inp.State, v.Path, "/"), llb.WithCustomNamef("set context %s to %s", k, v.Path))
@@ -784,10 +1048,15 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(t.ContextPath, "cwd://") {
return
}
if IsRemoteURL(t.ContextPath) {
if build.IsRemoteURL(t.ContextPath) {
return
}
st := llb.Scratch().File(llb.Copy(*inp.State, t.ContextPath, "/"), llb.WithCustomNamef("set context to %s", t.ContextPath))
st := llb.Scratch().File(
llb.Copy(*inp.State, t.ContextPath, "/", &llb.CopyInfo{
CopyDirContentsOnly: true,
}),
llb.WithCustomNamef("set context to %s", t.ContextPath),
)
t.ContextState = &st
}
@@ -820,7 +1089,7 @@ func validateContextsEntitlements(t build.Inputs, inp *Input) error {
}
func checkPath(p string) error {
if IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
return nil
}
p, err := filepath.EvalSymlinks(p)
@@ -830,6 +1099,10 @@ func checkPath(p string) error {
}
return err
}
p, err = filepath.Abs(p)
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return err
@@ -838,7 +1111,8 @@ func checkPath(p string) error {
if err != nil {
return err
}
if strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
parts := strings.Split(rel, string(os.PathSeparator))
if parts[0] == ".." {
return errors.Errorf("path %s is outside of the working directory, please set BAKE_ALLOW_REMOTE_FS_ACCESS=1", p)
}
return nil
@@ -856,16 +1130,90 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if t.Context != nil {
contextPath = *t.Context
}
if !strings.HasPrefix(contextPath, "cwd://") && !IsRemoteURL(contextPath) {
if !strings.HasPrefix(contextPath, "cwd://") && !build.IsRemoteURL(contextPath) {
contextPath = path.Clean(contextPath)
}
dockerfilePath := "Dockerfile"
if t.Dockerfile != nil {
dockerfilePath = *t.Dockerfile
}
if !strings.HasPrefix(dockerfilePath, "cwd://") {
dockerfilePath = path.Clean(dockerfilePath)
}
if !isRemoteResource(contextPath) && !path.IsAbs(dockerfilePath) {
dockerfilePath = path.Join(contextPath, dockerfilePath)
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if strings.HasPrefix(bi.DockerfilePath, "cwd://") {
// If Dockerfile is local for a remote invocation, we first check if
// it's not outside the working directory and then resolve it to an
// absolute path.
bi.DockerfilePath = path.Clean(strings.TrimPrefix(bi.DockerfilePath, "cwd://"))
if err := checkPath(bi.DockerfilePath); err != nil {
return nil, err
}
var err error
bi.DockerfilePath, err = filepath.Abs(bi.DockerfilePath)
if err != nil {
return nil, err
}
} else if !build.IsRemoteURL(bi.DockerfilePath) && strings.HasPrefix(bi.ContextPath, "cwd://") && (inp != nil && build.IsRemoteURL(inp.URL)) {
// We don't currently support reading a remote Dockerfile with a local
// context when doing a remote invocation because we automatically
// derive the dockerfile from the context atm:
//
// target "default" {
// context = BAKE_CMD_CONTEXT
// dockerfile = "Dockerfile.app"
// }
//
// > docker buildx bake https://github.com/foo/bar.git
// failed to solve: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount3004544897/Dockerfile.app: no such file or directory
//
// To avoid mistakenly reading a local Dockerfile, we check if the
// Dockerfile exists locally and if so, we error out.
if _, err := os.Stat(filepath.Join(path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://")), bi.DockerfilePath)); err == nil {
return nil, errors.Errorf("reading a dockerfile for a remote build invocation is currently not supported")
}
}
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
if !build.IsRemoteURL(bi.ContextPath) && bi.ContextState == nil && !path.IsAbs(bi.DockerfilePath) {
bi.DockerfilePath = path.Join(bi.ContextPath, bi.DockerfilePath)
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
args := map[string]string{}
for k, v := range t.Args {
if v == nil {
continue
}
args[k] = *v
}
labels := map[string]string{}
for k, v := range t.Labels {
if v == nil {
continue
}
labels[k] = *v
}
noCache := false
@@ -881,35 +1229,11 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
networkMode = *t.NetworkMode
}
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
bo := &build.Options{
Inputs: bi,
Tags: t.Tags,
BuildArgs: t.Args,
Labels: t.Labels,
BuildArgs: args,
Labels: labels,
NoCache: noCache,
NoCacheFilter: t.NoCacheFilter,
Pull: pull,
@@ -924,23 +1248,30 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
bo.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
bo.Session = append(bo.Session, authprovider.NewDockerAuthProvider(dockerConfig))
bo.Session = append(bo.Session, authprovider.NewDockerAuthProvider(dockerConfig, nil))
secrets, err := buildflags.ParseSecretSpecs(t.Secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, secrets)
sshSpecs := t.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(contextPath) {
sshSpecs = []string{"default"}
}
ssh, err := buildflags.ParseSSHSpecs(sshSpecs)
secretAttachment, err := controllerapi.CreateSecrets(secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, ssh)
bo.Session = append(bo.Session, secretAttachment)
sshSpecs, err := buildflags.ParseSSHSpecs(t.SSH)
if err != nil {
return nil, err
}
if len(sshSpecs) == 0 && (buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL))) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
sshAttachment, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, sshAttachment)
if t.Target != nil {
bo.Target = *t.Target
@@ -950,19 +1281,43 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if err != nil {
return nil, err
}
bo.CacheFrom = cacheImports
bo.CacheFrom = controllerapi.CreateCaches(cacheImports)
cacheExports, err := buildflags.ParseCacheEntry(t.CacheTo)
if err != nil {
return nil, err
}
bo.CacheTo = cacheExports
bo.CacheTo = controllerapi.CreateCaches(cacheExports)
outputs, err := buildflags.ParseOutputs(t.Outputs)
outputs, err := buildflags.ParseExports(t.Outputs)
if err != nil {
return nil, err
}
bo.Exports, err = controllerapi.CreateExports(outputs)
if err != nil {
return nil, err
}
annotations, err := buildflags.ParseAnnotations(t.Annotations)
if err != nil {
return nil, err
}
for _, e := range bo.Exports {
for k, v := range annotations {
e.Attrs[k.String()] = v
}
}
attests, err := buildflags.ParseAttests(t.Attest)
if err != nil {
return nil, err
}
bo.Attests = controllerapi.CreateAttestations(attests)
bo.SourcePolicy, err = build.ReadSourcePolicy()
if err != nil {
return nil, err
}
bo.Exports = outputs
return bo, nil
}
@@ -988,8 +1343,24 @@ func removeDupes(s []string) []string {
return s[:i]
}
func isRemoteResource(str string) bool {
return urlutil.IsGitURL(str) || urlutil.IsURL(str)
func removeAttestDupes(s []string) []string {
res := []string{}
m := map[string]int{}
for _, v := range s {
att, err := buildflags.ParseAttest(v)
if err != nil {
res = append(res, v)
continue
}
if i, ok := m[att.Type]; ok {
res[i] = v
} else {
m[att.Type] = len(res)
res = append(res, v)
}
}
return res
}
func parseOutputType(str string) string {

View File

@@ -3,15 +3,16 @@ package bake
import (
"context"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestReadTargets(t *testing.T) {
t.Parallel()
fp := File{
Name: "config.hcl",
Data: []byte(`
@@ -35,21 +36,23 @@ target "webapp" {
ctx := context.TODO()
t.Run("NoOverrides", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
require.Equal(t, "Dockerfile.webapp", *m["webapp"].Dockerfile)
require.Equal(t, ".", *m["webapp"].Context)
require.Equal(t, "webDEP", m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, ptrstr("webDEP"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, true, *m["webapp"].NoCache)
require.Nil(t, m["webapp"].Pull)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
t.Run("InvalidTargetOverrides", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"nosuchtarget.context=foo"}, nil)
require.NotNil(t, err)
require.Equal(t, err.Error(), "could not find any target matching 'nosuchtarget'")
@@ -57,8 +60,7 @@ target "webapp" {
t.Run("ArgsOverrides", func(t *testing.T) {
t.Run("leaf", func(t *testing.T) {
os.Setenv("VAR_FROMENV"+t.Name(), "fromEnv")
defer os.Unsetenv("VAR_FROM_ENV" + t.Name())
t.Setenv("VAR_FROMENV"+t.Name(), "fromEnv")
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{
"webapp.args.VAR_UNSET",
@@ -79,33 +81,35 @@ target "webapp" {
_, isSet = m["webapp"].Args["VAR_EMPTY"]
require.True(t, isSet, m["webapp"].Args["VAR_EMPTY"])
require.Equal(t, m["webapp"].Args["VAR_SET"], "bananas")
require.Equal(t, ptrstr("bananas"), m["webapp"].Args["VAR_SET"])
require.Equal(t, m["webapp"].Args["VAR_FROMENV"+t.Name()], "fromEnv")
require.Equal(t, ptrstr("fromEnv"), m["webapp"].Args["VAR_FROMENV"+t.Name()])
require.Equal(t, m["webapp"].Args["VAR_BOTH"], "webapp")
require.Equal(t, m["webapp"].Args["VAR_INHERITED"], "override")
require.Equal(t, ptrstr("webapp"), m["webapp"].Args["VAR_BOTH"])
require.Equal(t, ptrstr("override"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
// building leaf but overriding parent fields
t.Run("parent", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{
"webDEP.args.VAR_INHERITED=override",
"webDEP.args.VAR_BOTH=override",
}, nil)
require.NoError(t, err)
require.Equal(t, m["webapp"].Args["VAR_INHERITED"], "override")
require.Equal(t, m["webapp"].Args["VAR_BOTH"], "webapp")
require.Equal(t, ptrstr("override"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, ptrstr("webapp"), m["webapp"].Args["VAR_BOTH"])
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
})
t.Run("ContextOverride", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.context"}, nil)
require.NotNil(t, err)
@@ -113,44 +117,47 @@ target "webapp" {
require.NoError(t, err)
require.Equal(t, "foo", *m["webapp"].Context)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
t.Run("NoCacheOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.no-cache=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].NoCache)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
t.Run("PullOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.pull=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].Pull)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
t.Run("PatternOverride", func(t *testing.T) {
t.Parallel()
// same check for two cases
multiTargetCheck := func(t *testing.T, m map[string]*Target, g []*Group, err error) {
multiTargetCheck := func(t *testing.T, m map[string]*Target, g map[string]*Group, err error) {
require.NoError(t, err)
require.Equal(t, 2, len(m))
require.Equal(t, "foo", *m["webapp"].Dockerfile)
require.Equal(t, "webDEP", m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, ptrstr("webDEP"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, "foo", *m["webDEP"].Dockerfile)
require.Equal(t, "webDEP", m["webDEP"].Args["VAR_INHERITED"])
require.Equal(t, ptrstr("webDEP"), m["webDEP"].Args["VAR_INHERITED"])
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"webDEP", "webapp"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"webDEP", "webapp"}, g["default"].Targets)
}
cases := []struct {
name string
targets []string
overrides []string
check func(*testing.T, map[string]*Target, []*Group, error)
check func(*testing.T, map[string]*Target, map[string]*Group, error)
}{
{
name: "multi target single pattern",
@@ -168,20 +175,20 @@ target "webapp" {
name: "single target",
targets: []string{"webapp"},
overrides: []string{"web*.dockerfile=foo"},
check: func(t *testing.T, m map[string]*Target, g []*Group, err error) {
check: func(t *testing.T, m map[string]*Target, g map[string]*Group, err error) {
require.NoError(t, err)
require.Equal(t, 1, len(m))
require.Equal(t, "foo", *m["webapp"].Dockerfile)
require.Equal(t, "webDEP", m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, ptrstr("webDEP"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, 1, len(g))
require.Equal(t, []string{"webapp"}, g[0].Targets)
require.Equal(t, []string{"webapp"}, g["default"].Targets)
},
},
{
name: "nomatch",
targets: []string{"webapp"},
overrides: []string{"nomatch*.dockerfile=foo"},
check: func(t *testing.T, m map[string]*Target, g []*Group, err error) {
check: func(t *testing.T, m map[string]*Target, g map[string]*Group, err error) {
// NOTE: I am unsure whether failing to match should always error out
// instead of simply skipping that override.
// Let's enforce the error and we can relax it later if users complain.
@@ -290,6 +297,9 @@ services:
ctx := context.TODO()
cwd, err := os.Getwd()
require.NoError(t, err)
m, g, err := ReadTargets(ctx, []File{fp, fp2, fp3}, []string{"default"}, nil, nil)
require.NoError(t, err)
@@ -298,13 +308,13 @@ services:
require.True(t, ok)
require.Equal(t, "Dockerfile.webapp", *m["webapp"].Dockerfile)
require.Equal(t, ".", *m["webapp"].Context)
require.Equal(t, "1", m["webapp"].Args["buildno"])
require.Equal(t, "12", m["webapp"].Args["buildno2"])
require.Equal(t, cwd, *m["webapp"].Context)
require.Equal(t, ptrstr("1"), m["webapp"].Args["buildno"])
require.Equal(t, ptrstr("12"), m["webapp"].Args["buildno2"])
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"db", "newservice", "webapp"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"db", "newservice", "webapp"}, g["default"].Targets)
}
func TestReadTargetsWithDotCompose(t *testing.T) {
@@ -337,13 +347,16 @@ services:
ctx := context.TODO()
cwd, err := os.Getwd()
require.NoError(t, err)
m, _, err := ReadTargets(ctx, []File{fp}, []string{"web.app"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
_, ok := m["web_app"]
require.True(t, ok)
require.Equal(t, "Dockerfile.webapp", *m["web_app"].Dockerfile)
require.Equal(t, "1", m["web_app"].Args["buildno"])
require.Equal(t, ptrstr("1"), m["web_app"].Args["buildno"])
m, _, err = ReadTargets(ctx, []File{fp2}, []string{"web_app"}, nil, nil)
require.NoError(t, err)
@@ -351,7 +364,7 @@ services:
_, ok = m["web_app"]
require.True(t, ok)
require.Equal(t, "Dockerfile", *m["web_app"].Dockerfile)
require.Equal(t, "12", m["web_app"].Args["buildno2"])
require.Equal(t, ptrstr("12"), m["web_app"].Args["buildno2"])
m, g, err := ReadTargets(ctx, []File{fp, fp2}, []string{"default"}, nil, nil)
require.NoError(t, err)
@@ -359,16 +372,16 @@ services:
_, ok = m["web_app"]
require.True(t, ok)
require.Equal(t, "Dockerfile.webapp", *m["web_app"].Dockerfile)
require.Equal(t, ".", *m["web_app"].Context)
require.Equal(t, "1", m["web_app"].Args["buildno"])
require.Equal(t, "12", m["web_app"].Args["buildno2"])
require.Equal(t, cwd, *m["web_app"].Context)
require.Equal(t, ptrstr("1"), m["web_app"].Args["buildno"])
require.Equal(t, ptrstr("12"), m["web_app"].Args["buildno2"])
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"web_app"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"web_app"}, g["default"].Targets)
}
func TestHCLCwdPrefix(t *testing.T) {
func TestHCLContextCwdPrefix(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
@@ -381,18 +394,49 @@ func TestHCLCwdPrefix(t *testing.T) {
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
_, ok := m["app"]
require.True(t, ok)
_, err = TargetsToBuildOpt(m, &Input{})
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, "test", *m["app"].Dockerfile)
require.Equal(t, "foo", *m["app"].Context)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
assert.Equal(t, "test", *m["app"].Dockerfile)
assert.Equal(t, "foo", *m["app"].Context)
assert.Equal(t, "foo/test", bo["app"].Inputs.DockerfilePath)
assert.Equal(t, "foo", bo["app"].Inputs.ContextPath)
}
func TestHCLDockerfileCwdPrefix(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "app" {
context = "."
dockerfile = "cwd://Dockerfile.app"
}`),
}
ctx := context.TODO()
cwd, err := os.Getwd()
require.NoError(t, err)
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g[0].Targets)
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
assert.Equal(t, "cwd://Dockerfile.app", *m["app"].Dockerfile)
assert.Equal(t, ".", *m["app"].Context)
assert.Equal(t, filepath.Join(cwd, "Dockerfile.app"), bo["app"].Inputs.DockerfilePath)
assert.Equal(t, ".", bo["app"].Inputs.ContextPath)
}
func TestOverrideMerge(t *testing.T) {
@@ -537,6 +581,9 @@ services:
ctx := context.TODO()
cwd, err := os.Getwd()
require.NoError(t, err)
m, _, err := ReadTargets(ctx, []File{fp, fp2}, []string{"app1", "app2"}, nil, nil)
require.NoError(t, err)
@@ -549,7 +596,7 @@ services:
require.Equal(t, "Dockerfile", *m["app1"].Dockerfile)
require.Equal(t, ".", *m["app1"].Context)
require.Equal(t, "Dockerfile", *m["app2"].Dockerfile)
require.Equal(t, ".", *m["app2"].Context)
require.Equal(t, cwd, *m["app2"].Context)
}
func TestReadContextFromTargetChain(t *testing.T) {
@@ -696,7 +743,7 @@ target "image" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"image"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, []string{"image"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "test", *m["image"].Dockerfile)
}
@@ -717,8 +764,9 @@ target "image" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"image"}, g["foo"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "test", *m["image"].Dockerfile)
}
@@ -742,15 +790,17 @@ target "image" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"image"}, g["foo"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "test", *m["image"].Dockerfile)
m, g, err = ReadTargets(ctx, []File{f}, []string{"foo", "foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"image"}, g["foo"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "test", *m["image"].Dockerfile)
}
@@ -829,7 +879,7 @@ services:
m, g, err := ReadTargets(ctx, []File{fhcl}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, []string{"image"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, 1, len(m["image"].Outputs))
require.Equal(t, "type=docker", m["image"].Outputs[0])
@@ -837,7 +887,7 @@ services:
m, g, err = ReadTargets(ctx, []File{fhcl}, []string{"image-release"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image-release"}, g[0].Targets)
require.Equal(t, []string{"image-release"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, 1, len(m["image-release"].Outputs))
require.Equal(t, "type=image,push=true", m["image-release"].Outputs[0])
@@ -845,7 +895,7 @@ services:
m, g, err = ReadTargets(ctx, []File{fhcl}, []string{"image", "image-release"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image", "image-release"}, g[0].Targets)
require.Equal(t, []string{"image", "image-release"}, g["default"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, ".", *m["image"].Context)
require.Equal(t, 1, len(m["image-release"].Outputs))
@@ -854,22 +904,22 @@ services:
m, g, err = ReadTargets(ctx, []File{fyml, fhcl}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, []string{"image"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, ".", *m["image"].Context)
m, g, err = ReadTargets(ctx, []File{fjson}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"image"}, g[0].Targets)
require.Equal(t, []string{"image"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, ".", *m["image"].Context)
m, g, err = ReadTargets(ctx, []File{fyml}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"addon", "aws"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"addon", "aws"}, g["default"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, "./Dockerfile", *m["addon"].Dockerfile)
require.Equal(t, "./aws.Dockerfile", *m["aws"].Dockerfile)
@@ -877,8 +927,8 @@ services:
m, g, err = ReadTargets(ctx, []File{fyml, fhcl}, []string{"addon", "aws"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"addon", "aws"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"addon", "aws"}, g["default"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, "./Dockerfile", *m["addon"].Dockerfile)
require.Equal(t, "./aws.Dockerfile", *m["aws"].Dockerfile)
@@ -886,8 +936,8 @@ services:
m, g, err = ReadTargets(ctx, []File{fyml, fhcl}, []string{"addon", "aws", "image"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
sort.Strings(g[0].Targets)
require.Equal(t, []string{"addon", "aws", "image"}, g[0].Targets)
sort.Strings(g["default"].Targets)
require.Equal(t, []string{"addon", "aws", "image"}, g["default"].Targets)
require.Equal(t, 3, len(m))
require.Equal(t, ".", *m["image"].Context)
require.Equal(t, "./Dockerfile", *m["addon"].Dockerfile)
@@ -913,15 +963,17 @@ target "image" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"foo"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"foo"}, g["foo"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "bar", *m["foo"].Dockerfile)
m, g, err = ReadTargets(ctx, []File{f}, []string{"foo", "foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"foo"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"foo"}, g["foo"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, "bar", *m["foo"].Dockerfile)
}
@@ -945,16 +997,18 @@ target "image" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"foo"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"foo", "image"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo"}, g["default"].Targets)
require.Equal(t, []string{"foo", "image"}, g["foo"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, "bar", *m["foo"].Dockerfile)
require.Equal(t, "type=docker", m["image"].Outputs[0])
m, g, err = ReadTargets(ctx, []File{f}, []string{"foo", "image"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"foo", "image"}, g[0].Targets)
require.Equal(t, 2, len(g))
require.Equal(t, []string{"foo", "image"}, g["default"].Targets)
require.Equal(t, []string{"foo", "image"}, g["foo"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, "bar", *m["foo"].Dockerfile)
require.Equal(t, "type=docker", m["image"].Outputs[0])
@@ -991,22 +1045,22 @@ target "d" {
cases := []struct {
name string
overrides []string
want map[string]string
want map[string]*string
}{
{
name: "nested simple",
overrides: nil,
want: map[string]string{"bar": "234", "baz": "890", "foo": "123"},
want: map[string]*string{"bar": ptrstr("234"), "baz": ptrstr("890"), "foo": ptrstr("123")},
},
{
name: "nested with overrides first",
overrides: []string{"a.args.foo=321", "b.args.bar=432"},
want: map[string]string{"bar": "234", "baz": "890", "foo": "321"},
want: map[string]*string{"bar": ptrstr("234"), "baz": ptrstr("890"), "foo": ptrstr("321")},
},
{
name: "nested with overrides last",
overrides: []string{"a.args.foo=321", "c.args.bar=432"},
want: map[string]string{"bar": "432", "baz": "890", "foo": "321"},
want: map[string]*string{"bar": ptrstr("432"), "baz": ptrstr("890"), "foo": ptrstr("321")},
},
}
for _, tt := range cases {
@@ -1015,7 +1069,7 @@ target "d" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"d"}, tt.overrides, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"d"}, g[0].Targets)
require.Equal(t, []string{"d"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Equal(t, tt.want, m["d"].Args)
})
@@ -1059,26 +1113,26 @@ group "default" {
cases := []struct {
name string
overrides []string
wantch1 map[string]string
wantch2 map[string]string
wantch1 map[string]*string
wantch2 map[string]*string
}{
{
name: "nested simple",
overrides: nil,
wantch1: map[string]string{"BAR": "fuu", "FOO": "bar"},
wantch2: map[string]string{"BAR": "fuu", "FOO": "bar", "FOO2": "bar2"},
wantch1: map[string]*string{"BAR": ptrstr("fuu"), "FOO": ptrstr("bar")},
wantch2: map[string]*string{"BAR": ptrstr("fuu"), "FOO": ptrstr("bar"), "FOO2": ptrstr("bar2")},
},
{
name: "nested with overrides first",
overrides: []string{"grandparent.args.BAR=fii", "child1.args.FOO=baaar"},
wantch1: map[string]string{"BAR": "fii", "FOO": "baaar"},
wantch2: map[string]string{"BAR": "fii", "FOO": "bar", "FOO2": "bar2"},
wantch1: map[string]*string{"BAR": ptrstr("fii"), "FOO": ptrstr("baaar")},
wantch2: map[string]*string{"BAR": ptrstr("fii"), "FOO": ptrstr("bar"), "FOO2": ptrstr("bar2")},
},
{
name: "nested with overrides last",
overrides: []string{"grandparent.args.BAR=fii", "child2.args.FOO=baaar"},
wantch1: map[string]string{"BAR": "fii", "FOO": "bar"},
wantch2: map[string]string{"BAR": "fii", "FOO": "baaar", "FOO2": "bar2"},
wantch1: map[string]*string{"BAR": ptrstr("fii"), "FOO": ptrstr("bar")},
wantch2: map[string]*string{"BAR": ptrstr("fii"), "FOO": ptrstr("baaar"), "FOO2": ptrstr("bar2")},
},
}
for _, tt := range cases {
@@ -1087,7 +1141,7 @@ group "default" {
m, g, err := ReadTargets(ctx, []File{f}, []string{"default"}, tt.overrides, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"child1", "child2"}, g[0].Targets)
require.Equal(t, []string{"child1", "child2"}, g["default"].Targets)
require.Equal(t, 2, len(m))
require.Equal(t, tt.wantch1, m["child1"].Args)
require.Equal(t, []string{"type=docker"}, m["child1"].Outputs)
@@ -1184,44 +1238,67 @@ target "f" {
}`)}
cases := []struct {
name string
targets []string
ntargets int
names []string
targets []string
groups []string
count int
}{
{
name: "a",
targets: []string{"b", "c"},
ntargets: 1,
names: []string{"a"},
targets: []string{"a"},
groups: []string{"default", "a", "b", "c"},
count: 1,
},
{
name: "b",
targets: []string{"d"},
ntargets: 1,
names: []string{"b"},
targets: []string{"b"},
groups: []string{"default", "b"},
count: 1,
},
{
name: "c",
targets: []string{"b"},
ntargets: 1,
names: []string{"c"},
targets: []string{"c"},
groups: []string{"default", "b", "c"},
count: 1,
},
{
name: "d",
targets: []string{"d"},
ntargets: 1,
names: []string{"d"},
targets: []string{"d"},
groups: []string{"default"},
count: 1,
},
{
name: "e",
targets: []string{"a", "f"},
ntargets: 2,
names: []string{"e"},
targets: []string{"e"},
groups: []string{"default", "a", "b", "c", "e"},
count: 2,
},
{
names: []string{"a", "e"},
targets: []string{"a", "e"},
groups: []string{"default", "a", "b", "c", "e"},
count: 2,
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
m, g, err := ReadTargets(ctx, []File{f}, []string{tt.name}, nil, nil)
t.Run(strings.Join(tt.names, "+"), func(t *testing.T) {
m, g, err := ReadTargets(ctx, []File{f}, tt.names, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, tt.targets, g[0].Targets)
require.Equal(t, tt.ntargets, len(m))
var gnames []string
for _, g := range g {
gnames = append(gnames, g.Name)
}
sort.Strings(gnames)
sort.Strings(tt.groups)
require.Equal(t, tt.groups, gnames)
sort.Strings(g["default"].Targets)
sort.Strings(tt.targets)
require.Equal(t, tt.targets, g["default"].Targets)
require.Equal(t, tt.count, len(m))
require.Equal(t, ".", *m["d"].Context)
require.Equal(t, "./testdockerfile", *m["d"].Dockerfile)
})
@@ -1254,8 +1331,192 @@ services:
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, "foo", c.Targets[0].Args["v1"])
require.Equal(t, "bar", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("foo"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["v2"])
require.Equal(t, "dir", *c.Targets[0].Context)
require.Equal(t, "Dockerfile-alternate", *c.Targets[0].Dockerfile)
}
func TestHCLNullVars(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`variable "FOO" {
default = null
}
variable "BAR" {
default = null
}
target "default" {
args = {
foo = FOO
bar = "baz"
}
labels = {
"com.docker.app.bar" = BAR
"com.docker.app.baz" = "foo"
}
}`),
}
ctx := context.TODO()
m, _, err := ReadTargets(ctx, []File{fp}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
_, ok := m["default"]
require.True(t, ok)
_, err = TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, m["default"].Args)
require.Equal(t, map[string]*string{"com.docker.app.baz": ptrstr("foo")}, m["default"].Labels)
}
func TestJSONNullVars(t *testing.T) {
fp := File{
Name: "docker-bake.json",
Data: []byte(
`{
"variable": {
"FOO": {
"default": null
}
},
"target": {
"default": {
"args": {
"foo": "${FOO}",
"bar": "baz"
}
}
}
}`),
}
ctx := context.TODO()
m, _, err := ReadTargets(ctx, []File{fp}, []string{"default"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
_, ok := m["default"]
require.True(t, ok)
_, err = TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, m["default"].Args)
}
func TestReadLocalFilesDefault(t *testing.T) {
tests := []struct {
filenames []string
expected []string
}{
{
filenames: []string{"abc.yml", "docker-compose.yml"},
expected: []string{"docker-compose.yml"},
},
{
filenames: []string{"test.foo", "compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"test.txt", "compsoe.yaml"}, // intentional misspell
expected: []string{},
},
}
pwd, err := os.Getwd()
require.NoError(t, err)
for _, tt := range tests {
t.Run(strings.Join(tt.filenames, "-"), func(t *testing.T) {
dir := t.TempDir()
t.Cleanup(func() { _ = os.Chdir(pwd) })
require.NoError(t, os.Chdir(dir))
for _, tf := range tt.filenames {
require.NoError(t, os.WriteFile(tf, []byte(tf), 0644))
}
files, err := ReadLocalFiles(nil, nil, nil)
require.NoError(t, err)
if len(files) == 0 {
require.Equal(t, len(tt.expected), len(files))
} else {
found := false
for _, exp := range tt.expected {
for _, f := range files {
if f.Name == exp {
found = true
break
}
}
require.True(t, found, exp)
}
}
})
}
}
func TestAttestDuplicates(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "default" {
attest = ["type=sbom", "type=sbom,generator=custom", "type=sbom,foo=bar", "type=provenance,mode=max"]
}`),
}
ctx := context.TODO()
m, _, err := ReadTargets(ctx, []File{fp}, []string{"default"}, nil, nil)
require.Equal(t, []string{"type=sbom,foo=bar", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": ptrstr("type=sbom,foo=bar"),
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
m, _, err = ReadTargets(ctx, []File{fp}, []string{"default"}, []string{"*.attest=type=sbom,disabled=true"}, nil)
require.Equal(t, []string{"type=sbom,disabled=true", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err = TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": nil,
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
}
func TestAnnotations(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "app" {
output = ["type=image,name=foo"]
annotations = ["manifest[linux/amd64]:foo=bar"]
}`),
}
ctx := context.TODO()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
require.Equal(t, "type=image,name=foo", m["app"].Outputs[0])
require.Equal(t, "manifest[linux/amd64]:foo=bar", m["app"].Annotations[0])
require.Len(t, bo["app"].Exports, 1)
require.Equal(t, "bar", bo["app"].Exports[0].Attrs["annotation-manifest[linux/amd64].foo"])
}

View File

@@ -1,6 +1,7 @@
package bake
import (
"context"
"os"
"path/filepath"
"strings"
@@ -28,11 +29,16 @@ func ParseComposeFiles(fs []File) (*Config, error) {
}
func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, error) {
cfg, err := loader.Load(compose.ConfigDetails{
if envs == nil {
envs = make(map[string]string)
}
cfg, err := loader.LoadWithContext(context.Background(), compose.ConfigDetails{
ConfigFiles: cfgs,
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
options.Profiles = []string{"*"}
})
if err != nil {
return nil, err
@@ -46,6 +52,7 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
g := &Group{Name: "default"}
for _, s := range cfg.Services {
s := s
if s.Build == nil {
continue
}
@@ -65,6 +72,19 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
dockerfilePath := s.Build.Dockerfile
dockerfilePathP = &dockerfilePath
}
var dockerfileInlineP *string
if s.Build.DockerfileInline != "" {
dockerfileInline := s.Build.DockerfileInline
dockerfileInlineP = &dockerfileInline
}
var additionalContexts map[string]string
if s.Build.AdditionalContexts != nil {
additionalContexts = map[string]string{}
for k, v := range s.Build.AdditionalContexts {
additionalContexts[k] = v
}
}
var secrets []string
for _, bs := range s.Build.Secrets {
@@ -75,13 +95,22 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
secrets = append(secrets, secret)
}
// compose does not support nil values for labels
labels := map[string]*string{}
for k, v := range s.Build.Labels {
v := v
labels[k] = &v
}
g.Targets = append(g.Targets, targetName)
t := &Target{
Name: targetName,
Context: contextPathP,
Dockerfile: dockerfilePathP,
Tags: s.Build.Tags,
Labels: s.Build.Labels,
Name: targetName,
Context: contextPathP,
Contexts: additionalContexts,
Dockerfile: dockerfilePathP,
DockerfileInline: dockerfileInlineP,
Tags: s.Build.Tags,
Labels: labels,
Args: flatten(s.Build.Args.Resolve(func(val string) (string, bool) {
if val, ok := s.Environment[val]; ok && val != nil {
return *val, true
@@ -138,6 +167,7 @@ func validateCompose(dt []byte, envs map[string]string) error {
},
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
// consistency is checked later in ParseCompose to ensure multiple
// compose files can be merged together
@@ -178,7 +208,7 @@ func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string,
return nil, err
}
envs, err := dotenv.UnmarshalBytes(dt)
envs, err := dotenv.UnmarshalBytesWithLookup(dt, nil)
if err != nil {
return nil, err
}
@@ -193,16 +223,16 @@ func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string,
return curenv, nil
}
func flatten(in compose.MappingWithEquals) compose.Mapping {
func flatten(in compose.MappingWithEquals) map[string]*string {
if len(in) == 0 {
return nil
}
out := compose.Mapping{}
out := map[string]*string{}
for k, v := range in {
if v == nil {
continue
}
out[k] = *v
out[k] = v
}
return out
}
@@ -222,7 +252,7 @@ type xbake struct {
NoCacheFilter stringArray `yaml:"no-cache-filter,omitempty"`
Contexts stringMap `yaml:"contexts,omitempty"`
// don't forget to update documentation if you add a new field:
// docs/guides/bake/compose-file.md#extension-field-with-x-bake
// docs/manuals/bake/compose-file.md#extension-field-with-x-bake
}
type stringMap map[string]string

View File

@@ -21,6 +21,8 @@ services:
webapp:
build:
context: ./dir
additional_contexts:
foo: ./bar
dockerfile: Dockerfile-alternate
network:
none
@@ -33,6 +35,13 @@ services:
secrets:
- token
- aws
webapp2:
profiles:
- test
build:
context: ./dir
dockerfile_inline: |
FROM alpine
secrets:
token:
environment: ENV_TOKEN
@@ -40,27 +49,31 @@ secrets:
file: /root/.aws/credentials
`)
cwd, err := os.Getwd()
require.NoError(t, err)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
sort.Strings(c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp", "webapp2"}, c.Groups[0].Targets)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, 3, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, filepath.Join(cwd, "db"), *c.Targets[0].Context)
require.Equal(t, []string{"docker.io/tonistiigi/db"}, c.Targets[0].Tags)
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, "./dir", *c.Targets[1].Context)
require.Equal(t, filepath.Join(cwd, "dir"), *c.Targets[1].Context)
require.Equal(t, map[string]string{"foo": filepath.Join(cwd, "bar")}, c.Targets[1].Contexts)
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, "123", c.Targets[1].Args["buildno"])
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, []string{"type=local,src=path/to/cache"}, c.Targets[1].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache"}, c.Targets[1].CacheTo)
require.Equal(t, "none", *c.Targets[1].NetworkMode)
@@ -68,6 +81,10 @@ secrets:
"id=token,env=ENV_TOKEN",
"id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, filepath.Join(cwd, "dir"), *c.Targets[2].Context)
require.Equal(t, "FROM alpine\n", *c.Targets[2].DockerfileInline)
}
func TestNoBuildOutOfTreeService(t *testing.T) {
@@ -149,18 +166,15 @@ services:
BRB: FOO
`)
os.Setenv("FOO", "bar")
defer os.Unsetenv("FOO")
os.Setenv("BAR", "foo")
defer os.Unsetenv("BAR")
os.Setenv("ZZZ_BAR", "zzz_foo")
defer os.Unsetenv("ZZZ_BAR")
t.Setenv("FOO", "bar")
t.Setenv("BAR", "foo")
t.Setenv("ZZZ_BAR", "zzz_foo")
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, sliceToMap(os.Environ()))
require.NoError(t, err)
require.Equal(t, "bar", c.Targets[0].Args["FOO"])
require.Equal(t, "zzz_foo", c.Targets[0].Args["BAR"])
require.Equal(t, "FOO", c.Targets[0].Args["BRB"])
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["FOO"])
require.Equal(t, ptrstr("zzz_foo"), c.Targets[0].Args["BAR"])
require.Equal(t, ptrstr("FOO"), c.Targets[0].Args["BRB"])
}
func TestInconsistentComposeFile(t *testing.T) {
@@ -308,7 +322,7 @@ services:
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, map[string]string{"CT_ECR": "foo", "CT_TAG": "bar"}, c.Targets[0].Args)
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "CT_TAG": ptrstr("bar")}, c.Targets[0].Args)
require.Equal(t, []string{"ct-addon:baz", "ct-addon:foo", "ct-addon:alp"}, c.Targets[0].Tags)
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[0].Platforms)
require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
@@ -381,7 +395,7 @@ services:
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]string{"CT_ECR": "foo", "FOO": "bsdf -csdf", "NODE_ENV": "test"}, c.Targets[0].Args)
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "FOO": ptrstr("bsdf -csdf"), "NODE_ENV": ptrstr("test")}, c.Targets[0].Args)
}
func TestDotEnv(t *testing.T) {
@@ -405,7 +419,7 @@ services:
Data: dt,
}})
require.NoError(t, err)
require.Equal(t, map[string]string{"FOO": "bar"}, c.Targets[0].Args)
require.Equal(t, map[string]*string{"FOO": ptrstr("bar")}, c.Targets[0].Args)
}
func TestPorts(t *testing.T) {
@@ -629,6 +643,101 @@ target "default" {
}
}
func TestComposeNullArgs(t *testing.T) {
var dt = []byte(`
services:
scratch:
build:
context: .
args:
FOO: null
bar: "baz"
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, c.Targets[0].Args)
}
func TestDependsOn(t *testing.T) {
var dt = []byte(`
services:
foo:
build:
context: .
ports:
- 3306:3306
depends_on:
- bar
bar:
build:
context: .
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
func TestInclude(t *testing.T) {
tmpdir := t.TempDir()
err := os.WriteFile(filepath.Join(tmpdir, "compose-foo.yml"), []byte(`
services:
foo:
build:
context: .
target: buildfoo
ports:
- 3306:3306
`), 0644)
require.NoError(t, err)
var dt = []byte(`
include:
- compose-foo.yml
services:
bar:
build:
context: .
target: buildbar
`)
chdir(t, tmpdir)
c, err := ParseComposeFiles([]File{{
Name: "compose.yml",
Data: dt,
}})
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, "bar", c.Targets[0].Name)
require.Equal(t, "buildbar", *c.Targets[0].Target)
require.Equal(t, "foo", c.Targets[1].Name)
require.Equal(t, "buildfoo", *c.Targets[1].Target)
}
func TestDevelop(t *testing.T) {
var dt = []byte(`
services:
scratch:
build:
context: ./webapp
develop:
watch:
- path: ./webapp/html
action: sync
target: /var/www
ignore:
- node_modules/
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
// chdir changes the current working directory to the named directory,
// and then restore the original working directory at the end of the test.
func chdir(t *testing.T, dir string) {

View File

@@ -1,7 +1,7 @@
package bake
import (
"os"
"reflect"
"testing"
"github.com/stretchr/testify/require"
@@ -54,7 +54,7 @@ func TestHCLBasic(t *testing.T) {
require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, "123", c.Targets[1].Args["buildno"])
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms))
@@ -62,7 +62,7 @@ func TestHCLBasic(t *testing.T) {
require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]string{"IAMCROSS": "true"}, c.Targets[3].Args)
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
}
func TestHCLBasicInJSON(t *testing.T) {
@@ -114,7 +114,7 @@ func TestHCLBasicInJSON(t *testing.T) {
require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, "123", c.Targets[1].Args["buildno"])
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, c.Targets[2].Name, "cross")
require.Equal(t, 2, len(c.Targets[2].Platforms))
@@ -122,7 +122,7 @@ func TestHCLBasicInJSON(t *testing.T) {
require.Equal(t, c.Targets[3].Name, "webapp-plus")
require.Equal(t, 1, len(c.Targets[3].Args))
require.Equal(t, map[string]string{"IAMCROSS": "true"}, c.Targets[3].Args)
require.Equal(t, map[string]*string{"IAMCROSS": ptrstr("true")}, c.Targets[3].Args)
}
func TestHCLWithFunctions(t *testing.T) {
@@ -147,7 +147,7 @@ func TestHCLWithFunctions(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, "124", c.Targets[0].Args["buildno"])
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
}
func TestHCLWithUserDefinedFunctions(t *testing.T) {
@@ -177,7 +177,7 @@ func TestHCLWithUserDefinedFunctions(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, "124", c.Targets[0].Args["buildno"])
require.Equal(t, ptrstr("124"), c.Targets[0].Args["buildno"])
}
func TestHCLWithVariables(t *testing.T) {
@@ -206,9 +206,9 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, "123", c.Targets[0].Args["buildno"])
require.Equal(t, ptrstr("123"), c.Targets[0].Args["buildno"])
os.Setenv("BUILD_NUMBER", "456")
t.Setenv("BUILD_NUMBER", "456")
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
@@ -219,7 +219,7 @@ func TestHCLWithVariables(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, "456", c.Targets[0].Args["buildno"])
require.Equal(t, ptrstr("456"), c.Targets[0].Args["buildno"])
}
func TestHCLWithVariablesInFunctions(t *testing.T) {
@@ -244,7 +244,7 @@ func TestHCLWithVariablesInFunctions(t *testing.T) {
require.Equal(t, c.Targets[0].Name, "webapp")
require.Equal(t, []string{"user/repo:v1"}, c.Targets[0].Tags)
os.Setenv("REPO", "docker/buildx")
t.Setenv("REPO", "docker/buildx")
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
@@ -280,10 +280,10 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-abc", c.Targets[0].Args["v1"])
require.Equal(t, "abc-post", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("pre-abc"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("abc-post"), c.Targets[0].Args["v2"])
os.Setenv("FOO", "def")
t.Setenv("FOO", "def")
c, err = ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
@@ -293,12 +293,11 @@ func TestHCLMultiFileSharedVariables(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", c.Targets[0].Args["v1"])
require.Equal(t, "def-post", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("def-post"), c.Targets[0].Args["v2"])
}
func TestHCLVarsWithVars(t *testing.T) {
os.Unsetenv("FOO")
dt := []byte(`
variable "FOO" {
default = upper("${BASE}def")
@@ -330,10 +329,10 @@ func TestHCLVarsWithVars(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre--ABCDEF-", c.Targets[0].Args["v1"])
require.Equal(t, "ABCDEF-post", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("pre--ABCDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("ABCDEF-post"), c.Targets[0].Args["v2"])
os.Setenv("BASE", "new")
t.Setenv("BASE", "new")
c, err = ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
@@ -343,12 +342,11 @@ func TestHCLVarsWithVars(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre--NEWDEF-", c.Targets[0].Args["v1"])
require.Equal(t, "NEWDEF-post", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("pre--NEWDEF-"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("NEWDEF-post"), c.Targets[0].Args["v2"])
}
func TestHCLTypedVariables(t *testing.T) {
os.Unsetenv("FOO")
dt := []byte(`
variable "FOO" {
default = 3
@@ -369,33 +367,80 @@ func TestHCLTypedVariables(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "lower", c.Targets[0].Args["v1"])
require.Equal(t, "yes", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("lower"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("yes"), c.Targets[0].Args["v2"])
os.Setenv("FOO", "5.1")
os.Setenv("IS_FOO", "0")
t.Setenv("FOO", "5.1")
t.Setenv("IS_FOO", "0")
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "higher", c.Targets[0].Args["v1"])
require.Equal(t, "no", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("higher"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("no"), c.Targets[0].Args["v2"])
os.Setenv("FOO", "NaN")
t.Setenv("FOO", "NaN")
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
require.Contains(t, err.Error(), "failed to parse FOO as number")
os.Setenv("FOO", "0")
os.Setenv("IS_FOO", "maybe")
t.Setenv("FOO", "0")
t.Setenv("IS_FOO", "maybe")
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
require.Contains(t, err.Error(), "failed to parse IS_FOO as bool")
}
func TestHCLNullVariables(t *testing.T) {
dt := []byte(`
variable "FOO" {
default = null
}
target "default" {
args = {
foo = FOO
}
}`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, ptrstr(nil), c.Targets[0].Args["foo"])
t.Setenv("FOO", "bar")
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["foo"])
}
func TestJSONNullVariables(t *testing.T) {
dt := []byte(`{
"variable": {
"FOO": {
"default": null
}
},
"target": {
"default": {
"args": {
"foo": "${FOO}"
}
}
}
}`)
c, err := ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, ptrstr(nil), c.Targets[0].Args["foo"])
t.Setenv("FOO", "bar")
c, err = ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["foo"])
}
func TestHCLVariableCycle(t *testing.T) {
dt := []byte(`
variable "FOO" {
@@ -431,19 +476,107 @@ func TestHCLAttrs(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "attr-abcdef", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// env does not apply if no variable
os.Setenv("FOO", "bar")
t.Setenv("FOO", "bar")
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "attr-abcdef", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("attr-abcdef"), c.Targets[0].Args["v1"])
// attr-multifile
}
func TestHCLTargetAttrs(t *testing.T) {
dt := []byte(`
target "foo" {
dockerfile = "xxx"
context = target.bar.context
target = target.foo.dockerfile
}
target "bar" {
dockerfile = target.foo.dockerfile
context = "yyy"
target = target.bar.context
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "foo", c.Targets[0].Name)
require.Equal(t, "bar", c.Targets[1].Name)
require.Equal(t, "xxx", *c.Targets[0].Dockerfile)
require.Equal(t, "yyy", *c.Targets[0].Context)
require.Equal(t, "xxx", *c.Targets[0].Target)
require.Equal(t, "xxx", *c.Targets[1].Dockerfile)
require.Equal(t, "yyy", *c.Targets[1].Context)
require.Equal(t, "yyy", *c.Targets[1].Target)
}
func TestHCLTargetGlobal(t *testing.T) {
dt := []byte(`
target "foo" {
dockerfile = "x"
}
x = target.foo.dockerfile
y = x
target "bar" {
dockerfile = y
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "foo", c.Targets[0].Name)
require.Equal(t, "bar", c.Targets[1].Name)
require.Equal(t, "x", *c.Targets[0].Dockerfile)
require.Equal(t, "x", *c.Targets[1].Dockerfile)
}
func TestHCLTargetAttrName(t *testing.T) {
dt := []byte(`
target "foo" {
dockerfile = target.foo.name
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "foo", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
}
func TestHCLTargetAttrEmptyChain(t *testing.T) {
dt := []byte(`
target "foo" {
# dockerfile = Dockerfile
context = target.foo.dockerfile
target = target.foo.context
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "foo", c.Targets[0].Name)
require.Nil(t, c.Targets[0].Dockerfile)
require.Nil(t, c.Targets[0].Context)
require.Nil(t, c.Targets[0].Target)
}
func TestHCLAttrsCustomType(t *testing.T) {
dt := []byte(`
platforms=["linux/arm64", "linux/amd64"]
@@ -461,11 +594,10 @@ func TestHCLAttrsCustomType(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, []string{"linux/arm64", "linux/amd64"}, c.Targets[0].Platforms)
require.Equal(t, "linux/arm64", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("linux/arm64"), c.Targets[0].Args["v1"])
}
func TestHCLMultiFileAttrs(t *testing.T) {
os.Unsetenv("FOO")
dt := []byte(`
variable "FOO" {
default = "abc"
@@ -487,9 +619,9 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("pre-def"), c.Targets[0].Args["v1"])
os.Setenv("FOO", "ghi")
t.Setenv("FOO", "ghi")
c, err = ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
@@ -499,7 +631,507 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-ghi", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"])
}
func TestHCLMultiFileGlobalAttrs(t *testing.T) {
dt := []byte(`
FOO = "abc"
target "app" {
args = {
v1 = "pre-${FOO}"
}
}
`)
dt2 := []byte(`
FOO = "def"
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
{Data: dt2, Name: "c2.hcl"},
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", *c.Targets[0].Args["v1"])
}
func TestHCLDuplicateTarget(t *testing.T) {
dt := []byte(`
target "app" {
dockerfile = "x"
}
target "app" {
dockerfile = "y"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, "y", *c.Targets[0].Dockerfile)
}
func TestHCLRenameTarget(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
dockerfile = "foo"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "requires matrix")
}
func TestHCLRenameGroup(t *testing.T) {
dt := []byte(`
group "foo" {
name = "bar"
targets = ["x", "y"]
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
dt = []byte(`
group "foo" {
matrix = {
name = ["x", "y"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
}
func TestHCLRenameTargetAttrs(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.xyz.dockerfile
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "xyz", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "def", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "def" {
dockerfile = target.xyz.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "def", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "xyz", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.abc.dockerfile
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
dt = []byte(`
target "def" {
dockerfile = target.abc.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
}
func TestHCLRenameSplit(t *testing.T) {
dt := []byte(`
target "x" {
name = "y"
matrix = {}
dockerfile = "foo"
}
target "x" {
name = "z"
matrix = {}
dockerfile = "bar"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "y", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "z", c.Targets[1].Name)
require.Equal(t, "bar", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "x", c.Groups[0].Name)
require.Equal(t, []string{"y", "z"}, c.Groups[0].Targets)
}
func TestHCLRenameMultiFile(t *testing.T) {
dt := []byte(`
target "foo" {
name = "bar"
matrix = {}
dockerfile = "x"
}
`)
dt2 := []byte(`
target "foo" {
context = "y"
}
`)
dt3 := []byte(`
target "bar" {
target = "z"
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
{Data: dt2, Name: "c2.hcl"},
{Data: dt3, Name: "c3.hcl"},
}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "bar")
require.Equal(t, *c.Targets[0].Dockerfile, "x")
require.Equal(t, *c.Targets[0].Target, "z")
require.Equal(t, c.Targets[1].Name, "foo")
require.Equal(t, *c.Targets[1].Context, "y")
}
func TestHCLMatrixBasic(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["x", "y"]
}
name = foo
dockerfile = "${foo}.Dockerfile"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "x")
require.Equal(t, c.Targets[1].Name, "y")
require.Equal(t, *c.Targets[0].Dockerfile, "x.Dockerfile")
require.Equal(t, *c.Targets[1].Dockerfile, "y.Dockerfile")
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.Equal(t, []string{"x", "y"}, c.Groups[0].Targets)
}
func TestHCLMatrixMultipleKeys(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a"]
bar = ["b", "c"]
baz = ["d", "e", "f"]
}
name = "${foo}-${bar}-${baz}"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 6, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, names)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, c.Groups[0].Targets)
}
func TestHCLMatrixLists(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [["aa", "bb"], ["cc", "dd"]]
}
name = aa[0]
args = {
target = "val${aa[1]}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "aa", c.Targets[0].Name)
require.Equal(t, ptrstr("valbb"), c.Targets[0].Args["target"])
require.Equal(t, "cc", c.Targets[1].Name)
require.Equal(t, ptrstr("valdd"), c.Targets[1].Args["target"])
}
func TestHCLMatrixMaps(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [
{
foo = "aa"
bar = "bb"
},
{
foo = "cc"
bar = "dd"
}
]
}
name = aa.foo
args = {
target = "val${aa.bar}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "aa")
require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb"))
require.Equal(t, c.Targets[1].Name, "cc")
require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd"))
}
func TestHCLMatrixMultipleTargets(t *testing.T) {
dt := []byte(`
target "x" {
matrix = {
foo = ["a", "b"]
}
name = foo
}
target "y" {
matrix = {
bar = ["c", "d"]
}
name = bar
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 4, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a", "b", "c", "d"}, names)
require.Equal(t, 2, len(c.Groups))
names = make([]string, len(c.Groups))
for i, c := range c.Groups {
names[i] = c.Name
}
require.ElementsMatch(t, []string{"x", "y"}, names)
for _, g := range c.Groups {
switch g.Name {
case "x":
require.Equal(t, []string{"a", "b"}, g.Targets)
case "y":
require.Equal(t, []string{"c", "d"}, g.Targets)
}
}
}
func TestHCLMatrixDuplicateNames(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a", "b"]
}
name = "c"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestHCLMatrixArgs(t *testing.T) {
dt := []byte(`
a = 1
variable "b" {
default = 2
}
target "default" {
matrix = {
foo = [a, b]
}
name = foo
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "1", c.Targets[0].Name)
require.Equal(t, "2", c.Targets[1].Name)
}
func TestHCLMatrixArgsOverride(t *testing.T) {
dt := []byte(`
variable "ABC" {
default = "def"
}
target "bar" {
matrix = {
aa = split(",", ABC)
}
name = "bar-${aa}"
args = {
foo = aa
}
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "docker-bake.hcl"},
}, map[string]string{"ABC": "11,22,33"})
require.NoError(t, err)
require.Equal(t, 3, len(c.Targets))
require.Equal(t, "bar-11", c.Targets[0].Name)
require.Equal(t, "bar-22", c.Targets[1].Name)
require.Equal(t, "bar-33", c.Targets[2].Name)
require.Equal(t, ptrstr("11"), c.Targets[0].Args["foo"])
require.Equal(t, ptrstr("22"), c.Targets[1].Args["foo"])
require.Equal(t, ptrstr("33"), c.Targets[2].Args["foo"])
}
func TestHCLMatrixBadTypes(t *testing.T) {
dt := []byte(`
target "default" {
matrix = "test"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = ["test"]
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
["a"] = ["b"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
1 = 2
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
a = "b"
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestHCLMatrixWithGlobalTarget(t *testing.T) {
dt := []byte(`
target "x" {
tags = ["a", "b"]
}
target "default" {
tags = target.x.tags
matrix = {
dummy = [""]
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "x", c.Targets[0].Name)
require.Equal(t, "default", c.Targets[1].Name)
require.Equal(t, []string{"a", "b"}, c.Targets[1].Tags)
}
func TestJSONAttributes(t *testing.T) {
@@ -510,7 +1142,7 @@ func TestJSONAttributes(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-abc-def", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("pre-abc-def"), c.Targets[0].Args["v1"])
}
func TestJSONFunctions(t *testing.T) {
@@ -535,7 +1167,25 @@ func TestJSONFunctions(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-<FOO-abc>", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
}
func TestJSONInvalidFunctions(t *testing.T) {
dt := []byte(`{
"target": {
"app": {
"args": {
"v1": "myfunc(\"foo\")"
}
}
}}`)
c, err := ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
}
func TestHCLFunctionInAttr(t *testing.T) {
@@ -563,7 +1213,7 @@ func TestHCLFunctionInAttr(t *testing.T) {
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "FOO <> [baz]", c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("FOO <> [baz]"), c.Targets[0].Args["v1"])
}
func TestHCLCombineCompose(t *testing.T) {
@@ -594,8 +1244,8 @@ services:
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "foo", c.Targets[0].Args["v1"])
require.Equal(t, "bar", c.Targets[0].Args["v2"])
require.Equal(t, ptrstr("foo"), c.Targets[0].Args["v1"])
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["v2"])
require.Equal(t, "dir", *c.Targets[0].Context)
require.Equal(t, "Dockerfile-alternate", *c.Targets[0].Dockerfile)
}
@@ -740,10 +1390,10 @@ target "two" {
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "one")
require.Equal(t, map[string]string{"a": "pre-ghi-jkl"}, c.Targets[0].Args)
require.Equal(t, map[string]*string{"a": ptrstr("pre-ghi-jkl")}, c.Targets[0].Args)
require.Equal(t, c.Targets[1].Name, "two")
require.Equal(t, map[string]string{"b": "pre-jkl"}, c.Targets[1].Args)
require.Equal(t, map[string]*string{"b": ptrstr("pre-jkl")}, c.Targets[1].Args)
}
func TestEmptyVariableJSON(t *testing.T) {
@@ -782,3 +1432,24 @@ func TestFunctionNoResult(t *testing.T) {
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestVarUnsupportedType(t *testing.T) {
dt := []byte(`
variable "FOO" {
default = []
}
target "default" {}`)
t.Setenv("FOO", "bar")
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func ptrstr(s interface{}) *string {
var n *string
if reflect.ValueOf(s).Kind() == reflect.String {
ss := s.(string)
n = &ss
}
return n
}

103
bake/hclparser/body.go Normal file
View File

@@ -0,0 +1,103 @@
package hclparser
import (
"github.com/hashicorp/hcl/v2"
)
type filterBody struct {
body hcl.Body
schema *hcl.BodySchema
exclude bool
}
func FilterIncludeBody(body hcl.Body, schema *hcl.BodySchema) hcl.Body {
return &filterBody{
body: body,
schema: schema,
}
}
func FilterExcludeBody(body hcl.Body, schema *hcl.BodySchema) hcl.Body {
return &filterBody{
body: body,
schema: schema,
exclude: true,
}
}
func (b *filterBody) Content(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Diagnostics) {
if b.exclude {
schema = subtractSchemas(schema, b.schema)
} else {
schema = intersectSchemas(schema, b.schema)
}
content, _, diag := b.body.PartialContent(schema)
return content, diag
}
func (b *filterBody) PartialContent(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
if b.exclude {
schema = subtractSchemas(schema, b.schema)
} else {
schema = intersectSchemas(schema, b.schema)
}
return b.body.PartialContent(schema)
}
func (b *filterBody) JustAttributes() (hcl.Attributes, hcl.Diagnostics) {
return b.body.JustAttributes()
}
func (b *filterBody) MissingItemRange() hcl.Range {
return b.body.MissingItemRange()
}
func intersectSchemas(a, b *hcl.BodySchema) *hcl.BodySchema {
result := &hcl.BodySchema{}
for _, blockA := range a.Blocks {
for _, blockB := range b.Blocks {
if blockA.Type == blockB.Type {
result.Blocks = append(result.Blocks, blockA)
break
}
}
}
for _, attrA := range a.Attributes {
for _, attrB := range b.Attributes {
if attrA.Name == attrB.Name {
result.Attributes = append(result.Attributes, attrA)
break
}
}
}
return result
}
func subtractSchemas(a, b *hcl.BodySchema) *hcl.BodySchema {
result := &hcl.BodySchema{}
for _, blockA := range a.Blocks {
found := false
for _, blockB := range b.Blocks {
if blockA.Type == blockB.Type {
found = true
break
}
}
if !found {
result.Blocks = append(result.Blocks, blockA)
}
}
for _, attrA := range a.Attributes {
found := false
for _, attrB := range b.Attributes {
if attrA.Name == attrB.Name {
found = true
break
}
}
if !found {
result.Attributes = append(result.Attributes, attrA)
}
}
return result
}

View File

@@ -14,15 +14,7 @@ func funcCalls(exp hcl.Expression) ([]string, hcl.Diagnostics) {
if !ok {
fns, err := jsonFuncCallsRecursive(exp)
if err != nil {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
return nil, wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
return fns, nil
}
@@ -83,11 +75,11 @@ func appendJSONFuncCalls(exp hcl.Expression, m map[string]struct{}) error {
// hcl/v2/json/ast#stringVal
val := src.FieldByName("Value")
if val.IsZero() {
if !val.IsValid() || val.IsZero() {
return nil
}
rng := src.FieldByName("SrcRange")
if val.IsZero() {
if rng.IsZero() {
return nil
}
var stringVal struct {

View File

@@ -1,7 +1,9 @@
package hclparser
import (
"encoding/binary"
"fmt"
"hash/fnv"
"math"
"math/big"
"reflect"
@@ -13,6 +15,7 @@ import (
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/gocty"
)
type Opt struct {
@@ -48,30 +51,42 @@ type parser struct {
attrs map[string]*hcl.Attribute
funcs map[string]*functionDef
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block][]reflect.Value
blockEvalCtx map[*hcl.Block][]*hcl.EvalContext
blockNames map[*hcl.Block][]string
blockTypes map[string]reflect.Type
ectx *hcl.EvalContext
progress map[string]struct{}
progressF map[string]struct{}
doneF map[string]struct{}
progressV map[uint64]struct{}
progressF map[uint64]struct{}
progressB map[uint64]map[string]struct{}
doneB map[uint64]map[string]struct{}
}
func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.Diagnostics {
type WithEvalContexts interface {
GetEvalContexts(base *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error)
}
type WithGetName interface {
GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error)
}
var errUndefined = errors.New("undefined")
func (p *parser) loadDeps(ectx *hcl.EvalContext, exp hcl.Expression, exclude map[string]struct{}, allowMissing bool) hcl.Diagnostics {
fns, hcldiags := funcCalls(exp)
if hcldiags.HasErrors() {
return hcldiags
}
for _, fn := range fns {
if err := p.resolveFunction(fn); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
if err := p.resolveFunction(ectx, fn); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
@@ -79,15 +94,61 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
if _, ok := exclude[v.RootName()]; ok {
continue
}
if err := p.resolveValue(v.RootName()); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
if _, ok := p.blockTypes[v.RootName()]; ok {
blockType := v.RootName()
split := v.SimpleSplit().Rel
if len(split) == 0 {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: fmt.Sprintf("cannot access %s as a variable", blockType),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
}
blockName, ok := split[0].(hcl.TraverseAttr)
if !ok {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: fmt.Sprintf("cannot traverse %s without attribute", blockType),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
}
blocks := p.blocks[blockType][blockName.Name]
if len(blocks) == 0 {
continue
}
var target *hcl.BodySchema
if len(split) > 1 {
if attr, ok := split[1].(hcl.TraverseAttr); ok {
target = &hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: attr.Name}},
Blocks: []hcl.BlockHeaderSchema{{Type: attr.Name}},
}
}
}
for _, block := range blocks {
if err := p.resolveBlock(block, target); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
} else {
if err := p.resolveValue(ectx, v.RootName()); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
}
@@ -95,21 +156,23 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
return nil
}
func (p *parser) resolveFunction(name string) error {
if _, ok := p.doneF[name]; ok {
// resolveFunction forces evaluation of a function, storing the result into the
// parser.
func (p *parser) resolveFunction(ectx *hcl.EvalContext, name string) error {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
if _, ok := ectx.Functions[name]; ok {
return nil
}
f, ok := p.funcs[name]
if !ok {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
return errors.Errorf("undefined function %s", name)
return errors.Wrapf(errUndefined, "function %q does not exist", name)
}
if _, ok := p.progressF[name]; ok {
if _, ok := p.progressF[key(ectx, name)]; ok {
return errors.Errorf("function cycle not allowed for %s", name)
}
p.progressF[name] = struct{}{}
p.progressF[key(ectx, name)] = struct{}{}
if f.Result == nil {
return errors.Errorf("empty result not allowed for %s", name)
@@ -154,7 +217,7 @@ func (p *parser) resolveFunction(name string) error {
return diags
}
if diags := p.loadDeps(f.Result.Expr, params); diags.HasErrors() {
if diags := p.loadDeps(p.ectx, f.Result.Expr, params, false); diags.HasErrors() {
return diags
}
@@ -164,20 +227,24 @@ func (p *parser) resolveFunction(name string) error {
if diags.HasErrors() {
return diags
}
p.doneF[name] = struct{}{}
p.ectx.Functions[name] = v
return nil
}
func (p *parser) resolveValue(name string) (err error) {
// resolveValue forces evaluation of a named value, storing the result into the
// parser.
func (p *parser) resolveValue(ectx *hcl.EvalContext, name string) (err error) {
if _, ok := p.ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progress[name]; ok {
if _, ok := ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progressV[key(ectx, name)]; ok {
return errors.Errorf("variable cycle not allowed for %s", name)
}
p.progress[name] = struct{}{}
p.progressV[key(ectx, name)] = struct{}{}
var v *cty.Value
defer func() {
@@ -190,9 +257,10 @@ func (p *parser) resolveValue(name string) (err error) {
if _, builtin := p.opt.Vars[name]; !ok && !builtin {
vr, ok := p.vars[name]
if !ok {
return errors.Errorf("undefined variable %q", name)
return errors.Wrapf(errUndefined, "variable %q does not exist", name)
}
def = vr.Default
ectx = p.ectx
}
if def == nil {
@@ -205,10 +273,10 @@ func (p *parser) resolveValue(name string) (err error) {
return
}
if diags := p.loadDeps(def.Expr, nil); diags.HasErrors() {
if diags := p.loadDeps(ectx, def.Expr, nil, true); diags.HasErrors() {
return diags
}
vv, diags := def.Expr.Value(p.ectx)
vv, diags := def.Expr.Value(ectx)
if diags.HasErrors() {
return diags
}
@@ -216,19 +284,16 @@ func (p *parser) resolveValue(name string) (err error) {
_, isVar := p.vars[name]
if envv, ok := p.opt.LookupVar(name); ok && isVar {
if vv.Type().Equals(cty.Bool) {
switch {
case vv.Type().Equals(cty.Bool):
b, err := strconv.ParseBool(envv)
if err != nil {
return errors.Wrapf(err, "failed to parse %s as bool", name)
}
vv := cty.BoolVal(b)
v = &vv
return nil
} else if vv.Type().Equals(cty.String) {
vv := cty.StringVal(envv)
v = &vv
return nil
} else if vv.Type().Equals(cty.Number) {
vv = cty.BoolVal(b)
case vv.Type().Equals(cty.String), vv.Type().Equals(cty.DynamicPseudoType):
vv = cty.StringVal(envv)
case vv.Type().Equals(cty.Number):
n, err := strconv.ParseFloat(envv, 64)
if err == nil && (math.IsNaN(n) || math.IsInf(n, 0)) {
err = errors.Errorf("invalid number value")
@@ -236,19 +301,240 @@ func (p *parser) resolveValue(name string) (err error) {
if err != nil {
return errors.Wrapf(err, "failed to parse %s as number", name)
}
vv := cty.NumberVal(big.NewFloat(n))
v = &vv
return nil
} else {
vv = cty.NumberVal(big.NewFloat(n))
default:
// TODO: support lists with csv values
return errors.Errorf("unsupported type %s for variable %s", v.Type(), name)
return errors.Errorf("unsupported type %s for variable %s", vv.Type().FriendlyName(), name)
}
}
v = &vv
return nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
// resolveBlock force evaluates a block, storing the result in the parser. If a
// target schema is provided, only the attributes and blocks present in the
// schema will be evaluated.
func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) {
// prepare the variable map for this type
if _, ok := p.ectx.Variables[block.Type]; !ok {
p.ectx.Variables[block.Type] = cty.MapValEmpty(cty.Map(cty.String))
}
// prepare the output destination and evaluation context
t, ok := p.blockTypes[block.Type]
if !ok {
return nil
}
var outputs []reflect.Value
var ectxs []*hcl.EvalContext
if prev, ok := p.blockValues[block]; ok {
outputs = prev
ectxs = p.blockEvalCtx[block]
} else {
if v, ok := reflect.New(t).Interface().(WithEvalContexts); ok {
ectxs, err = v.GetEvalContexts(p.ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(p.ectx, expr, nil, true)
})
if err != nil {
return err
}
for _, ectx := range ectxs {
if ectx != p.ectx && ectx.Parent() != p.ectx {
return errors.Errorf("EvalContext must return a context with the correct parent")
}
}
} else {
ectxs = append([]*hcl.EvalContext{}, p.ectx)
}
for range ectxs {
outputs = append(outputs, reflect.New(t))
}
}
p.blockValues[block] = outputs
p.blockEvalCtx[block] = ectxs
for i, output := range outputs {
target := target
ectx := ectxs[i]
name := block.Labels[0]
if names, ok := p.blockNames[block]; ok {
name = names[i]
}
if _, ok := p.doneB[key(block, ectx)]; !ok {
p.doneB[key(block, ectx)] = map[string]struct{}{}
}
if _, ok := p.progressB[key(block, ectx)]; !ok {
p.progressB[key(block, ectx)] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[key(block, ectx)][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[key(block, ectx)][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[key(block, ectx)][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[key(block, ectx)][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[key(block, ectx)][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[key(block, ectx)] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
}
for _, a := range content.Attributes {
diag := p.loadDeps(ectx, a.Expr, nil, true)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
if err != nil {
return err
}
}
// decode!
diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() {
return diag
}
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
}
// store the result into the evaluation context (so it can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
}
return nil
}
// resolveBlockNames returns the names of the block, calling resolveBlock to
// evaluate any label fields to correctly resolve the name.
func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
if names, ok := p.blockNames[block]; ok {
return names, nil
}
if err := p.resolveBlock(block, &hcl.BodySchema{}); err != nil {
return nil, err
}
names := make([]string, 0, len(p.blockValues[block]))
for i, val := range p.blockValues[block] {
ectx := p.blockEvalCtx[block][i]
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
if v, ok := val.Interface().(WithGetName); ok {
var err error
name, err = v.GetName(ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(ectx, expr, nil, true)
})
if err != nil {
return nil, err
}
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
}
setName(val, name)
names = append(names, name)
}
found := map[string]struct{}{}
for _, name := range names {
if _, ok := found[name]; ok {
return nil, errors.Errorf("duplicate name %q", name)
}
found[name] = struct{}{}
}
p.blockNames[block] = names
return names, nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) (map[string]map[string][]string, hcl.Diagnostics) {
reserved := map[string]struct{}{}
schema, _ := gohcl.ImpliedBodySchema(val)
@@ -261,7 +547,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
var defs inputs
if err := gohcl.DecodeBody(b, nil, &defs); err != nil {
return err
return nil, err
}
defsSchema, _ := gohcl.ImpliedBodySchema(defs)
@@ -284,13 +570,20 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
attrs: map[string]*hcl.Attribute{},
funcs: map[string]*functionDef{},
progress: map[string]struct{}{},
progressF: map[string]struct{}{},
doneF: map[string]struct{}{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block][]reflect.Value{},
blockEvalCtx: map[*hcl.Block][]*hcl.EvalContext{},
blockNames: map[*hcl.Block][]string{},
blockTypes: map[string]reflect.Type{},
ectx: &hcl.EvalContext{
Variables: map[string]cty.Value{},
Functions: stdlibFunctions,
Functions: Stdlib(),
},
progressV: map[uint64]struct{}{},
progressF: map[uint64]struct{}{},
progressB: map[uint64]map[string]struct{}{},
doneB: map[uint64]map[string]struct{}{},
}
for _, v := range defs.Variables {
@@ -310,18 +603,18 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
content, b, diags := b.PartialContent(schema)
if diags.HasErrors() {
return diags
return nil, diags
}
blocks, b, diags := b.PartialContent(defsSchema)
if diags.HasErrors() {
return diags
return nil, diags
}
attrs, diags := b.JustAttributes()
if diags.HasErrors() {
if d := removeAttributesDiags(diags, reserved, p.vars); len(d) > 0 {
return d
if d := removeAttributesDiags(diags, reserved, p.vars, attrs); len(d) > 0 {
return nil, d
}
}
@@ -334,112 +627,55 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
delete(p.attrs, "function")
for k := range p.opt.Vars {
_ = p.resolveValue(k)
}
for k := range p.attrs {
if err := p.resolveValue(k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &p.attrs[k].Range,
Context: &p.attrs[k].Range,
},
}
}
}
for k := range p.vars {
if err := p.resolveValue(k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
}
r := p.vars[k].Body.MissingItemRange()
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid value",
Detail: err.Error(),
Subject: &r,
Context: &r,
},
}
}
}
for k := range p.funcs {
if err := p.resolveFunction(k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
}
var subject *hcl.Range
var context *hcl.Range
if p.funcs[k].Params != nil {
subject = &p.funcs[k].Params.Range
context = subject
} else {
for _, block := range blocks.Blocks {
if block.Type == "function" && len(block.Labels) == 1 && block.Labels[0] == k {
subject = &block.LabelRanges[0]
context = &block.DefRange
break
}
}
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid function",
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
_ = p.resolveValue(p.ectx, k)
}
for _, a := range content.Attributes {
return hcl.Diagnostics{
a := a
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: "global attributes currently not supported",
Subject: &a.Range,
Context: &a.Range,
Subject: a.Range.Ptr(),
Context: a.Range.Ptr(),
},
}
}
m := map[string]map[string][]*hcl.Block{}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
for k := range p.vars {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
r := p.vars[k].Body.MissingItemRange()
return nil, wrapErrorDiagnostic("Invalid value", err, &r, &r)
}
bm, ok := m[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
m[b.Type] = bm
}
lbl := b.Labels[0]
bm[lbl] = append(bm[lbl], b)
}
vt := reflect.ValueOf(val).Elem().Type()
numFields := vt.NumField()
for k := range p.funcs {
if err := p.resolveFunction(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
var subject *hcl.Range
var context *hcl.Range
if p.funcs[k].Params != nil {
subject = p.funcs[k].Params.Range.Ptr()
context = subject
} else {
for _, block := range blocks.Blocks {
block := block
if block.Type == "function" && len(block.Labels) == 1 && block.Labels[0] == k {
subject = block.LabelRanges[0].Ptr()
context = block.DefRange.Ptr()
break
}
}
}
return nil, wrapErrorDiagnostic("Invalid function", err, subject, context)
}
}
type value struct {
reflect.Value
@@ -451,96 +687,177 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
values map[string]value
}
types := map[string]field{}
for i := 0; i < numFields; i++ {
renamed := map[string]map[string][]string{}
vt := reflect.ValueOf(val).Elem().Type()
for i := 0; i < vt.NumField(); i++ {
tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",")
p.blockTypes[tags[0]] = vt.Field(i).Type.Elem().Elem()
types[tags[0]] = field{
idx: i,
typ: vt.Field(i).Type,
values: make(map[string]value),
}
renamed[tags[0]] = map[string][]string{}
}
diags = hcl.Diagnostics{}
tmpBlocks := map[string]map[string][]*hcl.Block{}
for _, b := range content.Blocks {
v := reflect.ValueOf(val)
t, ok := types[b.Type]
if !ok {
continue
}
vv := reflect.New(t.typ.Elem().Elem())
diag := gohcl.DecodeBody(b.Body, p.ectx, vv.Interface())
if diag.HasErrors() {
diags = append(diags, diag...)
continue
}
if err := opt.ValidateLabel(b.Labels[0]); err != nil {
return hcl.Diagnostics{
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid name",
Detail: err.Error(),
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
lblIndex := setLabel(vv, b.Labels[0])
bm, ok := tmpBlocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
tmpBlocks[b.Type] = bm
}
oldValue, exists := t.values[b.Labels[0]]
if !exists && lblIndex != -1 {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if b.Labels[0] == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
names, err := p.resolveBlockNames(b)
if err != nil {
return nil, wrapErrorDiagnostic("Invalid name", err, &b.LabelRanges[0], &b.LabelRanges[0])
}
for _, name := range names {
bm[name] = append(bm[name], b)
renamed[b.Type][b.Labels[0]] = append(renamed[b.Type][b.Labels[0]], name)
}
}
p.blocks = tmpBlocks
diags = hcl.Diagnostics{}
for _, b := range content.Blocks {
b := b
v := reflect.ValueOf(val)
err := p.resolveBlock(b, nil)
if err != nil {
if diag, ok := err.(hcl.Diagnostics); ok {
if diag.HasErrors() {
diags = append(diags, diag...)
continue
}
} else {
return nil, wrapErrorDiagnostic("Invalid block", err, b.LabelRanges[0].Ptr(), b.DefRange.Ptr())
}
}
vvs := p.blockValues[b]
for _, vv := range vvs {
t := types[b.Type]
lblIndex, lblExists := getNameIndex(vv)
lblName, _ := getName(vv)
oldValue, exists := t.values[lblName]
if !exists && lblExists {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
}
}
}
}
}
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
}
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[lblName] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
} else {
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[b.Labels[0]] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
}
if diags.HasErrors() {
return diags
return nil, diags
}
return nil
for k := range p.attrs {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
return nil, wrapErrorDiagnostic("Invalid attribute", err, &p.attrs[k].Range, &p.attrs[k].Range)
}
}
return renamed, nil
}
func setLabel(v reflect.Value, lbl string) int {
// cache field index?
// wrapErrorDiagnostic wraps an error into a hcl.Diagnostics object.
// If the error is already an hcl.Diagnostics object, it is returned as is.
func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context *hcl.Range) hcl.Diagnostics {
switch err := err.(type) {
case *hcl.Diagnostic:
return hcl.Diagnostics{err}
case hcl.Diagnostics:
return err
default:
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: message,
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
}
func setName(v reflect.Value, name string) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
for _, t := range strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
v.Elem().Field(i).Set(reflect.ValueOf(lbl))
return i
v.Elem().Field(i).Set(reflect.ValueOf(name))
}
}
}
return -1
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable) hcl.Diagnostics {
func getName(v reflect.Value) (string, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return v.Elem().Field(i).String(), true
}
}
}
return "", false
}
func getNameIndex(v reflect.Value) (int, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return i, true
}
}
}
return 0, false
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable, attrs hcl.Attributes) hcl.Diagnostics {
var fdiags hcl.Diagnostics
for _, d := range diags {
if fout := func(d *hcl.Diagnostic) bool {
@@ -562,6 +879,12 @@ func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{},
return true
}
}
for a := range attrs {
// Do the same for attributes
if strings.HasPrefix(d.Detail, fmt.Sprintf(`Argument "%s" was already set at `, a)) {
return true
}
}
return false
}(d); !fout {
fdiags = append(fdiags, d)
@@ -569,3 +892,21 @@ func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{},
}
return fdiags
}
// key returns a unique hash for the given values
func key(ks ...any) uint64 {
hash := fnv.New64a()
for _, k := range ks {
v := reflect.ValueOf(k)
switch v.Kind() {
case reflect.String:
hash.Write([]byte(v.String()))
case reflect.Pointer:
ptr := reflect.ValueOf(k).Pointer()
binary.Write(hash, binary.LittleEndian, uint64(ptr))
default:
panic(fmt.Sprintf("unknown key kind %s", v.Kind().String()))
}
}
return hash.Sum64()
}

230
bake/hclparser/merged.go Normal file
View File

@@ -0,0 +1,230 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
// Forked from https://github.com/hashicorp/hcl/blob/4679383728fe331fc8a6b46036a27b8f818d9bc0/merged.go
package hclparser
import (
"fmt"
"github.com/hashicorp/hcl/v2"
)
// MergeFiles combines the given files to produce a single body that contains
// configuration from all of the given files.
//
// The ordering of the given files decides the order in which contained
// elements will be returned. If any top-level attributes are defined with
// the same name across multiple files, a diagnostic will be produced from
// the Content and PartialContent methods describing this error in a
// user-friendly way.
func MergeFiles(files []*hcl.File) hcl.Body {
var bodies []hcl.Body
for _, file := range files {
bodies = append(bodies, file.Body)
}
return MergeBodies(bodies)
}
// MergeBodies is like MergeFiles except it deals directly with bodies, rather
// than with entire files.
func MergeBodies(bodies []hcl.Body) hcl.Body {
if len(bodies) == 0 {
// Swap out for our singleton empty body, to reduce the number of
// empty slices we have hanging around.
return emptyBody
}
// If any of the given bodies are already merged bodies, we'll unpack
// to flatten to a single mergedBodies, since that's conceptually simpler.
// This also, as a side-effect, eliminates any empty bodies, since
// empties are merged bodies with no inner bodies.
var newLen int
var flatten bool
for _, body := range bodies {
if children, merged := body.(mergedBodies); merged {
newLen += len(children)
flatten = true
} else {
newLen++
}
}
if !flatten { // not just newLen == len, because we might have mergedBodies with single bodies inside
return mergedBodies(bodies)
}
if newLen == 0 {
// Don't allocate a new empty when we already have one
return emptyBody
}
n := make([]hcl.Body, 0, newLen)
for _, body := range bodies {
if children, merged := body.(mergedBodies); merged {
n = append(n, children...)
} else {
n = append(n, body)
}
}
return mergedBodies(n)
}
var emptyBody = mergedBodies([]hcl.Body{})
// EmptyBody returns a body with no content. This body can be used as a
// placeholder when a body is required but no body content is available.
func EmptyBody() hcl.Body {
return emptyBody
}
type mergedBodies []hcl.Body
// Content returns the content produced by applying the given schema to all
// of the merged bodies and merging the result.
//
// Although required attributes _are_ supported, they should be used sparingly
// with merged bodies since in this case there is no contextual information
// with which to return good diagnostics. Applications working with merged
// bodies may wish to mark all attributes as optional and then check for
// required attributes afterwards, to produce better diagnostics.
func (mb mergedBodies) Content(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Diagnostics) {
// the returned body will always be empty in this case, because mergedContent
// will only ever call Content on the child bodies.
content, _, diags := mb.mergedContent(schema, false)
return content, diags
}
func (mb mergedBodies) PartialContent(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
return mb.mergedContent(schema, true)
}
func (mb mergedBodies) JustAttributes() (hcl.Attributes, hcl.Diagnostics) {
attrs := make(map[string]*hcl.Attribute)
var diags hcl.Diagnostics
for _, body := range mb {
thisAttrs, thisDiags := body.JustAttributes()
if len(thisDiags) != 0 {
diags = append(diags, thisDiags...)
}
if thisAttrs != nil {
for name, attr := range thisAttrs {
if existing := attrs[name]; existing != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate argument",
Detail: fmt.Sprintf(
"Argument %q was already set at %s",
name, existing.NameRange.String(),
),
Subject: thisAttrs[name].NameRange.Ptr(),
})
}
attrs[name] = attr
}
}
}
return attrs, diags
}
func (mb mergedBodies) MissingItemRange() hcl.Range {
if len(mb) == 0 {
// Nothing useful to return here, so we'll return some garbage.
return hcl.Range{
Filename: "<empty>",
}
}
// arbitrarily use the first body's missing item range
return mb[0].MissingItemRange()
}
func (mb mergedBodies) mergedContent(schema *hcl.BodySchema, partial bool) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
// We need to produce a new schema with none of the attributes marked as
// required, since _any one_ of our bodies can contribute an attribute value.
// We'll separately check that all required attributes are present at
// the end.
mergedSchema := &hcl.BodySchema{
Blocks: schema.Blocks,
}
for _, attrS := range schema.Attributes {
mergedAttrS := attrS
mergedAttrS.Required = false
mergedSchema.Attributes = append(mergedSchema.Attributes, mergedAttrS)
}
var mergedLeftovers []hcl.Body
content := &hcl.BodyContent{
Attributes: map[string]*hcl.Attribute{},
}
var diags hcl.Diagnostics
for _, body := range mb {
var thisContent *hcl.BodyContent
var thisLeftovers hcl.Body
var thisDiags hcl.Diagnostics
if partial {
thisContent, thisLeftovers, thisDiags = body.PartialContent(mergedSchema)
} else {
thisContent, thisDiags = body.Content(mergedSchema)
}
if thisLeftovers != nil {
mergedLeftovers = append(mergedLeftovers, thisLeftovers)
}
if len(thisDiags) != 0 {
diags = append(diags, thisDiags...)
}
if thisContent.Attributes != nil {
for name, attr := range thisContent.Attributes {
if existing := content.Attributes[name]; existing != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate argument",
Detail: fmt.Sprintf(
"Argument %q was already set at %s",
name, existing.NameRange.String(),
),
Subject: thisContent.Attributes[name].NameRange.Ptr(),
})
}
content.Attributes[name] = attr
}
}
if len(thisContent.Blocks) != 0 {
content.Blocks = append(content.Blocks, thisContent.Blocks...)
}
}
// Finally, we check for required attributes.
for _, attrS := range schema.Attributes {
if !attrS.Required {
continue
}
if content.Attributes[attrS.Name] == nil {
// We don't have any context here to produce a good diagnostic,
// which is why we warn in the Content docstring to minimize the
// use of required attributes on merged bodies.
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Missing required argument",
Detail: fmt.Sprintf(
"The argument %q is required, but was not set.",
attrS.Name,
),
})
}
}
leftoverBody := MergeBodies(mergedLeftovers)
return content, leftoverBody, diags
}

View File

@@ -31,21 +31,21 @@ var stdlibFunctions = map[string]function.Function{
"cidrnetmask": cidr.NetmaskFunc,
"cidrsubnet": cidr.SubnetFunc,
"cidrsubnets": cidr.SubnetsFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"convert": typeexpr.ConvertFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"distinct": stdlib.DistinctFunc,
"divide": stdlib.DivideFunc,
"element": stdlib.ElementFunc,
"equal": stdlib.EqualFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"formatdate": stdlib.FormatDateFunc,
"format": stdlib.FormatFunc,
"formatdate": stdlib.FormatDateFunc,
"formatlist": stdlib.FormatListFunc,
"greaterthan": stdlib.GreaterThanFunc,
"greaterthanorequalto": stdlib.GreaterThanOrEqualToFunc,
@@ -53,10 +53,10 @@ var stdlibFunctions = map[string]function.Function{
"indent": stdlib.IndentFunc,
"index": stdlib.IndexFunc,
"int": stdlib.IntFunc,
"join": stdlib.JoinFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"join": stdlib.JoinFunc,
"length": stdlib.LengthFunc,
"lessthan": stdlib.LessThanFunc,
"lessthanorequalto": stdlib.LessThanOrEqualToFunc,
@@ -70,15 +70,16 @@ var stdlibFunctions = map[string]function.Function{
"modulo": stdlib.ModuloFunc,
"multiply": stdlib.MultiplyFunc,
"negate": stdlib.NegateFunc,
"notequal": stdlib.NotEqualFunc,
"not": stdlib.NotFunc,
"notequal": stdlib.NotEqualFunc,
"or": stdlib.OrFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regexall": stdlib.RegexAllFunc,
"regex": stdlib.RegexFunc,
"regex_replace": stdlib.RegexReplaceFunc,
"regex": stdlib.RegexFunc,
"regexall": stdlib.RegexAllFunc,
"replace": stdlib.ReplaceFunc,
"reverse": stdlib.ReverseFunc,
"reverselist": stdlib.ReverseListFunc,
"rsadecrypt": crypto.RsaDecryptFunc,
@@ -124,3 +125,11 @@ var timestampFunc = function.New(&function.Spec{
return cty.StringVal(time.Now().UTC().Format(time.RFC3339)), nil
},
})
func Stdlib() map[string]function.Function {
funcs := make(map[string]function.Function, len(stdlibFunctions))
for k, v := range stdlibFunctions {
funcs[k] = v
}
return funcs
}

View File

@@ -4,14 +4,16 @@ import (
"archive/tar"
"bytes"
"context"
"strings"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
)
@@ -20,11 +22,17 @@ type Input struct {
URL string
}
func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
var session []session.Attachable
var filename string
st, ok := detectGitContext(url)
if !ok {
st, filename, ok = detectHTTPContext(url)
st, ok := dockerui.DetectGitContext(url, false)
if ok {
ssh, err := controllerapi.CreateSSH([]*controllerapi.SSH{{ID: "default"}})
if err == nil {
session = append(session, ssh)
}
} else {
st, filename, ok = dockerui.DetectHTTPContext(url)
if !ok {
return nil, nil, errors.Errorf("not url context")
}
@@ -33,25 +41,25 @@ func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, na
inp := &Input{State: st, URL: url}
var files []File
var di *build.DriverInfo
for _, d := range dis {
if d.Err == nil {
di = &d
var node *builder.Node
for i, n := range nodes {
if n.Err == nil {
node = &nodes[i]
continue
}
}
if di == nil {
if node == nil {
return nil, nil, nil
}
c, err := driver.Boot(ctx, ctx, di.Driver, pw)
c, err := driver.Boot(ctx, ctx, node.Driver, pw)
if err != nil {
return nil, nil, err
}
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Session: session, Internal: true}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
@@ -83,51 +91,6 @@ func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, na
return files, inp, nil
}
func IsRemoteURL(url string) bool {
if _, _, ok := detectHTTPContext(url); ok {
return true
}
if _, ok := detectGitContext(url); ok {
return true
}
return false
}
func detectHTTPContext(url string) (*llb.State, string, bool) {
if httpPrefix.MatchString(url) {
httpContext := llb.HTTP(url, llb.Filename("context"), llb.WithCustomName("[internal] load remote build context"))
return &httpContext, "context", true
}
return nil, "", false
}
func detectGitContext(ref string) (*llb.State, bool) {
found := false
if httpPrefix.MatchString(ref) && gitURLPathWithFragmentSuffix.MatchString(ref) {
found = true
}
for _, prefix := range []string{"git://", "github.com/", "git@"} {
if strings.HasPrefix(ref, prefix) {
found = true
break
}
}
if !found {
return nil, false
}
parts := strings.SplitN(ref, "#", 2)
branch := ""
if len(parts) > 1 {
branch = parts[1]
}
gitOpts := []llb.GitOption{llb.WithCustomName("[internal] load git source " + ref)}
st := llb.Git(parts[0], branch, gitOpts...)
return &st, true
}
func isArchive(header []byte) bool {
for _, m := range [][]byte{
{0x42, 0x5A, 0x68}, // bzip2

File diff suppressed because it is too large Load Diff

305
build/driver.go Normal file
View File

@@ -0,0 +1,305 @@
package build
import (
"context"
"fmt"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/util/flightcontrol"
"github.com/moby/buildkit/util/tracing"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup"
)
type resolvedNode struct {
resolver *nodeResolver
driverIndex int
platforms []specs.Platform
}
func (dp resolvedNode) Node() builder.Node {
return dp.resolver.nodes[dp.driverIndex]
}
func (dp resolvedNode) Client(ctx context.Context) (*client.Client, error) {
clients, err := dp.resolver.boot(ctx, []int{dp.driverIndex}, nil)
if err != nil {
return nil, err
}
return clients[0], nil
}
func (dp resolvedNode) BuildOpts(ctx context.Context) (gateway.BuildOpts, error) {
opts, err := dp.resolver.opts(ctx, []int{dp.driverIndex}, nil)
if err != nil {
return gateway.BuildOpts{}, err
}
return opts[0], nil
}
type matchMaker func(specs.Platform) platforms.MatchComparer
type nodeResolver struct {
nodes []builder.Node
clients flightcontrol.Group[*client.Client]
opt flightcontrol.Group[gateway.BuildOpts]
}
func resolveDrivers(ctx context.Context, nodes []builder.Node, opt map[string]Options, pw progress.Writer) (map[string][]*resolvedNode, error) {
driverRes := newDriverResolver(nodes)
drivers, err := driverRes.Resolve(ctx, opt, pw)
if err != nil {
return nil, err
}
return drivers, err
}
func newDriverResolver(nodes []builder.Node) *nodeResolver {
r := &nodeResolver{
nodes: nodes,
}
return r
}
func (r *nodeResolver) Resolve(ctx context.Context, opt map[string]Options, pw progress.Writer) (map[string][]*resolvedNode, error) {
if len(r.nodes) == 0 {
return nil, nil
}
nodes := map[string][]*resolvedNode{}
for k, opt := range opt {
node, perfect, err := r.resolve(ctx, opt.Platforms, pw, platforms.OnlyStrict, nil)
if err != nil {
return nil, err
}
if !perfect {
break
}
nodes[k] = node
}
if len(nodes) != len(opt) {
// if we didn't get a perfect match, we need to boot all drivers
allIndexes := make([]int, len(r.nodes))
for i := range allIndexes {
allIndexes[i] = i
}
clients, err := r.boot(ctx, allIndexes, pw)
if err != nil {
return nil, err
}
eg, egCtx := errgroup.WithContext(ctx)
workers := make([][]specs.Platform, len(clients))
for i, c := range clients {
i, c := i, c
if c == nil {
continue
}
eg.Go(func() error {
ww, err := c.ListWorkers(egCtx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
ps := make(map[string]specs.Platform, len(ww))
for _, w := range ww {
for _, p := range w.Platforms {
pk := platforms.Format(platforms.Normalize(p))
ps[pk] = p
}
}
for _, p := range ps {
workers[i] = append(workers[i], p)
}
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
// then we can attempt to match against all the available platforms
// (this time we don't care about imperfect matches)
nodes = map[string][]*resolvedNode{}
for k, opt := range opt {
node, _, err := r.resolve(ctx, opt.Platforms, pw, platforms.Only, func(idx int, n builder.Node) []specs.Platform {
return workers[idx]
})
if err != nil {
return nil, err
}
nodes[k] = node
}
}
idxs := make([]int, 0, len(r.nodes))
for _, nodes := range nodes {
for _, node := range nodes {
idxs = append(idxs, node.driverIndex)
}
}
// preload capabilities
span, ctx := tracing.StartSpan(ctx, "load buildkit capabilities", trace.WithSpanKind(trace.SpanKindInternal))
_, err := r.opts(ctx, idxs, pw)
tracing.FinishWithError(span, err)
if err != nil {
return nil, err
}
return nodes, nil
}
func (r *nodeResolver) resolve(ctx context.Context, ps []specs.Platform, pw progress.Writer, matcher matchMaker, additional func(idx int, n builder.Node) []specs.Platform) ([]*resolvedNode, bool, error) {
if len(r.nodes) == 0 {
return nil, true, nil
}
if len(ps) == 0 {
ps = []specs.Platform{platforms.DefaultSpec()}
}
perfect := true
nodeIdxs := make([]int, 0)
for _, p := range ps {
idx := r.get(p, matcher, additional)
if idx == -1 {
idx = 0
perfect = false
}
nodeIdxs = append(nodeIdxs, idx)
}
var nodes []*resolvedNode
for i, idx := range nodeIdxs {
nodes = append(nodes, &resolvedNode{
resolver: r,
driverIndex: idx,
platforms: []specs.Platform{ps[i]},
})
}
nodes = recombineNodes(nodes)
if _, err := r.boot(ctx, nodeIdxs, pw); err != nil {
return nil, false, err
}
return nodes, perfect, nil
}
func (r *nodeResolver) get(p specs.Platform, matcher matchMaker, additionalPlatforms func(int, builder.Node) []specs.Platform) int {
best := -1
bestPlatform := specs.Platform{}
for i, node := range r.nodes {
platforms := node.Platforms
if additionalPlatforms != nil {
platforms = append([]specs.Platform{}, platforms...)
platforms = append(platforms, additionalPlatforms(i, node)...)
}
for _, p2 := range platforms {
m := matcher(p2)
if !m.Match(p) {
continue
}
if best == -1 {
best = i
bestPlatform = p2
continue
}
if matcher(p2).Less(p, bestPlatform) {
best = i
bestPlatform = p2
}
}
}
return best
}
func (r *nodeResolver) boot(ctx context.Context, idxs []int, pw progress.Writer) ([]*client.Client, error) {
clients := make([]*client.Client, len(idxs))
baseCtx := ctx
eg, ctx := errgroup.WithContext(ctx)
for i, idx := range idxs {
i, idx := i, idx
eg.Go(func() error {
c, err := r.clients.Do(ctx, fmt.Sprint(idx), func(ctx context.Context) (*client.Client, error) {
if r.nodes[idx].Driver == nil {
return nil, nil
}
return driver.Boot(ctx, baseCtx, r.nodes[idx].Driver, pw)
})
if err != nil {
return err
}
clients[i] = c
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
return clients, nil
}
func (r *nodeResolver) opts(ctx context.Context, idxs []int, pw progress.Writer) ([]gateway.BuildOpts, error) {
clients, err := r.boot(ctx, idxs, pw)
if err != nil {
return nil, err
}
bopts := make([]gateway.BuildOpts, len(clients))
eg, ctx := errgroup.WithContext(ctx)
for i, idxs := range idxs {
i, idx := i, idxs
c := clients[i]
if c == nil {
continue
}
eg.Go(func() error {
opt, err := r.opt.Do(ctx, fmt.Sprint(idx), func(ctx context.Context) (gateway.BuildOpts, error) {
opt := gateway.BuildOpts{}
_, err := c.Build(ctx, client.SolveOpt{
Internal: true,
}, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
opt = c.BuildOpts()
return nil, nil
}, nil)
return opt, err
})
if err != nil {
return err
}
bopts[i] = opt
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
return bopts, nil
}
// recombineDriverPairs recombines resolved nodes that are on the same driver
// back together into a single node.
func recombineNodes(nodes []*resolvedNode) []*resolvedNode {
result := make([]*resolvedNode, 0, len(nodes))
lookup := map[int]int{}
for _, node := range nodes {
if idx, ok := lookup[node.driverIndex]; ok {
result[idx].platforms = append(result[idx].platforms, node.platforms...)
} else {
lookup[node.driverIndex] = len(result)
result = append(result, node)
}
}
return result
}

313
build/driver_test.go Normal file
View File

@@ -0,0 +1,313 @@
package build
import (
"context"
"sort"
"testing"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/builder"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/require"
)
func TestFindDriverSanity(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.DefaultSpec()},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.DefaultSpec()}, nil, platforms.OnlyStrict, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestFindDriverEmpty(t *testing.T) {
r := makeTestResolver(nil)
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.DefaultSpec()}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Nil(t, res)
}
func TestFindDriverWeirdName(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/foobar")},
})
// find first platform
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/foobar")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestFindDriverUnknown(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSelectNodeSinglePlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
// find first platform
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/amd64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
// find second platform
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
// find an unknown platform, should match the first driver
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/s390x")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSelectNodeMultiPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/arm64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/amd64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeNonStrict(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm64")},
})
// arm64 should match itself
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
// arm64 may support arm/v8
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
// arm64 may support arm/v7
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeNonStrictARM(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm64")},
"ccc": {platforms.MustParse("linux/arm/v8")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "ccc", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "ccc", res[0].Node().Builder)
}
func TestSelectNodeNonStrictLower(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm/v7")},
})
// v8 can't be built on v7 (so we should select the default)...
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
// ...but v6 can be built on v8
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v6")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodePreferStart(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/riscv64")},
"ccc": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodePreferExact(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/arm/v8")},
"bbb": {platforms.MustParse("linux/arm/v7")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeCurrentPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/foobar")},
"bbb": {platforms.DefaultSpec()},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeAdditionalPlatforms(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm/v8")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, func(idx int, n builder.Node) []specs.Platform {
if n.Builder == "aaa" {
return []specs.Platform{platforms.MustParse("linux/arm/v7")}
}
return nil
})
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSplitNodeMultiPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/arm64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/arm64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/riscv64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 2)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Equal(t, "bbb", res[1].Node().Builder)
}
func TestSplitNodeMultiPlatformNoUnify(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/riscv64")},
})
// the "best" choice would be the node with both platforms, but we're using
// a naive algorithm that doesn't try to unify the platforms
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/riscv64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 2)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Equal(t, "bbb", res[1].Node().Builder)
}
func makeTestResolver(nodes map[string][]specs.Platform) *nodeResolver {
var ns []builder.Node
for name, platforms := range nodes {
ns = append(ns, builder.Node{
Builder: name,
Platforms: platforms,
})
}
sort.Slice(ns, func(i, j int) bool {
return ns[i].Builder < ns[j].Builder
})
return newDriverResolver(ns)
}

115
build/git.go Normal file
View File

@@ -0,0 +1,115 @@
package build
import (
"context"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"github.com/docker/buildx/util/gitutil"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
const DockerfileLabel = "com.docker.image.source.entrypoint"
func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath string) (res map[string]string, _ error) {
res = make(map[string]string)
if contextPath == "" {
return
}
setGitLabels := false
if v, ok := os.LookupEnv("BUILDX_GIT_LABELS"); ok {
if v == "full" { // backward compatibility with old "full" mode
setGitLabels = true
} else if v, err := strconv.ParseBool(v); err == nil {
setGitLabels = v
}
}
setGitInfo := true
if v, ok := os.LookupEnv("BUILDX_GIT_INFO"); ok {
if v, err := strconv.ParseBool(v); err == nil {
setGitInfo = v
}
}
if !setGitLabels && !setGitInfo {
return
}
// figure out in which directory the git command needs to run in
var wd string
if filepath.IsAbs(contextPath) {
wd = contextPath
} else {
cwd, _ := os.Getwd()
wd, _ = filepath.Abs(filepath.Join(cwd, contextPath))
}
gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd))
if err != nil {
if st, err1 := os.Stat(path.Join(wd, ".git")); err1 == nil && st.IsDir() {
return res, errors.Wrap(err, "git was not found in the system")
}
return
}
if !gitc.IsInsideWorkTree() {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("failed to read current commit information with git rev-parse --is-inside-work-tree")
}
return res, nil
}
if sha, err := gitc.FullCommit(); err != nil && !gitutil.IsUnknownRevision(err) {
return res, errors.Wrap(err, "failed to get git commit")
} else if sha != "" {
checkDirty := false
if v, ok := os.LookupEnv("BUILDX_GIT_CHECK_DIRTY"); ok {
if v, err := strconv.ParseBool(v); err == nil {
checkDirty = v
}
}
if checkDirty && gitc.IsDirty() {
sha += "-dirty"
}
if setGitLabels {
res["label:"+specs.AnnotationRevision] = sha
}
if setGitInfo {
res["vcs:revision"] = sha
}
}
if rurl, err := gitc.RemoteURL(); err == nil && rurl != "" {
if setGitLabels {
res["label:"+specs.AnnotationSource] = rurl
}
if setGitInfo {
res["vcs:source"] = rurl
}
}
if setGitLabels {
if root, err := gitc.RootDir(); err != nil {
return res, errors.Wrap(err, "failed to get git root dir")
} else if root != "" {
if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile")
}
if !filepath.IsAbs(dockerfilePath) {
cwd, _ := os.Getwd()
dockerfilePath = filepath.Join(cwd, dockerfilePath)
}
dockerfilePath, _ = filepath.Rel(root, dockerfilePath)
if !strings.HasPrefix(dockerfilePath, "..") {
res["label:"+DockerfileLabel] = dockerfilePath
}
}
}
return
}

156
build/git_test.go Normal file
View File

@@ -0,0 +1,156 @@
package build
import (
"context"
"os"
"path"
"path/filepath"
"strings"
"testing"
"github.com/docker/buildx/util/gitutil"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func setupTest(tb testing.TB) {
gitutil.Mktmp(tb)
c, err := gitutil.New()
require.NoError(tb, err)
gitutil.GitInit(c, tb)
df := []byte("FROM alpine:latest\n")
assert.NoError(tb, os.WriteFile("Dockerfile", df, 0644))
gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit")
gitutil.GitSetRemote(c, tb, "origin", "git@github.com:docker/buildx.git")
}
func TestGetGitAttributesNotGitRepo(t *testing.T) {
_, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
assert.NoError(t, err)
}
func TestGetGitAttributesBadGitRepo(t *testing.T) {
tmp := t.TempDir()
require.NoError(t, os.MkdirAll(path.Join(tmp, ".git"), 0755))
_, err := getGitAttributes(context.Background(), tmp, "Dockerfile")
assert.Error(t, err)
}
func TestGetGitAttributesNoContext(t *testing.T) {
setupTest(t)
gitattrs, err := getGitAttributes(context.Background(), "", "Dockerfile")
assert.NoError(t, err)
assert.Empty(t, gitattrs)
}
func TestGetGitAttributes(t *testing.T) {
cases := []struct {
name string
envGitLabels string
envGitInfo string
expected []string
}{
{
name: "default",
envGitLabels: "",
envGitInfo: "",
expected: []string{
"vcs:revision",
"vcs:source",
},
},
{
name: "none",
envGitLabels: "false",
envGitInfo: "false",
expected: []string{},
},
{
name: "gitinfo",
envGitLabels: "false",
envGitInfo: "true",
expected: []string{
"vcs:revision",
"vcs:source",
},
},
{
name: "gitlabels",
envGitLabels: "true",
envGitInfo: "false",
expected: []string{
"label:" + DockerfileLabel,
"label:" + specs.AnnotationRevision,
"label:" + specs.AnnotationSource,
},
},
{
name: "both",
envGitLabels: "true",
envGitInfo: "",
expected: []string{
"label:" + DockerfileLabel,
"label:" + specs.AnnotationRevision,
"label:" + specs.AnnotationSource,
"vcs:revision",
"vcs:source",
},
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
setupTest(t)
if tt.envGitLabels != "" {
t.Setenv("BUILDX_GIT_LABELS", tt.envGitLabels)
}
if tt.envGitInfo != "" {
t.Setenv("BUILDX_GIT_INFO", tt.envGitInfo)
}
gitattrs, err := getGitAttributes(context.Background(), ".", "Dockerfile")
require.NoError(t, err)
for _, e := range tt.expected {
assert.Contains(t, gitattrs, e)
assert.NotEmpty(t, gitattrs[e])
if e == "label:"+DockerfileLabel {
assert.Equal(t, "Dockerfile", gitattrs[e])
} else if e == "label:"+specs.AnnotationSource || e == "vcs:source" {
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs[e])
}
}
})
}
}
func TestGetGitAttributesDirty(t *testing.T) {
setupTest(t)
t.Setenv("BUILDX_GIT_CHECK_DIRTY", "true")
// make a change to test dirty flag
df := []byte("FROM alpine:edge\n")
require.NoError(t, os.Mkdir("dir", 0755))
require.NoError(t, os.WriteFile(filepath.Join("dir", "Dockerfile"), df, 0644))
t.Setenv("BUILDX_GIT_LABELS", "true")
gitattrs, _ := getGitAttributes(context.Background(), ".", "Dockerfile")
assert.Equal(t, 5, len(gitattrs))
assert.Contains(t, gitattrs, "label:"+DockerfileLabel)
assert.Equal(t, "Dockerfile", gitattrs["label:"+DockerfileLabel])
assert.Contains(t, gitattrs, "label:"+specs.AnnotationSource)
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs["label:"+specs.AnnotationSource])
assert.Contains(t, gitattrs, "label:"+specs.AnnotationRevision)
assert.True(t, strings.HasSuffix(gitattrs["label:"+specs.AnnotationRevision], "-dirty"))
assert.Contains(t, gitattrs, "vcs:source")
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs["vcs:source"])
assert.Contains(t, gitattrs, "vcs:revision")
assert.True(t, strings.HasSuffix(gitattrs["vcs:revision"], "-dirty"))
}

138
build/invoke.go Normal file
View File

@@ -0,0 +1,138 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"io"
"sync"
"sync/atomic"
"syscall"
controllerapi "github.com/docker/buildx/controller/pb"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type Container struct {
cancelOnce sync.Once
containerCancel func()
isUnavailable atomic.Bool
initStarted atomic.Bool
container gateway.Container
releaseCh chan struct{}
resultCtx *ResultHandle
}
func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig) (*Container, error) {
mainCtx := ctx
ctrCh := make(chan *Container)
errCh := make(chan error)
go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancel(ctx)
go func() {
<-mainCtx.Done()
cancel()
}()
containerCfg, err := resultCtx.getContainerConfig(ctx, c, cfg)
if err != nil {
return nil, err
}
containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil {
return nil, err
}
releaseCh := make(chan struct{})
container := &Container{
containerCancel: containerCancel,
container: bkContainer,
releaseCh: releaseCh,
resultCtx: resultCtx,
}
doneCh := make(chan struct{})
defer close(doneCh)
resultCtx.registerCleanup(func() {
container.Cancel()
<-doneCh
})
ctrCh <- container
<-container.releaseCh
return nil, bkContainer.Release(ctx)
})
if err != nil {
errCh <- err
}
}()
select {
case ctr := <-ctrCh:
return ctr, nil
case err := <-errCh:
return nil, err
case <-mainCtx.Done():
return nil, mainCtx.Err()
}
}
func (c *Container) Cancel() {
c.markUnavailable()
c.cancelOnce.Do(func() {
if c.containerCancel != nil {
c.containerCancel()
}
close(c.releaseCh)
})
}
func (c *Container) IsUnavailable() bool {
return c.isUnavailable.Load()
}
func (c *Container) markUnavailable() {
c.isUnavailable.Store(true)
}
func (c *Container) Exec(ctx context.Context, cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if isInit := c.initStarted.CompareAndSwap(false, true); isInit {
defer func() {
// container can't be used after init exits
c.markUnavailable()
}()
}
err := exec(ctx, c.resultCtx, cfg, c.container, stdin, stdout, stderr)
if err != nil {
// Container becomes unavailable if one of the processes fails in it.
c.markUnavailable()
}
return err
}
func exec(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig, ctr gateway.Container, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
processCfg, err := resultCtx.getProcessConfig(cfg, stdin, stdout, stderr)
if err != nil {
return err
}
proc, err := ctr.Start(ctx, processCfg)
if err != nil {
return errors.Errorf("failed to start container: %v", err)
}
doneCh := make(chan struct{})
defer close(doneCh)
go func() {
select {
case <-ctx.Done():
if err := proc.Signal(ctx, syscall.SIGKILL); err != nil {
logrus.Warnf("failed to kill process: %v", err)
}
case <-doneCh:
}
}()
return proc.Wait()
}

43
build/localstate.go Normal file
View File

@@ -0,0 +1,43 @@
package build
import (
"path/filepath"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/moby/buildkit/client"
)
func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, configDir string) error {
var err error
if so.Ref == "" {
return nil
}
lp := opts.Inputs.ContextPath
dp := opts.Inputs.DockerfilePath
if lp != "" || dp != "" {
if lp != "" {
lp, err = filepath.Abs(lp)
if err != nil {
return err
}
}
if dp != "" {
dp, err = filepath.Abs(dp)
if err != nil {
return err
}
}
l, err := localstate.New(configDir)
if err != nil {
return err
}
return l.SaveRef(node.Builder, node.Name, so.Ref, localstate.State{
Target: target,
LocalPath: lp,
DockerfilePath: dp,
GroupRef: opts.GroupRef,
})
}
return nil
}

495
build/result.go Normal file
View File

@@ -0,0 +1,495 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"encoding/json"
"io"
"sync"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/solver/result"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
// NewResultHandle makes a call to client.Build, additionally returning a
// opaque ResultHandle alongside the standard response and error.
//
// This ResultHandle can be used to execute additional build steps in the same
// context as the build occurred, which can allow easy debugging of build
// failures and successes.
//
// If the returned ResultHandle is not nil, the caller must call Done() on it.
func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt, product string, buildFunc gateway.BuildFunc, ch chan *client.SolveStatus) (*ResultHandle, *client.SolveResponse, error) {
// Create a new context to wrap the original, and cancel it when the
// caller-provided context is cancelled.
//
// We derive the context from the background context so that we can forbid
// cancellation of the build request after <-done is closed (which we do
// before returning the ResultHandle).
baseCtx := ctx
ctx, cancel := context.WithCancelCause(context.Background())
done := make(chan struct{})
go func() {
select {
case <-baseCtx.Done():
cancel(baseCtx.Err())
case <-done:
// Once done is closed, we've recorded a ResultHandle, so we
// shouldn't allow cancelling the underlying build request anymore.
}
}()
// Create a new channel to forward status messages to the original.
//
// We do this so that we can discard status messages after the main portion
// of the build is complete. This is necessary for the solve error case,
// where the original gateway is kept open until the ResultHandle is
// closed - we don't want progress messages from operations in that
// ResultHandle to display after this function exits.
//
// Additionally, callers should wait for the progress channel to be closed.
// If we keep the session open and never close the progress channel, the
// caller will likely hang.
baseCh := ch
ch = make(chan *client.SolveStatus)
go func() {
for {
s, ok := <-ch
if !ok {
return
}
select {
case <-baseCh:
// base channel is closed, discard status messages
default:
baseCh <- s
}
}
}()
defer close(baseCh)
var resp *client.SolveResponse
var respErr error
var respHandle *ResultHandle
go func() {
defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result
var err error
resp, err = cc.Build(ctx, opt, product, func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
var err error
res, err = buildFunc(ctx, c)
if res != nil && err == nil {
// Force evaluation of the build result (otherwise, we likely
// won't get a solve error)
def, err2 := getDefinition(ctx, res)
if err2 != nil {
return nil, err2
}
res, err = evalDefinition(ctx, c, def)
}
if err != nil {
// Scenario 1: we failed to evaluate a node somewhere in the
// build graph.
//
// In this case, we construct a ResultHandle from this
// original Build session, and return it alongside the original
// build error. We then need to keep the gateway session open
// until the caller explicitly closes the ResultHandle.
var se *errdefs.SolveError
if errors.As(err, &se) {
respHandle = &ResultHandle{
done: make(chan struct{}),
solveErr: se,
gwClient: c,
gwCtx: ctx,
}
respErr = err // return original error to preserve stacktrace
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
}
}
return res, err
}, ch)
if respHandle != nil {
return
}
if err != nil {
// Something unexpected failed during the build, we didn't succeed,
// but we also didn't make it far enough to create a ResultHandle.
respErr = err
close(done)
return
}
// Scenario 2: we successfully built the image with no errors.
//
// In this case, the original gateway session has now been closed
// since the Build has been completed. So, we need to create a new
// gateway session to populate the ResultHandle. To do this, we
// need to re-evaluate the target result, in this new session. This
// should be instantaneous since the result should be cached.
def, err := getDefinition(ctx, res)
if err != nil {
respErr = err
close(done)
return
}
// NOTE: ideally this second connection should be lazily opened
opt := opt
opt.Ref = ""
opt.Exports = nil
opt.CacheExports = nil
opt.Internal = true
_, respErr = cc.Build(ctx, opt, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
res, err := evalDefinition(ctx, c, def)
if err != nil {
// This should probably not happen, since we've previously
// successfully evaluated the same result with no issues.
return nil, errors.Wrap(err, "inconsistent solve result")
}
respHandle = &ResultHandle{
done: make(chan struct{}),
res: res,
gwClient: c,
gwCtx: ctx,
}
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
return nil, ctx.Err()
}, nil)
if respHandle != nil {
return
}
close(done)
}()
// Block until the other thread signals that it's completed the build.
select {
case <-done:
case <-baseCtx.Done():
if respErr == nil {
respErr = baseCtx.Err()
}
}
return respHandle, resp, respErr
}
// getDefinition converts a gateway result into a collection of definitions for
// each ref in the result.
func getDefinition(ctx context.Context, res *gateway.Result) (*result.Result[*pb.Definition], error) {
return result.ConvertResult(res, func(ref gateway.Reference) (*pb.Definition, error) {
st, err := ref.ToState()
if err != nil {
return nil, err
}
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
}
return def.ToPB(), nil
})
}
// evalDefinition performs the reverse of getDefinition, converting a
// collection of definitions into a gateway result.
func evalDefinition(ctx context.Context, c gateway.Client, defs *result.Result[*pb.Definition]) (*gateway.Result, error) {
// force evaluation of all targets in parallel
results := make(map[*pb.Definition]*gateway.Result)
resultsMu := sync.Mutex{}
eg, egCtx := errgroup.WithContext(ctx)
defs.EachRef(func(def *pb.Definition) error {
eg.Go(func() error {
res, err := c.Solve(egCtx, gateway.SolveRequest{
Evaluate: true,
Definition: def,
})
if err != nil {
return err
}
resultsMu.Lock()
results[def] = res
resultsMu.Unlock()
return nil
})
return nil
})
if err := eg.Wait(); err != nil {
return nil, err
}
res, _ := result.ConvertResult(defs, func(def *pb.Definition) (gateway.Reference, error) {
if res, ok := results[def]; ok {
return res.Ref, nil
}
return nil, nil
})
return res, nil
}
// ResultHandle is a build result with the client that built it.
type ResultHandle struct {
res *gateway.Result
solveErr *errdefs.SolveError
done chan struct{}
doneOnce sync.Once
gwClient gateway.Client
gwCtx context.Context
cleanups []func()
cleanupsMu sync.Mutex
}
func (r *ResultHandle) Done() {
r.doneOnce.Do(func() {
r.cleanupsMu.Lock()
cleanups := r.cleanups
r.cleanups = nil
r.cleanupsMu.Unlock()
for _, f := range cleanups {
f()
}
close(r.done)
<-r.gwCtx.Done()
})
}
func (r *ResultHandle) registerCleanup(f func()) {
r.cleanupsMu.Lock()
r.cleanups = append(r.cleanups, f)
r.cleanupsMu.Unlock()
}
func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
_, err = buildFunc(r.gwCtx, r.gwClient)
return err
}
func (r *ResultHandle) getContainerConfig(ctx context.Context, c gateway.Client, cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(ctx, r.res, c, *cfg)
if err != nil {
return containerCfg, err
}
containerCfg = *ccfg
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available")
}
containerCfg = *ccfg
}
return containerCfg, nil
}
func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) (_ gateway.StartRequest, err error) {
processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err
}
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err
}
}
return processCfg, nil
}
func containerConfigFromResult(ctx context.Context, res *gateway.Result, c gateway.Client, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
}
ps, err := exptypes.ParsePlatforms(res.Metadata)
if err != nil {
return nil, err
}
ref, ok := res.FindRef(ps.Platforms[0].ID)
if !ok {
return nil, errors.Errorf("no reference found")
}
return &gateway.NewContainerRequest{
Mounts: []gateway.Mount{
{
Dest: "/",
MountType: pb.MountType_BIND,
Ref: ref,
},
},
}, nil
}
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image
if len(imgData) > 0 {
img = &specs.Image{}
if err := json.Unmarshal(imgData, img); err != nil {
return err
}
}
user := ""
if !cfg.NoUser {
user = cfg.User
} else if img != nil {
user = img.Config.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else if img != nil {
cwd = img.Config.WorkingDir
}
env := []string{}
if img != nil {
env = append(env, img.Config.Env...)
}
env = append(env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
} else if img != nil {
args = append(args, img.Config.Entrypoint...)
}
if !cfg.NoCmd {
args = append(args, cfg.Cmd...)
} else if img != nil {
args = append(args, img.Config.Cmd...)
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr)
if err != nil {
return nil, err
}
var mounts []gateway.Mount
for i, mnt := range exec.Mounts {
rid := solveErr.Solve.MountIDs[i]
if cfg.Initial {
rid = solveErr.Solve.InputIDs[i]
}
mounts = append(mounts, gateway.Mount{
Selector: mnt.Selector,
Dest: mnt.Dest,
ResultID: rid,
Readonly: mnt.Readonly,
MountType: mnt.MountType,
CacheOpt: mnt.CacheOpt,
SecretOpt: mnt.SecretOpt,
SSHOpt: mnt.SSHOpt,
})
}
return &gateway.NewContainerRequest{
Mounts: mounts,
NetMode: exec.Network,
}, nil
}
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr)
if err != nil {
return err
}
meta := exec.Meta
user := ""
if !cfg.NoUser {
user = cfg.User
} else {
user = meta.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else {
cwd = meta.Cwd
}
env := append(meta.Env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
}
if len(args) == 0 {
args = meta.Args
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func execOpFromError(solveErr *errdefs.SolveError) (*pb.ExecOp, error) {
if solveErr == nil {
return nil, errors.Errorf("no error is available")
}
switch op := solveErr.Solve.Op.GetOp().(type) {
case *pb.Op_Exec:
return op.Exec, nil
default:
return nil, errors.Errorf("invoke: unsupported error type")
}
// TODO: support other ops
}
func newStartRequest(stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) gateway.StartRequest {
return gateway.StartRequest{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
}
}

View File

@@ -13,7 +13,7 @@ import (
"github.com/pkg/errors"
)
func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url string, pw progress.Writer) (string, error) {
func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil {
return "", err
@@ -21,7 +21,7 @@ func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url strin
var out string
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Internal: true}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := llb.HTTP(url, llb.Filename("Dockerfile"), llb.WithCustomNamef("[internal] load %s", url)).Marshal(ctx)
if err != nil {
return nil, err

View File

@@ -3,16 +3,36 @@ package build
import (
"archive/tar"
"bytes"
"context"
"net"
"os"
"strings"
"github.com/docker/buildx/driver"
"github.com/docker/cli/opts"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
"github.com/pkg/errors"
)
// archiveHeaderSize is the number of bytes in an archive header
const archiveHeaderSize = 512
const (
// archiveHeaderSize is the number of bytes in an archive header
archiveHeaderSize = 512
// mobyHostGatewayName defines a special string which users can append to
// --add-host to add an extra entry in /etc/hosts that maps
// host.docker.internal to the host IP
mobyHostGatewayName = "host-gateway"
)
func IsRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}
func isLocalDir(c string) bool {
st, err := os.Stat(c)
@@ -39,18 +59,28 @@ func isArchive(header []byte) bool {
}
// toBuildkitExtraHosts converts hosts from docker key:value format to buildkit's csv format
func toBuildkitExtraHosts(inp []string) (string, error) {
func toBuildkitExtraHosts(ctx context.Context, inp []string, nodeDriver *driver.DriverHandle) (string, error) {
if len(inp) == 0 {
return "", nil
}
hosts := make([]string, 0, len(inp))
for _, h := range inp {
parts := strings.Split(h, ":")
if len(parts) != 2 || parts[0] == "" || net.ParseIP(parts[1]) == nil {
host, ip, ok := strings.Cut(h, ":")
if !ok || host == "" || ip == "" {
return "", errors.Errorf("invalid host %s", h)
}
hosts = append(hosts, parts[0]+"="+parts[1])
// If the IP Address is a "host-gateway", replace this value with the
// IP address provided by the worker's label.
if ip == mobyHostGatewayName {
hgip, err := nodeDriver.HostGatewayIP(ctx)
if err != nil {
return "", errors.Wrap(err, "unable to derive the IP value for host-gateway")
}
ip = hgip.String()
} else if net.ParseIP(ip) == nil {
return "", errors.Errorf("invalid host %s", h)
}
hosts = append(hosts, host+"="+ip)
}
return strings.Join(hosts, ","), nil
}

299
builder/builder.go Normal file
View File

@@ -0,0 +1,299 @@
package builder
import (
"context"
"os"
"sort"
"sync"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
// Builder represents an active builder object
type Builder struct {
*store.NodeGroup
driverFactory driverFactory
nodes []Node
opts builderOpts
err error
}
type builderOpts struct {
dockerCli command.Cli
name string
txn *store.Txn
contextPathHash string
validate bool
}
// Option provides a variadic option for configuring the builder.
type Option func(b *Builder)
// WithName sets builder name.
func WithName(name string) Option {
return func(b *Builder) {
b.opts.name = name
}
}
// WithStore sets a store instance used at init.
func WithStore(txn *store.Txn) Option {
return func(b *Builder) {
b.opts.txn = txn
}
}
// WithContextPathHash is used for determining pods in k8s driver instance.
func WithContextPathHash(contextPathHash string) Option {
return func(b *Builder) {
b.opts.contextPathHash = contextPathHash
}
}
// WithSkippedValidation skips builder context validation.
func WithSkippedValidation() Option {
return func(b *Builder) {
b.opts.validate = false
}
}
// New initializes a new builder client
func New(dockerCli command.Cli, opts ...Option) (_ *Builder, err error) {
b := &Builder{
opts: builderOpts{
dockerCli: dockerCli,
validate: true,
},
}
for _, opt := range opts {
opt(b)
}
if b.opts.txn == nil {
// if store instance is nil we create a short-lived one using the
// default store and ensure we release it on completion
var release func()
b.opts.txn, release, err = storeutil.GetStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
}
if b.opts.name != "" {
if b.NodeGroup, err = storeutil.GetNodeGroup(b.opts.txn, dockerCli, b.opts.name); err != nil {
return nil, err
}
} else {
if b.NodeGroup, err = storeutil.GetCurrentInstance(b.opts.txn, dockerCli); err != nil {
return nil, err
}
}
if b.opts.validate {
if err = b.Validate(); err != nil {
return nil, err
}
}
return b, nil
}
// Validate validates builder context
func (b *Builder) Validate() error {
if b.NodeGroup != nil && b.NodeGroup.DockerContext {
list, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return err
}
currentContext := b.opts.dockerCli.CurrentContext()
for _, l := range list {
if l.Name == b.Name && l.Name != currentContext {
return errors.Errorf("use `docker --context=%s buildx` to switch to context %q", l.Name, l.Name)
}
}
}
return nil
}
// ContextName returns builder context name if available.
func (b *Builder) ContextName() string {
ctxbuilders, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return ""
}
for _, cb := range ctxbuilders {
if b.NodeGroup.Driver == "docker" && len(b.NodeGroup.Nodes) == 1 && b.NodeGroup.Nodes[0].Endpoint == cb.Name {
return cb.Name
}
}
return ""
}
// ImageOpt returns registry auth configuration
func (b *Builder) ImageOpt() (imagetools.Opt, error) {
return storeutil.GetImageConfig(b.opts.dockerCli, b.NodeGroup)
}
// Boot bootstrap a builder
func (b *Builder) Boot(ctx context.Context) (bool, error) {
toBoot := make([]int, 0, len(b.nodes))
for idx, d := range b.nodes {
if d.Err != nil || d.Driver == nil || d.DriverInfo == nil {
continue
}
if d.DriverInfo.Status != driver.Running {
toBoot = append(toBoot, idx)
}
}
if len(toBoot) == 0 {
return false, nil
}
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, progressui.AutoMode)
if err != nil {
return false, err
}
baseCtx := ctx
eg, _ := errgroup.WithContext(ctx)
errCh := make(chan error, len(toBoot))
for _, idx := range toBoot {
func(idx int) {
eg.Go(func() error {
pw := progress.WithPrefix(printer, b.NodeGroup.Nodes[idx].Name, len(toBoot) > 1)
_, err := driver.Boot(ctx, baseCtx, b.nodes[idx].Driver, pw)
if err != nil {
b.nodes[idx].Err = err
errCh <- err
}
return nil
})
}(idx)
}
err = eg.Wait()
close(errCh)
err1 := printer.Wait()
if err == nil {
err = err1
}
if err == nil && len(errCh) == len(toBoot) {
return false, <-errCh
}
return true, err
}
// Inactive checks if all nodes are inactive for this builder.
func (b *Builder) Inactive() bool {
for _, d := range b.nodes {
if d.DriverInfo != nil && d.DriverInfo.Status == driver.Running {
return false
}
}
return true
}
// Err returns error if any.
func (b *Builder) Err() error {
return b.err
}
type driverFactory struct {
driver.Factory
once sync.Once
}
// Factory returns the driver factory.
func (b *Builder) Factory(ctx context.Context, dialMeta map[string][]string) (_ driver.Factory, err error) {
b.driverFactory.once.Do(func() {
if b.Driver != "" {
b.driverFactory.Factory, err = driver.GetFactory(b.Driver, true)
if err != nil {
return
}
} else {
// empty driver means nodegroup was implicitly created as a default
// driver for a docker context and allows falling back to a
// docker-container driver for older daemon that doesn't support
// buildkit (< 18.06).
ep := b.NodeGroup.Nodes[0].Endpoint
var dockerapi *dockerutil.ClientAPI
dockerapi, err = dockerutil.NewClientAPI(b.opts.dockerCli, b.NodeGroup.Nodes[0].Endpoint)
if err != nil {
return
}
// check if endpoint is healthy is needed to determine the driver type.
// if this fails then can't continue with driver selection.
if _, err = dockerapi.Ping(ctx); err != nil {
return
}
b.driverFactory.Factory, err = driver.GetDefaultFactory(ctx, ep, dockerapi, false, dialMeta)
if err != nil {
return
}
b.Driver = b.driverFactory.Factory.Name()
}
})
return b.driverFactory.Factory, err
}
// GetBuilders returns all builders
func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
storeng, err := txn.List()
if err != nil {
return nil, err
}
builders := make([]*Builder, len(storeng))
seen := make(map[string]struct{})
for i, ng := range storeng {
b, err := New(dockerCli,
WithName(ng.Name),
WithStore(txn),
WithSkippedValidation(),
)
if err != nil {
return nil, err
}
builders[i] = b
seen[b.NodeGroup.Name] = struct{}{}
}
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
for _, c := range contexts {
// if a context has the same name as an instance from the store, do not
// add it to the builders list. An instance from the store takes
// precedence over context builders.
if _, ok := seen[c.Name]; ok {
continue
}
b, err := New(dockerCli,
WithName(c.Name),
WithStore(txn),
WithSkippedValidation(),
)
if err != nil {
return nil, err
}
builders = append(builders, b)
}
return builders, nil
}

243
builder/node.go Normal file
View File

@@ -0,0 +1,243 @@
package builder
import (
"context"
"sort"
"github.com/docker/buildx/driver"
ctxkube "github.com/docker/buildx/driver/kubernetes/context"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/platformutil"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc/codes"
)
type Node struct {
store.Node
Builder string
Driver *driver.DriverHandle
DriverInfo *driver.Info
ImageOpt imagetools.Opt
ProxyConfig map[string]string
Version string
Err error
// worker settings
IDs []string
Platforms []ocispecs.Platform
GCPolicy []client.PruneInfo
Labels map[string]string
}
// Nodes returns nodes for this builder.
func (b *Builder) Nodes() []Node {
return b.nodes
}
type LoadNodesOption func(*loadNodesOptions)
type loadNodesOptions struct {
data bool
dialMeta map[string][]string
}
func WithData() LoadNodesOption {
return func(o *loadNodesOptions) {
o.data = true
}
}
func WithDialMeta(dialMeta map[string][]string) LoadNodesOption {
return func(o *loadNodesOptions) {
o.dialMeta = dialMeta
}
}
// LoadNodes loads and returns nodes for this builder.
// TODO: this should be a method on a Node object and lazy load data for each driver.
func (b *Builder) LoadNodes(ctx context.Context, opts ...LoadNodesOption) (_ []Node, err error) {
lno := loadNodesOptions{
data: false,
}
for _, opt := range opts {
opt(&lno)
}
eg, _ := errgroup.WithContext(ctx)
b.nodes = make([]Node, len(b.NodeGroup.Nodes))
defer func() {
if b.err == nil && err != nil {
b.err = err
}
}()
factory, err := b.Factory(ctx, lno.dialMeta)
if err != nil {
return nil, err
}
imageopt, err := b.ImageOpt()
if err != nil {
return nil, err
}
for i, n := range b.NodeGroup.Nodes {
func(i int, n store.Node) {
eg.Go(func() error {
node := Node{
Node: n,
ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli),
Platforms: n.Platforms,
Builder: b.Name,
}
defer func() {
b.nodes[i] = node
}()
dockerapi, err := dockerutil.NewClientAPI(b.opts.dockerCli, n.Endpoint)
if err != nil {
node.Err = err
return nil
}
contextStore := b.opts.dockerCli.ContextStore()
var kcc driver.KubeClientConfig
kcc, err = ctxkube.ConfigFromEndpoint(n.Endpoint, contextStore)
if err != nil {
// err is returned if n.Endpoint is non-context name like "unix:///var/run/docker.sock".
// try again with name="default".
// FIXME(@AkihiroSuda): n should retain real context name.
kcc, err = ctxkube.ConfigFromEndpoint("default", contextStore)
if err != nil {
logrus.Error(err)
}
}
tryToUseKubeConfigInCluster := false
if kcc == nil {
tryToUseKubeConfigInCluster = true
} else {
if _, err := kcc.ClientConfig(); err != nil {
tryToUseKubeConfigInCluster = true
}
}
if tryToUseKubeConfigInCluster {
kccInCluster := driver.KubeClientConfigInCluster{}
if _, err := kccInCluster.ClientConfig(); err == nil {
logrus.Debug("using kube config in cluster")
kcc = kccInCluster
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, factory, n.Endpoint, dockerapi, imageopt.Auth, kcc, n.Flags, n.Files, n.DriverOpts, n.Platforms, b.opts.contextPathHash, lno.dialMeta)
if err != nil {
node.Err = err
return nil
}
node.Driver = d
node.ImageOpt = imageopt
if lno.data {
if err := node.loadData(ctx); err != nil {
node.Err = err
}
}
return nil
})
}(i, n)
}
if err := eg.Wait(); err != nil {
return nil, err
}
// TODO: This should be done in the routine loading driver data
if lno.data {
kubernetesDriverCount := 0
for _, d := range b.nodes {
if d.DriverInfo != nil && len(d.DriverInfo.DynamicNodes) > 0 {
kubernetesDriverCount++
}
}
isAllKubernetesDrivers := len(b.nodes) == kubernetesDriverCount
if isAllKubernetesDrivers {
var nodes []Node
var dynamicNodes []store.Node
for _, di := range b.nodes {
// dynamic nodes are used in Kubernetes driver.
// Kubernetes' pods are dynamically mapped to BuildKit Nodes.
if di.DriverInfo != nil && len(di.DriverInfo.DynamicNodes) > 0 {
for i := 0; i < len(di.DriverInfo.DynamicNodes); i++ {
diClone := di
if pl := di.DriverInfo.DynamicNodes[i].Platforms; len(pl) > 0 {
diClone.Platforms = pl
}
nodes = append(nodes, di)
}
dynamicNodes = append(dynamicNodes, di.DriverInfo.DynamicNodes...)
}
}
// not append (remove the static nodes in the store)
b.NodeGroup.Nodes = dynamicNodes
b.nodes = nodes
b.NodeGroup.Dynamic = true
}
}
return b.nodes, nil
}
func (n *Node) loadData(ctx context.Context) error {
if n.Driver == nil {
return nil
}
info, err := n.Driver.Info(ctx)
if err != nil {
return err
}
n.DriverInfo = info
if n.DriverInfo.Status == driver.Running {
driverClient, err := n.Driver.Client(ctx)
if err != nil {
return err
}
workers, err := driverClient.ListWorkers(ctx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
for idx, w := range workers {
n.IDs = append(n.IDs, w.ID)
n.Platforms = append(n.Platforms, w.Platforms...)
if idx == 0 {
n.GCPolicy = w.GCPolicy
n.Labels = w.Labels
}
}
sort.Strings(n.IDs)
n.Platforms = platformutil.Dedupe(n.Platforms)
inf, err := driverClient.Info(ctx)
if err != nil {
if st, ok := grpcerrors.AsGRPCStatus(err); ok && st.Code() == codes.Unimplemented {
n.Version, err = n.Driver.Version(ctx)
if err != nil {
return errors.Wrap(err, "getting version")
}
}
} else {
n.Version = inf.BuildkitVersion.Version
}
}
return nil
}

View File

@@ -4,8 +4,8 @@ import (
"fmt"
"os"
"github.com/containerd/containerd/pkg/seed"
"github.com/docker/buildx/commands"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/manager"
@@ -16,10 +16,10 @@ import (
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack"
_ "k8s.io/client-go/plugin/pkg/client/auth/azure"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
@@ -28,7 +28,9 @@ import (
)
func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision)
}
@@ -85,6 +87,9 @@ func main() {
} else {
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
}
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
ebr.Print(cmd.Err())
}
os.Exit(1)
}

View File

@@ -4,28 +4,45 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"strings"
"github.com/containerd/console"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/bake"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/util/tracing"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type bakeOptions struct {
files []string
overrides []string
printOnly bool
commonOptions
files []string
overrides []string
printOnly bool
sbom string
provenance string
builder string
metadataFile string
exportPush bool
exportLoad bool
}
func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error) {
func runBake(dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
ctx := appcontext.Context()
ctx, end, err := tracing.TraceCurrentCommand(ctx, "bake")
@@ -40,11 +57,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
cmdContext := "cwd://"
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
url = targets[0]
targets = targets[1:]
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
cmdContext = targets[0]
targets = targets[1:]
}
@@ -65,17 +82,58 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
} else if in.exportLoad {
overrides = append(overrides, "*.output=type=docker")
}
if in.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *in.noCache))
if cFlags.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *cFlags.noCache))
}
if in.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *in.pull))
if cFlags.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *cFlags.pull))
}
if in.sbom != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("sbom", in.sbom)))
}
if in.provenance != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("provenance", in.provenance)))
}
contextPathHash, _ := os.Getwd()
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
var nodes []builder.Node
var progressConsoleDesc, progressTextDesc string
// instance only needed for reading remote bake files or building
if url != "" || !in.printOnly {
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err = b.LoadNodes(ctx)
if err != nil {
return err
}
progressConsoleDesc = fmt.Sprintf("%s:%s", b.Driver, b.Name)
progressTextDesc = fmt.Sprintf("building with %q instance using %s driver", b.Name, b.Driver)
}
var term bool
if _, err := console.ConsoleFromFile(os.Stderr); err == nil {
term = true
}
progressMode := progressui.DisplayMode(cFlags.progress)
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressMode,
progress.WithDesc(progressTextDesc, progressConsoleDesc),
)
if err != nil {
return err
}
defer func() {
if printer != nil {
@@ -83,29 +141,24 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
if err == nil {
err = err1
}
if err == nil && progressMode != progressui.QuietMode {
desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term)
}
}
}()
dis, err := getInstanceOrDefault(ctx, dockerCli, in.builder, contextPathHash)
files, inp, err := readBakeFiles(ctx, nodes, url, in.files, dockerCli.In(), printer)
if err != nil {
return err
}
var files []bake.File
var inp *bake.Input
if url != "" {
files, inp, err = bake.ReadRemoteFiles(ctx, dis, url, in.files, printer)
} else {
files, err = bake.ReadLocalFiles(in.files)
}
if err != nil {
return err
if len(files) == 0 {
return errors.New("couldn't find a bake definition")
}
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, map[string]string{
// don't forget to update documentation if you add a new
// built-in variable: docs/guides/bake/file-definition.md#built-in-variables
// built-in variable: docs/bake-reference.md#built-in-variables
"BAKE_CMD_CONTEXT": cmdContext,
"BAKE_LOCAL_PLATFORM": platforms.DefaultString(),
})
@@ -113,26 +166,35 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return err
}
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
// TODO: extract env var parsing to a method easily usable by library consumers
for _, t := range tgts {
if _, ok := t.Args["SOURCE_DATE_EPOCH"]; ok {
continue
}
if t.Args == nil {
t.Args = map[string]*string{}
}
t.Args["SOURCE_DATE_EPOCH"] = &v
}
}
// this function can update target context string from the input so call before printOnly check
bo, err := bake.TargetsToBuildOpt(tgts, inp)
if err != nil {
return err
}
def := struct {
Group map[string]*bake.Group `json:"group,omitempty"`
Target map[string]*bake.Target `json:"target"`
}{
Group: grps,
Target: tgts,
}
if in.printOnly {
var defg map[string]*bake.Group
if len(grps) == 1 {
defg = map[string]*bake.Group{
"default": grps[0],
}
}
dt, err := json.MarshalIndent(struct {
Group map[string]*bake.Group `json:"group,omitempty"`
Target map[string]*bake.Target `json:"target"`
}{
defg,
tgts,
}, "", " ")
dt, err := json.MarshalIndent(def, "", " ")
if err != nil {
return err
}
@@ -145,7 +207,29 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return nil
}
resp, err := build.Build(ctx, dis, bo, dockerAPI(dockerCli), confutil.ConfigDir(dockerCli), printer)
// local state group
groupRef := identity.NewID()
var refs []string
for k, b := range bo {
b.Ref = identity.NewID()
b.GroupRef = groupRef
refs = append(refs, b.Ref)
bo[k] = b
}
dt, err := json.Marshal(def)
if err != nil {
return err
}
if err := saveLocalStateGroup(dockerCli, groupRef, localstate.StateGroup{
Definition: dt,
Targets: targets,
Inputs: overrides,
Refs: refs,
}); err != nil {
return err
}
resp, err := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), printer)
if err != nil {
return wrapBuildError(err, true)
}
@@ -165,6 +249,7 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
var options bakeOptions
var cFlags commonFlags
cmd := &cobra.Command{
Use: "bake [OPTIONS] [TARGET...]",
@@ -173,14 +258,17 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
// reset to nil to avoid override is unset
if !cmd.Flags().Lookup("no-cache").Changed {
options.noCache = nil
cFlags.noCache = nil
}
if !cmd.Flags().Lookup("pull").Changed {
options.pull = nil
cFlags.pull = nil
}
options.commonOptions.builder = rootOpts.builder
return runBake(dockerCli, args, options)
options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function.
return runBake(dockerCli, args, options, cFlags)
},
ValidArgsFunction: completion.BakeTargets(options.files),
}
flags := cmd.Flags()
@@ -189,9 +277,58 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.BoolVar(&options.exportLoad, "load", false, `Shorthand for "--set=*.output=type=docker"`)
flags.BoolVar(&options.printOnly, "print", false, "Print the options without building")
flags.BoolVar(&options.exportPush, "push", false, `Shorthand for "--set=*.output=type=registry"`)
flags.StringVar(&options.sbom, "sbom", "", `Shorthand for "--set=*.attest=type=sbom"`)
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
flags.StringArrayVar(&options.overrides, "set", nil, `Override target value (e.g., "targetpattern.key=value")`)
commonBuildFlags(&options.commonOptions, flags)
commonBuildFlags(&cFlags, flags)
return cmd
}
func saveLocalStateGroup(dockerCli command.Cli, ref string, lsg localstate.StateGroup) error {
l, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
return l.SaveGroup(ref, lsg)
}
func readBakeFiles(ctx context.Context, nodes []builder.Node, url string, names []string, stdin io.Reader, pw progress.Writer) (files []bake.File, inp *bake.Input, err error) {
var lnames []string
var rnames []string
for _, v := range names {
if strings.HasPrefix(v, "cwd://") {
lnames = append(lnames, strings.TrimPrefix(v, "cwd://"))
} else {
rnames = append(rnames, v)
}
}
if url != "" {
var rfiles []bake.File
rfiles, inp, err = bake.ReadRemoteFiles(ctx, nodes, url, rnames, pw)
if err != nil {
return nil, nil, err
}
files = append(files, rfiles...)
}
if len(lnames) > 0 || url == "" {
var lfiles []bake.File
progress.Wrap("[internal] load local bake definitions", pw.Write, func(sub progress.SubLogger) error {
if url != "" {
lfiles, err = bake.ReadLocalFiles(lnames, stdin, sub)
} else {
lfiles, err = bake.ReadLocalFiles(append(lnames, rnames...), stdin, sub)
}
return nil
})
if err != nil {
return nil, nil, err
}
files = append(files, lfiles...)
}
return
}

File diff suppressed because it is too large Load Diff

View File

@@ -10,13 +10,20 @@ import (
"strings"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
k8sutil "github.com/docker/buildx/driver/kubernetes/util"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
dopts "github.com/docker/cli/opts"
"github.com/google/shlex"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
@@ -65,6 +72,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if err != nil {
return err
}
// Ensure the file lock gets released no matter what happens.
defer release()
name := in.name
@@ -115,7 +123,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if len(args) > 0 {
arg = args[0]
}
f, err := driver.GetDefaultFactory(ctx, arg, dockerCli.Client(), true)
f, err := driver.GetDefaultFactory(ctx, arg, dockerCli.Client(), true, nil)
if err != nil {
return err
}
@@ -165,18 +173,34 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if err := ng.Leave(in.nodeName); err != nil {
return err
}
ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
if err := ls.RemoveBuilderNode(ng.Name, in.nodeName); err != nil {
return err
}
} else {
switch {
case driverName == "kubernetes":
if len(args) > 0 {
logrus.Warnf("kubernetes driver does not support endpoint args %q", args[0])
}
// generate node name if not provided to avoid duplicated endpoint
// error: https://github.com/docker/setup-buildx-action/issues/215
nodeName := in.nodeName
if nodeName == "" {
nodeName, err = k8sutil.GenerateNodeName(name, txn)
if err != nil {
return err
}
}
// naming endpoint to make --append works
ep = (&url.URL{
Scheme: driverName,
Path: "/" + in.name,
Path: "/" + name,
RawQuery: (&url.Values{
"deployment": {in.nodeName},
"deployment": {nodeName},
"kubeconfig": {os.Getenv("KUBECONFIG")},
}).Encode(),
}).String()
@@ -204,7 +228,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if dockerCli.CurrentContext() == "default" && dockerCli.DockerEndpoint().TLSData != nil {
return errors.Errorf("could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with `docker buildx create <context-name>`")
}
ep, err = storeutil.GetCurrentEndpoint(dockerCli)
ep, err = dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
@@ -234,17 +258,26 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
return err
}
ngi := &nginfo{ng: ng}
b, err := builder.New(dockerCli,
builder.WithName(ng.Name),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
if err = loadNodeGroupData(timeoutCtx, dockerCli, ngi); err != nil {
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if err != nil {
return err
}
for _, info := range ngi.drivers {
if err := info.di.Err; err != nil {
err := errors.Errorf("failed to initialize builder %s (%s): %s", ng.Name, info.di.Name, err)
for _, node := range nodes {
if err := node.Err; err != nil {
err := errors.Errorf("failed to initialize builder %s (%s): %s", ng.Name, node.Name, err)
var err2 error
if ngOriginal == nil {
err2 = txn.Remove(ng.Name)
@@ -259,7 +292,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
if in.use && ep != "" {
current, err := storeutil.GetCurrentEndpoint(dockerCli)
current, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
@@ -268,8 +301,12 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
// The store is no longer used from this point.
// Release it so we aren't holding the file lock during the boot.
release()
if in.bootstrap {
if _, err = boot(ctx, ngi); err != nil {
if _, err = b.Boot(ctx); err != nil {
return err
}
}
@@ -296,6 +333,7 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
@@ -340,3 +378,27 @@ func csvToMap(in []string) (map[string]string, error) {
}
return m, nil
}
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
dem, err := dockerutil.GetDockerEndpoint(dockerCli, ep)
if err == nil && dem != nil {
if ep == "default" {
return dem.Host, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// validateBuildkitEndpoint validates that endpoint is a valid buildkit host
func validateBuildkitEndpoint(ep string) (string, error) {
if err := remoteutil.IsValidEndpoint(ep); err != nil {
return "", err
}
return ep, nil
}

96
commands/debug/root.go Normal file
View File

@@ -0,0 +1,96 @@
package debug
import (
"context"
"os"
"runtime"
"github.com/containerd/console"
"github.com/docker/buildx/controller"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/monitor"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// DebugConfig is a user-specified configuration for the debugger.
type DebugConfig struct {
// InvokeFlag is a flag to configure the launched debugger and the commaned executed on the debugger.
InvokeFlag string
// OnFlag is a flag to configure the timing of launching the debugger.
OnFlag string
}
// DebuggableCmd is a command that supports debugger with recognizing the user-specified DebugConfig.
type DebuggableCmd interface {
// NewDebugger returns the new *cobra.Command with support for the debugger with recognizing DebugConfig.
NewDebugger(*DebugConfig) *cobra.Command
}
func RootCmd(dockerCli command.Cli, children ...DebuggableCmd) *cobra.Command {
var controlOptions control.ControlOptions
var progressMode string
var options DebugConfig
cmd := &cobra.Command{
Use: "debug",
Short: "Start debugger",
Args: cobra.NoArgs,
Annotations: map[string]string{
"experimentalCLI": "",
},
RunE: func(cmd *cobra.Command, args []string) error {
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, progressui.DisplayMode(progressMode))
if err != nil {
return err
}
ctx := context.TODO()
c, err := controller.NewController(ctx, controlOptions, dockerCli, printer)
if err != nil {
return err
}
defer func() {
if err := c.Close(); err != nil {
logrus.Warnf("failed to close server connection %v", err)
}
}()
con := console.Current()
if err := con.SetRaw(); err != nil {
return errors.Errorf("failed to configure terminal: %v", err)
}
_, err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true,
}, c, dockerCli.In(), os.Stdout, os.Stderr, printer)
con.Reset()
return err
},
}
flags := cmd.Flags()
flags.StringVar(&options.InvokeFlag, "invoke", "", "Launch a monitor with executing specified command")
flags.SetAnnotation("invoke", "experimentalCLI", nil)
flags.StringVar(&options.OnFlag, "on", "error", "When to launch the monitor ([always, error])")
flags.SetAnnotation("on", "experimentalCLI", nil)
flags.StringVar(&controlOptions.Root, "root", "", "Specify root directory of server to connect for the monitor")
flags.SetAnnotation("root", "experimentalCLI", nil)
flags.BoolVar(&controlOptions.Detach, "detach", runtime.GOOS == "linux", "Detach buildx server for the monitor (supported only on linux)")
flags.SetAnnotation("detach", "experimentalCLI", nil)
flags.StringVar(&controlOptions.ServerConfig, "server-config", "", "Specify buildx server config file for the monitor (used only when launching new server)")
flags.SetAnnotation("server-config", "experimentalCLI", nil)
flags.StringVar(&progressMode, "progress", "auto", `Set type of progress output ("auto", "plain", "tty") for the monitor. Use plain to show container output`)
for _, c := range children {
cmd.AddCommand(c.NewDebugger(&options))
}
return cmd
}

View File

@@ -8,7 +8,8 @@ import (
"text/tabwriter"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
@@ -33,25 +34,29 @@ func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
return err
}
dis, err := getInstanceOrDefault(ctx, dockerCli, opts.builder, "")
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
for _, di := range dis {
if di.Err != nil {
return err
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
out := make([][]*client.UsageInfo, len(dis))
out := make([][]*client.UsageInfo, len(nodes))
eg, ctx := errgroup.WithContext(ctx)
for i, di := range dis {
func(i int, di build.DriverInfo) {
for i, node := range nodes {
func(i int, node builder.Node) {
eg.Go(func() error {
if di.Driver != nil {
c, err := di.Driver.Client(ctx)
if node.Driver != nil {
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
@@ -64,7 +69,7 @@ func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
}
return nil
})
}(i, di)
}(i, node)
}
if err := eg.Wait(); err != nil {
@@ -111,6 +116,7 @@ func duCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runDiskUsage(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -7,13 +7,14 @@ import (
"os"
"strings"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/distribution/reference"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/distribution/reference"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -25,6 +26,7 @@ type createOptions struct {
builder string
files []string
tags []string
annotations []string
dryrun bool
actionAppend bool
progress string
@@ -90,47 +92,34 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
for i, s := range srcs {
if s.Ref == nil && s.Desc.MediaType == "" && s.Desc.Digest != "" {
if s.Ref == nil {
if defaultRepo == nil {
return errors.Errorf("multiple repositories specified, cannot infer repository for %q", args[i])
}
n, err := reference.ParseNormalizedNamed(*defaultRepo)
if err != nil {
return err
}
r, err := reference.WithDigest(n, s.Desc.Digest)
if err != nil {
return err
if s.Desc.MediaType == "" && s.Desc.Digest != "" {
r, err := reference.WithDigest(n, s.Desc.Digest)
if err != nil {
return err
}
srcs[i].Ref = r
sourceRefs = true
} else {
srcs[i].Ref = reference.TagNameOnly(n)
}
srcs[i].Ref = r
sourceRefs = true
}
}
ctx := appcontext.Context()
txn, release, err := storeutil.GetStore(dockerCli)
b, err := builder.New(dockerCli, builder.WithName(in.builder))
if err != nil {
return err
}
defer release()
var ng *store.NodeGroup
if in.builder != "" {
ng, err = storeutil.GetNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
} else {
ng, err = storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
}
imageopt, err := storeutil.GetImageConfig(dockerCli, ng)
imageopt, err := b.ImageOpt()
if err != nil {
return err
}
@@ -167,7 +156,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
dt, desc, err := r.Combine(ctx, srcs)
dt, desc, err := r.Combine(ctx, srcs, in.annotations)
if err != nil {
return err
}
@@ -182,7 +171,10 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressui.DisplayMode(in.progress))
if err != nil {
return err
}
eg, _ := errgroup.WithContext(ctx)
pw := progress.WithPrefix(printer, "internal", true)
@@ -284,6 +276,7 @@ func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
options.builder = *opts.Builder
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
@@ -292,6 +285,7 @@ func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
flags.BoolVar(&options.dryrun, "dry-run", false, "Show final image instead of pushing")
flags.BoolVar(&options.actionAppend, "append", false, "Append to existing manifest")
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "plain", "tty"). Use plain to show container output`)
flags.StringArrayVarP(&options.annotations, "annotation", "", []string{}, "Add annotation to the image")
return cmd
}

View File

@@ -1,8 +1,8 @@
package commands
import (
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
@@ -25,27 +25,11 @@ func runInspect(dockerCli command.Cli, in inspectOptions, name string) error {
return errors.Errorf("format and raw cannot be used together")
}
txn, release, err := storeutil.GetStore(dockerCli)
b, err := builder.New(dockerCli, builder.WithName(in.builder))
if err != nil {
return err
}
defer release()
var ng *store.NodeGroup
if in.builder != "" {
ng, err = storeutil.GetNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
} else {
ng, err = storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
}
imageopt, err := storeutil.GetImageConfig(dockerCli, ng)
imageopt, err := b.ImageOpt()
if err != nil {
return err
}
@@ -69,6 +53,7 @@ func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
options.builder = *rootOpts.Builder
return runInspect(dockerCli, options, args[0])
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -1,6 +1,7 @@
package commands
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
@@ -11,8 +12,9 @@ type RootOptions struct {
func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "imagetools",
Short: "Commands to work on images in registry",
Use: "imagetools",
Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(

View File

@@ -4,15 +4,19 @@ import (
"context"
"fmt"
"os"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/docker/go-units"
"github.com/moby/buildkit/util/appcontext"
"github.com/spf13/cobra"
)
@@ -25,71 +29,46 @@ type inspectOptions struct {
func runInspect(dockerCli command.Cli, in inspectOptions) error {
ctx := appcontext.Context()
txn, release, err := storeutil.GetStore(dockerCli)
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
defer release()
var ng *store.NodeGroup
if in.builder != "" {
ng, err = storeutil.GetNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
} else {
ng, err = storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
}
if ng == nil {
ng = &store.NodeGroup{
Name: "default",
Nodes: []store.Node{{
Name: "default",
Endpoint: "default",
}},
}
}
ngi := &nginfo{ng: ng}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
err = loadNodeGroupData(timeoutCtx, dockerCli, ngi)
var bootNgi *nginfo
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if in.bootstrap {
var ok bool
ok, err = boot(ctx, ngi)
ok, err = b.Boot(ctx)
if err != nil {
return err
}
bootNgi = ngi
if ok {
ngi = &nginfo{ng: ng}
err = loadNodeGroupData(ctx, dockerCli, ngi)
nodes, err = b.LoadNodes(timeoutCtx, builder.WithData())
}
}
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "Name:\t%s\n", ngi.ng.Name)
fmt.Fprintf(w, "Driver:\t%s\n", ngi.ng.Driver)
fmt.Fprintf(w, "Name:\t%s\n", b.Name)
fmt.Fprintf(w, "Driver:\t%s\n", b.Driver)
if !b.NodeGroup.LastActivity.IsZero() {
fmt.Fprintf(w, "Last Activity:\t%v\n", b.NodeGroup.LastActivity)
}
if err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if ngi.err != nil {
fmt.Fprintf(w, "Error:\t%s\n", ngi.err.Error())
} else if b.Err() != nil {
fmt.Fprintf(w, "Error:\t%s\n", b.Err().Error())
}
if err == nil {
fmt.Fprintln(w, "")
fmt.Fprintln(w, "Nodes:")
for i, n := range ngi.ng.Nodes {
for i, n := range nodes {
if i != 0 {
fmt.Fprintln(w, "")
}
@@ -104,21 +83,52 @@ func runInspect(dockerCli command.Cli, in inspectOptions) error {
fmt.Fprintf(w, "Driver Options:\t%s\n", strings.Join(driverOpts, " "))
}
if err := ngi.drivers[i].di.Err; err != nil {
if err := n.Err; err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if err := ngi.drivers[i].err; err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if bootNgi != nil && len(bootNgi.drivers) > i && bootNgi.drivers[i].err != nil {
fmt.Fprintf(w, "Error:\t%s\n", bootNgi.drivers[i].err.Error())
} else {
fmt.Fprintf(w, "Status:\t%s\n", ngi.drivers[i].info.Status)
fmt.Fprintf(w, "Status:\t%s\n", nodes[i].DriverInfo.Status)
if len(n.Flags) > 0 {
fmt.Fprintf(w, "Flags:\t%s\n", strings.Join(n.Flags, " "))
}
if ngi.drivers[i].version != "" {
fmt.Fprintf(w, "Buildkit:\t%s\n", ngi.drivers[i].version)
if nodes[i].Version != "" {
fmt.Fprintf(w, "Buildkit:\t%s\n", nodes[i].Version)
}
platforms := platformutil.FormatInGroups(n.Node.Platforms, n.Platforms)
if len(platforms) > 0 {
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platforms, ", "))
}
if debug.IsEnabled() {
fmt.Fprintf(w, "Features:\n")
features := nodes[i].Driver.Features(ctx)
featKeys := make([]string, 0, len(features))
for k := range features {
featKeys = append(featKeys, string(k))
}
sort.Strings(featKeys)
for _, k := range featKeys {
fmt.Fprintf(w, "\t%s:\t%t\n", k, features[driver.Feature(k)])
}
}
if len(nodes[i].Labels) > 0 {
fmt.Fprintf(w, "Labels:\n")
for _, k := range sortedKeys(nodes[i].Labels) {
v := nodes[i].Labels[k]
fmt.Fprintf(w, "\t%s:\t%s\n", k, v)
}
}
for ri, rule := range nodes[i].GCPolicy {
fmt.Fprintf(w, "GC Policy rule#%d:\n", ri)
fmt.Fprintf(w, "\tAll:\t%v\n", rule.All)
if len(rule.Filter) > 0 {
fmt.Fprintf(w, "\tFilters:\t%s\n", strings.Join(rule.Filter, " "))
}
if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
}
if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
}
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.FormatInGroups(n.Platforms, ngi.drivers[i].platforms), ", "))
}
}
}
@@ -142,6 +152,7 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runInspect(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
@@ -149,3 +160,14 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
return cmd
}
func sortedKeys(m map[string]string) []string {
s := make([]string, len(m))
i := 0
for k := range m {
s[i] = k
i++
}
sort.Strings(s)
return s
}

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -46,7 +47,8 @@ func installCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runInstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -4,14 +4,14 @@ import (
"context"
"fmt"
"io"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/store"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -32,52 +32,24 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
}
defer release()
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
current, err := storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
builders, err := builder.GetBuilders(dockerCli, txn)
if err != nil {
return err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
ll, err := txn.List()
if err != nil {
return err
}
builders := make([]*nginfo, len(ll))
for i, ng := range ll {
builders[i] = &nginfo{ng: ng}
}
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
for _, c := range contexts {
ngi := &nginfo{ng: &store.NodeGroup{
Name: c.Name,
Nodes: []store.Node{{
Name: c.Name,
Endpoint: c.Name,
}},
}}
// if a context has the same name as an instance from the store, do not
// add it to the builders list. An instance from the store takes
// precedence over context builders.
if hasNodeGroup(builders, ngi) {
continue
}
builders = append(builders, ngi)
}
eg, _ := errgroup.WithContext(ctx)
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {
func(b *nginfo) {
func(b *builder.Builder) {
eg.Go(func() error {
err = loadNodeGroupData(ctx, dockerCli, b)
if b.err == nil && err != nil {
b.err = err
}
_, _ = b.LoadNodes(timeoutCtx, builder.WithData())
return nil
})
}(b)
@@ -87,29 +59,15 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
return err
}
currentName := "default"
current, err := storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
if current != nil {
currentName = current.Name
if current.Name == "default" {
currentName = current.Nodes[0].Endpoint
}
}
w := tabwriter.NewWriter(dockerCli.Out(), 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "NAME/NODE\tDRIVER/ENDPOINT\tSTATUS\tBUILDKIT\tPLATFORMS\n")
currentSet := false
printErr := false
for _, b := range builders {
if !currentSet && b.ng.Name == currentName {
b.ng.Name += " *"
currentSet = true
if current.Name == b.Name {
b.Name += " *"
}
if ok := printngi(w, b); !ok {
if ok := printBuilder(w, b); !ok {
printErr = true
}
}
@@ -119,19 +77,12 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
if printErr {
_, _ = fmt.Fprintf(dockerCli.Err(), "\n")
for _, b := range builders {
if b.err != nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "Cannot load builder %s: %s\n", b.ng.Name, strings.TrimSpace(b.err.Error()))
if b.Err() != nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "Cannot load builder %s: %s\n", b.Name, strings.TrimSpace(b.Err().Error()))
} else {
for idx, n := range b.ng.Nodes {
d := b.drivers[idx]
var nodeErr string
if d.err != nil {
nodeErr = d.err.Error()
} else if d.di.Err != nil {
nodeErr = d.di.Err.Error()
}
if nodeErr != "" {
_, _ = fmt.Fprintf(dockerCli.Err(), "Failed to get status for %s (%s): %s\n", b.ng.Name, n.Name, strings.TrimSpace(nodeErr))
for _, d := range b.Nodes() {
if d.Err != nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "Failed to get status for %s (%s): %s\n", b.Name, d.Name, strings.TrimSpace(d.Err.Error()))
}
}
}
@@ -141,26 +92,25 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
return nil
}
func printngi(w io.Writer, ngi *nginfo) (ok bool) {
func printBuilder(w io.Writer, b *builder.Builder) (ok bool) {
ok = true
var err string
if ngi.err != nil {
if b.Err() != nil {
ok = false
err = "error"
}
fmt.Fprintf(w, "%s\t%s\t%s\t\t\n", ngi.ng.Name, ngi.ng.Driver, err)
if ngi.err == nil {
for idx, n := range ngi.ng.Nodes {
d := ngi.drivers[idx]
fmt.Fprintf(w, "%s\t%s\t%s\t\t\n", b.Name, b.Driver, err)
if b.Err() == nil {
for _, n := range b.Nodes() {
var status string
if d.info != nil {
status = d.info.Status.String()
if n.DriverInfo != nil {
status = n.DriverInfo.Status.String()
}
if d.err != nil || d.di.Err != nil {
if n.Err != nil {
ok = false
fmt.Fprintf(w, " %s\t%s\t%s\t\t\n", n.Name, n.Endpoint, "error")
} else {
fmt.Fprintf(w, " %s\t%s\t%s\t%s\t%s\n", n.Name, n.Endpoint, status, d.version, strings.Join(platformutil.FormatInGroups(n.Platforms, d.platforms), ", "))
fmt.Fprintf(w, " %s\t%s\t%s\t%s\t%s\n", n.Name, n.Endpoint, status, n.Version, strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
}
}
}
@@ -177,6 +127,7 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runLs(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -1,48 +0,0 @@
package commands
import (
"fmt"
"io"
"log"
"os"
"github.com/docker/buildx/build"
"github.com/docker/docker/api/types/versions"
"github.com/moby/buildkit/frontend/subrequests"
"github.com/moby/buildkit/frontend/subrequests/outline"
"github.com/moby/buildkit/frontend/subrequests/targets"
)
func printResult(f *build.PrintFunc, res map[string]string) error {
switch f.Name {
case "outline":
return printValue(outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res)
case "targets":
return printValue(targets.PrintTargets, targets.SubrequestsTargetsDefinition.Version, f.Format, res)
case "subrequests.describe":
return printValue(subrequests.PrintDescribe, subrequests.SubrequestsDescribeDefinition.Version, f.Format, res)
default:
if dt, ok := res["result.txt"]; ok {
fmt.Print(dt)
} else {
log.Printf("%s %+v", f, res)
}
}
return nil
}
type printFunc func([]byte, io.Writer) error
func printValue(printer printFunc, version string, format string, res map[string]string) error {
if format == "json" {
fmt.Fprintln(os.Stdout, res["result.json"])
return nil
}
if res["version"] != "" && versions.LessThan(version, res["version"]) && res["result.txt"] != "" {
// structure is too new and we don't know how to print it
fmt.Fprint(os.Stdout, res["result.txt"])
return nil
}
return printer([]byte(res["result.json"]), os.Stdout)
}

View File

@@ -7,7 +7,8 @@ import (
"text/tabwriter"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
@@ -54,14 +55,18 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
return nil
}
dis, err := getInstanceOrDefault(ctx, dockerCli, opts.builder, "")
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
for _, di := range dis {
if di.Err != nil {
return err
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
@@ -90,11 +95,11 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
}()
eg, ctx := errgroup.WithContext(ctx)
for _, di := range dis {
func(di build.DriverInfo) {
for _, node := range nodes {
func(node builder.Node) {
eg.Go(func() error {
if di.Driver != nil {
c, err := di.Driver.Client(ctx)
if node.Driver != nil {
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
@@ -109,7 +114,7 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
}
return nil
})
}(di)
}(node)
}
if err := eg.Wait(); err != nil {
@@ -135,6 +140,7 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runPrune(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -5,8 +5,10 @@ import (
"fmt"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
@@ -44,41 +46,33 @@ func runRm(dockerCli command.Cli, in rmOptions) error {
return rmAllInactive(ctx, txn, dockerCli, in)
}
var ng *store.NodeGroup
if in.builder != "" {
ng, err = storeutil.GetNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
} else {
ng, err = storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
}
if ng == nil {
return nil
}
ctxbuilders, err := dockerCli.ContextStore().List()
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
for _, cb := range ctxbuilders {
if ng.Driver == "docker" && len(ng.Nodes) == 1 && ng.Nodes[0].Endpoint == cb.Name {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb.Name)
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
err1 := rm(ctx, dockerCli, in, ng)
if err := txn.Remove(ng.Name); err != nil {
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
err1 := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
if err1 != nil {
return err1
}
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", ng.Name)
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.Name)
return nil
}
@@ -99,6 +93,7 @@ func rmCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runRm(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
@@ -110,61 +105,53 @@ func rmCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
return cmd
}
func rm(ctx context.Context, dockerCli command.Cli, in rmOptions, ng *store.NodeGroup) error {
dis, err := driversForNodeGroup(ctx, dockerCli, ng, "")
if err != nil {
return err
}
for _, di := range dis {
if di.Driver == nil {
func rm(ctx context.Context, nodes []builder.Node, in rmOptions) (err error) {
for _, node := range nodes {
if node.Driver == nil {
continue
}
// Do not stop the buildkitd daemon when --keep-daemon is provided
if !in.keepDaemon {
if err := di.Driver.Stop(ctx, true); err != nil {
if err := node.Driver.Stop(ctx, true); err != nil {
return err
}
}
if err := di.Driver.Rm(ctx, true, !in.keepState, !in.keepDaemon); err != nil {
if err := node.Driver.Rm(ctx, true, !in.keepState, !in.keepDaemon); err != nil {
return err
}
if di.Err != nil {
err = di.Err
if node.Err != nil {
err = node.Err
}
}
return err
}
func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, in rmOptions) error {
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
ll, err := txn.List()
builders, err := builder.GetBuilders(dockerCli, txn)
if err != nil {
return err
}
builders := make([]*nginfo, len(ll))
for i, ng := range ll {
builders[i] = &nginfo{ng: ng}
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
eg, _ := errgroup.WithContext(ctx)
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {
func(b *nginfo) {
func(b *builder.Builder) {
eg.Go(func() error {
if err := loadNodeGroupData(ctx, dockerCli, b); err != nil {
return errors.Wrapf(err, "cannot load %s", b.ng.Name)
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if err != nil {
return errors.Wrapf(err, "cannot load %s", b.Name)
}
if b.ng.Dynamic {
if b.Dynamic {
return nil
}
if b.inactive() {
rmerr := rm(ctx, dockerCli, in, b.ng)
if err := txn.Remove(b.ng.Name); err != nil {
if b.Inactive() {
rmerr := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.ng.Name)
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.Name)
return rmerr
}
return nil

View File

@@ -3,12 +3,16 @@ package commands
import (
"os"
debugcmd "github.com/docker/buildx/commands/debug"
imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/logutil"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/plugin"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
@@ -22,6 +26,9 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
Annotations: map[string]string{
annotation.CodeDelimiter: `"`,
},
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
}
if isPlugin {
cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
@@ -35,6 +42,11 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
cmd.TraverseChildren = true
cmd.DisableFlagsInUseLine = true
cli.DisableFlagsInUseLine(cmd)
// DEBUG=1 should perform the same as --debug at the docker root level
if debug.IsEnabled() {
debug.Enable()
}
}
logrus.SetFormatter(&logutil.Formatter{})
@@ -47,17 +59,6 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
"using default config store",
))
// filter out useless commandConn.CloseWrite warning message that can occur
// when listing builder instances with "buildx ls" for those that are
// unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
// https://github.com/docker/cli/blob/3fb4fb83dfb5db0c0753a8316f21aea54dab32c5/cli/connhelper/commandconn/commandconn.go#L203-L214
logrus.AddHook(logutil.NewFilter([]logrus.Level{
logrus.WarnLevel,
},
"commandConn.CloseWrite:",
"commandConn.CloseRead:",
))
addCommands(cmd, dockerCli)
return cmd
}
@@ -71,7 +72,7 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
rootFlags(opts, cmd.PersistentFlags())
cmd.AddCommand(
buildCmd(dockerCli, opts),
buildCmd(dockerCli, opts, nil),
bakeCmd(dockerCli, opts),
createCmd(dockerCli),
rmCmd(dockerCli, opts),
@@ -86,6 +87,17 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
)
if isExperimental() {
cmd.AddCommand(debugcmd.RootCmd(dockerCli,
newDebuggableBuild(dockerCli, opts),
))
remote.AddControllerCommands(cmd, dockerCli)
}
cmd.RegisterFlagCompletionFunc( //nolint:errcheck
"builder",
completion.BuilderNames(dockerCli),
)
}
func rootFlags(options *rootOptions, flags *pflag.FlagSet) {

View File

@@ -3,8 +3,8 @@ package commands
import (
"context"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
@@ -18,32 +18,19 @@ type stopOptions struct {
func runStop(dockerCli command.Cli, in stopOptions) error {
ctx := appcontext.Context()
txn, release, err := storeutil.GetStore(dockerCli)
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
defer release()
if in.builder != "" {
ng, err := storeutil.GetNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
if err := stop(ctx, dockerCli, ng); err != nil {
return err
}
return nil
}
ng, err := storeutil.GetCurrentInstance(txn, dockerCli)
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
if ng != nil {
return stop(ctx, dockerCli, ng)
}
return stopCurrent(ctx, dockerCli)
return stop(ctx, nodes)
}
func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
@@ -60,42 +47,21 @@ func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runStop(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
return cmd
}
func stop(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup) error {
dis, err := driversForNodeGroup(ctx, dockerCli, ng, "")
if err != nil {
return err
}
for _, di := range dis {
if di.Driver != nil {
if err := di.Driver.Stop(ctx, true); err != nil {
func stop(ctx context.Context, nodes []builder.Node) (err error) {
for _, node := range nodes {
if node.Driver != nil {
if err := node.Driver.Stop(ctx, true); err != nil {
return err
}
}
if di.Err != nil {
err = di.Err
}
}
return err
}
func stopCurrent(ctx context.Context, dockerCli command.Cli) error {
dis, err := getDefaultDrivers(ctx, dockerCli, false, "")
if err != nil {
return err
}
for _, di := range dis {
if di.Driver != nil {
if err := di.Driver.Stop(ctx, true); err != nil {
return err
}
}
if di.Err != nil {
err = di.Err
if node.Err != nil {
err = node.Err
}
}
return err

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -52,7 +53,8 @@ func uninstallCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runUninstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -4,6 +4,8 @@ import (
"os"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
@@ -29,14 +31,11 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Errorf("run `docker context use default` to switch to default context")
}
if in.builder == "default" || in.builder == dockerCli.CurrentContext() {
ep, err := storeutil.GetCurrentEndpoint(dockerCli)
ep, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
if err := txn.SetCurrent(ep, "", false, false); err != nil {
return err
}
return nil
return txn.SetCurrent(ep, "", false, false)
}
list, err := dockerCli.ContextStore().List()
if err != nil {
@@ -52,15 +51,11 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Wrapf(err, "failed to find instance %q", in.builder)
}
ep, err := storeutil.GetCurrentEndpoint(dockerCli)
ep, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
if err := txn.SetCurrent(ep, in.builder, in.isGlobal, in.isDefault); err != nil {
return err
}
return nil
return txn.SetCurrent(ep, in.builder, in.isGlobal, in.isDefault)
}
func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
@@ -77,6 +72,7 @@ func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runUse(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()

View File

@@ -1,487 +0,0 @@
package commands
import (
"context"
"net/url"
"os"
"strings"
"github.com/docker/buildx/build"
"github.com/docker/buildx/driver"
ctxkube "github.com/docker/buildx/driver/kubernetes/context"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/context/docker"
ctxstore "github.com/docker/cli/cli/context/store"
dopts "github.com/docker/cli/opts"
dockerclient "github.com/docker/docker/client"
"github.com/moby/buildkit/util/grpcerrors"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc/codes"
"k8s.io/client-go/tools/clientcmd"
)
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
de, err := storeutil.GetDockerEndpoint(dockerCli, ep)
if err == nil && de != "" {
if ep == "default" {
return de, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// validateBuildkitEndpoint validates that endpoint is a valid buildkit host
func validateBuildkitEndpoint(ep string) (string, error) {
if err := remoteutil.IsValidEndpoint(ep); err != nil {
return "", err
}
return ep, nil
}
// driversForNodeGroup returns drivers for a nodegroup instance
func driversForNodeGroup(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, contextPathHash string) ([]build.DriverInfo, error) {
eg, _ := errgroup.WithContext(ctx)
dis := make([]build.DriverInfo, len(ng.Nodes))
var f driver.Factory
if ng.Driver != "" {
var err error
f, err = driver.GetFactory(ng.Driver, true)
if err != nil {
return nil, err
}
} else {
// empty driver means nodegroup was implicitly created as a default
// driver for a docker context and allows falling back to a
// docker-container driver for older daemon that doesn't support
// buildkit (< 18.06).
ep := ng.Nodes[0].Endpoint
dockerapi, err := clientForEndpoint(dockerCli, ep)
if err != nil {
return nil, err
}
// check if endpoint is healthy is needed to determine the driver type.
// if this fails then can't continue with driver selection.
if _, err = dockerapi.Ping(ctx); err != nil {
return nil, err
}
f, err = driver.GetDefaultFactory(ctx, ep, dockerapi, false)
if err != nil {
return nil, err
}
ng.Driver = f.Name()
}
imageopt, err := storeutil.GetImageConfig(dockerCli, ng)
if err != nil {
return nil, err
}
for i, n := range ng.Nodes {
func(i int, n store.Node) {
eg.Go(func() error {
di := build.DriverInfo{
Name: n.Name,
Platform: n.Platforms,
ProxyConfig: storeutil.GetProxyConfig(dockerCli),
}
defer func() {
dis[i] = di
}()
dockerapi, err := clientForEndpoint(dockerCli, n.Endpoint)
if err != nil {
di.Err = err
return nil
}
// TODO: replace the following line with dockerclient.WithAPIVersionNegotiation option in clientForEndpoint
dockerapi.NegotiateAPIVersion(ctx)
contextStore := dockerCli.ContextStore()
var kcc driver.KubeClientConfig
kcc, err = configFromContext(n.Endpoint, contextStore)
if err != nil {
// err is returned if n.Endpoint is non-context name like "unix:///var/run/docker.sock".
// try again with name="default".
// FIXME: n should retain real context name.
kcc, err = configFromContext("default", contextStore)
if err != nil {
logrus.Error(err)
}
}
tryToUseKubeConfigInCluster := false
if kcc == nil {
tryToUseKubeConfigInCluster = true
} else {
if _, err := kcc.ClientConfig(); err != nil {
tryToUseKubeConfigInCluster = true
}
}
if tryToUseKubeConfigInCluster {
kccInCluster := driver.KubeClientConfigInCluster{}
if _, err := kccInCluster.ClientConfig(); err == nil {
logrus.Debug("using kube config in cluster")
kcc = kccInCluster
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, f, n.Endpoint, dockerapi, imageopt.Auth, kcc, n.Flags, n.Files, n.DriverOpts, n.Platforms, contextPathHash)
if err != nil {
di.Err = err
return nil
}
di.Driver = d
di.ImageOpt = imageopt
return nil
})
}(i, n)
}
if err := eg.Wait(); err != nil {
return nil, err
}
return dis, nil
}
func configFromContext(endpointName string, s ctxstore.Reader) (clientcmd.ClientConfig, error) {
if strings.HasPrefix(endpointName, "kubernetes://") {
u, _ := url.Parse(endpointName)
if kubeconfig := u.Query().Get("kubeconfig"); kubeconfig != "" {
_ = os.Setenv(clientcmd.RecommendedConfigPathEnvVar, kubeconfig)
}
rules := clientcmd.NewDefaultClientConfigLoadingRules()
apiConfig, err := rules.Load()
if err != nil {
return nil, err
}
return clientcmd.NewDefaultClientConfig(*apiConfig, &clientcmd.ConfigOverrides{}), nil
}
return ctxkube.ConfigFromContext(endpointName, s)
}
// clientForEndpoint returns a docker client for an endpoint
func clientForEndpoint(dockerCli command.Cli, name string) (dockerclient.APIClient, error) {
list, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, l := range list {
if l.Name == name {
dep, ok := l.Endpoints["docker"]
if !ok {
return nil, errors.Errorf("context %q does not have a Docker endpoint", name)
}
epm, ok := dep.(docker.EndpointMeta)
if !ok {
return nil, errors.Errorf("endpoint %q is not of type EndpointMeta, %T", dep, dep)
}
ep, err := docker.WithTLSData(dockerCli.ContextStore(), name, epm)
if err != nil {
return nil, err
}
clientOpts, err := ep.ClientOpts()
if err != nil {
return nil, err
}
return dockerclient.NewClientWithOpts(clientOpts...)
}
}
ep := docker.Endpoint{
EndpointMeta: docker.EndpointMeta{
Host: name,
},
}
clientOpts, err := ep.ClientOpts()
if err != nil {
return nil, err
}
return dockerclient.NewClientWithOpts(clientOpts...)
}
func getInstanceOrDefault(ctx context.Context, dockerCli command.Cli, instance, contextPathHash string) ([]build.DriverInfo, error) {
var defaultOnly bool
if instance == "default" && instance != dockerCli.CurrentContext() {
return nil, errors.Errorf("use `docker --context=default buildx` to switch to default context")
}
if instance == "default" || instance == dockerCli.CurrentContext() {
instance = ""
defaultOnly = true
}
list, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, l := range list {
if l.Name == instance {
return nil, errors.Errorf("use `docker --context=%s buildx` to switch to context %s", instance, instance)
}
}
if instance != "" {
return getInstanceByName(ctx, dockerCli, instance, contextPathHash)
}
return getDefaultDrivers(ctx, dockerCli, defaultOnly, contextPathHash)
}
func getInstanceByName(ctx context.Context, dockerCli command.Cli, instance, contextPathHash string) ([]build.DriverInfo, error) {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
ng, err := txn.NodeGroupByName(instance)
if err != nil {
return nil, err
}
return driversForNodeGroup(ctx, dockerCli, ng, contextPathHash)
}
// getDefaultDrivers returns drivers based on current cli config
func getDefaultDrivers(ctx context.Context, dockerCli command.Cli, defaultOnly bool, contextPathHash string) ([]build.DriverInfo, error) {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
if !defaultOnly {
ng, err := storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return nil, err
}
if ng != nil {
return driversForNodeGroup(ctx, dockerCli, ng, contextPathHash)
}
}
imageopt, err := storeutil.GetImageConfig(dockerCli, nil)
if err != nil {
return nil, err
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_default", nil, "", dockerCli.Client(), imageopt.Auth, nil, nil, nil, nil, nil, contextPathHash)
if err != nil {
return nil, err
}
return []build.DriverInfo{
{
Name: "default",
Driver: d,
ImageOpt: imageopt,
ProxyConfig: storeutil.GetProxyConfig(dockerCli),
},
}, nil
}
func loadInfoData(ctx context.Context, d *dinfo) error {
if d.di.Driver == nil {
return nil
}
info, err := d.di.Driver.Info(ctx)
if err != nil {
return err
}
d.info = info
if info.Status == driver.Running {
c, err := d.di.Driver.Client(ctx)
if err != nil {
return err
}
workers, err := c.ListWorkers(ctx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
for _, w := range workers {
d.platforms = append(d.platforms, w.Platforms...)
}
d.platforms = platformutil.Dedupe(d.platforms)
inf, err := c.Info(ctx)
if err != nil {
if st, ok := grpcerrors.AsGRPCStatus(err); ok && st.Code() == codes.Unimplemented {
d.version, err = d.di.Driver.Version(ctx)
if err != nil {
return errors.Wrap(err, "getting version")
}
}
} else {
d.version = inf.BuildkitVersion.Version
}
}
return nil
}
func loadNodeGroupData(ctx context.Context, dockerCli command.Cli, ngi *nginfo) error {
eg, _ := errgroup.WithContext(ctx)
dis, err := driversForNodeGroup(ctx, dockerCli, ngi.ng, "")
if err != nil {
return err
}
ngi.drivers = make([]dinfo, len(dis))
for i, di := range dis {
d := di
ngi.drivers[i].di = &d
func(d *dinfo) {
eg.Go(func() error {
if err := loadInfoData(ctx, d); err != nil {
d.err = err
}
return nil
})
}(&ngi.drivers[i])
}
if eg.Wait(); err != nil {
return err
}
kubernetesDriverCount := 0
for _, di := range ngi.drivers {
if di.info != nil && len(di.info.DynamicNodes) > 0 {
kubernetesDriverCount++
}
}
isAllKubernetesDrivers := len(ngi.drivers) == kubernetesDriverCount
if isAllKubernetesDrivers {
var drivers []dinfo
var dynamicNodes []store.Node
for _, di := range ngi.drivers {
// dynamic nodes are used in Kubernetes driver.
// Kubernetes pods are dynamically mapped to BuildKit Nodes.
if di.info != nil && len(di.info.DynamicNodes) > 0 {
for i := 0; i < len(di.info.DynamicNodes); i++ {
// all []dinfo share *build.DriverInfo and *driver.Info
diClone := di
if pl := di.info.DynamicNodes[i].Platforms; len(pl) > 0 {
diClone.platforms = pl
}
drivers = append(drivers, di)
}
dynamicNodes = append(dynamicNodes, di.info.DynamicNodes...)
}
}
// not append (remove the static nodes in the store)
ngi.ng.Nodes = dynamicNodes
ngi.drivers = drivers
ngi.ng.Dynamic = true
}
return nil
}
func hasNodeGroup(list []*nginfo, ngi *nginfo) bool {
for _, l := range list {
if ngi.ng.Name == l.ng.Name {
return true
}
}
return false
}
func dockerAPI(dockerCli command.Cli) *api {
return &api{dockerCli: dockerCli}
}
type api struct {
dockerCli command.Cli
}
func (a *api) DockerAPI(name string) (dockerclient.APIClient, error) {
if name == "" {
name = a.dockerCli.CurrentContext()
}
return clientForEndpoint(a.dockerCli, name)
}
type dinfo struct {
di *build.DriverInfo
info *driver.Info
platforms []specs.Platform
version string
err error
}
type nginfo struct {
ng *store.NodeGroup
drivers []dinfo
err error
}
// inactive checks if all nodes are inactive for this builder
func (n *nginfo) inactive() bool {
for idx := range n.ng.Nodes {
d := n.drivers[idx]
if d.info != nil && d.info.Status == driver.Running {
return false
}
}
return true
}
func boot(ctx context.Context, ngi *nginfo) (bool, error) {
toBoot := make([]int, 0, len(ngi.drivers))
for i, d := range ngi.drivers {
if d.err != nil || d.di.Err != nil || d.di.Driver == nil || d.info == nil {
continue
}
if d.info.Status != driver.Running {
toBoot = append(toBoot, i)
}
}
if len(toBoot) == 0 {
return false, nil
}
printer := progress.NewPrinter(context.TODO(), os.Stderr, os.Stderr, "auto")
baseCtx := ctx
eg, _ := errgroup.WithContext(ctx)
for _, idx := range toBoot {
func(idx int) {
eg.Go(func() error {
pw := progress.WithPrefix(printer, ngi.ng.Nodes[idx].Name, len(toBoot) > 1)
_, err := driver.Boot(ctx, baseCtx, ngi.drivers[idx].di.Driver, pw)
if err != nil {
ngi.drivers[idx].err = err
}
return nil
})
}(idx)
}
err := eg.Wait()
err1 := printer.Wait()
if err == nil {
err = err1
}
return true, err
}

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -23,6 +24,7 @@ func versionCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runVersion(dockerCli)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

280
controller/build/build.go Normal file
View File

@@ -0,0 +1,280 @@
package build
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"sync"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
)
const defaultTargetName = "default"
// RunBuild runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
}
contexts := map[string]build.NamedContext{}
for name, path := range in.NamedContexts {
contexts[name] = build.NamedContext{Path: path}
}
opts := build.Options{
Inputs: build.Inputs{
ContextPath: in.ContextPath,
DockerfilePath: in.DockerfileName,
InStream: inStream,
NamedContexts: contexts,
},
Ref: in.Ref,
BuildArgs: in.BuildArgs,
CgroupParent: in.CgroupParent,
ExtraHosts: in.ExtraHosts,
Labels: in.Labels,
NetworkMode: in.NetworkMode,
NoCache: in.NoCache,
NoCacheFilter: in.NoCacheFilter,
Pull: in.Pull,
ShmSize: dockeropts.MemBytes(in.ShmSize),
Tags: in.Tags,
Target: in.Target,
Ulimits: controllerUlimitOpt2DockerUlimit(in.Ulimits),
GroupRef: in.GroupRef,
}
platforms, err := platformutil.Parse(in.Platforms)
if err != nil {
return nil, nil, err
}
opts.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(dockerConfig, nil))
secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, secrets)
sshSpecs := in.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(in.ContextPath) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, ssh)
outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil {
return nil, nil, err
}
if in.ExportPush {
if in.ExportLoad {
return nil, nil, errors.Errorf("push and load may not be set together at the moment")
}
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
}}
} else {
switch outputs[0].Type {
case "image":
outputs[0].Attrs["push"] = "true"
default:
return nil, nil, errors.Errorf("push and %q output can't be used together", outputs[0].Type)
}
}
}
if in.ExportLoad {
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "docker",
Attrs: map[string]string{},
}}
} else {
switch outputs[0].Type {
case "docker":
default:
return nil, nil, errors.Errorf("load and %q output can't be used together", outputs[0].Type)
}
}
}
annotations, err := buildflags.ParseAnnotations(in.Annotations)
if err != nil {
return nil, nil, err
}
for _, o := range outputs {
for k, v := range annotations {
o.Attrs[k.String()] = v
}
}
opts.Exports = outputs
opts.CacheFrom = controllerapi.CreateCaches(in.CacheFrom)
opts.CacheTo = controllerapi.CreateCaches(in.CacheTo)
opts.Attests = controllerapi.CreateAttestations(in.Attests)
opts.SourcePolicy = in.SourcePolicy
allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil {
return nil, nil, err
}
opts.Allow = allow
if in.PrintFunc != nil {
opts.PrintFunc = &build.PrintFunc{
Name: in.PrintFunc.Name,
Format: in.PrintFunc.Format,
}
}
// key string used for kubernetes "sticky" mode
contextPathHash, err := filepath.Abs(in.ContextPath)
if err != nil {
contextPathHash = in.ContextPath
}
// TODO: this should not be loaded this side of the controller api
b, err := builder.New(dockerCli,
builder.WithName(in.Builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return nil, nil, err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return nil, nil, err
}
resp, res, err := buildTargets(ctx, dockerCli, b.NodeGroup, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
err = wrapBuildError(err, false)
if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, err
}
return resp, res, nil
}
// buildTargets runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func buildTargets(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, nodes []builder.Node, opts map[string]build.Options, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
var res *build.ResultHandle
var resp map[string]*client.SolveResponse
var err error
if generateResult {
var mu sync.Mutex
var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock()
defer mu.Unlock()
if res == nil || driverIndex < idx {
idx, res = driverIndex, gotRes
}
})
} else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
}
if err != nil {
return nil, res, err
}
return resp[defaultTargetName], res, err
}
func wrapBuildError(err error, bake bool) error {
if err == nil {
return nil
}
st, ok := grpcerrors.AsGRPCStatus(err)
if ok {
if st.Code() == codes.Unimplemented && strings.Contains(st.Message(), "unsupported frontend capability moby.buildkit.frontend.contexts") {
msg := "current frontend does not support --build-context."
if bake {
msg = "current frontend does not support defining additional contexts for targets."
}
msg += " Named contexts are supported since Dockerfile v1.4. Use #syntax directive in Dockerfile or update to latest BuildKit."
return &wrapped{err, msg}
}
}
return err
}
type wrapped struct {
err error
msg string
}
func (w *wrapped) Error() string {
return w.msg
}
func (w *wrapped) Unwrap() error {
return w.err
}
func updateLastActivity(dockerCli command.Cli, ng *store.NodeGroup) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
return txn.UpdateLastActivity(ng)
}
func controllerUlimitOpt2DockerUlimit(u *controllerapi.UlimitOpt) *dockeropts.UlimitOpt {
if u == nil {
return nil
}
values := make(map[string]*units.Ulimit)
for k, v := range u.Values {
values[k] = &units.Ulimit{
Name: v.Name,
Hard: v.Hard,
Soft: v.Soft,
}
}
return dockeropts.NewUlimitOpt(&values)
}

View File

@@ -0,0 +1,32 @@
package control
import (
"context"
"io"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
)
type BuildxController interface {
Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process.
// If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error
Close() error
List(ctx context.Context) (refs []string, _ error)
Disconnect(ctx context.Context, ref string) error
ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error)
DisconnectProcess(ctx context.Context, ref, pid string) error
Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error)
}
type ControlOptions struct {
ServerConfig string
Root string
Detach bool
}

36
controller/controller.go Normal file
View File

@@ -0,0 +1,36 @@
package controller
import (
"context"
"fmt"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/controller/local"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
)
func NewController(ctx context.Context, opts control.ControlOptions, dockerCli command.Cli, pw progress.Writer) (control.BuildxController, error) {
var name string
if opts.Detach {
name = "remote"
} else {
name = "local"
}
var c control.BuildxController
err := progress.Wrap(fmt.Sprintf("[internal] connecting to %s controller", name), pw.Write, func(l progress.SubLogger) (err error) {
if opts.Detach {
c, err = remote.NewRemoteBuildxController(ctx, dockerCli, opts, l)
} else {
c = local.NewLocalBuildxController(ctx, dockerCli, l)
}
return err
})
if err != nil {
return nil, errors.Wrap(err, "failed to start buildx controller")
}
return c, nil
}

View File

@@ -0,0 +1,34 @@
package errdefs
import (
"github.com/containerd/typeurl/v2"
"github.com/moby/buildkit/util/grpcerrors"
)
func init() {
typeurl.Register((*Build)(nil), "github.com/docker/buildx", "errdefs.Build+json")
}
type BuildError struct {
Build
error
}
func (e *BuildError) Unwrap() error {
return e.error
}
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return &e.Build
}
func WrapBuild(err error, ref string) error {
if err == nil {
return nil
}
return &BuildError{Build: Build{Ref: ref}, error: err}
}
func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: *b}
}

View File

@@ -0,0 +1,77 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: errdefs.proto
package errdefs
import (
fmt "fmt"
proto "github.com/gogo/protobuf/proto"
_ "github.com/moby/buildkit/solver/pb"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct {
Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Build) Reset() { *m = Build{} }
func (m *Build) String() string { return proto.CompactTextString(m) }
func (*Build) ProtoMessage() {}
func (*Build) Descriptor() ([]byte, []int) {
return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
}
var xxx_messageInfo_Build proto.InternalMessageInfo
func (m *Build) GetRef() string {
if m != nil {
return m.Ref
}
return ""
}
func init() {
proto.RegisterType((*Build)(nil), "errdefs.Build")
}
func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var fileDescriptor_689dc58a5060aff5 = []byte{
// 111 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
}

View File

@@ -0,0 +1,9 @@
syntax = "proto3";
package errdefs;
import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build {
string Ref = 1;
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -0,0 +1,146 @@
package local
import (
"context"
"io"
"sync/atomic"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{
dockerCli: dockerCli,
ref: "local",
processes: processes.NewManager(),
}
}
type buildConfig struct {
// TODO: these two structs should be merged
// Discussion: https://github.com/docker/buildx/pull/1640#discussion_r1113279719
resultCtx *build.ResultHandle
buildOptions *controllerapi.BuildOptions
}
type localController struct {
dockerCli command.Cli
ref string
buildConfig buildConfig
processes *processes.Manager
buildOnGoing atomic.Bool
}
func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, errors.New("build ongoing")
}
defer b.buildOnGoing.Store(false)
resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil {
b.buildConfig = buildConfig{
resultCtx: res,
buildOptions: &options,
}
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, b.ref)
}
}
if buildErr != nil {
return "", nil, buildErr
}
return b.ref, resp, nil
}
func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return b.processes.ListProcesses(), nil
}
func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
return b.processes.DeleteProcess(pid)
}
func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses()
}
func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
proc, ok := b.processes.Get(pid)
if !ok {
// Start a new process.
if b.buildConfig.resultCtx == nil {
return errors.New("no build result is registered")
}
var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select {
case <-ioCancelledCh:
return errors.Errorf("io cancelled")
case err := <-proc.Done():
return err
case <-ctx.Done():
return ctx.Err()
}
}
func (b *localController) Kill(context.Context) error {
b.Close()
return nil
}
func (b *localController) Close() error {
b.cancelRunningProcesses()
if b.buildConfig.resultCtx != nil {
b.buildConfig.resultCtx.Done()
}
// TODO: cancel ongoing builds?
return nil
}
func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.ref}, nil
}
func (b *localController) Disconnect(ctx context.Context, key string) error {
b.Close()
return nil
}
func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
}

20
controller/pb/attest.go Normal file
View File

@@ -0,0 +1,20 @@
package pb
func CreateAttestations(attests []*Attest) map[string]*string {
result := map[string]*string{}
for _, attest := range attests {
// ignore duplicates
if _, ok := result[attest.Type]; ok {
continue
}
if attest.Disabled {
result[attest.Type] = nil
continue
}
attrs := attest.Attrs
result[attest.Type] = &attrs
}
return result
}

21
controller/pb/cache.go Normal file
View File

@@ -0,0 +1,21 @@
package pb
import "github.com/moby/buildkit/client"
func CreateCaches(entries []*CacheOptionsEntry) []client.CacheOptionsEntry {
var outs []client.CacheOptionsEntry
if len(entries) == 0 {
return nil
}
for _, entry := range entries {
out := client.CacheOptionsEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
outs = append(outs, out)
}
return outs
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,248 @@
syntax = "proto3";
package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "pb";
service Controller {
rpc Build(BuildRequest) returns (BuildResponse);
rpc Inspect(InspectRequest) returns (InspectResponse);
rpc Status(StatusRequest) returns (stream StatusResponse);
rpc Input(stream InputMessage) returns (InputResponse);
rpc Invoke(stream Message) returns (stream Message);
rpc List(ListRequest) returns (ListResponse);
rpc Disconnect(DisconnectRequest) returns (DisconnectResponse);
rpc Info(InfoRequest) returns (InfoResponse);
rpc ListProcesses(ListProcessesRequest) returns (ListProcessesResponse);
rpc DisconnectProcess(DisconnectProcessRequest) returns (DisconnectProcessResponse);
}
message ListProcessesRequest {
string Ref = 1;
}
message ListProcessesResponse {
repeated ProcessInfo Infos = 1;
}
message ProcessInfo {
string ProcessID = 1;
InvokeConfig InvokeConfig = 2;
}
message DisconnectProcessRequest {
string Ref = 1;
string ProcessID = 2;
}
message DisconnectProcessResponse {
}
message BuildRequest {
string Ref = 1;
BuildOptions Options = 2;
}
message BuildOptions {
string ContextPath = 1;
string DockerfileName = 2;
PrintFunc PrintFunc = 3;
map<string, string> NamedContexts = 4;
repeated string Allow = 5;
repeated Attest Attests = 6;
map<string, string> BuildArgs = 7;
repeated CacheOptionsEntry CacheFrom = 8;
repeated CacheOptionsEntry CacheTo = 9;
string CgroupParent = 10;
repeated ExportEntry Exports = 11;
repeated string ExtraHosts = 12;
map<string, string> Labels = 13;
string NetworkMode = 14;
repeated string NoCacheFilter = 15;
repeated string Platforms = 16;
repeated Secret Secrets = 17;
int64 ShmSize = 18;
repeated SSH SSH = 19;
repeated string Tags = 20;
string Target = 21;
UlimitOpt Ulimits = 22;
string Builder = 23;
bool NoCache = 24;
bool Pull = 25;
bool ExportPush = 26;
bool ExportLoad = 27;
moby.buildkit.v1.sourcepolicy.Policy SourcePolicy = 28;
string Ref = 29;
string GroupRef = 30;
repeated string Annotations = 31;
}
message ExportEntry {
string Type = 1;
map<string, string> Attrs = 2;
string Destination = 3;
}
message CacheOptionsEntry {
string Type = 1;
map<string, string> Attrs = 2;
}
message Attest {
string Type = 1;
bool Disabled = 2;
string Attrs = 3;
}
message SSH {
string ID = 1;
repeated string Paths = 2;
}
message Secret {
string ID = 1;
string FilePath = 2;
string Env = 3;
}
message PrintFunc {
string Name = 1;
string Format = 2;
}
message InspectRequest {
string Ref = 1;
}
message InspectResponse {
BuildOptions Options = 1;
}
message UlimitOpt {
map<string, Ulimit> values = 1;
}
message Ulimit {
string Name = 1;
int64 Hard = 2;
int64 Soft = 3;
}
message BuildResponse {
map<string, string> ExporterResponse = 1;
}
message DisconnectRequest {
string Ref = 1;
}
message DisconnectResponse {}
message ListRequest {
string Ref = 1;
}
message ListResponse {
repeated string keys = 1;
}
message InputMessage {
oneof Input {
InputInitMessage Init = 1;
DataMessage Data = 2;
}
}
message InputInitMessage {
string Ref = 1;
}
message DataMessage {
bool EOF = 1; // true if eof was reached
bytes Data = 2; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message InputResponse {}
message Message {
oneof Input {
InitMessage Init = 1;
// FdMessage used from client to server for input (stdin) and
// from server to client for output (stdout, stderr)
FdMessage File = 2;
// ResizeMessage used from client to server for terminal resize events
ResizeMessage Resize = 3;
// SignalMessage is used from client to server to send signal events
SignalMessage Signal = 4;
}
}
message InitMessage {
string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored.
string ProcessID = 2;
InvokeConfig InvokeConfig = 3;
}
message InvokeConfig {
repeated string Entrypoint = 1;
repeated string Cmd = 2;
bool NoCmd = 11; // Do not set cmd but use the image's default
repeated string Env = 3;
string User = 4;
bool NoUser = 5; // Do not set user but use the image's default
string Cwd = 6;
bool NoCwd = 7; // Do not set cwd but use the image's default
bool Tty = 8;
bool Rollback = 9; // Kill all process in the container and recreate it.
bool Initial = 10; // Run container from the initial state of that stage (supported only on the failed step)
}
message FdMessage {
uint32 Fd = 1; // what fd the data was from
bool EOF = 2; // true if eof was reached
bytes Data = 3; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message ResizeMessage {
uint32 Rows = 1;
uint32 Cols = 2;
}
message SignalMessage {
// we only send name (ie HUP, INT) because the int values
// are platform dependent.
string Name = 1;
}
message StatusRequest {
string Ref = 1;
}
message StatusResponse {
repeated moby.buildkit.v1.Vertex vertexes = 1;
repeated moby.buildkit.v1.VertexStatus statuses = 2;
repeated moby.buildkit.v1.VertexLog logs = 3;
repeated moby.buildkit.v1.VertexWarning warnings = 4;
}
message InfoRequest {}
message InfoResponse {
BuildxVersion buildxVersion = 1;
}
message BuildxVersion {
string package = 1;
string version = 2;
string revision = 3;
}

100
controller/pb/export.go Normal file
View File

@@ -0,0 +1,100 @@
package pb
import (
"io"
"os"
"strconv"
"github.com/containerd/console"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
if len(entries) == 0 {
return nil, nil
}
for _, entry := range entries {
if entry.Type == "" {
return nil, errors.Errorf("type is required for output")
}
out := client.ExportEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
supportFile := false
supportDir := false
switch out.Type {
case client.ExporterLocal:
supportDir = true
case client.ExporterTar:
supportFile = true
case client.ExporterOCI, client.ExporterDocker:
tar, err := strconv.ParseBool(out.Attrs["tar"])
if err != nil {
tar = true
}
supportFile = tar
supportDir = !tar
case "registry":
out.Type = client.ExporterImage
}
if supportDir {
if entry.Destination == "" {
return nil, errors.Errorf("dest is required for %s exporter", out.Type)
}
if entry.Destination == "-" {
return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
}
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
}
if err == nil && !fi.IsDir() {
return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
}
out.OutputDir = entry.Destination
}
if supportFile {
if entry.Destination == "" && out.Type != client.ExporterDocker {
entry.Destination = "-"
}
if entry.Destination == "-" {
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
} else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
}
if err == nil && fi.IsDir() {
return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
}
f, err := os.Create(entry.Destination)
if err != nil {
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
}
}
outs = append(outs, out)
}
return outs, nil
}
func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) {
return func(map[string]string) (io.WriteCloser, error) {
return wc, nil
}
}

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

175
controller/pb/path.go Normal file
View File

@@ -0,0 +1,175 @@
package pb
import (
"path/filepath"
"strings"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
)
// ResolveOptionPaths resolves all paths contained in BuildOptions
// and replaces them to absolute paths.
func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
localContext := false
if options.ContextPath != "" && options.ContextPath != "-" {
if !isRemoteURL(options.ContextPath) {
localContext = true
options.ContextPath, err = filepath.Abs(options.ContextPath)
if err != nil {
return nil, err
}
}
}
if options.DockerfileName != "" && options.DockerfileName != "-" {
if localContext && !urlutil.IsURL(options.DockerfileName) {
options.DockerfileName, err = filepath.Abs(options.DockerfileName)
if err != nil {
return nil, err
}
}
}
var contexts map[string]string
for k, v := range options.NamedContexts {
if isRemoteURL(v) || strings.HasPrefix(v, "docker-image://") {
// url prefix, this is a remote path
} else if strings.HasPrefix(v, "oci-layout://") {
// oci layout prefix, this is a local path
p := strings.TrimPrefix(v, "oci-layout://")
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
v = "oci-layout://" + p
} else {
// no prefix, assume local path
v, err = filepath.Abs(v)
if err != nil {
return nil, err
}
}
if contexts == nil {
contexts = make(map[string]string)
}
contexts[k] = v
}
options.NamedContexts = contexts
var cacheFrom []*CacheOptionsEntry
for _, co := range options.CacheFrom {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "src":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheFrom = append(cacheFrom, co)
default:
cacheFrom = append(cacheFrom, co)
}
}
options.CacheFrom = cacheFrom
var cacheTo []*CacheOptionsEntry
for _, co := range options.CacheTo {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "dest":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheTo = append(cacheTo, co)
default:
cacheTo = append(cacheTo, co)
}
}
options.CacheTo = cacheTo
var exports []*ExportEntry
for _, e := range options.Exports {
if e.Destination != "" && e.Destination != "-" {
e.Destination, err = filepath.Abs(e.Destination)
if err != nil {
return nil, err
}
}
exports = append(exports, e)
}
options.Exports = exports
var secrets []*Secret
for _, s := range options.Secrets {
if s.FilePath != "" {
s.FilePath, err = filepath.Abs(s.FilePath)
if err != nil {
return nil, err
}
}
secrets = append(secrets, s)
}
options.Secrets = secrets
var ssh []*SSH
for _, s := range options.SSH {
var ps []string
for _, pt := range s.Paths {
p := pt
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
ps = append(ps, p)
}
s.Paths = ps
ssh = append(ssh, s)
}
options.SSH = ssh
return options, nil
}
func isRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}

248
controller/pb/path_test.go Normal file
View File

@@ -0,0 +1,248 @@
package pb
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/stretchr/testify/require"
)
func TestResolvePaths(t *testing.T) {
tmpwd, err := os.MkdirTemp("", "testresolvepaths")
require.NoError(t, err)
defer os.Remove(tmpwd)
require.NoError(t, os.Chdir(tmpwd))
tests := []struct {
name string
options BuildOptions
want BuildOptions
}{
{
name: "contextpath",
options: BuildOptions{ContextPath: "test"},
want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
},
{
name: "contextpath-cwd",
options: BuildOptions{ContextPath: "."},
want: BuildOptions{ContextPath: tmpwd},
},
{
name: "contextpath-dash",
options: BuildOptions{ContextPath: "-"},
want: BuildOptions{ContextPath: "-"},
},
{
name: "contextpath-ssh",
options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "dockerfilename",
options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
},
{
name: "dockerfilename-dash",
options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
},
{
name: "dockerfilename-remote",
options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "contexts",
options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
},
{
name: "cache-from",
options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "cache-to",
options: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "exports",
options: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: "test1",
},
{
Type: "tar",
Destination: "test3",
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: "test4",
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
want: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: filepath.Join(tmpwd, "test1"),
},
{
Type: "tar",
Destination: filepath.Join(tmpwd, "test3"),
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: filepath.Join(tmpwd, "test4"),
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
},
{
name: "secrets",
options: BuildOptions{
Secrets: []*Secret{
{
FilePath: "test1",
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: "test3",
},
},
},
want: BuildOptions{
Secrets: []*Secret{
{
FilePath: filepath.Join(tmpwd, "test1"),
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: filepath.Join(tmpwd, "test3"),
},
},
},
},
{
name: "ssh",
options: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{"test1", "test2"},
},
{
ID: "a",
Paths: []string{"test3"},
},
},
},
want: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{filepath.Join(tmpwd, "test1"), filepath.Join(tmpwd, "test2")},
},
{
ID: "a",
Paths: []string{filepath.Join(tmpwd, "test3")},
},
},
},
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err)
if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, *got)
}
})
}
}

126
controller/pb/progress.go Normal file
View File

@@ -0,0 +1,126 @@
package pb
import (
"github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest"
)
type writer struct {
ch chan<- *StatusResponse
}
func NewProgressWriter(ch chan<- *StatusResponse) progress.Writer {
return &writer{ch: ch}
}
func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status)
}
func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true
}
func (w *writer) ClearLogSource(interface{}) {}
func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{}
for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: v.Vertex,
Stream: int64(v.Stream),
Msg: v.Data,
Timestamp: v.Timestamp,
})
}
for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: v.Vertex,
Level: int64(v.Level),
Short: v.Short,
Detail: v.Detail,
Url: v.URL,
Info: v.SourceInfo,
Ranges: v.Range,
})
}
return &resp
}
func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{}
for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{
Vertex: v.Vertex,
Stream: int(v.Stream),
Data: v.Msg,
Timestamp: v.Timestamp,
})
}
for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: v.Vertex,
Level: int(v.Level),
Short: v.Short,
Detail: v.Detail,
URL: v.Url,
SourceInfo: v.Info,
Range: v.Ranges,
})
}
return &s
}

22
controller/pb/secrets.go Normal file
View File

@@ -0,0 +1,22 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/secrets/secretsprovider"
)
func CreateSecrets(secrets []*Secret) (session.Attachable, error) {
fs := make([]secretsprovider.Source, 0, len(secrets))
for _, secret := range secrets {
fs = append(fs, secretsprovider.Source{
ID: secret.ID,
FilePath: secret.FilePath,
Env: secret.Env,
})
}
store, err := secretsprovider.NewStore(fs)
if err != nil {
return nil, err
}
return secretsprovider.NewSecretProvider(store), nil
}

18
controller/pb/ssh.go Normal file
View File

@@ -0,0 +1,18 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider"
)
func CreateSSH(ssh []*SSH) (session.Attachable, error) {
configs := make([]sshprovider.AgentConfig, 0, len(ssh))
for _, ssh := range ssh {
cfg := sshprovider.AgentConfig{
ID: ssh.ID,
Paths: append([]string{}, ssh.Paths...),
}
configs = append(configs, cfg)
}
return sshprovider.NewSSHAgentProvider(configs)
}

View File

@@ -0,0 +1,149 @@
package processes
import (
"context"
"sync"
"sync/atomic"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/ioset"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// Process provides methods to control a process.
type Process struct {
inEnd *ioset.Forwarder
invokeConfig *pb.InvokeConfig
errCh chan error
processCancel func()
serveIOCancel func()
}
// ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil {
f()
}
p.serveIOCancel = ioCancelCallback
}
// Done returns a channel where error or nil will be sent
// when the process exits.
// TODO: change this to Wait()
func (p *Process) Done() <-chan error {
return p.errCh
}
// Manager manages a set of proceses.
type Manager struct {
container atomic.Value
processes sync.Map
}
// NewManager creates and returns a Manager.
func NewManager() *Manager {
return &Manager{}
}
// Get returns the specified process.
func (m *Manager) Get(id string) (*Process, bool) {
v, ok := m.processes.Load(id)
if !ok {
return nil, false
}
return v.(*Process), true
}
// CancelRunningProcesses cancels execution of all running processes.
func (m *Manager) CancelRunningProcesses() {
var funcs []func()
m.processes.Range(func(key, value any) bool {
funcs = append(funcs, value.(*Process).processCancel)
m.processes.Delete(key)
return true
})
for _, f := range funcs {
f()
}
}
// ListProcesses lists all running processes.
func (m *Manager) ListProcesses() (res []*pb.ProcessInfo) {
m.processes.Range(func(key, value any) bool {
res = append(res, &pb.ProcessInfo{
ProcessID: key.(string),
InvokeConfig: value.(*Process).invokeConfig,
})
return true
})
return res
}
// DeleteProcess deletes the specified process.
func (m *Manager) DeleteProcess(id string) error {
p, ok := m.processes.LoadAndDelete(id)
if !ok {
return errors.Errorf("unknown process %q", id)
}
p.(*Process).processCancel()
return nil
}
// StartProcess starts a process in the container.
// When a container isn't available (i.e. first time invoking or the container has exited) or cfg.Rollback is set,
// this method will start a new container and run the process in it. Otherwise, this method starts a new process in the
// existing container.
func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *pb.InvokeConfig) (*Process, error) {
// Get the target result to invoke a container from
var ctr *build.Container
if a := m.container.Load(); a != nil {
ctr = a.(*build.Container)
}
if cfg.Rollback || ctr == nil || ctr.IsUnavailable() {
go m.CancelRunningProcesses()
// (Re)create a new container if this is rollback or first time to invoke a process.
if ctr != nil {
go ctr.Cancel() // Finish the existing container
}
var err error
ctr, err = build.NewContainer(context.TODO(), resultCtx, cfg)
if err != nil {
return nil, errors.Errorf("failed to create container %v", err)
}
m.container.Store(ctr)
}
// [client(ForwardIO)] <-forwarder(switchable)-> [out] <-pipe-> [in] <- [process]
in, out := ioset.Pipe()
f := ioset.NewForwarder()
f.PropagateStdinClose = false
f.SetOut(&out)
// Register process
ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once
processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
p := &Process{
inEnd: f,
invokeConfig: cfg,
processCancel: processCancelFunc,
errCh: make(chan error),
}
m.processes.Store(pid, p)
go func() {
var err error
if err = ctr.Exec(ctx, cfg, in.Stdin, in.Stdout, in.Stderr); err != nil {
logrus.Debugf("process error: %v", err)
}
logrus.Debugf("finished process %s %v", pid, cfg.Entrypoint)
m.processes.Delete(pid)
processCancelFunc()
p.errCh <- err
}()
return p, nil
}

240
controller/remote/client.go Normal file
View File

@@ -0,0 +1,240 @@
package remote
import (
"context"
"io"
"sync"
"time"
"github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure"
)
func NewClient(ctx context.Context, addr string) (*Client, error) {
backoffConfig := backoff.DefaultConfig
backoffConfig.MaxDelay = 3 * time.Second
connParams := grpc.ConnectParams{
Backoff: backoffConfig,
}
gopts := []grpc.DialOption{
grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams),
grpc.WithContextDialer(dialer.ContextDialer),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(defaults.DefaultMaxRecvMsgSize)),
grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(defaults.DefaultMaxSendMsgSize)),
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
}
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil {
return nil, err
}
return &Client{conn: conn}, nil
}
type Client struct {
conn *grpc.ClientConn
closeOnce sync.Once
}
func (c *Client) Close() (err error) {
c.closeOnce.Do(func() {
err = c.conn.Close()
})
return
}
func (c *Client) Version(ctx context.Context) (string, string, string, error) {
res, err := c.client().Info(ctx, &pb.InfoRequest{})
if err != nil {
return "", "", "", err
}
v := res.BuildxVersion
return v.Package, v.Version, v.Revision, nil
}
func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
res, err := c.client().List(ctx, &pb.ListRequest{})
if err != nil {
return nil, err
}
return res.Keys, nil
}
func (c *Client) Disconnect(ctx context.Context, key string) error {
if key == "" {
return nil
}
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err
}
func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil {
return nil, err
}
return res.Infos, nil
}
func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err
}
func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if ref == "" || pid == "" {
return errors.New("build reference must be specified")
}
stream, err := c.client().Invoke(ctx)
if err != nil {
return err
}
return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in,
stdout: stdout,
stderr: stderr,
// TODO: Signal, Resize
})
}
func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
}
func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID()
statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx)
var resp *client.SolveResponse
eg.Go(func() error {
defer close(statusChan)
var err error
resp, err = c.build(egCtx, ref, options, in, statusChan)
return err
})
eg.Go(func() error {
for s := range statusChan {
st := s
progress.Write(st)
}
return nil
})
return ref, resp, eg.Wait()
}
func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{})
var resp *client.SolveResponse
eg.Go(func() error {
defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
Ref: ref,
Options: &options,
})
if err != nil {
return err
}
resp = &client.SolveResponse{
ExporterResponse: pbResp.ExporterResponse,
}
return nil
})
eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{
Ref: ref,
})
if err != nil {
return err
}
for {
resp, err := stream.Recv()
if err != nil {
if err == io.EOF {
return nil
}
return errors.Wrap(err, "failed to receive status")
}
statusChan <- pb.FromControlStatus(resp)
}
})
if in != nil {
eg.Go(func() error {
stream, err := c.client().Input(egCtx)
if err != nil {
return err
}
if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{
Ref: ref,
},
},
}); err != nil {
return errors.Wrap(err, "failed to init input")
}
inReader, inWriter := io.Pipe()
eg2, _ := errgroup.WithContext(ctx)
eg2.Go(func() error {
<-done
return inWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(inWriter, in)
inWriter.Close()
}()
eg2.Go(func() error {
for {
buf := make([]byte, 32*1024)
n, err := inReader.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
EOF: true,
},
},
})
})
return eg2.Wait()
})
}
return resp, eg.Wait()
}
func (c *Client) client() pb.ControllerClient {
return pb.NewControllerClient(c.conn)
}

View File

@@ -0,0 +1,333 @@
//go:build linux
package remote
import (
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"syscall"
"time"
"github.com/containerd/log"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pelletier/go-toml"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc"
)
const (
serveCommandName = "_INTERNAL_SERVE"
)
var (
defaultLogFilename = fmt.Sprintf("buildx.%s.log", version.Revision)
defaultSocketFilename = fmt.Sprintf("buildx.%s.sock", version.Revision)
defaultPIDFilename = fmt.Sprintf("buildx.%s.pid", version.Revision)
)
type serverConfig struct {
// Specify buildx server root
Root string `toml:"root"`
// LogLevel sets the logging level [trace, debug, info, warn, error, fatal, panic]
LogLevel string `toml:"log_level"`
// Specify file to output buildx server log
LogFile string `toml:"log_file"`
}
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
rootDir := opts.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running
ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server")
}
} else {
return &buildxController{c, serverRoot}, nil
}
// start buildx server via subcommand
err = logger.Wrap("no buildx server found; launching...", func() error {
launchFlags := []string{}
if opts.ServerConfig != "" {
launchFlags = append(launchFlags, "--config", opts.ServerConfig)
}
logFile, err := getLogFilePath(dockerCli, opts.ServerConfig)
if err != nil {
return err
}
wait, err := launch(ctx, logFile, append([]string{serveCommandName}, launchFlags...)...)
if err != nil {
return err
}
go wait()
// wait for buildx server to be ready
ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server")
}
return nil
})
if err != nil {
return nil, err
}
return &buildxController{c, serverRoot}, nil
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
serveCmd(dockerCli),
)
}
func serveCmd(dockerCli command.Cli) *cobra.Command {
var serverConfigPath string
cmd := &cobra.Command{
Use: fmt.Sprintf("%s [OPTIONS]", serveCommandName),
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Parse config
config, err := getConfig(dockerCli, serverConfigPath)
if err != nil {
return err
}
if config.LogLevel == "" {
logrus.SetLevel(logrus.InfoLevel)
} else {
lvl, err := logrus.ParseLevel(config.LogLevel)
if err != nil {
return errors.Wrap(err, "failed to prepare logger")
}
logrus.SetLevel(lvl)
}
logrus.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: log.RFC3339NanoFixed,
})
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return err
}
pidF := filepath.Join(root, defaultPIDFilename)
if err := os.WriteFile(pidF, []byte(fmt.Sprintf("%d", os.Getpid())), 0600); err != nil {
return err
}
defer func() {
if err := os.Remove(pidF); err != nil {
logrus.Errorf("failed to clean up info file %q: %v", pidF, err)
}
}()
// prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
})
defer b.Close()
// serve server
addr := filepath.Join(root, defaultSocketFilename)
if err := os.Remove(addr); err != nil && !os.IsNotExist(err) { // avoid EADDRINUSE
return err
}
defer func() {
if err := os.Remove(addr); err != nil {
logrus.Errorf("failed to clean up socket %q: %v", addr, err)
}
}()
logrus.Infof("starting server at %q", addr)
l, err := net.Listen("unix", addr)
if err != nil {
return err
}
rpc := grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
)
controllerapi.RegisterControllerServer(rpc, b)
doneCh := make(chan struct{})
errCh := make(chan error, 1)
go func() {
defer close(doneCh)
if err := rpc.Serve(l); err != nil {
errCh <- errors.Wrapf(err, "error on serving via socket %q", addr)
}
}()
var s os.Signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT)
signal.Notify(sigCh, syscall.SIGTERM)
select {
case err := <-errCh:
logrus.Errorf("got error %s, exiting", err)
return err
case s = <-sigCh:
logrus.Infof("got signal %s, exiting", s)
return nil
case <-doneCh:
logrus.Infof("rpc server done, exiting")
return nil
}
},
}
flags := cmd.Flags()
flags.StringVar(&serverConfigPath, "config", "", "Specify buildx server config file")
return cmd
}
func getLogFilePath(dockerCli command.Cli, configPath string) (string, error) {
config, err := getConfig(dockerCli, configPath)
if err != nil {
return "", err
}
if config.LogFile == "" {
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return "", err
}
return filepath.Join(root, defaultLogFilename), nil
}
return config.LogFile, nil
}
func getConfig(dockerCli command.Cli, configPath string) (*serverConfig, error) {
var defaultConfigPath bool
if configPath == "" {
defaultRoot := rootDataDir(dockerCli)
configPath = filepath.Join(defaultRoot, "config.toml")
defaultConfigPath = true
}
var config serverConfig
tree, err := toml.LoadFile(configPath)
if err != nil && !(os.IsNotExist(err) && defaultConfigPath) {
return nil, errors.Wrapf(err, "failed to read config %q", configPath)
} else if err == nil {
if err := tree.Unmarshal(&config); err != nil {
return nil, errors.Wrapf(err, "failed to unmarshal config %q", configPath)
}
}
return &config, nil
}
func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error) {
rootDir := config.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
if rootDir == "" {
return "", errors.New("buildx root dir must be determined")
}
if err := os.MkdirAll(rootDir, 0700); err != nil {
return "", err
}
serverRoot := filepath.Join(rootDir, "shared")
if err := os.MkdirAll(serverRoot, 0700); err != nil {
return "", err
}
return serverRoot, nil
}
func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
}
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {
c, err := NewClient(ctx, addr)
if err != nil {
return nil, err
}
p, v, r, err := c.Version(ctx)
if err != nil {
return nil, err
}
logrus.Debugf("connected to server (\"%v %v %v\")", p, v, r)
if !(p == version.Package && v == version.Version && r == version.Revision) {
return nil, errors.Errorf("version mismatch (client: \"%v %v %v\", server: \"%v %v %v\")", version.Package, version.Version, version.Revision, p, v, r)
}
return c, nil
}
type buildxController struct {
*Client
serverRoot string
}
func (c *buildxController) Kill(ctx context.Context) error {
pidB, err := os.ReadFile(filepath.Join(c.serverRoot, defaultPIDFilename))
if err != nil {
return err
}
pid, err := strconv.ParseInt(string(pidB), 10, 64)
if err != nil {
return err
}
if pid <= 0 {
return errors.New("no PID is recorded for buildx server")
}
p, err := os.FindProcess(int(pid))
if err != nil {
return err
}
if err := p.Signal(syscall.SIGINT); err != nil {
return err
}
// TODO: Should we send SIGKILL if process doesn't finish?
return nil
}
func launch(ctx context.Context, logFile string, args ...string) (func() error, error) {
// set absolute path of binary, since we set the working directory to the root
pathname, err := os.Executable()
if err != nil {
return nil, err
}
bCmd := exec.CommandContext(ctx, pathname, args...)
if logFile != "" {
f, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, err
}
defer f.Close()
bCmd.Stdout = f
bCmd.Stderr = f
}
bCmd.Stdin = nil
bCmd.Dir = "/"
bCmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
}
if err := bCmd.Start(); err != nil {
return nil, err
}
return bCmd.Wait, nil
}

View File

@@ -0,0 +1,19 @@
//go:build !linux
package remote
import (
"context"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
return nil, errors.New("remote buildx unsupported")
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {}

430
controller/remote/io.go Normal file
View File

@@ -0,0 +1,430 @@
package remote
import (
"context"
"io"
"syscall"
"time"
"github.com/docker/buildx/controller/pb"
"github.com/moby/sys/signal"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
type msgStream interface {
Send(*pb.Message) error
Recv() (*pb.Message, error)
}
type ioServerConfig struct {
stdin io.WriteCloser
stdout, stderr io.ReadCloser
// signalFn is a callback function called when a signal is reached to the client.
signalFn func(context.Context, syscall.Signal) error
// resizeFn is a callback function called when a resize event is reached to the client.
resizeFn func(context.Context, winSize) error
}
func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessage) error, ioConfig *ioServerConfig) (err error) {
stdin, stdout, stderr := ioConfig.stdin, ioConfig.stdout, ioConfig.stderr
stream := &debugStream{srv, "server=" + time.Now().String()}
eg, ctx := errgroup.WithContext(attachCtx)
done := make(chan struct{})
msg, err := receive(ctx, stream)
if err != nil {
return err
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
}
ref := init.Ref
if ref == "" {
return errors.New("no ref is provided")
}
if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server")
}
if stdout != nil {
stdoutReader, stdoutWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdoutWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdoutWriter, stdout)
stdoutWriter.Close()
}()
eg.Go(func() error {
defer stdoutReader.Close()
return copyToStream(1, stream, stdoutReader)
})
}
if stderr != nil {
stderrReader, stderrWriter := io.Pipe()
eg.Go(func() error {
<-done
return stderrWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stderrWriter, stderr)
stderrWriter.Close()
}()
eg.Go(func() error {
defer stderrReader.Close()
return copyToStream(2, stream, stderrReader)
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if file.Fd != 0 {
return errors.Errorf("unexpected fd: %v", file.Fd)
}
if stdin == nil {
continue // no stdin destination is specified so ignore the data
}
if len(file.Data) > 0 {
_, err := stdin.Write(file.Data)
if err != nil {
return err
}
}
if file.EOF {
stdin.Close()
}
} else if resize := msg.GetResize(); resize != nil {
if ioConfig.resizeFn != nil {
ioConfig.resizeFn(ctx, winSize{
cols: resize.Cols,
rows: resize.Rows,
})
}
} else if sig := msg.GetSignal(); sig != nil {
if ioConfig.signalFn != nil {
syscallSignal, ok := signal.SignalMap[sig.Name]
if !ok {
continue
}
ioConfig.signalFn(ctx, syscallSignal)
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
type ioAttachConfig struct {
stdin io.ReadCloser
stdout, stderr io.WriteCloser
signal <-chan syscall.Signal
resize <-chan winSize
}
type winSize struct {
rows uint32
cols uint32
}
func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage, cfg ioAttachConfig) (retErr error) {
eg, ctx := errgroup.WithContext(ctx)
done := make(chan struct{})
if err := stream.Send(&pb.Message{
Input: &pb.Message_Init{
Init: initMessage,
},
}); err != nil {
return errors.Wrap(err, "failed to init")
}
if cfg.stdin != nil {
stdinReader, stdinWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdinWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdinWriter, cfg.stdin)
stdinWriter.Close()
}()
eg.Go(func() error {
defer stdinReader.Close()
return copyToStream(0, stream, stdinReader)
})
}
if cfg.signal != nil {
eg.Go(func() error {
for {
var sig syscall.Signal
select {
case sig = <-cfg.signal:
case <-done:
return nil
case <-ctx.Done():
return nil
}
name := sigToName[sig]
if name == "" {
continue
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Signal{
Signal: &pb.SignalMessage{
Name: name,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send signal")
}
}
})
}
if cfg.resize != nil {
eg.Go(func() error {
for {
var win winSize
select {
case win = <-cfg.resize:
case <-done:
return nil
case <-ctx.Done():
return nil
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Resize{
Resize: &pb.ResizeMessage{
Rows: win.rows,
Cols: win.cols,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send resize")
}
}
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
eofs := make(map[uint32]struct{})
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if _, ok := eofs[file.Fd]; ok {
continue
}
var out io.WriteCloser
switch file.Fd {
case 1:
out = cfg.stdout
case 2:
out = cfg.stderr
default:
return errors.Errorf("unsupported fd %d", file.Fd)
}
if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
continue
}
if len(file.Data) > 0 {
if _, err := out.Write(file.Data); err != nil {
return err
}
}
if file.EOF {
eofs[file.Fd] = struct{}{}
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
msgCh := make(chan *pb.Message)
errCh := make(chan error)
go func() {
msg, err := stream.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
return
}
errCh <- err
return
}
msgCh <- msg
}()
select {
case msg := <-msgCh:
return msg, nil
case err := <-errCh:
return nil, err
case <-ctx.Done():
return nil, ctx.Err()
}
}
func copyToStream(fd uint32, snd msgStream, r io.Reader) error {
for {
buf := make([]byte, 32*1024)
n, err := r.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
EOF: true,
},
},
})
}
var sigToName = map[syscall.Signal]string{}
func init() {
for name, value := range signal.SignalMap {
sigToName[value] = name
}
}
type debugStream struct {
msgStream
prefix string
}
func (s *debugStream) Send(msg *pb.Message) error {
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|---> Resize Message (sender:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|---> Signal Message (sender:%v): %s", s.prefix, m.Signal.Name)
}
return s.msgStream.Send(msg)
}
func (s *debugStream) Recv() (*pb.Message, error) {
msg, err := s.msgStream.Recv()
if err != nil {
return nil, err
}
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|<--- Resize Message (receiver:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|<--- Signal Message (receiver:%v): %s", s.prefix, m.Signal.Name)
}
return msg, nil
}

439
controller/remote/server.go Normal file
View File

@@ -0,0 +1,439 @@
package remote
import (
"context"
"io"
"sync"
"sync/atomic"
"time"
"github.com/docker/buildx/build"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server {
return &Server{
buildFunc: buildFunc,
}
}
type Server struct {
buildFunc BuildFunc
session map[string]*session
sessionMu sync.Mutex
}
type session struct {
buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse
cancelBuild func()
buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter
result *build.ResultHandle
processes *processes.Manager
}
func (s *session) cancelRunningProcesses() {
s.processes.CancelRunningProcesses()
}
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
res = new(pb.ListProcessesResponse)
res.Infos = append(res.Infos, s.processes.ListProcesses()...)
return res, nil
}
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
return res, s.processes.DeleteProcess(req.ProcessID)
}
func (m *Server) Info(ctx context.Context, req *pb.InfoRequest) (res *pb.InfoResponse, err error) {
return &pb.InfoResponse{
BuildxVersion: &pb.BuildxVersion{
Package: version.Package,
Version: version.Version,
Revision: version.Revision,
},
}, nil
}
func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListResponse, err error) {
keys := make(map[string]struct{})
m.sessionMu.Lock()
for k := range m.session {
keys[k] = struct{}{}
}
m.sessionMu.Unlock()
var keysL []string
for k := range keys {
keysL = append(keysL, k)
}
return &pb.ListResponse{
Keys: keysL,
}, nil
}
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
key := req.Ref
if key == "" {
return nil, errors.New("disconnect: empty key")
}
m.sessionMu.Lock()
if s, ok := m.session[key]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
if s.result != nil {
s.result.Done()
}
}
delete(m.session, key)
m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil
}
func (m *Server) Close() error {
m.sessionMu.Lock()
for k := range m.session {
if s, ok := m.session[k]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
}
}
m.sessionMu.Unlock()
return nil
}
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("inspect: empty key")
}
var bo *pb.BuildOptions
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
bo = s.buildOptions
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", ref)
}
m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil
}
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("build: empty key")
}
// Prepare status channel and session
m.sessionMu.Lock()
if m.session == nil {
m.session = make(map[string]*session)
}
s, ok := m.session[ref]
if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock()
return &pb.BuildResponse{}, errors.New("build ongoing")
}
s.cancelRunningProcesses()
s.result = nil
} else {
s = &session{}
s.buildOnGoing.Store(true)
}
s.processes = processes.NewManager()
statusChan := make(chan *pb.StatusResponse)
s.statusChan = statusChan
inR, inW := io.Pipe()
defer inR.Close()
s.inputPipe = inW
m.session[ref] = s
m.sessionMu.Unlock()
defer func() {
close(statusChan)
m.sessionMu.Lock()
s, ok := m.session[ref]
if ok {
s.statusChan = nil
s.buildOnGoing.Store(false)
}
m.sessionMu.Unlock()
}()
pw := pb.NewProgressWriter(statusChan)
// Build the specified request
ctx, cancel := context.WithCancel(ctx)
defer cancel()
resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil {
s.result = res
s.cancelBuild = cancel
s.buildOptions = req.Options
m.session[ref] = s
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, ref)
}
}
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown key %v", ref)
}
m.sessionMu.Unlock()
if buildErr != nil {
return nil, buildErr
}
if resp == nil {
resp = &client.SolveResponse{}
}
return &pb.BuildResponse{
ExporterResponse: resp.ExporterResponse,
}, nil
}
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
ref := req.Ref
if ref == "" {
return errors.New("status: empty key")
}
// Wait and get status channel prepared by Build()
var statusChan <-chan *pb.StatusResponse
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
statusChan = m.session[ref].statusChan
m.sessionMu.Unlock()
break
}
// forward status
for ss := range statusChan {
if ss == nil {
break
}
if err := stream.Send(ss); err != nil {
return err
}
}
return nil
}
func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
// Get the target ref from init message
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
}
ref := init.Ref
if ref == "" {
return errors.New("input: no ref is provided")
}
// Wait and get input stream pipe prepared by Build()
var inputPipeW *io.PipeWriter
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock()
break
}
// Forward input stream
eg, ctx := errgroup.WithContext(context.TODO())
done := make(chan struct{})
msgCh := make(chan *pb.InputMessage)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() (retErr error) {
defer close(done)
defer func() {
if retErr != nil {
inputPipeW.CloseWithError(retErr)
return
}
inputPipeW.Close()
}()
for {
var msg *pb.InputMessage
select {
case msg = <-msgCh:
case <-ctx.Done():
return errors.Wrap(ctx.Err(), "canceled")
}
if msg == nil {
return nil
}
if data := msg.GetData(); data != nil {
if len(data.Data) > 0 {
_, err := inputPipeW.Write(data.Data)
if err != nil {
return err
}
}
if data.EOF {
return nil
}
}
}
})
return eg.Wait()
}
func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
containerIn, containerOut := ioset.Pipe()
defer func() { containerOut.Close(); containerIn.Close() }()
initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error {
defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() {
if retErr != nil {
initErrCh <- retErr
}
}()
ref := initMessage.Ref
cfg := initMessage.InvokeConfig
m.sessionMu.Lock()
s, ok := m.session[ref]
if !ok {
m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown key %v", ref)
}
m.sessionMu.Unlock()
pid := initMessage.ProcessID
if pid == "" {
return errors.Errorf("invoke: specify process ID")
}
proc, ok := s.processes.Get(pid)
if !ok {
// Start a new process.
if cfg == nil {
return errors.New("no container config is provided")
}
var err error
proc, err = s.processes.StartProcess(pid, s.result, cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
proc.ForwardIO(&containerIn, srvIOCancel)
initDoneCh <- proc
return nil
}, &ioServerConfig{
stdin: containerOut.Stdin,
stdout: containerOut.Stdout,
stderr: containerOut.Stderr,
// TODO: signal, resize
})
})
eg.Go(func() (rErr error) {
defer srvIOCancel()
// Wait for init done
var proc *processes.Process
select {
case p := <-initDoneCh:
proc = p
case err := <-initErrCh:
return err
case <-egCtx.Done():
return egCtx.Err()
}
// Wait for IO done
select {
case <-srvIOCtx.Done():
return srvIOCtx.Err()
case err := <-proc.Done():
return err
case <-egCtx.Done():
return egCtx.Err()
}
})
return eg.Wait()
}

View File

@@ -1,17 +1,14 @@
variable "GO_VERSION" {
default = "1.18"
}
variable "BIN_OUT" {
default = "./bin"
}
variable "RELEASE_OUT" {
default = "./release-out"
default = null
}
variable "DOCS_FORMATS" {
default = "md"
}
variable "DESTDIR" {
default = "./bin"
}
// Special target: https://github.com/docker/metadata-action#bake-definition
# Special target: https://github.com/docker/metadata-action#bake-definition
target "meta-helper" {
tags = ["docker/buildx-bin:local"]
}
@@ -20,7 +17,6 @@ target "_common" {
args = {
GO_VERSION = GO_VERSION
BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1
BUILDX_EXPERIMENTAL = 1
}
}
@@ -49,6 +45,7 @@ target "validate-docs" {
inherits = ["_common"]
args = {
FORMATS = DOCS_FORMATS
BUILDX_EXPERIMENTAL = 1 // enables experimental cmds/flags for docs generation
}
dockerfile = "./hack/dockerfiles/docs.Dockerfile"
target = "validate"
@@ -62,6 +59,13 @@ target "validate-authors" {
output = ["type=cacheonly"]
}
target "validate-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "update-vendor" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -73,6 +77,7 @@ target "update-docs" {
inherits = ["_common"]
args = {
FORMATS = DOCS_FORMATS
BUILDX_EXPERIMENTAL = 1 // enables experimental cmds/flags for docs generation
}
dockerfile = "./hack/dockerfiles/docs.Dockerfile"
target = "update"
@@ -86,6 +91,13 @@ target "update-authors" {
output = ["."]
}
target "update-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "update"
output = ["."]
}
target "mod-outdated" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -97,13 +109,13 @@ target "mod-outdated" {
target "test" {
inherits = ["_common"]
target = "test-coverage"
output = ["./coverage"]
output = ["${DESTDIR}/coverage"]
}
target "binaries" {
inherits = ["_common"]
target = "binaries"
output = [BIN_OUT]
output = ["${DESTDIR}/build"]
platforms = ["local"]
}
@@ -127,7 +139,7 @@ target "binaries-cross" {
target "release" {
inherits = ["binaries-cross"]
target = "release"
output = [RELEASE_OUT]
output = ["${DESTDIR}/release"]
}
target "image" {
@@ -144,3 +156,29 @@ target "image-local" {
inherits = ["image"]
output = ["type=docker"]
}
variable "HTTP_PROXY" {
default = ""
}
variable "HTTPS_PROXY" {
default = ""
}
variable "NO_PROXY" {
default = ""
}
target "integration-test-base" {
inherits = ["_common"]
args = {
HTTP_PROXY = HTTP_PROXY
HTTPS_PROXY = HTTPS_PROXY
NO_PROXY = NO_PROXY
}
target = "integration-test-base"
output = ["type=cacheonly"]
}
target "integration-test" {
inherits = ["integration-test-base"]
target = "integration-test"
}

1073
docs/bake-reference.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,74 +0,0 @@
# Defining additional build contexts and linking targets
In addition to the main `context` key that defines the build context each target
can also define additional named contexts with a map defined with key `contexts`.
These values map to the `--build-context` flag in the [build command](https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context).
Inside the Dockerfile these contexts can be used with the `FROM` instruction or `--from` flag.
The value can be a local source directory, container image (with `docker-image://` prefix),
Git URL, HTTP URL or a name of another target in the Bake file (with `target:` prefix).
## Pinning alpine image
```dockerfile
# syntax=docker/dockerfile:1
FROM alpine
RUN echo "Hello world"
```
```hcl
# docker-bake.hcl
target "app" {
contexts = {
alpine = "docker-image://alpine:3.13"
}
}
```
## Using a secondary source directory
```dockerfile
# syntax=docker/dockerfile:1
FROM scratch AS src
FROM golang
COPY --from=src . .
```
```hcl
# docker-bake.hcl
target "app" {
contexts = {
src = "../path/to/source"
}
}
```
## Using a result of one target as a base image in another target
To use a result of one target as a build context of another, specity the target
name with `target:` prefix.
```dockerfile
# syntax=docker/dockerfile:1
FROM baseapp
RUN echo "Hello world"
```
```hcl
# docker-bake.hcl
target "base" {
dockerfile = "baseapp.Dockerfile"
}
target "app" {
contexts = {
baseapp = "target:base"
}
}
```
Please note that in most cases you should just use a single multi-stage
Dockerfile with multiple targets for similar behavior. This case is recommended
when you have multiple Dockerfiles that can't be easily merged into one.

View File

@@ -1,270 +0,0 @@
# Building from Compose file
## Specification
Bake uses the [compose-spec](https://docs.docker.com/compose/compose-file/) to
parse a compose file and translate each service to a [target](file-definition.md#target).
```yaml
# docker-compose.yml
services:
webapp-dev:
build: &build-dev
dockerfile: Dockerfile.webapp
tags:
- docker.io/username/webapp:latest
cache_from:
- docker.io/username/webapp:cache
cache_to:
- docker.io/username/webapp:cache
webapp-release:
build:
<<: *build-dev
x-bake:
platforms:
- linux/amd64
- linux/arm64
db:
image: docker.io/username/db
build:
dockerfile: Dockerfile.db
```
```console
$ docker buildx bake --print
```
```json
{
"group": {
"default": {
"targets": [
"db",
"webapp-dev",
"webapp-release"
]
}
},
"target": {
"db": {
"context": ".",
"dockerfile": "Dockerfile.db",
"tags": [
"docker.io/username/db"
]
},
"webapp-dev": {
"context": ".",
"dockerfile": "Dockerfile.webapp",
"tags": [
"docker.io/username/webapp:latest"
],
"cache-from": [
"docker.io/username/webapp:cache"
],
"cache-to": [
"docker.io/username/webapp:cache"
]
},
"webapp-release": {
"context": ".",
"dockerfile": "Dockerfile.webapp",
"tags": [
"docker.io/username/webapp:latest"
],
"cache-from": [
"docker.io/username/webapp:cache"
],
"cache-to": [
"docker.io/username/webapp:cache"
],
"platforms": [
"linux/amd64",
"linux/arm64"
]
}
}
}
```
Unlike the [HCL format](file-definition.md#hcl-definition), there are some
limitations with the compose format:
* Specifying variables or global scope attributes is not yet supported
* `inherits` service field is not supported, but you can use [YAML anchors](https://docs.docker.com/compose/compose-file/#fragments) to reference other services like the example above
## `.env` file
You can declare default environment variables in an environment file named
`.env`. This file will be loaded from the current working directory,
where the command is executed and applied to compose definitions passed
with `-f`.
```yaml
# docker-compose.yml
services:
webapp:
image: docker.io/username/webapp:${TAG:-v1.0.0}
build:
dockerfile: Dockerfile
```
```
# .env
TAG=v1.1.0
```
```console
$ docker buildx bake --print
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"docker.io/username/webapp:v1.1.0"
]
}
}
}
```
> **Note**
>
> System environment variables take precedence over environment variables
> in `.env` file.
## Extension field with `x-bake`
Even if some fields are not (yet) available in the compose specification, you
can use the [special extension](https://docs.docker.com/compose/compose-file/#extension)
field `x-bake` in your compose file to evaluate extra fields:
```yaml
# docker-compose.yml
services:
addon:
image: ct-addon:bar
build:
context: .
dockerfile: ./Dockerfile
args:
CT_ECR: foo
CT_TAG: bar
x-bake:
tags:
- ct-addon:foo
- ct-addon:alp
platforms:
- linux/amd64
- linux/arm64
cache-from:
- user/app:cache
- type=local,src=path/to/cache
cache-to:
- type=local,dest=path/to/cache
pull: true
aws:
image: ct-fake-aws:bar
build:
dockerfile: ./aws.Dockerfile
args:
CT_ECR: foo
CT_TAG: bar
x-bake:
secret:
- id=mysecret,src=./secret
- id=mysecret2,src=./secret2
platforms: linux/arm64
output: type=docker
no-cache: true
```
```console
$ docker buildx bake --print
```
```json
{
"group": {
"default": {
"targets": [
"aws",
"addon"
]
}
},
"target": {
"addon": {
"context": ".",
"dockerfile": "./Dockerfile",
"args": {
"CT_ECR": "foo",
"CT_TAG": "bar"
},
"tags": [
"ct-addon:foo",
"ct-addon:alp"
],
"cache-from": [
"user/app:cache",
"type=local,src=path/to/cache"
],
"cache-to": [
"type=local,dest=path/to/cache"
],
"platforms": [
"linux/amd64",
"linux/arm64"
],
"pull": true
},
"aws": {
"context": ".",
"dockerfile": "./aws.Dockerfile",
"args": {
"CT_ECR": "foo",
"CT_TAG": "bar"
},
"tags": [
"ct-fake-aws:bar"
],
"secret": [
"id=mysecret,src=./secret",
"id=mysecret2,src=./secret2"
],
"platforms": [
"linux/arm64"
],
"output": [
"type=docker"
],
"no-cache": true
}
}
}
```
Complete list of valid fields for `x-bake`:
* `cache-from`
* `cache-to`
* `contexts`
* `no-cache`
* `no-cache-filter`
* `output`
* `platforms`
* `pull`
* `secret`
* `ssh`
* `tags`

View File

@@ -1,216 +0,0 @@
# Configuring builds
Bake supports loading build definition from files, but sometimes you need even
more flexibility to configure this definition.
For this use case, you can define variables inside the bake files that can be
set by the user with environment variables or by [attribute definitions](#global-scope-attributes)
in other bake files. If you wish to change a specific value for a single
invocation you can use the `--set` flag [from the command line](#from-command-line).
## Global scope attributes
You can define global scope attributes in HCL/JSON and use them for code reuse
and setting values for variables. This means you can do a "data-only" HCL file
with the values you want to set/override and use it in the list of regular
output files.
```hcl
# docker-bake.hcl
variable "FOO" {
default = "abc"
}
target "app" {
args = {
v1 = "pre-${FOO}"
}
}
```
You can use this file directly:
```console
$ docker buildx bake --print app
```
```json
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"v1": "pre-abc"
}
}
}
}
```
Or create an override configuration file:
```hcl
# env.hcl
WHOAMI="myuser"
FOO="def-${WHOAMI}"
```
And invoke bake together with both of the files:
```console
$ docker buildx bake -f docker-bake.hcl -f env.hcl --print app
```
```json
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"v1": "pre-def-myuser"
}
}
}
}
```
## From command line
You can also override target configurations from the command line with the
[`--set` flag](https://docs.docker.com/engine/reference/commandline/buildx_bake/#set):
```hcl
# docker-bake.hcl
target "app" {
args = {
mybuildarg = "foo"
}
}
```
```console
$ docker buildx bake --set app.args.mybuildarg=bar --set app.platform=linux/arm64 app --print
```
```json
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"mybuildarg": "bar"
},
"platforms": [
"linux/arm64"
]
}
}
}
```
Pattern matching syntax defined in [https://golang.org/pkg/path/#Match](https://golang.org/pkg/path/#Match)
is also supported:
```console
$ docker buildx bake --set foo*.args.mybuildarg=value # overrides build arg for all targets starting with "foo"
$ docker buildx bake --set *.platform=linux/arm64 # overrides platform for all targets
$ docker buildx bake --set foo*.no-cache # bypass caching only for targets starting with "foo"
```
Complete list of overridable fields:
* `args`
* `cache-from`
* `cache-to`
* `context`
* `dockerfile`
* `labels`
* `no-cache`
* `output`
* `platform`
* `pull`
* `secrets`
* `ssh`
* `tags`
* `target`
## Using variables in variables across files
When multiple files are specified, one file can use variables defined in
another file.
```hcl
# docker-bake1.hcl
variable "FOO" {
default = upper("${BASE}def")
}
variable "BAR" {
default = "-${FOO}-"
}
target "app" {
args = {
v1 = "pre-${BAR}"
}
}
```
```hcl
# docker-bake2.hcl
variable "BASE" {
default = "abc"
}
target "app" {
args = {
v2 = "${FOO}-post"
}
}
```
```console
$ docker buildx bake -f docker-bake1.hcl -f docker-bake2.hcl --print app
```
```json
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"v1": "pre--ABCDEF-",
"v2": "ABCDEF-post"
}
}
}
}
```

View File

@@ -1,440 +0,0 @@
# Bake file definition
`buildx bake` supports HCL, JSON and Compose file format for defining build
[groups](#group), [targets](#target) as well as [variables](#variable) and
[functions](#functions). It looks for build definition files in the current
directory in the following order:
* `docker-compose.yml`
* `docker-compose.yaml`
* `docker-bake.json`
* `docker-bake.override.json`
* `docker-bake.hcl`
* `docker-bake.override.hcl`
## Specification
Inside a bake file you can declare group, target and variable blocks to define
project specific reusable build flows.
### Target
A target reflects a single docker build invocation with the same options that
you would specify for `docker build`:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
```
```console
$ docker buildx bake webapp-dev
```
> **Note**
>
> In the case of compose files, each service corresponds to a target.
> If compose service name contains a dot it will be replaced with an underscore.
Complete list of valid target fields available for [HCL](#hcl-definition) and
[JSON](#json-definition) definitions:
| Name | Type | Description |
|---------------------|--------|-------------------------------------------------------------------------------------------------------------------------------------------------|
| `inherits` | List | [Inherit build options](#merging-and-inheritance) from other targets |
| `args` | Map | Set build-time variables (same as [`--build-arg` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `cache-from` | List | External cache sources (same as [`--cache-from` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `cache-to` | List | Cache export destinations (same as [`--cache-to` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `context` | String | Set of files located in the specified path or URL |
| `contexts` | Map | Additional build contexts (same as [`--build-context` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `dockerfile` | String | Name of the Dockerfile (same as [`--file` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `dockerfile-inline` | String | Inline Dockerfile content |
| `labels` | Map | Set metadata for an image (same as [`--label` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `no-cache` | Bool | Do not use cache when building the image (same as [`--no-cache` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `no-cache-filter` | List | Do not cache specified stages (same as [`--no-cache-filter` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `output` | List | Output destination (same as [`--output` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `platforms` | List | Set target platforms for build (same as [`--platform` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `pull` | Bool | Always attempt to pull all referenced images (same as [`--pull` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `secret` | List | Secret to expose to the build (same as [`--secret` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `ssh` | List | SSH agent socket or keys to expose to the build (same as [`--ssh` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `tags` | List | Name and optionally a tag in the format `name:tag` (same as [`--tag` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
| `target` | String | Set the target build stage to build (same as [`--target` flag](https://docs.docker.com/engine/reference/commandline/buildx_build/)) |
### Group
A group is a grouping of targets:
```hcl
# docker-bake.hcl
group "build" {
targets = ["db", "webapp-dev"]
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
target "db" {
dockerfile = "Dockerfile.db"
tags = ["docker.io/username/db"]
}
```
```console
$ docker buildx bake build
```
### Variable
Similar to how Terraform provides a way to [define variables](https://www.terraform.io/docs/configuration/variables.html#declaring-an-input-variable),
the HCL file format also supports variable block definitions. These can be used
to define variables with values provided by the current environment, or a
default value when unset:
```hcl
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:${TAG}"]
}
```
```console
$ docker buildx bake webapp-dev # will use the default value "latest"
$ TAG=dev docker buildx bake webapp-dev # will use the TAG environment variable value
```
> **Tip**
>
> See also the [Configuring builds](configuring-build.md) page for advanced usage.
### Functions
A [set of generally useful functions](https://github.com/docker/buildx/blob/master/bake/hclparser/stdlib.go)
provided by [go-cty](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib)
are available for use in HCL files:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${add(123, 1)}"
}
}
```
In addition, [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc)
are also supported:
```hcl
# docker-bake.hcl
function "increment" {
params = [number]
result = number + 1
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${increment(123)}"
}
}
```
> **Note**
>
> See [User defined HCL functions](hcl-funcs.md) page for more details.
## Built-in variables
* `BAKE_CMD_CONTEXT` can be used to access the main `context` for bake command
from a bake file that has been [imported remotely](file-definition.md#remote-definition).
* `BAKE_LOCAL_PLATFORM` returns the current platform's default platform
specification (e.g. `linux/amd64`).
## Merging and inheritance
Multiple files can include the same target and final build options will be
determined by merging them together:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
```
```hcl
# docker-bake2.hcl
target "webapp-dev" {
tags = ["docker.io/username/webapp:dev"]
}
```
```console
$ docker buildx bake -f docker-bake.hcl -f docker-bake2.hcl webapp-dev
```
A group can specify its list of targets with the `targets` option. A target can
inherit build options by setting the `inherits` option to the list of targets or
groups to inherit from:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:${TAG}"]
}
target "webapp-release" {
inherits = ["webapp-dev"]
platforms = ["linux/amd64", "linux/arm64"]
}
```
## `default` target/group
When you invoke `bake` you specify what targets/groups you want to build. If no
arguments is specified, the group/target named `default` will be built:
```hcl
# docker-bake.hcl
target "default" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
```
```console
$ docker buildx bake
```
## Definitions
### HCL definition
HCL definition file is recommended as its experience is more aligned with buildx UX
and also allows better code reuse, different target groups and extended features.
```hcl
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["db", "webapp-dev"]
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:${TAG}"]
}
target "webapp-release" {
inherits = ["webapp-dev"]
platforms = ["linux/amd64", "linux/arm64"]
}
target "db" {
dockerfile = "Dockerfile.db"
tags = ["docker.io/username/db"]
}
```
### JSON definition
```json
{
"variable": {
"TAG": {
"default": "latest"
}
},
"group": {
"default": {
"targets": [
"db",
"webapp-dev"
]
}
},
"target": {
"webapp-dev": {
"dockerfile": "Dockerfile.webapp",
"tags": [
"docker.io/username/webapp:${TAG}"
]
},
"webapp-release": {
"inherits": [
"webapp-dev"
],
"platforms": [
"linux/amd64",
"linux/arm64"
]
},
"db": {
"dockerfile": "Dockerfile.db",
"tags": [
"docker.io/username/db"
]
}
}
}
```
### Compose file
```yaml
# docker-compose.yml
services:
webapp:
image: docker.io/username/webapp:latest
build:
dockerfile: Dockerfile.webapp
db:
image: docker.io/username/db
build:
dockerfile: Dockerfile.db
```
> **Note**
>
> See [Building from Compose file](compose-file.md) page for more details.
## Remote definition
You can also build bake files directly from a remote Git repository or HTTPS URL:
```console
$ docker buildx bake "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/docker/cli.git#v20.10.11
#1 0.745 e8f1871b077b64bcb4a13334b7146492773769f7 refs/tags/v20.10.11
#1 2.022 From https://github.com/docker/cli
#1 2.022 * [new tag] v20.10.11 -> v20.10.11
#1 DONE 2.9s
```
```json
{
"group": {
"default": {
"targets": [
"binary"
]
}
},
"target": {
"binary": {
"context": "https://github.com/docker/cli.git#v20.10.11",
"dockerfile": "Dockerfile",
"args": {
"BASE_VARIANT": "alpine",
"GO_STRIP": "",
"VERSION": ""
},
"target": "binary",
"platforms": [
"local"
],
"output": [
"build"
]
}
}
}
```
As you can see the context is fixed to `https://github.com/docker/cli.git` even if
[no context is actually defined](https://github.com/docker/cli/blob/2776a6d694f988c0c1df61cad4bfac0f54e481c8/docker-bake.hcl#L17-L26)
in the definition.
If you want to access the main context for bake command from a bake file
that has been imported remotely, you can use the [`BAKE_CMD_CONTEXT` built-in var](#built-in-variables).
```console
$ cat https://raw.githubusercontent.com/tonistiigi/buildx/remote-test/docker-bake.hcl
```
```hcl
target "default" {
context = BAKE_CMD_CONTEXT
dockerfile-inline = <<EOT
FROM alpine
WORKDIR /src
COPY . .
RUN ls -l && stop
EOT
}
```
```console
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" --print
```
```json
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile",
"dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
}
}
}
```
```console
$ touch foo bar
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test"
```
```text
...
> [4/4] RUN ls -l && stop:
#8 0.101 total 0
#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 bar
#8 0.102 -rw-r--r-- 1 root root 0 Jul 27 18:47 foo
#8 0.102 /bin/sh: stop: not found
```
```console
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/tonistiigi/buildx.git#remote-test
#1 0.429 577303add004dd7efeb13434d69ea030d35f7888 refs/heads/remote-test
#1 CACHED
```
```json
{
"target": {
"default": {
"context": "https://github.com/docker/cli.git#v20.10.11",
"dockerfile": "Dockerfile",
"dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
}
}
}
```
```console
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11"
```
```text
...
> [4/4] RUN ls -l && stop:
#8 0.136 drwxrwxrwx 5 root root 4096 Jul 27 18:31 kubernetes
#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 man
#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 opts
#8 0.136 -rw-rw-rw- 1 root root 1893 Jul 27 18:31 poule.yml
#8 0.136 drwxrwxrwx 7 root root 4096 Jul 27 18:31 scripts
#8 0.136 drwxrwxrwx 3 root root 4096 Jul 27 18:31 service
#8 0.136 drwxrwxrwx 2 root root 4096 Jul 27 18:31 templates
#8 0.136 drwxrwxrwx 10 root root 4096 Jul 27 18:31 vendor
#8 0.136 -rwxrwxrwx 1 root root 9620 Jul 27 18:31 vendor.conf
#8 0.136 /bin/sh: stop: not found
```

View File

@@ -1,327 +0,0 @@
# User defined HCL functions
## Using interpolation to tag an image with the git sha
As shown in the [File definition](file-definition.md#variable) page, `bake`
supports variable blocks which are assigned to matching environment variables
or default values:
```hcl
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
tags = ["docker.io/username/webapp:${TAG}"]
}
```
alternatively, in json format:
```json
{
"variable": {
"TAG": {
"default": "latest"
}
},
"group": {
"default": {
"targets": ["webapp"]
}
},
"target": {
"webapp": {
"tags": ["docker.io/username/webapp:${TAG}"]
}
}
}
```
```console
$ docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"docker.io/username/webapp:latest"
]
}
}
}
```
```console
$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"docker.io/username/webapp:985e9e9"
]
}
}
}
```
## Using the `add` function
You can use [`go-cty` stdlib functions](https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib).
Here we are using the `add` function.
```hcl
# docker-bake.hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
args = {
buildno = "${add(123, 1)}"
}
}
```
```console
$ docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"buildno": "124"
}
}
}
}
```
## Defining an `increment` function
It also supports [user defined functions](https://github.com/hashicorp/hcl/tree/main/ext/userfunc).
The following example defines a simple an `increment` function.
```hcl
# docker-bake.hcl
function "increment" {
params = [number]
result = number + 1
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
args = {
buildno = "${increment(123)}"
}
}
```
```console
$ docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"buildno": "124"
}
}
}
}
```
## Only adding tags if a variable is not empty using an `notequal`
Here we are using the conditional `notequal` function which is just for
symmetry with the `equal` one.
```hcl
# docker-bake.hcl
variable "TAG" {default="" }
group "default" {
targets = [
"webapp",
]
}
target "webapp" {
context="."
dockerfile="Dockerfile"
tags = [
"my-image:latest",
notequal("",TAG) ? "my-image:${TAG}": "",
]
}
```
```console
$ docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"my-image:latest"
]
}
}
}
```
## Using variables in functions
You can refer variables to other variables like the target blocks can. Stdlib
functions can also be called but user functions can't at the moment.
```hcl
# docker-bake.hcl
variable "REPO" {
default = "user/repo"
}
function "tag" {
params = [tag]
result = ["${REPO}:${tag}"]
}
target "webapp" {
tags = tag("v1")
}
```
```console
$ docker buildx bake --print webapp
```
```json
{
"group": {
"default": {
"targets": [
"webapp"
]
}
},
"target": {
"webapp": {
"context": ".",
"dockerfile": "Dockerfile",
"tags": [
"user/repo:v1"
]
}
}
}
```
## Using typed variables
Non-string variables are also accepted. The value passed with env is parsed
into suitable type first.
```hcl
# docker-bake.hcl
variable "FOO" {
default = 3
}
variable "IS_FOO" {
default = true
}
target "app" {
args = {
v1 = FOO > 5 ? "higher" : "lower"
v2 = IS_FOO ? "yes" : "no"
}
}
```
```console
$ docker buildx bake --print app
```
```json
{
"group": {
"default": {
"targets": [
"app"
]
}
},
"target": {
"app": {
"context": ".",
"dockerfile": "Dockerfile",
"args": {
"v1": "lower",
"v2": "yes"
}
}
}
}
```

View File

@@ -1,36 +0,0 @@
# High-level build options with Bake
> This command is experimental.
>
> The design of bake is in early stages, and we are looking for [feedback from users](https://github.com/docker/buildx/issues).
{: .experimental }
Buildx also aims to provide support for high-level build concepts that go beyond
invoking a single build command. We want to support building all the images in
your application together and let the users define project specific reusable
build flows that can then be easily invoked by anyone.
[BuildKit](https://github.com/moby/buildkit) efficiently handles multiple
concurrent build requests and de-duplicating work. The build commands can be
combined with general-purpose command runners (for example, `make`). However,
these tools generally invoke builds in sequence and therefore cannot leverage
the full potential of BuildKit parallelization, or combine BuildKit's output
for the user. For this use case, we have added a command called
[`docker buildx bake`](https://docs.docker.com/engine/reference/commandline/buildx_bake/).
The `bake` command supports building images from HCL, JSON and Compose files.
This is similar to [`docker compose build`](https://docs.docker.com/compose/reference/build/),
but allowing all the services to be built concurrently as part of a single
request. If multiple files are specified they are all read and configurations are
combined.
We recommend using HCL files as its experience is more aligned with buildx UX
and also allows better code reuse, different target groups and extended features.
## Next steps
* [File definition](file-definition.md)
* [Configuring builds](configuring-build.md)
* [User defined HCL functions](hcl-funcs.md)
* [Defining additional build contexts and linking targets](build-contexts.md)
* [Building from Compose file](compose-file.md)

View File

@@ -1,48 +1,3 @@
# CI/CD
## GitHub Actions
Docker provides a [GitHub Action that will build and push your image](https://github.com/docker/build-push-action/#about)
using Buildx. Here is a simple workflow:
```yaml
name: ci
on:
push:
branches:
- 'main'
jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v2
with:
push: true
tags: user/app:latest
```
In this example we are also using 3 other actions:
* [`setup-buildx`](https://github.com/docker/setup-buildx-action) action will create and boot a builder using by
default the `docker-container` [builder driver](../reference/buildx_create.md#driver).
This is **not required but recommended** using it to be able to build multi-platform images, export cache, etc.
* [`setup-qemu`](https://github.com/docker/setup-qemu-action) action can be useful if you want
to add emulation support with QEMU to be able to build against more platforms.
* [`login`](https://github.com/docker/login-action) action will take care to log
in against a Docker registry.
This page has moved to [Docker Docs website](https://docs.docker.com/build/ci/)

View File

@@ -1,23 +1,3 @@
# CNI networking
It can be useful to use a bridge network for your builder if for example you
encounter a network port contention during multiple builds. If you're using
the BuildKit image, CNI is not yet available in it, but you can create
[a custom BuildKit image with CNI support](https://github.com/moby/buildkit/blob/master/docs/cni-networking.md).
Now build this image:
```console
$ docker buildx build --tag buildkit-cni:local --load .
```
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) that
will use this image:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "image=buildkit-cni:local" \
--buildkitd-flags "--oci-worker-net=cni"
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#cni-networking)

View File

@@ -1,20 +1,3 @@
# Color output controls
Buildx has support for modifying the colors that are used to output information
to the terminal. You can set the environment variable `BUILDKIT_COLORS` to
something like `run=123,20,245:error=yellow:cancel=blue:warning=white` to set
the colors that you would like to use:
![Progress output custom colors](https://user-images.githubusercontent.com/1951866/180584033-24522385-cafd-4a54-a4a2-18f5ce74eb27.png)
Setting `NO_COLOR` to anything will disable any colorized output as recommended
by [no-color.org](https://no-color.org/):
![Progress output no color](https://user-images.githubusercontent.com/1951866/180584037-e28f9997-dd4c-49cf-8b26-04864815de19.png)
> **Note**
>
> Parsing errors will be reported but ignored. This will result in default
> color values being used where needed.
See also [the list of pre-defined colors](https://github.com/moby/buildkit/blob/master/util/progress/progressui/colors.go).
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/env-vars/#buildkit_colors)

View File

@@ -1,34 +1,3 @@
# Using a custom network
[Create a network](https://docs.docker.com/engine/reference/commandline/network_create/)
named `foonet`:
```console
$ docker network create foonet
```
[Create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
named `mybuilder` that will use this network:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "network=foonet"
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
[Inspect the builder container](https://docs.docker.com/engine/reference/commandline/inspect/)
and see what network is being used:
{% raw %}
```console
$ docker inspect buildx_buildkit_mybuilder0 --format={{.NetworkSettings.Networks}}
map[foonet:0xc00018c0c0]
```
{% endraw %}
This page has moved to [Docker Docs website](https://docs.docker.com/build/drivers/docker-container/#custom-network)

View File

@@ -1,63 +1,3 @@
# Using a custom registry configuration
If you [create a `docker-container` or `kubernetes` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) and
have specified certificates for registries in the [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md),
the files will be copied into the container under `/etc/buildkit/certs` and
configuration will be updated to reflect that.
Take the following `buildkitd.toml` configuration that will be used for
pushing an image to this registry using self-signed certificates:
```toml
# /etc/buildkitd.toml
debug = true
[registry."myregistry.com"]
ca=["/etc/certs/myregistry.pem"]
[[registry."myregistry.com".keypair]]
key="/etc/certs/myregistry_key.pem"
cert="/etc/certs/myregistry_cert.pem"
```
Here we have configured a self-signed certificate for `myregistry.com` registry.
Now [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use this BuildKit configuration:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--config /etc/buildkitd.toml
```
Inspecting the builder container, you can see that buildkitd configuration
has changed:
```console
$ docker exec -it buildx_buildkit_mybuilder0 cat /etc/buildkit/buildkitd.toml
```
```toml
debug = true
[registry]
[registry."myregistry.com"]
ca = ["/etc/buildkit/certs/myregistry.com/myregistry.pem"]
[[registry."myregistry.com".keypair]]
cert = "/etc/buildkit/certs/myregistry.com/myregistry_cert.pem"
key = "/etc/buildkit/certs/myregistry.com/myregistry_key.pem"
```
And certificates copied inside the container:
```console
$ docker exec -it buildx_buildkit_mybuilder0 ls /etc/buildkit/certs/myregistry.com/
myregistry.pem myregistry_cert.pem myregistry_key.pem
```
Now you should be able to push to the registry with this builder:
```console
$ docker buildx build --push --tag myregistry.com/myimage:latest .
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#setting-registry-certificates)

Some files were not shown because too many files have changed in this diff Show More