Compare commits

...

934 Commits

Author SHA1 Message Date
Tõnis Tiigi
e273a53c88 Merge pull request #2194 from LaurentGoderre/sbom-dsse
Add support for DSSE envelope for attestation in imagetools
2024-02-28 20:08:07 -08:00
Tonis Tiigi
dcdcce6c52 imagetools: supress warnings for dsse mediatypes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-28 19:25:42 -08:00
Tõnis Tiigi
c5b4ce9e7b Merge pull request #2292 from crazy-max/go-minor
pin to go 1.21
2024-02-28 14:26:28 -08:00
Tõnis Tiigi
8f484f6ac1 Merge pull request #2290 from tonistiigi/multi-export
build: allow multiple exports if supported by buildkit
2024-02-28 14:20:21 -08:00
Laurent Goderre
b748185f48 Add support for DSSE envelope for attestation and provenance in imagetools
Signed-off-by: Laurent Goderre <laurent.goderre@docker.com>
2024-02-28 16:45:51 -05:00
CrazyMax
a6228ed78f Merge pull request #2293 from docker/dependabot/github_actions/peter-evans/create-pull-request-6.0.1
build(deps): bump peter-evans/create-pull-request from 6.0.0 to 6.0.1
2024-02-28 22:31:55 +01:00
Tonis Tiigi
fcbe2803c8 build: allow multiple exports if supported by buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-28 13:16:15 -08:00
Tõnis Tiigi
83c30c6c5a Merge pull request #2291 from crazy-max/update-buildkit
vendor: github.com/moby/buildkit v0.13.0-rc3
2024-02-28 13:13:06 -08:00
Tõnis Tiigi
8db86e4031 Merge pull request #2287 from iankingori/fix-dialer
remote: use winio DialPipeContext for named pipes
2024-02-28 13:12:29 -08:00
Tõnis Tiigi
e705cafcd5 Merge pull request #2289 from tonistiigi/prompt-cancel
commands: handle ctrl-c on active prompt
2024-02-28 13:11:48 -08:00
dependabot[bot]
32f17b0de1 build(deps): bump peter-evans/create-pull-request from 6.0.0 to 6.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 6.0.0 to 6.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](b1ddad2c99...a4f52f8033)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-28 18:23:29 +00:00
Ian King'ori
d40c4bb046 remote: use winio DialPipeContext for named pipes
Signed-off-by: Ian King'ori <kingorim.ian@gmail.com>
2024-02-28 16:19:58 +03:00
CrazyMax
25f8011825 pin to go 1.21
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-28 13:18:42 +01:00
CrazyMax
d0f9655aa2 vendor: github.com/moby/buildkit v0.13.0-rc3
full diff: https://github.com/moby/buildkit/compare/v0.13.0-rc2...v0.13.0-rc3

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-28 09:46:36 +01:00
Tonis Tiigi
ce9a486a0e commands: handle ctrl-c on active prompt
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-27 17:35:27 -08:00
CrazyMax
85abcc413e Merge pull request #2283 from crazy-max/update-compose
vendor: update compose-go to v2.0.0-rc.8
2024-02-27 08:55:04 +01:00
Tõnis Tiigi
e5acb010c9 Merge pull request #2284 from crazy-max/update-uuid
vendor: update github.com/google/uuid to v1.6.0
2024-02-26 08:39:01 -08:00
Akihiro Suda
79f50ad924 Merge pull request #2285 from crazy-max/update-hashring
vendor: github.com/serialx/hashring 22c0c7ab6b1b (master)
2024-02-26 21:00:42 +09:00
Tõnis Tiigi
5723ceefb6 Merge pull request #2281 from crazy-max/update-buildkit
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-25 22:19:45 -08:00
CrazyMax
95185e9525 vendor: update compose-go to v2.0.0-rc.8
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-24 17:15:27 +01:00
CrazyMax
e423a67f7b vendor: github.com/moby/buildkit v0.13.0-rc2
full diff: https://github.com/moby/buildkit/compare/8e3fe35738c2...v0.13.0-rc2

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-24 17:14:01 +01:00
CrazyMax
545a5c97c6 Merge pull request #2282 from crazy-max/update-k8s
vendor: bump k8s dependencies to v0.29.2
2024-02-24 17:12:39 +01:00
CrazyMax
625d90b983 vendor: github.com/serialx/hashring 22c0c7ab6b1b (master)
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-24 16:56:59 +01:00
CrazyMax
9999fc63e8 vendor: update github.com/google/uuid to v1.6.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-24 16:52:21 +01:00
CrazyMax
303e509bbf vendor: bump k8s dependencies to v0.29.2
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-24 16:41:41 +01:00
CrazyMax
ae0a5e495a Merge pull request #2263 from crazy-max/resp-build-ref
build: set build ref in response
2024-02-23 23:21:26 +01:00
CrazyMax
2edb7a04a9 build: set build ref in response
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-23 23:01:41 +01:00
CrazyMax
a0599c1c31 Merge pull request #2279 from crazy-max/build-test-shmsize-ulimit
test: build shm-size and ulimit
2024-02-23 23:00:31 +01:00
CrazyMax
eedf9f10e8 test: build shm-size and ulimit
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-23 22:44:30 +01:00
CrazyMax
d891634fc6 Merge pull request #2266 from crazy-max/container-driver-host-entl
driver: set network.host entitlement by default for container drivers
2024-02-23 22:41:12 +01:00
Tõnis Tiigi
af75d0bd7d Merge pull request #2235 from jsternberg/build-context-transfer-metric
metrics: measure context transfers for local source operations
2024-02-23 13:25:58 -08:00
CrazyMax
e008b846bb driver: set network.host entitlement by default for container drivers
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-23 22:23:27 +01:00
Tõnis Tiigi
fd11d93381 Merge pull request #2275 from crazy-max/buildkitd-flags-network-mode
driver: docs to set buildkitd network mode and add tests
2024-02-23 13:20:20 -08:00
CrazyMax
aa518f9b88 driver: test bridge network mode
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 23:07:50 +01:00
CrazyMax
b16bd02f95 vendor: github.com/moby/buildkit 8e3fe35738c2 (v0.13.0-dev)
full diff: 8e3fe35738...d6e142600e

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 23:07:50 +01:00
CrazyMax
69bd408964 docs(driver): set buildkitd network mode
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 23:07:50 +01:00
Tõnis Tiigi
d8e9c7f5b5 Merge pull request #2268 from crazy-max/dirver-align-flags
driver: make buildkitd "config" and "flags" names consistent
2024-02-22 13:50:24 -08:00
CrazyMax
fd54daf184 Merge pull request #2276 from crazy-max/codecov-token
ci: set codecov token
2024-02-22 22:43:11 +01:00
CrazyMax
9057bd27af ci: set codecov token
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 18:05:08 +01:00
CrazyMax
5a466918f9 build: enhance error message for unsupported attestations
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 15:30:53 +01:00
CrazyMax
56fc68eb7e driver: make buildkitd "config" and "flags" names consistent
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 10:26:18 +01:00
CrazyMax
ccfcf4bc37 Merge pull request #2272 from crazy-max/fix-docs-upstream
ci: update docs-upstream workflow
2024-02-22 10:25:28 +01:00
CrazyMax
560eaf0e78 ci: update docs-upstream workflow
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-22 10:07:48 +01:00
Tõnis Tiigi
daaa8f2482 Merge pull request #2242 from crazy-max/bake-ulimits-shmsize
bake: ulimits and shm-size support
2024-02-21 16:30:27 -08:00
Jonathan A. Sternberg
97052cf203 metrics: measure context transfers for local source operations
Measure the transfer size and duration of context transfers for various
categories of local source transfers from the progress stream that's
returned during the build.

Local source transfers are split into one of four categories:
* context
* dockerfile
* dockerignore
* namedcontext

Named contexts that are different names will be categorized under the
same metric.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-02-21 14:52:03 -06:00
Tõnis Tiigi
2eccaadce5 Merge pull request #2258 from jsternberg/docker-otel-non-experimental
metricutil: remove BUILDX_EXPERIMENTAL from internal docker reporting
2024-02-21 11:45:23 -08:00
Tõnis Tiigi
aa4317bfce Merge pull request #2267 from crazy-max/update-buildkit
vendor: github.com/moby/buildkit db304eb93126 (v0.13.0-dev)
2024-02-21 10:22:14 -08:00
CrazyMax
953cbf6696 vendor: github.com/moby/buildkit db304eb93126 (v0.13.0-dev)
full diff: d6e142600e...db304eb931

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-21 11:54:00 +01:00
CrazyMax
414f215929 Merge pull request #2265 from tonistiigi/bake-parent-eval
bake: avoid evaluating parent targets before child LLB loaded
2024-02-21 11:46:12 +01:00
Tonis Tiigi
698eb840a3 bake: avoid evaluating parent targets before child LLB loaded
Because of the way buildkit cache works if you have request
with external cache, if some vertices from the request have
already been evaluated and are available in the shared graph
BuildKit will not load cache keys from external source for such
vertices. This may mean that children of such vertices will
not load cache because there isn't a cache path through the parent.

To work around it, wait before child definition is loaded before
evaluating the parent.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-20 19:52:17 -08:00
Justin Chadwell
714b85aaaf Merge pull request #2262 from Granjow/patch-1
Fix typo in URL
2024-02-20 14:45:18 +00:00
Simon A. Eugster
fb604d4b57 Fix typo in URL
Signed-off-by: Simon A. Eugster <simon.eu@gmail.com>
2024-02-20 14:19:04 +01:00
CrazyMax
73d8969158 docs: more context around shm-size and ulimit usage
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 11:29:13 +01:00
CrazyMax
64e2b2532a bake: ulimits support
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 11:23:42 +01:00
CrazyMax
c2befc0c12 bake: shm-size support
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 11:23:42 +01:00
CrazyMax
345551ae0d test: fix message output
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-20 11:23:42 +01:00
CrazyMax
97e8fa7aaf Merge pull request #2253 from dvdksn/docs-cli-reference-urlscheme
docs: use absolute links and update link targets
2024-02-20 09:29:14 +01:00
David Karlsson
cdfc35d0b6 docs: update external link paths
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-02-20 08:55:41 +01:00
David Karlsson
ce66d8830d vendor: github.com/docker/cli-docs-tool v0.7.0
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-02-20 08:34:13 +01:00
Jonathan A. Sternberg
fe08cf2981 metricutil: remove BUILDX_EXPERIMENTAL from internal docker reporting
The `BUILDX_EXPERIMENTAL` check is removed from the docker otel
collector. We'll send metrics to the OTLP endpoint for docker desktop if
it is present and enabled regardless of experimental status.

The user-facing `OTEL` endpoints for enabling the metric reporting for
external use is still hidden behind the experimental flag. We'll likely
remove the experimental flag for this feature for v0.14.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-02-15 14:17:58 -06:00
Tõnis Tiigi
c9d1c41d20 Merge pull request #2225 from jsternberg/command-duration-metric
metrics: add build command duration metric
2024-02-14 15:38:41 -08:00
Jonathan A. Sternberg
bda968ad5d metrics: add build command duration metric
This adds a build duration metric for the build command with attributes
related to the buildx driver, the error type (if any), and which options
were used to perform the build from a subset of the options.

This also refactors some of the utility methods used by the git tool to
determine filepaths into its own separate package so they can be reused
in another place.

Also adds a test to ensure the resource is initialized correctly and
doesn't error. The otel handler logging message is suppressed on buildx
invocations so we never see the error if there's a problem with the
schema url. It's so easy to mess up the schema url when upgrading OTEL
that we need a proper test to make sure we haven't broken the
functionality.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-02-14 15:58:52 -06:00
Tõnis Tiigi
481384b185 Merge pull request #2112 from cpuguy83/dialstdio
Add dial-stdio command
2024-02-09 17:13:46 -08:00
Tõnis Tiigi
67d9385ce0 Merge pull request #2252 from ndeloof/rawjson
don't print build details when progress is rawjson
2024-02-09 16:40:50 -08:00
Nicolas De Loof
598bc16e5d don't print build details when progress is rawjson
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2024-02-09 13:28:44 +01:00
Brian Goff
760244ee3e Add dial-stdio command
This allows the buildx CLI to act a proxy to the configured instance.
It allows external code to use buildx itself as a driver for connecting
to buildkitd instances.

Instance and node selection should follow the same semantics as as
`buildx build`, including taking into account the `BUILDX_BUILDER` env
var and the `--builder` global flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2024-02-08 22:16:00 +00:00
CrazyMax
d0177c6da3 Merge pull request #1271 from crazy-max/ctn-restart
docker-container: restart-policy opt
2024-02-08 19:31:57 +01:00
Justin Chadwell
8f8ed68b61 Merge pull request #2250 from iankingori/add-npipe
driver: add npipe url scheme support
2024-02-07 13:46:01 +00:00
Ian King'ori
981cc8c2aa add npipe url scheme support
- enables remote builder and buildx create on windows
Signed-off-by: Ian King'ori <kingorim.ian@gmail.com>
2024-02-07 13:47:12 +03:00
CrazyMax
9822409b67 docker-container: restart-policy opt
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-06 14:33:36 +01:00
CrazyMax
328666dc6a Merge pull request #2249 from crazy-max/dockerfile-bump-docker
Dockerfile: update to Docker Engine v25.0.2
2024-02-06 11:03:35 +01:00
CrazyMax
42d2719b08 Merge pull request #2248 from crazy-max/bump-xx
update xx to 1.4.0
2024-02-06 11:00:09 +01:00
CrazyMax
3b33ac48d2 Dockerfile: update to Docker Engine v25.0.2
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-06 10:45:59 +01:00
CrazyMax
e0303dd65a update xx to 1.4.0
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-06 10:44:03 +01:00
CrazyMax
dab7af617a Merge pull request #2245 from thaJeztah/bump_buildkit
vendor: github.com/moby/buildkit 6bd81372ad6f (v0.13.0-dev)
2024-02-05 23:15:42 +01:00
CrazyMax
0326d2a5b1 Merge pull request #2246 from thaJeztah/bump_c8d_console
vendor: github.com/containerd/console v1.0.4
2024-02-05 18:49:20 +01:00
Sebastiaan van Stijn
b4c81a4d27 vendor: github.com/containerd/console v1.0.4
no diff; same commit, but tagged: https://github.com/containerd/console/compare/8f6c4e4faef5...v1.0.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 18:24:02 +01:00
Sebastiaan van Stijn
7b3c4fc714 vendor: github.com/moby/buildkit 6bd81372ad6f (v0.13.0-dev)
full diff: 6bd81372ad...d6e142600e

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 18:10:02 +01:00
Sebastiaan van Stijn
43ed470208 vendor: github.com/aws/aws-sdk-go-v2/config v1.26.6
vendor github.com/aws/aws-sdk-go-v2/config v1.26.6 and related dependencies.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 18:08:03 +01:00
Sebastiaan van Stijn
089982153f vendor: github.com/docker/cli v25.0.2
no changes in vendored code

full diff: https://github.com/docker/cli/compare/v25.0.1...v25.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 18:03:10 +01:00
Sebastiaan van Stijn
7393650008 vendor: github.com/docker/docker v25.0.2
no changes in vendored code

full diff: https://github.com/docker/docker/compare/v25.0.1...v25.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-05 18:01:55 +01:00
CrazyMax
b36c5196dd Merge pull request #2243 from crazy-max/bump-codecov
ci: bump codecov/codecov-action to 4
2024-02-05 16:29:37 +01:00
CrazyMax
1484862a50 Merge pull request #2244 from kushmansingh/upgrade-buildkit
Upgrade buildkit to v0.12.5
2024-02-05 16:25:19 +01:00
Kushagra Mansingh
e5c3fa5293 Upgrade buildkit to v0.12.5
Contains important security fixes https://github.com/moby/buildkit/releases/tag/v0.12.5

Signed-off-by: Kushagra Mansingh <12158241+kushmansingh@users.noreply.github.com>
2024-02-05 09:54:07 -05:00
CrazyMax
2c58e6003f ci: bump codecov/codecov-action to 4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-05 13:54:55 +01:00
CrazyMax
30ae5ceb6e Merge pull request #2238 from crazy-max/fix-win-console
vendor: github.com/containerd/console 8f6c4e4
2024-02-05 13:10:50 +01:00
CrazyMax
6ffb77dcda vendor: github.com/containerd/console 8f6c4e4
full diff: https://github.com/containerd/console/compare/v1.0.3...8f6c4e4faef5a326d2cd907097d04c0239ee5e2f

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-05 12:16:39 +01:00
CrazyMax
2c1f46450a Merge pull request #2237 from crazy-max/fix-bake-read-order
bake: fix definitions merge order
2024-02-02 13:42:17 +01:00
CrazyMax
052f279de7 bake: fix definitions merge order
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-02-02 10:55:25 +01:00
Justin Chadwell
89684021b3 Merge pull request #2230 from jedevc/imagetools-resolver-copy-dupe
fix: avoid modifying source during resolver.Copy
2024-02-01 10:30:32 +00:00
Justin Chadwell
95bdecc145 fix: avoid modifying source during resolver.Copy
Signed-off-by: Justin Chadwell <me@jedevc.com>
2024-01-31 14:44:10 +00:00
CrazyMax
082d5d70b2 Merge pull request #2228 from docker/dependabot/github_actions/peter-evans/create-pull-request-6.0.0
build(deps): bump peter-evans/create-pull-request from 5.0.2 to 6.0.0
2024-01-31 15:06:24 +01:00
dependabot[bot]
5b75930a6d build(deps): bump peter-evans/create-pull-request from 5.0.2 to 6.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.2 to 6.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](153407881e...b1ddad2c99)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-31 13:46:27 +00:00
CrazyMax
e41ab8d10d Merge pull request #2227 from crazy-max/fix-dependabot-conf
chore: ignore docker/docs deps with dependabot
2024-01-31 14:46:00 +01:00
CrazyMax
4b408c79fe Merge pull request #2205 from crazy-max/bump-compose-go
vendor: update compose-go to v2.0.0-rc.3
2024-01-31 14:31:50 +01:00
CrazyMax
cff7baff1c chore: ignore docker/docs deps with dependabot
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-31 14:23:34 +01:00
CrazyMax
5130700981 test: revert non-deterministic compose context path
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-31 14:15:57 +01:00
CrazyMax
13beda8b11 vendor: update compose-go to v2.0.0-rc.3
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-31 14:15:57 +01:00
Tõnis Tiigi
8babd5a147 Merge pull request #2215 from thaJeztah/bump_cobra_1.8
vendor: github.com/spf13/cobra v1.8.0
2024-01-30 17:57:41 -08:00
Tõnis Tiigi
cb856682e9 Merge pull request #2224 from jsternberg/otelsdk-package
otel: include service instance id attribute to resource and move to metricutil package
2024-01-30 14:45:24 -08:00
Jonathan A. Sternberg
c65b7ed24f otel: include service instance id attribute to resource and move to metricutil package
Add the service instance id to the resource attributes to prevent
downstream OTEL processors and exporters from thinking that the CLI
invocations are a single process that keeps restarting. The unique id
can be removed through downstream aggregation to prevent cardinality
issues, but we need some way to tell OTEL that it shouldn't reset the
counters.

Move the check for the experimental flag to its own package and then use
that invocation to prevent creating exporters so metrics are disabled
completely. This makes it so we don't have to check for the experimental
flag in every place we add metrics until we decide to make metrics
stable in general.

This also moves the OTEL initialization to a `util/metricutil` package
to be more consistent with the existing util naming and to differentiate
it from the upstream `metric` name. Using both `metrics` and `metric` as
import names was confusing since `metric` was an upstream dependency and
`metrics` was a local utility. `metricutil` matches with the existing
utilities and makes clear that it isn't a spelling mistake.

The record version metric has been removed since we weren't planning on
keeping that metric anyway and most of the information is now included
in the instrumentation library name and version. That function is
included as a utility in the `otel/sdk/metric` package to retrieve the
appropriate meter from the meter provider.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-01-30 16:27:02 -06:00
Tõnis Tiigi
2c3d7dab3f Merge pull request #2212 from crazy-max/bump-gotest-annotations
bump gotest-annotations to fa6141aedf23596fb8bdcceab9cce8dadaa31bd9
2024-01-28 17:34:36 -08:00
Sebastiaan van Stijn
13467c1f5d vendor: github.com/spf13/cobra v1.8.0
- release notes: https://github.com/spf13/cobra/releases/tag/v1.8.0
- full diff: https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0

Release notes highlights:

Features

- Support usage as plugin for tools like kubectl - this means that programs
  that utilize a "plugin-like" structure have much better support and usage
  (like for completions, command paths, etc.)
- Move documentation sources to site/content
- Add 'one required flag' group - this includes a new MarkFlagsOneRequired API
  for flags which can be used to mark a flag group as required and cause command
  failure if at least one is not used when invoked.
- Customizable error message prefix - This adds the SetErrPrefix and ErrPrefix
  APIs on the Command struct to allow for setting a custom prefix for errors
- feat: add getters for flag completions
- Feature: allow running persistent run hooks of all parents
- Improve API to get flag completion function

Bug fixes

- Fix typo in fish completions
- Fix grammar: 'allows to'
- powershell: escape variable with curly brackets
- Don't complete --help flag when flag parsing disabled
- Replace all non-alphanumerics in active help env var program prefix

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 16:11:21 +01:00
CrazyMax
d0c4bed484 Merge pull request #2198 from thaJeztah/bump_buildkit
vendor: github.com/moby/buildkit 6bd81372ad6f (master)
2024-01-26 14:35:29 +01:00
Sebastiaan van Stijn
dbaad32f49 vendor: github.com/moby/buildkit 6bd81372ad6f (master)
- tests: implement NetNSDetached method

full diff: 6e200afad5...6bd81372ad

Co-authored-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 13:05:26 +01:00
Sebastiaan van Stijn
528e3ba259 vendor: github.com/docker/cli v25.0.1
- cli-plugins: move socket code into common package
- cli-plugins: don't use abstract sockets on macOS
    - fixes CLI leaving behind plugin socket mount-points
- socket: return from loop after EOF

full diff: https://github.com/docker/cli/compare/v25.0.0-rc.1...v25.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 13:02:24 +01:00
Sebastiaan van Stijn
1ff261d38e vendor: github.com/docker/docker v25.0.1
- pkg/system: return even richer xattr errors

full diff: https://github.com/moby/moby/compare/v25.0.0-rc.1...v25.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:56:23 +01:00
Sebastiaan van Stijn
bef5d567b0 vendor: github.com/moby/sys/mountinfo v0.7.1
full diff: https://github.com/moby/sys/compare/mountinfo/v0.6.2...mountinfo/v0.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:40 +01:00
Sebastiaan van Stijn
da95d9f0ca vendor: golang.org/x/tools v0.14.0, golang.org/x/mod v0.13.0, golang.org/x/sync v0.4.0
full diff:

- https://github.com/golang/tools/compare/v0.10.0...v0.14.0
- https://github.com/golang/mod/compare/v0.11.0...v0.13.0
- https://github.com/golang/sync/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:40 +01:00
Sebastiaan van Stijn
7206e2d179 vendor: golang.org/x/sys v0.16.0
full diff: https://github.com/golang/sys/compare/v0.15.0...v0.16.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:40 +01:00
Sebastiaan van Stijn
736094794c vendor: github.com/google/uuid v1.5.0
full diff: https://github.com/google/uuid/compare/v1.3.1...v1.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:40 +01:00
Sebastiaan van Stijn
a399a97949 vendor: github.com/google/go-cmp v0.6.0
full diff: https://github.com/google/go-cmp/compare/v0.5.9...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:39 +01:00
Sebastiaan van Stijn
62a416fe12 vendor: github.com/containerd/containerd v1.7.12
full diff: https://github.com/containerd/containerd/compare/v1.7.11...v1.7.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-26 12:54:37 +01:00
CrazyMax
f6564c3147 bump gotest-annotations to fa6141aedf23596fb8bdcceab9cce8dadaa31bd9
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-26 10:04:30 +01:00
CrazyMax
b49911416c Merge pull request #2210 from crazy-max/bump-artifacts-action
bump actions/upload-artifact and actions/download-artifact to 4
2024-01-26 10:01:04 +01:00
CrazyMax
22c2538466 Merge pull request #2211 from docker/dependabot/github_actions/actions/setup-go-5
build(deps): bump actions/setup-go from 4 to 5
2024-01-26 09:46:20 +01:00
CrazyMax
1861405b1e ci(docs-upstream): pin reusable workflow
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-26 09:37:40 +01:00
CrazyMax
c9aeca19ce bump actions/upload-artifact and actions/download-artifact to 4
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-26 09:33:50 +01:00
dependabot[bot]
59827f5c27 build(deps): bump actions/setup-go from 4 to 5
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-26 08:28:22 +00:00
CrazyMax
827622421e Merge pull request #2206 from crazy-max/test-os
ci: test-unit job matrix for win/macos/ubuntu
2024-01-26 09:09:28 +01:00
Tõnis Tiigi
f0c5dfaf48 Merge pull request #2184 from laurazard/cli-signal-handling
Use Cobra's `command.Context()`, cancel execution when CLI is signalled
2024-01-25 10:37:45 -08:00
CrazyMax
703c765ec8 gitutil: check git bash env when testing
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-25 15:06:32 +01:00
CrazyMax
fb2c62a038 build: resolve 8.3 filename format to long one on Windows
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-25 15:06:32 +01:00
CrazyMax
eabbee797b ci: test-unit job matrix for win/macos/ubuntu
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-25 10:53:01 +01:00
CrazyMax
7e4021a43d Merge pull request #2204 from thaJeztah/ci_update_docker_version
Dockerfile: update to Docker Engine v25.0.1
2024-01-24 11:59:38 +01:00
Sebastiaan van Stijn
2478f300aa Dockerfile: update to Docker Engine v25.0.1
Update DOCKER_VERSION to the current release, so that tests run
against that version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 11:01:43 +01:00
CrazyMax
620c57c86c Merge pull request #2203 from thaJeztah/ci_update_buildkit_version
Dockerfile: update to buildkit v0.12.4
2024-01-24 10:30:24 +01:00
Sebastiaan van Stijn
8bea1cb417 Dockerfile: update to BuildKit v0.12.4
Update BUILDKIT_VERSION to the current release, so that tests run
against that version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-24 09:35:18 +01:00
Tonis Tiigi
147c7135b0 simplify signal handling for cobra context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-01-19 17:18:42 -08:00
Laura Brehm
650a7af0ae cobra/commands: cancel command context on signal
See https://github.com/docker/cli/pull/4599 and
https://github.com/docker/cli/pull/4769.

Since we switched to using the cobra command context instead of
`appcontext`, we need to set up the signal handling that was being
provided by `appcontext`, as well as configuring the context with
the OTEL tracing utilities also used by `appcontext`.

This commit introduces `cobrautil.ConfigureContext` which implements
the pre-existing signal handling logic from `appcontext` and cancels
the command's context when a signal is received, as well as doing
the relevant OTEL config.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-01-19 16:30:52 -08:00
Laura Brehm
4f738020fd deps: remove appcontext, use cmd.Context
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-01-19 16:29:12 -08:00
CrazyMax
d852568a29 Merge pull request #2186 from dvdksn/docs-fix-cli-reference-link
docs: update link to `docker build` reference
2024-01-19 16:20:54 +01:00
David Karlsson
68c3ac4f66 docs: update link to docker build reference
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2024-01-19 15:28:37 +01:00
Tõnis Tiigi
38afdf1f52 Merge pull request #2188 from AkihiroSuda/mark-experimental-flags
Mark experimental flags in `--help`
2024-01-17 21:21:29 -08:00
CrazyMax
b2e723e2a3 Merge pull request #2190 from thaJeztah/bump_golang
update to go1.21.6
2024-01-17 10:53:34 +01:00
Akihiro Suda
02c2073feb Mark experimental flags in --help
Prior to this commit, experimental flags were not distinguishable from
regular flags in `--help`

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2024-01-13 19:53:56 +09:00
Sebastiaan van Stijn
61dff684ad update to go1.21.6
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-12 23:28:24 +01:00
Tõnis Tiigi
78adfc80a9 Merge pull request #2155 from jsternberg/otel-exporter
metrics: send metrics to the otel collector endpoint when active
2024-01-08 14:46:58 -08:00
Tõnis Tiigi
7c590ecb9a Merge pull request #2140 from crazy-max/rm-multi
rm: support removing multiple builders at once
2024-01-08 11:08:46 -08:00
CrazyMax
24e043e375 rm: support removing multiple builders at once
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-08 01:02:03 +01:00
Jonathan A. Sternberg
7094eb86c9 metrics: send metrics to the otel collector endpoint when active
Introduce a meter provider to the buildx cli that will send metrics to
the otel-collector included in docker desktop if enabled.

This will send usage metrics to the desktop application but also send
metrics to a user-provided otlp receiver endpoint through the standard
environment variables.

This introduces a single metric which is the cli count for build and
bake along with the command name and a few additional attributes.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-01-05 14:50:54 -06:00
Tõnis Tiigi
81ea718ea4 Merge pull request #2179 from thaJeztah/go_connection_0.5.0
vendor: github.com/docker/go-connections v0.5.0
2024-01-05 12:46:33 -08:00
Tõnis Tiigi
9060cab077 Merge pull request #2175 from jsternberg/vendor-update
deps: update buildkit, vendor changes
2024-01-05 12:45:07 -08:00
Sebastiaan van Stijn
3cd6d8d6e4 vendor: github.com/docker/go-connections v0.5.0
no diff, as the tag is the same commit as we used already;
https://github.com/docker/go-connections/compare/fa09c952e3ea...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-01-05 18:29:16 +01:00
Jonathan A. Sternberg
ba43fe08f4 deps: update buildkit, vendor changes
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2024-01-05 11:17:43 -06:00
Tõnis Tiigi
6b63e7e3de Merge pull request #2176 from crazy-max/fix-builder-creation
driver(container): fix conditional statement for error handling
2024-01-05 08:31:00 -08:00
CrazyMax
57d737a13c driver(container): fix conditional statement for error handling
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-01-05 16:35:09 +01:00
Tõnis Tiigi
671347dc35 Merge pull request #2156 from crazy-max/frontend-attr-localdirs
build: set local dirs as frontend attributes
2024-01-02 23:58:32 -08:00
CrazyMax
02bc4e8992 build: set local dirs as frontend attributes
Set local dirs metadata if relative to VCS directory so
dockerfile path is tracked accurately in case vcs information
is not fulfilled.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-21 11:38:11 +01:00
CrazyMax
1cdefbe901 Merge pull request #2167 from crazy-max/fix-git-root-wsl
gitutil: sanitize root dir on WSL
2023-12-20 11:03:28 +01:00
CrazyMax
7694f0b9d8 Merge pull request #2166 from crazy-max/multi-golangci-lint
enable golangci-lint for supported platforms
2023-12-20 11:02:58 +01:00
CrazyMax
fa9126c61f Merge pull request #2171 from dvdksn/docs-no-cache-filter
docs: add --no-cache-filter example
2023-12-20 10:59:53 +01:00
David Karlsson
ebae070f7e docs: add --no-cache-filter example
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-12-20 08:58:52 +01:00
CrazyMax
617f538cb3 Merge pull request #2170 from laurazard/bump-buildkit
deps: update buildkit, vendor changes
2023-12-19 15:08:29 +01:00
Laura Brehm
0f45b629ad deps: update buildkit, vendor changes
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-12-19 14:01:05 +00:00
CrazyMax
ac5b3241b1 gitutil: sanitize root dir on WSL
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-19 09:27:32 +01:00
CrazyMax
ee24a36c4f enable golangci-lint for supported platforms
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-14 22:49:37 +01:00
CrazyMax
8484fcdd57 Merge pull request #2162 from crazy-max/ci-username-var
ci: use org-wide var as username
2023-12-14 14:35:19 +01:00
CrazyMax
45deb29f09 ci: use org-wide var as username
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-14 14:24:56 +01:00
CrazyMax
6641167e7d Merge pull request #2159 from docker/dependabot/github_actions/github/codeql-action-3
build(deps): bump github/codeql-action from 2 to 3
2023-12-14 10:13:58 +01:00
CrazyMax
9f4987997c Merge pull request #2154 from dvdksn/doc-annotations
docs: annotations
2023-12-14 10:07:18 +01:00
CrazyMax
8337c25fa4 Merge pull request #2157 from crazy-max/swtich-account-push-bin
ci: use public bot account to push bin image
2023-12-13 20:49:23 +01:00
dependabot[bot]
6b048e2316 build(deps): bump github/codeql-action from 2 to 3
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-13 18:52:37 +00:00
CrazyMax
54a1f0f0ea ci: use public bot account to push bin image
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-13 15:33:22 +01:00
CrazyMax
57dc45774a Merge pull request #2131 from crazy-max/fix-driver-infer-platform
build: infer platform from first node if none set
2023-12-12 13:27:51 +01:00
CrazyMax
9d8ac1ce2d build: move solveOpt to local struct type
*client.SolveOpt in driver code is only used by build code.
For a clear separation of concerns, move it to an internal
struct type only accessible by BuildWithResultHandler func.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-11 19:53:05 +01:00
CrazyMax
0a0252d9b3 Merge pull request #2150 from crazy-max/builder-create
builder: move builder creation logic to builder package
2023-12-11 01:34:15 -08:00
David Karlsson
c6535e9675 docs: add levels to bake file target.annotations
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-12-10 21:16:08 +01:00
David Karlsson
d762c76a68 docs: build --annotation
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-12-10 21:16:08 +01:00
David Karlsson
1091707bd5 docs: add lang tag for plaintext code blocks
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-12-10 20:35:08 +01:00
David Karlsson
a4c392f4db docs: imagetools create --annotation
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-12-10 20:35:08 +01:00
CrazyMax
e4880c5dd1 Merge pull request #2151 from docker/dependabot/github_actions/actions/setup-go-5
build(deps): bump actions/setup-go from 4 to 5
2023-12-07 07:43:59 -08:00
dependabot[bot]
b2510c6b94 build(deps): bump actions/setup-go from 4 to 5
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-06 18:12:24 +00:00
CrazyMax
5b5c4c8c9d Merge pull request #2146 from tonistiigi/gitutil-default-remote
gitutil: find default remote by tracking branch
2023-12-04 08:08:11 -08:00
CrazyMax
ceb5bc807c builder: move builder creation logic to builder package
This moves the builder creation logic to the builder package as we
plan to allow builder creation in the build ui of Docker Desktop so we
avoid drifting with the same logic in command package.

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-12-04 14:08:08 +01:00
CrazyMax
b2f705ad71 Merge pull request #2147 from tonistiigi/bake-shared-authprovider
bake: use same auth provider for bake targets
2023-12-01 03:24:17 -08:00
Tonis Tiigi
6028094e6b gitutil: find default remote by tracking branch
Using this command resolves remote based on remote
tracking branch of the curently selected branch and
should be more precise in case we can't predict if user
uses origin to mark upstream or their fork.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-11-30 23:33:51 -08:00
Tonis Tiigi
9516ce8e25 bake: use same auth provider for bake targets
The results from credential plugins are cached
and this reduces the lookup times.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-11-30 22:44:53 -08:00
CrazyMax
d82637582c Merge pull request #2145 from crazy-max/update-buildkit
vendor: github.com/moby/buildkit c9ee8491d74f (master)
2023-11-30 06:57:58 -08:00
CrazyMax
1e80c70990 vendor: github.com/moby/buildkit c9ee8491d74f (master)
full diff:

- https://github.com/containerd/containerd/compare/v1.7.8...v1.7.9
- 5ae9b23c40...c9ee8491d7

Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-11-29 16:40:13 +01:00
CrazyMax
54032316f9 build: infer platform from first node if none set
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-11-28 11:55:47 +01:00
Justin Chadwell
aac7a47469 build: fix incorrect solve opt platform from being set
This regression was introduced in 616fb3e55c,
with the node resolution refactor.

The core issue here is just that we would unconditionally set the
solve opt's platform to the default current platform, which was
incorrect. We can prevent this easily by having a special case for the
default case, like we had before, and then not setting the platforms
field on this (which keeping the resolution behavior which was
introduced).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-28 10:11:51 +00:00
Justin Chadwell
aa0aeac297 build: move solve opt out of duplicate map
This was more error prone, as opposed to the approach used prior to
616fb3e55c.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-28 10:10:50 +00:00
CrazyMax
cec4496d3b Merge pull request #1787 from crazy-max/ls-format
ls: format opt
2023-11-23 06:00:17 -08:00
CrazyMax
9368ecb67e Merge pull request #2121 from robmry/align_add-host_with_cli
Permit '=' separator and '[ipv6]' in --add-host
2023-11-23 02:33:23 -08:00
CrazyMax
20c947990c ls: format opt
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-11-22 21:11:24 +01:00
Rob Murray
eeeff1cf23 Permit '=' separator and '[ipv6]' in --add-host
Fixes docker/cli#4648

Make it easier to specify IPv6 addresses in the '--add-host' option by
permitting 'host=ip' in addition to 'host:ip', and allowing square
brackets around the address.

For example:

    --add-host=hostname:127.0.0.1
    --add-host=hostname:::1
    --add-host=hostname=::1
    --add-host=hostname=[::1]

Signed-off-by: Rob Murray <rob.murray@docker.com>
2023-11-22 10:52:14 +00:00
Tõnis Tiigi
752680e289 Merge pull request #2136 from dvdksn/docs-du-verbose
docs: add buildx du verbose example
2023-11-21 16:49:55 -08:00
David Karlsson
5bf02d9f7b docs: add buildx du verbose example
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-11-21 15:55:50 +01:00
Tõnis Tiigi
0962fdbb04 Merge pull request #2130 from jsternberg/remote-bootstrap-timeout
driver: add status reporting and a timeout to the remote driver bootstrap
2023-11-17 14:12:33 -08:00
Jonathan A. Sternberg
1f5562315b driver: add status reporting and a timeout to the remote driver bootstrap
This adds status reporting for the remote driver so it shows the length
of time it is spending waiting for the connection. If the service is
already present, this logger isn't shown but it should help provide a
message to show the user why the build is stalled.

A timeout of 20 seconds has been added to the bootstrap.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-11-17 15:33:28 -06:00
CrazyMax
a102d33738 Merge pull request #2128 from dvdksn/docs-du-reftypes
docs: update du cmd description
2023-11-17 04:51:11 -08:00
David Karlsson
940e0a4a3c docs: update du cmd description
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-11-17 11:02:20 +01:00
Tõnis Tiigi
a978b2b7a3 Merge pull request #2117 from dvdksn/docs-cli-reference-refresh
docs: minor cli reference editorial updates
2023-11-16 11:25:52 -08:00
David Karlsson
1326634c7d chore: add docs reminder comments for driver opts
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-11-16 17:00:37 +01:00
David Karlsson
7a724ac445 docs: minor cli reference editorial updates
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-11-16 16:05:38 +01:00
CrazyMax
55e164a540 Merge pull request #2122 from crazy-max/docs-fix-image-inspect
docs: fix imagetools inspect json format
2023-11-16 06:47:37 -08:00
CrazyMax
707ae87060 docs: fix imagetools inspect json format
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2023-11-16 14:50:56 +01:00
CrazyMax
cb37886658 Merge pull request #2118 from thaJeztah/update_engine_25
vendor: buildkit 5ae9b23c40a9 (master / v0.13.0-dev), docker v25.0.0-beta.1
2023-11-15 07:32:06 -08:00
Sebastiaan van Stijn
c855277d53 vendor: github.com/moby/buildkit 5ae9b23c40a9 (master / v0.13.0-dev)
full diff:

- 36ef4d8c0d...f098008783
- d5c1d785b0...5ae9b23c40

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:59:23 +01:00
Sebastiaan van Stijn
898a8eeddf vendor: github.com/docker/docker, github.com/docker/cli v25.0.0-beta.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:55:37 +01:00
Sebastiaan van Stijn
c857eaa380 vendor: github.com/docker/go-connections fa09c952e3ea (v0.5.0-dev)
full diff: https://github.com/docker/go-connections/compare/v0.4.0...fa09c952e3ea

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:54:44 +01:00
Sebastiaan van Stijn
55db25c21c vendor: github.com/docker/docker-credential-helpers v0.8.0
full diff: https://github.com/docker/docker-credential-helpers/compare/v0.7.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:53:58 +01:00
Sebastiaan van Stijn
f353814390 vendor: github.com/klauspost/compress v1.17.2
full diff: https://github.com/klauspost/compress/compare/v1.16.3...v1.17.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:52:35 +01:00
Sebastiaan van Stijn
271a467612 vendor: github.com/go-logr/logr v1.2.4
full diff: https://github.com/go-logr/logr/compare/v1.2.3...v1.2.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:51:13 +01:00
CrazyMax
b3b8c62ad4 Merge pull request #2111 from docker/dependabot/github_actions/actions/github-script-7
build(deps): bump actions/github-script from 6 to 7
2023-11-15 06:50:56 -08:00
Sebastiaan van Stijn
eacf2bdf3d vendor: github.com/cenkalti/backoff/v4 v4.2.1
no changes to vendored files

full diff: https://github.com/cenkalti/backoff/compare/v4.2.0...v4.2.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-15 15:48:18 +01:00
Tõnis Tiigi
d537b9e418 Merge pull request #2115 from crazy-max/rework-driver-resolution2
build: rework node resolution
2023-11-14 08:43:03 -08:00
Justin Chadwell
616fb3e55c build: rework node resolution
This patch reworks and updates the node resolution logic for selecting a
node from a builder.

The new implementation reworks the logic to make use of containerd's
platforms.Matcher interface instead of manually associated strings, and
additionally provides a few behavioural changes over the original
implementation, namely platforms can be matched with non-strict
semantics. e.g. i386 builds can be scheduled on an amd64 node, arm/v6
builds can be scheduled on an arm/v7 node.

We also add a new collection of tests for tracking regressions and
making the intended behaviour clearer.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-14 15:08:30 +01:00
Sebastiaan van Stijn
da5f853b44 vendor: github.com/containerd/containerd v1.7.8
full diff: https://github.com/containerd/containerd/compare/v1.7.7...v1.7.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 14:33:27 +01:00
Sebastiaan van Stijn
4932eecc3f vendor: google.golang.org/grpc v1.58.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 14:31:24 +01:00
Sebastiaan van Stijn
7f93616ff1 vendor: golang.org/x/oauth2 v0.10.0
full diff: https://github.com/golang/oauth2/compare/v0.5.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 14:30:28 +01:00
Sebastiaan van Stijn
ab58333311 vendor: google.golang.org/protobuf v1.31.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 14:29:26 +01:00
Sebastiaan van Stijn
9efaa2793d vendor: golang.org/x/tools v0.10.0
full diff: https://github.com/golang/tools/compare/v0.7.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-11-14 14:26:38 +01:00
Tõnis Tiigi
80aa28f75c Merge pull request #2113 from tonistiigi/wait-child-release2
build: wait from child targets to complete before session release
2023-11-13 18:24:48 -08:00
dependabot[bot]
8819ac1b65 build(deps): bump actions/github-script from 6 to 7
Bumps [actions/github-script](https://github.com/actions/github-script) from 6 to 7.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-11-13 18:55:01 +00:00
Tõnis Tiigi
0408f3ac45 Merge pull request #2093 from jsternberg/rootless-init-config
driver: docker-container driver uses --config correctly in rootless mode
2023-11-13 09:10:07 -08:00
Tonis Tiigi
7683ef9137 build: wait from child targets to complete before session release
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-11-07 14:41:33 -08:00
Jonathan A. Sternberg
3f423468df driver: docker-container driver uses --config correctly in rootless mode
The `docker-container` driver relies on the default config file location
for buildkit when writing the configuration file. When run in a rootless
version of docker (dind), the default location is different.

Instead of trying to figure out where the appropriate default location
is, this just writes the files to the same location and sets the
`--config` parameter explicitly. This flag is placed first so a
user-specified config option in `--buildkitd-flags` will take precedence
over the implicit config parameter.

This also fixes the `--config` option with the rootless image.
Previously, the config directory was being copied in a way that rendered
`/etc` unreadable and the configuration file wasn't readable either. It
also wasn't copied to the correct place. Now, `--config` is used to
specify the directory, `/etc` isn't included in the copied archive (so
the permissions aren't overwritten), and the directory is set as world
readable to be readable from the rootless buildkit process`.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-11-07 16:20:07 -06:00
CrazyMax
ff8bca206b Merge pull request #2100 from dvdksn/bake-empty-definition
bake: improve error message for empty definition
2023-11-06 03:17:12 -08:00
CrazyMax
08a70ecdcc Merge pull request #2106 from jedevc/fix-bake-matrix-with-target-access
bake: fix global target access when using a matrix
2023-11-06 03:16:09 -08:00
Justin Chadwell
d83da63320 bake: fix global target access when using a matrix
Previously, we would fail while trying to use the global "target" field
when using a matrix. The contents of the matrix really don't matter for
this.

What was happening was that we would copy the "target" property into the
child evaluation context, so that when it was updated on the parent, it
wouldn't propagate to the child. The correct behavior here is to avoid
copying variables from the target evaluation context if it is the
root.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-11-02 17:49:58 +00:00
CrazyMax
639e0bc5ed Merge pull request #1838 from c-ameron/feat/local-remote-files
command(bake): Specify local and remote bake files
2023-11-02 05:19:40 -07:00
David Karlsson
d0a9a81e2e bake: improve error message for empty definition
Closes #2095

Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-31 15:36:50 +01:00
CrazyMax
de1a560f07 Merge pull request #2099 from crazy-max/ci-codeql
ci: enable SAST scanning with CodeQL
2023-10-30 02:23:39 -07:00
CrazyMax
e168fd826c ci: enable SAST scanning with CodeQL
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-28 08:01:20 +02:00
Tõnis Tiigi
2f1b7a0131 Merge pull request #2098 from crazy-max/annotations-push
build: make annotations work with push flag
2023-10-26 09:14:23 -07:00
CrazyMax
f3871b158f build: make annotations work with push flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-26 12:01:10 +02:00
Tõnis Tiigi
deb9dbe9bb Merge pull request #1062 from crazy-max/bake-fix-attrs
bake: set attribute even if diagnosed as duplicated
2023-10-25 09:31:59 -07:00
Tõnis Tiigi
6f71ea8904 Merge pull request #2086 from ktock/debug-errormessage
debug: Following-up patches for error messages
2023-10-25 09:27:44 -07:00
CrazyMax
e437f7ba04 Merge pull request #1869 from crazy-max/fix-boot
builder: return error if all nodes fail to boot
2023-10-25 08:12:05 -07:00
Cameron Adams
abfc04f621 command(bake): Specify local and remote bake files
This adds the ability to source additional local build definition files when
sourcing Bake files via a remote url.
Prefixing a file with 'cwd://' will source a bake file on the local
machine, instead of the remote location.
Local files will be read/have precedence before remote files.

Usage:
```
docker buildx bake https://github.com/example/upstream.git --file cwd://docker-bake.override.hcl --print
```
This will source a default file from the example/upstream repository,
and also source a build definition from the local machine.

Also moves remote and local files reading logic to a func

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Cameron Adams <pnzreba@gmail.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-24 19:37:04 +02:00
CrazyMax
612dfdd813 builder: return error if all nodes fail to boot
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-24 10:59:04 +02:00
CrazyMax
ee19ce5ef2 Merge pull request #2076 from crazy-max/bake-display-files
bake: display read definition files in build output
2023-10-24 01:56:05 -07:00
CrazyMax
23c2498dee bake: display read definition files in build output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-23 16:26:18 +02:00
CrazyMax
390eedc50b Merge pull request #2096 from jedevc/vendor-buildkit
vendor: update buildkit to master@d5c1d785b042
2023-10-23 07:14:42 -07:00
Justin Chadwell
adc839aa40 vendor: update buildkit to master@d5c1d785b042
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 14:48:50 +01:00
CrazyMax
7838ade9f3 Merge pull request #2094 from crazy-max/update-compose
vendor: update compose-go to v1.20.0
2023-10-23 01:40:04 -07:00
CrazyMax
c043c9229e compose: fix deprecated Load func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-21 08:30:51 +02:00
CrazyMax
05a0fdf744 vendor: update compose-go to v1.20.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-21 08:29:01 +02:00
CrazyMax
dfb557b34f chore: fix lint issues
G601: Implicit memory aliasing in for loop. (gosec)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:39 +02:00
CrazyMax
d0d8bfbca4 bake: set attribute even if diagnosed as duplicated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:38 +02:00
CrazyMax
21e4eb89b2 bake: fork merged bodies interface logic from hcl repo and use it
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-20 20:07:38 +02:00
Kohei Tokunaga
14834e6085 debug: print error message on monitor instead of processes pkg
This can catch and print more errors on monitor side not only process-related
errors.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-20 14:09:32 +09:00
Kohei Tokunaga
267e30a19c debug: monitor: return last build result
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-20 14:09:28 +09:00
Tõnis Tiigi
be4fd7110d Merge pull request #2089 from dvdksn/docs-bakefile-lookup-order
docs: clarify bake file lookup order and merge
2023-10-19 09:22:29 -07:00
Tõnis Tiigi
24668122d9 Merge pull request #2088 from crazy-max/update-hcl
vendor: update hcl dependencies
2023-10-19 09:19:07 -07:00
Tõnis Tiigi
31d021a9ca Merge pull request #2015 from crazy-max/fix-bake-cwd-dockerfile
bake: local dockerfile support for remote definition
2023-10-19 09:18:20 -07:00
CrazyMax
7497e6481e bake: deny access to local dockerfile for remote invocation with local context
we don't currently support reading a remote Dockerfile with a local
context when doing a remote invocation because we automatically derive
the dockerfile from the context atm. To avoid mistakenly reading a local
Dockerfile, we check if the Dockerfile exists locally and if so, we
error out.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 17:41:05 +02:00
CrazyMax
de9d253f45 bake: local dockerfile support for remote definition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 17:41:05 +02:00
David Karlsson
f4f511201b docs: clarify bake file lookup order and merge
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-19 16:05:30 +02:00
CrazyMax
beca8b6adf vendor: update github.com/zclconf/go-cty to v1.14.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:52:01 +02:00
CrazyMax
457dc402d3 vendor: update github.com/hashicorp/go-cty-funcs to a090f58aa992
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:50:44 +02:00
CrazyMax
34b9a629a0 vendor: update github.com/hashicorp/hcl/v2 to v2.19.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 14:49:10 +02:00
CrazyMax
ad674e2666 Merge pull request #2084 from crazy-max/go-1.21
update to go 1.21.3
2023-10-19 03:55:10 -07:00
CrazyMax
503d483731 fix lint issues after update
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 12:31:24 +02:00
CrazyMax
6e5aefbb98 update golangci-lint to 1.54.2
also update config to show all issues at once and
set same exlucde rules as buildkit ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 12:24:16 +02:00
Tõnis Tiigi
7d2c9d5ef5 Merge pull request #2078 from crazy-max/fix-rm-state-group
build: fix racy localstate group removal
2023-10-18 16:54:21 -07:00
CrazyMax
1734abbb76 build: fix racy localstate group removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-18 19:12:46 +02:00
Kohei Tokunaga
b06a55cf53 debug: do not print error when error is nil
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-18 12:20:17 +09:00
Kohei Tokunaga
38137b29dd debug: reload: print error message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-18 11:25:24 +09:00
CrazyMax
fc7144f61d update to go 1.21.3
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-18 03:21:44 +02:00
Tõnis Tiigi
ac93a7fbfb Merge pull request #2006 from ktock/debug-command
debug: Add `buildx debug` command
2023-10-17 18:20:33 -07:00
CrazyMax
48f9b86b9a Merge pull request #2083 from crazy-max/test-host-gateway
test: docker host-gateway
2023-10-17 04:59:55 -07:00
CrazyMax
6c32a8c4c1 test: docker host-gateway
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 12:57:46 +02:00
CrazyMax
7a08248c4e Merge pull request #2082 from dvdksn/fix-docs-upstream-ci
ci: update docs upstream validation workflow
2023-10-16 16:39:05 -07:00
CrazyMax
05af608774 Merge pull request #2018 from crazy-max/driver-client-meta
driver(docker): opt to set additional dial meta to the client
2023-10-16 16:14:34 -07:00
CrazyMax
511e41386f driver(docker): opt to set additional dial meta to the client
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 01:01:17 +02:00
CrazyMax
fd251d2a7b builder: use variadic options in LoadNodes func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
CrazyMax
5836c24e7d driver: move Features and HostGatewayIP to specific driver
Adds a new HostGatewayIP entry in the Driver interface so we can
move Features and HostGatewayIP handling with cache to the very
specific driver (docker).

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
David Karlsson
c8f8a106ed ci: update docs upstream validation workflow
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-10-16 20:42:47 +02:00
Kohei Tokunaga
198764f116 debug: update docs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:36 +09:00
Kohei Tokunaga
0dd89f6029 monitor: print error information before launching monitor
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:35 +09:00
Kohei Tokunaga
8da8ee2aea controller: return original error to preserve stacktrace
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:35 +09:00
Kohei Tokunaga
6db8569f09 process: Do not print error log when process is canceled
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:30 +09:00
Kohei Tokunaga
5a0e4c1023 debug: set on=error by default
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:27 +09:00
Kohei Tokunaga
ded91da575 exec, rollback: return error when no session found
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:26 +09:00
Kohei Tokunaga
508b2ef0c6 debug: Add buildx debug command
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:21 +09:00
Tõnis Tiigi
05b8821625 Merge pull request #2048 from doreamon-design/feat/support-memory-opt-for-docker-container
feat: support memory/cpu driver options for docker-container
2023-10-12 10:05:54 -07:00
Tõnis Tiigi
01245e72ab Merge pull request #2072 from silvin-lubecki/undo-client-interface
Revert "Introduce a client interface, to abstract from buildkit client."
2023-10-11 16:40:28 -07:00
Silvin Lubecki
22e9e3342b Revert "Introduce a client interface, to abstract from buildkit client."
This reverts commit 2aa6d52b06.

Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
2023-10-11 22:13:23 +02:00
Silvin Lubecki
0e3911147a Merge pull request #2071 from silvin-lubecki/client-interface
(drivers): Introduce a client interface
2023-10-11 22:08:35 +02:00
Silvin Lubecki
2aa6d52b06 Introduce a client interface, to abstract from buildkit client.
This will allow further enhancement like caching remote driver responses.

Signed-off-by: Silvin Lubecki <silvin.lubecki@docker.com>
2023-10-06 14:24:04 +02:00
CrazyMax
561a4330cf Merge pull request #2065 from dvdksn/update-compose-go
vendor: update compose-go to v1.19.0
2023-10-03 22:26:17 +02:00
CrazyMax
7b4bc4f00a Merge pull request #2063 from crazy-max/node-workerid
builder: set node worker ids
2023-10-02 09:11:27 +02:00
CrazyMax
a012e0043b Merge pull request #2066 from jsternberg/early-flock-release
commands: release the file lock before attempting to boot during create
2023-09-30 12:40:02 +02:00
Justin Chadwell
2c2294fa43 Merge pull request #2068 from crazy-max/inspect-fix-platforms 2023-09-30 09:25:02 +01:00
CrazyMax
197824c6f2 builder: set node worker ids
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 07:10:02 +02:00
CrazyMax
22e61ef06f inspect: do not display platforms field if empty
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 07:08:47 +02:00
CrazyMax
159eac42f3 compose: test with develop section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 06:46:51 +02:00
CrazyMax
6c77b76b7b Merge pull request #2038 from crazy-max/localstate-group
build: support local state group
2023-09-30 06:01:14 +02:00
CrazyMax
130e9fe093 localstate: tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
e9fb769c60 build: handle local state group file removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
3dcb03452c build: support local state group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
9b7d30c9a0 build: opt to set build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:03:00 +02:00
CrazyMax
2134a1e104 build: set target to local state
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:03:00 +02:00
CrazyMax
cc6957d1cc build: move localstate logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:02:45 +02:00
Jonathan A. Sternberg
0878d5b22b commands: release the file lock before attempting to boot during create
If the boot command hung or took a long time, it blocked any read
operations (such as `buildx ls`).

When the boot happens, we no longer need to hold the file lock so we can
release it. Releasing multiple times is legal and causes whichever
release that is second to be a no-op so the defer is kept to ensure the
lock is released even when an error condition happens.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-09-29 15:53:38 -05:00
David Karlsson
c8002e58a4 vendor: update compose-go to v1.19.0
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-09-29 20:59:14 +02:00
Zero
cfcd1d9420 feat: support memory and cpu driver options for docker-container
Signed-off-by: Zero <tobewhatwewant@outlook.com>
2023-09-29 23:57:19 +08:00
CrazyMax
e6756d951a Merge pull request #2035 from jedevc/update-buildkit
vendor: update buildkit to master
2023-09-28 15:14:00 +02:00
Justin Chadwell
b9aad03e7a progress: create new display object after unpause
Without this, we attempt to overwrite the previous display in the lines
above which have been filled with logs from the monitor (see when used
with `--invoke`).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell
0bd6f3c7f5 remote: remove explicit backoff config from client opts
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
Justin Chadwell
e2ebab5f26 vendor: update buildkit to master@cbfd4023383d
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
CrazyMax
e018f8b6fb Merge pull request #1994 from crazy-max/load-progress
build: sublogger to show docker load progress output
2023-09-13 08:21:44 +02:00
CrazyMax
03bedfb3c3 build: sublogger to show docker load progress output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-12 22:24:29 +02:00
CrazyMax
bdaaca40a2 Merge pull request #2045 from docker/dependabot/github_actions/docker/bake-action-4
build(deps): bump docker/bake-action from 3 to 4
2023-09-12 21:23:04 +02:00
dependabot[bot]
bc021c89a8 build(deps): bump docker/bake-action from 3 to 4
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 3 to 4.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:55:11 +00:00
CrazyMax
798402314c Merge pull request #2046 from docker/dependabot/github_actions/docker/login-action-3
build(deps): bump docker/login-action from 2 to 3
2023-09-12 20:54:59 +02:00
CrazyMax
7cfb440136 Merge pull request #2044 from docker/dependabot/github_actions/docker/metadata-action-5
build(deps): bump docker/metadata-action from 4 to 5
2023-09-12 20:54:30 +02:00
CrazyMax
80358842c4 Merge pull request #2043 from docker/dependabot/github_actions/docker/setup-buildx-action-3
build(deps): bump docker/setup-buildx-action from 2 to 3
2023-09-12 20:54:16 +02:00
CrazyMax
77aedb751e Merge pull request #2042 from docker/dependabot/github_actions/docker/setup-qemu-action-3
build(deps): bump docker/setup-qemu-action from 2 to 3
2023-09-12 20:54:01 +02:00
dependabot[bot]
739ec964db build(deps): bump docker/login-action from 2 to 3
Bumps [docker/login-action](https://github.com/docker/login-action) from 2 to 3.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:26 +00:00
dependabot[bot]
320a3109f3 build(deps): bump docker/metadata-action from 4 to 5
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 4 to 5.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Upgrade guide](https://github.com/docker/metadata-action/blob/master/UPGRADE.md)
- [Commits](https://github.com/docker/metadata-action/compare/v4...v5)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:13 +00:00
dependabot[bot]
2c986bc184 build(deps): bump docker/setup-buildx-action from 2 to 3
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:07 +00:00
dependabot[bot]
6c31f43cc9 build(deps): bump docker/setup-qemu-action from 2 to 3
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-12 18:18:01 +00:00
CrazyMax
7b049b99c5 Merge pull request #1998 from dvdksn/build/docker-driver-errmsgs
build: improve error messages for docker driver
2023-09-12 11:32:32 +02:00
David Karlsson
bf5a70023c build: improve error messages for docker driver
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
Co-authored-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-09-12 09:39:51 +02:00
Justin Chadwell
8d001e338f Merge pull request #2040 from crazy-max/validate-build-args 2023-09-11 16:51:20 +01:00
CrazyMax
73ea0826ca build: validate build args and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-11 16:56:50 +02:00
Justin Chadwell
66e6dab26b Merge pull request #2020 from jedevc/build-annotation-flags 2023-09-11 15:55:01 +01:00
Justin Chadwell
0138f2a00f bake: add annotations field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:07 +01:00
Justin Chadwell
a59058e8a5 build: add --annotation shortcut flag
This extracts the same logic for parsing annotations from the imagetools
create command, and allows the same flags to be attached to the build
command.

These annotations are then merged into all provided exporters.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:31:04 +01:00
CrazyMax
f6b7a3c522 Merge pull request #2039 from thaJeztah/update_golang_1.20.8
update to go1.20.8
2023-09-11 16:30:46 +02:00
Justin Chadwell
8fe2070d10 imagetools: make annotation parser more generic
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Justin Chadwell
54bb799d15 imagetools: simplify return type of annotation parser
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-11 15:30:31 +01:00
Sebastiaan van Stijn
957044825f update to go1.20.8
go1.20.8 (released 2023-09-06) includes two security fixes to the html/template
package, as well as bug fixes to the compiler, the go command, the runtime,
and the crypto/tls, go/types, net/http, and path/filepath packages. See the
Go 1.20.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.8+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.7...go1.20.8

From the security mailing:

[security] Go 1.21.1 and Go 1.20.8 are released

Hello gophers,

We have just released Go versions 1.21.1 and 1.20.8, minor point releases.

These minor releases include 4 security fixes following the security policy:

- cmd/go: go.mod toolchain directive allows arbitrary execution
  The go.mod toolchain directive, introduced in Go 1.21, could be leveraged to
  execute scripts and binaries relative to the root of the module when the "go"
  command was executed within the module. This applies to modules downloaded using
  the "go" command from the module proxy, as well as modules downloaded directly
  using VCS software.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-39320 and Go issue https://go.dev/issue/62198.

- html/template: improper handling of HTML-like comments within script contexts
  The html/template package did not properly handle HMTL-like "<!--" and "-->"
  comment tokens, nor hashbang "#!" comment tokens, in <script> contexts. This may
  cause the template parser to improperly interpret the contents of <script>
  contexts, causing actions to be improperly escaped. This could be leveraged to
  perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39318 and Go issue https://go.dev/issue/62196.

- html/template: improper handling of special tags within script contexts
  The html/template package did not apply the proper rules for handling occurrences
  of "<script", "<!--", and "</script" within JS literals in <script> contexts.
  This may cause the template parser to improperly consider script contexts to be
  terminated early, causing actions to be improperly escaped. This could be
  leveraged to perform an XSS attack.

  Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this
  issue.

  This is CVE-2023-39319 and Go issue https://go.dev/issue/62197.

- crypto/tls: panic when processing post-handshake message on QUIC connections
  Processing an incomplete post-handshake message for a QUIC connection caused a panic.

  Thanks to Marten Seemann for reporting this issue.

  This is CVE-2023-39321 and CVE-2023-39322 and Go issue https://go.dev/issue/62266.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-11 15:53:03 +02:00
CrazyMax
42a0f3d504 Merge pull request #2037 from kd-collective/fix_broken_link
docs: remove broken link
2023-09-09 09:03:15 +02:00
Kijima Daigo
84ad208985 docs: remove broken link
Signed-off-by: Kijima Daigo <norimaking777@gmail.com>
2023-09-09 11:51:56 +09:00
CrazyMax
3631dc17c9 Merge pull request #2027 from thaJeztah/swap_reference
migrate reference github.com/distribution/reference
2023-09-08 15:49:08 +02:00
Sebastiaan van Stijn
bafdc63b8c migrate to github.com/distribution/reference v0.5.0
The "reference" package was moved to a separate module, which was extracted
from b9b19409cf

Also update compose-go, which also switched to distribution/reference;

full diff: https://github.com/compose-spec/compose-go/compare/v1.18.3...v1.18.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:41:35 +02:00
Sebastiaan van Stijn
51c94cd2a6 vendor: github.com/compose-spec/compose-go v1.18.3
- Parse service device count to int if possible
- introduce ResourceResolver to accept remote resources
- use include.env_file to resolve variables in included compose.yaml file
- remove potential dependencies to disabled services in ForServices
- ability to convert a mapping (back) to KEY=VALUE strings
- load: include details about included files on Project
- include disabled services
- local environment to override included .env
- load: move env var profile detection to option
- add support for multi-document yaml

full diff: https://github.com/compose-spec/compose-go/compare/v1.17.0...v1.18.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-06 17:41:35 +02:00
Tõnis Tiigi
31d88398bc Merge pull request #2031 from thaJeztah/bump_notary
vendor: github.com/theupdateframework/notary v0.7.0
2023-09-06 08:37:17 -07:00
Justin Chadwell
fbf6594758 Merge pull request #2034 from tonistiigi/trace-delegate-driver 2023-09-06 11:20:56 +01:00
Tonis Tiigi
f54a67de6d enable trace delegation for docker/remote driver
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-09-05 22:04:57 -07:00
CrazyMax
f35b2b7cab Merge pull request #2032 from docker/dependabot/github_actions/actions/checkout-4
build(deps): bump actions/checkout from 3 to 4
2023-09-05 08:36:03 +02:00
dependabot[bot]
29ba5ecef6 build(deps): bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 18:51:00 +00:00
Sebastiaan van Stijn
fb50d82fd8 vendor: github.com/theupdateframework/notary v0.7.0
update the dependency to v0.7.0 to be closer to what docker/cli uses;
https://github.com/theupdateframework/notary/compare/v0.6.1...v0.7.0

Note that docker/cli is slightly ahead of v0.7.0, and uses bf96a202a09a;
https://github.com/theupdateframework/notary/compare/v0.7.0...bf96a202a09a

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-09-04 15:37:45 +02:00
Justin Chadwell
87e8e4b847 Merge pull request #2029 from testwill/loop
chore: slice loop replace
2023-08-31 07:10:10 +02:00
guoguangwu
a71a24c0f4 chore: slice loop replace
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-08-31 12:24:02 +08:00
Tõnis Tiigi
76119b0f61 Merge pull request #2026 from ktock/fix-invoke-flag
debug: fix short-form custom command name on `--invoke` isn't used
2023-08-30 10:05:49 -07:00
Kohei Tokunaga
7843b5f417 debug: fix short-form custom command name on --invoke isn't used
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-08-30 18:41:16 +09:00
Justin Chadwell
da6662975f Merge pull request #2019 from jedevc/chore-bake-field-reorder 2023-08-24 14:17:55 +01:00
Justin Chadwell
de4dbb7d00 chore: reorder target fields
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-24 14:01:14 +01:00
Justin Chadwell
3bd4bca994 Merge pull request #2013 from jedevc/tests-add-imagetools-create
tests: add imagetools tests for copying manifests and indexes
2023-08-23 13:31:45 +01:00
CrazyMax
296832c90e Merge pull request #2014 from crazy-max/fix-compose-test
test: fix non-deterministic compose context path
2023-08-23 11:27:47 +02:00
CrazyMax
56d55a4137 test: fix non-deterministic compose context path
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-23 11:15:50 +02:00
CrazyMax
626e6f8fa3 Merge pull request #1905 from thaJeztah/cgroup_parent_description
update flag-description for --cgroup-parent
2023-08-23 10:15:47 +02:00
Justin Chadwell
5941bf0494 tests: add imagetools tests for copying manifests and indexes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 17:38:36 +01:00
CrazyMax
29a496cdab Merge pull request #2012 from jedevc/git-propogate-errors
git: propogate failure to locate git binary
2023-08-22 13:30:36 +02:00
Justin Chadwell
a43d9a67c7 git: fix error wrapping to ensure internal errors are propogated
Also, tidy up the error printing, so that now we always print out the
"current commit information was not captured by the build" message,
instead of just for not locating the git binary.

Before:

	WARNING: buildx: git was not found in the system. Current commit information was not captured by the build

After:

	WARNING: current commit information was not captured by the build: git was not found in the system: <error message>

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:17:17 +01:00
Justin Chadwell
c47eb3bf5a git: propogate failure to locate git binary
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-22 12:16:19 +01:00
CrazyMax
a97e1641a4 Merge pull request #2000 from jedevc/fix-race-container-creation
docker-container: avoid fail if container conflict
2023-08-21 16:11:02 +02:00
Akihiro Suda
86ae8ea854 Merge pull request #1999 from crazy-max/update-k8s
vendor: bump k8s to v0.26.7
2023-08-10 23:26:24 +09:00
Justin Chadwell
d37d483097 docker-container: avoid fail if container conflict
Fixes the race condition where two boots are executed simultaneously
across multiple processes.

We initially check to see if the container exists, but if during
container creation we get a name conflict, we don't treat this error as
a hard failure, and instead move immediately into waiting for the node
to boot.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-10 14:29:36 +01:00
CrazyMax
4e96faa201 vendor: bump k8s to v0.26.7
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-10 15:22:24 +02:00
Tõnis Tiigi
e5419ef6d7 Merge pull request #1927 from crazy-max/fix-load-status
build: read body response to check for erroneous image export to docker
2023-08-10 12:29:03 +03:00
CrazyMax
14747a490a Merge pull request #1971 from glours/bump-compose-go-v1.17.0
bump compose-go version to v1.17.0 to fix issue with depends_on
2023-08-08 21:03:59 +02:00
Justin Chadwell
e5cee892ed Merge pull request #1965 from mqasimsarfraz/qasim/oci-annotations 2023-08-08 14:36:26 +01:00
CrazyMax
ef4b984df4 build: read body response to check for erroneous image export to docker
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-08 06:34:24 +02:00
CrazyMax
a8f402e28d Merge pull request #1993 from kenyon/typo-fix
README: fix typo
2023-08-08 06:32:44 +02:00
Kenyon Ralph
2eba99b40b README: fix whitespace
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-07 11:34:13 -07:00
Kenyon Ralph
7686fa1f16 README: fix typo
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-07 11:33:54 -07:00
Justin Chadwell
51b9bab245 Merge pull request #1987 from jedevc/vendor-buildkit-master-tests 2023-08-07 17:08:16 +01:00
CrazyMax
6b5758f4cd Merge pull request #1821 from jedevc/allow-debug-env
commands: consume DEBUG environment variable
2023-08-07 15:06:47 +02:00
CrazyMax
bd375a14a8 Merge pull request #1940 from kenyon/patch-1
README: clarify the Linux package install instructions
2023-08-07 14:43:29 +02:00
CrazyMax
b01693f63e bake: test compose include
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-08-04 17:24:30 +02:00
Guillaume Lours
4a059d5144 adapt compose unit tests, build context is now transformed to absolute paths by compose-go
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-08-04 17:23:12 +02:00
Guillaume Lours
f3775c0046 bump compose-go version to v1.17.0 to fix issue with depends_on
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-08-04 17:23:04 +02:00
Justin Chadwell
50fbdd86f9 Merge pull request #1985 from jsternberg/integration-test/version 2023-08-04 16:14:42 +01:00
Justin Chadwell
1f61de0fcc Merge pull request #1988 from jedevc/fix-attests-on-docker-driver 2023-08-04 16:07:21 +01:00
Justin Chadwell
e206c585bb build: error on attests on non-multiplatform driver
On drivers that do not support multi-platform builds (the default
`docker` driver), we do not support building attestations (unless using
the containerd store).

We need to check this feature before attempting to build using
attestations.

Also adds a test to ensure that attestations can be pushed to registries
at all, and that it adequately fails on the docker driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 15:28:32 +01:00
Justin Chadwell
5e46d8057d tests: add unsupported features detection skeleton
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell
4e7709e54c vendor: update buildkit to master@b49a8873179b
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 12:00:18 +01:00
Justin Chadwell
5ed8f1b7d9 tests: avoid hardcoded driver check in testImageIDOutput
To detect if there is a docker daemon available, we can use the sandbox
.DockerAddress() function.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-08-04 11:22:10 +01:00
Jonathan A. Sternberg
1d12c1f5b3 Integration test for docker buildx version
An integration test for `docker buildx version` has been created. The
integration test checks that there is one line output, the output is
composed of three sections, and that these sections could feasibly be
the package path, version, and revision information.

The intention of the checks is to find obvious errors in the output like
the package path not existing or the version and revision being swapped.
It is not intended to assert that these values must be certain values
because it is assumed these values may vary depending on the build
process for buildx.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
2023-08-03 09:51:03 -05:00
Qasim Sarfraz
3ef93e081c tests: add testImagetoolsAnnotation integration test
Signed-off-by: Qasim Sarfraz <qasimsarfraz@microsoft.com>
2023-08-03 12:16:40 +02:00
Qasim Sarfraz
18894a8e3a allow annotations for OCI image index
Signed-off-by: Qasim Sarfraz <qasimsarfraz@microsoft.com>
2023-08-03 12:08:04 +02:00
Justin Chadwell
13ec635988 Merge pull request #1914 from ktock/updatedebugdocs 2023-08-03 09:52:13 +01:00
Tõnis Tiigi
f804b8fa4b Merge pull request #1982 from thaJeztah/update_go1.20.7
update to go1.20.7
2023-08-03 10:59:18 +03:00
Kenyon Ralph
21a55ff9a1 README: clarify the Linux package install instructions
Signed-off-by: Kenyon Ralph <quic_kralph@quicinc.com>
2023-08-02 13:55:22 -07:00
Sebastiaan van Stijn
dd350284df update to go1.20.7
Includes a fix for CVE-2023-29409

go1.20.7 (released 2023-08-01) includes a security fix to the crypto/tls
package, as well as bug fixes to the assembler and the compiler. See the
Go 1.20.7 milestone on our issue tracker for details:

- https://github.com/golang/go/issues?q=milestone%3AGo1.20.7+label%3ACherryPickApproved
- full diff: https://github.com/golang/go/compare/go1.20.6...go1.20.7

From the mailing list announcement:

[security] Go 1.20.7 and Go 1.19.12 are released

Hello gophers,

We have just released Go versions 1.20.7 and 1.19.12, minor point releases.

These minor releases include 1 security fixes following the security policy:

- crypto/tls: restrict RSA keys in certificates to <= 8192 bits

  Extremely large RSA keys in certificate chains can cause a client/server
  to expend significant CPU time verifying signatures. Limit this by
  restricting the size of RSA keys transmitted during handshakes to <=
  8192 bits.

  Based on a survey of publicly trusted RSA keys, there are currently only
  three certificates in circulation with keys larger than this, and all
  three appear to be test certificates that are not actively deployed. It
  is possible there are larger keys in use in private PKIs, but we target
  the web PKI, so causing breakage here in the interests of increasing the
  default safety of users of crypto/tls seems reasonable.

  Thanks to Mateusz Poliwczak for reporting this issue.

View the release notes for more information:
https://go.dev/doc/devel/release#go1.20.7

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-08-02 00:06:05 +02:00
Tõnis Tiigi
c010d3de8d Merge pull request #1967 from thaJeztah/update_cli
vendor: github.com/docker/docker, github.com/docker/cli v24.0.5
2023-08-01 11:51:44 +03:00
Justin Chadwell
d11dbbf9f7 Merge pull request #1978 from tonistiigi/imagetools-test 2023-07-31 15:39:42 +01:00
Tonis Tiigi
75cdceb9f1 tests: add integration test for imagetools create
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-07-31 17:17:11 +03:00
Tõnis Tiigi
10ff93f190 Merge pull request #1968 from jedevc/reset-dont-modify-input
progress: don't modify ResetTime inputs
2023-07-31 16:02:23 +03:00
CrazyMax
bf00185809 Merge pull request #1681 from crazy-max/fup-buildinfo
docs: update since buildinfo removal
2023-07-31 10:55:30 +02:00
CrazyMax
90f03e57c2 Merge pull request #1972 from crazy-max/docs-fix-create
docs: fix platform example for create command
2023-07-31 10:54:26 +02:00
Justin Chadwell
a59fd3ebfe Merge pull request #1970 from ktock/entrypointconfig 2023-07-27 15:21:37 +01:00
Kohei Tokunaga
3eb490153d remote controller: Fix entrypoint interaction bugs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-07-27 09:12:52 +09:00
CrazyMax
d957d8b987 docs: fix platform example for create command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-26 17:50:58 +02:00
Justin Chadwell
5a1f252bd9 progress: don't modify ResetTime inputs
No other parts of the progress rendering modify the inputs, so we should
avoid this as well.

This actually fixes an edge case in pushWithMoby which writes the same
VertexStatus multiple times, modifying the timestamps and similar.
However, if the operation takes long enough the small time difference
can accumulate, and move the Start time far into the past.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-25 09:35:43 +01:00
Sebastiaan van Stijn
ab4585f38c vendor: github.com/docker/cli v24.0.5
Fix a panic when `auths: null` is found in the CLI config file.

full diff: https://github.com/docker/cli/compare/v24.0.4...v24.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:09:50 +02:00
Sebastiaan van Stijn
3003045c0b vendor: github.com/docker/docker v24.0.5
- client: Client.postHijacked: use Client.buildRequest

full diff: https://github.com/moby/moby/compare/36e9e796c6fc...v24.0.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:09:02 +02:00
Sebastiaan van Stijn
a6f3f290b4 vendor: golang.org/x/net v0.10.0
- http2: properly discard data received after request/response body is closed
- http2: don't reuse connections that are experiencing errors
- internal/socks: permit authenticating with an empty password

full diff: https://github.com/golang/net/compare/v0.8.0...v0.10.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:05:20 +02:00
Sebastiaan van Stijn
27d072a099 vendor: golang.org/x/text v0.9.0
no changes in vendored files

full diff: https://github.com/golang/text/compare/v0.8.0...v0.9.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:04:17 +02:00
Sebastiaan van Stijn
8e3df1943c vendor: golang.org/x/term v0.8.0
no changes in vendored files

full diff: https://github.com/golang/term/compare/v0.6.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:03:14 +02:00
Sebastiaan van Stijn
8c54de66ce vendor: golang.org/x/sys v0.8.0
full diff: https://github.com/golang/sys/compare/v0.7.0...v0.8.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 09:02:03 +02:00
Sebastiaan van Stijn
06b9ac2dc4 vendor: github.com/sirupsen/logrus v1.9.3
Fix a potential denial of service in logrus.Writer() that could be triggered
by logging text longer than 64kb without newlines.

full diff: https://github.com/sirupsen/logrus/compare/v1.9.0...v1.9.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-25 08:59:00 +02:00
Justin Chadwell
b8739d7441 Merge pull request #1934 from jedevc/use-buildkit-client-wait 2023-07-19 16:23:54 +01:00
Justin Chadwell
23fe02993b Merge pull request #1963 from jedevc/split-to-driver-pairs-fix-type 2023-07-19 15:57:43 +01:00
Justin Chadwell
1d177f00d2 chore: tidy splitToDriverPairs to avoid unneccessary int
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 14:43:50 +01:00
CrazyMax
ceaba7011f Merge pull request #1938 from jonapich/feature/custom-annotations
kubernetes driver // allow custom annotations and labels
2023-07-19 14:21:12 +02:00
Jonathan Piché
9c06f383ba allow custom annotations and labels into kubernetes manifests
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>

Signed-off-by: Jonathan Piché <jpiche@coveo.com>
2023-07-19 07:48:26 -04:00
Justin Chadwell
e11c5e3e96 remote: use buildkit's client.Wait method to bootstrap
This native implementation uses GRPC level waiting, instead of starting
a busy loop. We also a manual max backoff of one second to improve
responsiveness.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-19 12:39:19 +01:00
CrazyMax
f5719f3017 Merge pull request #1959 from thaJeztah/update_cli
vendor: github.com/docker/cli v24.0.4
2023-07-19 11:57:02 +02:00
CrazyMax
163babdca7 Merge pull request #1961 from crazy-max/fix-internal-build
build: set remote bake def and remote dockerfile as internal solve
2023-07-19 11:47:52 +02:00
Sebastiaan van Stijn
094d1aded8 commands: NewRootCmd: remove obsolete logrus filter hook
This hook was added in 278f94a8b6 and
72758fef22 to suppress spurious warnings
printed by the CLI's cli/connhelper/commandconn package;
3fb4fb83df/cli/connhelper/commandconn/commandconn.go (L203-L214)

Those logs were removed in a5ebe2282a
so we can remove the hook.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-19 11:04:45 +02:00
CrazyMax
05ef20b434 build: set remote bake def and remote dockerfile as internal solve
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-19 04:18:28 +02:00
Sebastiaan van Stijn
cc718b3444 vendor: github.com/docker/cli v24.0.4
full diff: https://github.com/docker/cli/compare/v24.0.2...v24.0.4

notable changes:

- ssh: fix error on commandconn close, add ping and default
- commandconn: return original error while closing

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-18 17:13:42 +02:00
Justin Chadwell
e98e8f6ac9 Merge pull request #1948 from thaJeztah/buildkit_0.12 2023-07-18 15:05:18 +01:00
CrazyMax
36541ed9d5 Merge pull request #1954 from crazy-max/result-handle-internal
build: mark result handle build as internal
2023-07-17 17:05:34 +02:00
CrazyMax
418ea82d3a build: mark result handle build as internal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 16:53:12 +02:00
Sebastiaan van Stijn
130bbda00e vendor: github.com/moby/buildkit v0.12.1-0.20230717122532-faa0cc7da353
full diff:

- https://github.com/moby/buildkit/compare/20230620112432...v0.12.0
- https://github.com/moby/buildkit/compare/v0.12.0...faa0cc7da3536923d85b74b2bb2d13c12a6ecc99

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:37:27 +02:00
Sebastiaan van Stijn
2666bd6996 vendor: github.com/tonistiigi/vt100 v0.0.0-20230623042737-f9a4f7ef6531
full diff: 8066bb9726...f9a4f7ef65

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:43 +02:00
Sebastiaan van Stijn
ff2c8da803 vendor: github.com/tonistiigi/fsutil v0.0.0-20230629203738-36ef4d8c0dbb
full diff: 9e7a6df485...36ef4d8c0d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:43 +02:00
Sebastiaan van Stijn
e094296f37 vendor: github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb
full diff: 4e3ac2762d...02993c407b

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 15:35:40 +02:00
Justin Chadwell
7c3b77fb36 Merge pull request #1950 from thaJeztah/remove_imageutil_dead_code 2023-07-17 12:26:05 +01:00
CrazyMax
fb4c4f07ca Merge pull request #1941 from crazy-max/fix-kube-config
k8s: fix missing kubeconfig check from endpoint
2023-07-17 12:36:27 +02:00
Sebastiaan van Stijn
b9e25e82cf util/imagetools: remove unused Resolver.ImageConfig
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-17 12:35:42 +02:00
CrazyMax
089036da29 Merge pull request #1946 from crazy-max/update-go
update go to 1.20.6
2023-07-17 12:01:26 +02:00
CrazyMax
1123bfed10 hack(generated-files): bump golang image to bookworm
#7 [internal] load metadata for docker.io/library/golang:1.20.6-buster
#7 ERROR: docker.io/library/golang:1.20.6-buster: not found

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 11:18:37 +02:00
CrazyMax
7f2293308b update go to 1.20.6
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 10:30:00 +02:00
CrazyMax
a65131f9d3 Merge pull request #1945 from crazy-max/bump-docker
vendor: github.com/docker/docker@24.0 36e9e79
2023-07-17 10:29:40 +02:00
CrazyMax
8a3a646c61 vendor: github.com/docker/docker@24.0 36e9e79
client: define a "dummy" hostname to use for local connections
fixes "http: invalid Host header" errors when compiling with go1.20.6
or go1.19.11

full diff: https://github.com/docker/docker/compare/v24.0.2...36e9e796c6fc

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-17 10:06:04 +02:00
CrazyMax
4384947be1 k8s: fix missing kubeconfig check from endpoint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-07-15 21:51:56 +02:00
Tõnis Tiigi
69421182ca Merge pull request #1891 from droopy4096/auth-token
Add Bearer token support
2023-07-12 11:57:47 -07:00
Justin Chadwell
068382f5df Merge pull request #1936 from jedevc/hack-set-go-version-to-1.20.5 2023-07-12 14:49:59 +01:00
Justin Chadwell
c4bec05466 hack: force go version to 1.20.5
A temporary workaround for "http: invalid Host header" introduced in
go 1.20.6.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-12 14:31:23 +01:00
CrazyMax
89e1ac0a6e Merge pull request #1894 from crazy-max/fix-host-gateway
build: fix host-gateway handling
2023-07-03 21:53:27 +02:00
Justin Chadwell
b84e0e11b4 Merge pull request #1918 from crazy-max/docs-hidden-fix 2023-07-03 11:50:04 +01:00
Justin Chadwell
d95f5f8f3b Merge pull request #1925 from dvdksn/test/build-progress 2023-07-03 10:48:52 +01:00
David Karlsson
b4c0941683 tests: add basic build progress test
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-07-02 21:09:33 +02:00
Justin Chadwell
cf9798cede Merge pull request #1919 from crazy-max/fix-build-details-link 2023-06-30 13:51:23 +01:00
CrazyMax
20d2501edc test: build details output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-30 13:48:02 +02:00
CrazyMax
d45601fdc6 build: missing newline when printing build details on error
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-30 13:48:02 +02:00
CrazyMax
c81a9a89cf Merge pull request #1913 from yastanotheruser/master
Controller: Include CgroupParent in build.Options
2023-06-30 00:07:15 +02:00
CrazyMax
87b9f9ecfb docs: update generated content
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-27 17:21:05 +02:00
CrazyMax
cbc473359a vendor: update cli-docs-tool to 0.6.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-27 17:21:04 +02:00
Justin Chadwell
2eba60db75 Merge pull request #1916 from jedevc/add-local-bake-test 2023-06-27 11:28:47 +01:00
Justin Chadwell
0dcbed3f53 tests: add simple local bake test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-27 11:16:44 +01:00
Kohei Tokunaga
ca08eb65e2 docs: debug: update the output of help command to the latest message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-27 15:17:01 +09:00
Kohei Tokunaga
6f37d9bee7 monitor: attach: fix typo in long help message
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-27 15:16:32 +09:00
Jhan S. Álvarez
e65f6b8c8b controller: include CgroupParent in build.Options
Signed-off-by: Jhan S. Álvarez <alvarezpcuser@gmail.com>
2023-06-26 23:18:53 -05:00
CrazyMax
707dc43d55 Merge pull request #1903 from crazy-max/fix-bake-compose-profiles
bake: ignore profiles in compose definitions
2023-06-26 15:02:53 +02:00
CrazyMax
8cbb7a9319 build: fix host-gateway handling
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-24 17:47:48 +02:00
CrazyMax
4f5a56aadb Merge pull request #1904 from thaJeztah/cleanup_ParseEntitlements
utils/buildflags: ParseEntitlements(): use BuildKit's parsing
2023-06-23 14:26:44 +02:00
Sebastiaan van Stijn
399beb53d9 utils/buildflags: ParseEntitlements(): use BuildKit's parsing
Use buildkit's parsing of entitlements to make sure that accepted
values match what's accepted by BuildKit.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-23 10:44:30 +02:00
Sebastiaan van Stijn
7dec9fd6e7 update flag-description for --cgroup-parent
This attempts to make it clearer that the --cgroup-parent option is only used
for the containers used during build. Instead of mentioning "build container",
I opted for using "RUN instructions" (to match the --network description),
although this may not be ideal (as it assumes the "Dockerfile" front-end, which
of course may not be the case).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-23 10:28:50 +02:00
CrazyMax
120f3a8918 bake: ignore profiles in compose definitions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-22 17:44:35 +02:00
CrazyMax
bd672eaf5b Merge pull request #1886 from crazy-max/docker-local
build: prefer local image resolution for docker driver
2023-06-21 11:20:24 +02:00
CrazyMax
c2500ea2d8 build: prefer local image resolution for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 11:05:06 +02:00
Justin Chadwell
a4663b4b2e Merge pull request #1900 from cyphar/build-go_extra_flags 2023-06-21 09:59:19 +01:00
Aleksa Sarai
57c618b83a build: add GO_EXTRA_FLAGS argument
This is useful for setting things like -buildmode=pie when packaging
docker-buildx for distributions.

Signed-off-by: Aleksa Sarai <cyphar@cyphar.com>
2023-06-21 18:48:05 +10:00
CrazyMax
b3a4f95110 Merge pull request #1897 from crazy-max/test-containerd-snap
test: register docker worker with containerd snapshotter
2023-06-21 00:11:31 +02:00
CrazyMax
28a1eb3527 test: fix testImageIDOutput
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
75ecc15958 test: fix inspect and ls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
2235ebce2f test: register docker worker with containerd snapshotter
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 17:04:21 +02:00
CrazyMax
7147463418 dockerfile: update docker to 24.0.2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 16:50:38 +02:00
Justin Chadwell
010e4c8d54 Merge pull request #1890 from jedevc/tests-share-docker-container-backend 2023-06-20 14:24:09 +01:00
Justin Chadwell
6f394a0691 tests: set a dedicated buildx config dir for each worker
This should help reduce any unexpected config conflict between workers.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:40:45 +01:00
Justin Chadwell
efd7279118 ci: run docker-container tests in parallel
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:51 +01:00
Justin Chadwell
601056f3a7 tests: share single docker between docker-container backends
This means that we can run our docker-container tests in parallel again,
which can help speed up our test runs by a *significant* amount.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:28:49 +01:00
Justin Chadwell
0a7f96cbfb vendor: update buildkit to master@2d91ddcceedc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-20 12:27:46 +01:00
CrazyMax
1c530c2fe0 Merge pull request #1896 from dvdksn/docs/add-experimental-debugmonitor
docs: add experimental annotation for debug-shell command
2023-06-20 12:46:26 +02:00
David Karlsson
1e576dd7c6 chore: make docs
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
2023-06-20 12:05:30 +02:00
CrazyMax
7a5472153b docs: set experimental annotation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-20 11:42:02 +02:00
Dmitry Makovey
b986ce566b Add Bearer token
Signed-off-by: Dmitry Makovey <dmakovey@gitlab.com>
2023-06-16 14:55:09 -06:00
CrazyMax
daba16f4be Merge pull request #1879 from crazy-max/fix-ctx-validation
builder: skip name validation for docker context
2023-06-15 14:05:37 +02:00
CrazyMax
ee36e2264e Merge pull request #1880 from jedevc/fix-dockerfile-cwd-join
bake: fix incorrect dockerfile resolution against `cwd://` context
2023-06-15 14:05:13 +02:00
CrazyMax
329e98d9f0 Merge pull request #1883 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.2
build(deps): Bump peter-evans/create-pull-request from 5.0.1 to 5.0.2
2023-06-15 13:55:26 +02:00
dependabot[bot]
f4513f7028 build(deps): Bump peter-evans/create-pull-request from 5.0.1 to 5.0.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.1 to 5.0.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](284f54f989...153407881e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-06-14 18:59:43 +00:00
CrazyMax
b1c5449428 builder: skip name validation for docker context
Although a builder from the store cannot be created unless
it has a valid name, this is not the case for a Docker context.

We should skip name validation when checking a node from the
store and fall back to finding one from Docker context instead.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-14 14:19:30 +02:00
Justin Chadwell
431732f5d1 bake: fix incorrect dockerfile resolution against cwd:// context
We need to resolve the strip the cwd:// prefix before attempting to
resolve the dockerfile. Otherwise, we'll get the cwd:// prefix in the
dockerfile name, which isn't stripped out later.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-14 12:18:53 +01:00
CrazyMax
687feca9e8 Merge pull request #1877 from jedevc/cache-dockerutil-features
dockerutil: cache feature collection
2023-06-13 11:42:22 +02:00
Justin Chadwell
d4a2c8d0c3 dockerutil: cache feature collection
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-13 09:47:38 +01:00
Justin Chadwell
bef42b2441 Merge pull request #1876 from tonistiigi/remove-history-request 2023-06-13 09:38:05 +01:00
CrazyMax
2de333fdd3 check history api support once
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-13 10:29:22 +02:00
Tonis Tiigi
1138789f20 avoid extra client for history API detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-12 17:39:09 -07:00
Justin Chadwell
1f4ac09ffb Merge pull request #1874 from thaJeztah/bump_engine_24.0.2 2023-06-09 11:39:13 +01:00
CrazyMax
26a8ffb393 Merge pull request #1873 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@67a08623b95a
2023-06-09 12:14:11 +02:00
CrazyMax
9b7aada99b Merge pull request #1716 from distorhead/use-docker-cli-in-stream
build(cli): allow passing in-stream using command.Cli
2023-06-09 12:11:47 +02:00
Sebastiaan van Stijn
fd6207695b vendor: github.com/docker/cli v24.0.2
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:44 +02:00
Sebastiaan van Stijn
def96d2bf4 vendor: github.com/docker/docker v24.0.2
no changes in vendored files

full diff: https://github.com/docker/docker/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:01 +02:00
Justin Chadwell
f5f00e68ef bake(cli): allow passing in-stream using command.Cli
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:53 +01:00
Justin Chadwell
14aebe713e debug-shell(cli): allow passing in-stream using command.Cli
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:35 +01:00
Justin Chadwell
9d2388e6f5 vendor: update buildkit to master@67a08623b95a
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:36:52 +01:00
Timofey Kirillov
75e2c46295 build(cli): allow passing in-stream using command.Cli
Use command.Cli::In() which is always initialized either to os.Stdin, or to user-specified stream:
* 5be21394cb/cli/command/cli.go (L494)
* https://github.com/docker/cli/blob/master/cli/command/cli_options.go#L16C1-L26

Signed-off-by: Timofey Kirillov <timofey.kirillov@flant.com>
2023-06-09 12:19:02 +03:00
Justin Chadwell
2c02db8db4 Merge pull request #1835 from ktock/long-form-json 2023-06-08 17:10:37 +01:00
Kohei Tokunaga
e304a05d2a docs: monitor: mention about long form flag
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:03 +01:00
Kohei Tokunaga
14c1ea0e11 invoke: Allow JSON array for long form flags
This commit allows specifying a JSON array to the long-form arg, entrypoint and
envvars.
Non-JSON-array value can still be specified. Buildx treats the value as a JSON
array only when it can be parsed as a JSON array.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:02 +01:00
Justin Chadwell
c30bcade2c Merge pull request #1870 from tianon/invoke-entrypoint-fixes 2023-06-08 16:03:24 +01:00
Tianon Gravi
62bfb19db4 Fix a couple --invoke entrypoint interaction bugs
When running `--invoke` against images that have `Cmd` set, the interactions with `Entrypoint` start to cause issues like the following:

    /usr/local/bin/bash: /usr/local/bin/bash: cannot execute binary file

Or:

    sh: can't open 'bash': No such file or directory

This patch fixes those by explicitly setting `Cmd` to be empty if it is unspecified and `Entrypoint` is being set, which matches `docker`'s behavior:

    $ docker image inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' bash
    ["docker-entrypoint.sh"] + ["bash"]
    $ docker create --name foo --entrypoint bash bash
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["bash"] + null
    $ docker rm foo
    $ docker create --name foo bash ls
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["docker-entrypoint.sh"] + ["ls"]

(There are still some weird edge cases in the interaction between the `InvokeConfig` and the original image config, but this fixes the most irritating for me and the rest are going to be deeper changes that are possibly less acceptable. 😅)

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-07 16:33:28 -07:00
Justin Chadwell
47e34f2684 Merge pull request #1813 from jedevc/oci-export-on-containerd 2023-06-07 15:19:54 +02:00
Justin Chadwell
3d981be4ad Merge pull request #1850 from ktock/monitor-longhelp 2023-06-07 15:09:35 +02:00
CrazyMax
5d94b0fcc7 Merge pull request #1866 from crazy-max/bump-compose
vendor: github.com/compose-spec/compose-go v1.14.0
2023-06-07 14:55:55 +02:00
CrazyMax
569c66fb62 Merge pull request #1867 from crazy-max/bump-testify
vendor: github.com/stretchr/testify v1.8.4
2023-06-07 12:19:33 +02:00
CrazyMax
93f7fbdd78 Merge pull request #1865 from crazy-max/bump-semver
vendor: github.com/Masterminds/semver/v3 v3.2.1
2023-06-07 12:19:17 +02:00
Tõnis Tiigi
ea06685c11 Merge pull request #1858 from potherca-contrib/fix/typo-Shortand
Fix typo "Shortand" -> "Shorthand"
2023-06-06 14:03:40 -07:00
Ben Peachey
eaba4fa9e6 Update docs with result from make docs command.
Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-06-06 21:43:41 +02:00
CrazyMax
99e3882e2a Merge pull request #1841 from crazy-max/desktop
build: display build details link
2023-06-06 19:06:19 +02:00
CrazyMax
0a2f35970c build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
CrazyMax
ab5f5e4169 build: fix quiet mode
We are using the quiet flag option and we are not taking
progress quiet mode into account

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:47:55 +02:00
Justin Chadwell
696770d29c Merge pull request #1840 from jedevc/fix-check-path-for-bake-cmd-context 2023-06-06 17:49:04 +02:00
CrazyMax
b47b4e5957 vendor: github.com/stretchr/testify v1.8.4
full diff https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:44:58 +02:00
CrazyMax
9a125afba0 vendor: github.com/compose-spec/compose-go v1.14.0
https: //github.com/compose-spec/compose-go/compare/v1.13.4...v1.14.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:34:34 +02:00
Justin Chadwell
d34103b0d9 bake: fix potential context entitlements escape
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
c820350b5e tests: refactor cmd helpers to allow configuring cwd, etc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
61a7854659 tests: add bake test for remote subdir context
Fixed in 12b6a3ad9a, but now we have
regression tests! So we can add a check that we don't break this
behavior again.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
e859ebc12e tests: add bake test for remote cmd context override
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
ef997fd6d0 bake: fix BAKE_CMD_CONTEXT relative path resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
76c96347ff tests: add basic remote bake context test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
48d7dafbd5 git: update gitutil test utilities
- Adds a new GitServeHTTP function to start an http server to serve a
  target git repository.
- Adds a new GitDir helper method to get the path to the .git
  directory
- Updates the GitAdd method to take a variable number of files

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
d03e93f6f1 test: tmpdir can be a test helper
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
fcb7810a38 Merge pull request #1854 from crazy-max/inspect-features 2023-06-06 16:59:58 +02:00
CrazyMax
459d94bdf1 vendor: github.com/Masterminds/semver/v3 v3.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 16:55:59 +02:00
Justin Chadwell
7cef021a8a Merge pull request #1804 from jedevc/fixup-solve 2023-06-06 09:12:10 +02:00
Justin Chadwell
c6db4cf342 build: clarify NewResultHandle comment
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 09:08:30 +02:00
CrazyMax
6c9436fbd5 inspect: display builder features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-05 23:11:58 +02:00
Justin Chadwell
a906149930 Merge pull request #1862 from jedevc/update-contributing-tests
docs: update CONTRIBUTING.md with specific test notes
2023-06-01 21:55:12 +01:00
Justin Chadwell
af328fe413 docs: update CONTRIBUTING.md with specific test notes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-01 11:34:20 +01:00
Justin Chadwell
183a73abae build: docker exporter should instead use oci with containerd
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:59:55 +01:00
Justin Chadwell
b7f0b3d763 build: clear exports for secondary solve request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:56 +01:00
Justin Chadwell
5b27d5a9f6 build: cleanup res if returned in basic build
In practice, this shouldn't happen, but the check is good to include
anyways.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:03 +01:00
Ben Peachey
8f24c58f4d Fix typo in commands/build.go and docs/reference/buildx_build.md
`Shortand` -> `Shorthand`

Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-05-31 13:40:58 +02:00
Justin Chadwell
cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell
8d822fb06c build: move main solve request into main gateway call
Now, we always perform the full solve request in the main gateway call.
This ensures that progress works properly, and makes the lifetime
semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to
always return a single ResultContext, even though the details of how a
gateway is created is different:
- For a failed build, we can just keep the gateway open.
- For a successful build, we immediately open another gateway and
  re-evaluate the build definition in that gateway. This should give an
  instant cache hit (since the build was just successful).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:46:46 +01:00
CrazyMax
0758a9b75d Merge pull request #1856 from crazy-max/history-feature-update
driver: update history api check
2023-05-31 10:27:54 +02:00
CrazyMax
f8fa526678 driver: update history api check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-31 10:10:54 +02:00
Tõnis Tiigi
4abff3ce12 Merge pull request #1852 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@c36941f4a10e
2023-05-30 16:14:26 -07:00
CrazyMax
e7034f66bc Merge pull request #1846 from crazy-max/history-capability
driver: check history capability
2023-05-30 20:25:31 +02:00
CrazyMax
8c65e4fc1d driver: add context to Features interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
d196ac347e driver: cache features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
9b723ece46 driver: check history capability
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:17 +02:00
Justin Chadwell
5e2f8bd64a Merge pull request #1853 from jedevc/logrus-show-error 2023-05-30 10:29:49 +01:00
Justin Chadwell
5788ab33d2 logutil: print error from WithError if found
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 10:00:24 +01:00
CrazyMax
f1788002e1 Merge pull request #1851 from crazy-max/test-flow
test: update workflow
2023-05-30 10:58:17 +02:00
Justin Chadwell
6c62225d1b vendor: update buildkit to master@c36941f4a10e
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 09:10:52 +01:00
CrazyMax
38b4eef5c6 ci: make release and bin-image depend on test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
a4db138c5e docs: update CONTRIBUTING.md to run tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
55377b2b0f make: test-unit and test-integration targets
Also switch test target to run the test script.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
98dedd3225 ci: cache integration tests base and build in dedicated step
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
74b121be66 test: opt to avoid building the integration tests image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
b9cf46785b test: use trap to remove cache volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
ecf8dd0a26 test: type integration not needed
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
73c17ef4d2 ci: send test reports to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
e762e46b4b ci: merge test workflow to build one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
Kohei Tokunaga
cafeedba79 monitor: add long help for commands
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-29 23:23:56 +09:00
CrazyMax
17bdbbd3c3 Merge pull request #1831 from crazy-max/fix-ctn-certs-path
driver(docker-container): fix incorrect path when writing certs
2023-05-26 16:15:21 +02:00
Justin Chadwell
2dae553d18 build: update outline fallback image
Subrequests have been included in docker/dockerfile:1.5 labs, so we can
update the fallback to point to this release.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin Chadwell
91c17f25fb build: tidy up print func
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
CrazyMax
63fc01e08a Merge pull request #1842 from crazy-max/update-xx
update xx to 1.2.1
2023-05-26 11:22:33 +02:00
Justin Chadwell
354ccc9469 Merge pull request #1844 from tonistiigi/imageid-fixes 2023-05-26 10:21:53 +01:00
Tonis Tiigi
68ae67720a build: fix writing correct image ID with -q
Container driver wrote manifest digest that had a
mismatch with --iidfile output.

When --iidfile was set the --metadata-file was not
written.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:20:29 -07:00
Tonis Tiigi
b273db20c3 remote: allow docker exporter on remote driver
There is no reason why one can't make a docker tarball or
load to Docker instance from remote driver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:18:57 -07:00
CrazyMax
0ae88ecc4d update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-25 15:26:56 +02:00
Justin Chadwell
341fb65f6f Merge pull request #1830 from jedevc/fix-missing-driver-in-build 2023-05-25 11:35:01 +01:00
Justin Chadwell
69a9c6609a Merge pull request #1699 from jedevc/bake-attestation-override 2023-05-25 09:54:34 +01:00
CrazyMax
1c96fdaf03 driver(docker-container): fix incorrect path when writing certs
Replace invalid characters in the name of the base directory of
certificates using the name of the registry which can contain
for example a colon when the port is specified.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-23 15:36:38 +02:00
Justin Chadwell
c77bd8a578 build: fix missing "docker" driver name in build progress
This was missing, since the driver property can only be fully populated
after loading nodes from disk. So we add logic to load the nodes, and
check for an error, which ensures that the "docker" driver is always
correctly present in the progress description.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:37:22 +01:00
Justin Chadwell
e5f701351c Merge pull request #1829 from jedevc/vendor-buildkit-master 2023-05-23 10:23:24 +01:00
Justin Chadwell
09798cdebd vendor: update buildkit to master@212ab16a39b1
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:16:09 +01:00
CrazyMax
0dfc35d558 Merge pull request #1822 from thaJeztah/vendor_engine_24.0.1
vendor: github.com/docker/docker and github.com/docker/cli v24.0.1
2023-05-23 11:11:34 +02:00
CrazyMax
8085f57a3a Merge pull request #1825 from thaJeztah/remove_k8s_replace
go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
2023-05-23 11:10:31 +02:00
Sebastiaan van Stijn
d582a21acd go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
Replace rules are not inherited by consumers of buildx as a module, and as
such would default to use the v0.26.2 version. Removing the replace rules
also removes various (indirect) dependencies (although brings in some new
packages from k8s itself).

The "azure" and "gcp" authentication packages in k8s.io/go-client are now
no longer functional, so removing those imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 16:10:49 +02:00
CrazyMax
580820a4de Merge pull request #1824 from thaJeztah/k8s_remove_openstack
cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
2023-05-22 16:09:30 +02:00
Sebastiaan van Stijn
b7e8afc61b cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
Kubernetes v1.26 removed the in-tree openstack cloud provider. While buildx
has a replace rule to pin the k8s dependencies to an older version, such
replace rules are non-transitive, which means that consumers of buildx as
a module could potentially end up with non-resolvable dependencies.

The used code was already non-functional since c41b006be1
(buildx v0.5.0-rc1);

a6b0959276/vendor/k8s.io/client-go/plugin/pkg/client/auth/openstack/openstack_stub.go (L26-L36)

See:

- 9bbf01bae9
- 98e81a7784

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 13:01:54 +02:00
Sebastiaan van Stijn
a8a637e19d vendor: github.com/docker/cli v24.0.1
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:16:37 +02:00
Sebastiaan van Stijn
79632a4c4c vendor: github.com/docker/docker v24.0.1
no changes in vendored files

full diff: https://github.com/moby/moby/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:08:30 +02:00
CrazyMax
a6b0959276 Merge pull request #1440 from crazy-max/gc-policies
builder: add worker gc policies and labels
2023-05-22 11:46:46 +02:00
CrazyMax
6d7142b057 Merge pull request #1820 from jedevc/use-docker-distribution
vendor: use docker/distribution instead of distribution/distribution
2023-05-22 11:17:01 +02:00
Justin Chadwell
d0bff18cee commands: consume DEBUG environment variable
When running in standalone mode, the --debug flag passed to docker
cannot be passed. The docker cli also supports a DEBUG env var, however,
in standalone mode this won't be consumed.

This patch reads the contents of the DEBUG environment variable, and
enables debugging logs when it's been set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:52:23 +01:00
Justin Chadwell
7e39644f69 Merge pull request #1816 from bpaquet/fix_session_token 2023-05-22 09:34:07 +01:00
Bertrand Paquet
adc6349b28 Fix AWS Authentication when mixing static creds and IAM profile
When the user supply static creds, we must not enrich them with a
session token which is unrelated.

Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
2023-05-22 09:26:28 +01:00
Justin Chadwell
f558fd8b22 vendor: use docker/distribution instead of distribution/distribution
This doesn't require pulling in a new direct dependency, we already use
docker/distribution throughout imagetools and build.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:24:41 +01:00
Justin Chadwell
432e16ef70 Merge pull request #1806 from thaJeztah/update_engine2 2023-05-22 09:19:35 +01:00
Sebastiaan van Stijn
8c86c2242a vendor: github.com/docker/cli v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:47:38 +02:00
Sebastiaan van Stijn
75ad5d732b vendor: github.com/docker/docker v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:23 +02:00
Sebastiaan van Stijn
9bd0202312 vendor: github.com/spf13/cobra v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:20 +02:00
Sebastiaan van Stijn
367f114cc7 vendor: github.com/opencontainers/runc v1.1.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:43:16 +02:00
Sebastiaan van Stijn
2959ce205e vendor: github.com/klauspost/compress v1.16.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:41:29 +02:00
Sebastiaan van Stijn
75b5c6560f vendor: github.com/moby/term v0.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:39:47 +02:00
Justin Chadwell
4429ccbcc2 Merge pull request #1817 from jedevc/docs-monitor-invoke-parts 2023-05-19 16:40:47 +01:00
Justin Chadwell
c59fc18325 Merge pull request #1770 from jedevc/integration-tests 2023-05-19 16:18:31 +01:00
Justin Chadwell
4ce80856b3 docs: add info for on-error and debug-shell values for invoke
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:51 +01:00
Justin Chadwell
af3feec4ea docs: fixup typo in buildkit console output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:30 +01:00
Justin Chadwell
90c849f5ef bake: merge attest entries
This ensures that `target.attest=["type=sbom,<value>"]` can be
appropriately merged when `--sbom=true` or `--set
target.attest=type=sbom`.

To merge, we simply naively take the last valid value.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:32:58 +01:00
Justin Chadwell
6024212ac8 ci: add integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
2d124e0ce9 test: add basic integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
e61a8cf637 vendor: update buildkit to master@31c870e82a48
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:25:31 +01:00
Justin Chadwell
167cd16acb Merge pull request #1797 from crazy-max/fix-image-create 2023-05-18 09:06:45 +01:00
Tõnis Tiigi
1dd31fefcb Merge pull request #1810 from nicks/patch-1
fix two bugs in bake-reference.md
2023-05-17 21:12:19 -07:00
Nick Santos
5a12b25bab Update bake-reference.md
Before this change, there were two bugs:
- the HCL was not valid. in hcl, argument names can't be quoted
- the target argument should be a real target

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-05-17 19:09:42 -04:00
CrazyMax
b702188b65 imagetools(create): set correct media type when combining manifests
When using imagetools create and combining multiple sources
we should check the media type of each manifest and set
the right media type for the manifest list.

If there is a mismatch we set OCI index as best effort.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-16 11:46:58 +02:00
Justin Chadwell
060ac842bb Merge pull request #1794 from jedevc/fix-printer-start-race 2023-05-12 09:43:58 +01:00
Justin Chadwell
31d1b778ff Merge pull request #1795 from thaJeztah/vendor_distribution_v2.8.2 2023-05-12 09:43:07 +01:00
Sebastiaan van Stijn
1cd4b54810 vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2-beta.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 19:08:16 +02:00
CrazyMax
c54926c5b2 Merge pull request #1767 from jedevc/issue-template-no-bug-feature
github: don't auto-add kind/ labels
2023-05-11 17:22:23 +02:00
Justin Chadwell
10aea8e970 Merge pull request #1791 from jedevc/docs/bake-reference-matrix
Document new bake matrix builds
2023-05-11 15:43:49 +01:00
Justin Chadwell
be6542911f progress: make printer ready only after pausing logrus
This fixes a possible race where messages printed directly after calls
to NewPrinter may appear before the printer starts. With this change, we
delay all of the logs until after.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 15:23:41 +01:00
CrazyMax
9b07f6510a Merge pull request #1793 from jedevc/move-path-resolution-to-controller
controller: move path resolution into controller package
2023-05-11 15:01:21 +02:00
Justin Chadwell
9ee19520dd controller: move path resolution into controller package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:56:05 +01:00
Justin Chadwell
878faae332 docs: add matrix docs for bake
Co-authored-by: David Karlsson <david.karlsson@docker.com
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:44:28 +01:00
Justin Chadwell
eaf38570e7 github: don't auto-add kind/bug labels
kind labels should only be manually added by maintainers after
confirming the issue. Otherwise, it can be tricky to sort through all
the issues when filtering for legitimate bugs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:43:25 +01:00
CrazyMax
167340df17 Merge pull request #1780 from techvelocity/elranshefer/add-support-for-proxy-url
kubernetes driver: add support for proxy-url
2023-05-11 14:40:23 +02:00
Justin Chadwell
e61a1da7fc Merge pull request #1748 from dvdksn/docs/bake-reference
docs: move and rewrite bake reference
2023-05-11 13:00:21 +01:00
David Karlsson
f8483d7243 docs: refactor bake file reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-05-11 12:54:08 +01:00
Justin Chadwell
2c8a9aad76 Merge pull request #1782 from jedevc/docs-monitor
docs: add initial guide about debugging monitor mode
2023-05-11 11:29:21 +01:00
Justin Chadwell
32009a701c Merge pull request #1786 from jedevc/debug-multi-platform
debug: evaluate all refs to allow multi-platform debugging
2023-05-11 09:54:13 +01:00
Justin Chadwell
0cbc316f76 docs: add initial guide about debugging monitor mode
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:53:21 +01:00
Justin Chadwell
45fccef3f3 debug: evaluate all refs to allow multi-platform debugging
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:44:50 +01:00
Justin Chadwell
fdcb4e2fb9 Merge pull request #1784 from jedevc/support-new-compose-features
Support new compose features in bake
2023-05-11 09:44:02 +01:00
Justin Chadwell
4a0a67d7a2 Merge pull request #1783 from jedevc/fix-bake-sub-dir-context
bake: copy remote context contents to root
2023-05-11 09:40:25 +01:00
CrazyMax
855d49ff58 Merge pull request #1788 from crazy-max/update-docker
vendor: github.com/docker/docker v23.0.6
2023-05-10 17:22:14 +02:00
CrazyMax
384e873db0 Merge pull request #1790 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-05-10 16:17:53 +02:00
CrazyMax
60e72ba989 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:43:24 +02:00
CrazyMax
45a2ae6762 vendor: github.com/docker/docker v23.0.6
full diff:
- https://github.com/docker/docker/compare/v23.0.1...v23.0.6
- https://github.com/docker/cli/compare/v23.0.1...v23.0.6

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:26:20 +02:00
Justin Chadwell
2eeef180ea Merge pull request #1746 from ktock/resultcleanup
controller: avoid "context canceled" error on cleanup
2023-05-10 13:24:12 +01:00
Kohei Tokunaga
8fd81f5cfd controller: avoid "context canceled" error on cleanup
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-10 12:56:33 +01:00
Justin Chadwell
1eb9ad979e Merge pull request #1785 from jedevc/default-to-local-controller
controller: default to using local controller
2023-05-10 11:18:55 +01:00
Justin Chadwell
77e0e860f8 controller: default to using local controller
--detach shouldn't be the default yet, since it still has the potential
to leak some sessions in odd edge cases, and is slightly more painful to
debug.

For now, we should set the local controller as the default, with the
idea that we can change it back in the future.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:51:58 +01:00
Justin Chadwell
e228c398f4 bake: support compose additional contexts
compose-go v1.13.0 supports the new additional_contexts to allow passing
additional build context during build, so we should map this to bake's
contexts property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:41:37 +01:00
Justin Chadwell
5d06406f26 bake: support compose inline dockerfile
compose-go v1.13.0 supports the new dockerfile_inline to allow including
a dockerfile inline in the compose definition, so we should map this to
bake's dockerfile-inline property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:39:00 +01:00
Justin Chadwell
cb061b684c Merge pull request #1753 from nicksieger/compose-go-v1.13.4
compose go v1.13.4
2023-05-10 09:25:10 +01:00
Elran Shefer
29b427ce13 kubernetes driver: add support for proxy-url
Signed-off-by: Elran Shefer <elran.shefer@velocity.tech>
2023-05-10 08:36:39 +03:00
Tõnis Tiigi
4fa7cd1fc2 Merge pull request #1772 from tonistiigi/container-hang-fix
container-driver: avoid hanging on container panic
2023-05-09 10:27:11 -07:00
Justin Chadwell
12b6a3ad9a bake: copy remote context contents to root
When resolving remote contexts locally in bake, then we need to ensure
that we properly unpack the contents of that context to the root
directory, instead of leaving it in the subdirectory.

Otherwise, any files will be found in the wrong location. Along with
this change, we also need a change to the dockerfile location lookup to
ensure that it is found at the root instead of in the subdirectory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:19:26 +01:00
Justin Chadwell
22e1901581 bake: reorganize input creation together
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:15:44 +01:00
Justin Chadwell
e23c37fa96 Merge pull request #1771 from jedevc/unset-other-build-client-opt
debug: secondary client build should not reuse solve opts
2023-05-09 14:57:23 +01:00
Justin Chadwell
e5a0ed1149 debug: secondary client build should not reuse solve opt ref
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 14:23:41 +01:00
Justin Chadwell
c9c1303e31 Merge pull request #1777 from ktock/fixerrormsg
monitor: improve error messages
2023-05-09 13:49:07 +01:00
Justin Chadwell
ae3299d9d4 Merge pull request #1628 from cpuguy83/policy_file
Support for passing through raw buildkit policies
2023-05-09 11:50:10 +01:00
CrazyMax
a948cc14c5 Merge pull request #1769 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.1
build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
2023-05-09 11:41:08 +02:00
CrazyMax
621b07c799 Merge pull request #1727 from crazy-max/cmd-completion
cli: add shell completion
2023-05-09 10:59:32 +02:00
Justin Chadwell
7ad970f93a Merge pull request #1776 from tonistiigi/fix-conn-leak
docker: fix connection leak on features check
2023-05-09 09:52:18 +01:00
Kohei Tokunaga
437fe55104 monitor: improve error messages
Print more understandable messages on error:

- When ps fails because the monitor doesn't attach to any session, print "no
  attaching session" instead of "unknown ref".
- Avoid disconnect silently fails when the monitor doesn't attach to any
  session. Print "no attaching session" error instead.
- Fix error message of "attach"'s arguments. ("server name must be passed" ->
  "ID of session or process must be passed")

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-08 15:42:37 +09:00
Tonis Tiigi
bebd244e33 docker: fix connection leak on features check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-07 21:58:19 -07:00
Tonis Tiigi
9f2143e3df container-driver: avoid hanging on container panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-03 17:11:08 -07:00
Brian Goff
98efe7af10 Support for passing through raw buildkit policies
This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-03 23:07:44 +00:00
Justin Chadwell
c7c37c3591 Merge pull request #1760 from ktock/monitor-commands
monitor: Move commands to a separated package
2023-05-03 14:33:08 +01:00
Justin Chadwell
a43837d26c monitor: extend monitor interface from controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-03 22:08:30 +09:00
dependabot[bot]
f115abb509 build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.0 to 5.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](5b4a9f6a9e...284f54f989)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-02 18:59:56 +00:00
Justin Chadwell
43a07f3997 Merge pull request #1597 from robertlestak/k8s-sa
kubernetes driver: add serviceaccount opt
2023-05-02 12:56:07 +01:00
CrazyMax
41e1693be0 Merge pull request #1756 from crazy-max/fix-bin-image
ci: fix bin-image job
2023-04-27 12:20:16 +02:00
Justin Chadwell
9d5af461b2 Merge pull request #1755 from jedevc/move-across-controller-boundary
Move metadata and subrequest output outside of controller
2023-04-27 11:15:54 +01:00
CrazyMax
b38c9c7db4 Merge pull request #1759 from crazy-max/bump-gomod
bump go.mod to 1.20
2023-04-27 09:00:31 +02:00
Kohei Tokunaga
9f884edbbf monitor: use shlex
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:38 +09:00
Kohei Tokunaga
0a7a2b1882 monitor: Move commands to a separated package
This commit moves monitor commands to `monior/commands` package.
Commands still need access to the `monitor` object and buildx controller so this
commit enables this via `Monitor` interface stored in `monitor/types`.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:35 +09:00
Justin Chadwell
6bec8f6e00 docs: add serviceaccount driver-opt for k8s driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:44:03 +01:00
robertlestak
65037e4611 kubernetes driver: add serviceaccount opt
Signed-off-by: robertlestak <robert.lestak@umusic.com>
2023-04-26 16:40:07 +01:00
Justin Chadwell
ba92989a94 controller: print result outside of controller
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2bf996d9ad controller: don't write metadata file in controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
75ed3e296b Merge pull request #1737 from jedevc/improved-controller-progress
controller: refactor progress api
2023-04-26 10:17:00 +01:00
CrazyMax
e14e0521cf Merge pull request #1732 from milas/builder-init-panic
builder: extra init error handling
2023-04-26 09:23:01 +02:00
Tõnis Tiigi
28e6995f7c Merge pull request #1735 from tonistiigi/localstate
build: support for saving local state by build ref
2023-04-25 15:17:08 -07:00
CrazyMax
8f72fb353c bump go.mod to 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 21:45:40 +02:00
Milas Bowman
14f5d490ef builder: extra init error handling
* Return errors from creating the `NodeGroup`
* Ensure that `b.NodeGroup != nil` before reading from
  it during validation

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 11:32:54 -04:00
CrazyMax
c9095e8eab ci: fix bin-image job
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 13:39:00 +02:00
CrazyMax
0589f69206 build: null values support with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
b724a173a9 build: fix JSON formatting for --print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
e5ccb64617 fix: changed attest type from Map to List
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
08d114195f chore: format with prettier
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
caf7d2ec9b build: added attest field
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
David Karlsson
2dffed3f3a build: sort bake fields alphabetically
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
Justin Chadwell
784dc2223d lint: add empty line between console and json code blocks
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 12:05:03 +02:00
CrazyMax
c3fd1e8b79 build: fix links and formatting
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
6f0c550ee9 build: move bake under the build section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
5d551dbbc1 build: customizing builds section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
043cb3a0db build: add bake section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:02 +02:00
Justin Chadwell
16d5b38f2b debug: display build warnings after each build
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 09:58:02 +01:00
Nick Sieger
956a1be656 vendor: update to compose-go 1.13.4
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:29:20 -05:00
Justin Chadwell
afcaa8df5f Merge pull request #1752 from nicksieger/compose-go-default-filenames
bake: use default filenames from compose-go
2023-04-21 17:28:32 +01:00
Nick Sieger
12885c01ad bake: use default filenames from compose-go
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:11:37 -05:00
Justin Chadwell
2ab8749052 controller: replace logrus status messages with progress messages
logrus info messages aren't particularly in-theme with the rest of the
progress output (and are also frustratingly racy). The progress output
is a lot neater, so we refactor it into that.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 12:33:04 +01:00
Justin Chadwell
e826141af4 controller: refactor progress api
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.

This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
  they can be displayed correctly from the client side.

Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.

To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:17:43 +01:00
Justin Chadwell
0c1fd31226 build: refactor out common build command components
We had some duplicated code between the basic runBuild and
launchControllerAndRunBuild.

This patch refactors out the common logic (since it's only really like
to keep growing), and has runBuild call into either the controller or
directly start the build depending on whether BUILDX_EXPERIMENTAL is
set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:13:58 +01:00
Justin Chadwell
0e9804901b controller: refactor status message conversion to pb package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:59:16 +01:00
Justin Chadwell
2402607846 build: use gateway's solve context to allow cancelling getResultAt
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:52:03 +01:00
CrazyMax
3d49bbd23a Merge pull request #1749 from docker/dependabot/github_actions/docker/bake-action-3
build(deps): Bump docker/bake-action from 2 to 3
2023-04-19 11:07:06 +02:00
dependabot[bot]
33b1fdbf39 build(deps): Bump docker/bake-action from 2 to 3
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 2 to 3.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-18 18:10:42 +00:00
Justin Chadwell
de4cdab411 Merge pull request #1747 from jedevc/only-capture-result-for-controller
controller: only capture the result when invoked through a controller
2023-04-18 18:28:08 +01:00
Justin Chadwell
a7e471b7b3 controller: only capture the result when invoked through a controller
This ensures that the code used to capture and evaluated a result is
only executed when built through the controller. Otherwise, no build
result should be recorded.

This ensures that new code added to capture and store the build result
for debugging isn't used when BUILDX_EXPERIMENTAL is not set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 16:35:42 +01:00
Justin Chadwell
ba6e5cddb0 Merge pull request #1745 from jedevc/remote-unneccessary-exec-server-name
monitor: update check for exec args
2023-04-18 11:29:01 +01:00
Justin Chadwell
e4ff82f864 monitor: update exec failure message when no args are passed
"exec" does not take a server name, so we can rename the error check
here.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 11:16:12 +01:00
Justin Chadwell
48b733d6da Merge pull request #1740 from jedevc/resolve-paths-context-dockerfile
build: avoid resolution of dockerfile if context is remote
2023-04-18 10:54:41 +01:00
Justin Chadwell
0b432cc5f2 Merge pull request #1640 from ktock/monitor-invoke-mode-restore
monitor: add `debug-shell` and `on-error`
2023-04-18 10:41:04 +01:00
Justin Chadwell
f6cccefffc build: avoid resolution of dockerfile if context is remote
In 566f41b598, we added a check to ensure
that we avoid resolving http URLs for Dockerfile. However, we have
another circumstance we should not resolve the path in - if the context
is a remote context, the dockerfile is resolved in that context (see
build.go#LoadInputs for more information).

Therefore, we should only resolve the dockerfile to a local directory if
the context is also resolved to a local directory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-17 11:18:49 +01:00
Kohei Tokunaga
fd5d90c699 remove unused fields from ResultContext
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
06399630a2 remove ResultContextError
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
20693aa808 Fix monitor to be aware of Inspect API and invokations from errors
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
f373b91cc3 Add flags and subcommand
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
ce48b1ae84 Enable to restore build options from the server
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:05 +09:00
Kohei Tokunaga
b3340cc7ba Enable to create container from error
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:36:33 +09:00
Kohei Tokunaga
1303715aba Allow passing ResultContext from server to the client through grpcerror
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:11:32 +09:00
Justin Chadwell
b716e48926 Merge pull request #1734 from jedevc/more-ssh-context-fixes
Various context fixups
2023-04-14 09:27:36 +01:00
Justin Chadwell
7d35a3b8d8 Merge pull request #1733 from jedevc/use-dockerui-context-detection
Use dockerui context detection
2023-04-14 09:25:07 +01:00
CrazyMax
200058b505 build: support for saving local state by build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-12 21:37:03 -07:00
Justin Chadwell
566f41b598 build: avoid file resolution of dockerfile urls
Dockerfiles can be HTTP URLs as well as local paths 🤦

We just copy the same logic we use for resolving context paths, and
apply it here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
6c0547e7e6 bake: ensure remote files behind ssh expose agent
The updateContext function may make modifications to the build inputs,
creating either an SSH URL, or an SSH llb.State. In these cases, we need
to ensure that we appropriately expose the client's default agent.

Previously, we would only expose it if the remote context was a git URL,
however, we need to also ensure that if the input was used to override
the context (in the case of ReadRemoteFiles), that we expose the agent
here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
871f865ac8 bake: update ReadRemoteFiles to use buildkit api
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 11:15:25 +01:00
Justin Chadwell
62a21520ea vendor: update buildkit to master@333ee9158128
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 10:11:46 +01:00
CrazyMax
a597266a52 cli: add completion to list builder names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:45:59 +02:00
CrazyMax
14b66817fb build: add completion to list context folders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:35:50 +02:00
CrazyMax
af011d6ca3 bake: add completion to list targets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:34:40 +02:00
CrazyMax
8a02cf8717 cli: set default completion
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:32:48 +02:00
CrazyMax
672eeed9a6 Merge pull request #1731 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-04-11 11:26:32 +02:00
CrazyMax
1b816ff838 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:10:01 +02:00
CrazyMax
10365ddf22 Merge pull request #1723 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.0
build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
2023-04-11 09:39:17 +02:00
dependabot[bot]
a28cb1491d build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.4 to 5.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](38e0b6e68b...5b4a9f6a9e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-05 18:59:48 +00:00
Justin Chadwell
1e149bb84f Merge pull request #1722 from jedevc/bake-matrix-check-map-type
bake: fix map type checking to detect matrix property set to list
2023-04-05 16:03:50 +01:00
Justin Chadwell
9827abbf76 bake: fix map type checking to detect matrix property set to list
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 13:41:03 +01:00
Justin Chadwell
a3293cdaaa Merge pull request #1720 from jedevc/bake-replace-func
Add `replace` function to bake stdlib
2023-04-05 11:59:40 +01:00
Justin Chadwell
f7d8bd2055 Merge pull request #1690 from jedevc/bake-matrix
Implement matrix for bake targets
2023-04-05 11:58:59 +01:00
Justin Chadwell
5d33a3af22 bake: add replace func to stdlib
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
Justin Chadwell
87f900ce77 chore: sort bake stdlib map
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
CrazyMax
bb5c93cafc Merge pull request #1719 from dvdksn/docs/update-readme-links
docs: prune links in README toc, add external reference
2023-04-05 11:30:29 +02:00
David Karlsson
c6ce0964b9 docs: prune links in README toc, add external reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-04-05 11:00:06 +02:00
Justin Chadwell
5c21e80a83 Merge pull request #1711 from jedevc/bake-remote-file-git
bake: forward default ssh agent for remote bake file
2023-04-04 11:50:42 +01:00
CrazyMax
498cc9ba0a Merge pull request #1715 from crazy-max/ci-restrict-repo
ci: check repo origin on push tag event
2023-04-03 21:16:51 +02:00
CrazyMax
805f3a199d ci: check repo origin on push tag event
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-03 20:58:37 +02:00
Justin Chadwell
91fdb0423d Merge pull request #1713 from ktock/monitor-buildapi-options
controller: Extract nested CommonOptions on controller API
2023-04-03 14:48:06 +01:00
Kohei Tokunaga
8ba8659496 controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Justin Chadwell
16e41ba297 Merge pull request #1710 from jedevc/use-buildkit-gitutil-parsegitref
Use buildkit's gitutil package to detect remote git files
2023-04-03 11:14:08 +01:00
Justin Chadwell
387ce5be7c bake: forward default ssh agent for remote bake file
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:50:43 +01:00
Justin Chadwell
87a120e8e3 bake: use build.IsRemoteURL
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:48:31 +01:00
Justin Chadwell
589d4e4cf5 build: use buildkit's gitutil package to detect remote files
BuildKit's gitutil package behaves slightly differently than moby's
urlutil, so we should rely on BuildKit's gitutil when detecting URLs to
avoid cases of accidentally producing invalid build requests that can
confuse users.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:23:59 +01:00
CrazyMax
6535f16aec Merge pull request #1705 from docker/dependabot/go_modules/github.com/opencontainers/runc-1.1.5
build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
2023-04-03 10:53:26 +02:00
Justin Chadwell
a1520ea1b2 bake: additional validation for matrixes
This adds the following constraints to the new features:
- Explicit renaming with the `name` property is *only* permitted when
  used with the `matrix` property.
- Group does not support either `name` or `matrix` (we may choose to
  relax this constraint over time).
- All generated names must be unique.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 09:23:28 +01:00
Tõnis Tiigi
0844213897 Merge pull request #1707 from crazy-max/fix-ls-docs
docs: update ls command output sample
2023-03-30 19:33:55 -07:00
CrazyMax
989ba55d9a docs: update ls command output sample
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 17:54:23 +02:00
CrazyMax
33388d6ede Merge pull request #1552 from crazy-max/moby-buildkit-version
driver: resolve BuildKit version for docker driver
2023-03-30 17:31:47 +02:00
dependabot[bot]
bfadbecb96 build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 15:25:26 +00:00
CrazyMax
f815f4acf7 Merge pull request #1703 from jedevc/vendor-buildkit-master
Bump BuildKit to latest master
2023-03-30 17:23:38 +02:00
CrazyMax
81d7decd13 driver: resolve BuildKit version for docker driver
Buildx currently returns the Docker Engine version for the docker driver
and it can be confusing. This is because BuildKit before 0.11 version does
not support the Info API that returns the BuildKit version.

This change resolves the BuildKit version from the Docker engine one.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 15:52:29 +02:00
Justin Chadwell
d699d08399 bake: add additional tests for matrix behavior
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:39:38 +01:00
Justin Chadwell
9541457c54 vendor: update buildkit to master@8b7bcb900d3c
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:35:49 +01:00
Justin Chadwell
c6cdcb02cf build: replace removed buildkit BUILDINFO field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:24:46 +01:00
Justin Chadwell
799715ea24 Merge pull request #1701 from jedevc/fixup-resolve-paths-context-path
build: fixup resolvePaths for remote context path
2023-03-29 13:59:36 +01:00
Justin Chadwell
b5c6b3f10b build: fixup resolvePaths for remote context path
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-29 10:10:20 +01:00
Tõnis Tiigi
3f59b27cf4 Merge pull request #1698 from crazy-max/fix-issue-template
chore: fix issue template
2023-03-27 09:10:56 -07:00
CrazyMax
00b18558dd chore: fix issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-27 13:30:33 +02:00
Tõnis Tiigi
948414e1b2 Merge pull request #1693 from jedevc/remote-tls-missing-key-cert
remote: avoid tls error if both key and cert are not set
2023-03-23 10:58:12 -07:00
Justin Chadwell
56876ab825 remote: avoid tls error if both key and cert are not set
Previously, we would explicitly error if all TLS parameters were not
available. However, it is a perfectly valid use case to connect to a
buildkit server that only provides TLS in one direction to verify the
server (which is possible today with buildctl).

To support this use case, we only need to error if only one of key or
cert is set, and the other is not - if both are unspecified, the client
will not present a certificate to the server.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-23 10:43:10 +00:00
Justin Chadwell
0806870261 bake: generate implicit groups for matrixes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-22 17:07:04 +00:00
Tõnis Tiigi
fd8eaab2df Merge pull request #1675 from jedevc/source-date-epoch-fixes
SOURCE_DATE_EPOCH build arg injection fixes
2023-03-21 22:29:24 -07:00
Justin Chadwell
77252f161c bake: add matrix to target block
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-21 12:05:14 +00:00
Justin Chadwell
4437802e63 bake: allow overriding name property
Previously, the name property could not be set in the body of a bake
target and could only be set for a label. This patch allows the body to
override the values of label fields, though the default is still the
label.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 17:55:28 +00:00
Justin Chadwell
1613fde55c bake: allow interception to create derived contexts
This patch allows high level clients to define an EvalContext method
which can derive a new context given a block and the base parent
context.

This allows users of the package to intercept evaluation before it
begins, and define additional variables and functions that are bound to
a single block.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 12:18:36 +00:00
CrazyMax
624bc064d8 Merge pull request #1677 from crazy-max/bump-k8s
vendor: bump k8s to v0.25.4
2023-03-19 17:05:55 +00:00
CrazyMax
0c4a68555e Merge pull request #1682 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.4
build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
2023-03-16 09:50:44 +00:00
dependabot[bot]
476ac18d2c build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](2b011faafd...38e0b6e68b)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-15 19:00:53 +00:00
CrazyMax
8ad30d0a35 docs: update since buildinfo removal
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-15 17:00:54 +01:00
Justin Chadwell
780531425b bake: move SOURCE_DATE_EPOCH parsing to overrides
Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-15 10:06:53 +00:00
CrazyMax
92d2dc8263 k8s: corev1.Handler has been replaced by corev1.ProbeHandler
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
CrazyMax
cfa6b4f7c8 vendor: bump k8s to v0.25.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
Justin Chadwell
5d4223e4f8 build: move SOURCE_DATE_EPOCH parsing into option generation
This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-14 10:21:32 +00:00
Tõnis Tiigi
4a73abfd64 Merge pull request #1673 from crazy-max/fix-k8s-deploy-name
k8s: generate node name if not provided
2023-03-13 17:20:51 -07:00
Justin Chadwell
6f722da04d Merge pull request #1670 from jedevc/version-v-prefix
version: add "v" prefix to version for tagging convention consistency
2023-03-13 08:00:46 +00:00
CrazyMax
527d57540e k8s: generate node name if not provided
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-11 21:22:10 +01:00
Justin Chadwell
b65f49622e version: add "v" prefix to version for tagging convention consistency
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-08 16:25:00 +00:00
Tõnis Tiigi
c5ce08bf3c Merge pull request #1626 from ktock/monitor-exec
monitor: Enable to exec into the container
2023-03-07 17:46:22 +00:00
Tõnis Tiigi
71b35ae42e Merge pull request #1663 from crazy-max/stripcreds
build: strip credentials from remote url on collecting Git provenance info
2023-03-06 16:26:09 +00:00
CrazyMax
15eb6418e8 build: strip credentials from remote url on collecting Git provenance info
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 17:00:05 +01:00
Justin Chadwell
2a83723d57 Merge pull request #1581 from ktock/resolvepath
monitor: resolve paths arguments in client
2023-03-02 11:09:34 +00:00
Kohei Tokunaga
e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:30:29 +09:00
Justin Chadwell
b5ea989eee monitor: resolve oci layout path in client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-02 10:28:56 +00:00
Kohei Tokunaga
17105bfc50 monitor: resolve paths arguments in client
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:13:45 +09:00
CrazyMax
eefe27ff42 Merge pull request #1658 from crazy-max/remove-guides
docs: guides moved to docs website
2023-03-01 10:40:12 +01:00
CrazyMax
1ea71e358a docs: guides moved to docs website
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-01 10:21:02 +01:00
Tõnis Tiigi
14d8f95ec9 Merge pull request #1657 from ktock/argpath
controller: use os.Executable() for getting the exceutable path
2023-03-01 00:12:36 -08:00
Kohei Tokunaga
b0728c96d3 controller: use os.Executable() for getting the exceutable path
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-01 15:49:07 +09:00
Tõnis Tiigi
5e685c0e04 Merge pull request #1650 from crazy-max/dirty-optin
build: makes git dirty check opt-in
2023-02-28 10:49:17 -08:00
CrazyMax
f2ac30f431 Merge pull request #1651 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-02-24 15:43:32 +01:00
CrazyMax
6808c0e585 ci: update docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:13:10 +01:00
CrazyMax
9de12bb9c8 docs: do not generate ref for internal serve command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:12:15 +01:00
CrazyMax
0645acfd79 build: makes git dirty check opt-in
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 13:19:33 +01:00
CrazyMax
439d58ddbd Merge pull request #1037 from crazy-max/bug-template
chore: github issue template
2023-02-24 13:06:55 +01:00
CrazyMax
c0a9274d64 chore: github issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 12:33:24 +01:00
Justin Chadwell
f3a4cd5176 Merge pull request #1614 from jedevc/typed-controller-api
controller: strongly type the controller api
2023-02-24 09:44:45 +00:00
Justin Chadwell
c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:43:15 +00:00
Justin Chadwell
c4ad930e2a Merge pull request #1644 from crazy-max/update-frontend
Dockerfile: align frontend version
2023-02-23 11:00:13 +00:00
CrazyMax
3d0c88695e Merge pull request #1646 from docker/dependabot/go_modules/golang.org/x/net-0.7.0
build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
2023-02-23 11:36:36 +01:00
dependabot[bot]
7332140fdf build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-22 19:50:31 +00:00
CrazyMax
132fababb0 Dockerfile: align frontend version
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 17:47:48 +01:00
CrazyMax
71507c0b58 Merge pull request #1643 from crazy-max/go-1.20
update to go 1.20
2023-02-22 17:42:59 +01:00
CrazyMax
7888fdee58 fix lint issue
should drop = nil from declaration of var n; it is the zero value

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:57:29 +01:00
CrazyMax
fb61fde581 fix cannot set environment variables in parallel tests
> FAIL: TestReadTargets/ArgsOverrides/leaf (0.00s)
> panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:56:46 +01:00
CrazyMax
5258e44030 remove deprecated golangci-lint linters
Linters 'deadcode' and 'varcheck' are deprecated since v1.49.0
and had been replaced by 'unused'

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:40:14 +01:00
CrazyMax
e16c1b289b update golangci-lint to v1.51.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:38:17 +01:00
CrazyMax
376b73f078 update to go 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:37:50 +01:00
Justin Chadwell
1c6060f27d Merge pull request #1612 from cpuguy83/env_no_provenance
Add env var to disable default attestations
2023-02-22 11:36:35 +00:00
Justin Chadwell
ed4fd965ff Merge pull request #1620 from jedevc/remote-controller-fixes
Improvements for remote controller code
2023-02-21 10:52:58 +00:00
Brian Goff
bc9cb2c66a Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-02-16 17:25:40 +00:00
CrazyMax
aa05f4c207 Merge pull request #1584 from dvdksn/docs/build-platform-local
docs: added --platform=local example
2023-02-16 18:08:25 +01:00
CrazyMax
62fbef22d0 Merge pull request #1177 from crazy-max/build-display-builder
build: print instance being used
2023-02-16 18:07:30 +01:00
David Karlsson
2563685d27 docs: added --platform=local example
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-02-16 17:24:45 +01:00
CrazyMax
598f1f0a62 build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
8311b0963a vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
b1949b7388 Merge pull request #1634 from dgageot/fix-1633
build: fix git ambiguous argument
2023-02-16 14:05:32 +01:00
CrazyMax
3341bd1740 Merge pull request #1610 from thaJeztah/update_docker
vendor: github.com/docker/docker, github.com/docker/cli v23.0.0
2023-02-16 13:43:12 +01:00
Justin Chadwell
74f64f88a7 Merge pull request #1617 from jedevc/process-multiple-attestation-annotations
imagetools: process `com.docker.reference.*` annotations
2023-02-16 12:05:20 +00:00
David Gageot
d4a4aaf509 Remove git warning: buildx/1633
Signed-off-by: David Gageot <david.gageot@docker.com>
2023-02-16 07:36:24 +01:00
Justin Chadwell
1f73f4fd5d Merge pull request #1630 from jedevc/debug-help-message-typo
monitor: fix typo in help message
2023-02-16 00:06:06 +00:00
Justin Chadwell
77f83d4171 monitor: fix typo in help message
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-15 14:48:56 +00:00
Justin Chadwell
642f28f439 imagetools: process com.docker.reference.* annotations
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.

Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:24:40 +00:00
Justin Chadwell
54f4dc8f6e controller: set absolute path of server binary before execution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:21:18 +00:00
CrazyMax
89d99b1694 inspect: display labels and gc policies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:56 +01:00
CrazyMax
9753f63f57 builder: add worker gc policies and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:15 +01:00
CrazyMax
04804ff355 Merge pull request #1557 from crazy-max/gomod-1.19
go.mod: bump to Go 1.19
2023-02-11 22:47:51 +01:00
Justin Chadwell
ed9ea2476d controller: use unique files per buildx version
This ensures that we should never accidentally connect to a server with
a mismatched version, while also allowing us to run multiple buildx
servers at a time.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:23 +00:00
Justin Chadwell
d0d29168a5 controller: use grpc with contexts for improved timeouts
This patch improves timeout logic for testing and creating a buildx
server. Instead of needing to have a custom loop, we can just use the
primitives provided by go and grpc for easier handling.

Additionally, without the explicit time.Sleeps, we defer to GRPCs
exponential backoff algorithm which should provide substantially better
results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:19 +00:00
Justin Chadwell
abda257763 controller: exit cleanly on SIGTERM
This signal may be sent using an external tool such as pkill, and since
we can handle it neatly, we should.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:15 +00:00
Justin Chadwell
1b91bc2e02 controller: add more informative server exit messages
When exiting, we should ideally always print a message, and give details
as to exactly what error we received.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:26 +00:00
Justin Chadwell
56b9e785e5 build: don't kill remote controller after build
We don't know if other builds might be running, etc, so we should allow
the server to decide when to exit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:25 +00:00
Sebastiaan van Stijn
081447c9b1 vendor: github.com/docker/cli v23.0.0
full diff: https://github.com/docker/cli/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:32:57 +01:00
Sebastiaan van Stijn
260117289b vendor: github.com/docker/docker v23.0.0
full diff: https://github.com/docker/docker/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:30:17 +01:00
Sebastiaan van Stijn
73dca749ca vendor: golang.org/x/oauth2 v0.1.0
no changes in vendored code

full diff: https://github.com/golang/oauth2/compare/6fdb5e3db783...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:26:17 +01:00
Sebastiaan van Stijn
8ac380bfb3 vendor: golang.org/x/net v0.5.0
full diff: https://github.com/golang/net/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:21:05 +01:00
Sebastiaan van Stijn
aeac7e08f9 vendor: golang.org/x/term v0.4.0
no changes in vendored code

full diff: https://github.com/golang/term/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:20:00 +01:00
Sebastiaan van Stijn
7c9cdc4353 vendor: golang.org/x/text v0.6.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:19:53 +01:00
Sebastiaan van Stijn
67572785cf vendor: golang.org/x/sys v0.4.0
full diff: https://github.com/golang/sys/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:12:32 +01:00
Sebastiaan van Stijn
8a70e7634d vendor: github.com/google/certificate-transparency-go v1.1.4
full diff:

- https://github.com/google/certificate-transparency-go/compare/v1.0.21...v1.1.4
- https://github.com/kubernetes/klog/compare/v2.30.0...v2.80.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:03:26 +01:00
Sebastiaan van Stijn
6dd5589a9c vendor: github.com/containerd/containerd v1.6.16
full diff: https://github.com/containerd/containerd/compare/1709cfe273d9...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 14:55:44 +01:00
CrazyMax
78058ce5f3 Merge pull request #1592 from crazy-max/fix-build-git
build: check reachable git commits
2023-02-08 14:23:52 +01:00
CrazyMax
fd5884189c build: check reachable git commits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-08 12:46:04 +01:00
Justin Chadwell
ab7a9f008d Merge pull request #1607 from jedevc/bake-fix-error-traces
bake: avoid nesting error diagnostics
2023-02-07 13:20:07 +00:00
CrazyMax
a8eb2a7fbe gitutil: override the locale to ensure consistent output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-07 13:26:28 +01:00
Justin Chadwell
fbb4f4dec8 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-07 11:40:20 +00:00
Justin Chadwell
46fd0a61ba Merge pull request #1605 from jedevc/bake-fix-errors
bake: avoid early-exit for resolution failures
2023-02-07 10:36:05 +00:00
Justin Chadwell
6444c813dc Merge pull request #1601 from jedevc/buildx-document-attestation-booleans
build: add docs for boolean attestation flags
2023-02-06 15:15:01 +00:00
Justin Chadwell
dc8a2b0398 bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 15:05:45 +00:00
CrazyMax
d9780e27cd Merge pull request #1602 from crazy-max/rm-releases-json
ci: remove releases-json workflow
2023-02-06 13:44:57 +01:00
CrazyMax
ab44d03771 ci: remove releases-json workflow
This is now directly handled by our actions-toolkit. We keep
`.github/releases.json` file for now but will be removed on
next v0.11 release.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
crazy-max
b53cb256e5 github: update .github/releases.json
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
CrazyMax
c3075923f4 Merge pull request #1591 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.9.0
2023-02-06 13:28:24 +01:00
CrazyMax
a32881313b Merge pull request #1596 from crazy-max/builder-fix-validation
builder: fix docker context not validated
2023-02-06 12:31:04 +01:00
Justin Chadwell
07548bc898 build: add docs for boolean attestation flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 10:53:49 +00:00
CrazyMax
0e544fe835 builder: fix docker context not validated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 21:23:29 +01:00
CrazyMax
21ac4c34fb update github.com/compose-spec/compose-go to v1.9.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 03:19:59 +01:00
Justin Chadwell
d2fa4a5724 Merge pull request #1582 from jedevc/lint-fixups
Linting tidy-ups
2023-02-01 16:08:07 +00:00
Justin Chadwell
4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:31:01 +00:00
Justin Chadwell
5da09f0c23 lint: enable gosec
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:50 +00:00
Justin Chadwell
48357ee0c6 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:43 +00:00
Justin Chadwell
6506166f02 Merge pull request #1579 from jedevc/refactor-controllers
Refactor controllers, moving them into a separate package
2023-02-01 09:57:32 +00:00
Justin Chadwell
5f130b25ad hack: move syntax directive to top of generated-files
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
cb94298a02 Merge pull request #1296 from ktock/monitor-list
monitor: Enable to run build and invoke in background
2023-01-31 14:14:57 +00:00
CrazyMax
046084c0b8 Merge pull request #1574 from crazy-max/ci-fix-workflow
ci: fix pr branch name in releases-json workflow
2023-01-31 15:00:01 +01:00
CrazyMax
18760253b9 ci: fix pr branch name in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 13:51:30 +01:00
CrazyMax
ded6376ece Merge pull request #1573 from crazy-max/fix-releases-json
ci: add workflow_dispatch in releases-json workflow
2023-01-31 13:48:33 +01:00
CrazyMax
a4d60a451d ci: add workflow_dispatch in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 11:38:46 +01:00
Justin Chadwell
0f4030de5d Merge pull request #1572 from crazy-max/ci-update-ver
ci: update buildx and buildkit to latest
2023-01-31 10:22:58 +00:00
CrazyMax
f1a5a3ec50 ci: update buildx and buildkit to latest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 22:13:27 +01:00
CrazyMax
87beaefbb8 Merge pull request #1571 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-01-30 21:18:54 +01:00
CrazyMax
451847183d ci: fix docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 21:01:27 +01:00
CrazyMax
7625a3a4b0 Merge pull request #1568 from crazy-max/silent-git-no-upstream
build: silently fail if git remote not found
2023-01-30 20:32:12 +01:00
CrazyMax
6db696748b build: better message output for git provenance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 20:24:09 +01:00
Tõnis Tiigi
14f9ae679d Merge pull request #1566 from tonistiigi/fix-result-descriptor
build: fix multi-node merge to read descriptor from result
2023-01-30 09:04:40 -08:00
CrazyMax
4789d2219c build: silently fail if git remote not found
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 14:33:16 +01:00
CrazyMax
eacecf657c Merge pull request #1561 from crazy-max/fix-preferred-platform
build: fix preferred platform not taken into account
2023-01-30 12:22:08 +01:00
CrazyMax
1de0be240f Merge pull request #1563 from crazy-max/releases-json
ci: generate releases.json on release event
2023-01-30 11:51:08 +01:00
CrazyMax
ea4bec2bad github: update .github/releases.json
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
CrazyMax
36d95bd3b9 ci: generate releases.json on release event
We are currently using the GitHub API in our setup-buildx-action
to check for latest and tagged releases to make sure they exist
before download. But this requires using a token to avoid
rate-limit. It's fine for public runners but GHES runners don't
have the `github.token` populated automatically. They need to
create a PAT.

This PR will solve this issue by generating and pushing a
`releases.json` file in this repo when we publish a GitHub Release
that will then be fetched through `raw.githubusercontent.com`
endpoint on `setup-buildx-action` repo. This endpoint is better
served for our purpose with 5000 requests per hour compared to the
GitHub API endpoint that is limited to 60 requests per hour (unauth)
and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an
action in a workflow run can't trigger a new workflow run anyway.
See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
Tonis Tiigi
c33b310b48 build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-29 22:52:41 -08:00
Justin Chadwell
8af76c68a8 Merge pull request #1564 from AkihiroSuda/fix-1562
bake: SOURCE_DATE_EPOCH: fix `panic: assignment to entry in nil map`
2023-01-29 22:07:20 +00:00
Akihiro Suda
1f56f51740 bake: SOURCE_DATE_EPOCH: fix panic: assignment to entry in nil map
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-29 08:43:06 +09:00
CrazyMax
49b3c0dba5 build: fix preferred platform not taken account
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-27 20:02:13 +01:00
Tõnis Tiigi
a718d07f64 Merge pull request #1559 from tonistiigi/update-buildkit-v0.11.2
vendor: update buildkit to v0.11.2
2023-01-26 10:57:21 -08:00
Tonis Tiigi
f6da7ee135 vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-26 10:33:20 -08:00
CrazyMax
7eb266de69 go.mod: bump to Go 1.19
Go 1.20 will be there soon, I think it's time to move our go.mod to
latest stable. We can then remove the compat in our vendor.Dockerfile

Downstream projects like compose or github.com/linuxkit/linuxkit should
not be affected.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-26 16:08:09 +01:00
CrazyMax
9f821dabeb Merge pull request #1548 from crazy-max/git-ls-remote
build: set remote origin url
2023-01-26 11:03:18 +01:00
Kohei Tokunaga
a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-26 17:17:07 +09:00
Tõnis Tiigi
b1b4e64c97 Merge pull request #1504 from jedevc/fix-1497
Refactor BuildWithResultHandler to simplify concurrency usage
2023-01-25 10:52:17 -08:00
CrazyMax
c1058c17aa build: set remote origin url
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-25 18:15:05 +01:00
Tõnis Tiigi
059c347fc2 Merge pull request #1549 from jedevc/silence-intoto-warnings
imagetools: silence intoto warnings
2023-01-24 17:03:49 -08:00
Justin Chadwell
7145e021f9 imagetools: silence intoto warnings
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 16:49:28 +00:00
CrazyMax
9723f4f76c Merge pull request #1547 from crazy-max/fix-rm
rm: do not check for context builders when removing inactive
2023-01-24 13:39:36 +01:00
Justin Chadwell
db72d0cc05 Merge pull request #1543 from jedevc/bump-ci-buildkit
ci: update buildkit release version in build pipeline
2023-01-24 12:17:34 +00:00
Justin Chadwell
00b7d5b858 Merge pull request #1505 from jedevc/inspect-lazy-attestations
Lazily load attestation data in `imagetools inspect`
2023-01-24 12:10:11 +00:00
CrazyMax
6cd0c11ab1 rm: do not check for context builders when removing inactive
This change has been introduced in e7b5ee7518
but we should not check context builders when removing inactive
ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-24 13:08:18 +01:00
Justin Chadwell
c1ab55a3f2 ci: update buildkit release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 11:47:58 +00:00
Justin Chadwell
c756e3ba96 Merge pull request #1534 from jedevc/readme-minimum-docker-version
docs: add note about minimum docker version
2023-01-24 11:02:21 +00:00
Justin Chadwell
566f37b65b docs: add note about minimum docker version
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 10:25:58 +00:00
CrazyMax
6d1ff27410 Merge pull request #1538 from tonistiigi/caps-req-internal
build: mark capabilities request as internal
2023-01-24 11:12:37 +01:00
Tonis Tiigi
be55b41427 build: mark capabilities request as internal
So it doesn't show up in the History API.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-24 00:48:35 -08:00
Tõnis Tiigi
a4f01b41a4 Merge pull request #1529 from thaJeztah/update_buildkit
vendor: github.com/moby/buildkit v0.11.1
2023-01-18 12:56:26 -08:00
Sebastiaan van Stijn
01e1c28dd9 vendor: github.com/moby/buildkit v0.11.1
full diff: https://github.com/moby/buildkit/compare/v0.11.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 20:56:54 +01:00
Tõnis Tiigi
51e41b16db Merge pull request #1511 from jedevc/error-on-attestations-docker
build: error when using docker exporter and attestations
2023-01-17 09:57:48 -08:00
Justin Chadwell
9e9cdc2e6d Merge pull request #1525 from jedevc/ci-bump-buildx
ci: update buildx release version in build pipeline
2023-01-17 16:39:39 +00:00
Justin Chadwell
bc1d590ca7 ci: update buildx release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 15:21:52 +00:00
Justin Chadwell
900d9c294d Merge pull request #1523 from jedevc/docs-reference-attest
docs: add reference for new attest family of flags
2023-01-17 13:47:10 +00:00
CrazyMax
65aac16139 Merge pull request #1512 from nicks/patch-1
docs: small tweak to opentelemetry
2023-01-17 13:30:33 +01:00
Justin Chadwell
4903f462f6 docs: add reference for new attest family of flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 11:45:38 +00:00
Justin Chadwell
44b5a19c13 Merge pull request #1516 from dvdksn/docs/fix-broken-bake-link-in-cli-reference
docs: fix broken link in buildx_bake CLI reference
2023-01-17 11:11:54 +00:00
David Karlsson
ba8fa6c403 docs: fix broken link in buildx_bake CLI reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-01-13 15:33:31 +01:00
CrazyMax
5b3083e9e1 Merge pull request #1515 from crazy-max/fix-docs-release
ci: fix typo in docs-release workflow
2023-01-13 15:16:16 +01:00
CrazyMax
523a16aa35 ci: fix typo in docs-release workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-13 15:10:03 +01:00
Justin Chadwell
43a748fd15 build: error when using docker exporter and attestations
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:41:53 +00:00
Justin Chadwell
15a80b56b5 build: avoid compatability error when attestations disabled
We should avoid erroring with attestations support compatability errors
when a user has specified --provenance=false.

A user may wish to enable --provenance=false that works across buildkit
versions, but currently it will fail on old versions - this patch fixes
this, to silently ignore the provenance flag for this check if it's set
to disabled.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:36:55 +00:00
Nick Santos
b14bfb9fa2 docs: small tweak to opentelemetry
The buildx builder persists across Desktop restarts. So when we start up Jaeger, we should recommend that people configure it to persist across restarts as well.

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-01-11 12:09:05 -05:00
Justin Chadwell
56950ece69 inspect: lazily load attestation data
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.

With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 13:00:42 +00:00
Justin Chadwell
1d2ac78443 inspect: move attestation loading to struct methods
This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 12:40:42 +00:00
Justin Chadwell
8b7aa1a168 build: create error group per opt
Using the syncronization primitive, we can avoid needing to create a
separate wait group.

This allows us to sidestep the issue where the wait group could be
completed, but the build invocation functions had not terminated - if
one of the functions was to terminate with an error, then it was
possible to encounter a race condition, where the result handling code
would begin executing, despite an error.

The refactor to use a separate error group which more elegantly handles
the concept of function returns and errors, ensures that we can't
encounter this issue.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 11:02:27 +00:00
Justin Chadwell
1180d919f5 build: reorder error group funcs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
347417ee12 build: use copy for BuildWithResultHandler loop vars
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
fb27e3f919 Merge pull request #1502 from developer-guy/fix/readme
fix the directory of the buildx binary
2023-01-10 10:28:04 +00:00
Batuhan Apaydın
edb16f8aab fix the directory of the buildx binary
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2023-01-10 11:26:11 +03:00
Tõnis Tiigi
5c56e947fe Merge pull request #1500 from tonistiigi/update-buildkit-v0.11.0
github: update CI to buildkit v0.11
2023-01-09 15:59:25 -08:00
Tonis Tiigi
571871b084 github: update CI to buildkit v0.11
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-09 15:50:56 -08:00
Tõnis Tiigi
8340c40647 Merge pull request #1498 from jedevc/attestation-printing
Improved attestation inspect
2023-01-09 11:22:20 -08:00
Justin Chadwell
9818055b0e docs: update with new inspect output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:07:10 +00:00
Justin Chadwell
484823c97d inspect: change additional spdxs to not have duplicates
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:03:23 +00:00
Justin Chadwell
3ce17b01dc inspect: provide access to multiple spdx documents
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:42:26 +00:00
Justin Chadwell
e68c566c1c inspect: parse sbom and provenance into json structs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:43 +00:00
Justin Chadwell
19d16aa941 inspect: break after first matching attestation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:06 +00:00
CrazyMax
6852713121 Merge pull request #1494 from thaJeztah/docs_update_docs_tools
go.mod: update cli-docs-tool v0.5.1
2023-01-09 13:14:01 +01:00
Sebastiaan van Stijn
c97500b117 go.mod: update cli-docs-tool v0.5.1 and re-generate docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:03:01 +01:00
CrazyMax
85040a9067 Merge pull request #1493 from thaJeztah/docs_fix_anchors
docs: update anchor links
2023-01-09 12:19:58 +01:00
Sebastiaan van Stijn
b8285c17e6 docs: update anchor links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-07 15:12:08 +01:00
Tõnis Tiigi
332dfb4b92 Merge pull request #1444 from crazy-max/inspect-attest
imagetools inspect: handle provenance and sboms
2023-01-06 16:29:33 -08:00
Tõnis Tiigi
cb279bb14b Merge pull request #1491 from jedevc/ociindex-refactor
vendor: update buildkit to v0.11.0-rc4
2023-01-06 16:29:18 -08:00
Justin Chadwell
60c9cf74ce vendor: update buildkit to v0.11.0-rc4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-06 15:51:17 -08:00
Tõnis Tiigi
ff6754eb04 Merge pull request #1456 from jedevc/oci-layout-reference-parsing
build: refactor reference parsing for oci image layouts
2023-01-05 23:50:25 -08:00
CrazyMax
e6b9aba997 imagetools inspect: handle provenance and sbom
use stub structs for SLSA/SBOM while waiting for
go-imageinspect library to be public.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 17:34:30 +01:00
CrazyMax
0302894bfb Merge pull request #1463 from crazy-max/e2e-k3s
e2e: use native k3s installation script
2023-01-05 16:50:45 +01:00
CrazyMax
e46394c3be e2e: use native k3s installation script
debianmaster/actions-k3s action gives some warnings in our e2e
workflow. This commit brings https://github.com/debianmaster/actions-k3s/blob/master/index.js
directly in the workflow through actions/github-script with
some changes to properly wait for nodes to be up.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 15:37:19 +01:00
Justin Chadwell
1885e41789 docs: update oci layout with tag resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-05 13:47:42 +00:00
CrazyMax
2fb9db994b imagetools inspect: missing annotations key
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-04 15:52:16 +01:00
Tõnis Tiigi
287aaf1696 Merge pull request #1482 from AkihiroSuda/auto-propagate-source-date-epoch
Propagate SOURCE_DATE_EPOCH from the client env
2022-12-27 17:32:08 -08:00
Akihiro Suda
0e6f5a155e Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-24 01:32:15 +09:00
Justin Chadwell
70682b043e build: refactor reference parsing for image layouts
We allow any valid image reference format for the oci-layout, not just
limiting to name@digest, we additionally allow images of the form
name:tag@digest now.

The name of the reference is used to find the local directory to lookup
the store in, while the tag and digest are attached to a random identity
to generate the dummy reference sent to the oci-layout context.

This separation of the target to replace and the value to replace it
with ensures that any tag or digest set in the client is properly sent
across to the server. The tag is used when a digest was not specified,
and it is resolved in the context of the local directory before being
sent, using the same helpers as we use for the local cache expoter.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-14 16:53:10 +00:00
4414 changed files with 442584 additions and 133852 deletions

View File

@@ -116,6 +116,60 @@ commit automatically with `git commit -s`.
### Run the unit- and integration-tests
Running tests:
```bash
make test
```
This runs all unit and integration tests, in a containerized environment.
Locally, every package can be tested separately with standard Go tools, but
integration tests are skipped if local user doesn't have enough permissions or
worker binaries are not installed.
```bash
# run unit tests only
make test-unit
# run integration tests only
make test-integration
# test a specific package
TESTPKGS=./bake make test
# run all integration tests with a specific worker
TESTFLAGS="--run=//worker=remote -v" make test-integration
# run a specific integration test
TESTFLAGS="--run /TestBuild/worker=remote/ -v" make test-integration
# run a selection of integration tests using a regexp
TESTFLAGS="--run /TestBuild.*/worker=remote/ -v" make test-integration
```
> **Note**
>
> Set `TEST_KEEP_CACHE=1` for the test framework to keep external dependant
> images in a docker volume if you are repeatedly calling `make test`. This
> helps to avoid rate limiting on the remote registry side.
> **Note**
>
> Set `TEST_DOCKERD=1` for the test framework to enable the docker workers,
> specifically the `docker` and `docker-container` drivers.
>
> The docker tests cannot be run in parallel, so require passing `--parallel=1`
> in `TESTFLAGS`.
> **Note**
>
> If you are working behind a proxy, you can set some of or all
> `HTTP_PROXY=http://ip:port`, `HTTPS_PROXY=http://ip:port`, `NO_PROXY=http://ip:port`
> for the test framework to specify the proxy build args.
### Run the helper commands
To enter a demo container environment and experiment, you may run:
```

124
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,124 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Bug Report
description: Report a bug
labels:
- status/triage
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to report a bug!
If this is a security issue please report it to the [Docker Security team](mailto:security@docker.com).
- type: checkboxes
attributes:
label: Contributing guidelines
description: |
Please read the contributing guidelines before proceeding.
options:
- label: I've read the [contributing guidelines](https://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree
required: true
- type: checkboxes
attributes:
label: I've found a bug and checked that ...
description: |
Make sure that your request fulfills all of the following requirements.
If one requirement cannot be satisfied, explain in detail why.
options:
- label: ... the documentation does not mention anything about my problem
- label: ... there are no open or closed issues that are related to my problem
- type: textarea
attributes:
label: Description
description: |
Please provide a brief description of the bug in 1-2 sentences.
validations:
required: true
- type: textarea
attributes:
label: Expected behaviour
description: |
Please describe precisely what you'd expect to happen.
validations:
required: true
- type: textarea
attributes:
label: Actual behaviour
description: |
Please describe precisely what is actually happening.
validations:
required: true
- type: input
attributes:
label: Buildx version
description: |
Output of `docker buildx version` command.
Example: `github.com/docker/buildx v0.8.1 5fac64c2c49dae1320f2b51f1a899ca451935554`
validations:
required: true
- type: textarea
attributes:
label: Docker info
description: |
Output of `docker info` command.
render: text
- type: textarea
attributes:
label: Builders list
description: |
Output of `docker buildx ls` command.
render: text
validations:
required: true
- type: textarea
attributes:
label: Configuration
description: >
Please provide a minimal Dockerfile, bake definition (if applicable) and
invoked commands to help reproducing your issue.
placeholder: |
```dockerfile
FROM alpine
echo hello
```
```hcl
group "default" {
targets = ["app"]
}
target "app" {
dockerfile = "Dockerfile"
target = "build"
}
```
```console
$ docker buildx build .
$ docker buildx bake
```
validations:
required: true
- type: textarea
attributes:
label: Build logs
description: |
Please provide logs output (and/or BuildKit logs if applicable).
render: text
validations:
required: false
- type: textarea
attributes:
label: Additional info
description: |
Please provide any additional information that could be useful.

12
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
blank_issues_enabled: true
contact_links:
- name: Questions and Discussions
url: https://github.com/docker/buildx/discussions/new
about: Use Github Discussions to ask questions and/or open discussion topics.
- name: Command line reference
url: https://docs.docker.com/engine/reference/commandline/buildx/
about: Read the command line reference.
- name: Documentation
url: https://docs.docker.com/build/
about: Read the documentation.

15
.github/ISSUE_TEMPLATE/feature.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Feature request
description: Missing functionality? Come tell us about it!
labels:
- kind/enhancement
- status/triage
body:
- type: textarea
id: description
attributes:
label: Description
description: What is the feature you want to see?
validations:
required: true

12
.github/SECURITY.md vendored Normal file
View File

@@ -0,0 +1,12 @@
# Reporting security issues
The project maintainers take security seriously. If you discover a security
issue, please bring it to their attention right away!
**Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Security reports are greatly appreciated, and we will publicly thank you for it.
We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
ruling it out in the future.

View File

@@ -5,6 +5,11 @@ updates:
directory: "/"
schedule:
interval: "daily"
ignore:
# ignore this dependency
# it seems a bug with dependabot as pining to commit sha should not
# trigger a new version: https://github.com/docker/buildx/pull/2222#issuecomment-1919092153
- dependency-name: "docker/docs"
labels:
- "dependencies"
- "bot"

735
.github/releases.json vendored Normal file
View File

@@ -0,0 +1,735 @@
{
"latest": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.2": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.1": {
"id": 90346950,
"tag_name": "v0.10.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/checksums.txt"
]
},
"v0.10.0": {
"id": 88388110,
"tag_name": "v0.10.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/checksums.txt"
]
},
"v0.10.0-rc3": {
"id": 88191592,
"tag_name": "v0.10.0-rc3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/checksums.txt"
]
},
"v0.10.0-rc2": {
"id": 86248476,
"tag_name": "v0.10.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/checksums.txt"
]
},
"v0.10.0-rc1": {
"id": 85963900,
"tag_name": "v0.10.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/checksums.txt"
]
},
"v0.9.1": {
"id": 74760068,
"tag_name": "v0.9.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/checksums.txt"
]
},
"v0.9.0": {
"id": 74546589,
"tag_name": "v0.9.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/checksums.txt"
]
},
"v0.9.0-rc2": {
"id": 74052235,
"tag_name": "v0.9.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/checksums.txt"
]
},
"v0.9.0-rc1": {
"id": 73389692,
"tag_name": "v0.9.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/checksums.txt"
]
},
"v0.8.2": {
"id": 63479740,
"tag_name": "v0.8.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/checksums.txt"
]
},
"v0.8.1": {
"id": 62289050,
"tag_name": "v0.8.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/checksums.txt"
]
},
"v0.8.0": {
"id": 61423774,
"tag_name": "v0.8.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/checksums.txt"
]
},
"v0.8.0-rc1": {
"id": 60513568,
"tag_name": "v0.8.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/checksums.txt"
]
},
"v0.7.1": {
"id": 54098347,
"tag_name": "v0.7.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/checksums.txt"
]
},
"v0.7.0": {
"id": 53109422,
"tag_name": "v0.7.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/checksums.txt"
]
},
"v0.7.0-rc1": {
"id": 52726324,
"tag_name": "v0.7.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/checksums.txt"
]
},
"v0.6.3": {
"id": 48691641,
"tag_name": "v0.6.3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-arm64.exe"
]
},
"v0.6.2": {
"id": 48207405,
"tag_name": "v0.6.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-arm64.exe"
]
},
"v0.6.1": {
"id": 47064772,
"tag_name": "v0.6.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-arm64.exe"
]
},
"v0.6.0": {
"id": 46343260,
"tag_name": "v0.6.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-arm64.exe"
]
},
"v0.6.0-rc1": {
"id": 46230351,
"tag_name": "v0.6.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-arm64.exe"
]
},
"v0.5.1": {
"id": 35276550,
"tag_name": "v0.5.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.windows-amd64.exe"
]
},
"v0.5.0": {
"id": 35268960,
"tag_name": "v0.5.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.windows-amd64.exe"
]
},
"v0.5.0-rc1": {
"id": 35015334,
"tag_name": "v0.5.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.windows-amd64.exe"
]
},
"v0.4.2": {
"id": 30007794,
"tag_name": "v0.4.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.windows-amd64.exe"
]
},
"v0.4.1": {
"id": 26067509,
"tag_name": "v0.4.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.windows-amd64.exe"
]
},
"v0.4.0": {
"id": 26028174,
"tag_name": "v0.4.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.windows-amd64.exe"
]
},
"v0.3.1": {
"id": 20316235,
"tag_name": "v0.3.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.windows-amd64.exe"
]
},
"v0.3.0": {
"id": 19029664,
"tag_name": "v0.3.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.windows-amd64.exe"
]
},
"v0.2.2": {
"id": 17671545,
"tag_name": "v0.2.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.windows-amd64.exe"
]
},
"v0.2.1": {
"id": 17582885,
"tag_name": "v0.2.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.windows-amd64.exe"
]
},
"v0.2.0": {
"id": 16965310,
"tag_name": "v0.2.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.windows-amd64.exe"
]
}
}

View File

@@ -13,55 +13,205 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
BUILDX_VERSION: "v0.10.0-rc1"
BUILDKIT_IMAGE: "moby/buildkit:v0.11.0-rc3"
BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:latest"
REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test"
TESTFLAGS: "-v --parallel=6 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose"
GO_VERSION: "1.21"
GOTESTSUM_VERSION: "v1.9.0" # same as one in Dockerfile
jobs:
test:
prepare-test-integration:
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Test
uses: docker/bake-action@v2
name: Build
uses: docker/bake-action@v4
with:
targets: test
targets: integration-test-base
set: |
*.cache-from=type=gha,scope=test
*.cache-to=type=gha,scope=test
-
name: Upload coverage
uses: codecov/codecov-action@v3
with:
directory: ${{ env.DESTDIR }}/coverage
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.cache-to=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
prepare:
test-integration:
runs-on: ubuntu-22.04
needs:
- prepare-test-integration
env:
TESTFLAGS_DOCKER: "-v --parallel=1 --timeout=30m"
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildx-tests"
strategy:
fail-fast: false
matrix:
worker:
- docker
- docker\+containerd # same as docker, but with containerd snapshotter
- docker-container
- remote
pkg:
- ./tests
steps:
-
name: Prepare
run: |
echo "TESTREPORTS_NAME=${{ github.job }}-$(echo "${{ matrix.pkg }}-${{ matrix.worker }}" | tr -dc '[:alnum:]-\n\r' | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
-
name: Set up QEMU
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build test image
uses: docker/bake-action@v4
with:
targets: integration-test
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |
./hack/test
env:
TEST_REPORT_SUFFIX: "-${{ env.TESTREPORTS_NAME }}"
TEST_DOCKERD: "${{ startsWith(matrix.worker, 'docker') && '1' || '0' }}"
TESTFLAGS: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker\\+containerd') && env.TESTFLAGS_DOCKER || env.TESTFLAGS }} --run=//worker=${{ matrix.worker }}$"
TESTPKGS: "${{ matrix.pkg }}"
-
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v4
with:
directory: ./bin/testreports
flags: integration
token: ${{ secrets.CODECOV_TOKEN }}
-
name: Generate annotations
if: always()
uses: crazy-max/.github/.github/actions/gotest-annotations@fa6141aedf23596fb8bdcceab9cce8dadaa31bd9
with:
directory: ./bin/testreports
-
name: Upload test reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ env.TESTREPORTS_NAME }}
path: ./bin/testreports
test-unit:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os:
- ubuntu-22.04
- macos-12
- windows-2022
env:
SKIP_INTEGRATION_TESTS: 1
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: "${{ env.GO_VERSION }}"
-
name: Prepare
run: |
testreportsName=${{ github.job }}--${{ matrix.os }}
testreportsBaseDir=./bin/testreports
testreportsDir=$testreportsBaseDir/$testreportsName
echo "TESTREPORTS_NAME=$testreportsName" >> $GITHUB_ENV
echo "TESTREPORTS_BASEDIR=$testreportsBaseDir" >> $GITHUB_ENV
echo "TESTREPORTS_DIR=$testreportsDir" >> $GITHUB_ENV
mkdir -p $testreportsDir
shell: bash
-
name: Install gotestsum
run: |
go install gotest.tools/gotestsum@${{ env.GOTESTSUM_VERSION }}
-
name: Test
env:
TMPDIR: ${{ runner.temp }}
run: |
gotestsum \
--jsonfile="${{ env.TESTREPORTS_DIR }}/go-test-report.json" \
--junitfile="${{ env.TESTREPORTS_DIR }}/junit-report.xml" \
--packages="./..." \
-- \
"-mod=vendor" \
"-coverprofile" "${{ env.TESTREPORTS_DIR }}/coverage.txt" \
"-covermode" "atomic" ${{ env.TESTFLAGS }}
shell: bash
-
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v4
with:
directory: ${{ env.TESTREPORTS_DIR }}
env_vars: RUNNER_OS
flags: unit
token: ${{ secrets.CODECOV_TOKEN }}
-
name: Generate annotations
if: always()
uses: crazy-max/.github/.github/actions/gotest-annotations@fa6141aedf23596fb8bdcceab9cce8dadaa31bd9
with:
directory: ${{ env.TESTREPORTS_DIR }}
-
name: Upload test reports
if: always()
uses: actions/upload-artifact@v4
with:
name: test-reports-${{ env.TESTREPORTS_NAME }}
path: ${{ env.TESTREPORTS_BASEDIR }}
prepare-binaries:
runs-on: ubuntu-22.04
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Create matrix
id: platforms
@@ -75,11 +225,11 @@ jobs:
binaries:
runs-on: ubuntu-22.04
needs:
- prepare
- prepare-binaries
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.matrix) }}
platform: ${{ fromJson(needs.prepare-binaries.outputs.matrix) }}
steps:
-
name: Prepare
@@ -88,13 +238,13 @@ jobs:
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
@@ -109,25 +259,28 @@ jobs:
CACHE_TO: type=gha,scope=binaries-${{ env.PLATFORM_PAIR }},mode=max
-
name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: buildx
name: buildx-${{ env.PLATFORM_PAIR }}
path: ${{ env.DESTDIR }}/*
if-no-files-found: error
bin-image:
runs-on: ubuntu-22.04
if: ${{ github.event_name != 'pull_request' }}
needs:
- test-integration
- test-unit
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
@@ -135,7 +288,7 @@ jobs:
-
name: Docker meta
id: meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
with:
images: |
${{ env.REPO_SLUG }}
@@ -147,39 +300,41 @@ jobs:
-
name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
username: ${{ vars.DOCKERPUBLICBOT_USERNAME }}
password: ${{ secrets.DOCKERPUBLICBOT_WRITE_PAT }}
-
name: Build and push image
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max
*.attest=type=sbom
*.attest=type=provenance,mode=max,builder-id=https://github.com/${{ env.GITHUB_REPOSITORY }}/actions/runs/${{ env.GITHUB_RUN_ID }}
release:
runs-on: ubuntu-22.04
needs:
- test-integration
- test-unit
- binaries
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Download binaries
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: buildx
path: ${{ env.DESTDIR }}
pattern: buildx-*
merge-multiple: true
-
name: Create checksums
run: ./hack/hash-files
@@ -207,13 +362,13 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=moby/buildkit:master
@@ -221,6 +376,6 @@ jobs:
-
# Just run a bake target to check eveything runs fine
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: binaries

42
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
name: codeql
on:
push:
branches:
- 'master'
- 'v[0-9]*'
pull_request:
permissions:
actions: read
contents: read
security-events: write
env:
GO_VERSION: "1.21"
jobs:
codeql:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v4
-
name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
-
name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: go
-
name: Autobuild
uses: github/codeql-action/autobuild@v3
-
name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:go"

View File

@@ -3,16 +3,16 @@ name: docs-release
on:
release:
types:
- releases
- released
jobs:
open-pr:
runs-on: ubuntu-22.04
if: "!github.event.release.prerelease"
if: ${{ github.event.release.prerelease != true && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout docs repo
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
repository: docker/docs
@@ -23,10 +23,10 @@ jobs:
rm -rf ./_data/buildx/*
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
-
name: Build docs
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ github.event.release.name }}
targets: update-docs
@@ -44,7 +44,7 @@ jobs:
git add -A .
-
name: Create PR on docs repo
uses: peter-evans/create-pull-request@2b011faafdcbc9ceb11414d64d0573f37c774b04
uses: peter-evans/create-pull-request@a4f52f8033a6168103c2538976c07b467e8163bc
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io

View File

@@ -26,15 +26,15 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-
name: Build reference YAML docs
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: update-docs
set: |
@@ -45,17 +45,18 @@ jobs:
DOCS_FORMATS: yaml
-
name: Upload reference YAML docs
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: docs-yaml
path: /tmp/buildx-docs/out/reference
retention-days: 1
validate:
uses: docker/docs/.github/workflows/validate-upstream.yml@main
uses: docker/docs/.github/workflows/validate-upstream.yml@6b73b05acb21edf7995cc5b3c6672d8e314cee7a # pin for artifact v4 support: https://github.com/docker/docs/pull/19220
needs:
- docs-yaml
with:
repo: https://github.com/${{ github.repository }}
module-name: docker/buildx
data-files-id: docs-yaml
data-files-folder: buildx
create-placeholder-stubs: true

View File

@@ -11,30 +11,29 @@ on:
- 'master'
- 'v[0-9]*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
DESTDIR: "./bin"
K3S_VERSION: "v1.21.2-k3s1"
jobs:
build:
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v4
with:
targets: binaries
set: |
@@ -47,7 +46,7 @@ jobs:
mv ${{ env.DESTDIR }}/build/buildx ${{ env.DESTDIR }}/build/docker-buildx
-
name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: binary
path: ${{ env.DESTDIR }}/build
@@ -97,14 +96,14 @@ jobs:
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
uses: docker/setup-qemu-action@v3
if: matrix.driver == 'docker' || matrix.driver == 'docker-container'
-
name: Install buildx
uses: actions/download-artifact@v3
uses: actions/download-artifact@v4
with:
name: binary
path: /home/runner/.docker/cli-plugins
@@ -133,20 +132,67 @@ jobs:
-
name: Install k3s
if: matrix.driver == 'kubernetes'
uses: debianmaster/actions-k3s@b9cf3f599fd118699a3c8a0d18a2f2bda6cf4ce4
id: k3s
uses: actions/github-script@v7
with:
version: v1.21.2-k3s1
script: |
const fs = require('fs');
let wait = function(milliseconds) {
return new Promise((resolve, reject) => {
if (typeof(milliseconds) !== 'number') {
throw new Error('milleseconds not a number');
}
setTimeout(() => resolve("done!"), milliseconds)
});
}
try {
const kubeconfig="/tmp/buildkit-k3s/kubeconfig.yaml";
core.info(`storing kubeconfig in ${kubeconfig}`);
await exec.exec('docker', ["run", "-d",
"--privileged",
"--name=buildkit-k3s",
"-e", "K3S_KUBECONFIG_OUTPUT="+kubeconfig,
"-e", "K3S_KUBECONFIG_MODE=666",
"-v", "/tmp/buildkit-k3s:/tmp/buildkit-k3s",
"-p", "6443:6443",
"-p", "80:80",
"-p", "443:443",
"-p", "8080:8080",
"rancher/k3s:${{ env.K3S_VERSION }}", "server"
]);
await wait(10000);
core.exportVariable('KUBECONFIG', kubeconfig);
let nodeName;
for (let count = 1; count <= 5; count++) {
try {
const nodeNameOutput = await exec.getExecOutput("kubectl get nodes --no-headers -oname");
nodeName = nodeNameOutput.stdout
} catch (error) {
core.info(`Unable to resolve node name (${error.message}). Attempt ${count} of 5.`)
} finally {
if (nodeName) {
break;
}
await wait(5000);
}
}
if (!nodeName) {
throw new Error(`Unable to resolve node name after 5 attempts.`);
}
await exec.exec(`kubectl wait --for=condition=Ready ${nodeName}`);
} catch (error) {
core.setFailed(error.message);
}
-
name: Config k3s
name: Print KUBECONFIG
if: matrix.driver == 'kubernetes'
run: |
(set -x ; cat ${{ steps.k3s.outputs.kubeconfig }})
-
name: Check k3s nodes
if: matrix.driver == 'kubernetes'
run: |
kubectl get nodes
yq ${{ env.KUBECONFIG }}
-
name: Launch remote buildkitd
if: matrix.driver == 'remote'

View File

@@ -13,13 +13,14 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
jobs:
validate:
runs-on: ubuntu-22.04
env:
GOLANGCI_LINT_MULTIPLATFORM: 1
strategy:
fail-fast: false
matrix:
@@ -27,13 +28,14 @@ jobs:
- lint
- validate-vendor
- validate-docs
- validate-generated-files
steps:
-
name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
uses: docker/setup-buildx-action@v3
with:
version: latest
-

View File

@@ -1,5 +1,5 @@
run:
timeout: 10m
timeout: 30m
skip-files:
- ".*\\.pb\\.go$"
@@ -11,30 +11,59 @@ linters:
enable:
- gofmt
- govet
- deadcode
- depguard
- goimports
- ineffassign
- misspell
- unused
- varcheck
- revive
- staticcheck
- typecheck
- nolintlint
- gosec
- forbidigo
disable-all: true
linters-settings:
depguard:
list-type: blacklist
include-go-root: true
packages:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- io/ioutil
rules:
main:
deny:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- pkg: "io/ioutil"
desc: The io/ioutil package has been deprecated.
forbidigo:
forbid:
- '^fmt\.Errorf(# use errors\.Errorf instead)?$'
gosec:
excludes:
- G204 # Audit use of command execution
- G402 # TLS MinVersion too low
config:
G306: "0644"
issues:
exclude-rules:
- linters:
- revive
text: "stutters"
- linters:
- revive
text: "empty-block"
- linters:
- revive
text: "superfluous-else"
- linters:
- revive
text: "unused-parameter"
- linters:
- revive
text: "redefines-builtin-id"
- linters:
- revive
text: "if-return"
# show all
max-issues-per-linter: 0
max-same-issues: 0

View File

@@ -1,10 +1,12 @@
# syntax=docker/dockerfile-upstream:master
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.19
ARG XX_VERSION=1.1.2
ARG DOCKERD_VERSION=20.10.14
ARG GO_VERSION=1.21
ARG XX_VERSION=1.4.0
FROM docker:$DOCKERD_VERSION AS dockerd-release
ARG DOCKER_VERSION=25.0.2
ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.12.5
# xx is a helper for cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
@@ -18,6 +20,33 @@ ENV GOFLAGS=-mod=vendor
ENV CGO_ENABLED=0
WORKDIR /src
FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM gobase AS docker
ARG TARGETPLATFORM
ARG DOCKER_VERSION
WORKDIR /opt/docker
RUN DOCKER_ARCH=$(case ${TARGETPLATFORM:-linux/amd64} in \
"linux/amd64") echo "x86_64" ;; \
"linux/arm/v6") echo "armel" ;; \
"linux/arm/v7") echo "armhf" ;; \
"linux/arm64") echo "aarch64" ;; \
"linux/ppc64le") echo "ppc64le" ;; \
"linux/s390x") echo "s390x" ;; \
*) echo "" ;; esac) \
&& echo "DOCKER_ARCH=$DOCKER_ARCH" \
&& wget -qO- "https://download.docker.com/linux/static/stable/${DOCKER_ARCH}/docker-${DOCKER_VERSION}.tgz" | tar xvz --strip 1
RUN ./dockerd --version && ./containerd --version && ./ctr --version && ./runc --version
FROM gobase AS gotestsum
ARG GOTESTSUM_VERSION
ENV GOFLAGS=
RUN --mount=target=/root/.cache,type=cache \
GOBIN=/out/ go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" && \
/out/gotestsum --version
FROM gobase AS buildx-version
RUN --mount=type=bind,target=. <<EOT
set -e
@@ -39,6 +68,7 @@ RUN --mount=type=bind,target=. \
EOT
FROM gobase AS test
ENV SKIP_INTEGRATION_TESTS=1
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
@@ -61,6 +91,28 @@ FROM binaries-$TARGETOS AS binaries
# enable scanning for this stage
ARG BUILDKIT_SBOM_SCAN_STAGE=true
FROM gobase AS integration-test-base
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN apk add --no-cache \
btrfs-progs \
e2fsprogs \
e2fsprogs-extra \
ip6tables \
iptables \
openssl \
shadow-uidmap \
xfsprogs \
xz
COPY --link --from=gotestsum /out/gotestsum /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=docker /opt/docker/* /usr/bin/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/
FROM integration-test-base AS integration-test
COPY . .
# Release
FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work
@@ -76,7 +128,7 @@ FROM scratch AS release
COPY --from=releaser /out/ /
# Shell
FROM docker:$DOCKERD_VERSION AS dockerd-release
FROM docker:$DOCKER_VERSION AS dockerd-release
FROM alpine AS shell
RUN apk add --no-cache iptables tmux git vim less openssh
RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx

View File

@@ -37,7 +37,7 @@ release:
./hack/release
.PHONY: validate-all
validate-all: lint test validate-vendor validate-docs
validate-all: lint test validate-vendor validate-docs validate-generated-files
.PHONY: lint
lint:
@@ -45,7 +45,15 @@ lint:
.PHONY: test
test:
$(BUILDX_CMD) bake test
./hack/test
.PHONY: test-unit
test-unit:
TESTPKGS=./... SKIP_INTEGRATION_TESTS=1 ./hack/test
.PHONY: test
test-integration:
TESTPKGS=./tests ./hack/test
.PHONY: validate-vendor
validate-vendor:
@@ -59,6 +67,10 @@ validate-docs:
validate-authors:
$(BUILDX_CMD) bake validate-authors
.PHONY: validate-generated-files
validate-generated-files:
$(BUILDX_CMD) bake validate-generated-files
.PHONY: test-driver
test-driver:
./hack/test-driver
@@ -78,3 +90,7 @@ authors:
.PHONY: mod-outdated
mod-outdated:
$(BUILDX_CMD) bake mod-outdated
.PHONY: generated-files
generated-files:
$(BUILDX_CMD) bake update-generated-files

View File

@@ -32,19 +32,6 @@ Key features:
- [Building with buildx](#building-with-buildx)
- [Working with builder instances](#working-with-builder-instances)
- [Building multi-platform images](#building-multi-platform-images)
- [Manuals](docs/manuals)
- [High-level build options with Bake](docs/manuals/bake/index.md)
- [Drivers](docs/manuals/drivers/index.md)
- [Exporters](docs/manuals/exporters/index.md)
- [Cache backends](docs/manuals/cache/backends/index.md)
- [Guides](docs/guides)
- [CI/CD](docs/guides/cicd.md)
- [CNI networking](docs/guides/cni-networking.md)
- [Using a custom network](docs/guides/custom-network.md)
- [Using a custom registry configuration](docs/guides/custom-registry-config.md)
- [OpenTelemetry support](docs/guides/opentelemetry.md)
- [Registry mirror](docs/guides/registry-mirror.md)
- [Resource limiting](docs/guides/resource-limiting.md)
- [Reference](docs/reference/buildx.md)
- [`buildx bake`](docs/reference/buildx_bake.md)
- [`buildx build`](docs/reference/buildx_build.md)
@@ -54,21 +41,26 @@ Key features:
- [`buildx imagetools create`](docs/reference/buildx_imagetools_create.md)
- [`buildx imagetools inspect`](docs/reference/buildx_imagetools_inspect.md)
- [`buildx inspect`](docs/reference/buildx_inspect.md)
- [`buildx install`](docs/reference/buildx_install.md)
- [`buildx ls`](docs/reference/buildx_ls.md)
- [`buildx prune`](docs/reference/buildx_prune.md)
- [`buildx rm`](docs/reference/buildx_rm.md)
- [`buildx stop`](docs/reference/buildx_stop.md)
- [`buildx uninstall`](docs/reference/buildx_uninstall.md)
- [`buildx use`](docs/reference/buildx_use.md)
- [`buildx version`](docs/reference/buildx_version.md)
- [Contributing](#contributing)
For more information on how to use Buildx, see
[Docker Build docs](https://docs.docker.com/build/).
# Installing
Using `buildx` as a docker CLI plugin requires using Docker 19.03 or newer.
A limited set of functionality works with older versions of Docker when
invoking the binary directly.
Using `buildx` with Docker requires Docker engine 19.03 or newer.
> **Warning**
>
> Using an incompatible version of Docker may result in unexpected behavior,
> and will likely cause issues, especially when using Buildx builders with more
> recent versions of BuildKit.
## Windows and macOS
@@ -77,8 +69,9 @@ for Windows and macOS.
## Linux packages
Docker Linux packages also include Docker Buildx when installed using the
[DEB or RPM packages](https://docs.docker.com/engine/install/).
Docker Engine package repositories contain Docker Buildx packages when installed according to the
[Docker Engine install documentation](https://docs.docker.com/engine/install/). Install the
`docker-buildx-plugin` package to install the Buildx plugin.
## Manual download
@@ -147,14 +140,14 @@ To remove this alias, run [`docker buildx uninstall`](docs/reference/buildx_unin
# Buildx 0.6+
$ docker buildx bake "https://github.com/docker/buildx.git"
$ mkdir -p ~/.docker/cli-plugins
$ mv ./bin/buildx ~/.docker/cli-plugins/docker-buildx
$ mv ./bin/build/buildx ~/.docker/cli-plugins/docker-buildx
# Docker 19.03+
$ DOCKER_BUILDKIT=1 docker build --platform=local -o . "https://github.com/docker/buildx.git"
$ mkdir -p ~/.docker/cli-plugins
$ mv buildx ~/.docker/cli-plugins/docker-buildx
# Local
# Local
$ git clone https://github.com/docker/buildx.git && cd buildx
$ make install
```
@@ -246,7 +239,7 @@ When you invoke a build, you can set the `--platform` flag to specify the target
platform for the build output, (for example, `linux/amd64`, `linux/arm64`, or
`darwin/amd64`).
When the current builder instance is backed by the `docker-container` or
When the current builder instance is backed by the `docker-container` or
`kubernetes` driver, you can specify multiple platforms together. In this case,
it builds a manifest list which contains images for all specified architectures.
When you use this image in [`docker run`](https://docs.docker.com/engine/reference/commandline/run/)
@@ -316,7 +309,7 @@ cross-compilation helpers for more advanced use-cases.
## High-level build options
See [`docs/manuals/bake/index.md`](docs/manuals/bake/index.md) for more details.
See [High-level builds with Bake](https://docs.docker.com/build/bake/) for more details.
# Contributing

View File

@@ -3,7 +3,6 @@ package bake
import (
"context"
"encoding/csv"
"fmt"
"io"
"os"
"path"
@@ -12,23 +11,27 @@ import (
"sort"
"strconv"
"strings"
"time"
composecli "github.com/compose-spec/compose-go/v2/cli"
"github.com/docker/buildx/bake/hclparser"
"github.com/docker/buildx/build"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/config"
"github.com/docker/docker/builder/remotecontext/urlutil"
dockeropts "github.com/docker/cli/opts"
hcl "github.com/hashicorp/hcl/v2"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
)
var (
httpPrefix = regexp.MustCompile(`^https?://`)
gitURLPathWithFragmentSuffix = regexp.MustCompile(`\.git(?:#.+)?$`)
validTargetNameChars = `[a-zA-Z0-9_-]+`
targetNamePattern = regexp.MustCompile(`^` + validTargetNameChars + `$`)
)
@@ -44,17 +47,18 @@ type Override struct {
}
func defaultFilenames() []string {
return []string{
"docker-compose.yml", // support app
"docker-compose.yaml", // support app
names := []string{}
names = append(names, composecli.DefaultFileNames...)
names = append(names, []string{
"docker-bake.json",
"docker-bake.override.json",
"docker-bake.hcl",
"docker-bake.override.hcl",
}
}...)
return names
}
func ReadLocalFiles(names []string) ([]File, error) {
func ReadLocalFiles(names []string, stdin io.Reader, l progress.SubLogger) ([]File, error) {
isDefault := false
if len(names) == 0 {
isDefault = true
@@ -62,20 +66,26 @@ func ReadLocalFiles(names []string) ([]File, error) {
}
out := make([]File, 0, len(names))
setStatus := func(st *client.VertexStatus) {
if l != nil {
l.SetStatus(st)
}
}
for _, n := range names {
var dt []byte
var err error
if n == "-" {
dt, err = io.ReadAll(os.Stdin)
dt, err = readWithProgress(stdin, setStatus)
if err != nil {
return nil, err
}
} else {
dt, err = os.ReadFile(n)
dt, err = readFileWithProgress(n, isDefault, setStatus)
if dt == nil && err == nil {
continue
}
if err != nil {
if isDefault && errors.Is(err, os.ErrNotExist) {
continue
}
return nil, err
}
}
@@ -84,6 +94,103 @@ func ReadLocalFiles(names []string) ([]File, error) {
return out, nil
}
func readFileWithProgress(fname string, isDefault bool, setStatus func(st *client.VertexStatus)) (dt []byte, err error) {
st := &client.VertexStatus{
ID: "reading " + fname,
}
defer func() {
now := time.Now()
st.Completed = &now
if dt != nil || err != nil {
setStatus(st)
}
}()
now := time.Now()
st.Started = &now
f, err := os.Open(fname)
if err != nil {
if isDefault && errors.Is(err, os.ErrNotExist) {
return nil, nil
}
return nil, err
}
defer f.Close()
setStatus(st)
info, err := f.Stat()
if err != nil {
return nil, err
}
st.Total = info.Size()
setStatus(st)
buf := make([]byte, 1024)
for {
n, err := f.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
dt = append(dt, buf[:n]...)
st.Current += int64(n)
setStatus(st)
}
return dt, nil
}
func readWithProgress(r io.Reader, setStatus func(st *client.VertexStatus)) (dt []byte, err error) {
st := &client.VertexStatus{
ID: "reading from stdin",
}
defer func() {
now := time.Now()
st.Completed = &now
setStatus(st)
}()
now := time.Now()
st.Started = &now
setStatus(st)
buf := make([]byte, 1024)
for {
n, err := r.Read(buf)
if err == io.EOF {
break
}
if err != nil {
return nil, err
}
dt = append(dt, buf[:n]...)
st.Current += int64(n)
setStatus(st)
}
return dt, nil
}
func ListTargets(files []File) ([]string, error) {
c, err := ParseFiles(files, nil)
if err != nil {
return nil, err
}
var targets []string
for _, g := range c.Groups {
targets = append(targets, g.Name)
}
for _, t := range c.Targets {
targets = append(targets, t.Name)
}
return dedupSlice(targets), nil
}
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, map[string]*Group, error) {
c, err := ParseFiles(files, defaults)
if err != nil {
@@ -215,7 +322,7 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
hclFiles = append(hclFiles, hf)
} else if composeErr != nil {
return nil, fmt.Errorf("failed to parse %s: parsing yaml: %v, parsing hcl: %w", f.Name, composeErr, err)
return nil, errors.Wrapf(err, "failed to parse %s: parsing yaml: %v, parsing hcl", f.Name, composeErr)
} else {
return nil, err
}
@@ -232,13 +339,28 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
if len(hclFiles) > 0 {
if err := hclparser.Parse(hcl.MergeFiles(hclFiles), hclparser.Opt{
renamed, err := hclparser.Parse(hclparser.MergeFiles(hclFiles), hclparser.Opt{
LookupVar: os.LookupEnv,
Vars: defaults,
ValidateLabel: validateTargetName,
}, &c); err.HasErrors() {
}, &c)
if err.HasErrors() {
return nil, err
}
for _, renamed := range renamed {
for oldName, newNames := range renamed {
newNames = dedupSlice(newNames)
if len(newNames) == 1 && oldName == newNames[0] {
continue
}
c.Groups = append(c.Groups, &Group{
Name: oldName,
Targets: newNames,
})
}
}
c = dedupeConfig(c)
}
return &c, nil
@@ -556,9 +678,10 @@ type Target struct {
Name string `json:"-" hcl:"name,label" cty:"name"`
// Inherits is the only field that cannot be overridden with --set
Attest []string `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Inherits []string `json:"inherits,omitempty" hcl:"inherits,optional" cty:"inherits"`
Annotations []string `json:"annotations,omitempty" hcl:"annotations,optional" cty:"annotations"`
Attest []string `json:"attest,omitempty" hcl:"attest,optional" cty:"attest"`
Context *string `json:"context,omitempty" hcl:"context,optional" cty:"context"`
Contexts map[string]string `json:"contexts,omitempty" hcl:"contexts,optional" cty:"contexts"`
Dockerfile *string `json:"dockerfile,omitempty" hcl:"dockerfile,optional" cty:"dockerfile"`
@@ -577,14 +700,22 @@ type Target struct {
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"-" hcl:"-" cty:"-"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/manuals/bake/file-definition.md.
ShmSize *string `json:"shm-size,omitempty" hcl:"shm-size,optional"`
Ulimits []string `json:"ulimits,omitempty" hcl:"ulimits,optional"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/bake-reference.md.
// linked is a private field to mark a target used as a linked one
linked bool
}
var _ hclparser.WithEvalContexts = &Target{}
var _ hclparser.WithGetName = &Target{}
var _ hclparser.WithEvalContexts = &Group{}
var _ hclparser.WithGetName = &Group{}
func (t *Target) normalize() {
t.Attest = removeDupes(t.Attest)
t.Annotations = removeDupes(t.Annotations)
t.Attest = removeAttestDupes(t.Attest)
t.Tags = removeDupes(t.Tags)
t.Secrets = removeDupes(t.Secrets)
t.SSH = removeDupes(t.SSH)
@@ -593,6 +724,7 @@ func (t *Target) normalize() {
t.CacheTo = removeDupes(t.CacheTo)
t.Outputs = removeDupes(t.Outputs)
t.NoCacheFilter = removeDupes(t.NoCacheFilter)
t.Ulimits = removeDupes(t.Ulimits)
for k, v := range t.Contexts {
if v == "" {
@@ -644,8 +776,12 @@ func (t *Target) Merge(t2 *Target) {
if t2.Target != nil {
t.Target = t2.Target
}
if t2.Annotations != nil { // merge
t.Annotations = append(t.Annotations, t2.Annotations...)
}
if t2.Attest != nil { // merge
t.Attest = append(t.Attest, t2.Attest...)
t.Attest = removeAttestDupes(t.Attest)
}
if t2.Secrets != nil { // merge
t.Secrets = append(t.Secrets, t2.Secrets...)
@@ -677,6 +813,12 @@ func (t *Target) Merge(t2 *Target) {
if t2.NoCacheFilter != nil { // merge
t.NoCacheFilter = append(t.NoCacheFilter, t2.NoCacheFilter...)
}
if t2.ShmSize != nil { // no merge
t.ShmSize = t2.ShmSize
}
if t2.Ulimits != nil { // merge
t.Ulimits = append(t.Ulimits, t2.Ulimits...)
}
t.Inherits = append(t.Inherits, t2.Inherits...)
}
@@ -729,6 +871,8 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
t.Platforms = o.ArrValue
case "output":
t.Outputs = o.ArrValue
case "annotations":
t.Annotations = append(t.Annotations, o.ArrValue...)
case "attest":
t.Attest = append(t.Attest, o.ArrValue...)
case "no-cache":
@@ -739,6 +883,10 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
t.NoCache = &noCache
case "no-cache-filter":
t.NoCacheFilter = o.ArrValue
case "shm-size":
t.ShmSize = &value
case "ulimits":
t.Ulimits = o.ArrValue
case "pull":
pull, err := strconv.ParseBool(value)
if err != nil {
@@ -766,13 +914,128 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
return nil
}
func (g *Group) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
if _, ok := content.Attributes["matrix"]; ok {
return nil, errors.Errorf("matrix is not supported for groups")
}
return []*hcl.EvalContext{ectx}, nil
}
func (t *Target) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
attr, ok := content.Attributes["matrix"]
if !ok {
return []*hcl.EvalContext{ectx}, nil
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return nil, diags
}
value, err := attr.Expr.Value(ectx)
if err != nil {
return nil, err
}
if !value.Type().IsMapType() && !value.Type().IsObjectType() {
return nil, errors.Errorf("matrix must be a map")
}
matrix := value.AsValueMap()
ectxs := []*hcl.EvalContext{ectx}
for k, expr := range matrix {
if !expr.CanIterateElements() {
return nil, errors.Errorf("matrix values must be a list")
}
ectxs2 := []*hcl.EvalContext{}
for _, v := range expr.AsValueSlice() {
for _, e := range ectxs {
e2 := ectx.NewChild()
e2.Variables = make(map[string]cty.Value)
if e != ectx {
for k, v := range e.Variables {
e2.Variables[k] = v
}
}
e2.Variables[k] = v
ectxs2 = append(ectxs2, e2)
}
}
ectxs = ectxs2
}
return ectxs, nil
}
func (g *Group) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
if _, ok := content.Attributes["name"]; ok {
return "", errors.Errorf("name is not supported for groups")
}
if _, ok := content.Attributes["matrix"]; ok {
return "", errors.Errorf("matrix is not supported for groups")
}
return block.Labels[0], nil
}
func (t *Target) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
attr, ok := content.Attributes["name"]
if !ok {
return block.Labels[0], nil
}
if _, ok := content.Attributes["matrix"]; !ok {
return "", errors.Errorf("name requires matrix")
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return "", diags
}
value, diags := attr.Expr.Value(ectx)
if diags != nil {
return "", diags
}
value, err := convert.Convert(value, cty.String)
if err != nil {
return "", err
}
return value.AsString(), nil
}
func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) {
// make sure local credentials are loaded multiple times for different targets
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
authProvider := authprovider.NewDockerAuthProvider(dockerConfig, nil)
m2 := make(map[string]build.Options, len(m))
for k, v := range m {
bo, err := toBuildOpt(v, inp)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, authProvider)
m2[k] = *bo
}
return m2, nil
@@ -790,7 +1053,7 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(v.Path, "cwd://") || strings.HasPrefix(v.Path, "target:") || strings.HasPrefix(v.Path, "docker-image:") {
continue
}
if IsRemoteURL(v.Path) {
if build.IsRemoteURL(v.Path) {
continue
}
st := llb.Scratch().File(llb.Copy(*inp.State, v.Path, "/"), llb.WithCustomNamef("set context %s to %s", k, v.Path))
@@ -804,10 +1067,15 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(t.ContextPath, "cwd://") {
return
}
if IsRemoteURL(t.ContextPath) {
if build.IsRemoteURL(t.ContextPath) {
return
}
st := llb.Scratch().File(llb.Copy(*inp.State, t.ContextPath, "/"), llb.WithCustomNamef("set context to %s", t.ContextPath))
st := llb.Scratch().File(
llb.Copy(*inp.State, t.ContextPath, "/", &llb.CopyInfo{
CopyDirContentsOnly: true,
}),
llb.WithCustomNamef("set context to %s", t.ContextPath),
)
t.ContextState = &st
}
@@ -840,7 +1108,7 @@ func validateContextsEntitlements(t build.Inputs, inp *Input) error {
}
func checkPath(p string) error {
if IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
return nil
}
p, err := filepath.EvalSymlinks(p)
@@ -850,6 +1118,10 @@ func checkPath(p string) error {
}
return err
}
p, err = filepath.Abs(p)
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return err
@@ -858,7 +1130,8 @@ func checkPath(p string) error {
if err != nil {
return err
}
if strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
parts := strings.Split(rel, string(os.PathSeparator))
if parts[0] == ".." {
return errors.Errorf("path %s is outside of the working directory, please set BAKE_ALLOW_REMOTE_FS_ACCESS=1", p)
}
return nil
@@ -876,18 +1149,76 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if t.Context != nil {
contextPath = *t.Context
}
if !strings.HasPrefix(contextPath, "cwd://") && !IsRemoteURL(contextPath) {
if !strings.HasPrefix(contextPath, "cwd://") && !build.IsRemoteURL(contextPath) {
contextPath = path.Clean(contextPath)
}
dockerfilePath := "Dockerfile"
if t.Dockerfile != nil {
dockerfilePath = *t.Dockerfile
}
if !isRemoteResource(contextPath) && !path.IsAbs(dockerfilePath) {
dockerfilePath = path.Join(contextPath, dockerfilePath)
if !strings.HasPrefix(dockerfilePath, "cwd://") {
dockerfilePath = path.Clean(dockerfilePath)
}
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if strings.HasPrefix(bi.DockerfilePath, "cwd://") {
// If Dockerfile is local for a remote invocation, we first check if
// it's not outside the working directory and then resolve it to an
// absolute path.
bi.DockerfilePath = path.Clean(strings.TrimPrefix(bi.DockerfilePath, "cwd://"))
if err := checkPath(bi.DockerfilePath); err != nil {
return nil, err
}
var err error
bi.DockerfilePath, err = filepath.Abs(bi.DockerfilePath)
if err != nil {
return nil, err
}
} else if !build.IsRemoteURL(bi.DockerfilePath) && strings.HasPrefix(bi.ContextPath, "cwd://") && (inp != nil && build.IsRemoteURL(inp.URL)) {
// We don't currently support reading a remote Dockerfile with a local
// context when doing a remote invocation because we automatically
// derive the dockerfile from the context atm:
//
// target "default" {
// context = BAKE_CMD_CONTEXT
// dockerfile = "Dockerfile.app"
// }
//
// > docker buildx bake https://github.com/foo/bar.git
// failed to solve: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount3004544897/Dockerfile.app: no such file or directory
//
// To avoid mistakenly reading a local Dockerfile, we check if the
// Dockerfile exists locally and if so, we error out.
if _, err := os.Stat(filepath.Join(path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://")), bi.DockerfilePath)); err == nil {
return nil, errors.Errorf("reading a dockerfile for a remote build invocation is currently not supported")
}
}
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
if !build.IsRemoteURL(bi.ContextPath) && bi.ContextState == nil && !path.IsAbs(bi.DockerfilePath) {
bi.DockerfilePath = path.Join(bi.ContextPath, bi.DockerfilePath)
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
args := map[string]string{}
for k, v := range t.Args {
if v == nil {
@@ -916,31 +1247,13 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if t.NetworkMode != nil {
networkMode = *t.NetworkMode
}
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
shmSize := new(dockeropts.MemBytes)
if t.ShmSize != nil {
if err := shmSize.Set(*t.ShmSize); err != nil {
return nil, errors.Errorf("invalid value %s for membytes key shm-size", *t.ShmSize)
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
bo := &build.Options{
Inputs: bi,
Tags: t.Tags,
@@ -951,6 +1264,7 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
Pull: pull,
NetworkMode: networkMode,
Linked: t.linked,
ShmSize: *shmSize,
}
platforms, err := platformutil.Parse(t.Platforms)
@@ -959,24 +1273,28 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
}
bo.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
bo.Session = append(bo.Session, authprovider.NewDockerAuthProvider(dockerConfig))
secrets, err := buildflags.ParseSecretSpecs(t.Secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, secrets)
sshSpecs := t.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(contextPath) {
sshSpecs = []string{"default"}
}
ssh, err := buildflags.ParseSSHSpecs(sshSpecs)
secretAttachment, err := controllerapi.CreateSecrets(secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, ssh)
bo.Session = append(bo.Session, secretAttachment)
sshSpecs, err := buildflags.ParseSSHSpecs(t.SSH)
if err != nil {
return nil, err
}
if len(sshSpecs) == 0 && (buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL))) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
sshAttachment, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, sshAttachment)
if t.Target != nil {
bo.Target = *t.Target
@@ -986,25 +1304,51 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if err != nil {
return nil, err
}
bo.CacheFrom = cacheImports
bo.CacheFrom = controllerapi.CreateCaches(cacheImports)
cacheExports, err := buildflags.ParseCacheEntry(t.CacheTo)
if err != nil {
return nil, err
}
bo.CacheTo = cacheExports
bo.CacheTo = controllerapi.CreateCaches(cacheExports)
outputs, err := buildflags.ParseOutputs(t.Outputs)
outputs, err := buildflags.ParseExports(t.Outputs)
if err != nil {
return nil, err
}
bo.Exports = outputs
bo.Exports, err = controllerapi.CreateExports(outputs)
if err != nil {
return nil, err
}
annotations, err := buildflags.ParseAnnotations(t.Annotations)
if err != nil {
return nil, err
}
for _, e := range bo.Exports {
for k, v := range annotations {
e.Attrs[k.String()] = v
}
}
attests, err := buildflags.ParseAttests(t.Attest)
if err != nil {
return nil, err
}
bo.Attests = attests
bo.Attests = controllerapi.CreateAttestations(attests)
bo.SourcePolicy, err = build.ReadSourcePolicy()
if err != nil {
return nil, err
}
ulimits := dockeropts.NewUlimitOpt(nil)
for _, field := range t.Ulimits {
if err := ulimits.Set(field); err != nil {
return nil, err
}
}
bo.Ulimits = ulimits
return bo, nil
}
@@ -1030,8 +1374,24 @@ func removeDupes(s []string) []string {
return s[:i]
}
func isRemoteResource(str string) bool {
return urlutil.IsGitURL(str) || urlutil.IsURL(str)
func removeAttestDupes(s []string) []string {
res := []string{}
m := map[string]int{}
for _, v := range s {
att, err := buildflags.ParseAttest(v)
if err != nil {
res = append(res, v)
continue
}
if i, ok := m[att.Type]; ok {
res[i] = v
} else {
m[att.Type] = len(res)
res = append(res, v)
}
}
return res
}
func parseOutputType(str string) string {

View File

@@ -2,16 +2,17 @@ package bake
import (
"context"
"os"
"path/filepath"
"sort"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestReadTargets(t *testing.T) {
t.Parallel()
fp := File{
Name: "config.hcl",
Data: []byte(`
@@ -21,6 +22,8 @@ target "webDEP" {
VAR_BOTH = "webDEP"
}
no-cache = true
shm-size = "128m"
ulimits = ["nofile=1024:1024"]
}
target "webapp" {
@@ -35,6 +38,7 @@ target "webapp" {
ctx := context.TODO()
t.Run("NoOverrides", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
@@ -43,6 +47,8 @@ target "webapp" {
require.Equal(t, ".", *m["webapp"].Context)
require.Equal(t, ptrstr("webDEP"), m["webapp"].Args["VAR_INHERITED"])
require.Equal(t, true, *m["webapp"].NoCache)
require.Equal(t, "128m", *m["webapp"].ShmSize)
require.Equal(t, []string{"nofile=1024:1024"}, m["webapp"].Ulimits)
require.Nil(t, m["webapp"].Pull)
require.Equal(t, 1, len(g))
@@ -50,6 +56,7 @@ target "webapp" {
})
t.Run("InvalidTargetOverrides", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"nosuchtarget.context=foo"}, nil)
require.NotNil(t, err)
require.Equal(t, err.Error(), "could not find any target matching 'nosuchtarget'")
@@ -91,6 +98,7 @@ target "webapp" {
// building leaf but overriding parent fields
t.Run("parent", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{
"webDEP.args.VAR_INHERITED=override",
"webDEP.args.VAR_BOTH=override",
@@ -105,6 +113,7 @@ target "webapp" {
})
t.Run("ContextOverride", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.context"}, nil)
require.NotNil(t, err)
@@ -116,6 +125,7 @@ target "webapp" {
})
t.Run("NoCacheOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.no-cache=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].NoCache)
@@ -123,7 +133,14 @@ target "webapp" {
require.Equal(t, []string{"webapp"}, g["default"].Targets)
})
t.Run("ShmSizeOverride", func(t *testing.T) {
m, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.shm-size=256m"}, nil)
require.NoError(t, err)
require.Equal(t, "256m", *m["webapp"].ShmSize)
})
t.Run("PullOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.pull=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].Pull)
@@ -132,6 +149,7 @@ target "webapp" {
})
t.Run("PatternOverride", func(t *testing.T) {
t.Parallel()
// same check for two cases
multiTargetCheck := func(t *testing.T, m map[string]*Target, g map[string]*Group, err error) {
require.NoError(t, err)
@@ -367,7 +385,7 @@ services:
require.Equal(t, []string{"web_app"}, g["default"].Targets)
}
func TestHCLCwdPrefix(t *testing.T) {
func TestHCLContextCwdPrefix(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
@@ -380,18 +398,49 @@ func TestHCLCwdPrefix(t *testing.T) {
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
_, ok := m["app"]
require.True(t, ok)
_, err = TargetsToBuildOpt(m, &Input{})
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, "test", *m["app"].Dockerfile)
require.Equal(t, "foo", *m["app"].Context)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
assert.Equal(t, "test", *m["app"].Dockerfile)
assert.Equal(t, "foo", *m["app"].Context)
assert.Equal(t, "foo/test", bo["app"].Inputs.DockerfilePath)
assert.Equal(t, "foo", bo["app"].Inputs.ContextPath)
}
func TestHCLDockerfileCwdPrefix(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "app" {
context = "."
dockerfile = "cwd://Dockerfile.app"
}`),
}
ctx := context.TODO()
cwd, err := os.Getwd()
require.NoError(t, err)
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
assert.Equal(t, "cwd://Dockerfile.app", *m["app"].Dockerfile)
assert.Equal(t, ".", *m["app"].Context)
assert.Equal(t, filepath.Join(cwd, "Dockerfile.app"), bo["app"].Inputs.DockerfilePath)
assert.Equal(t, ".", bo["app"].Inputs.ContextPath)
}
func TestOverrideMerge(t *testing.T) {
@@ -1358,3 +1407,117 @@ func TestJSONNullVars(t *testing.T) {
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, m["default"].Args)
}
func TestReadLocalFilesDefault(t *testing.T) {
tests := []struct {
filenames []string
expected []string
}{
{
filenames: []string{"abc.yml", "docker-compose.yml"},
expected: []string{"docker-compose.yml"},
},
{
filenames: []string{"test.foo", "compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"test.txt", "compsoe.yaml"}, // intentional misspell
expected: []string{},
},
}
pwd, err := os.Getwd()
require.NoError(t, err)
for _, tt := range tests {
t.Run(strings.Join(tt.filenames, "-"), func(t *testing.T) {
dir := t.TempDir()
t.Cleanup(func() { _ = os.Chdir(pwd) })
require.NoError(t, os.Chdir(dir))
for _, tf := range tt.filenames {
require.NoError(t, os.WriteFile(tf, []byte(tf), 0644))
}
files, err := ReadLocalFiles(nil, nil, nil)
require.NoError(t, err)
if len(files) == 0 {
require.Equal(t, len(tt.expected), len(files))
} else {
found := false
for _, exp := range tt.expected {
for _, f := range files {
if f.Name == exp {
found = true
break
}
}
require.True(t, found, exp)
}
}
})
}
}
func TestAttestDuplicates(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "default" {
attest = ["type=sbom", "type=sbom,generator=custom", "type=sbom,foo=bar", "type=provenance,mode=max"]
}`),
}
ctx := context.TODO()
m, _, err := ReadTargets(ctx, []File{fp}, []string{"default"}, nil, nil)
require.Equal(t, []string{"type=sbom,foo=bar", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": ptrstr("type=sbom,foo=bar"),
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
m, _, err = ReadTargets(ctx, []File{fp}, []string{"default"}, []string{"*.attest=type=sbom,disabled=true"}, nil)
require.Equal(t, []string{"type=sbom,disabled=true", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err = TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": nil,
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
}
func TestAnnotations(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "app" {
output = ["type=image,name=foo"]
annotations = ["manifest[linux/amd64]:foo=bar"]
}`),
}
ctx := context.TODO()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"app"}, nil, nil)
require.NoError(t, err)
bo, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, 1, len(g))
require.Equal(t, []string{"app"}, g["default"].Targets)
require.Equal(t, 1, len(m))
require.Contains(t, m, "app")
require.Equal(t, "type=image,name=foo", m["app"].Outputs[0])
require.Equal(t, "manifest[linux/amd64]:foo=bar", m["app"].Annotations[0])
require.Len(t, bo["app"].Exports, 1)
require.Equal(t, "bar", bo["app"].Exports[0].Attrs["annotation-manifest[linux/amd64].foo"])
}

View File

@@ -1,13 +1,17 @@
package bake
import (
"context"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/compose-spec/compose-go/dotenv"
"github.com/compose-spec/compose-go/loader"
compose "github.com/compose-spec/compose-go/types"
"github.com/compose-spec/compose-go/v2/dotenv"
"github.com/compose-spec/compose-go/v2/loader"
composetypes "github.com/compose-spec/compose-go/v2/types"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/pkg/errors"
"gopkg.in/yaml.v3"
)
@@ -17,9 +21,9 @@ func ParseComposeFiles(fs []File) (*Config, error) {
if err != nil {
return nil, err
}
var cfgs []compose.ConfigFile
var cfgs []composetypes.ConfigFile
for _, f := range fs {
cfgs = append(cfgs, compose.ConfigFile{
cfgs = append(cfgs, composetypes.ConfigFile{
Filename: f.Name,
Content: f.Data,
})
@@ -27,12 +31,17 @@ func ParseComposeFiles(fs []File) (*Config, error) {
return ParseCompose(cfgs, envs)
}
func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, error) {
cfg, err := loader.Load(compose.ConfigDetails{
func ParseCompose(cfgs []composetypes.ConfigFile, envs map[string]string) (*Config, error) {
if envs == nil {
envs = make(map[string]string)
}
cfg, err := loader.LoadWithContext(context.Background(), composetypes.ConfigDetails{
ConfigFiles: cfgs,
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
options.Profiles = []string{"*"}
})
if err != nil {
return nil, err
@@ -46,6 +55,7 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
g := &Group{Name: "default"}
for _, s := range cfg.Services {
s := s
if s.Build == nil {
continue
}
@@ -65,6 +75,37 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
dockerfilePath := s.Build.Dockerfile
dockerfilePathP = &dockerfilePath
}
var dockerfileInlineP *string
if s.Build.DockerfileInline != "" {
dockerfileInline := s.Build.DockerfileInline
dockerfileInlineP = &dockerfileInline
}
var additionalContexts map[string]string
if s.Build.AdditionalContexts != nil {
additionalContexts = map[string]string{}
for k, v := range s.Build.AdditionalContexts {
additionalContexts[k] = v
}
}
var shmSize *string
if s.Build.ShmSize > 0 {
shmSizeBytes := dockeropts.MemBytes(s.Build.ShmSize)
shmSizeStr := shmSizeBytes.String()
shmSize = &shmSizeStr
}
var ulimits []string
if s.Build.Ulimits != nil {
for n, u := range s.Build.Ulimits {
ulimit, err := units.ParseUlimit(fmt.Sprintf("%s=%d:%d", n, u.Soft, u.Hard))
if err != nil {
return nil, err
}
ulimits = append(ulimits, ulimit.String())
}
}
var secrets []string
for _, bs := range s.Build.Secrets {
@@ -78,16 +119,19 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
// compose does not support nil values for labels
labels := map[string]*string{}
for k, v := range s.Build.Labels {
v := v
labels[k] = &v
}
g.Targets = append(g.Targets, targetName)
t := &Target{
Name: targetName,
Context: contextPathP,
Dockerfile: dockerfilePathP,
Tags: s.Build.Tags,
Labels: labels,
Name: targetName,
Context: contextPathP,
Contexts: additionalContexts,
Dockerfile: dockerfilePathP,
DockerfileInline: dockerfileInlineP,
Tags: s.Build.Tags,
Labels: labels,
Args: flatten(s.Build.Args.Resolve(func(val string) (string, bool) {
if val, ok := s.Environment[val]; ok && val != nil {
return *val, true
@@ -99,6 +143,8 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
CacheTo: s.Build.CacheTo,
NetworkMode: &s.Build.Network,
Secrets: secrets,
ShmSize: shmSize,
Ulimits: ulimits,
}
if err = t.composeExtTarget(s.Build.Extensions); err != nil {
return nil, err
@@ -136,14 +182,15 @@ func validateComposeFile(dt []byte, fn string) (bool, error) {
}
func validateCompose(dt []byte, envs map[string]string) error {
_, err := loader.Load(compose.ConfigDetails{
ConfigFiles: []compose.ConfigFile{
_, err := loader.Load(composetypes.ConfigDetails{
ConfigFiles: []composetypes.ConfigFile{
{
Content: dt,
},
},
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
// consistency is checked later in ParseCompose to ensure multiple
// compose files can be merged together
@@ -184,7 +231,7 @@ func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string,
return nil, err
}
envs, err := dotenv.UnmarshalBytes(dt)
envs, err := dotenv.UnmarshalBytesWithLookup(dt, nil)
if err != nil {
return nil, err
}
@@ -199,7 +246,7 @@ func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string,
return curenv, nil
}
func flatten(in compose.MappingWithEquals) map[string]*string {
func flatten(in composetypes.MappingWithEquals) map[string]*string {
if len(in) == 0 {
return nil
}
@@ -303,8 +350,8 @@ func (t *Target) composeExtTarget(exts map[string]interface{}) error {
// composeToBuildkitSecret converts secret from compose format to buildkit's
// csv format.
func composeToBuildkitSecret(inp compose.ServiceSecretConfig, psecret compose.SecretConfig) (string, error) {
if psecret.External.External {
func composeToBuildkitSecret(inp composetypes.ServiceSecretConfig, psecret composetypes.SecretConfig) (string, error) {
if psecret.External {
return "", errors.Errorf("unsupported external secret %s", psecret.Name)
}

View File

@@ -6,7 +6,7 @@ import (
"sort"
"testing"
compose "github.com/compose-spec/compose-go/types"
composetypes "github.com/compose-spec/compose-go/v2/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -21,6 +21,8 @@ services:
webapp:
build:
context: ./dir
additional_contexts:
foo: ./bar
dockerfile: Dockerfile-alternate
network:
none
@@ -33,6 +35,13 @@ services:
secrets:
- token
- aws
webapp2:
profiles:
- test
build:
context: ./dir
dockerfile_inline: |
FROM alpine
secrets:
token:
environment: ENV_TOKEN
@@ -40,24 +49,25 @@ secrets:
file: /root/.aws/credentials
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
sort.Strings(c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp", "webapp2"}, c.Groups[0].Targets)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, 3, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, "db", *c.Targets[0].Context)
require.Equal(t, []string{"docker.io/tonistiigi/db"}, c.Targets[0].Tags)
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, "./dir", *c.Targets[1].Context)
require.Equal(t, "dir", *c.Targets[1].Context)
require.Equal(t, map[string]string{"foo": "bar"}, c.Targets[1].Contexts)
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
@@ -68,6 +78,10 @@ secrets:
"id=token,env=ENV_TOKEN",
"id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, "dir", *c.Targets[2].Context)
require.Equal(t, "FROM alpine\n", *c.Targets[2].DockerfileInline)
}
func TestNoBuildOutOfTreeService(t *testing.T) {
@@ -78,7 +92,7 @@ services:
webapp:
build: ./db
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, 1, len(c.Targets))
@@ -97,7 +111,7 @@ services:
target: webapp
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
@@ -122,7 +136,7 @@ services:
target: webapp
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
@@ -153,7 +167,7 @@ services:
t.Setenv("BAR", "foo")
t.Setenv("ZZZ_BAR", "zzz_foo")
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, sliceToMap(os.Environ()))
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, sliceToMap(os.Environ()))
require.NoError(t, err)
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["FOO"])
require.Equal(t, ptrstr("zzz_foo"), c.Targets[0].Args["BAR"])
@@ -167,7 +181,7 @@ services:
entrypoint: echo 1
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.Error(t, err)
}
@@ -192,7 +206,7 @@ networks:
gateway: 10.5.0.254
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
@@ -209,7 +223,7 @@ services:
- bar
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, []string{"foo", "bar"}, c.Targets[0].Tags)
}
@@ -246,7 +260,7 @@ networks:
name: test-net
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
@@ -289,6 +303,11 @@ services:
args:
CT_ECR: foo
CT_TAG: bar
shm_size: 128m
ulimits:
nofile:
soft: 1024
hard: 1024
x-bake:
secret:
- id=mysecret,src=/local/secret
@@ -299,7 +318,7 @@ services:
no-cache: true
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
@@ -318,6 +337,8 @@ services:
require.Equal(t, []string{"linux/arm64"}, c.Targets[1].Platforms)
require.Equal(t, []string{"type=docker"}, c.Targets[1].Outputs)
require.Equal(t, newBool(true), c.Targets[1].NoCache)
require.Equal(t, ptrstr("128MiB"), c.Targets[1].ShmSize)
require.Equal(t, []string{"nofile=1024:1024"}, c.Targets[1].Ulimits)
}
func TestComposeExtDedup(t *testing.T) {
@@ -343,7 +364,7 @@ services:
- type=local,dest=path/to/cache
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"ct-addon:foo", "ct-addon:baz"}, c.Targets[0].Tags)
@@ -376,7 +397,7 @@ services:
- ` + envf.Name() + `
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "FOO": ptrstr("bsdf -csdf"), "NODE_ENV": ptrstr("test")}, c.Targets[0].Args)
}
@@ -422,7 +443,7 @@ services:
published: "3306"
protocol: tcp
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
@@ -468,7 +489,7 @@ func TestServiceName(t *testing.T) {
for _, tt := range cases {
tt := tt
t.Run(tt.svc, func(t *testing.T) {
_, err := ParseCompose([]compose.ConfigFile{{Content: []byte(`
_, err := ParseCompose([]composetypes.ConfigFile{{Content: []byte(`
services:
` + tt.svc + `:
build:
@@ -539,7 +560,7 @@ services:
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
_, err := ParseCompose([]compose.ConfigFile{{Content: tt.dt}}, nil)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: tt.dt}}, nil)
if tt.wantErr {
require.Error(t, err)
} else {
@@ -637,11 +658,90 @@ services:
bar: "baz"
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
c, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, c.Targets[0].Args)
}
func TestDependsOn(t *testing.T) {
var dt = []byte(`
services:
foo:
build:
context: .
ports:
- 3306:3306
depends_on:
- bar
bar:
build:
context: .
`)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
func TestInclude(t *testing.T) {
tmpdir := t.TempDir()
err := os.WriteFile(filepath.Join(tmpdir, "compose-foo.yml"), []byte(`
services:
foo:
build:
context: .
target: buildfoo
ports:
- 3306:3306
`), 0644)
require.NoError(t, err)
var dt = []byte(`
include:
- compose-foo.yml
services:
bar:
build:
context: .
target: buildbar
`)
chdir(t, tmpdir)
c, err := ParseComposeFiles([]File{{
Name: "composetypes.yml",
Data: dt,
}})
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, "bar", c.Targets[0].Name)
require.Equal(t, "buildbar", *c.Targets[0].Target)
require.Equal(t, "foo", c.Targets[1].Name)
require.Equal(t, "buildfoo", *c.Targets[1].Target)
}
func TestDevelop(t *testing.T) {
var dt = []byte(`
services:
scratch:
build:
context: ./webapp
develop:
watch:
- path: ./webapp/html
action: sync
target: /var/www
ignore:
- node_modules/
`)
_, err := ParseCompose([]composetypes.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
// chdir changes the current working directory to the named directory,
// and then restore the original working directory at the end of the test.
func chdir(t *testing.T, dir string) {

View File

@@ -634,6 +634,506 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"])
}
func TestHCLMultiFileGlobalAttrs(t *testing.T) {
dt := []byte(`
FOO = "abc"
target "app" {
args = {
v1 = "pre-${FOO}"
}
}
`)
dt2 := []byte(`
FOO = "def"
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
{Data: dt2, Name: "c2.hcl"},
}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, "pre-def", *c.Targets[0].Args["v1"])
}
func TestHCLDuplicateTarget(t *testing.T) {
dt := []byte(`
target "app" {
dockerfile = "x"
}
target "app" {
dockerfile = "y"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, "y", *c.Targets[0].Dockerfile)
}
func TestHCLRenameTarget(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
dockerfile = "foo"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "requires matrix")
}
func TestHCLRenameGroup(t *testing.T) {
dt := []byte(`
group "foo" {
name = "bar"
targets = ["x", "y"]
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
dt = []byte(`
group "foo" {
matrix = {
name = ["x", "y"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
}
func TestHCLRenameTargetAttrs(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.xyz.dockerfile
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "xyz", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "def", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "def" {
dockerfile = target.xyz.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "def", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "xyz", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.abc.dockerfile
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
dt = []byte(`
target "def" {
dockerfile = target.abc.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
}
func TestHCLRenameSplit(t *testing.T) {
dt := []byte(`
target "x" {
name = "y"
matrix = {}
dockerfile = "foo"
}
target "x" {
name = "z"
matrix = {}
dockerfile = "bar"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "y", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "z", c.Targets[1].Name)
require.Equal(t, "bar", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "x", c.Groups[0].Name)
require.Equal(t, []string{"y", "z"}, c.Groups[0].Targets)
}
func TestHCLRenameMultiFile(t *testing.T) {
dt := []byte(`
target "foo" {
name = "bar"
matrix = {}
dockerfile = "x"
}
`)
dt2 := []byte(`
target "foo" {
context = "y"
}
`)
dt3 := []byte(`
target "bar" {
target = "z"
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
{Data: dt2, Name: "c2.hcl"},
{Data: dt3, Name: "c3.hcl"},
}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "bar")
require.Equal(t, *c.Targets[0].Dockerfile, "x")
require.Equal(t, *c.Targets[0].Target, "z")
require.Equal(t, c.Targets[1].Name, "foo")
require.Equal(t, *c.Targets[1].Context, "y")
}
func TestHCLMatrixBasic(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["x", "y"]
}
name = foo
dockerfile = "${foo}.Dockerfile"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "x")
require.Equal(t, c.Targets[1].Name, "y")
require.Equal(t, *c.Targets[0].Dockerfile, "x.Dockerfile")
require.Equal(t, *c.Targets[1].Dockerfile, "y.Dockerfile")
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.Equal(t, []string{"x", "y"}, c.Groups[0].Targets)
}
func TestHCLMatrixMultipleKeys(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a"]
bar = ["b", "c"]
baz = ["d", "e", "f"]
}
name = "${foo}-${bar}-${baz}"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 6, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, names)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, c.Groups[0].Targets)
}
func TestHCLMatrixLists(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [["aa", "bb"], ["cc", "dd"]]
}
name = aa[0]
args = {
target = "val${aa[1]}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "aa", c.Targets[0].Name)
require.Equal(t, ptrstr("valbb"), c.Targets[0].Args["target"])
require.Equal(t, "cc", c.Targets[1].Name)
require.Equal(t, ptrstr("valdd"), c.Targets[1].Args["target"])
}
func TestHCLMatrixMaps(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [
{
foo = "aa"
bar = "bb"
},
{
foo = "cc"
bar = "dd"
}
]
}
name = aa.foo
args = {
target = "val${aa.bar}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "aa")
require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb"))
require.Equal(t, c.Targets[1].Name, "cc")
require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd"))
}
func TestHCLMatrixMultipleTargets(t *testing.T) {
dt := []byte(`
target "x" {
matrix = {
foo = ["a", "b"]
}
name = foo
}
target "y" {
matrix = {
bar = ["c", "d"]
}
name = bar
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 4, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a", "b", "c", "d"}, names)
require.Equal(t, 2, len(c.Groups))
names = make([]string, len(c.Groups))
for i, c := range c.Groups {
names[i] = c.Name
}
require.ElementsMatch(t, []string{"x", "y"}, names)
for _, g := range c.Groups {
switch g.Name {
case "x":
require.Equal(t, []string{"a", "b"}, g.Targets)
case "y":
require.Equal(t, []string{"c", "d"}, g.Targets)
}
}
}
func TestHCLMatrixDuplicateNames(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a", "b"]
}
name = "c"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestHCLMatrixArgs(t *testing.T) {
dt := []byte(`
a = 1
variable "b" {
default = 2
}
target "default" {
matrix = {
foo = [a, b]
}
name = foo
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "1", c.Targets[0].Name)
require.Equal(t, "2", c.Targets[1].Name)
}
func TestHCLMatrixArgsOverride(t *testing.T) {
dt := []byte(`
variable "ABC" {
default = "def"
}
target "bar" {
matrix = {
aa = split(",", ABC)
}
name = "bar-${aa}"
args = {
foo = aa
}
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "docker-bake.hcl"},
}, map[string]string{"ABC": "11,22,33"})
require.NoError(t, err)
require.Equal(t, 3, len(c.Targets))
require.Equal(t, "bar-11", c.Targets[0].Name)
require.Equal(t, "bar-22", c.Targets[1].Name)
require.Equal(t, "bar-33", c.Targets[2].Name)
require.Equal(t, ptrstr("11"), c.Targets[0].Args["foo"])
require.Equal(t, ptrstr("22"), c.Targets[1].Args["foo"])
require.Equal(t, ptrstr("33"), c.Targets[2].Args["foo"])
}
func TestHCLMatrixBadTypes(t *testing.T) {
dt := []byte(`
target "default" {
matrix = "test"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = ["test"]
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
["a"] = ["b"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
1 = 2
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
a = "b"
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestHCLMatrixWithGlobalTarget(t *testing.T) {
dt := []byte(`
target "x" {
tags = ["a", "b"]
}
target "default" {
tags = target.x.tags
matrix = {
dummy = [""]
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "x", c.Targets[0].Name)
require.Equal(t, "default", c.Targets[1].Name)
require.Equal(t, []string{"a", "b"}, c.Targets[1].Tags)
}
func TestJSONAttributes(t *testing.T) {
dt := []byte(`{"FOO": "abc", "variable": {"BAR": {"default": "def"}}, "target": { "app": { "args": {"v1": "pre-${FOO}-${BAR}"}} } }`)
@@ -670,6 +1170,24 @@ func TestJSONFunctions(t *testing.T) {
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
}
func TestJSONInvalidFunctions(t *testing.T) {
dt := []byte(`{
"target": {
"app": {
"args": {
"v1": "myfunc(\"foo\")"
}
}
}}`)
c, err := ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
}
func TestHCLFunctionInAttr(t *testing.T) {
dt := []byte(`
function "brace" {
@@ -928,7 +1446,7 @@ func TestVarUnsupportedType(t *testing.T) {
}
func ptrstr(s interface{}) *string {
var n *string = nil
var n *string
if reflect.ValueOf(s).Kind() == reflect.String {
ss := s.(string)
n = &ss

View File

@@ -14,15 +14,7 @@ func funcCalls(exp hcl.Expression) ([]string, hcl.Diagnostics) {
if !ok {
fns, err := jsonFuncCallsRecursive(exp)
if err != nil {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
return nil, wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
return fns, nil
}

View File

@@ -1,7 +1,9 @@
package hclparser
import (
"encoding/binary"
"fmt"
"hash/fnv"
"math"
"math/big"
"reflect"
@@ -49,36 +51,42 @@ type parser struct {
attrs map[string]*hcl.Attribute
funcs map[string]*functionDef
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block]reflect.Value
blockTypes map[string]reflect.Type
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block][]reflect.Value
blockEvalCtx map[*hcl.Block][]*hcl.EvalContext
blockNames map[*hcl.Block][]string
blockTypes map[string]reflect.Type
ectx *hcl.EvalContext
progress map[string]struct{}
progressF map[string]struct{}
progressB map[*hcl.Block]map[string]struct{}
doneF map[string]struct{}
doneB map[*hcl.Block]map[string]struct{}
progressV map[uint64]struct{}
progressF map[uint64]struct{}
progressB map[uint64]map[string]struct{}
doneB map[uint64]map[string]struct{}
}
func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.Diagnostics {
type WithEvalContexts interface {
GetEvalContexts(base *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error)
}
type WithGetName interface {
GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error)
}
var errUndefined = errors.New("undefined")
func (p *parser) loadDeps(ectx *hcl.EvalContext, exp hcl.Expression, exclude map[string]struct{}, allowMissing bool) hcl.Diagnostics {
fns, hcldiags := funcCalls(exp)
if hcldiags.HasErrors() {
return hcldiags
}
for _, fn := range fns {
if err := p.resolveFunction(fn); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
if err := p.resolveFunction(ectx, fn); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
@@ -127,28 +135,20 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
}
}
}
if err := p.resolveBlock(blocks[0], target); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
for _, block := range blocks {
if err := p.resolveBlock(block, target); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
} else {
if err := p.resolveValue(v.RootName()); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
if err := p.resolveValue(ectx, v.RootName()); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
}
@@ -158,21 +158,21 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
// resolveFunction forces evaluation of a function, storing the result into the
// parser.
func (p *parser) resolveFunction(name string) error {
if _, ok := p.doneF[name]; ok {
func (p *parser) resolveFunction(ectx *hcl.EvalContext, name string) error {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
if _, ok := ectx.Functions[name]; ok {
return nil
}
f, ok := p.funcs[name]
if !ok {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
return errors.Errorf("undefined function %s", name)
return errors.Wrapf(errUndefined, "function %q does not exist", name)
}
if _, ok := p.progressF[name]; ok {
if _, ok := p.progressF[key(ectx, name)]; ok {
return errors.Errorf("function cycle not allowed for %s", name)
}
p.progressF[name] = struct{}{}
p.progressF[key(ectx, name)] = struct{}{}
if f.Result == nil {
return errors.Errorf("empty result not allowed for %s", name)
@@ -217,7 +217,7 @@ func (p *parser) resolveFunction(name string) error {
return diags
}
if diags := p.loadDeps(f.Result.Expr, params); diags.HasErrors() {
if diags := p.loadDeps(p.ectx, f.Result.Expr, params, false); diags.HasErrors() {
return diags
}
@@ -227,7 +227,6 @@ func (p *parser) resolveFunction(name string) error {
if diags.HasErrors() {
return diags
}
p.doneF[name] = struct{}{}
p.ectx.Functions[name] = v
return nil
@@ -235,14 +234,17 @@ func (p *parser) resolveFunction(name string) error {
// resolveValue forces evaluation of a named value, storing the result into the
// parser.
func (p *parser) resolveValue(name string) (err error) {
func (p *parser) resolveValue(ectx *hcl.EvalContext, name string) (err error) {
if _, ok := p.ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progress[name]; ok {
if _, ok := ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progressV[key(ectx, name)]; ok {
return errors.Errorf("variable cycle not allowed for %s", name)
}
p.progress[name] = struct{}{}
p.progressV[key(ectx, name)] = struct{}{}
var v *cty.Value
defer func() {
@@ -255,9 +257,10 @@ func (p *parser) resolveValue(name string) (err error) {
if _, builtin := p.opt.Vars[name]; !ok && !builtin {
vr, ok := p.vars[name]
if !ok {
return errors.Errorf("undefined variable %q", name)
return errors.Wrapf(errUndefined, "variable %q does not exist", name)
}
def = vr.Default
ectx = p.ectx
}
if def == nil {
@@ -270,10 +273,10 @@ func (p *parser) resolveValue(name string) (err error) {
return
}
if diags := p.loadDeps(def.Expr, nil); diags.HasErrors() {
if diags := p.loadDeps(ectx, def.Expr, nil, true); diags.HasErrors() {
return diags
}
vv, diags := def.Expr.Value(p.ectx)
vv, diags := def.Expr.Value(ectx)
if diags.HasErrors() {
return diags
}
@@ -312,154 +315,226 @@ func (p *parser) resolveValue(name string) (err error) {
// target schema is provided, only the attributes and blocks present in the
// schema will be evaluated.
func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) {
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid name",
Detail: err.Error(),
Subject: &block.LabelRanges[0],
},
}
// prepare the variable map for this type
if _, ok := p.ectx.Variables[block.Type]; !ok {
p.ectx.Variables[block.Type] = cty.MapValEmpty(cty.Map(cty.String))
}
if _, ok := p.doneB[block]; !ok {
p.doneB[block] = map[string]struct{}{}
}
if _, ok := p.progressB[block]; !ok {
p.progressB[block] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[block][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[block][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[block][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[block][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[block][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[block][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[block] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
// prepare the output destination and evaluation context
t, ok := p.blockTypes[block.Type]
if !ok {
return nil
}
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
var outputs []reflect.Value
var ectxs []*hcl.EvalContext
if prev, ok := p.blockValues[block]; ok {
outputs = prev
ectxs = p.blockEvalCtx[block]
} else {
if v, ok := reflect.New(t).Interface().(WithEvalContexts); ok {
ectxs, err = v.GetEvalContexts(p.ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(p.ectx, expr, nil, true)
})
if err != nil {
return err
}
for _, ectx := range ectxs {
if ectx != p.ectx && ectx.Parent() != p.ectx {
return errors.Errorf("EvalContext must return a context with the correct parent")
}
}
} else {
ectxs = append([]*hcl.EvalContext{}, p.ectx)
}
for range ectxs {
outputs = append(outputs, reflect.New(t))
}
}
for _, a := range content.Attributes {
diag := p.loadDeps(a.Expr, nil)
p.blockValues[block] = outputs
p.blockEvalCtx[block] = ectxs
for i, output := range outputs {
target := target
ectx := ectxs[i]
name := block.Labels[0]
if names, ok := p.blockNames[block]; ok {
name = names[i]
}
if _, ok := p.doneB[key(block, ectx)]; !ok {
p.doneB[key(block, ectx)] = map[string]struct{}{}
}
if _, ok := p.progressB[key(block, ectx)]; !ok {
p.progressB[key(block, ectx)] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[key(block, ectx)][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[key(block, ectx)][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[key(block, ectx)][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[key(block, ectx)][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[key(block, ectx)][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[key(block, ectx)] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
for _, a := range content.Attributes {
diag := p.loadDeps(ectx, a.Expr, nil, true)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
if err != nil {
return err
}
}
// decode!
diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() {
return diag
}
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
}
// store the result into the evaluation context (so it can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
}
// decode!
var output reflect.Value
if prev, ok := p.blockValues[block]; ok {
output = prev
} else {
output = reflect.New(t)
setLabel(output, block.Labels[0]) // early attach labels, so we can reference them
}
diag = gohcl.DecodeBody(body(), p.ectx, output.Interface())
if diag.HasErrors() {
return diag
}
p.blockValues[block] = output
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[block][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[block][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[block][a.Name] = struct{}{}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
for _, b := range target.Blocks {
p.doneB[block][b.Type] = struct{}{}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
}
// store the result into the evaluation context (so if can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
return nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
// resolveBlockNames returns the names of the block, calling resolveBlock to
// evaluate any label fields to correctly resolve the name.
func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
if names, ok := p.blockNames[block]; ok {
return names, nil
}
if err := p.resolveBlock(block, &hcl.BodySchema{}); err != nil {
return nil, err
}
names := make([]string, 0, len(p.blockValues[block]))
for i, val := range p.blockValues[block] {
ectx := p.blockEvalCtx[block][i]
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
if v, ok := val.Interface().(WithGetName); ok {
var err error
name, err = v.GetName(ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(ectx, expr, nil, true)
})
if err != nil {
return nil, err
}
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
}
setName(val, name)
names = append(names, name)
}
found := map[string]struct{}{}
for _, name := range names {
if _, ok := found[name]; ok {
return nil, errors.Errorf("duplicate name %q", name)
}
found[name] = struct{}{}
}
p.blockNames[block] = names
return names, nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) (map[string]map[string][]string, hcl.Diagnostics) {
reserved := map[string]struct{}{}
schema, _ := gohcl.ImpliedBodySchema(val)
@@ -472,7 +547,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
var defs inputs
if err := gohcl.DecodeBody(b, nil, &defs); err != nil {
return err
return nil, err
}
defsSchema, _ := gohcl.ImpliedBodySchema(defs)
@@ -495,20 +570,20 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
attrs: map[string]*hcl.Attribute{},
funcs: map[string]*functionDef{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block]reflect.Value{},
blockTypes: map[string]reflect.Type{},
progress: map[string]struct{}{},
progressF: map[string]struct{}{},
progressB: map[*hcl.Block]map[string]struct{}{},
doneF: map[string]struct{}{},
doneB: map[*hcl.Block]map[string]struct{}{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block][]reflect.Value{},
blockEvalCtx: map[*hcl.Block][]*hcl.EvalContext{},
blockNames: map[*hcl.Block][]string{},
blockTypes: map[string]reflect.Type{},
ectx: &hcl.EvalContext{
Variables: map[string]cty.Value{},
Functions: stdlibFunctions,
Functions: Stdlib(),
},
progressV: map[uint64]struct{}{},
progressF: map[uint64]struct{}{},
progressB: map[uint64]map[string]struct{}{},
doneB: map[uint64]map[string]struct{}{},
}
for _, v := range defs.Variables {
@@ -528,18 +603,18 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
content, b, diags := b.PartialContent(schema)
if diags.HasErrors() {
return diags
return nil, diags
}
blocks, b, diags := b.PartialContent(defsSchema)
if diags.HasErrors() {
return diags
return nil, diags
}
attrs, diags := b.JustAttributes()
if diags.HasErrors() {
if d := removeAttributesDiags(diags, reserved, p.vars); len(d) > 0 {
return d
if d := removeAttributesDiags(diags, reserved, p.vars, attrs); len(d) > 0 {
return nil, d
}
}
@@ -552,92 +627,56 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
delete(p.attrs, "function")
for k := range p.opt.Vars {
_ = p.resolveValue(k)
_ = p.resolveValue(p.ectx, k)
}
for _, a := range content.Attributes {
return hcl.Diagnostics{
a := a
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: "global attributes currently not supported",
Subject: &a.Range,
Context: &a.Range,
Subject: a.Range.Ptr(),
Context: a.Range.Ptr(),
},
}
}
for k := range p.vars {
if err := p.resolveValue(k); err != nil {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
return nil, diags
}
r := p.vars[k].Body.MissingItemRange()
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid value",
Detail: err.Error(),
Subject: &r,
Context: &r,
},
}
return nil, wrapErrorDiagnostic("Invalid value", err, &r, &r)
}
}
for k := range p.funcs {
if err := p.resolveFunction(k); err != nil {
if err := p.resolveFunction(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
return nil, diags
}
var subject *hcl.Range
var context *hcl.Range
if p.funcs[k].Params != nil {
subject = &p.funcs[k].Params.Range
subject = p.funcs[k].Params.Range.Ptr()
context = subject
} else {
for _, block := range blocks.Blocks {
block := block
if block.Type == "function" && len(block.Labels) == 1 && block.Labels[0] == k {
subject = &block.LabelRanges[0]
context = &block.DefRange
subject = block.LabelRanges[0].Ptr()
context = block.DefRange.Ptr()
break
}
}
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid function",
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
return nil, wrapErrorDiagnostic("Invalid function", err, subject, context)
}
}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
bm, ok := p.blocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
p.blocks[b.Type] = bm
}
lbl := b.Labels[0]
bm[lbl] = append(bm[lbl], b)
}
type value struct {
reflect.Value
idx int
@@ -648,7 +687,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
values map[string]value
}
types := map[string]field{}
renamed := map[string]map[string][]string{}
vt := reflect.ValueOf(val).Elem().Type()
for i := 0; i < vt.NumField(); i++ {
tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",")
@@ -659,10 +698,43 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
typ: vt.Field(i).Type,
values: make(map[string]value),
}
renamed[tags[0]] = map[string][]string{}
}
tmpBlocks := map[string]map[string][]*hcl.Block{}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
bm, ok := tmpBlocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
tmpBlocks[b.Type] = bm
}
names, err := p.resolveBlockNames(b)
if err != nil {
return nil, wrapErrorDiagnostic("Invalid name", err, &b.LabelRanges[0], &b.LabelRanges[0])
}
for _, name := range names {
bm[name] = append(bm[name], b)
renamed[b.Type][b.Labels[0]] = append(renamed[b.Type][b.Labels[0]], name)
}
}
p.blocks = tmpBlocks
diags = hcl.Diagnostics{}
for _, b := range content.Blocks {
b := b
v := reflect.ValueOf(val)
err := p.resolveBlock(b, nil)
@@ -673,89 +745,119 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
continue
}
} else {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &b.LabelRanges[0],
Context: &b.DefRange,
},
}
return nil, wrapErrorDiagnostic("Invalid block", err, b.LabelRanges[0].Ptr(), b.DefRange.Ptr())
}
}
vv := p.blockValues[b]
t := types[b.Type]
lblIndex := setLabel(vv, b.Labels[0])
oldValue, exists := t.values[b.Labels[0]]
if !exists && lblIndex != -1 {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if b.Labels[0] == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
vvs := p.blockValues[b]
for _, vv := range vvs {
t := types[b.Type]
lblIndex, lblExists := getNameIndex(vv)
lblName, _ := getName(vv)
oldValue, exists := t.values[lblName]
if !exists && lblExists {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
}
}
}
}
}
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
}
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[lblName] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
} else {
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[b.Labels[0]] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
}
if diags.HasErrors() {
return diags
return nil, diags
}
for k := range p.attrs {
if err := p.resolveValue(k); err != nil {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &p.attrs[k].Range,
Context: &p.attrs[k].Range,
},
return nil, diags
}
return nil, wrapErrorDiagnostic("Invalid attribute", err, &p.attrs[k].Range, &p.attrs[k].Range)
}
}
return nil
return renamed, nil
}
func setLabel(v reflect.Value, lbl string) int {
// cache field index?
// wrapErrorDiagnostic wraps an error into a hcl.Diagnostics object.
// If the error is already an hcl.Diagnostics object, it is returned as is.
func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context *hcl.Range) hcl.Diagnostics {
switch err := err.(type) {
case *hcl.Diagnostic:
return hcl.Diagnostics{err}
case hcl.Diagnostics:
return err
default:
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: message,
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
}
func setName(v reflect.Value, name string) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
for _, t := range strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
v.Elem().Field(i).Set(reflect.ValueOf(lbl))
return i
v.Elem().Field(i).Set(reflect.ValueOf(name))
}
}
}
return -1
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable) hcl.Diagnostics {
func getName(v reflect.Value) (string, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return v.Elem().Field(i).String(), true
}
}
}
return "", false
}
func getNameIndex(v reflect.Value) (int, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return i, true
}
}
}
return 0, false
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable, attrs hcl.Attributes) hcl.Diagnostics {
var fdiags hcl.Diagnostics
for _, d := range diags {
if fout := func(d *hcl.Diagnostic) bool {
@@ -777,6 +879,12 @@ func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{},
return true
}
}
for a := range attrs {
// Do the same for attributes
if strings.HasPrefix(d.Detail, fmt.Sprintf(`Argument "%s" was already set at `, a)) {
return true
}
}
return false
}(d); !fout {
fdiags = append(fdiags, d)
@@ -784,3 +892,21 @@ func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{},
}
return fdiags
}
// key returns a unique hash for the given values
func key(ks ...any) uint64 {
hash := fnv.New64a()
for _, k := range ks {
v := reflect.ValueOf(k)
switch v.Kind() {
case reflect.String:
hash.Write([]byte(v.String()))
case reflect.Pointer:
ptr := reflect.ValueOf(k).Pointer()
binary.Write(hash, binary.LittleEndian, uint64(ptr))
default:
panic(fmt.Sprintf("unknown key kind %s", v.Kind().String()))
}
}
return hash.Sum64()
}

230
bake/hclparser/merged.go Normal file
View File

@@ -0,0 +1,230 @@
// Copyright (c) HashiCorp, Inc.
// SPDX-License-Identifier: MPL-2.0
// Forked from https://github.com/hashicorp/hcl/blob/4679383728fe331fc8a6b46036a27b8f818d9bc0/merged.go
package hclparser
import (
"fmt"
"github.com/hashicorp/hcl/v2"
)
// MergeFiles combines the given files to produce a single body that contains
// configuration from all of the given files.
//
// The ordering of the given files decides the order in which contained
// elements will be returned. If any top-level attributes are defined with
// the same name across multiple files, a diagnostic will be produced from
// the Content and PartialContent methods describing this error in a
// user-friendly way.
func MergeFiles(files []*hcl.File) hcl.Body {
var bodies []hcl.Body
for _, file := range files {
bodies = append(bodies, file.Body)
}
return MergeBodies(bodies)
}
// MergeBodies is like MergeFiles except it deals directly with bodies, rather
// than with entire files.
func MergeBodies(bodies []hcl.Body) hcl.Body {
if len(bodies) == 0 {
// Swap out for our singleton empty body, to reduce the number of
// empty slices we have hanging around.
return emptyBody
}
// If any of the given bodies are already merged bodies, we'll unpack
// to flatten to a single mergedBodies, since that's conceptually simpler.
// This also, as a side-effect, eliminates any empty bodies, since
// empties are merged bodies with no inner bodies.
var newLen int
var flatten bool
for _, body := range bodies {
if children, merged := body.(mergedBodies); merged {
newLen += len(children)
flatten = true
} else {
newLen++
}
}
if !flatten { // not just newLen == len, because we might have mergedBodies with single bodies inside
return mergedBodies(bodies)
}
if newLen == 0 {
// Don't allocate a new empty when we already have one
return emptyBody
}
n := make([]hcl.Body, 0, newLen)
for _, body := range bodies {
if children, merged := body.(mergedBodies); merged {
n = append(n, children...)
} else {
n = append(n, body)
}
}
return mergedBodies(n)
}
var emptyBody = mergedBodies([]hcl.Body{})
// EmptyBody returns a body with no content. This body can be used as a
// placeholder when a body is required but no body content is available.
func EmptyBody() hcl.Body {
return emptyBody
}
type mergedBodies []hcl.Body
// Content returns the content produced by applying the given schema to all
// of the merged bodies and merging the result.
//
// Although required attributes _are_ supported, they should be used sparingly
// with merged bodies since in this case there is no contextual information
// with which to return good diagnostics. Applications working with merged
// bodies may wish to mark all attributes as optional and then check for
// required attributes afterwards, to produce better diagnostics.
func (mb mergedBodies) Content(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Diagnostics) {
// the returned body will always be empty in this case, because mergedContent
// will only ever call Content on the child bodies.
content, _, diags := mb.mergedContent(schema, false)
return content, diags
}
func (mb mergedBodies) PartialContent(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
return mb.mergedContent(schema, true)
}
func (mb mergedBodies) JustAttributes() (hcl.Attributes, hcl.Diagnostics) {
attrs := make(map[string]*hcl.Attribute)
var diags hcl.Diagnostics
for _, body := range mb {
thisAttrs, thisDiags := body.JustAttributes()
if len(thisDiags) != 0 {
diags = append(diags, thisDiags...)
}
if thisAttrs != nil {
for name, attr := range thisAttrs {
if existing := attrs[name]; existing != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate argument",
Detail: fmt.Sprintf(
"Argument %q was already set at %s",
name, existing.NameRange.String(),
),
Subject: thisAttrs[name].NameRange.Ptr(),
})
}
attrs[name] = attr
}
}
}
return attrs, diags
}
func (mb mergedBodies) MissingItemRange() hcl.Range {
if len(mb) == 0 {
// Nothing useful to return here, so we'll return some garbage.
return hcl.Range{
Filename: "<empty>",
}
}
// arbitrarily use the first body's missing item range
return mb[0].MissingItemRange()
}
func (mb mergedBodies) mergedContent(schema *hcl.BodySchema, partial bool) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
// We need to produce a new schema with none of the attributes marked as
// required, since _any one_ of our bodies can contribute an attribute value.
// We'll separately check that all required attributes are present at
// the end.
mergedSchema := &hcl.BodySchema{
Blocks: schema.Blocks,
}
for _, attrS := range schema.Attributes {
mergedAttrS := attrS
mergedAttrS.Required = false
mergedSchema.Attributes = append(mergedSchema.Attributes, mergedAttrS)
}
var mergedLeftovers []hcl.Body
content := &hcl.BodyContent{
Attributes: map[string]*hcl.Attribute{},
}
var diags hcl.Diagnostics
for _, body := range mb {
var thisContent *hcl.BodyContent
var thisLeftovers hcl.Body
var thisDiags hcl.Diagnostics
if partial {
thisContent, thisLeftovers, thisDiags = body.PartialContent(mergedSchema)
} else {
thisContent, thisDiags = body.Content(mergedSchema)
}
if thisLeftovers != nil {
mergedLeftovers = append(mergedLeftovers, thisLeftovers)
}
if len(thisDiags) != 0 {
diags = append(diags, thisDiags...)
}
if thisContent.Attributes != nil {
for name, attr := range thisContent.Attributes {
if existing := content.Attributes[name]; existing != nil {
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Duplicate argument",
Detail: fmt.Sprintf(
"Argument %q was already set at %s",
name, existing.NameRange.String(),
),
Subject: thisContent.Attributes[name].NameRange.Ptr(),
})
}
content.Attributes[name] = attr
}
}
if len(thisContent.Blocks) != 0 {
content.Blocks = append(content.Blocks, thisContent.Blocks...)
}
}
// Finally, we check for required attributes.
for _, attrS := range schema.Attributes {
if !attrS.Required {
continue
}
if content.Attributes[attrS.Name] == nil {
// We don't have any context here to produce a good diagnostic,
// which is why we warn in the Content docstring to minimize the
// use of required attributes on merged bodies.
diags = diags.Append(&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Missing required argument",
Detail: fmt.Sprintf(
"The argument %q is required, but was not set.",
attrS.Name,
),
})
}
}
leftoverBody := MergeBodies(mergedLeftovers)
return content, leftoverBody, diags
}

View File

@@ -31,21 +31,21 @@ var stdlibFunctions = map[string]function.Function{
"cidrnetmask": cidr.NetmaskFunc,
"cidrsubnet": cidr.SubnetFunc,
"cidrsubnets": cidr.SubnetsFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"convert": typeexpr.ConvertFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"distinct": stdlib.DistinctFunc,
"divide": stdlib.DivideFunc,
"element": stdlib.ElementFunc,
"equal": stdlib.EqualFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"formatdate": stdlib.FormatDateFunc,
"format": stdlib.FormatFunc,
"formatdate": stdlib.FormatDateFunc,
"formatlist": stdlib.FormatListFunc,
"greaterthan": stdlib.GreaterThanFunc,
"greaterthanorequalto": stdlib.GreaterThanOrEqualToFunc,
@@ -53,10 +53,10 @@ var stdlibFunctions = map[string]function.Function{
"indent": stdlib.IndentFunc,
"index": stdlib.IndexFunc,
"int": stdlib.IntFunc,
"join": stdlib.JoinFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"join": stdlib.JoinFunc,
"length": stdlib.LengthFunc,
"lessthan": stdlib.LessThanFunc,
"lessthanorequalto": stdlib.LessThanOrEqualToFunc,
@@ -70,15 +70,16 @@ var stdlibFunctions = map[string]function.Function{
"modulo": stdlib.ModuloFunc,
"multiply": stdlib.MultiplyFunc,
"negate": stdlib.NegateFunc,
"notequal": stdlib.NotEqualFunc,
"not": stdlib.NotFunc,
"notequal": stdlib.NotEqualFunc,
"or": stdlib.OrFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regexall": stdlib.RegexAllFunc,
"regex": stdlib.RegexFunc,
"regex_replace": stdlib.RegexReplaceFunc,
"regex": stdlib.RegexFunc,
"regexall": stdlib.RegexAllFunc,
"replace": stdlib.ReplaceFunc,
"reverse": stdlib.ReverseFunc,
"reverselist": stdlib.ReverseListFunc,
"rsadecrypt": crypto.RsaDecryptFunc,
@@ -124,3 +125,11 @@ var timestampFunc = function.New(&function.Spec{
return cty.StringVal(time.Now().UTC().Format(time.RFC3339)), nil
},
})
func Stdlib() map[string]function.Function {
funcs := make(map[string]function.Function, len(stdlibFunctions))
for k, v := range stdlibFunctions {
funcs[k] = v
}
return funcs
}

View File

@@ -4,14 +4,16 @@ import (
"archive/tar"
"bytes"
"context"
"strings"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
)
@@ -21,10 +23,16 @@ type Input struct {
}
func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
var session []session.Attachable
var filename string
st, ok := detectGitContext(url)
if !ok {
st, filename, ok = detectHTTPContext(url)
st, ok := dockerui.DetectGitContext(url, false)
if ok {
ssh, err := controllerapi.CreateSSH([]*controllerapi.SSH{{ID: "default"}})
if err == nil {
session = append(session, ssh)
}
} else {
st, filename, ok = dockerui.DetectHTTPContext(url)
if !ok {
return nil, nil, errors.Errorf("not url context")
}
@@ -34,9 +42,9 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
var files []File
var node *builder.Node
for _, n := range nodes {
for i, n := range nodes {
if n.Err == nil {
node = &n
node = &nodes[i]
continue
}
}
@@ -51,7 +59,7 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Session: session, Internal: true}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
@@ -83,51 +91,6 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
return files, inp, nil
}
func IsRemoteURL(url string) bool {
if _, _, ok := detectHTTPContext(url); ok {
return true
}
if _, ok := detectGitContext(url); ok {
return true
}
return false
}
func detectHTTPContext(url string) (*llb.State, string, bool) {
if httpPrefix.MatchString(url) {
httpContext := llb.HTTP(url, llb.Filename("context"), llb.WithCustomName("[internal] load remote build context"))
return &httpContext, "context", true
}
return nil, "", false
}
func detectGitContext(ref string) (*llb.State, bool) {
found := false
if httpPrefix.MatchString(ref) && gitURLPathWithFragmentSuffix.MatchString(ref) {
found = true
}
for _, prefix := range []string{"git://", "github.com/", "git@"} {
if strings.HasPrefix(ref, prefix) {
found = true
break
}
}
if !found {
return nil, false
}
parts := strings.SplitN(ref, "#", 2)
branch := ""
if len(parts) > 1 {
branch = parts[1]
}
gitOpts := []llb.GitOption{llb.WithCustomName("[internal] load git source " + ref)}
st := llb.Git(parts[0], branch, gitOpts...)
return &st, true
}
func isArchive(header []byte) bool {
for _, m := range [][]byte{
{0x42, 0x5A, 0x68}, // bzip2

File diff suppressed because it is too large Load Diff

62
build/dial.go Normal file
View File

@@ -0,0 +1,62 @@
package build
import (
"context"
stderrors "errors"
"net"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/progress"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
func Dial(ctx context.Context, nodes []builder.Node, pw progress.Writer, platform *v1.Platform) (net.Conn, error) {
nodes, err := filterAvailableNodes(nodes)
if err != nil {
return nil, err
}
if len(nodes) == 0 {
return nil, errors.New("no nodes available")
}
var pls []v1.Platform
if platform != nil {
pls = []v1.Platform{*platform}
}
opts := map[string]Options{"default": {Platforms: pls}}
resolved, err := resolveDrivers(ctx, nodes, opts, pw)
if err != nil {
return nil, err
}
var dialError error
for _, ls := range resolved {
for _, rn := range ls {
if platform != nil {
p := *platform
var found bool
for _, pp := range rn.platforms {
if platforms.Only(p).Match(pp) {
found = true
break
}
}
if !found {
continue
}
}
conn, err := nodes[rn.driverIndex].Driver.Dial(ctx)
if err == nil {
return conn, nil
}
dialError = stderrors.Join(err)
}
}
return nil, errors.Wrap(dialError, "no nodes available")
}

312
build/driver.go Normal file
View File

@@ -0,0 +1,312 @@
package build
import (
"context"
"fmt"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/util/flightcontrol"
"github.com/moby/buildkit/util/tracing"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"go.opentelemetry.io/otel/trace"
"golang.org/x/sync/errgroup"
)
type resolvedNode struct {
resolver *nodeResolver
driverIndex int
platforms []specs.Platform
}
func (dp resolvedNode) Node() builder.Node {
return dp.resolver.nodes[dp.driverIndex]
}
func (dp resolvedNode) Client(ctx context.Context) (*client.Client, error) {
clients, err := dp.resolver.boot(ctx, []int{dp.driverIndex}, nil)
if err != nil {
return nil, err
}
return clients[0], nil
}
func (dp resolvedNode) BuildOpts(ctx context.Context) (gateway.BuildOpts, error) {
opts, err := dp.resolver.opts(ctx, []int{dp.driverIndex}, nil)
if err != nil {
return gateway.BuildOpts{}, err
}
return opts[0], nil
}
type matchMaker func(specs.Platform) platforms.MatchComparer
type nodeResolver struct {
nodes []builder.Node
clients flightcontrol.Group[*client.Client]
opt flightcontrol.Group[gateway.BuildOpts]
}
func resolveDrivers(ctx context.Context, nodes []builder.Node, opt map[string]Options, pw progress.Writer) (map[string][]*resolvedNode, error) {
driverRes := newDriverResolver(nodes)
drivers, err := driverRes.Resolve(ctx, opt, pw)
if err != nil {
return nil, err
}
return drivers, err
}
func newDriverResolver(nodes []builder.Node) *nodeResolver {
r := &nodeResolver{
nodes: nodes,
}
return r
}
func (r *nodeResolver) Resolve(ctx context.Context, opt map[string]Options, pw progress.Writer) (map[string][]*resolvedNode, error) {
if len(r.nodes) == 0 {
return nil, nil
}
nodes := map[string][]*resolvedNode{}
for k, opt := range opt {
node, perfect, err := r.resolve(ctx, opt.Platforms, pw, platforms.OnlyStrict, nil)
if err != nil {
return nil, err
}
if !perfect {
break
}
nodes[k] = node
}
if len(nodes) != len(opt) {
// if we didn't get a perfect match, we need to boot all drivers
allIndexes := make([]int, len(r.nodes))
for i := range allIndexes {
allIndexes[i] = i
}
clients, err := r.boot(ctx, allIndexes, pw)
if err != nil {
return nil, err
}
eg, egCtx := errgroup.WithContext(ctx)
workers := make([][]specs.Platform, len(clients))
for i, c := range clients {
i, c := i, c
if c == nil {
continue
}
eg.Go(func() error {
ww, err := c.ListWorkers(egCtx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
ps := make(map[string]specs.Platform, len(ww))
for _, w := range ww {
for _, p := range w.Platforms {
pk := platforms.Format(platforms.Normalize(p))
ps[pk] = p
}
}
for _, p := range ps {
workers[i] = append(workers[i], p)
}
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
// then we can attempt to match against all the available platforms
// (this time we don't care about imperfect matches)
nodes = map[string][]*resolvedNode{}
for k, opt := range opt {
node, _, err := r.resolve(ctx, opt.Platforms, pw, platforms.Only, func(idx int, n builder.Node) []specs.Platform {
return workers[idx]
})
if err != nil {
return nil, err
}
nodes[k] = node
}
}
idxs := make([]int, 0, len(r.nodes))
for _, nodes := range nodes {
for _, node := range nodes {
idxs = append(idxs, node.driverIndex)
}
}
// preload capabilities
span, ctx := tracing.StartSpan(ctx, "load buildkit capabilities", trace.WithSpanKind(trace.SpanKindInternal))
_, err := r.opts(ctx, idxs, pw)
tracing.FinishWithError(span, err)
if err != nil {
return nil, err
}
return nodes, nil
}
func (r *nodeResolver) resolve(ctx context.Context, ps []specs.Platform, pw progress.Writer, matcher matchMaker, additional func(idx int, n builder.Node) []specs.Platform) ([]*resolvedNode, bool, error) {
if len(r.nodes) == 0 {
return nil, true, nil
}
perfect := true
nodeIdxs := make([]int, 0)
for _, p := range ps {
idx := r.get(p, matcher, additional)
if idx == -1 {
idx = 0
perfect = false
}
nodeIdxs = append(nodeIdxs, idx)
}
var nodes []*resolvedNode
if len(nodeIdxs) == 0 {
nodes = append(nodes, &resolvedNode{
resolver: r,
driverIndex: 0,
})
} else {
for i, idx := range nodeIdxs {
node := &resolvedNode{
resolver: r,
driverIndex: idx,
}
if len(ps) > 0 {
node.platforms = []specs.Platform{ps[i]}
}
nodes = append(nodes, node)
}
}
nodes = recombineNodes(nodes)
if _, err := r.boot(ctx, nodeIdxs, pw); err != nil {
return nil, false, err
}
return nodes, perfect, nil
}
func (r *nodeResolver) get(p specs.Platform, matcher matchMaker, additionalPlatforms func(int, builder.Node) []specs.Platform) int {
best := -1
bestPlatform := specs.Platform{}
for i, node := range r.nodes {
platforms := node.Platforms
if additionalPlatforms != nil {
platforms = append([]specs.Platform{}, platforms...)
platforms = append(platforms, additionalPlatforms(i, node)...)
}
for _, p2 := range platforms {
m := matcher(p2)
if !m.Match(p) {
continue
}
if best == -1 {
best = i
bestPlatform = p2
continue
}
if matcher(p2).Less(p, bestPlatform) {
best = i
bestPlatform = p2
}
}
}
return best
}
func (r *nodeResolver) boot(ctx context.Context, idxs []int, pw progress.Writer) ([]*client.Client, error) {
clients := make([]*client.Client, len(idxs))
baseCtx := ctx
eg, ctx := errgroup.WithContext(ctx)
for i, idx := range idxs {
i, idx := i, idx
eg.Go(func() error {
c, err := r.clients.Do(ctx, fmt.Sprint(idx), func(ctx context.Context) (*client.Client, error) {
if r.nodes[idx].Driver == nil {
return nil, nil
}
return driver.Boot(ctx, baseCtx, r.nodes[idx].Driver, pw)
})
if err != nil {
return err
}
clients[i] = c
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
return clients, nil
}
func (r *nodeResolver) opts(ctx context.Context, idxs []int, pw progress.Writer) ([]gateway.BuildOpts, error) {
clients, err := r.boot(ctx, idxs, pw)
if err != nil {
return nil, err
}
bopts := make([]gateway.BuildOpts, len(clients))
eg, ctx := errgroup.WithContext(ctx)
for i, idxs := range idxs {
i, idx := i, idxs
c := clients[i]
if c == nil {
continue
}
eg.Go(func() error {
opt, err := r.opt.Do(ctx, fmt.Sprint(idx), func(ctx context.Context) (gateway.BuildOpts, error) {
opt := gateway.BuildOpts{}
_, err := c.Build(ctx, client.SolveOpt{
Internal: true,
}, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
opt = c.BuildOpts()
return nil, nil
}, nil)
return opt, err
})
if err != nil {
return err
}
bopts[i] = opt
return nil
})
}
if err := eg.Wait(); err != nil {
return nil, err
}
return bopts, nil
}
// recombineDriverPairs recombines resolved nodes that are on the same driver
// back together into a single node.
func recombineNodes(nodes []*resolvedNode) []*resolvedNode {
result := make([]*resolvedNode, 0, len(nodes))
lookup := map[int]int{}
for _, node := range nodes {
if idx, ok := lookup[node.driverIndex]; ok {
result[idx].platforms = append(result[idx].platforms, node.platforms...)
} else {
lookup[node.driverIndex] = len(result)
result = append(result, node)
}
}
return result
}

315
build/driver_test.go Normal file
View File

@@ -0,0 +1,315 @@
package build
import (
"context"
"sort"
"testing"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/builder"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/require"
)
func TestFindDriverSanity(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.DefaultSpec()},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.DefaultSpec()}, nil, platforms.OnlyStrict, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Equal(t, []specs.Platform{platforms.DefaultSpec()}, res[0].platforms)
}
func TestFindDriverEmpty(t *testing.T) {
r := makeTestResolver(nil)
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.DefaultSpec()}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Nil(t, res)
}
func TestFindDriverWeirdName(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/foobar")},
})
// find first platform
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/foobar")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestFindDriverUnknown(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSelectNodeSinglePlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
// find first platform
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/amd64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
// find second platform
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
// find an unknown platform, should match the first driver
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/s390x")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSelectNodeMultiPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/arm64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/amd64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 0, res[0].driverIndex)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, 1, res[0].driverIndex)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeNonStrict(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm64")},
})
// arm64 should match itself
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
// arm64 may support arm/v8
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
// arm64 may support arm/v7
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeNonStrictARM(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm64")},
"ccc": {platforms.MustParse("linux/arm/v8")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "ccc", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "ccc", res[0].Node().Builder)
}
func TestSelectNodeNonStrictLower(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm/v7")},
})
// v8 can't be built on v7 (so we should select the default)...
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v8")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.False(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
// ...but v6 can be built on v8
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v6")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodePreferStart(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/riscv64")},
"ccc": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/riscv64")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodePreferExact(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/arm/v8")},
"bbb": {platforms.MustParse("linux/arm/v7")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
}
func TestSelectNodeNoPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/foobar")},
"bbb": {platforms.DefaultSpec()},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Empty(t, res[0].platforms)
}
func TestSelectNodeAdditionalPlatforms(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/arm/v8")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "bbb", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{platforms.MustParse("linux/arm/v7")}, nil, platforms.Only, func(idx int, n builder.Node) []specs.Platform {
if n.Builder == "aaa" {
return []specs.Platform{platforms.MustParse("linux/arm/v7")}
}
return nil
})
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
}
func TestSplitNodeMultiPlatform(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/arm64")},
"bbb": {platforms.MustParse("linux/riscv64")},
})
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/arm64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 1)
require.Equal(t, "aaa", res[0].Node().Builder)
res, perfect, err = r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/riscv64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 2)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Equal(t, "bbb", res[1].Node().Builder)
}
func TestSplitNodeMultiPlatformNoUnify(t *testing.T) {
r := makeTestResolver(map[string][]specs.Platform{
"aaa": {platforms.MustParse("linux/amd64")},
"bbb": {platforms.MustParse("linux/amd64"), platforms.MustParse("linux/riscv64")},
})
// the "best" choice would be the node with both platforms, but we're using
// a naive algorithm that doesn't try to unify the platforms
res, perfect, err := r.resolve(context.TODO(), []specs.Platform{
platforms.MustParse("linux/amd64"),
platforms.MustParse("linux/riscv64"),
}, nil, platforms.Only, nil)
require.NoError(t, err)
require.True(t, perfect)
require.Len(t, res, 2)
require.Equal(t, "aaa", res[0].Node().Builder)
require.Equal(t, "bbb", res[1].Node().Builder)
}
func makeTestResolver(nodes map[string][]specs.Platform) *nodeResolver {
var ns []builder.Node
for name, platforms := range nodes {
ns = append(ns, builder.Node{
Builder: name,
Platforms: platforms,
})
}
sort.Slice(ns, func(i, j int) bool {
return ns[i].Builder < ns[j].Builder
})
return newDriverResolver(ns)
}

View File

@@ -9,16 +9,18 @@ import (
"strings"
"github.com/docker/buildx/util/gitutil"
"github.com/docker/buildx/util/osutil"
"github.com/moby/buildkit/client"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
const DockerfileLabel = "com.docker.image.source.entrypoint"
func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath string) (res map[string]string, _ error) {
res = make(map[string]string)
func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath string) (map[string]string, func(*client.SolveOpt), error) {
res := make(map[string]string)
if contextPath == "" {
return
return nil, nil, nil
}
setGitLabels := false
@@ -37,7 +39,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
}
if !setGitLabels && !setGitInfo {
return
return nil, nil, nil
}
// figure out in which directory the git command needs to run in
@@ -45,29 +47,40 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
if filepath.IsAbs(contextPath) {
wd = contextPath
} else {
cwd, _ := os.Getwd()
wd, _ = filepath.Abs(filepath.Join(cwd, contextPath))
wd, _ = filepath.Abs(filepath.Join(osutil.GetWd(), contextPath))
}
wd = osutil.SanitizePath(wd)
gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd))
if err != nil {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("git was not found in the system. Current commit information was not captured by the build")
if st, err1 := os.Stat(path.Join(wd, ".git")); err1 == nil && st.IsDir() {
return res, nil, errors.Wrap(err, "git was not found in the system")
}
return
return nil, nil, nil
}
if !gitc.IsInsideWorkTree() {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("failed to read current commit information with git rev-parse --is-inside-work-tree")
return res, nil, errors.New("failed to read current commit information with git rev-parse --is-inside-work-tree")
}
return res, nil
return nil, nil, nil
}
if sha, err := gitc.FullCommit(); err != nil {
return res, errors.Wrapf(err, "failed to get git commit")
root, err := gitc.RootDir()
if err != nil {
return res, nil, errors.Wrap(err, "failed to get git root dir")
}
if sha, err := gitc.FullCommit(); err != nil && !gitutil.IsUnknownRevision(err) {
return res, nil, errors.Wrap(err, "failed to get git commit")
} else if sha != "" {
if gitc.IsDirty() {
checkDirty := false
if v, ok := os.LookupEnv("BUILDX_GIT_CHECK_DIRTY"); ok {
if v, err := strconv.ParseBool(v); err == nil {
checkDirty = v
}
}
if checkDirty && gitc.IsDirty() {
sha += "-dirty"
}
if setGitLabels {
@@ -78,9 +91,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
}
}
if rurl, err := gitc.RemoteURL(); err != nil {
return res, errors.Wrapf(err, "failed to get git remote url")
} else if rurl != "" {
if rurl, err := gitc.RemoteURL(); err == nil && rurl != "" {
if setGitLabels {
res["label:"+specs.AnnotationSource] = rurl
}
@@ -89,23 +100,38 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
}
}
if setGitLabels {
if root, err := gitc.RootDir(); err != nil {
return res, errors.Wrapf(err, "failed to get git root dir")
} else if root != "" {
if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile")
}
if !filepath.IsAbs(dockerfilePath) {
cwd, _ := os.Getwd()
dockerfilePath = filepath.Join(cwd, dockerfilePath)
}
dockerfilePath, _ = filepath.Rel(root, dockerfilePath)
if !strings.HasPrefix(dockerfilePath, "..") {
res["label:"+DockerfileLabel] = dockerfilePath
}
if setGitLabels && root != "" {
if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile")
}
if !filepath.IsAbs(dockerfilePath) {
dockerfilePath = filepath.Join(osutil.GetWd(), dockerfilePath)
}
if r, err := filepath.Rel(root, dockerfilePath); err == nil && !strings.HasPrefix(r, "..") {
res["label:"+DockerfileLabel] = r
}
}
return
return res, func(so *client.SolveOpt) {
if !setGitInfo || root == "" {
return
}
for k, dir := range so.LocalDirs {
dir, err = filepath.EvalSymlinks(dir)
if err != nil {
continue
}
dir, err = filepath.Abs(dir)
if err != nil {
continue
}
if lp, err := osutil.GetLongPathName(dir); err == nil {
dir = lp
}
dir = osutil.SanitizePath(dir)
if r, err := filepath.Rel(root, dir); err == nil && !strings.HasPrefix(r, "..") {
so.FrontendAttrs["vcs:localdir:"+k] = r
}
}
}, nil
}

View File

@@ -9,6 +9,7 @@ import (
"testing"
"github.com/docker/buildx/util/gitutil"
"github.com/moby/buildkit/client"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -26,11 +27,11 @@ func setupTest(tb testing.TB) {
gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit")
gitutil.GitSetRemote(c, tb, "git@github.com:docker/buildx.git")
gitutil.GitSetRemote(c, tb, "origin", "git@github.com:docker/buildx.git")
}
func TestGetGitAttributesNotGitRepo(t *testing.T) {
_, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
_, _, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
assert.NoError(t, err)
}
@@ -38,14 +39,14 @@ func TestGetGitAttributesBadGitRepo(t *testing.T) {
tmp := t.TempDir()
require.NoError(t, os.MkdirAll(path.Join(tmp, ".git"), 0755))
_, err := getGitAttributes(context.Background(), tmp, "Dockerfile")
_, _, err := getGitAttributes(context.Background(), tmp, "Dockerfile")
assert.Error(t, err)
}
func TestGetGitAttributesNoContext(t *testing.T) {
setupTest(t)
gitattrs, err := getGitAttributes(context.Background(), "", "Dockerfile")
gitattrs, _, err := getGitAttributes(context.Background(), "", "Dockerfile")
assert.NoError(t, err)
assert.Empty(t, gitattrs)
}
@@ -114,7 +115,7 @@ func TestGetGitAttributes(t *testing.T) {
if tt.envGitInfo != "" {
t.Setenv("BUILDX_GIT_INFO", tt.envGitInfo)
}
gitattrs, err := getGitAttributes(context.Background(), ".", "Dockerfile")
gitattrs, _, err := getGitAttributes(context.Background(), ".", "Dockerfile")
require.NoError(t, err)
for _, e := range tt.expected {
assert.Contains(t, gitattrs, e)
@@ -131,6 +132,7 @@ func TestGetGitAttributes(t *testing.T) {
func TestGetGitAttributesDirty(t *testing.T) {
setupTest(t)
t.Setenv("BUILDX_GIT_CHECK_DIRTY", "true")
// make a change to test dirty flag
df := []byte("FROM alpine:edge\n")
@@ -138,7 +140,7 @@ func TestGetGitAttributesDirty(t *testing.T) {
require.NoError(t, os.WriteFile(filepath.Join("dir", "Dockerfile"), df, 0644))
t.Setenv("BUILDX_GIT_LABELS", "true")
gitattrs, _ := getGitAttributes(context.Background(), ".", "Dockerfile")
gitattrs, _, _ := getGitAttributes(context.Background(), ".", "Dockerfile")
assert.Equal(t, 5, len(gitattrs))
assert.Contains(t, gitattrs, "label:"+DockerfileLabel)
@@ -153,3 +155,59 @@ func TestGetGitAttributesDirty(t *testing.T) {
assert.Contains(t, gitattrs, "vcs:revision")
assert.True(t, strings.HasSuffix(gitattrs["vcs:revision"], "-dirty"))
}
func TestLocalDirs(t *testing.T) {
setupTest(t)
so := &client.SolveOpt{
FrontendAttrs: map[string]string{},
LocalDirs: map[string]string{
"context": ".",
"dockerfile": ".",
},
}
_, addVCSLocalDir, err := getGitAttributes(context.Background(), ".", "Dockerfile")
require.NoError(t, err)
require.NotNil(t, addVCSLocalDir)
addVCSLocalDir(so)
require.Contains(t, so.FrontendAttrs, "vcs:localdir:context")
assert.Equal(t, ".", so.FrontendAttrs["vcs:localdir:context"])
require.Contains(t, so.FrontendAttrs, "vcs:localdir:dockerfile")
assert.Equal(t, ".", so.FrontendAttrs["vcs:localdir:dockerfile"])
}
func TestLocalDirsSub(t *testing.T) {
gitutil.Mktmp(t)
c, err := gitutil.New()
require.NoError(t, err)
gitutil.GitInit(c, t)
df := []byte("FROM alpine:latest\n")
assert.NoError(t, os.MkdirAll("app", 0755))
assert.NoError(t, os.WriteFile("app/Dockerfile", df, 0644))
gitutil.GitAdd(c, t, "app/Dockerfile")
gitutil.GitCommit(c, t, "initial commit")
gitutil.GitSetRemote(c, t, "origin", "git@github.com:docker/buildx.git")
so := &client.SolveOpt{
FrontendAttrs: map[string]string{},
LocalDirs: map[string]string{
"context": ".",
"dockerfile": "app",
},
}
_, addVCSLocalDir, err := getGitAttributes(context.Background(), ".", "app/Dockerfile")
require.NoError(t, err)
require.NotNil(t, addVCSLocalDir)
addVCSLocalDir(so)
require.Contains(t, so.FrontendAttrs, "vcs:localdir:context")
assert.Equal(t, ".", so.FrontendAttrs["vcs:localdir:context"])
require.Contains(t, so.FrontendAttrs, "vcs:localdir:dockerfile")
assert.Equal(t, "app", so.FrontendAttrs["vcs:localdir:dockerfile"])
}

138
build/invoke.go Normal file
View File

@@ -0,0 +1,138 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"io"
"sync"
"sync/atomic"
"syscall"
controllerapi "github.com/docker/buildx/controller/pb"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type Container struct {
cancelOnce sync.Once
containerCancel func()
isUnavailable atomic.Bool
initStarted atomic.Bool
container gateway.Container
releaseCh chan struct{}
resultCtx *ResultHandle
}
func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig) (*Container, error) {
mainCtx := ctx
ctrCh := make(chan *Container)
errCh := make(chan error)
go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancel(ctx)
go func() {
<-mainCtx.Done()
cancel()
}()
containerCfg, err := resultCtx.getContainerConfig(ctx, c, cfg)
if err != nil {
return nil, err
}
containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil {
return nil, err
}
releaseCh := make(chan struct{})
container := &Container{
containerCancel: containerCancel,
container: bkContainer,
releaseCh: releaseCh,
resultCtx: resultCtx,
}
doneCh := make(chan struct{})
defer close(doneCh)
resultCtx.registerCleanup(func() {
container.Cancel()
<-doneCh
})
ctrCh <- container
<-container.releaseCh
return nil, bkContainer.Release(ctx)
})
if err != nil {
errCh <- err
}
}()
select {
case ctr := <-ctrCh:
return ctr, nil
case err := <-errCh:
return nil, err
case <-mainCtx.Done():
return nil, mainCtx.Err()
}
}
func (c *Container) Cancel() {
c.markUnavailable()
c.cancelOnce.Do(func() {
if c.containerCancel != nil {
c.containerCancel()
}
close(c.releaseCh)
})
}
func (c *Container) IsUnavailable() bool {
return c.isUnavailable.Load()
}
func (c *Container) markUnavailable() {
c.isUnavailable.Store(true)
}
func (c *Container) Exec(ctx context.Context, cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if isInit := c.initStarted.CompareAndSwap(false, true); isInit {
defer func() {
// container can't be used after init exits
c.markUnavailable()
}()
}
err := exec(ctx, c.resultCtx, cfg, c.container, stdin, stdout, stderr)
if err != nil {
// Container becomes unavailable if one of the processes fails in it.
c.markUnavailable()
}
return err
}
func exec(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig, ctr gateway.Container, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
processCfg, err := resultCtx.getProcessConfig(cfg, stdin, stdout, stderr)
if err != nil {
return err
}
proc, err := ctr.Start(ctx, processCfg)
if err != nil {
return errors.Errorf("failed to start container: %v", err)
}
doneCh := make(chan struct{})
defer close(doneCh)
go func() {
select {
case <-ctx.Done():
if err := proc.Signal(ctx, syscall.SIGKILL); err != nil {
logrus.Warnf("failed to kill process: %v", err)
}
case <-doneCh:
}
}()
return proc.Wait()
}

43
build/localstate.go Normal file
View File

@@ -0,0 +1,43 @@
package build
import (
"path/filepath"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/moby/buildkit/client"
)
func saveLocalState(so *client.SolveOpt, target string, opts Options, node builder.Node, configDir string) error {
var err error
if so.Ref == "" {
return nil
}
lp := opts.Inputs.ContextPath
dp := opts.Inputs.DockerfilePath
if lp != "" || dp != "" {
if lp != "" {
lp, err = filepath.Abs(lp)
if err != nil {
return err
}
}
if dp != "" {
dp, err = filepath.Abs(dp)
if err != nil {
return err
}
}
l, err := localstate.New(configDir)
if err != nil {
return err
}
return l.SaveRef(node.Builder, node.Name, so.Ref, localstate.State{
Target: target,
LocalPath: lp,
DockerfilePath: dp,
GroupRef: opts.GroupRef,
})
}
return nil
}

495
build/result.go Normal file
View File

@@ -0,0 +1,495 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"encoding/json"
"io"
"sync"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/solver/result"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
// NewResultHandle makes a call to client.Build, additionally returning a
// opaque ResultHandle alongside the standard response and error.
//
// This ResultHandle can be used to execute additional build steps in the same
// context as the build occurred, which can allow easy debugging of build
// failures and successes.
//
// If the returned ResultHandle is not nil, the caller must call Done() on it.
func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt, product string, buildFunc gateway.BuildFunc, ch chan *client.SolveStatus) (*ResultHandle, *client.SolveResponse, error) {
// Create a new context to wrap the original, and cancel it when the
// caller-provided context is cancelled.
//
// We derive the context from the background context so that we can forbid
// cancellation of the build request after <-done is closed (which we do
// before returning the ResultHandle).
baseCtx := ctx
ctx, cancel := context.WithCancelCause(context.Background())
done := make(chan struct{})
go func() {
select {
case <-baseCtx.Done():
cancel(baseCtx.Err())
case <-done:
// Once done is closed, we've recorded a ResultHandle, so we
// shouldn't allow cancelling the underlying build request anymore.
}
}()
// Create a new channel to forward status messages to the original.
//
// We do this so that we can discard status messages after the main portion
// of the build is complete. This is necessary for the solve error case,
// where the original gateway is kept open until the ResultHandle is
// closed - we don't want progress messages from operations in that
// ResultHandle to display after this function exits.
//
// Additionally, callers should wait for the progress channel to be closed.
// If we keep the session open and never close the progress channel, the
// caller will likely hang.
baseCh := ch
ch = make(chan *client.SolveStatus)
go func() {
for {
s, ok := <-ch
if !ok {
return
}
select {
case <-baseCh:
// base channel is closed, discard status messages
default:
baseCh <- s
}
}
}()
defer close(baseCh)
var resp *client.SolveResponse
var respErr error
var respHandle *ResultHandle
go func() {
defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result
var err error
resp, err = cc.Build(ctx, opt, product, func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
var err error
res, err = buildFunc(ctx, c)
if res != nil && err == nil {
// Force evaluation of the build result (otherwise, we likely
// won't get a solve error)
def, err2 := getDefinition(ctx, res)
if err2 != nil {
return nil, err2
}
res, err = evalDefinition(ctx, c, def)
}
if err != nil {
// Scenario 1: we failed to evaluate a node somewhere in the
// build graph.
//
// In this case, we construct a ResultHandle from this
// original Build session, and return it alongside the original
// build error. We then need to keep the gateway session open
// until the caller explicitly closes the ResultHandle.
var se *errdefs.SolveError
if errors.As(err, &se) {
respHandle = &ResultHandle{
done: make(chan struct{}),
solveErr: se,
gwClient: c,
gwCtx: ctx,
}
respErr = err // return original error to preserve stacktrace
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
}
}
return res, err
}, ch)
if respHandle != nil {
return
}
if err != nil {
// Something unexpected failed during the build, we didn't succeed,
// but we also didn't make it far enough to create a ResultHandle.
respErr = err
close(done)
return
}
// Scenario 2: we successfully built the image with no errors.
//
// In this case, the original gateway session has now been closed
// since the Build has been completed. So, we need to create a new
// gateway session to populate the ResultHandle. To do this, we
// need to re-evaluate the target result, in this new session. This
// should be instantaneous since the result should be cached.
def, err := getDefinition(ctx, res)
if err != nil {
respErr = err
close(done)
return
}
// NOTE: ideally this second connection should be lazily opened
opt := opt
opt.Ref = ""
opt.Exports = nil
opt.CacheExports = nil
opt.Internal = true
_, respErr = cc.Build(ctx, opt, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
res, err := evalDefinition(ctx, c, def)
if err != nil {
// This should probably not happen, since we've previously
// successfully evaluated the same result with no issues.
return nil, errors.Wrap(err, "inconsistent solve result")
}
respHandle = &ResultHandle{
done: make(chan struct{}),
res: res,
gwClient: c,
gwCtx: ctx,
}
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
return nil, ctx.Err()
}, nil)
if respHandle != nil {
return
}
close(done)
}()
// Block until the other thread signals that it's completed the build.
select {
case <-done:
case <-baseCtx.Done():
if respErr == nil {
respErr = baseCtx.Err()
}
}
return respHandle, resp, respErr
}
// getDefinition converts a gateway result into a collection of definitions for
// each ref in the result.
func getDefinition(ctx context.Context, res *gateway.Result) (*result.Result[*pb.Definition], error) {
return result.ConvertResult(res, func(ref gateway.Reference) (*pb.Definition, error) {
st, err := ref.ToState()
if err != nil {
return nil, err
}
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
}
return def.ToPB(), nil
})
}
// evalDefinition performs the reverse of getDefinition, converting a
// collection of definitions into a gateway result.
func evalDefinition(ctx context.Context, c gateway.Client, defs *result.Result[*pb.Definition]) (*gateway.Result, error) {
// force evaluation of all targets in parallel
results := make(map[*pb.Definition]*gateway.Result)
resultsMu := sync.Mutex{}
eg, egCtx := errgroup.WithContext(ctx)
defs.EachRef(func(def *pb.Definition) error {
eg.Go(func() error {
res, err := c.Solve(egCtx, gateway.SolveRequest{
Evaluate: true,
Definition: def,
})
if err != nil {
return err
}
resultsMu.Lock()
results[def] = res
resultsMu.Unlock()
return nil
})
return nil
})
if err := eg.Wait(); err != nil {
return nil, err
}
res, _ := result.ConvertResult(defs, func(def *pb.Definition) (gateway.Reference, error) {
if res, ok := results[def]; ok {
return res.Ref, nil
}
return nil, nil
})
return res, nil
}
// ResultHandle is a build result with the client that built it.
type ResultHandle struct {
res *gateway.Result
solveErr *errdefs.SolveError
done chan struct{}
doneOnce sync.Once
gwClient gateway.Client
gwCtx context.Context
cleanups []func()
cleanupsMu sync.Mutex
}
func (r *ResultHandle) Done() {
r.doneOnce.Do(func() {
r.cleanupsMu.Lock()
cleanups := r.cleanups
r.cleanups = nil
r.cleanupsMu.Unlock()
for _, f := range cleanups {
f()
}
close(r.done)
<-r.gwCtx.Done()
})
}
func (r *ResultHandle) registerCleanup(f func()) {
r.cleanupsMu.Lock()
r.cleanups = append(r.cleanups, f)
r.cleanupsMu.Unlock()
}
func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
_, err = buildFunc(r.gwCtx, r.gwClient)
return err
}
func (r *ResultHandle) getContainerConfig(ctx context.Context, c gateway.Client, cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(ctx, r.res, c, *cfg)
if err != nil {
return containerCfg, err
}
containerCfg = *ccfg
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available")
}
containerCfg = *ccfg
}
return containerCfg, nil
}
func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) (_ gateway.StartRequest, err error) {
processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err
}
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err
}
}
return processCfg, nil
}
func containerConfigFromResult(ctx context.Context, res *gateway.Result, c gateway.Client, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
}
ps, err := exptypes.ParsePlatforms(res.Metadata)
if err != nil {
return nil, err
}
ref, ok := res.FindRef(ps.Platforms[0].ID)
if !ok {
return nil, errors.Errorf("no reference found")
}
return &gateway.NewContainerRequest{
Mounts: []gateway.Mount{
{
Dest: "/",
MountType: pb.MountType_BIND,
Ref: ref,
},
},
}, nil
}
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image
if len(imgData) > 0 {
img = &specs.Image{}
if err := json.Unmarshal(imgData, img); err != nil {
return err
}
}
user := ""
if !cfg.NoUser {
user = cfg.User
} else if img != nil {
user = img.Config.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else if img != nil {
cwd = img.Config.WorkingDir
}
env := []string{}
if img != nil {
env = append(env, img.Config.Env...)
}
env = append(env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
} else if img != nil {
args = append(args, img.Config.Entrypoint...)
}
if !cfg.NoCmd {
args = append(args, cfg.Cmd...)
} else if img != nil {
args = append(args, img.Config.Cmd...)
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr)
if err != nil {
return nil, err
}
var mounts []gateway.Mount
for i, mnt := range exec.Mounts {
rid := solveErr.Solve.MountIDs[i]
if cfg.Initial {
rid = solveErr.Solve.InputIDs[i]
}
mounts = append(mounts, gateway.Mount{
Selector: mnt.Selector,
Dest: mnt.Dest,
ResultID: rid,
Readonly: mnt.Readonly,
MountType: mnt.MountType,
CacheOpt: mnt.CacheOpt,
SecretOpt: mnt.SecretOpt,
SSHOpt: mnt.SSHOpt,
})
}
return &gateway.NewContainerRequest{
Mounts: mounts,
NetMode: exec.Network,
}, nil
}
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr)
if err != nil {
return err
}
meta := exec.Meta
user := ""
if !cfg.NoUser {
user = cfg.User
} else {
user = meta.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else {
cwd = meta.Cwd
}
env := append(meta.Env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
}
if len(args) == 0 {
args = meta.Args
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func execOpFromError(solveErr *errdefs.SolveError) (*pb.ExecOp, error) {
if solveErr == nil {
return nil, errors.Errorf("no error is available")
}
switch op := solveErr.Solve.Op.GetOp().(type) {
case *pb.Op_Exec:
return op.Exec, nil
default:
return nil, errors.Errorf("invoke: unsupported error type")
}
// TODO: support other ops
}
func newStartRequest(stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) gateway.StartRequest {
return gateway.StartRequest{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
}
}

View File

@@ -13,7 +13,7 @@ import (
"github.com/pkg/errors"
)
func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url string, pw progress.Writer) (string, error) {
func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil {
return "", err
@@ -21,7 +21,7 @@ func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url strin
var out string
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Internal: true}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := llb.HTTP(url, llb.Filename("Dockerfile"), llb.WithCustomNamef("[internal] load %s", url)).Marshal(ctx)
if err != nil {
return nil, err

View File

@@ -3,11 +3,14 @@ package build
import (
"archive/tar"
"bytes"
"context"
"net"
"os"
"strings"
"github.com/docker/buildx/driver"
"github.com/docker/cli/opts"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
"github.com/pkg/errors"
)
@@ -20,9 +23,14 @@ const (
mobyHostGatewayName = "host-gateway"
)
func isLocalDir(c string) bool {
st, err := os.Stat(c)
return err == nil && st.IsDir()
func IsRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}
func isArchive(header []byte) bool {
@@ -45,18 +53,34 @@ func isArchive(header []byte) bool {
}
// toBuildkitExtraHosts converts hosts from docker key:value format to buildkit's csv format
func toBuildkitExtraHosts(inp []string, mobyDriver bool) (string, error) {
func toBuildkitExtraHosts(ctx context.Context, inp []string, nodeDriver *driver.DriverHandle) (string, error) {
if len(inp) == 0 {
return "", nil
}
hosts := make([]string, 0, len(inp))
for _, h := range inp {
host, ip, ok := strings.Cut(h, ":")
host, ip, ok := strings.Cut(h, "=")
if !ok {
host, ip, ok = strings.Cut(h, ":")
}
if !ok || host == "" || ip == "" {
return "", errors.Errorf("invalid host %s", h)
}
// Skip IP address validation for "host-gateway" string with moby driver
if !mobyDriver || ip != mobyHostGatewayName {
// If the IP Address is a "host-gateway", replace this value with the
// IP address provided by the worker's label.
if ip == mobyHostGatewayName {
hgip, err := nodeDriver.HostGatewayIP(ctx)
if err != nil {
return "", errors.Wrap(err, "unable to derive the IP value for host-gateway")
}
ip = hgip.String()
} else {
// If the address is enclosed in square brackets, extract it (for IPv6, but
// permit it for IPv4 as well; we don't know the address family here, but it's
// unambiguous).
if len(ip) > 2 && ip[0] == '[' && ip[len(ip)-1] == ']' {
ip = ip[1 : len(ip)-1]
}
if net.ParseIP(ip) == nil {
return "", errors.Errorf("invalid host %s", h)
}

148
build/utils_test.go Normal file
View File

@@ -0,0 +1,148 @@
package build
import (
"context"
"strings"
"testing"
"github.com/stretchr/testify/require"
)
func TestToBuildkitExtraHosts(t *testing.T) {
tests := []struct {
doc string
input []string
expectedOut string // Expect output==input if not set.
expectedErr string // Expect success if not set.
}{
{
doc: "IPv4, colon sep",
input: []string{`myhost:192.168.0.1`},
expectedOut: `myhost=192.168.0.1`,
},
{
doc: "IPv4, eq sep",
input: []string{`myhost=192.168.0.1`},
},
{
doc: "Weird but permitted, IPv4 with brackets",
input: []string{`myhost=[192.168.0.1]`},
expectedOut: `myhost=192.168.0.1`,
},
{
doc: "Host and domain",
input: []string{`host.and.domain.invalid:10.0.2.1`},
expectedOut: `host.and.domain.invalid=10.0.2.1`,
},
{
doc: "IPv6, colon sep",
input: []string{`anipv6host:2003:ab34:e::1`},
expectedOut: `anipv6host=2003:ab34:e::1`,
},
{
doc: "IPv6, colon sep, brackets",
input: []string{`anipv6host:[2003:ab34:e::1]`},
expectedOut: `anipv6host=2003:ab34:e::1`,
},
{
doc: "IPv6, eq sep, brackets",
input: []string{`anipv6host=[2003:ab34:e::1]`},
expectedOut: `anipv6host=2003:ab34:e::1`,
},
{
doc: "IPv6 localhost, colon sep",
input: []string{`ipv6local:::1`},
expectedOut: `ipv6local=::1`,
},
{
doc: "IPv6 localhost, eq sep",
input: []string{`ipv6local=::1`},
},
{
doc: "IPv6 localhost, eq sep, brackets",
input: []string{`ipv6local=[::1]`},
expectedOut: `ipv6local=::1`,
},
{
doc: "IPv6 localhost, non-canonical, colon sep",
input: []string{`ipv6local:0:0:0:0:0:0:0:1`},
expectedOut: `ipv6local=0:0:0:0:0:0:0:1`,
},
{
doc: "IPv6 localhost, non-canonical, eq sep",
input: []string{`ipv6local=0:0:0:0:0:0:0:1`},
},
{
doc: "IPv6 localhost, non-canonical, eq sep, brackets",
input: []string{`ipv6local=[0:0:0:0:0:0:0:1]`},
expectedOut: `ipv6local=0:0:0:0:0:0:0:1`,
},
{
doc: "Bad address, colon sep",
input: []string{`myhost:192.notanipaddress.1`},
expectedErr: `invalid IP address in add-host: "192.notanipaddress.1"`,
},
{
doc: "Bad address, eq sep",
input: []string{`myhost=192.notanipaddress.1`},
expectedErr: `invalid IP address in add-host: "192.notanipaddress.1"`,
},
{
doc: "No sep",
input: []string{`thathost-nosemicolon10.0.0.1`},
expectedErr: `bad format for add-host: "thathost-nosemicolon10.0.0.1"`,
},
{
doc: "Bad IPv6",
input: []string{`anipv6host:::::1`},
expectedErr: `invalid IP address in add-host: "::::1"`,
},
{
doc: "Bad IPv6, trailing colons",
input: []string{`ipv6local:::0::`},
expectedErr: `invalid IP address in add-host: "::0::"`,
},
{
doc: "Bad IPv6, missing close bracket",
input: []string{`ipv6addr=[::1`},
expectedErr: `invalid IP address in add-host: "[::1"`,
},
{
doc: "Bad IPv6, missing open bracket",
input: []string{`ipv6addr=::1]`},
expectedErr: `invalid IP address in add-host: "::1]"`,
},
{
doc: "Missing address, colon sep",
input: []string{`myhost.invalid:`},
expectedErr: `invalid IP address in add-host: ""`,
},
{
doc: "Missing address, eq sep",
input: []string{`myhost.invalid=`},
expectedErr: `invalid IP address in add-host: ""`,
},
{
doc: "No input",
input: []string{``},
expectedErr: `bad format for add-host: ""`,
},
}
for _, tc := range tests {
tc := tc
if tc.expectedOut == "" {
tc.expectedOut = strings.Join(tc.input, ",")
}
t.Run(tc.doc, func(t *testing.T) {
actualOut, actualErr := toBuildkitExtraHosts(context.TODO(), tc.input, nil)
if tc.expectedErr == "" {
require.Equal(t, tc.expectedOut, actualOut)
require.Nil(t, actualErr)
} else {
require.Zero(t, actualOut)
require.Error(t, actualErr, tc.expectedErr)
}
})
}
}

View File

@@ -2,18 +2,31 @@ package builder
import (
"context"
"encoding/csv"
"encoding/json"
"net/url"
"os"
"sort"
"strings"
"sync"
"time"
"github.com/docker/buildx/driver"
k8sutil "github.com/docker/buildx/driver/kubernetes/util"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
dopts "github.com/docker/cli/opts"
"github.com/google/shlex"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/pflag"
"golang.org/x/sync/errgroup"
)
@@ -108,7 +121,7 @@ func New(dockerCli command.Cli, opts ...Option) (_ *Builder, err error) {
// Validate validates builder context
func (b *Builder) Validate() error {
if b.NodeGroup.DockerContext {
if b.NodeGroup != nil && b.NodeGroup.DockerContext {
list, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return err
@@ -157,13 +170,14 @@ func (b *Builder) Boot(ctx context.Context) (bool, error) {
return false, nil
}
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, os.Stderr, progress.PrinterModeAuto)
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, progressui.AutoMode)
if err != nil {
return false, err
}
baseCtx := ctx
eg, _ := errgroup.WithContext(ctx)
errCh := make(chan error, len(toBoot))
for _, idx := range toBoot {
func(idx int) {
eg.Go(func() error {
@@ -171,6 +185,7 @@ func (b *Builder) Boot(ctx context.Context) (bool, error) {
_, err := driver.Boot(ctx, baseCtx, b.nodes[idx].Driver, pw)
if err != nil {
b.nodes[idx].Err = err
errCh <- err
}
return nil
})
@@ -178,11 +193,15 @@ func (b *Builder) Boot(ctx context.Context) (bool, error) {
}
err = eg.Wait()
close(errCh)
err1 := printer.Wait()
if err == nil {
err = err1
}
if err == nil && len(errCh) == len(toBoot) {
return false, <-errCh
}
return true, err
}
@@ -207,7 +226,7 @@ type driverFactory struct {
}
// Factory returns the driver factory.
func (b *Builder) Factory(ctx context.Context) (_ driver.Factory, err error) {
func (b *Builder) Factory(ctx context.Context, dialMeta map[string][]string) (_ driver.Factory, err error) {
b.driverFactory.once.Do(func() {
if b.Driver != "" {
b.driverFactory.Factory, err = driver.GetFactory(b.Driver, true)
@@ -230,7 +249,7 @@ func (b *Builder) Factory(ctx context.Context) (_ driver.Factory, err error) {
if _, err = dockerapi.Ping(ctx); err != nil {
return
}
b.driverFactory.Factory, err = driver.GetDefaultFactory(ctx, ep, dockerapi, false)
b.driverFactory.Factory, err = driver.GetDefaultFactory(ctx, ep, dockerapi, false, dialMeta)
if err != nil {
return
}
@@ -240,6 +259,28 @@ func (b *Builder) Factory(ctx context.Context) (_ driver.Factory, err error) {
return b.driverFactory.Factory, err
}
func (b *Builder) MarshalJSON() ([]byte, error) {
var berr string
if b.err != nil {
berr = strings.TrimSpace(b.err.Error())
}
return json.Marshal(struct {
Name string
Driver string
LastActivity time.Time `json:",omitempty"`
Dynamic bool
Nodes []Node
Err string `json:",omitempty"`
}{
Name: b.Name,
Driver: b.Driver,
LastActivity: b.LastActivity,
Dynamic: b.Dynamic,
Nodes: b.nodes,
Err: berr,
})
}
// GetBuilders returns all builders
func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
storeng, err := txn.List()
@@ -290,3 +331,347 @@ func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
return builders, nil
}
type CreateOpts struct {
Name string
Driver string
NodeName string
Platforms []string
BuildkitdFlags string
BuildkitdConfigFile string
DriverOpts []string
Use bool
Endpoint string
Append bool
}
func Create(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts CreateOpts) (*Builder, error) {
var err error
if opts.Name == "default" {
return nil, errors.Errorf("default is a reserved name and cannot be used to identify builder instance")
} else if opts.Append && opts.Name == "" {
return nil, errors.Errorf("append requires a builder name")
}
name := opts.Name
if name == "" {
name, err = store.GenerateName(txn)
if err != nil {
return nil, err
}
}
if !opts.Append {
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, c := range contexts {
if c.Name == name {
return nil, errors.Errorf("instance name %q already exists as context builder", name)
}
}
}
ng, err := txn.NodeGroupByName(name)
if err != nil {
if os.IsNotExist(errors.Cause(err)) {
if opts.Append && opts.Name != "" {
return nil, errors.Errorf("failed to find instance %q for append", opts.Name)
}
} else {
return nil, err
}
}
buildkitHost := os.Getenv("BUILDKIT_HOST")
driverName := opts.Driver
if driverName == "" {
if ng != nil {
driverName = ng.Driver
} else if opts.Endpoint == "" && buildkitHost != "" {
driverName = "remote"
} else {
f, err := driver.GetDefaultFactory(ctx, opts.Endpoint, dockerCli.Client(), true, nil)
if err != nil {
return nil, err
}
if f == nil {
return nil, errors.Errorf("no valid drivers found")
}
driverName = f.Name()
}
}
if ng != nil {
if opts.NodeName == "" && !opts.Append {
return nil, errors.Errorf("existing instance for %q but no append mode, specify the node name to make changes for existing instances", name)
}
if driverName != ng.Driver {
return nil, errors.Errorf("existing instance for %q but has mismatched driver %q", name, ng.Driver)
}
}
if _, err := driver.GetFactory(driverName, true); err != nil {
return nil, err
}
ngOriginal := ng
if ngOriginal != nil {
ngOriginal = ngOriginal.Copy()
}
if ng == nil {
ng = &store.NodeGroup{
Name: name,
Driver: driverName,
}
}
driverOpts, err := csvToMap(opts.DriverOpts)
if err != nil {
return nil, err
}
buildkitdFlags, err := parseBuildkitdFlags(opts.BuildkitdFlags, driverName, driverOpts)
if err != nil {
return nil, err
}
var ep string
var setEp bool
switch {
case driverName == "kubernetes":
if opts.Endpoint != "" {
return nil, errors.Errorf("kubernetes driver does not support endpoint args %q", opts.Endpoint)
}
// generate node name if not provided to avoid duplicated endpoint
// error: https://github.com/docker/setup-buildx-action/issues/215
nodeName := opts.NodeName
if nodeName == "" {
nodeName, err = k8sutil.GenerateNodeName(name, txn)
if err != nil {
return nil, err
}
}
// naming endpoint to make append works
ep = (&url.URL{
Scheme: driverName,
Path: "/" + name,
RawQuery: (&url.Values{
"deployment": {nodeName},
"kubeconfig": {os.Getenv("KUBECONFIG")},
}).Encode(),
}).String()
setEp = false
case driverName == "remote":
if opts.Endpoint != "" {
ep = opts.Endpoint
} else if buildkitHost != "" {
ep = buildkitHost
} else {
return nil, errors.Errorf("no remote endpoint provided")
}
ep, err = validateBuildkitEndpoint(ep)
if err != nil {
return nil, err
}
setEp = true
case opts.Endpoint != "":
ep, err = validateEndpoint(dockerCli, opts.Endpoint)
if err != nil {
return nil, err
}
setEp = true
default:
if dockerCli.CurrentContext() == "default" && dockerCli.DockerEndpoint().TLSData != nil {
return nil, errors.Errorf("could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with context set to <context-name>")
}
ep, err = dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return nil, err
}
setEp = false
}
buildkitdConfigFile := opts.BuildkitdConfigFile
if buildkitdConfigFile == "" {
// if buildkit daemon config is not provided, check if the default one
// is available and use it
if f, ok := confutil.DefaultConfigFile(dockerCli); ok {
buildkitdConfigFile = f
}
}
if err := ng.Update(opts.NodeName, ep, opts.Platforms, setEp, opts.Append, buildkitdFlags, buildkitdConfigFile, driverOpts); err != nil {
return nil, err
}
if err := txn.Save(ng); err != nil {
return nil, err
}
b, err := New(dockerCli,
WithName(ng.Name),
WithStore(txn),
WithSkippedValidation(),
)
if err != nil {
return nil, err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, WithData())
if err != nil {
return nil, err
}
for _, node := range nodes {
if err := node.Err; err != nil {
err := errors.Errorf("failed to initialize builder %s (%s): %s", ng.Name, node.Name, err)
var err2 error
if ngOriginal == nil {
err2 = txn.Remove(ng.Name)
} else {
err2 = txn.Save(ngOriginal)
}
if err2 != nil {
return nil, errors.Errorf("could not rollback to previous state: %s", err2)
}
return nil, err
}
}
if opts.Use && ep != "" {
current, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return nil, err
}
if err := txn.SetCurrent(current, ng.Name, false, false); err != nil {
return nil, err
}
}
return b, nil
}
type LeaveOpts struct {
Name string
NodeName string
}
func Leave(ctx context.Context, txn *store.Txn, dockerCli command.Cli, opts LeaveOpts) error {
if opts.Name == "" {
return errors.Errorf("leave requires instance name")
}
if opts.NodeName == "" {
return errors.Errorf("leave requires node name")
}
ng, err := txn.NodeGroupByName(opts.Name)
if err != nil {
if os.IsNotExist(errors.Cause(err)) {
return errors.Errorf("failed to find instance %q for leave", opts.Name)
}
return err
}
if err := ng.Leave(opts.NodeName); err != nil {
return err
}
ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
if err := ls.RemoveBuilderNode(ng.Name, opts.NodeName); err != nil {
return err
}
return txn.Save(ng)
}
func csvToMap(in []string) (map[string]string, error) {
if len(in) == 0 {
return nil, nil
}
m := make(map[string]string, len(in))
for _, s := range in {
csvReader := csv.NewReader(strings.NewReader(s))
fields, err := csvReader.Read()
if err != nil {
return nil, err
}
for _, v := range fields {
p := strings.SplitN(v, "=", 2)
if len(p) != 2 {
return nil, errors.Errorf("invalid value %q, expecting k=v", v)
}
m[p[0]] = p[1]
}
}
return m, nil
}
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
dem, err := dockerutil.GetDockerEndpoint(dockerCli, ep)
if err == nil && dem != nil {
if ep == "default" {
return dem.Host, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// validateBuildkitEndpoint validates that endpoint is a valid buildkit host
func validateBuildkitEndpoint(ep string) (string, error) {
if err := remoteutil.IsValidEndpoint(ep); err != nil {
return "", err
}
return ep, nil
}
// parseBuildkitdFlags parses buildkit flags
func parseBuildkitdFlags(inp string, driver string, driverOpts map[string]string) (res []string, err error) {
if inp != "" {
res, err = shlex.Split(inp)
if err != nil {
return nil, errors.Wrap(err, "failed to parse buildkit flags")
}
}
var allowInsecureEntitlements []string
flags := pflag.NewFlagSet("buildkitd", pflag.ContinueOnError)
flags.Usage = func() {}
flags.StringArrayVar(&allowInsecureEntitlements, "allow-insecure-entitlement", nil, "")
_ = flags.Parse(res)
var hasNetworkHostEntitlement bool
for _, e := range allowInsecureEntitlements {
if e == "network.host" {
hasNetworkHostEntitlement = true
break
}
}
if v, ok := driverOpts["network"]; ok && v == "host" && !hasNetworkHostEntitlement && driver == "docker-container" {
// always set network.host entitlement if user has set network=host
res = append(res, "--allow-insecure-entitlement=network.host")
} else if len(allowInsecureEntitlements) == 0 && (driver == "kubernetes" || driver == "docker-container") {
// set network.host entitlement if user does not provide any as
// network is isolated for container drivers.
res = append(res, "--allow-insecure-entitlement=network.host")
}
return res, nil
}

139
builder/builder_test.go Normal file
View File

@@ -0,0 +1,139 @@
package builder
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCsvToMap(t *testing.T) {
d := []string{
"\"tolerations=key=foo,value=bar;key=foo2,value=bar2\",replicas=1",
"namespace=default",
}
r, err := csvToMap(d)
require.NoError(t, err)
require.Contains(t, r, "tolerations")
require.Equal(t, r["tolerations"], "key=foo,value=bar;key=foo2,value=bar2")
require.Contains(t, r, "replicas")
require.Equal(t, r["replicas"], "1")
require.Contains(t, r, "namespace")
require.Equal(t, r["namespace"], "default")
}
func TestParseBuildkitdFlags(t *testing.T) {
testCases := []struct {
name string
flags string
driver string
driverOpts map[string]string
expected []string
wantErr bool
}{
{
"docker-container no flags",
"",
"docker-container",
nil,
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
{
"kubernetes no flags",
"",
"kubernetes",
nil,
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
{
"remote no flags",
"",
"remote",
nil,
nil,
false,
},
{
"docker-container with insecure flag",
"--allow-insecure-entitlement=security.insecure",
"docker-container",
nil,
[]string{
"--allow-insecure-entitlement=security.insecure",
},
false,
},
{
"docker-container with insecure and host flag",
"--allow-insecure-entitlement=network.host --allow-insecure-entitlement=security.insecure",
"docker-container",
nil,
[]string{
"--allow-insecure-entitlement=network.host",
"--allow-insecure-entitlement=security.insecure",
},
false,
},
{
"docker-container with network host opt",
"",
"docker-container",
map[string]string{"network": "host"},
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
{
"docker-container with host flag and network host opt",
"--allow-insecure-entitlement=network.host",
"docker-container",
map[string]string{"network": "host"},
[]string{
"--allow-insecure-entitlement=network.host",
},
false,
},
{
"docker-container with insecure, host flag and network host opt",
"--allow-insecure-entitlement=network.host --allow-insecure-entitlement=security.insecure",
"docker-container",
map[string]string{"network": "host"},
[]string{
"--allow-insecure-entitlement=network.host",
"--allow-insecure-entitlement=security.insecure",
},
false,
},
{
"error parsing flags",
"foo'",
"docker-container",
nil,
nil,
true,
},
}
for _, tt := range testCases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
flags, err := parseBuildkitdFlags(tt.flags, tt.driver, tt.driverOpts)
if tt.wantErr {
require.Error(t, err)
return
}
require.NoError(t, err)
assert.Equal(t, tt.expected, flags)
})
}
}

View File

@@ -2,7 +2,11 @@ package builder
import (
"context"
"encoding/json"
"sort"
"strings"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/driver"
ctxkube "github.com/docker/buildx/driver/kubernetes/context"
"github.com/docker/buildx/store"
@@ -10,6 +14,7 @@ import (
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/platformutil"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -20,13 +25,19 @@ import (
type Node struct {
store.Node
Driver driver.Driver
Builder string
Driver *driver.DriverHandle
DriverInfo *driver.Info
Platforms []ocispecs.Platform
ImageOpt imagetools.Opt
ProxyConfig map[string]string
Version string
Err error
// worker settings
IDs []string
Platforms []ocispecs.Platform
GCPolicy []client.PruneInfo
Labels map[string]string
}
// Nodes returns nodes for this builder.
@@ -34,9 +45,35 @@ func (b *Builder) Nodes() []Node {
return b.nodes
}
type LoadNodesOption func(*loadNodesOptions)
type loadNodesOptions struct {
data bool
dialMeta map[string][]string
}
func WithData() LoadNodesOption {
return func(o *loadNodesOptions) {
o.data = true
}
}
func WithDialMeta(dialMeta map[string][]string) LoadNodesOption {
return func(o *loadNodesOptions) {
o.dialMeta = dialMeta
}
}
// LoadNodes loads and returns nodes for this builder.
// TODO: this should be a method on a Node object and lazy load data for each driver.
func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err error) {
func (b *Builder) LoadNodes(ctx context.Context, opts ...LoadNodesOption) (_ []Node, err error) {
lno := loadNodesOptions{
data: false,
}
for _, opt := range opts {
opt(&lno)
}
eg, _ := errgroup.WithContext(ctx)
b.nodes = make([]Node, len(b.NodeGroup.Nodes))
@@ -46,7 +83,7 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
}
}()
factory, err := b.Factory(ctx)
factory, err := b.Factory(ctx, lno.dialMeta)
if err != nil {
return nil, err
}
@@ -62,6 +99,8 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
node := Node{
Node: n,
ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli),
Platforms: n.Platforms,
Builder: b.Name,
}
defer func() {
b.nodes[i] = node
@@ -76,12 +115,12 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
contextStore := b.opts.dockerCli.ContextStore()
var kcc driver.KubeClientConfig
kcc, err = ctxkube.ConfigFromContext(n.Endpoint, contextStore)
kcc, err = ctxkube.ConfigFromEndpoint(n.Endpoint, contextStore)
if err != nil {
// err is returned if n.Endpoint is non-context name like "unix:///var/run/docker.sock".
// try again with name="default".
// FIXME(@AkihiroSuda): n should retain real context name.
kcc, err = ctxkube.ConfigFromContext("default", contextStore)
kcc, err = ctxkube.ConfigFromEndpoint("default", contextStore)
if err != nil {
logrus.Error(err)
}
@@ -103,7 +142,7 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, factory, n.Endpoint, dockerapi, imageopt.Auth, kcc, n.Flags, n.Files, n.DriverOpts, n.Platforms, b.opts.contextPathHash)
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, factory, n.Endpoint, dockerapi, imageopt.Auth, kcc, n.BuildkitdFlags, n.Files, n.DriverOpts, n.Platforms, b.opts.contextPathHash, lno.dialMeta)
if err != nil {
node.Err = err
return nil
@@ -111,7 +150,7 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
node.Driver = d
node.ImageOpt = imageopt
if withData {
if lno.data {
if err := node.loadData(ctx); err != nil {
node.Err = err
}
@@ -126,7 +165,7 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
}
// TODO: This should be done in the routine loading driver data
if withData {
if lno.data {
kubernetesDriverCount := 0
for _, d := range b.nodes {
if d.DriverInfo != nil && len(d.DriverInfo.DynamicNodes) > 0 {
@@ -163,6 +202,51 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
return b.nodes, nil
}
func (n *Node) MarshalJSON() ([]byte, error) {
var status string
if n.DriverInfo != nil {
status = n.DriverInfo.Status.String()
}
var nerr string
if n.Err != nil {
status = "error"
nerr = strings.TrimSpace(n.Err.Error())
}
var pp []string
for _, p := range n.Platforms {
pp = append(pp, platforms.Format(p))
}
return json.Marshal(struct {
Name string
Endpoint string
BuildkitdFlags []string `json:"Flags,omitempty"`
DriverOpts map[string]string `json:",omitempty"`
Files map[string][]byte `json:",omitempty"`
Status string `json:",omitempty"`
ProxyConfig map[string]string `json:",omitempty"`
Version string `json:",omitempty"`
Err string `json:",omitempty"`
IDs []string `json:",omitempty"`
Platforms []string `json:",omitempty"`
GCPolicy []client.PruneInfo `json:",omitempty"`
Labels map[string]string `json:",omitempty"`
}{
Name: n.Name,
Endpoint: n.Endpoint,
BuildkitdFlags: n.BuildkitdFlags,
DriverOpts: n.DriverOpts,
Files: n.Files,
Status: status,
ProxyConfig: n.ProxyConfig,
Version: n.Version,
Err: nerr,
IDs: n.IDs,
Platforms: pp,
GCPolicy: n.GCPolicy,
Labels: n.Labels,
})
}
func (n *Node) loadData(ctx context.Context) error {
if n.Driver == nil {
return nil
@@ -181,9 +265,15 @@ func (n *Node) loadData(ctx context.Context) error {
if err != nil {
return errors.Wrap(err, "listing workers")
}
for _, w := range workers {
for idx, w := range workers {
n.IDs = append(n.IDs, w.ID)
n.Platforms = append(n.Platforms, w.Platforms...)
if idx == 0 {
n.GCPolicy = w.GCPolicy
n.Labels = w.Labels
}
}
sort.Strings(n.IDs)
n.Platforms = platformutil.Dedupe(n.Platforms)
inf, err := driverClient.Info(ctx)
if err != nil {

View File

@@ -4,8 +4,8 @@ import (
"fmt"
"os"
"github.com/containerd/containerd/pkg/seed"
"github.com/docker/buildx/commands"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/manager"
@@ -16,10 +16,10 @@ import (
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack"
_ "k8s.io/client-go/plugin/pkg/client/auth/azure"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
@@ -28,7 +28,9 @@ import (
)
func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision)
}
@@ -85,6 +87,9 @@ func main() {
} else {
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
}
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
ebr.Print(cmd.Err())
}
os.Exit(1)
}

View File

@@ -4,33 +4,44 @@ import (
"context"
"encoding/json"
"fmt"
"io"
"os"
"strings"
"github.com/containerd/console"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/bake"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/util/tracing"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type bakeOptions struct {
files []string
overrides []string
printOnly bool
commonOptions
files []string
overrides []string
printOnly bool
sbom string
provenance string
builder string
metadataFile string
exportPush bool
exportLoad bool
}
func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error) {
ctx := appcontext.Context()
func runBake(ctx context.Context, dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
ctx, end, err := tracing.TraceCurrentCommand(ctx, "bake")
if err != nil {
return err
@@ -43,11 +54,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
cmdContext := "cwd://"
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
url = targets[0]
targets = targets[1:]
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
cmdContext = targets[0]
targets = targets[1:]
}
@@ -68,11 +79,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
} else if in.exportLoad {
overrides = append(overrides, "*.output=type=docker")
}
if in.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *in.noCache))
if cFlags.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *cFlags.noCache))
}
if in.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *in.pull))
if cFlags.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *cFlags.pull))
}
if in.sbom != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("sbom", in.sbom)))
@@ -84,23 +95,9 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
if err != nil {
return err
}
defer func() {
if printer != nil {
err1 := printer.Wait()
if err == nil {
err = err1
}
}
}()
var nodes []builder.Node
var files []bake.File
var inp *bake.Input
var progressConsoleDesc, progressTextDesc string
// instance only needed for reading remote bake files or building
if url != "" || !in.printOnly {
@@ -114,24 +111,51 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err = b.LoadNodes(ctx, false)
nodes, err = b.LoadNodes(ctx)
if err != nil {
return err
}
progressConsoleDesc = fmt.Sprintf("%s:%s", b.Driver, b.Name)
progressTextDesc = fmt.Sprintf("building with %q instance using %s driver", b.Name, b.Driver)
}
if url != "" {
files, inp, err = bake.ReadRemoteFiles(ctx, nodes, url, in.files, printer)
} else {
files, err = bake.ReadLocalFiles(in.files)
var term bool
if _, err := console.ConsoleFromFile(os.Stderr); err == nil {
term = true
}
progressMode := progressui.DisplayMode(cFlags.progress)
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressMode,
progress.WithDesc(progressTextDesc, progressConsoleDesc),
)
if err != nil {
return err
}
defer func() {
if printer != nil {
err1 := printer.Wait()
if err == nil {
err = err1
}
if err == nil && progressMode != progressui.QuietMode && progressMode != progressui.RawJSONMode {
desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term)
}
}
}()
files, inp, err := readBakeFiles(ctx, nodes, url, in.files, dockerCli.In(), printer)
if err != nil {
return err
}
if len(files) == 0 {
return errors.New("couldn't find a bake definition")
}
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, map[string]string{
// don't forget to update documentation if you add a new
// built-in variable: docs/manuals/bake/file-definition.md#built-in-variables
// built-in variable: docs/bake-reference.md#built-in-variables
"BAKE_CMD_CONTEXT": cmdContext,
"BAKE_LOCAL_PLATFORM": platforms.DefaultString(),
})
@@ -139,20 +163,35 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return err
}
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
// TODO: extract env var parsing to a method easily usable by library consumers
for _, t := range tgts {
if _, ok := t.Args["SOURCE_DATE_EPOCH"]; ok {
continue
}
if t.Args == nil {
t.Args = map[string]*string{}
}
t.Args["SOURCE_DATE_EPOCH"] = &v
}
}
// this function can update target context string from the input so call before printOnly check
bo, err := bake.TargetsToBuildOpt(tgts, inp)
if err != nil {
return err
}
def := struct {
Group map[string]*bake.Group `json:"group,omitempty"`
Target map[string]*bake.Target `json:"target"`
}{
Group: grps,
Target: tgts,
}
if in.printOnly {
dt, err := json.MarshalIndent(struct {
Group map[string]*bake.Group `json:"group,omitempty"`
Target map[string]*bake.Target `json:"target"`
}{
grps,
tgts,
}, "", " ")
dt, err := json.MarshalIndent(def, "", " ")
if err != nil {
return err
}
@@ -165,6 +204,28 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return nil
}
// local state group
groupRef := identity.NewID()
var refs []string
for k, b := range bo {
b.Ref = identity.NewID()
b.GroupRef = groupRef
refs = append(refs, b.Ref)
bo[k] = b
}
dt, err := json.Marshal(def)
if err != nil {
return err
}
if err := saveLocalStateGroup(dockerCli, groupRef, localstate.StateGroup{
Definition: dt,
Targets: targets,
Inputs: overrides,
Refs: refs,
}); err != nil {
return err
}
resp, err := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), printer)
if err != nil {
return wrapBuildError(err, true)
@@ -185,6 +246,7 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
var options bakeOptions
var cFlags commonFlags
cmd := &cobra.Command{
Use: "bake [OPTIONS] [TARGET...]",
@@ -193,14 +255,17 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
// reset to nil to avoid override is unset
if !cmd.Flags().Lookup("no-cache").Changed {
options.noCache = nil
cFlags.noCache = nil
}
if !cmd.Flags().Lookup("pull").Changed {
options.pull = nil
cFlags.pull = nil
}
options.commonOptions.builder = rootOpts.builder
return runBake(dockerCli, args, options)
options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function.
return runBake(cmd.Context(), dockerCli, args, options, cFlags)
},
ValidArgsFunction: completion.BakeTargets(options.files),
}
flags := cmd.Flags()
@@ -213,7 +278,58 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
flags.StringArrayVar(&options.overrides, "set", nil, `Override target value (e.g., "targetpattern.key=value")`)
commonBuildFlags(&options.commonOptions, flags)
commonBuildFlags(&cFlags, flags)
return cmd
}
func saveLocalStateGroup(dockerCli command.Cli, ref string, lsg localstate.StateGroup) error {
l, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
return l.SaveGroup(ref, lsg)
}
func readBakeFiles(ctx context.Context, nodes []builder.Node, url string, names []string, stdin io.Reader, pw progress.Writer) (files []bake.File, inp *bake.Input, err error) {
var lnames []string // local
var rnames []string // remote
var anames []string // both
for _, v := range names {
if strings.HasPrefix(v, "cwd://") {
tname := strings.TrimPrefix(v, "cwd://")
lnames = append(lnames, tname)
anames = append(anames, tname)
} else {
rnames = append(rnames, v)
anames = append(anames, v)
}
}
if url != "" {
var rfiles []bake.File
rfiles, inp, err = bake.ReadRemoteFiles(ctx, nodes, url, rnames, pw)
if err != nil {
return nil, nil, err
}
files = append(files, rfiles...)
}
if len(lnames) > 0 || url == "" {
var lfiles []bake.File
progress.Wrap("[internal] load local bake definitions", pw.Write, func(sub progress.SubLogger) error {
if url != "" {
lfiles, err = bake.ReadLocalFiles(lnames, stdin, sub)
} else {
lfiles, err = bake.ReadLocalFiles(anames, stdin, sub)
}
return nil
})
if err != nil {
return nil, nil, err
}
files = append(files, lfiles...)
}
return
}

File diff suppressed because it is too large Load Diff

View File

@@ -3,283 +3,72 @@ package commands
import (
"bytes"
"context"
"encoding/csv"
"fmt"
"net/url"
"os"
"strings"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
dopts "github.com/docker/cli/opts"
"github.com/google/shlex"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
type createOptions struct {
name string
driver string
nodeName string
platform []string
actionAppend bool
actionLeave bool
use bool
flags string
configFile string
driverOpts []string
bootstrap bool
name string
driver string
nodeName string
platform []string
actionAppend bool
actionLeave bool
use bool
driverOpts []string
buildkitdFlags string
buildkitdConfigFile string
bootstrap bool
// upgrade bool // perform upgrade of the driver
}
func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
ctx := appcontext.Context()
if in.name == "default" {
return errors.Errorf("default is a reserved name and cannot be used to identify builder instance")
}
if in.actionLeave {
if in.name == "" {
return errors.Errorf("leave requires instance name")
}
if in.nodeName == "" {
return errors.Errorf("leave requires node name but --node not set")
}
}
if in.actionAppend {
if in.name == "" {
logrus.Warnf("append used without name, creating a new instance instead")
}
}
func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, args []string) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
// Ensure the file lock gets released no matter what happens.
defer release()
name := in.name
if name == "" {
name, err = store.GenerateName(txn)
if err != nil {
return err
}
}
if !in.actionLeave && !in.actionAppend {
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return err
}
for _, c := range contexts {
if c.Name == name {
logrus.Warnf("instance name %q already exists as context builder", name)
break
}
}
}
ng, err := txn.NodeGroupByName(name)
if err != nil {
if os.IsNotExist(errors.Cause(err)) {
if in.actionAppend && in.name != "" {
logrus.Warnf("failed to find %q for append, creating a new instance instead", in.name)
}
if in.actionLeave {
return errors.Errorf("failed to find instance %q for leave", in.name)
}
} else {
return err
}
}
buildkitHost := os.Getenv("BUILDKIT_HOST")
driverName := in.driver
if driverName == "" {
if ng != nil {
driverName = ng.Driver
} else if len(args) == 0 && buildkitHost != "" {
driverName = "remote"
} else {
var arg string
if len(args) > 0 {
arg = args[0]
}
f, err := driver.GetDefaultFactory(ctx, arg, dockerCli.Client(), true)
if err != nil {
return err
}
if f == nil {
return errors.Errorf("no valid drivers found")
}
driverName = f.Name()
}
}
if ng != nil {
if in.nodeName == "" && !in.actionAppend {
return errors.Errorf("existing instance for %q but no append mode, specify --node to make changes for existing instances", name)
}
if driverName != ng.Driver {
return errors.Errorf("existing instance for %q but has mismatched driver %q", name, ng.Driver)
}
}
if _, err := driver.GetFactory(driverName, true); err != nil {
return err
}
ngOriginal := ng
if ngOriginal != nil {
ngOriginal = ngOriginal.Copy()
}
if ng == nil {
ng = &store.NodeGroup{
Name: name,
Driver: driverName,
}
}
var flags []string
if in.flags != "" {
flags, err = shlex.Split(in.flags)
if err != nil {
return errors.Wrap(err, "failed to parse buildkit flags")
}
if in.actionLeave {
return builder.Leave(ctx, txn, dockerCli, builder.LeaveOpts{
Name: in.name,
NodeName: in.nodeName,
})
}
var ep string
var setEp bool
if in.actionLeave {
if err := ng.Leave(in.nodeName); err != nil {
return err
}
} else {
switch {
case driverName == "kubernetes":
if len(args) > 0 {
logrus.Warnf("kubernetes driver does not support endpoint args %q", args[0])
}
// naming endpoint to make --append works
ep = (&url.URL{
Scheme: driverName,
Path: "/" + in.name,
RawQuery: (&url.Values{
"deployment": {in.nodeName},
"kubeconfig": {os.Getenv("KUBECONFIG")},
}).Encode(),
}).String()
setEp = false
case driverName == "remote":
if len(args) > 0 {
ep = args[0]
} else if buildkitHost != "" {
ep = buildkitHost
} else {
return errors.Errorf("no remote endpoint provided")
}
ep, err = validateBuildkitEndpoint(ep)
if err != nil {
return err
}
setEp = true
case len(args) > 0:
ep, err = validateEndpoint(dockerCli, args[0])
if err != nil {
return err
}
setEp = true
default:
if dockerCli.CurrentContext() == "default" && dockerCli.DockerEndpoint().TLSData != nil {
return errors.Errorf("could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with `docker buildx create <context-name>`")
}
ep, err = dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
setEp = false
}
m, err := csvToMap(in.driverOpts)
if err != nil {
return err
}
if in.configFile == "" {
// if buildkit config is not provided, check if the default one is
// available and use it
if f, ok := confutil.DefaultConfigFile(dockerCli); ok {
logrus.Warnf("Using default BuildKit config in %s", f)
in.configFile = f
}
}
if err := ng.Update(in.nodeName, ep, in.platform, setEp, in.actionAppend, flags, in.configFile, m); err != nil {
return err
}
if len(args) > 0 {
ep = args[0]
}
if err := txn.Save(ng); err != nil {
return err
}
b, err := builder.New(dockerCli,
builder.WithName(ng.Name),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
b, err := builder.Create(ctx, txn, dockerCli, builder.CreateOpts{
Name: in.name,
Driver: in.driver,
NodeName: in.nodeName,
Platforms: in.platform,
DriverOpts: in.driverOpts,
BuildkitdFlags: in.buildkitdFlags,
BuildkitdConfigFile: in.buildkitdConfigFile,
Use: in.use,
Endpoint: ep,
Append: in.actionAppend,
})
if err != nil {
return err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, true)
if err != nil {
return err
}
for _, node := range nodes {
if err := node.Err; err != nil {
err := errors.Errorf("failed to initialize builder %s (%s): %s", ng.Name, node.Name, err)
var err2 error
if ngOriginal == nil {
err2 = txn.Remove(ng.Name)
} else {
err2 = txn.Save(ngOriginal)
}
if err2 != nil {
logrus.Warnf("Could not rollback to previous state: %s", err2)
}
return err
}
}
if in.use && ep != "" {
current, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
if err := txn.SetCurrent(current, ng.Name, false, false); err != nil {
return err
}
}
// The store is no longer used from this point.
// Release it so we aren't holding the file lock during the boot.
release()
if in.bootstrap {
if _, err = b.Boot(ctx); err != nil {
@@ -287,7 +76,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
fmt.Printf("%s\n", ng.Name)
fmt.Printf("%s\n", b.Name)
return nil
}
@@ -307,8 +96,9 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
Short: "Create a new builder instance",
Args: cli.RequiresMaxArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return runCreate(dockerCli, options, args)
return runCreate(cmd.Context(), dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
@@ -316,12 +106,16 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
flags.StringVar(&options.name, "name", "", "Builder instance name")
flags.StringVar(&options.driver, "driver", "", fmt.Sprintf("Driver to use (available: %s)", drivers.String()))
flags.StringVar(&options.nodeName, "node", "", "Create/modify node with given name")
flags.StringVar(&options.flags, "buildkitd-flags", "", "Flags for buildkitd daemon")
flags.StringVar(&options.configFile, "config", "", "BuildKit config file")
flags.StringArrayVar(&options.platform, "platform", []string{}, "Fixed platforms for current node")
flags.StringArrayVar(&options.driverOpts, "driver-opt", []string{}, "Options for the driver")
flags.BoolVar(&options.bootstrap, "bootstrap", false, "Boot builder after creation")
flags.StringVar(&options.buildkitdFlags, "buildkitd-flags", "", "BuildKit daemon flags")
// we allow for both "--config" and "--buildkitd-config", although the latter is the recommended way to avoid ambiguity.
flags.StringVar(&options.buildkitdConfigFile, "buildkitd-config", "", "BuildKit daemon config file")
flags.StringVar(&options.buildkitdConfigFile, "config", "", "BuildKit daemon config file")
flags.MarkHidden("config")
flags.BoolVar(&options.bootstrap, "bootstrap", false, "Boot builder after creation")
flags.BoolVar(&options.actionAppend, "append", false, "Append a node to builder instead of changing it")
flags.BoolVar(&options.actionLeave, "leave", false, "Remove a node from builder instead of changing it")
flags.BoolVar(&options.use, "use", false, "Set the current builder instance")
@@ -331,49 +125,3 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
return cmd
}
func csvToMap(in []string) (map[string]string, error) {
if len(in) == 0 {
return nil, nil
}
m := make(map[string]string, len(in))
for _, s := range in {
csvReader := csv.NewReader(strings.NewReader(s))
fields, err := csvReader.Read()
if err != nil {
return nil, err
}
for _, v := range fields {
p := strings.SplitN(v, "=", 2)
if len(p) != 2 {
return nil, errors.Errorf("invalid value %q, expecting k=v", v)
}
m[p[0]] = p[1]
}
}
return m, nil
}
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
dem, err := dockerutil.GetDockerEndpoint(dockerCli, ep)
if err == nil && dem != nil {
if ep == "default" {
return dem.Host, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// validateBuildkitEndpoint validates that endpoint is a valid buildkit host
func validateBuildkitEndpoint(ep string) (string, error) {
if err := remoteutil.IsValidEndpoint(ep); err != nil {
return "", err
}
return ep, nil
}

View File

@@ -1,26 +0,0 @@
package commands
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestCsvToMap(t *testing.T) {
d := []string{
"\"tolerations=key=foo,value=bar;key=foo2,value=bar2\",replicas=1",
"namespace=default",
}
r, err := csvToMap(d)
require.NoError(t, err)
require.Contains(t, r, "tolerations")
require.Equal(t, r["tolerations"], "key=foo,value=bar;key=foo2,value=bar2")
require.Contains(t, r, "replicas")
require.Equal(t, r["replicas"], "1")
require.Contains(t, r, "namespace")
require.Equal(t, r["namespace"], "default")
}

92
commands/debug/root.go Normal file
View File

@@ -0,0 +1,92 @@
package debug
import (
"context"
"os"
"runtime"
"github.com/containerd/console"
"github.com/docker/buildx/controller"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/monitor"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// DebugConfig is a user-specified configuration for the debugger.
type DebugConfig struct {
// InvokeFlag is a flag to configure the launched debugger and the commaned executed on the debugger.
InvokeFlag string
// OnFlag is a flag to configure the timing of launching the debugger.
OnFlag string
}
// DebuggableCmd is a command that supports debugger with recognizing the user-specified DebugConfig.
type DebuggableCmd interface {
// NewDebugger returns the new *cobra.Command with support for the debugger with recognizing DebugConfig.
NewDebugger(*DebugConfig) *cobra.Command
}
func RootCmd(dockerCli command.Cli, children ...DebuggableCmd) *cobra.Command {
var controlOptions control.ControlOptions
var progressMode string
var options DebugConfig
cmd := &cobra.Command{
Use: "debug",
Short: "Start debugger",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, progressui.DisplayMode(progressMode))
if err != nil {
return err
}
ctx := context.TODO()
c, err := controller.NewController(ctx, controlOptions, dockerCli, printer)
if err != nil {
return err
}
defer func() {
if err := c.Close(); err != nil {
logrus.Warnf("failed to close server connection %v", err)
}
}()
con := console.Current()
if err := con.SetRaw(); err != nil {
return errors.Errorf("failed to configure terminal: %v", err)
}
_, err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true,
}, c, dockerCli.In(), os.Stdout, os.Stderr, printer)
con.Reset()
return err
},
}
cobrautil.MarkCommandExperimental(cmd)
flags := cmd.Flags()
flags.StringVar(&options.InvokeFlag, "invoke", "", "Launch a monitor with executing specified command")
flags.StringVar(&options.OnFlag, "on", "error", "When to launch the monitor ([always, error])")
flags.StringVar(&controlOptions.Root, "root", "", "Specify root directory of server to connect for the monitor")
flags.BoolVar(&controlOptions.Detach, "detach", runtime.GOOS == "linux", "Detach buildx server for the monitor (supported only on linux)")
flags.StringVar(&controlOptions.ServerConfig, "server-config", "", "Specify buildx server config file for the monitor (used only when launching new server)")
flags.StringVar(&progressMode, "progress", "auto", `Set type of progress output ("auto", "plain", "tty") for the monitor. Use plain to show container output`)
cobrautil.MarkFlagsExperimental(flags, "invoke", "on", "root", "detach", "server-config")
for _, c := range children {
cmd.AddCommand(c.NewDebugger(&options))
}
return cmd
}

132
commands/dial_stdio.go Normal file
View File

@@ -0,0 +1,132 @@
package commands
import (
"io"
"net"
"os"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/util/progress/progressui"
v1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type stdioOptions struct {
builder string
platform string
progress string
}
func runDialStdio(dockerCli command.Cli, opts stdioOptions) error {
ctx := appcontext.Context()
contextPathHash, _ := os.Getwd()
b, err := builder.New(dockerCli,
builder.WithName(opts.builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
printer, err := progress.NewPrinter(ctx, os.Stderr, progressui.DisplayMode(opts.progress), progress.WithPhase("dial-stdio"), progress.WithDesc("builder: "+b.Name, "builder:"+b.Name))
if err != nil {
return err
}
var p *v1.Platform
if opts.platform != "" {
pp, err := platforms.Parse(opts.platform)
if err != nil {
return errors.Wrapf(err, "invalid platform %q", opts.platform)
}
p = &pp
}
defer printer.Wait()
return progress.Wrap("Proxying to builder", printer.Write, func(sub progress.SubLogger) error {
var conn net.Conn
err := sub.Wrap("Dialing builder", func() error {
conn, err = build.Dial(ctx, nodes, printer, p)
if err != nil {
return err
}
return nil
})
if err != nil {
return err
}
defer conn.Close()
go func() {
<-ctx.Done()
closeWrite(conn)
}()
var eg errgroup.Group
eg.Go(func() error {
_, err := io.Copy(conn, os.Stdin)
closeWrite(conn)
return err
})
eg.Go(func() error {
_, err := io.Copy(os.Stdout, conn)
closeRead(conn)
return err
})
return eg.Wait()
})
}
func closeRead(conn net.Conn) error {
if c, ok := conn.(interface{ CloseRead() error }); ok {
return c.CloseRead()
}
return conn.Close()
}
func closeWrite(conn net.Conn) error {
if c, ok := conn.(interface{ CloseWrite() error }); ok {
return c.CloseWrite()
}
return conn.Close()
}
func dialStdioCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
opts := stdioOptions{}
cmd := &cobra.Command{
Use: "dial-stdio",
Short: "Proxy current stdio streams to builder instance",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
opts.builder = rootOpts.builder
return runDialStdio(dockerCli, opts)
},
}
flags := cmd.Flags()
cmd.Flags()
flags.StringVar(&opts.platform, "platform", os.Getenv("DOCKER_DEFAULT_PLATFORM"), "Target platform: this is used for node selection")
flags.StringVar(&opts.progress, "progress", "quiet", "Set type of progress output (auto, plain, tty).")
return cmd
}

View File

@@ -1,6 +1,7 @@
package commands
import (
"context"
"fmt"
"io"
"os"
@@ -9,12 +10,12 @@ import (
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/appcontext"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
@@ -25,9 +26,7 @@ type duOptions struct {
verbose bool
}
func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
ctx := appcontext.Context()
func runDiskUsage(ctx context.Context, dockerCli command.Cli, opts duOptions) error {
pi, err := toBuildkitPruneInfo(opts.filter.Value())
if err != nil {
return err
@@ -38,7 +37,7 @@ func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
return err
}
nodes, err := b.LoadNodes(ctx, false)
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
@@ -113,8 +112,9 @@ func duCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
Args: cli.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = rootOpts.builder
return runDiskUsage(dockerCli, options)
return runDiskUsage(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -7,12 +7,13 @@ import (
"os"
"strings"
"github.com/distribution/reference"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/distribution/reference"
"github.com/moby/buildkit/util/appcontext"
"github.com/moby/buildkit/util/progress/progressui"
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -24,12 +25,13 @@ type createOptions struct {
builder string
files []string
tags []string
annotations []string
dryrun bool
actionAppend bool
progress string
}
func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
func runCreate(ctx context.Context, dockerCli command.Cli, in createOptions, args []string) error {
if len(args) == 0 && len(in.files) == 0 {
return errors.Errorf("no sources specified")
}
@@ -110,8 +112,6 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
ctx := appcontext.Context()
b, err := builder.New(dockerCli, builder.WithName(in.builder))
if err != nil {
return err
@@ -153,7 +153,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
dt, desc, err := r.Combine(ctx, srcs)
dt, desc, err := r.Combine(ctx, srcs, in.annotations)
if err != nil {
return err
}
@@ -168,7 +168,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
printer, err := progress.NewPrinter(ctx2, os.Stderr, progressui.DisplayMode(in.progress))
if err != nil {
return err
}
@@ -271,8 +271,9 @@ func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
Short: "Create a new image based on source images",
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *opts.Builder
return runCreate(dockerCli, options, args)
return runCreate(cmd.Context(), dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
@@ -281,6 +282,7 @@ func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
flags.BoolVar(&options.dryrun, "dry-run", false, "Show final image instead of pushing")
flags.BoolVar(&options.actionAppend, "append", false, "Append to existing manifest")
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "plain", "tty"). Use plain to show container output`)
flags.StringArrayVarP(&options.annotations, "annotation", "", []string{}, "Add annotation to the image")
return cmd
}

View File

@@ -1,12 +1,14 @@
package commands
import (
"context"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
@@ -17,9 +19,7 @@ type inspectOptions struct {
raw bool
}
func runInspect(dockerCli command.Cli, in inspectOptions, name string) error {
ctx := appcontext.Context()
func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions, name string) error {
if in.format != "" && in.raw {
return errors.Errorf("format and raw cannot be used together")
}
@@ -50,8 +50,9 @@ func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
Args: cli.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *rootOpts.Builder
return runInspect(dockerCli, options, args[0])
return runInspect(cmd.Context(), dockerCli, options, args[0])
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -1,6 +1,7 @@
package commands
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
@@ -11,8 +12,9 @@ type RootOptions struct {
func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "imagetools",
Short: "Commands to work on images in registry",
Use: "imagetools",
Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(

View File

@@ -4,15 +4,19 @@ import (
"context"
"fmt"
"os"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/docker/cli/cli/debug"
"github.com/docker/go-units"
"github.com/spf13/cobra"
)
@@ -21,9 +25,7 @@ type inspectOptions struct {
builder string
}
func runInspect(dockerCli command.Cli, in inspectOptions) error {
ctx := appcontext.Context()
func runInspect(ctx context.Context, dockerCli command.Cli, in inspectOptions) error {
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
@@ -35,7 +37,7 @@ func runInspect(dockerCli command.Cli, in inspectOptions) error {
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, true)
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if in.bootstrap {
var ok bool
ok, err = b.Boot(ctx)
@@ -43,7 +45,7 @@ func runInspect(dockerCli command.Cli, in inspectOptions) error {
return err
}
if ok {
nodes, err = b.LoadNodes(timeoutCtx, true)
nodes, err = b.LoadNodes(timeoutCtx, builder.WithData())
}
}
@@ -82,13 +84,48 @@ func runInspect(dockerCli command.Cli, in inspectOptions) error {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else {
fmt.Fprintf(w, "Status:\t%s\n", nodes[i].DriverInfo.Status)
if len(n.Flags) > 0 {
fmt.Fprintf(w, "Flags:\t%s\n", strings.Join(n.Flags, " "))
if len(n.BuildkitdFlags) > 0 {
fmt.Fprintf(w, "BuildKit daemon flags:\t%s\n", strings.Join(n.BuildkitdFlags, " "))
}
if nodes[i].Version != "" {
fmt.Fprintf(w, "Buildkit:\t%s\n", nodes[i].Version)
fmt.Fprintf(w, "BuildKit version:\t%s\n", nodes[i].Version)
}
platforms := platformutil.FormatInGroups(n.Node.Platforms, n.Platforms)
if len(platforms) > 0 {
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platforms, ", "))
}
if debug.IsEnabled() {
fmt.Fprintf(w, "Features:\n")
features := nodes[i].Driver.Features(ctx)
featKeys := make([]string, 0, len(features))
for k := range features {
featKeys = append(featKeys, string(k))
}
sort.Strings(featKeys)
for _, k := range featKeys {
fmt.Fprintf(w, "\t%s:\t%t\n", k, features[driver.Feature(k)])
}
}
if len(nodes[i].Labels) > 0 {
fmt.Fprintf(w, "Labels:\n")
for _, k := range sortedKeys(nodes[i].Labels) {
v := nodes[i].Labels[k]
fmt.Fprintf(w, "\t%s:\t%s\n", k, v)
}
}
for ri, rule := range nodes[i].GCPolicy {
fmt.Fprintf(w, "GC Policy rule#%d:\n", ri)
fmt.Fprintf(w, "\tAll:\t%v\n", rule.All)
if len(rule.Filter) > 0 {
fmt.Fprintf(w, "\tFilters:\t%s\n", strings.Join(rule.Filter, " "))
}
if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
}
if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
}
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
}
}
}
@@ -110,8 +147,9 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
if len(args) > 0 {
options.builder = args[0]
}
return runInspect(dockerCli, options)
return runInspect(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
@@ -119,3 +157,14 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
return cmd
}
func sortedKeys(m map[string]string) []string {
s := make([]string, len(m))
i := 0
for k := range m {
s[i] = k
i++
}
sort.Strings(s)
return s
}

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -46,7 +47,8 @@ func installCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runInstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -2,29 +2,43 @@ package commands
import (
"context"
"encoding/json"
"fmt"
"io"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/docker/cli/cli/command/formatter"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
const (
lsNameNodeHeader = "NAME/NODE"
lsDriverEndpointHeader = "DRIVER/ENDPOINT"
lsStatusHeader = "STATUS"
lsLastActivityHeader = "LAST ACTIVITY"
lsBuildkitHeader = "BUILDKIT"
lsPlatformsHeader = "PLATFORMS"
lsIndent = ` \_ `
lsDefaultTableFormat = "table {{.Name}}\t{{.DriverEndpoint}}\t{{.Status}}\t{{.Buildkit}}\t{{.Platforms}}"
)
type lsOptions struct {
format string
}
func runLs(dockerCli command.Cli, in lsOptions) error {
ctx := appcontext.Context()
func runLs(ctx context.Context, dockerCli command.Cli, in lsOptions) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
@@ -48,7 +62,7 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
for _, b := range builders {
func(b *builder.Builder) {
eg.Go(func() error {
_, _ = b.LoadNodes(timeoutCtx, true)
_, _ = b.LoadNodes(timeoutCtx, builder.WithData())
return nil
})
}(b)
@@ -58,22 +72,9 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
return err
}
w := tabwriter.NewWriter(dockerCli.Out(), 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "NAME/NODE\tDRIVER/ENDPOINT\tSTATUS\tBUILDKIT\tPLATFORMS\n")
printErr := false
for _, b := range builders {
if current.Name == b.Name {
b.Name += " *"
}
if ok := printBuilder(w, b); !ok {
printErr = true
}
}
w.Flush()
if printErr {
if hasErrors, err := lsPrint(dockerCli, current, builders, in.format); err != nil {
return err
} else if hasErrors {
_, _ = fmt.Fprintf(dockerCli.Err(), "\n")
for _, b := range builders {
if b.Err() != nil {
@@ -91,31 +92,6 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
return nil
}
func printBuilder(w io.Writer, b *builder.Builder) (ok bool) {
ok = true
var err string
if b.Err() != nil {
ok = false
err = "error"
}
fmt.Fprintf(w, "%s\t%s\t%s\t\t\n", b.Name, b.Driver, err)
if b.Err() == nil {
for _, n := range b.Nodes() {
var status string
if n.DriverInfo != nil {
status = n.DriverInfo.Status.String()
}
if n.Err != nil {
ok = false
fmt.Fprintf(w, " %s\t%s\t%s\t\t\n", n.Name, n.Endpoint, "error")
} else {
fmt.Fprintf(w, " %s\t%s\t%s\t%s\t%s\n", n.Name, n.Endpoint, status, n.Version, strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
}
}
}
return
}
func lsCmd(dockerCli command.Cli) *cobra.Command {
var options lsOptions
@@ -124,12 +100,175 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
Short: "List builder instances",
Args: cli.ExactArgs(0),
RunE: func(cmd *cobra.Command, args []string) error {
return runLs(dockerCli, options)
return runLs(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.format, "format", formatter.TableFormatKey, "Format the output")
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}
func lsPrint(dockerCli command.Cli, current *store.NodeGroup, builders []*builder.Builder, format string) (hasErrors bool, _ error) {
if format == formatter.TableFormatKey {
format = lsDefaultTableFormat
}
ctx := formatter.Context{
Output: dockerCli.Out(),
Format: formatter.Format(format),
}
sort.SliceStable(builders, func(i, j int) bool {
ierr := builders[i].Err() != nil
jerr := builders[j].Err() != nil
if ierr && !jerr {
return false
} else if !ierr && jerr {
return true
}
return i < j
})
render := func(format func(subContext formatter.SubContext) error) error {
for _, b := range builders {
if err := format(&lsContext{
Builder: &lsBuilder{
Builder: b,
Current: b.Name == current.Name,
},
format: ctx.Format,
}); err != nil {
return err
}
if b.Err() != nil {
if ctx.Format.IsTable() {
hasErrors = true
}
continue
}
for _, n := range b.Nodes() {
if n.Err != nil {
if ctx.Format.IsTable() {
hasErrors = true
}
}
if err := format(&lsContext{
format: ctx.Format,
Builder: &lsBuilder{
Builder: b,
Current: b.Name == current.Name,
},
node: n,
}); err != nil {
return err
}
}
}
return nil
}
lsCtx := lsContext{}
lsCtx.Header = formatter.SubHeaderContext{
"Name": lsNameNodeHeader,
"DriverEndpoint": lsDriverEndpointHeader,
"LastActivity": lsLastActivityHeader,
"Status": lsStatusHeader,
"Buildkit": lsBuildkitHeader,
"Platforms": lsPlatformsHeader,
}
return hasErrors, ctx.Write(&lsCtx, render)
}
type lsBuilder struct {
*builder.Builder
Current bool
}
type lsContext struct {
formatter.HeaderContext
Builder *lsBuilder
format formatter.Format
node builder.Node
}
func (c *lsContext) MarshalJSON() ([]byte, error) {
return json.Marshal(c.Builder)
}
func (c *lsContext) Name() string {
if c.node.Name == "" {
name := c.Builder.Name
if c.Builder.Current && c.format.IsTable() {
name += "*"
}
return name
}
if c.format.IsTable() {
return lsIndent + c.node.Name
}
return c.node.Name
}
func (c *lsContext) DriverEndpoint() string {
if c.node.Name == "" {
return c.Builder.Driver
}
if c.format.IsTable() {
return lsIndent + c.node.Endpoint
}
return c.node.Endpoint
}
func (c *lsContext) LastActivity() string {
if c.node.Name != "" || c.Builder.LastActivity.IsZero() {
return ""
}
return c.Builder.LastActivity.UTC().Format(time.RFC3339)
}
func (c *lsContext) Status() string {
if c.node.Name == "" {
if c.Builder.Err() != nil {
return "error"
}
return ""
}
if c.node.Err != nil {
return "error"
}
if c.node.DriverInfo != nil {
return c.node.DriverInfo.Status.String()
}
return ""
}
func (c *lsContext) Buildkit() string {
if c.node.Name == "" {
return ""
}
return c.node.Version
}
func (c *lsContext) Platforms() string {
if c.node.Name == "" {
return ""
}
return strings.Join(platformutil.FormatInGroups(c.node.Node.Platforms, c.node.Platforms), ", ")
}
func (c *lsContext) Error() string {
if c.node.Name != "" && c.node.Err != nil {
return c.node.Err.Error()
} else if err := c.Builder.Err(); err != nil {
return err.Error()
}
return ""
}

View File

@@ -1,48 +0,0 @@
package commands
import (
"fmt"
"io"
"log"
"os"
"github.com/docker/buildx/build"
"github.com/docker/docker/api/types/versions"
"github.com/moby/buildkit/frontend/subrequests"
"github.com/moby/buildkit/frontend/subrequests/outline"
"github.com/moby/buildkit/frontend/subrequests/targets"
)
func printResult(f *build.PrintFunc, res map[string]string) error {
switch f.Name {
case "outline":
return printValue(outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res)
case "targets":
return printValue(targets.PrintTargets, targets.SubrequestsTargetsDefinition.Version, f.Format, res)
case "subrequests.describe":
return printValue(subrequests.PrintDescribe, subrequests.SubrequestsDescribeDefinition.Version, f.Format, res)
default:
if dt, ok := res["result.txt"]; ok {
fmt.Print(dt)
} else {
log.Printf("%s %+v", f, res)
}
}
return nil
}
type printFunc func([]byte, io.Writer) error
func printValue(printer printFunc, version string, format string, res map[string]string) error {
if format == "json" {
fmt.Fprintln(os.Stdout, res["result.json"])
return nil
}
if res["version"] != "" && versions.LessThan(version, res["version"]) && res["result.txt"] != "" {
// structure is too new and we don't know how to print it
fmt.Fprint(os.Stdout, res["result.txt"])
return nil
}
return printer([]byte(res["result.json"]), os.Stdout)
}

View File

@@ -1,6 +1,7 @@
package commands
import (
"context"
"fmt"
"os"
"strings"
@@ -8,13 +9,13 @@ import (
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
"github.com/docker/docker/api/types/filters"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
@@ -34,9 +35,7 @@ const (
allCacheWarning = `WARNING! This will remove all build cache. Are you sure you want to continue?`
)
func runPrune(dockerCli command.Cli, opts pruneOptions) error {
ctx := appcontext.Context()
func runPrune(ctx context.Context, dockerCli command.Cli, opts pruneOptions) error {
pruneFilters := opts.filter.Value()
pruneFilters = command.PruneFilters(dockerCli, pruneFilters)
@@ -50,8 +49,12 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
warning = allCacheWarning
}
if !opts.force && !command.PromptForConfirmation(dockerCli.In(), dockerCli.Out(), warning) {
return nil
if !opts.force {
if ok, err := prompt(ctx, dockerCli.In(), dockerCli.Out(), warning); err != nil {
return err
} else if !ok {
return nil
}
}
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
@@ -59,7 +62,7 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
return err
}
nodes, err := b.LoadNodes(ctx, false)
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
@@ -137,8 +140,9 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
Args: cli.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = rootOpts.builder
return runPrune(dockerCli, options)
return runPrune(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -8,16 +8,15 @@ import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/cli/cli"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type rmOptions struct {
builder string
builders []string
keepState bool
keepDaemon bool
allInactive bool
@@ -28,11 +27,13 @@ const (
rmInactiveWarning = `WARNING! This will remove all builders that are not in running state. Are you sure you want to continue?`
)
func runRm(dockerCli command.Cli, in rmOptions) error {
ctx := appcontext.Context()
if in.allInactive && !in.force && !command.PromptForConfirmation(dockerCli.In(), dockerCli.Out(), rmInactiveWarning) {
return nil
func runRm(ctx context.Context, dockerCli command.Cli, in rmOptions) error {
if in.allInactive && !in.force {
if ok, err := prompt(ctx, dockerCli.In(), dockerCli.Out(), rmInactiveWarning); err != nil {
return err
} else if !ok {
return nil
}
}
txn, release, err := storeutil.GetStore(dockerCli)
@@ -45,33 +46,52 @@ func runRm(dockerCli command.Cli, in rmOptions) error {
return rmAllInactive(ctx, txn, dockerCli, in)
}
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
eg, _ := errgroup.WithContext(ctx)
for _, name := range in.builders {
func(name string) {
eg.Go(func() (err error) {
defer func() {
if err == nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", name)
} else {
_, _ = fmt.Fprintf(dockerCli.Err(), "failed to remove %s: %v\n", name, err)
}
}()
b, err := builder.New(dockerCli,
builder.WithName(name),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
err1 := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
if err1 != nil {
return err1
}
return nil
})
}(name)
}
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return err
if err := eg.Wait(); err != nil {
return errors.New("failed to remove one or more builders")
}
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
err1 := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
if err1 != nil {
return err1
}
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.Name)
return nil
}
@@ -79,24 +99,24 @@ func rmCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
var options rmOptions
cmd := &cobra.Command{
Use: "rm [NAME]",
Short: "Remove a builder instance",
Args: cli.RequiresMaxArgs(1),
Use: "rm [OPTIONS] [NAME] [NAME...]",
Short: "Remove one or more builder instances",
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = rootOpts.builder
options.builders = []string{rootOpts.builder}
if len(args) > 0 {
if options.allInactive {
return errors.New("cannot specify builder name when --all-inactive is set")
}
options.builder = args[0]
options.builders = args
}
return runRm(dockerCli, options)
return runRm(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
flags.BoolVar(&options.keepState, "keep-state", false, "Keep BuildKit state")
flags.BoolVar(&options.keepDaemon, "keep-daemon", false, "Keep the buildkitd daemon running")
flags.BoolVar(&options.keepDaemon, "keep-daemon", false, "Keep the BuildKit daemon running")
flags.BoolVar(&options.allInactive, "all-inactive", false, "Remove all inactive builders")
flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation")
@@ -137,13 +157,10 @@ func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, i
for _, b := range builders {
func(b *builder.Builder) {
eg.Go(func() error {
nodes, err := b.LoadNodes(timeoutCtx, true)
nodes, err := b.LoadNodes(timeoutCtx, builder.WithData())
if err != nil {
return errors.Wrapf(err, "cannot load %s", b.Name)
}
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
if b.Dynamic {
return nil
}

View File

@@ -3,12 +3,17 @@ package commands
import (
"os"
debugcmd "github.com/docker/buildx/commands/debug"
imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/logutil"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/plugin"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/moby/buildkit/util/appcontext"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
@@ -22,12 +27,18 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
Annotations: map[string]string{
annotation.CodeDelimiter: `"`,
},
}
if isPlugin {
cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
cmd.SetContext(appcontext.Context())
if !isPlugin {
return nil
}
return plugin.PersistentPreRunE(cmd, args)
}
} else {
},
}
if !isPlugin {
// match plugin behavior for standalone mode
// https://github.com/docker/cli/blob/6c9eb708fa6d17765d71965f90e1c59cea686ee9/cli-plugins/plugin/plugin.go#L117-L127
cmd.SilenceUsage = true
@@ -35,6 +46,11 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
cmd.TraverseChildren = true
cmd.DisableFlagsInUseLine = true
cli.DisableFlagsInUseLine(cmd)
// DEBUG=1 should perform the same as --debug at the docker root level
if debug.IsEnabled() {
debug.Enable()
}
}
logrus.SetFormatter(&logutil.Formatter{})
@@ -47,16 +63,9 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
"using default config store",
))
// filter out useless commandConn.CloseWrite warning message that can occur
// when listing builder instances with "buildx ls" for those that are
// unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
// https://github.com/docker/cli/blob/3fb4fb83dfb5db0c0753a8316f21aea54dab32c5/cli/connhelper/commandconn/commandconn.go#L203-L214
logrus.AddHook(logutil.NewFilter([]logrus.Level{
logrus.WarnLevel,
},
"commandConn.CloseWrite:",
"commandConn.CloseRead:",
))
if !isExperimental() {
cmd.SetHelpTemplate(cmd.HelpTemplate() + "\nExperimental commands and flags are hidden. Set BUILDX_EXPERIMENTAL=1 to show them.\n")
}
addCommands(cmd, dockerCli)
return cmd
@@ -71,9 +80,10 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
rootFlags(opts, cmd.PersistentFlags())
cmd.AddCommand(
buildCmd(dockerCli, opts),
buildCmd(dockerCli, opts, nil),
bakeCmd(dockerCli, opts),
createCmd(dockerCli),
dialStdioCmd(dockerCli, opts),
rmCmd(dockerCli, opts),
lsCmd(dockerCli),
useCmd(dockerCli, opts),
@@ -86,6 +96,17 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
)
if isExperimental() {
cmd.AddCommand(debugcmd.RootCmd(dockerCli,
newDebuggableBuild(dockerCli, opts),
))
remote.AddControllerCommands(cmd, dockerCli)
}
cmd.RegisterFlagCompletionFunc( //nolint:errcheck
"builder",
completion.BuilderNames(dockerCli),
)
}
func rootFlags(options *rootOptions, flags *pflag.FlagSet) {

View File

@@ -4,9 +4,9 @@ import (
"context"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/spf13/cobra"
)
@@ -14,9 +14,7 @@ type stopOptions struct {
builder string
}
func runStop(dockerCli command.Cli, in stopOptions) error {
ctx := appcontext.Context()
func runStop(ctx context.Context, dockerCli command.Cli, in stopOptions) error {
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
@@ -24,7 +22,7 @@ func runStop(dockerCli command.Cli, in stopOptions) error {
if err != nil {
return err
}
nodes, err := b.LoadNodes(ctx, false)
nodes, err := b.LoadNodes(ctx)
if err != nil {
return err
}
@@ -44,8 +42,9 @@ func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
if len(args) > 0 {
options.builder = args[0]
}
return runStop(dockerCli, options)
return runStop(cmd.Context(), dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
return cmd

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -52,7 +53,8 @@ func uninstallCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runUninstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -34,10 +35,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
if err != nil {
return err
}
if err := txn.SetCurrent(ep, "", false, false); err != nil {
return err
}
return nil
return txn.SetCurrent(ep, "", false, false)
}
list, err := dockerCli.ContextStore().List()
if err != nil {
@@ -57,11 +55,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
if err != nil {
return err
}
if err := txn.SetCurrent(ep, in.builder, in.isGlobal, in.isDefault); err != nil {
return err
}
return nil
return txn.SetCurrent(ep, in.builder, in.isGlobal, in.isDefault)
}
func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
@@ -78,6 +72,7 @@ func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runUse(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()

23
commands/util.go Normal file
View File

@@ -0,0 +1,23 @@
package commands
import (
"context"
"io"
"github.com/docker/cli/cli/command"
)
func prompt(ctx context.Context, ins io.Reader, out io.Writer, msg string) (bool, error) {
done := make(chan struct{})
var ok bool
go func() {
ok = command.PromptForConfirmation(ins, out, msg)
close(done)
}()
select {
case <-ctx.Done():
return false, context.Cause(ctx)
case <-done:
return ok, nil
}
}

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -23,6 +24,7 @@ func versionCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runVersion(dockerCli)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

280
controller/build/build.go Normal file
View File

@@ -0,0 +1,280 @@
package build
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"sync"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
)
const defaultTargetName = "default"
// RunBuild runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
}
contexts := map[string]build.NamedContext{}
for name, path := range in.NamedContexts {
contexts[name] = build.NamedContext{Path: path}
}
opts := build.Options{
Inputs: build.Inputs{
ContextPath: in.ContextPath,
DockerfilePath: in.DockerfileName,
InStream: inStream,
NamedContexts: contexts,
},
Ref: in.Ref,
BuildArgs: in.BuildArgs,
CgroupParent: in.CgroupParent,
ExtraHosts: in.ExtraHosts,
Labels: in.Labels,
NetworkMode: in.NetworkMode,
NoCache: in.NoCache,
NoCacheFilter: in.NoCacheFilter,
Pull: in.Pull,
ShmSize: dockeropts.MemBytes(in.ShmSize),
Tags: in.Tags,
Target: in.Target,
Ulimits: controllerUlimitOpt2DockerUlimit(in.Ulimits),
GroupRef: in.GroupRef,
}
platforms, err := platformutil.Parse(in.Platforms)
if err != nil {
return nil, nil, err
}
opts.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(dockerConfig, nil))
secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, secrets)
sshSpecs := in.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(in.ContextPath) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, ssh)
outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil {
return nil, nil, err
}
if in.ExportPush {
if in.ExportLoad {
return nil, nil, errors.Errorf("push and load may not be set together at the moment")
}
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
}}
} else {
switch outputs[0].Type {
case "image":
outputs[0].Attrs["push"] = "true"
default:
return nil, nil, errors.Errorf("push and %q output can't be used together", outputs[0].Type)
}
}
}
if in.ExportLoad {
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "docker",
Attrs: map[string]string{},
}}
} else {
switch outputs[0].Type {
case "docker":
default:
return nil, nil, errors.Errorf("load and %q output can't be used together", outputs[0].Type)
}
}
}
annotations, err := buildflags.ParseAnnotations(in.Annotations)
if err != nil {
return nil, nil, err
}
for _, o := range outputs {
for k, v := range annotations {
o.Attrs[k.String()] = v
}
}
opts.Exports = outputs
opts.CacheFrom = controllerapi.CreateCaches(in.CacheFrom)
opts.CacheTo = controllerapi.CreateCaches(in.CacheTo)
opts.Attests = controllerapi.CreateAttestations(in.Attests)
opts.SourcePolicy = in.SourcePolicy
allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil {
return nil, nil, err
}
opts.Allow = allow
if in.PrintFunc != nil {
opts.PrintFunc = &build.PrintFunc{
Name: in.PrintFunc.Name,
Format: in.PrintFunc.Format,
}
}
// key string used for kubernetes "sticky" mode
contextPathHash, err := filepath.Abs(in.ContextPath)
if err != nil {
contextPathHash = in.ContextPath
}
// TODO: this should not be loaded this side of the controller api
b, err := builder.New(dockerCli,
builder.WithName(in.Builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return nil, nil, err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx)
if err != nil {
return nil, nil, err
}
resp, res, err := buildTargets(ctx, dockerCli, b.NodeGroup, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
err = wrapBuildError(err, false)
if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, err
}
return resp, res, nil
}
// buildTargets runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func buildTargets(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, nodes []builder.Node, opts map[string]build.Options, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
var res *build.ResultHandle
var resp map[string]*client.SolveResponse
var err error
if generateResult {
var mu sync.Mutex
var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock()
defer mu.Unlock()
if res == nil || driverIndex < idx {
idx, res = driverIndex, gotRes
}
})
} else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
}
if err != nil {
return nil, res, err
}
return resp[defaultTargetName], res, err
}
func wrapBuildError(err error, bake bool) error {
if err == nil {
return nil
}
st, ok := grpcerrors.AsGRPCStatus(err)
if ok {
if st.Code() == codes.Unimplemented && strings.Contains(st.Message(), "unsupported frontend capability moby.buildkit.frontend.contexts") {
msg := "current frontend does not support --build-context."
if bake {
msg = "current frontend does not support defining additional contexts for targets."
}
msg += " Named contexts are supported since Dockerfile v1.4. Use #syntax directive in Dockerfile or update to latest BuildKit."
return &wrapped{err, msg}
}
}
return err
}
type wrapped struct {
err error
msg string
}
func (w *wrapped) Error() string {
return w.msg
}
func (w *wrapped) Unwrap() error {
return w.err
}
func updateLastActivity(dockerCli command.Cli, ng *store.NodeGroup) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
return txn.UpdateLastActivity(ng)
}
func controllerUlimitOpt2DockerUlimit(u *controllerapi.UlimitOpt) *dockeropts.UlimitOpt {
if u == nil {
return nil
}
values := make(map[string]*units.Ulimit)
for k, v := range u.Values {
values[k] = &units.Ulimit{
Name: v.Name,
Hard: v.Hard,
Soft: v.Soft,
}
}
return dockeropts.NewUlimitOpt(&values)
}

View File

@@ -0,0 +1,32 @@
package control
import (
"context"
"io"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
)
type BuildxController interface {
Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process.
// If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error
Close() error
List(ctx context.Context) (refs []string, _ error)
Disconnect(ctx context.Context, ref string) error
ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error)
DisconnectProcess(ctx context.Context, ref, pid string) error
Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error)
}
type ControlOptions struct {
ServerConfig string
Root string
Detach bool
}

36
controller/controller.go Normal file
View File

@@ -0,0 +1,36 @@
package controller
import (
"context"
"fmt"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/controller/local"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
)
func NewController(ctx context.Context, opts control.ControlOptions, dockerCli command.Cli, pw progress.Writer) (control.BuildxController, error) {
var name string
if opts.Detach {
name = "remote"
} else {
name = "local"
}
var c control.BuildxController
err := progress.Wrap(fmt.Sprintf("[internal] connecting to %s controller", name), pw.Write, func(l progress.SubLogger) (err error) {
if opts.Detach {
c, err = remote.NewRemoteBuildxController(ctx, dockerCli, opts, l)
} else {
c = local.NewLocalBuildxController(ctx, dockerCli, l)
}
return err
})
if err != nil {
return nil, errors.Wrap(err, "failed to start buildx controller")
}
return c, nil
}

View File

@@ -0,0 +1,34 @@
package errdefs
import (
"github.com/containerd/typeurl/v2"
"github.com/moby/buildkit/util/grpcerrors"
)
func init() {
typeurl.Register((*Build)(nil), "github.com/docker/buildx", "errdefs.Build+json")
}
type BuildError struct {
Build
error
}
func (e *BuildError) Unwrap() error {
return e.error
}
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return &e.Build
}
func WrapBuild(err error, ref string) error {
if err == nil {
return nil
}
return &BuildError{Build: Build{Ref: ref}, error: err}
}
func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: *b}
}

View File

@@ -0,0 +1,77 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: errdefs.proto
package errdefs
import (
fmt "fmt"
proto "github.com/gogo/protobuf/proto"
_ "github.com/moby/buildkit/solver/pb"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct {
Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Build) Reset() { *m = Build{} }
func (m *Build) String() string { return proto.CompactTextString(m) }
func (*Build) ProtoMessage() {}
func (*Build) Descriptor() ([]byte, []int) {
return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
}
var xxx_messageInfo_Build proto.InternalMessageInfo
func (m *Build) GetRef() string {
if m != nil {
return m.Ref
}
return ""
}
func init() {
proto.RegisterType((*Build)(nil), "errdefs.Build")
}
func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var fileDescriptor_689dc58a5060aff5 = []byte{
// 111 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
}

View File

@@ -0,0 +1,9 @@
syntax = "proto3";
package errdefs;
import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build {
string Ref = 1;
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -0,0 +1,146 @@
package local
import (
"context"
"io"
"sync/atomic"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{
dockerCli: dockerCli,
ref: "local",
processes: processes.NewManager(),
}
}
type buildConfig struct {
// TODO: these two structs should be merged
// Discussion: https://github.com/docker/buildx/pull/1640#discussion_r1113279719
resultCtx *build.ResultHandle
buildOptions *controllerapi.BuildOptions
}
type localController struct {
dockerCli command.Cli
ref string
buildConfig buildConfig
processes *processes.Manager
buildOnGoing atomic.Bool
}
func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, errors.New("build ongoing")
}
defer b.buildOnGoing.Store(false)
resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil {
b.buildConfig = buildConfig{
resultCtx: res,
buildOptions: &options,
}
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, b.ref)
}
}
if buildErr != nil {
return "", nil, buildErr
}
return b.ref, resp, nil
}
func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return b.processes.ListProcesses(), nil
}
func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
return b.processes.DeleteProcess(pid)
}
func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses()
}
func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
proc, ok := b.processes.Get(pid)
if !ok {
// Start a new process.
if b.buildConfig.resultCtx == nil {
return errors.New("no build result is registered")
}
var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select {
case <-ioCancelledCh:
return errors.Errorf("io cancelled")
case err := <-proc.Done():
return err
case <-ctx.Done():
return ctx.Err()
}
}
func (b *localController) Kill(context.Context) error {
b.Close()
return nil
}
func (b *localController) Close() error {
b.cancelRunningProcesses()
if b.buildConfig.resultCtx != nil {
b.buildConfig.resultCtx.Done()
}
// TODO: cancel ongoing builds?
return nil
}
func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.ref}, nil
}
func (b *localController) Disconnect(ctx context.Context, key string) error {
b.Close()
return nil
}
func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
}

20
controller/pb/attest.go Normal file
View File

@@ -0,0 +1,20 @@
package pb
func CreateAttestations(attests []*Attest) map[string]*string {
result := map[string]*string{}
for _, attest := range attests {
// ignore duplicates
if _, ok := result[attest.Type]; ok {
continue
}
if attest.Disabled {
result[attest.Type] = nil
continue
}
attrs := attest.Attrs
result[attest.Type] = &attrs
}
return result
}

21
controller/pb/cache.go Normal file
View File

@@ -0,0 +1,21 @@
package pb
import "github.com/moby/buildkit/client"
func CreateCaches(entries []*CacheOptionsEntry) []client.CacheOptionsEntry {
var outs []client.CacheOptionsEntry
if len(entries) == 0 {
return nil
}
for _, entry := range entries {
out := client.CacheOptionsEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
outs = append(outs, out)
}
return outs
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,248 @@
syntax = "proto3";
package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "pb";
service Controller {
rpc Build(BuildRequest) returns (BuildResponse);
rpc Inspect(InspectRequest) returns (InspectResponse);
rpc Status(StatusRequest) returns (stream StatusResponse);
rpc Input(stream InputMessage) returns (InputResponse);
rpc Invoke(stream Message) returns (stream Message);
rpc List(ListRequest) returns (ListResponse);
rpc Disconnect(DisconnectRequest) returns (DisconnectResponse);
rpc Info(InfoRequest) returns (InfoResponse);
rpc ListProcesses(ListProcessesRequest) returns (ListProcessesResponse);
rpc DisconnectProcess(DisconnectProcessRequest) returns (DisconnectProcessResponse);
}
message ListProcessesRequest {
string Ref = 1;
}
message ListProcessesResponse {
repeated ProcessInfo Infos = 1;
}
message ProcessInfo {
string ProcessID = 1;
InvokeConfig InvokeConfig = 2;
}
message DisconnectProcessRequest {
string Ref = 1;
string ProcessID = 2;
}
message DisconnectProcessResponse {
}
message BuildRequest {
string Ref = 1;
BuildOptions Options = 2;
}
message BuildOptions {
string ContextPath = 1;
string DockerfileName = 2;
PrintFunc PrintFunc = 3;
map<string, string> NamedContexts = 4;
repeated string Allow = 5;
repeated Attest Attests = 6;
map<string, string> BuildArgs = 7;
repeated CacheOptionsEntry CacheFrom = 8;
repeated CacheOptionsEntry CacheTo = 9;
string CgroupParent = 10;
repeated ExportEntry Exports = 11;
repeated string ExtraHosts = 12;
map<string, string> Labels = 13;
string NetworkMode = 14;
repeated string NoCacheFilter = 15;
repeated string Platforms = 16;
repeated Secret Secrets = 17;
int64 ShmSize = 18;
repeated SSH SSH = 19;
repeated string Tags = 20;
string Target = 21;
UlimitOpt Ulimits = 22;
string Builder = 23;
bool NoCache = 24;
bool Pull = 25;
bool ExportPush = 26;
bool ExportLoad = 27;
moby.buildkit.v1.sourcepolicy.Policy SourcePolicy = 28;
string Ref = 29;
string GroupRef = 30;
repeated string Annotations = 31;
}
message ExportEntry {
string Type = 1;
map<string, string> Attrs = 2;
string Destination = 3;
}
message CacheOptionsEntry {
string Type = 1;
map<string, string> Attrs = 2;
}
message Attest {
string Type = 1;
bool Disabled = 2;
string Attrs = 3;
}
message SSH {
string ID = 1;
repeated string Paths = 2;
}
message Secret {
string ID = 1;
string FilePath = 2;
string Env = 3;
}
message PrintFunc {
string Name = 1;
string Format = 2;
}
message InspectRequest {
string Ref = 1;
}
message InspectResponse {
BuildOptions Options = 1;
}
message UlimitOpt {
map<string, Ulimit> values = 1;
}
message Ulimit {
string Name = 1;
int64 Hard = 2;
int64 Soft = 3;
}
message BuildResponse {
map<string, string> ExporterResponse = 1;
}
message DisconnectRequest {
string Ref = 1;
}
message DisconnectResponse {}
message ListRequest {
string Ref = 1;
}
message ListResponse {
repeated string keys = 1;
}
message InputMessage {
oneof Input {
InputInitMessage Init = 1;
DataMessage Data = 2;
}
}
message InputInitMessage {
string Ref = 1;
}
message DataMessage {
bool EOF = 1; // true if eof was reached
bytes Data = 2; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message InputResponse {}
message Message {
oneof Input {
InitMessage Init = 1;
// FdMessage used from client to server for input (stdin) and
// from server to client for output (stdout, stderr)
FdMessage File = 2;
// ResizeMessage used from client to server for terminal resize events
ResizeMessage Resize = 3;
// SignalMessage is used from client to server to send signal events
SignalMessage Signal = 4;
}
}
message InitMessage {
string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored.
string ProcessID = 2;
InvokeConfig InvokeConfig = 3;
}
message InvokeConfig {
repeated string Entrypoint = 1;
repeated string Cmd = 2;
bool NoCmd = 11; // Do not set cmd but use the image's default
repeated string Env = 3;
string User = 4;
bool NoUser = 5; // Do not set user but use the image's default
string Cwd = 6;
bool NoCwd = 7; // Do not set cwd but use the image's default
bool Tty = 8;
bool Rollback = 9; // Kill all process in the container and recreate it.
bool Initial = 10; // Run container from the initial state of that stage (supported only on the failed step)
}
message FdMessage {
uint32 Fd = 1; // what fd the data was from
bool EOF = 2; // true if eof was reached
bytes Data = 3; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message ResizeMessage {
uint32 Rows = 1;
uint32 Cols = 2;
}
message SignalMessage {
// we only send name (ie HUP, INT) because the int values
// are platform dependent.
string Name = 1;
}
message StatusRequest {
string Ref = 1;
}
message StatusResponse {
repeated moby.buildkit.v1.Vertex vertexes = 1;
repeated moby.buildkit.v1.VertexStatus statuses = 2;
repeated moby.buildkit.v1.VertexLog logs = 3;
repeated moby.buildkit.v1.VertexWarning warnings = 4;
}
message InfoRequest {}
message InfoResponse {
BuildxVersion buildxVersion = 1;
}
message BuildxVersion {
string package = 1;
string version = 2;
string revision = 3;
}

View File

@@ -1,66 +1,32 @@
package buildflags
package pb
import (
"encoding/csv"
"io"
"os"
"strconv"
"strings"
"github.com/containerd/console"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func ParseOutputs(inp []string) ([]client.ExportEntry, error) {
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
if len(inp) == 0 {
if len(entries) == 0 {
return nil, nil
}
for _, s := range inp {
csvReader := csv.NewReader(strings.NewReader(s))
fields, err := csvReader.Read()
if err != nil {
return nil, err
var stdoutUsed bool
for _, entry := range entries {
if entry.Type == "" {
return nil, errors.Errorf("type is required for output")
}
out := client.ExportEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
if len(fields) == 1 && fields[0] == s && !strings.HasPrefix(s, "type=") {
if s != "-" {
outs = append(outs, client.ExportEntry{
Type: client.ExporterLocal,
OutputDir: s,
})
continue
}
out = client.ExportEntry{
Type: client.ExporterTar,
Attrs: map[string]string{
"dest": s,
},
}
}
if out.Type == "" {
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) != 2 {
return nil, errors.Errorf("invalid value %s", field)
}
key := strings.TrimSpace(strings.ToLower(parts[0]))
value := parts[1]
switch key {
case "type":
out.Type = value
default:
out.Attrs[key] = value
}
}
}
if out.Type == "" {
return nil, errors.Errorf("type is required for output")
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
supportFile := false
@@ -79,56 +45,53 @@ func ParseOutputs(inp []string) ([]client.ExportEntry, error) {
supportDir = !tar
case "registry":
out.Type = client.ExporterImage
if _, ok := out.Attrs["push"]; !ok {
out.Attrs["push"] = "true"
}
}
dest, ok := out.Attrs["dest"]
if supportDir {
if !ok {
if entry.Destination == "" {
return nil, errors.Errorf("dest is required for %s exporter", out.Type)
}
if dest == "-" {
if entry.Destination == "-" {
return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
}
fi, err := os.Stat(dest)
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination directory: %s", dest)
return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
}
if err == nil && !fi.IsDir() {
return nil, errors.Errorf("destination directory %s is a file", dest)
return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
}
out.OutputDir = dest
out.OutputDir = entry.Destination
}
if supportFile {
if !ok && out.Type != client.ExporterDocker {
dest = "-"
if entry.Destination == "" && out.Type != client.ExporterDocker {
entry.Destination = "-"
}
if dest == "-" {
if entry.Destination == "-" {
if stdoutUsed {
return nil, errors.Errorf("multiple outputs configured to write to stdout")
}
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
} else if dest != "" {
fi, err := os.Stat(dest)
stdoutUsed = true
} else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination file: %s", dest)
return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
}
if err == nil && fi.IsDir() {
return nil, errors.Errorf("destination file %s is a directory", dest)
return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
}
f, err := os.Create(dest)
f, err := os.Create(entry.Destination)
if err != nil {
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
}
}
if supportFile || supportDir {
delete(out.Attrs, "dest")
}
outs = append(outs, out)
}

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

175
controller/pb/path.go Normal file
View File

@@ -0,0 +1,175 @@
package pb
import (
"path/filepath"
"strings"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
)
// ResolveOptionPaths resolves all paths contained in BuildOptions
// and replaces them to absolute paths.
func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
localContext := false
if options.ContextPath != "" && options.ContextPath != "-" {
if !isRemoteURL(options.ContextPath) {
localContext = true
options.ContextPath, err = filepath.Abs(options.ContextPath)
if err != nil {
return nil, err
}
}
}
if options.DockerfileName != "" && options.DockerfileName != "-" {
if localContext && !urlutil.IsURL(options.DockerfileName) {
options.DockerfileName, err = filepath.Abs(options.DockerfileName)
if err != nil {
return nil, err
}
}
}
var contexts map[string]string
for k, v := range options.NamedContexts {
if isRemoteURL(v) || strings.HasPrefix(v, "docker-image://") {
// url prefix, this is a remote path
} else if strings.HasPrefix(v, "oci-layout://") {
// oci layout prefix, this is a local path
p := strings.TrimPrefix(v, "oci-layout://")
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
v = "oci-layout://" + p
} else {
// no prefix, assume local path
v, err = filepath.Abs(v)
if err != nil {
return nil, err
}
}
if contexts == nil {
contexts = make(map[string]string)
}
contexts[k] = v
}
options.NamedContexts = contexts
var cacheFrom []*CacheOptionsEntry
for _, co := range options.CacheFrom {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "src":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheFrom = append(cacheFrom, co)
default:
cacheFrom = append(cacheFrom, co)
}
}
options.CacheFrom = cacheFrom
var cacheTo []*CacheOptionsEntry
for _, co := range options.CacheTo {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "dest":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheTo = append(cacheTo, co)
default:
cacheTo = append(cacheTo, co)
}
}
options.CacheTo = cacheTo
var exports []*ExportEntry
for _, e := range options.Exports {
if e.Destination != "" && e.Destination != "-" {
e.Destination, err = filepath.Abs(e.Destination)
if err != nil {
return nil, err
}
}
exports = append(exports, e)
}
options.Exports = exports
var secrets []*Secret
for _, s := range options.Secrets {
if s.FilePath != "" {
s.FilePath, err = filepath.Abs(s.FilePath)
if err != nil {
return nil, err
}
}
secrets = append(secrets, s)
}
options.Secrets = secrets
var ssh []*SSH
for _, s := range options.SSH {
var ps []string
for _, pt := range s.Paths {
p := pt
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
ps = append(ps, p)
}
s.Paths = ps
ssh = append(ssh, s)
}
options.SSH = ssh
return options, nil
}
func isRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}

248
controller/pb/path_test.go Normal file
View File

@@ -0,0 +1,248 @@
package pb
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/stretchr/testify/require"
)
func TestResolvePaths(t *testing.T) {
tmpwd, err := os.MkdirTemp("", "testresolvepaths")
require.NoError(t, err)
defer os.Remove(tmpwd)
require.NoError(t, os.Chdir(tmpwd))
tests := []struct {
name string
options BuildOptions
want BuildOptions
}{
{
name: "contextpath",
options: BuildOptions{ContextPath: "test"},
want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
},
{
name: "contextpath-cwd",
options: BuildOptions{ContextPath: "."},
want: BuildOptions{ContextPath: tmpwd},
},
{
name: "contextpath-dash",
options: BuildOptions{ContextPath: "-"},
want: BuildOptions{ContextPath: "-"},
},
{
name: "contextpath-ssh",
options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "dockerfilename",
options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
},
{
name: "dockerfilename-dash",
options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
},
{
name: "dockerfilename-remote",
options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "contexts",
options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
},
{
name: "cache-from",
options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "cache-to",
options: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "exports",
options: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: "test1",
},
{
Type: "tar",
Destination: "test3",
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: "test4",
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
want: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: filepath.Join(tmpwd, "test1"),
},
{
Type: "tar",
Destination: filepath.Join(tmpwd, "test3"),
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: filepath.Join(tmpwd, "test4"),
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
},
{
name: "secrets",
options: BuildOptions{
Secrets: []*Secret{
{
FilePath: "test1",
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: "test3",
},
},
},
want: BuildOptions{
Secrets: []*Secret{
{
FilePath: filepath.Join(tmpwd, "test1"),
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: filepath.Join(tmpwd, "test3"),
},
},
},
},
{
name: "ssh",
options: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{"test1", "test2"},
},
{
ID: "a",
Paths: []string{"test3"},
},
},
},
want: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{filepath.Join(tmpwd, "test1"), filepath.Join(tmpwd, "test2")},
},
{
ID: "a",
Paths: []string{filepath.Join(tmpwd, "test3")},
},
},
},
},
}
for _, tt := range tests {
tt := tt
t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err)
if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, *got)
}
})
}
}

126
controller/pb/progress.go Normal file
View File

@@ -0,0 +1,126 @@
package pb
import (
"github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest"
)
type writer struct {
ch chan<- *StatusResponse
}
func NewProgressWriter(ch chan<- *StatusResponse) progress.Writer {
return &writer{ch: ch}
}
func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status)
}
func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true
}
func (w *writer) ClearLogSource(interface{}) {}
func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{}
for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: v.Vertex,
Stream: int64(v.Stream),
Msg: v.Data,
Timestamp: v.Timestamp,
})
}
for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: v.Vertex,
Level: int64(v.Level),
Short: v.Short,
Detail: v.Detail,
Url: v.URL,
Info: v.SourceInfo,
Ranges: v.Range,
})
}
return &resp
}
func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{}
for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{
Vertex: v.Vertex,
Stream: int(v.Stream),
Data: v.Msg,
Timestamp: v.Timestamp,
})
}
for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: v.Vertex,
Level: int(v.Level),
Short: v.Short,
Detail: v.Detail,
URL: v.Url,
SourceInfo: v.Info,
Range: v.Ranges,
})
}
return &s
}

22
controller/pb/secrets.go Normal file
View File

@@ -0,0 +1,22 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/secrets/secretsprovider"
)
func CreateSecrets(secrets []*Secret) (session.Attachable, error) {
fs := make([]secretsprovider.Source, 0, len(secrets))
for _, secret := range secrets {
fs = append(fs, secretsprovider.Source{
ID: secret.ID,
FilePath: secret.FilePath,
Env: secret.Env,
})
}
store, err := secretsprovider.NewStore(fs)
if err != nil {
return nil, err
}
return secretsprovider.NewSecretProvider(store), nil
}

18
controller/pb/ssh.go Normal file
View File

@@ -0,0 +1,18 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider"
)
func CreateSSH(ssh []*SSH) (session.Attachable, error) {
configs := make([]sshprovider.AgentConfig, 0, len(ssh))
for _, ssh := range ssh {
cfg := sshprovider.AgentConfig{
ID: ssh.ID,
Paths: append([]string{}, ssh.Paths...),
}
configs = append(configs, cfg)
}
return sshprovider.NewSSHAgentProvider(configs)
}

View File

@@ -0,0 +1,149 @@
package processes
import (
"context"
"sync"
"sync/atomic"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/ioset"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// Process provides methods to control a process.
type Process struct {
inEnd *ioset.Forwarder
invokeConfig *pb.InvokeConfig
errCh chan error
processCancel func()
serveIOCancel func()
}
// ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil {
f()
}
p.serveIOCancel = ioCancelCallback
}
// Done returns a channel where error or nil will be sent
// when the process exits.
// TODO: change this to Wait()
func (p *Process) Done() <-chan error {
return p.errCh
}
// Manager manages a set of proceses.
type Manager struct {
container atomic.Value
processes sync.Map
}
// NewManager creates and returns a Manager.
func NewManager() *Manager {
return &Manager{}
}
// Get returns the specified process.
func (m *Manager) Get(id string) (*Process, bool) {
v, ok := m.processes.Load(id)
if !ok {
return nil, false
}
return v.(*Process), true
}
// CancelRunningProcesses cancels execution of all running processes.
func (m *Manager) CancelRunningProcesses() {
var funcs []func()
m.processes.Range(func(key, value any) bool {
funcs = append(funcs, value.(*Process).processCancel)
m.processes.Delete(key)
return true
})
for _, f := range funcs {
f()
}
}
// ListProcesses lists all running processes.
func (m *Manager) ListProcesses() (res []*pb.ProcessInfo) {
m.processes.Range(func(key, value any) bool {
res = append(res, &pb.ProcessInfo{
ProcessID: key.(string),
InvokeConfig: value.(*Process).invokeConfig,
})
return true
})
return res
}
// DeleteProcess deletes the specified process.
func (m *Manager) DeleteProcess(id string) error {
p, ok := m.processes.LoadAndDelete(id)
if !ok {
return errors.Errorf("unknown process %q", id)
}
p.(*Process).processCancel()
return nil
}
// StartProcess starts a process in the container.
// When a container isn't available (i.e. first time invoking or the container has exited) or cfg.Rollback is set,
// this method will start a new container and run the process in it. Otherwise, this method starts a new process in the
// existing container.
func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *pb.InvokeConfig) (*Process, error) {
// Get the target result to invoke a container from
var ctr *build.Container
if a := m.container.Load(); a != nil {
ctr = a.(*build.Container)
}
if cfg.Rollback || ctr == nil || ctr.IsUnavailable() {
go m.CancelRunningProcesses()
// (Re)create a new container if this is rollback or first time to invoke a process.
if ctr != nil {
go ctr.Cancel() // Finish the existing container
}
var err error
ctr, err = build.NewContainer(context.TODO(), resultCtx, cfg)
if err != nil {
return nil, errors.Errorf("failed to create container %v", err)
}
m.container.Store(ctr)
}
// [client(ForwardIO)] <-forwarder(switchable)-> [out] <-pipe-> [in] <- [process]
in, out := ioset.Pipe()
f := ioset.NewForwarder()
f.PropagateStdinClose = false
f.SetOut(&out)
// Register process
ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once
processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
p := &Process{
inEnd: f,
invokeConfig: cfg,
processCancel: processCancelFunc,
errCh: make(chan error),
}
m.processes.Store(pid, p)
go func() {
var err error
if err = ctr.Exec(ctx, cfg, in.Stdin, in.Stdout, in.Stderr); err != nil {
logrus.Debugf("process error: %v", err)
}
logrus.Debugf("finished process %s %v", pid, cfg.Entrypoint)
m.processes.Delete(pid)
processCancelFunc()
p.errCh <- err
}()
return p, nil
}

240
controller/remote/client.go Normal file
View File

@@ -0,0 +1,240 @@
package remote
import (
"context"
"io"
"sync"
"time"
"github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure"
)
func NewClient(ctx context.Context, addr string) (*Client, error) {
backoffConfig := backoff.DefaultConfig
backoffConfig.MaxDelay = 3 * time.Second
connParams := grpc.ConnectParams{
Backoff: backoffConfig,
}
gopts := []grpc.DialOption{
grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams),
grpc.WithContextDialer(dialer.ContextDialer),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(defaults.DefaultMaxRecvMsgSize)),
grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(defaults.DefaultMaxSendMsgSize)),
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
}
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil {
return nil, err
}
return &Client{conn: conn}, nil
}
type Client struct {
conn *grpc.ClientConn
closeOnce sync.Once
}
func (c *Client) Close() (err error) {
c.closeOnce.Do(func() {
err = c.conn.Close()
})
return
}
func (c *Client) Version(ctx context.Context) (string, string, string, error) {
res, err := c.client().Info(ctx, &pb.InfoRequest{})
if err != nil {
return "", "", "", err
}
v := res.BuildxVersion
return v.Package, v.Version, v.Revision, nil
}
func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
res, err := c.client().List(ctx, &pb.ListRequest{})
if err != nil {
return nil, err
}
return res.Keys, nil
}
func (c *Client) Disconnect(ctx context.Context, key string) error {
if key == "" {
return nil
}
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err
}
func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil {
return nil, err
}
return res.Infos, nil
}
func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err
}
func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if ref == "" || pid == "" {
return errors.New("build reference must be specified")
}
stream, err := c.client().Invoke(ctx)
if err != nil {
return err
}
return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in,
stdout: stdout,
stderr: stderr,
// TODO: Signal, Resize
})
}
func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
}
func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID()
statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx)
var resp *client.SolveResponse
eg.Go(func() error {
defer close(statusChan)
var err error
resp, err = c.build(egCtx, ref, options, in, statusChan)
return err
})
eg.Go(func() error {
for s := range statusChan {
st := s
progress.Write(st)
}
return nil
})
return ref, resp, eg.Wait()
}
func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{})
var resp *client.SolveResponse
eg.Go(func() error {
defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
Ref: ref,
Options: &options,
})
if err != nil {
return err
}
resp = &client.SolveResponse{
ExporterResponse: pbResp.ExporterResponse,
}
return nil
})
eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{
Ref: ref,
})
if err != nil {
return err
}
for {
resp, err := stream.Recv()
if err != nil {
if err == io.EOF {
return nil
}
return errors.Wrap(err, "failed to receive status")
}
statusChan <- pb.FromControlStatus(resp)
}
})
if in != nil {
eg.Go(func() error {
stream, err := c.client().Input(egCtx)
if err != nil {
return err
}
if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{
Ref: ref,
},
},
}); err != nil {
return errors.Wrap(err, "failed to init input")
}
inReader, inWriter := io.Pipe()
eg2, _ := errgroup.WithContext(ctx)
eg2.Go(func() error {
<-done
return inWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(inWriter, in)
inWriter.Close()
}()
eg2.Go(func() error {
for {
buf := make([]byte, 32*1024)
n, err := inReader.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
EOF: true,
},
},
})
})
return eg2.Wait()
})
}
return resp, eg.Wait()
}
func (c *Client) client() pb.ControllerClient {
return pb.NewControllerClient(c.conn)
}

View File

@@ -0,0 +1,333 @@
//go:build linux
package remote
import (
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"syscall"
"time"
"github.com/containerd/log"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pelletier/go-toml"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc"
)
const (
serveCommandName = "_INTERNAL_SERVE"
)
var (
defaultLogFilename = fmt.Sprintf("buildx.%s.log", version.Revision)
defaultSocketFilename = fmt.Sprintf("buildx.%s.sock", version.Revision)
defaultPIDFilename = fmt.Sprintf("buildx.%s.pid", version.Revision)
)
type serverConfig struct {
// Specify buildx server root
Root string `toml:"root"`
// LogLevel sets the logging level [trace, debug, info, warn, error, fatal, panic]
LogLevel string `toml:"log_level"`
// Specify file to output buildx server log
LogFile string `toml:"log_file"`
}
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
rootDir := opts.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running
ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server")
}
} else {
return &buildxController{c, serverRoot}, nil
}
// start buildx server via subcommand
err = logger.Wrap("no buildx server found; launching...", func() error {
launchFlags := []string{}
if opts.ServerConfig != "" {
launchFlags = append(launchFlags, "--config", opts.ServerConfig)
}
logFile, err := getLogFilePath(dockerCli, opts.ServerConfig)
if err != nil {
return err
}
wait, err := launch(ctx, logFile, append([]string{serveCommandName}, launchFlags...)...)
if err != nil {
return err
}
go wait()
// wait for buildx server to be ready
ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server")
}
return nil
})
if err != nil {
return nil, err
}
return &buildxController{c, serverRoot}, nil
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
serveCmd(dockerCli),
)
}
func serveCmd(dockerCli command.Cli) *cobra.Command {
var serverConfigPath string
cmd := &cobra.Command{
Use: fmt.Sprintf("%s [OPTIONS]", serveCommandName),
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Parse config
config, err := getConfig(dockerCli, serverConfigPath)
if err != nil {
return err
}
if config.LogLevel == "" {
logrus.SetLevel(logrus.InfoLevel)
} else {
lvl, err := logrus.ParseLevel(config.LogLevel)
if err != nil {
return errors.Wrap(err, "failed to prepare logger")
}
logrus.SetLevel(lvl)
}
logrus.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: log.RFC3339NanoFixed,
})
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return err
}
pidF := filepath.Join(root, defaultPIDFilename)
if err := os.WriteFile(pidF, []byte(fmt.Sprintf("%d", os.Getpid())), 0600); err != nil {
return err
}
defer func() {
if err := os.Remove(pidF); err != nil {
logrus.Errorf("failed to clean up info file %q: %v", pidF, err)
}
}()
// prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
})
defer b.Close()
// serve server
addr := filepath.Join(root, defaultSocketFilename)
if err := os.Remove(addr); err != nil && !os.IsNotExist(err) { // avoid EADDRINUSE
return err
}
defer func() {
if err := os.Remove(addr); err != nil {
logrus.Errorf("failed to clean up socket %q: %v", addr, err)
}
}()
logrus.Infof("starting server at %q", addr)
l, err := net.Listen("unix", addr)
if err != nil {
return err
}
rpc := grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
)
controllerapi.RegisterControllerServer(rpc, b)
doneCh := make(chan struct{})
errCh := make(chan error, 1)
go func() {
defer close(doneCh)
if err := rpc.Serve(l); err != nil {
errCh <- errors.Wrapf(err, "error on serving via socket %q", addr)
}
}()
var s os.Signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT)
signal.Notify(sigCh, syscall.SIGTERM)
select {
case err := <-errCh:
logrus.Errorf("got error %s, exiting", err)
return err
case s = <-sigCh:
logrus.Infof("got signal %s, exiting", s)
return nil
case <-doneCh:
logrus.Infof("rpc server done, exiting")
return nil
}
},
}
flags := cmd.Flags()
flags.StringVar(&serverConfigPath, "config", "", "Specify buildx server config file")
return cmd
}
func getLogFilePath(dockerCli command.Cli, configPath string) (string, error) {
config, err := getConfig(dockerCli, configPath)
if err != nil {
return "", err
}
if config.LogFile == "" {
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return "", err
}
return filepath.Join(root, defaultLogFilename), nil
}
return config.LogFile, nil
}
func getConfig(dockerCli command.Cli, configPath string) (*serverConfig, error) {
var defaultConfigPath bool
if configPath == "" {
defaultRoot := rootDataDir(dockerCli)
configPath = filepath.Join(defaultRoot, "config.toml")
defaultConfigPath = true
}
var config serverConfig
tree, err := toml.LoadFile(configPath)
if err != nil && !(os.IsNotExist(err) && defaultConfigPath) {
return nil, errors.Wrapf(err, "failed to read config %q", configPath)
} else if err == nil {
if err := tree.Unmarshal(&config); err != nil {
return nil, errors.Wrapf(err, "failed to unmarshal config %q", configPath)
}
}
return &config, nil
}
func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error) {
rootDir := config.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
if rootDir == "" {
return "", errors.New("buildx root dir must be determined")
}
if err := os.MkdirAll(rootDir, 0700); err != nil {
return "", err
}
serverRoot := filepath.Join(rootDir, "shared")
if err := os.MkdirAll(serverRoot, 0700); err != nil {
return "", err
}
return serverRoot, nil
}
func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
}
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {
c, err := NewClient(ctx, addr)
if err != nil {
return nil, err
}
p, v, r, err := c.Version(ctx)
if err != nil {
return nil, err
}
logrus.Debugf("connected to server (\"%v %v %v\")", p, v, r)
if !(p == version.Package && v == version.Version && r == version.Revision) {
return nil, errors.Errorf("version mismatch (client: \"%v %v %v\", server: \"%v %v %v\")", version.Package, version.Version, version.Revision, p, v, r)
}
return c, nil
}
type buildxController struct {
*Client
serverRoot string
}
func (c *buildxController) Kill(ctx context.Context) error {
pidB, err := os.ReadFile(filepath.Join(c.serverRoot, defaultPIDFilename))
if err != nil {
return err
}
pid, err := strconv.ParseInt(string(pidB), 10, 64)
if err != nil {
return err
}
if pid <= 0 {
return errors.New("no PID is recorded for buildx server")
}
p, err := os.FindProcess(int(pid))
if err != nil {
return err
}
if err := p.Signal(syscall.SIGINT); err != nil {
return err
}
// TODO: Should we send SIGKILL if process doesn't finish?
return nil
}
func launch(ctx context.Context, logFile string, args ...string) (func() error, error) {
// set absolute path of binary, since we set the working directory to the root
pathname, err := os.Executable()
if err != nil {
return nil, err
}
bCmd := exec.CommandContext(ctx, pathname, args...)
if logFile != "" {
f, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, err
}
defer f.Close()
bCmd.Stdout = f
bCmd.Stderr = f
}
bCmd.Stdin = nil
bCmd.Dir = "/"
bCmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
}
if err := bCmd.Start(); err != nil {
return nil, err
}
return bCmd.Wait, nil
}

View File

@@ -0,0 +1,19 @@
//go:build !linux
package remote
import (
"context"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
return nil, errors.New("remote buildx unsupported")
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {}

430
controller/remote/io.go Normal file
View File

@@ -0,0 +1,430 @@
package remote
import (
"context"
"io"
"syscall"
"time"
"github.com/docker/buildx/controller/pb"
"github.com/moby/sys/signal"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
type msgStream interface {
Send(*pb.Message) error
Recv() (*pb.Message, error)
}
type ioServerConfig struct {
stdin io.WriteCloser
stdout, stderr io.ReadCloser
// signalFn is a callback function called when a signal is reached to the client.
signalFn func(context.Context, syscall.Signal) error
// resizeFn is a callback function called when a resize event is reached to the client.
resizeFn func(context.Context, winSize) error
}
func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessage) error, ioConfig *ioServerConfig) (err error) {
stdin, stdout, stderr := ioConfig.stdin, ioConfig.stdout, ioConfig.stderr
stream := &debugStream{srv, "server=" + time.Now().String()}
eg, ctx := errgroup.WithContext(attachCtx)
done := make(chan struct{})
msg, err := receive(ctx, stream)
if err != nil {
return err
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
}
ref := init.Ref
if ref == "" {
return errors.New("no ref is provided")
}
if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server")
}
if stdout != nil {
stdoutReader, stdoutWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdoutWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdoutWriter, stdout)
stdoutWriter.Close()
}()
eg.Go(func() error {
defer stdoutReader.Close()
return copyToStream(1, stream, stdoutReader)
})
}
if stderr != nil {
stderrReader, stderrWriter := io.Pipe()
eg.Go(func() error {
<-done
return stderrWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stderrWriter, stderr)
stderrWriter.Close()
}()
eg.Go(func() error {
defer stderrReader.Close()
return copyToStream(2, stream, stderrReader)
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if file.Fd != 0 {
return errors.Errorf("unexpected fd: %v", file.Fd)
}
if stdin == nil {
continue // no stdin destination is specified so ignore the data
}
if len(file.Data) > 0 {
_, err := stdin.Write(file.Data)
if err != nil {
return err
}
}
if file.EOF {
stdin.Close()
}
} else if resize := msg.GetResize(); resize != nil {
if ioConfig.resizeFn != nil {
ioConfig.resizeFn(ctx, winSize{
cols: resize.Cols,
rows: resize.Rows,
})
}
} else if sig := msg.GetSignal(); sig != nil {
if ioConfig.signalFn != nil {
syscallSignal, ok := signal.SignalMap[sig.Name]
if !ok {
continue
}
ioConfig.signalFn(ctx, syscallSignal)
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
type ioAttachConfig struct {
stdin io.ReadCloser
stdout, stderr io.WriteCloser
signal <-chan syscall.Signal
resize <-chan winSize
}
type winSize struct {
rows uint32
cols uint32
}
func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage, cfg ioAttachConfig) (retErr error) {
eg, ctx := errgroup.WithContext(ctx)
done := make(chan struct{})
if err := stream.Send(&pb.Message{
Input: &pb.Message_Init{
Init: initMessage,
},
}); err != nil {
return errors.Wrap(err, "failed to init")
}
if cfg.stdin != nil {
stdinReader, stdinWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdinWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdinWriter, cfg.stdin)
stdinWriter.Close()
}()
eg.Go(func() error {
defer stdinReader.Close()
return copyToStream(0, stream, stdinReader)
})
}
if cfg.signal != nil {
eg.Go(func() error {
for {
var sig syscall.Signal
select {
case sig = <-cfg.signal:
case <-done:
return nil
case <-ctx.Done():
return nil
}
name := sigToName[sig]
if name == "" {
continue
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Signal{
Signal: &pb.SignalMessage{
Name: name,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send signal")
}
}
})
}
if cfg.resize != nil {
eg.Go(func() error {
for {
var win winSize
select {
case win = <-cfg.resize:
case <-done:
return nil
case <-ctx.Done():
return nil
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Resize{
Resize: &pb.ResizeMessage{
Rows: win.rows,
Cols: win.cols,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send resize")
}
}
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
eofs := make(map[uint32]struct{})
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if _, ok := eofs[file.Fd]; ok {
continue
}
var out io.WriteCloser
switch file.Fd {
case 1:
out = cfg.stdout
case 2:
out = cfg.stderr
default:
return errors.Errorf("unsupported fd %d", file.Fd)
}
if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
continue
}
if len(file.Data) > 0 {
if _, err := out.Write(file.Data); err != nil {
return err
}
}
if file.EOF {
eofs[file.Fd] = struct{}{}
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
msgCh := make(chan *pb.Message)
errCh := make(chan error)
go func() {
msg, err := stream.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
return
}
errCh <- err
return
}
msgCh <- msg
}()
select {
case msg := <-msgCh:
return msg, nil
case err := <-errCh:
return nil, err
case <-ctx.Done():
return nil, ctx.Err()
}
}
func copyToStream(fd uint32, snd msgStream, r io.Reader) error {
for {
buf := make([]byte, 32*1024)
n, err := r.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
EOF: true,
},
},
})
}
var sigToName = map[syscall.Signal]string{}
func init() {
for name, value := range signal.SignalMap {
sigToName[value] = name
}
}
type debugStream struct {
msgStream
prefix string
}
func (s *debugStream) Send(msg *pb.Message) error {
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|---> Resize Message (sender:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|---> Signal Message (sender:%v): %s", s.prefix, m.Signal.Name)
}
return s.msgStream.Send(msg)
}
func (s *debugStream) Recv() (*pb.Message, error) {
msg, err := s.msgStream.Recv()
if err != nil {
return nil, err
}
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|<--- Resize Message (receiver:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|<--- Signal Message (receiver:%v): %s", s.prefix, m.Signal.Name)
}
return msg, nil
}

439
controller/remote/server.go Normal file
View File

@@ -0,0 +1,439 @@
package remote
import (
"context"
"io"
"sync"
"sync/atomic"
"time"
"github.com/docker/buildx/build"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server {
return &Server{
buildFunc: buildFunc,
}
}
type Server struct {
buildFunc BuildFunc
session map[string]*session
sessionMu sync.Mutex
}
type session struct {
buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse
cancelBuild func()
buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter
result *build.ResultHandle
processes *processes.Manager
}
func (s *session) cancelRunningProcesses() {
s.processes.CancelRunningProcesses()
}
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
res = new(pb.ListProcessesResponse)
res.Infos = append(res.Infos, s.processes.ListProcesses()...)
return res, nil
}
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
return res, s.processes.DeleteProcess(req.ProcessID)
}
func (m *Server) Info(ctx context.Context, req *pb.InfoRequest) (res *pb.InfoResponse, err error) {
return &pb.InfoResponse{
BuildxVersion: &pb.BuildxVersion{
Package: version.Package,
Version: version.Version,
Revision: version.Revision,
},
}, nil
}
func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListResponse, err error) {
keys := make(map[string]struct{})
m.sessionMu.Lock()
for k := range m.session {
keys[k] = struct{}{}
}
m.sessionMu.Unlock()
var keysL []string
for k := range keys {
keysL = append(keysL, k)
}
return &pb.ListResponse{
Keys: keysL,
}, nil
}
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
key := req.Ref
if key == "" {
return nil, errors.New("disconnect: empty key")
}
m.sessionMu.Lock()
if s, ok := m.session[key]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
if s.result != nil {
s.result.Done()
}
}
delete(m.session, key)
m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil
}
func (m *Server) Close() error {
m.sessionMu.Lock()
for k := range m.session {
if s, ok := m.session[k]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
}
}
m.sessionMu.Unlock()
return nil
}
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("inspect: empty key")
}
var bo *pb.BuildOptions
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
bo = s.buildOptions
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", ref)
}
m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil
}
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("build: empty key")
}
// Prepare status channel and session
m.sessionMu.Lock()
if m.session == nil {
m.session = make(map[string]*session)
}
s, ok := m.session[ref]
if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock()
return &pb.BuildResponse{}, errors.New("build ongoing")
}
s.cancelRunningProcesses()
s.result = nil
} else {
s = &session{}
s.buildOnGoing.Store(true)
}
s.processes = processes.NewManager()
statusChan := make(chan *pb.StatusResponse)
s.statusChan = statusChan
inR, inW := io.Pipe()
defer inR.Close()
s.inputPipe = inW
m.session[ref] = s
m.sessionMu.Unlock()
defer func() {
close(statusChan)
m.sessionMu.Lock()
s, ok := m.session[ref]
if ok {
s.statusChan = nil
s.buildOnGoing.Store(false)
}
m.sessionMu.Unlock()
}()
pw := pb.NewProgressWriter(statusChan)
// Build the specified request
ctx, cancel := context.WithCancel(ctx)
defer cancel()
resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil {
s.result = res
s.cancelBuild = cancel
s.buildOptions = req.Options
m.session[ref] = s
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, ref)
}
}
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown key %v", ref)
}
m.sessionMu.Unlock()
if buildErr != nil {
return nil, buildErr
}
if resp == nil {
resp = &client.SolveResponse{}
}
return &pb.BuildResponse{
ExporterResponse: resp.ExporterResponse,
}, nil
}
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
ref := req.Ref
if ref == "" {
return errors.New("status: empty key")
}
// Wait and get status channel prepared by Build()
var statusChan <-chan *pb.StatusResponse
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
statusChan = m.session[ref].statusChan
m.sessionMu.Unlock()
break
}
// forward status
for ss := range statusChan {
if ss == nil {
break
}
if err := stream.Send(ss); err != nil {
return err
}
}
return nil
}
func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
// Get the target ref from init message
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
}
ref := init.Ref
if ref == "" {
return errors.New("input: no ref is provided")
}
// Wait and get input stream pipe prepared by Build()
var inputPipeW *io.PipeWriter
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock()
break
}
// Forward input stream
eg, ctx := errgroup.WithContext(context.TODO())
done := make(chan struct{})
msgCh := make(chan *pb.InputMessage)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() (retErr error) {
defer close(done)
defer func() {
if retErr != nil {
inputPipeW.CloseWithError(retErr)
return
}
inputPipeW.Close()
}()
for {
var msg *pb.InputMessage
select {
case msg = <-msgCh:
case <-ctx.Done():
return errors.Wrap(ctx.Err(), "canceled")
}
if msg == nil {
return nil
}
if data := msg.GetData(); data != nil {
if len(data.Data) > 0 {
_, err := inputPipeW.Write(data.Data)
if err != nil {
return err
}
}
if data.EOF {
return nil
}
}
}
})
return eg.Wait()
}
func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
containerIn, containerOut := ioset.Pipe()
defer func() { containerOut.Close(); containerIn.Close() }()
initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error {
defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() {
if retErr != nil {
initErrCh <- retErr
}
}()
ref := initMessage.Ref
cfg := initMessage.InvokeConfig
m.sessionMu.Lock()
s, ok := m.session[ref]
if !ok {
m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown key %v", ref)
}
m.sessionMu.Unlock()
pid := initMessage.ProcessID
if pid == "" {
return errors.Errorf("invoke: specify process ID")
}
proc, ok := s.processes.Get(pid)
if !ok {
// Start a new process.
if cfg == nil {
return errors.New("no container config is provided")
}
var err error
proc, err = s.processes.StartProcess(pid, s.result, cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
proc.ForwardIO(&containerIn, srvIOCancel)
initDoneCh <- proc
return nil
}, &ioServerConfig{
stdin: containerOut.Stdin,
stdout: containerOut.Stdout,
stderr: containerOut.Stderr,
// TODO: signal, resize
})
})
eg.Go(func() (rErr error) {
defer srvIOCancel()
// Wait for init done
var proc *processes.Process
select {
case p := <-initDoneCh:
proc = p
case err := <-initErrCh:
return err
case <-egCtx.Done():
return egCtx.Err()
}
// Wait for IO done
select {
case <-srvIOCtx.Done():
return srvIOCtx.Err()
case err := <-proc.Done():
return err
case <-egCtx.Done():
return egCtx.Err()
}
})
return eg.Wait()
}

View File

@@ -1,5 +1,5 @@
variable "GO_VERSION" {
default = "1.19"
default = null
}
variable "DOCS_FORMATS" {
default = "md"
@@ -7,6 +7,9 @@ variable "DOCS_FORMATS" {
variable "DESTDIR" {
default = "./bin"
}
variable "GOLANGCI_LINT_MULTIPLATFORM" {
default = null
}
# Special target: https://github.com/docker/metadata-action#bake-definition
target "meta-helper" {
@@ -32,6 +35,17 @@ target "lint" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/lint.Dockerfile"
output = ["type=cacheonly"]
platforms = GOLANGCI_LINT_MULTIPLATFORM != null ? [
"darwin/amd64",
"darwin/arm64",
"linux/amd64",
"linux/arm64",
"linux/s390x",
"linux/ppc64le",
"linux/riscv64",
"windows/amd64",
"windows/arm64"
] : []
}
target "validate-vendor" {
@@ -59,6 +73,13 @@ target "validate-authors" {
output = ["type=cacheonly"]
}
target "validate-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "update-vendor" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -84,6 +105,13 @@ target "update-authors" {
output = ["."]
}
target "update-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "update"
output = ["."]
}
target "mod-outdated" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -142,3 +170,29 @@ target "image-local" {
inherits = ["image"]
output = ["type=docker"]
}
variable "HTTP_PROXY" {
default = ""
}
variable "HTTPS_PROXY" {
default = ""
}
variable "NO_PROXY" {
default = ""
}
target "integration-test-base" {
inherits = ["_common"]
args = {
HTTP_PROXY = HTTP_PROXY
HTTPS_PROXY = HTTPS_PROXY
NO_PROXY = NO_PROXY
}
target = "integration-test-base"
output = ["type=cacheonly"]
}
target "integration-test" {
inherits = ["integration-test-base"]
target = "integration-test"
}

1139
docs/bake-reference.md Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,7 @@ package main
import (
"log"
"os"
"strings"
"github.com/docker/buildx/commands"
clidocstool "github.com/docker/cli-docs-tool"
@@ -26,6 +27,28 @@ type options struct {
formats []string
}
// fixUpExperimentalCLI trims the " (EXPERIMENTAL)" suffix from the CLI output,
// as docs.docker.com already displays "experimental (CLI)",
//
// https://github.com/docker/buildx/pull/2188#issuecomment-1889487022
func fixUpExperimentalCLI(cmd *cobra.Command) {
const (
annotationExperimentalCLI = "experimentalCLI"
suffixExperimental = " (EXPERIMENTAL)"
)
if _, ok := cmd.Annotations[annotationExperimentalCLI]; ok {
cmd.Short = strings.TrimSuffix(cmd.Short, suffixExperimental)
}
cmd.Flags().VisitAll(func(f *pflag.Flag) {
if _, ok := f.Annotations[annotationExperimentalCLI]; ok {
f.Usage = strings.TrimSuffix(f.Usage, suffixExperimental)
}
})
for _, c := range cmd.Commands() {
fixUpExperimentalCLI(c)
}
}
func gen(opts *options) error {
log.SetFlags(0)
@@ -57,6 +80,8 @@ func gen(opts *options) error {
return err
}
case "yaml":
// fix up is needed only for yaml (used for generating docs.docker.com contents)
fixUpExperimentalCLI(cmd)
if err = c.GenYamlTree(cmd); err != nil {
return err
}

View File

@@ -1,48 +1,3 @@
# CI/CD
## GitHub Actions
Docker provides a [GitHub Action that will build and push your image](https://github.com/docker/build-push-action/#about)
using Buildx. Here is a simple workflow:
```yaml
name: ci
on:
push:
branches:
- 'main'
jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v2
with:
push: true
tags: user/app:latest
```
In this example we are also using 3 other actions:
* [`setup-buildx`](https://github.com/docker/setup-buildx-action) action will create and boot a builder using by
default the `docker-container` [builder driver](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver).
This is **not required but recommended** using it to be able to build multi-platform images, export cache, etc.
* [`setup-qemu`](https://github.com/docker/setup-qemu-action) action can be useful if you want
to add emulation support with QEMU to be able to build against more platforms.
* [`login`](https://github.com/docker/login-action) action will take care to log
in against a Docker registry.
This page has moved to [Docker Docs website](https://docs.docker.com/build/ci/)

View File

@@ -1,23 +1,3 @@
# CNI networking
It can be useful to use a bridge network for your builder if for example you
encounter a network port contention during multiple builds. If you're using
the BuildKit image, CNI is not yet available in it, but you can create
[a custom BuildKit image with CNI support](https://github.com/moby/buildkit/blob/master/docs/cni-networking.md).
Now build this image:
```console
$ docker buildx build --tag buildkit-cni:local --load .
```
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) that
will use this image:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "image=buildkit-cni:local" \
--buildkitd-flags "--oci-worker-net=cni"
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#cni-networking)

View File

@@ -1,20 +1,3 @@
# Color output controls
Buildx has support for modifying the colors that are used to output information
to the terminal. You can set the environment variable `BUILDKIT_COLORS` to
something like `run=123,20,245:error=yellow:cancel=blue:warning=white` to set
the colors that you would like to use:
![Progress output custom colors](https://user-images.githubusercontent.com/1951866/180584033-24522385-cafd-4a54-a4a2-18f5ce74eb27.png)
Setting `NO_COLOR` to anything will disable any colorized output as recommended
by [no-color.org](https://no-color.org/):
![Progress output no color](https://user-images.githubusercontent.com/1951866/180584037-e28f9997-dd4c-49cf-8b26-04864815de19.png)
> **Note**
>
> Parsing errors will be reported but ignored. This will result in default
> color values being used where needed.
See also [the list of pre-defined colors](https://github.com/moby/buildkit/blob/master/util/progress/progressui/colors.go).
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/env-vars/#buildkit_colors)

View File

@@ -1,34 +1,3 @@
# Using a custom network
[Create a network](https://docs.docker.com/engine/reference/commandline/network_create/)
named `foonet`:
```console
$ docker network create foonet
```
[Create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
named `mybuilder` that will use this network:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "network=foonet"
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
[Inspect the builder container](https://docs.docker.com/engine/reference/commandline/inspect/)
and see what network is being used:
{% raw %}
```console
$ docker inspect buildx_buildkit_mybuilder0 --format={{.NetworkSettings.Networks}}
map[foonet:0xc00018c0c0]
```
{% endraw %}
This page has moved to [Docker Docs website](https://docs.docker.com/build/drivers/docker-container/#custom-network)

View File

@@ -1,63 +1,3 @@
# Using a custom registry configuration
If you [create a `docker-container` or `kubernetes` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) and
have specified certificates for registries in the [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md),
the files will be copied into the container under `/etc/buildkit/certs` and
configuration will be updated to reflect that.
Take the following `buildkitd.toml` configuration that will be used for
pushing an image to this registry using self-signed certificates:
```toml
# /etc/buildkitd.toml
debug = true
[registry."myregistry.com"]
ca=["/etc/certs/myregistry.pem"]
[[registry."myregistry.com".keypair]]
key="/etc/certs/myregistry_key.pem"
cert="/etc/certs/myregistry_cert.pem"
```
Here we have configured a self-signed certificate for `myregistry.com` registry.
Now [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use this BuildKit configuration:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--config /etc/buildkitd.toml
```
Inspecting the builder container, you can see that buildkitd configuration
has changed:
```console
$ docker exec -it buildx_buildkit_mybuilder0 cat /etc/buildkit/buildkitd.toml
```
```toml
debug = true
[registry]
[registry."myregistry.com"]
ca = ["/etc/buildkit/certs/myregistry.com/myregistry.pem"]
[[registry."myregistry.com".keypair]]
cert = "/etc/buildkit/certs/myregistry.com/myregistry_cert.pem"
key = "/etc/buildkit/certs/myregistry.com/myregistry_key.pem"
```
And certificates copied inside the container:
```console
$ docker exec -it buildx_buildkit_mybuilder0 ls /etc/buildkit/certs/myregistry.com/
myregistry.pem myregistry_cert.pem myregistry_key.pem
```
Now you should be able to push to the registry with this builder:
```console
$ docker buildx build --push --tag myregistry.com/myimage:latest .
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#setting-registry-certificates)

166
docs/guides/debugging.md Normal file
View File

@@ -0,0 +1,166 @@
# Debug monitor
To assist with creating and debugging complex builds, Buildx provides a
debugger to help you step through the build process and easily inspect the
state of the build environment at any point.
> **Note**
>
> The debug monitor is a new experimental feature in recent versions of Buildx.
> There are rough edges, known bugs, and missing features. Please try it out
> and let us know what you think!
## Starting the debugger
To start the debugger, first, ensure that `BUILDX_EXPERIMENTAL=1` is set in
your environment.
```console
$ export BUILDX_EXPERIMENTAL=1
```
To start a debug session for a build, you can use the `buildx debug` command with `--invoke` flag to specify a command to launch in the resulting image.
`buildx debug` command provides `buildx debug build` subcommand that provides the same features as the normal `buildx build` command but allows launching the debugger session after the build.
Arguments available after `buildx debug build` are the same as the normal `buildx build`.
```console
$ docker buildx debug --invoke /bin/sh build .
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
...
Launching interactive container. Press Ctrl-a-c to switch to monitor console
Interactive container was restarted with process "dzz7pjb4pk1mj29xqrx0ac3oj". Press Ctrl-a-c to switch to the new container
Switched IO
/ #
```
This launches a `/bin/sh` process in the final stage of the image, and allows
you to explore the contents of the image, without needing to export or load the
image outside of the builder.
For example, you can use `ls` to see the contents of the image:
```console
/ # ls
bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr work
```
Optional long form allows you specifying detailed configurations of the process.
It must be CSV-styled comma-separated key-value pairs.
Supported keys are `args` (can be JSON array format), `entrypoint` (can be JSON array format), `env` (can be JSON array format), `user`, `cwd` and `tty` (bool).
Example:
```
$ docker buildx debug --invoke 'entrypoint=["sh"],"args=[""-c"", ""env | grep -e FOO -e AAA""]","env=[""FOO=bar"", ""AAA=bbb""]"' build .
```
#### `on` flag
If you want to start a debug session when a build fails, you can use
`--on=error` to start a debug session when the build fails.
```console
$ docker buildx debug --on=error build .
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
...
=> ERROR [shell 10/10] RUN bad-command
------
> [shell 10/10] RUN bad-command:
#0 0.049 /bin/sh: bad-command: not found
------
Launching interactive container. Press Ctrl-a-c to switch to monitor console
Interactive container was restarted with process "edmzor60nrag7rh1mbi4o9lm8". Press Ctrl-a-c to switch to the new container
/ #
```
This allows you to explore the state of the image when the build failed.
#### Launch the debug session directly with `buildx debug` subcommand
If you want to drop into a debug session without first starting the build, you
can use `buildx debug` command to start a debug session.
```
$ docker buildx debug
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
(buildx)
```
You can then use the commands available in [monitor mode](#monitor-mode) to
start and observe the build.
## Monitor mode
By default, when debugging, you'll be dropped into a shell in the final stage.
When you're in a debug shell, you can use the `Ctrl-a-c` key combination (press
`Ctrl`+`a` together, lift, then press `c`) to toggle between the debug shell
and the monitor mode. In monitor mode, you can run commands that control the
debug environment.
```console
(buildx) help
Available commands are:
attach attach to a buildx server or a process in the container
disconnect disconnect a client from a buildx server. Specific session ID can be specified an arg
exec execute a process in the interactive container
exit exits monitor
help shows this message. Optionally pass a command name as an argument to print the detailed usage.
kill kill buildx server
list list buildx sessions
ps list processes invoked by "exec". Use "attach" to attach IO to that process
reload reloads the context and build it
rollback re-runs the interactive container with the step's rootfs contents
```
## Build controllers
Debugging is performed using a buildx "controller", which provides a high-level
abstraction to perform builds. By default, the local controller is used for a
more stable experience which runs all builds in-process. However, you can also
use the remote controller to detach the build process from the CLI.
To detach the build process from the CLI, you can use the `--detach=true` flag with
the build command.
```console
$ docker buildx debug --invoke /bin/sh build --detach=true .
```
If you start a debugging session using the `--invoke` flag with a detached
build, then you can attach to it using the `buildx debug` command to
immediately enter the monitor mode.
```console
$ docker buildx debug
[+] Building 0.0s (1/1) FINISHED
=> [internal] connecting to remote controller
(buildx) list
ID CURRENT_SESSION
xfe1162ovd9def8yapb4ys66t false
(buildx) attach xfe1162ovd9def8yapb4ys66t
Attached to process "". Press Ctrl-a-c to switch to the new container
(buildx) ps
PID CURRENT_SESSION COMMAND
3ug8iqaufiwwnukimhqqt06jz false [sh]
(buildx) attach 3ug8iqaufiwwnukimhqqt06jz
Attached to process "3ug8iqaufiwwnukimhqqt06jz". Press Ctrl-a-c to switch to the new container
(buildx) Switched IO
/ # ls
bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr work
/ #
```

View File

@@ -1,31 +1,3 @@
# OpenTelemetry support
To capture the trace to [Jaeger](https://github.com/jaegertracing/jaeger), set
`JAEGER_TRACE` environment variable to the collection address using a `driver-opt`.
First create a Jaeger container:
```console
$ docker run -d --name jaeger -p "6831:6831/udp" -p "16686:16686" jaegertracing/all-in-one
```
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use the Jaeger instance via the `JAEGER_TRACE` env var:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "network=host" \
--driver-opt "env.JAEGER_TRACE=localhost:6831"
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
Buildx commands should be traced at `http://127.0.0.1:16686/`:
![OpenTelemetry Buildx Bake](https://user-images.githubusercontent.com/1951866/124468052-ef085400-dd98-11eb-84ab-7ac8e261dd52.png)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/opentelemetry/)

Some files were not shown because too many files have changed in this diff Show More