Compare commits

...

458 Commits

Author SHA1 Message Date
CrazyMax
687feca9e8 Merge pull request #1877 from jedevc/cache-dockerutil-features
dockerutil: cache feature collection
2023-06-13 11:42:22 +02:00
Justin Chadwell
d4a2c8d0c3 dockerutil: cache feature collection
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-13 09:47:38 +01:00
Justin Chadwell
bef42b2441 Merge pull request #1876 from tonistiigi/remove-history-request 2023-06-13 09:38:05 +01:00
CrazyMax
2de333fdd3 check history api support once
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-13 10:29:22 +02:00
Tonis Tiigi
1138789f20 avoid extra client for history API detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-12 17:39:09 -07:00
Justin Chadwell
1f4ac09ffb Merge pull request #1874 from thaJeztah/bump_engine_24.0.2 2023-06-09 11:39:13 +01:00
CrazyMax
26a8ffb393 Merge pull request #1873 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@67a08623b95a
2023-06-09 12:14:11 +02:00
CrazyMax
9b7aada99b Merge pull request #1716 from distorhead/use-docker-cli-in-stream
build(cli): allow passing in-stream using command.Cli
2023-06-09 12:11:47 +02:00
Sebastiaan van Stijn
fd6207695b vendor: github.com/docker/cli v24.0.2
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:44 +02:00
Sebastiaan van Stijn
def96d2bf4 vendor: github.com/docker/docker v24.0.2
no changes in vendored files

full diff: https://github.com/docker/docker/compare/v24.0.1...v24.0.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-09 12:07:01 +02:00
Justin Chadwell
f5f00e68ef bake(cli): allow passing in-stream using command.Cli
ReadLocalFiles should allow passing the stdin file as an argument, which
allows us to read from dockerCli.Stdin() to be consistent with other
commands in the same package.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:53 +01:00
Justin Chadwell
14aebe713e debug-shell(cli): allow passing in-stream using command.Cli
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:50:35 +01:00
Justin Chadwell
9d2388e6f5 vendor: update buildkit to master@67a08623b95a
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-09 10:36:52 +01:00
Timofey Kirillov
75e2c46295 build(cli): allow passing in-stream using command.Cli
Use command.Cli::In() which is always initialized either to os.Stdin, or to user-specified stream:
* 5be21394cb/cli/command/cli.go (L494)
* https://github.com/docker/cli/blob/master/cli/command/cli_options.go#L16C1-L26

Signed-off-by: Timofey Kirillov <timofey.kirillov@flant.com>
2023-06-09 12:19:02 +03:00
Justin Chadwell
2c02db8db4 Merge pull request #1835 from ktock/long-form-json 2023-06-08 17:10:37 +01:00
Kohei Tokunaga
e304a05d2a docs: monitor: mention about long form flag
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:03 +01:00
Kohei Tokunaga
14c1ea0e11 invoke: Allow JSON array for long form flags
This commit allows specifying a JSON array to the long-form arg, entrypoint and
envvars.
Non-JSON-array value can still be specified. Buildx treats the value as a JSON
array only when it can be parsed as a JSON array.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-06-08 17:05:02 +01:00
Justin Chadwell
c30bcade2c Merge pull request #1870 from tianon/invoke-entrypoint-fixes 2023-06-08 16:03:24 +01:00
Tianon Gravi
62bfb19db4 Fix a couple --invoke entrypoint interaction bugs
When running `--invoke` against images that have `Cmd` set, the interactions with `Entrypoint` start to cause issues like the following:

    /usr/local/bin/bash: /usr/local/bin/bash: cannot execute binary file

Or:

    sh: can't open 'bash': No such file or directory

This patch fixes those by explicitly setting `Cmd` to be empty if it is unspecified and `Entrypoint` is being set, which matches `docker`'s behavior:

    $ docker image inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' bash
    ["docker-entrypoint.sh"] + ["bash"]
    $ docker create --name foo --entrypoint bash bash
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["bash"] + null
    $ docker rm foo
    $ docker create --name foo bash ls
    $ docker container inspect --format '{{ json .Config.Entrypoint }} + {{ json .Config.Cmd }}' foo
    ["docker-entrypoint.sh"] + ["ls"]

(There are still some weird edge cases in the interaction between the `InvokeConfig` and the original image config, but this fixes the most irritating for me and the rest are going to be deeper changes that are possibly less acceptable. 😅)

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-06-07 16:33:28 -07:00
Justin Chadwell
47e34f2684 Merge pull request #1813 from jedevc/oci-export-on-containerd 2023-06-07 15:19:54 +02:00
Justin Chadwell
3d981be4ad Merge pull request #1850 from ktock/monitor-longhelp 2023-06-07 15:09:35 +02:00
CrazyMax
5d94b0fcc7 Merge pull request #1866 from crazy-max/bump-compose
vendor: github.com/compose-spec/compose-go v1.14.0
2023-06-07 14:55:55 +02:00
CrazyMax
569c66fb62 Merge pull request #1867 from crazy-max/bump-testify
vendor: github.com/stretchr/testify v1.8.4
2023-06-07 12:19:33 +02:00
CrazyMax
93f7fbdd78 Merge pull request #1865 from crazy-max/bump-semver
vendor: github.com/Masterminds/semver/v3 v3.2.1
2023-06-07 12:19:17 +02:00
Tõnis Tiigi
ea06685c11 Merge pull request #1858 from potherca-contrib/fix/typo-Shortand
Fix typo "Shortand" -> "Shorthand"
2023-06-06 14:03:40 -07:00
Ben Peachey
eaba4fa9e6 Update docs with result from make docs command.
Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-06-06 21:43:41 +02:00
CrazyMax
99e3882e2a Merge pull request #1841 from crazy-max/desktop
build: display build details link
2023-06-06 19:06:19 +02:00
CrazyMax
0a2f35970c build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
CrazyMax
ab5f5e4169 build: fix quiet mode
We are using the quiet flag option and we are not taking
progress quiet mode into account

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:47:55 +02:00
Justin Chadwell
696770d29c Merge pull request #1840 from jedevc/fix-check-path-for-bake-cmd-context 2023-06-06 17:49:04 +02:00
CrazyMax
b47b4e5957 vendor: github.com/stretchr/testify v1.8.4
full diff https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:44:58 +02:00
CrazyMax
9a125afba0 vendor: github.com/compose-spec/compose-go v1.14.0
https: //github.com/compose-spec/compose-go/compare/v1.13.4...v1.14.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:34:34 +02:00
Justin Chadwell
d34103b0d9 bake: fix potential context entitlements escape
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
c820350b5e tests: refactor cmd helpers to allow configuring cwd, etc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
61a7854659 tests: add bake test for remote subdir context
Fixed in 12b6a3ad9a, but now we have
regression tests! So we can add a check that we don't break this
behavior again.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
e859ebc12e tests: add bake test for remote cmd context override
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
ef997fd6d0 bake: fix BAKE_CMD_CONTEXT relative path resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
76c96347ff tests: add basic remote bake context test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
48d7dafbd5 git: update gitutil test utilities
- Adds a new GitServeHTTP function to start an http server to serve a
  target git repository.
- Adds a new GitDir helper method to get the path to the .git
  directory
- Updates the GitAdd method to take a variable number of files

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
d03e93f6f1 test: tmpdir can be a test helper
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
fcb7810a38 Merge pull request #1854 from crazy-max/inspect-features 2023-06-06 16:59:58 +02:00
CrazyMax
459d94bdf1 vendor: github.com/Masterminds/semver/v3 v3.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 16:55:59 +02:00
Justin Chadwell
7cef021a8a Merge pull request #1804 from jedevc/fixup-solve 2023-06-06 09:12:10 +02:00
Justin Chadwell
c6db4cf342 build: clarify NewResultHandle comment
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 09:08:30 +02:00
CrazyMax
6c9436fbd5 inspect: display builder features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-05 23:11:58 +02:00
Justin Chadwell
a906149930 Merge pull request #1862 from jedevc/update-contributing-tests
docs: update CONTRIBUTING.md with specific test notes
2023-06-01 21:55:12 +01:00
Justin Chadwell
af328fe413 docs: update CONTRIBUTING.md with specific test notes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-01 11:34:20 +01:00
Justin Chadwell
183a73abae build: docker exporter should instead use oci with containerd
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:59:55 +01:00
Justin Chadwell
b7f0b3d763 build: clear exports for secondary solve request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:56 +01:00
Justin Chadwell
5b27d5a9f6 build: cleanup res if returned in basic build
In practice, this shouldn't happen, but the check is good to include
anyways.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:03 +01:00
Ben Peachey
8f24c58f4d Fix typo in commands/build.go and docs/reference/buildx_build.md
`Shortand` -> `Shorthand`

Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-05-31 13:40:58 +02:00
Justin Chadwell
cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell
8d822fb06c build: move main solve request into main gateway call
Now, we always perform the full solve request in the main gateway call.
This ensures that progress works properly, and makes the lifetime
semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to
always return a single ResultContext, even though the details of how a
gateway is created is different:
- For a failed build, we can just keep the gateway open.
- For a successful build, we immediately open another gateway and
  re-evaluate the build definition in that gateway. This should give an
  instant cache hit (since the build was just successful).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:46:46 +01:00
CrazyMax
0758a9b75d Merge pull request #1856 from crazy-max/history-feature-update
driver: update history api check
2023-05-31 10:27:54 +02:00
CrazyMax
f8fa526678 driver: update history api check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-31 10:10:54 +02:00
Tõnis Tiigi
4abff3ce12 Merge pull request #1852 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@c36941f4a10e
2023-05-30 16:14:26 -07:00
CrazyMax
e7034f66bc Merge pull request #1846 from crazy-max/history-capability
driver: check history capability
2023-05-30 20:25:31 +02:00
CrazyMax
8c65e4fc1d driver: add context to Features interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
d196ac347e driver: cache features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
9b723ece46 driver: check history capability
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:17 +02:00
Justin Chadwell
5e2f8bd64a Merge pull request #1853 from jedevc/logrus-show-error 2023-05-30 10:29:49 +01:00
Justin Chadwell
5788ab33d2 logutil: print error from WithError if found
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 10:00:24 +01:00
CrazyMax
f1788002e1 Merge pull request #1851 from crazy-max/test-flow
test: update workflow
2023-05-30 10:58:17 +02:00
Justin Chadwell
6c62225d1b vendor: update buildkit to master@c36941f4a10e
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 09:10:52 +01:00
CrazyMax
38b4eef5c6 ci: make release and bin-image depend on test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
a4db138c5e docs: update CONTRIBUTING.md to run tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
55377b2b0f make: test-unit and test-integration targets
Also switch test target to run the test script.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
98dedd3225 ci: cache integration tests base and build in dedicated step
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
74b121be66 test: opt to avoid building the integration tests image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
b9cf46785b test: use trap to remove cache volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
ecf8dd0a26 test: type integration not needed
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
73c17ef4d2 ci: send test reports to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
e762e46b4b ci: merge test workflow to build one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
Kohei Tokunaga
cafeedba79 monitor: add long help for commands
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-29 23:23:56 +09:00
CrazyMax
17bdbbd3c3 Merge pull request #1831 from crazy-max/fix-ctn-certs-path
driver(docker-container): fix incorrect path when writing certs
2023-05-26 16:15:21 +02:00
Justin Chadwell
2dae553d18 build: update outline fallback image
Subrequests have been included in docker/dockerfile:1.5 labs, so we can
update the fallback to point to this release.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin Chadwell
91c17f25fb build: tidy up print func
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
CrazyMax
63fc01e08a Merge pull request #1842 from crazy-max/update-xx
update xx to 1.2.1
2023-05-26 11:22:33 +02:00
Justin Chadwell
354ccc9469 Merge pull request #1844 from tonistiigi/imageid-fixes 2023-05-26 10:21:53 +01:00
Tonis Tiigi
68ae67720a build: fix writing correct image ID with -q
Container driver wrote manifest digest that had a
mismatch with --iidfile output.

When --iidfile was set the --metadata-file was not
written.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:20:29 -07:00
Tonis Tiigi
b273db20c3 remote: allow docker exporter on remote driver
There is no reason why one can't make a docker tarball or
load to Docker instance from remote driver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:18:57 -07:00
CrazyMax
0ae88ecc4d update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-25 15:26:56 +02:00
Justin Chadwell
341fb65f6f Merge pull request #1830 from jedevc/fix-missing-driver-in-build 2023-05-25 11:35:01 +01:00
Justin Chadwell
69a9c6609a Merge pull request #1699 from jedevc/bake-attestation-override 2023-05-25 09:54:34 +01:00
CrazyMax
1c96fdaf03 driver(docker-container): fix incorrect path when writing certs
Replace invalid characters in the name of the base directory of
certificates using the name of the registry which can contain
for example a colon when the port is specified.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-23 15:36:38 +02:00
Justin Chadwell
c77bd8a578 build: fix missing "docker" driver name in build progress
This was missing, since the driver property can only be fully populated
after loading nodes from disk. So we add logic to load the nodes, and
check for an error, which ensures that the "docker" driver is always
correctly present in the progress description.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:37:22 +01:00
Justin Chadwell
e5f701351c Merge pull request #1829 from jedevc/vendor-buildkit-master 2023-05-23 10:23:24 +01:00
Justin Chadwell
09798cdebd vendor: update buildkit to master@212ab16a39b1
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:16:09 +01:00
CrazyMax
0dfc35d558 Merge pull request #1822 from thaJeztah/vendor_engine_24.0.1
vendor: github.com/docker/docker and github.com/docker/cli v24.0.1
2023-05-23 11:11:34 +02:00
CrazyMax
8085f57a3a Merge pull request #1825 from thaJeztah/remove_k8s_replace
go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
2023-05-23 11:10:31 +02:00
Sebastiaan van Stijn
d582a21acd go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
Replace rules are not inherited by consumers of buildx as a module, and as
such would default to use the v0.26.2 version. Removing the replace rules
also removes various (indirect) dependencies (although brings in some new
packages from k8s itself).

The "azure" and "gcp" authentication packages in k8s.io/go-client are now
no longer functional, so removing those imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 16:10:49 +02:00
CrazyMax
580820a4de Merge pull request #1824 from thaJeztah/k8s_remove_openstack
cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
2023-05-22 16:09:30 +02:00
Sebastiaan van Stijn
b7e8afc61b cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
Kubernetes v1.26 removed the in-tree openstack cloud provider. While buildx
has a replace rule to pin the k8s dependencies to an older version, such
replace rules are non-transitive, which means that consumers of buildx as
a module could potentially end up with non-resolvable dependencies.

The used code was already non-functional since c41b006be1
(buildx v0.5.0-rc1);

a6b0959276/vendor/k8s.io/client-go/plugin/pkg/client/auth/openstack/openstack_stub.go (L26-L36)

See:

- 9bbf01bae9
- 98e81a7784

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 13:01:54 +02:00
Sebastiaan van Stijn
a8a637e19d vendor: github.com/docker/cli v24.0.1
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:16:37 +02:00
Sebastiaan van Stijn
79632a4c4c vendor: github.com/docker/docker v24.0.1
no changes in vendored files

full diff: https://github.com/moby/moby/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:08:30 +02:00
CrazyMax
a6b0959276 Merge pull request #1440 from crazy-max/gc-policies
builder: add worker gc policies and labels
2023-05-22 11:46:46 +02:00
CrazyMax
6d7142b057 Merge pull request #1820 from jedevc/use-docker-distribution
vendor: use docker/distribution instead of distribution/distribution
2023-05-22 11:17:01 +02:00
Justin Chadwell
7e39644f69 Merge pull request #1816 from bpaquet/fix_session_token 2023-05-22 09:34:07 +01:00
Bertrand Paquet
adc6349b28 Fix AWS Authentication when mixing static creds and IAM profile
When the user supply static creds, we must not enrich them with a
session token which is unrelated.

Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
2023-05-22 09:26:28 +01:00
Justin Chadwell
f558fd8b22 vendor: use docker/distribution instead of distribution/distribution
This doesn't require pulling in a new direct dependency, we already use
docker/distribution throughout imagetools and build.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:24:41 +01:00
Justin Chadwell
432e16ef70 Merge pull request #1806 from thaJeztah/update_engine2 2023-05-22 09:19:35 +01:00
Sebastiaan van Stijn
8c86c2242a vendor: github.com/docker/cli v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:47:38 +02:00
Sebastiaan van Stijn
75ad5d732b vendor: github.com/docker/docker v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:23 +02:00
Sebastiaan van Stijn
9bd0202312 vendor: github.com/spf13/cobra v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:20 +02:00
Sebastiaan van Stijn
367f114cc7 vendor: github.com/opencontainers/runc v1.1.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:43:16 +02:00
Sebastiaan van Stijn
2959ce205e vendor: github.com/klauspost/compress v1.16.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:41:29 +02:00
Sebastiaan van Stijn
75b5c6560f vendor: github.com/moby/term v0.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:39:47 +02:00
Justin Chadwell
4429ccbcc2 Merge pull request #1817 from jedevc/docs-monitor-invoke-parts 2023-05-19 16:40:47 +01:00
Justin Chadwell
c59fc18325 Merge pull request #1770 from jedevc/integration-tests 2023-05-19 16:18:31 +01:00
Justin Chadwell
4ce80856b3 docs: add info for on-error and debug-shell values for invoke
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:51 +01:00
Justin Chadwell
af3feec4ea docs: fixup typo in buildkit console output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:30 +01:00
Justin Chadwell
90c849f5ef bake: merge attest entries
This ensures that `target.attest=["type=sbom,<value>"]` can be
appropriately merged when `--sbom=true` or `--set
target.attest=type=sbom`.

To merge, we simply naively take the last valid value.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:32:58 +01:00
Justin Chadwell
6024212ac8 ci: add integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
2d124e0ce9 test: add basic integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
e61a8cf637 vendor: update buildkit to master@31c870e82a48
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:25:31 +01:00
Justin Chadwell
167cd16acb Merge pull request #1797 from crazy-max/fix-image-create 2023-05-18 09:06:45 +01:00
Tõnis Tiigi
1dd31fefcb Merge pull request #1810 from nicks/patch-1
fix two bugs in bake-reference.md
2023-05-17 21:12:19 -07:00
Nick Santos
5a12b25bab Update bake-reference.md
Before this change, there were two bugs:
- the HCL was not valid. in hcl, argument names can't be quoted
- the target argument should be a real target

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-05-17 19:09:42 -04:00
CrazyMax
b702188b65 imagetools(create): set correct media type when combining manifests
When using imagetools create and combining multiple sources
we should check the media type of each manifest and set
the right media type for the manifest list.

If there is a mismatch we set OCI index as best effort.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-16 11:46:58 +02:00
Justin Chadwell
060ac842bb Merge pull request #1794 from jedevc/fix-printer-start-race 2023-05-12 09:43:58 +01:00
Justin Chadwell
31d1b778ff Merge pull request #1795 from thaJeztah/vendor_distribution_v2.8.2 2023-05-12 09:43:07 +01:00
Sebastiaan van Stijn
1cd4b54810 vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2-beta.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 19:08:16 +02:00
CrazyMax
c54926c5b2 Merge pull request #1767 from jedevc/issue-template-no-bug-feature
github: don't auto-add kind/ labels
2023-05-11 17:22:23 +02:00
Justin Chadwell
10aea8e970 Merge pull request #1791 from jedevc/docs/bake-reference-matrix
Document new bake matrix builds
2023-05-11 15:43:49 +01:00
Justin Chadwell
be6542911f progress: make printer ready only after pausing logrus
This fixes a possible race where messages printed directly after calls
to NewPrinter may appear before the printer starts. With this change, we
delay all of the logs until after.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 15:23:41 +01:00
CrazyMax
9b07f6510a Merge pull request #1793 from jedevc/move-path-resolution-to-controller
controller: move path resolution into controller package
2023-05-11 15:01:21 +02:00
Justin Chadwell
9ee19520dd controller: move path resolution into controller package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:56:05 +01:00
Justin Chadwell
878faae332 docs: add matrix docs for bake
Co-authored-by: David Karlsson <david.karlsson@docker.com
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:44:28 +01:00
Justin Chadwell
eaf38570e7 github: don't auto-add kind/bug labels
kind labels should only be manually added by maintainers after
confirming the issue. Otherwise, it can be tricky to sort through all
the issues when filtering for legitimate bugs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:43:25 +01:00
CrazyMax
167340df17 Merge pull request #1780 from techvelocity/elranshefer/add-support-for-proxy-url
kubernetes driver: add support for proxy-url
2023-05-11 14:40:23 +02:00
Justin Chadwell
e61a1da7fc Merge pull request #1748 from dvdksn/docs/bake-reference
docs: move and rewrite bake reference
2023-05-11 13:00:21 +01:00
David Karlsson
f8483d7243 docs: refactor bake file reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-05-11 12:54:08 +01:00
Justin Chadwell
2c8a9aad76 Merge pull request #1782 from jedevc/docs-monitor
docs: add initial guide about debugging monitor mode
2023-05-11 11:29:21 +01:00
Justin Chadwell
32009a701c Merge pull request #1786 from jedevc/debug-multi-platform
debug: evaluate all refs to allow multi-platform debugging
2023-05-11 09:54:13 +01:00
Justin Chadwell
0cbc316f76 docs: add initial guide about debugging monitor mode
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:53:21 +01:00
Justin Chadwell
45fccef3f3 debug: evaluate all refs to allow multi-platform debugging
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:44:50 +01:00
Justin Chadwell
fdcb4e2fb9 Merge pull request #1784 from jedevc/support-new-compose-features
Support new compose features in bake
2023-05-11 09:44:02 +01:00
Justin Chadwell
4a0a67d7a2 Merge pull request #1783 from jedevc/fix-bake-sub-dir-context
bake: copy remote context contents to root
2023-05-11 09:40:25 +01:00
CrazyMax
855d49ff58 Merge pull request #1788 from crazy-max/update-docker
vendor: github.com/docker/docker v23.0.6
2023-05-10 17:22:14 +02:00
CrazyMax
384e873db0 Merge pull request #1790 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-05-10 16:17:53 +02:00
CrazyMax
60e72ba989 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:43:24 +02:00
CrazyMax
45a2ae6762 vendor: github.com/docker/docker v23.0.6
full diff:
- https://github.com/docker/docker/compare/v23.0.1...v23.0.6
- https://github.com/docker/cli/compare/v23.0.1...v23.0.6

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:26:20 +02:00
Justin Chadwell
2eeef180ea Merge pull request #1746 from ktock/resultcleanup
controller: avoid "context canceled" error on cleanup
2023-05-10 13:24:12 +01:00
Kohei Tokunaga
8fd81f5cfd controller: avoid "context canceled" error on cleanup
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-10 12:56:33 +01:00
Justin Chadwell
1eb9ad979e Merge pull request #1785 from jedevc/default-to-local-controller
controller: default to using local controller
2023-05-10 11:18:55 +01:00
Justin Chadwell
77e0e860f8 controller: default to using local controller
--detach shouldn't be the default yet, since it still has the potential
to leak some sessions in odd edge cases, and is slightly more painful to
debug.

For now, we should set the local controller as the default, with the
idea that we can change it back in the future.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:51:58 +01:00
Justin Chadwell
e228c398f4 bake: support compose additional contexts
compose-go v1.13.0 supports the new additional_contexts to allow passing
additional build context during build, so we should map this to bake's
contexts property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:41:37 +01:00
Justin Chadwell
5d06406f26 bake: support compose inline dockerfile
compose-go v1.13.0 supports the new dockerfile_inline to allow including
a dockerfile inline in the compose definition, so we should map this to
bake's dockerfile-inline property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:39:00 +01:00
Justin Chadwell
cb061b684c Merge pull request #1753 from nicksieger/compose-go-v1.13.4
compose go v1.13.4
2023-05-10 09:25:10 +01:00
Elran Shefer
29b427ce13 kubernetes driver: add support for proxy-url
Signed-off-by: Elran Shefer <elran.shefer@velocity.tech>
2023-05-10 08:36:39 +03:00
Tõnis Tiigi
4fa7cd1fc2 Merge pull request #1772 from tonistiigi/container-hang-fix
container-driver: avoid hanging on container panic
2023-05-09 10:27:11 -07:00
Justin Chadwell
12b6a3ad9a bake: copy remote context contents to root
When resolving remote contexts locally in bake, then we need to ensure
that we properly unpack the contents of that context to the root
directory, instead of leaving it in the subdirectory.

Otherwise, any files will be found in the wrong location. Along with
this change, we also need a change to the dockerfile location lookup to
ensure that it is found at the root instead of in the subdirectory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:19:26 +01:00
Justin Chadwell
22e1901581 bake: reorganize input creation together
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:15:44 +01:00
Justin Chadwell
e23c37fa96 Merge pull request #1771 from jedevc/unset-other-build-client-opt
debug: secondary client build should not reuse solve opts
2023-05-09 14:57:23 +01:00
Justin Chadwell
e5a0ed1149 debug: secondary client build should not reuse solve opt ref
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 14:23:41 +01:00
Justin Chadwell
c9c1303e31 Merge pull request #1777 from ktock/fixerrormsg
monitor: improve error messages
2023-05-09 13:49:07 +01:00
Justin Chadwell
ae3299d9d4 Merge pull request #1628 from cpuguy83/policy_file
Support for passing through raw buildkit policies
2023-05-09 11:50:10 +01:00
CrazyMax
a948cc14c5 Merge pull request #1769 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.1
build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
2023-05-09 11:41:08 +02:00
CrazyMax
621b07c799 Merge pull request #1727 from crazy-max/cmd-completion
cli: add shell completion
2023-05-09 10:59:32 +02:00
Justin Chadwell
7ad970f93a Merge pull request #1776 from tonistiigi/fix-conn-leak
docker: fix connection leak on features check
2023-05-09 09:52:18 +01:00
Kohei Tokunaga
437fe55104 monitor: improve error messages
Print more understandable messages on error:

- When ps fails because the monitor doesn't attach to any session, print "no
  attaching session" instead of "unknown ref".
- Avoid disconnect silently fails when the monitor doesn't attach to any
  session. Print "no attaching session" error instead.
- Fix error message of "attach"'s arguments. ("server name must be passed" ->
  "ID of session or process must be passed")

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-08 15:42:37 +09:00
Tonis Tiigi
bebd244e33 docker: fix connection leak on features check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-07 21:58:19 -07:00
Tonis Tiigi
9f2143e3df container-driver: avoid hanging on container panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-03 17:11:08 -07:00
Brian Goff
98efe7af10 Support for passing through raw buildkit policies
This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-03 23:07:44 +00:00
Justin Chadwell
c7c37c3591 Merge pull request #1760 from ktock/monitor-commands
monitor: Move commands to a separated package
2023-05-03 14:33:08 +01:00
Justin Chadwell
a43837d26c monitor: extend monitor interface from controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-03 22:08:30 +09:00
dependabot[bot]
f115abb509 build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.0 to 5.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](5b4a9f6a9e...284f54f989)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-02 18:59:56 +00:00
Justin Chadwell
43a07f3997 Merge pull request #1597 from robertlestak/k8s-sa
kubernetes driver: add serviceaccount opt
2023-05-02 12:56:07 +01:00
CrazyMax
41e1693be0 Merge pull request #1756 from crazy-max/fix-bin-image
ci: fix bin-image job
2023-04-27 12:20:16 +02:00
Justin Chadwell
9d5af461b2 Merge pull request #1755 from jedevc/move-across-controller-boundary
Move metadata and subrequest output outside of controller
2023-04-27 11:15:54 +01:00
CrazyMax
b38c9c7db4 Merge pull request #1759 from crazy-max/bump-gomod
bump go.mod to 1.20
2023-04-27 09:00:31 +02:00
Kohei Tokunaga
9f884edbbf monitor: use shlex
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:38 +09:00
Kohei Tokunaga
0a7a2b1882 monitor: Move commands to a separated package
This commit moves monitor commands to `monior/commands` package.
Commands still need access to the `monitor` object and buildx controller so this
commit enables this via `Monitor` interface stored in `monitor/types`.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:35 +09:00
Justin Chadwell
6bec8f6e00 docs: add serviceaccount driver-opt for k8s driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:44:03 +01:00
robertlestak
65037e4611 kubernetes driver: add serviceaccount opt
Signed-off-by: robertlestak <robert.lestak@umusic.com>
2023-04-26 16:40:07 +01:00
Justin Chadwell
ba92989a94 controller: print result outside of controller
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2bf996d9ad controller: don't write metadata file in controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
75ed3e296b Merge pull request #1737 from jedevc/improved-controller-progress
controller: refactor progress api
2023-04-26 10:17:00 +01:00
CrazyMax
e14e0521cf Merge pull request #1732 from milas/builder-init-panic
builder: extra init error handling
2023-04-26 09:23:01 +02:00
Tõnis Tiigi
28e6995f7c Merge pull request #1735 from tonistiigi/localstate
build: support for saving local state by build ref
2023-04-25 15:17:08 -07:00
CrazyMax
8f72fb353c bump go.mod to 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 21:45:40 +02:00
Milas Bowman
14f5d490ef builder: extra init error handling
* Return errors from creating the `NodeGroup`
* Ensure that `b.NodeGroup != nil` before reading from
  it during validation

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 11:32:54 -04:00
CrazyMax
c9095e8eab ci: fix bin-image job
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 13:39:00 +02:00
CrazyMax
0589f69206 build: null values support with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
b724a173a9 build: fix JSON formatting for --print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
e5ccb64617 fix: changed attest type from Map to List
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
08d114195f chore: format with prettier
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
caf7d2ec9b build: added attest field
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
David Karlsson
2dffed3f3a build: sort bake fields alphabetically
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
Justin Chadwell
784dc2223d lint: add empty line between console and json code blocks
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 12:05:03 +02:00
CrazyMax
c3fd1e8b79 build: fix links and formatting
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
6f0c550ee9 build: move bake under the build section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
5d551dbbc1 build: customizing builds section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
043cb3a0db build: add bake section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:02 +02:00
Justin Chadwell
16d5b38f2b debug: display build warnings after each build
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 09:58:02 +01:00
Nick Sieger
956a1be656 vendor: update to compose-go 1.13.4
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:29:20 -05:00
Justin Chadwell
afcaa8df5f Merge pull request #1752 from nicksieger/compose-go-default-filenames
bake: use default filenames from compose-go
2023-04-21 17:28:32 +01:00
Nick Sieger
12885c01ad bake: use default filenames from compose-go
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:11:37 -05:00
Justin Chadwell
2ab8749052 controller: replace logrus status messages with progress messages
logrus info messages aren't particularly in-theme with the rest of the
progress output (and are also frustratingly racy). The progress output
is a lot neater, so we refactor it into that.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 12:33:04 +01:00
Justin Chadwell
e826141af4 controller: refactor progress api
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.

This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
  they can be displayed correctly from the client side.

Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.

To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:17:43 +01:00
Justin Chadwell
0c1fd31226 build: refactor out common build command components
We had some duplicated code between the basic runBuild and
launchControllerAndRunBuild.

This patch refactors out the common logic (since it's only really like
to keep growing), and has runBuild call into either the controller or
directly start the build depending on whether BUILDX_EXPERIMENTAL is
set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:13:58 +01:00
Justin Chadwell
0e9804901b controller: refactor status message conversion to pb package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:59:16 +01:00
Justin Chadwell
2402607846 build: use gateway's solve context to allow cancelling getResultAt
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:52:03 +01:00
CrazyMax
3d49bbd23a Merge pull request #1749 from docker/dependabot/github_actions/docker/bake-action-3
build(deps): Bump docker/bake-action from 2 to 3
2023-04-19 11:07:06 +02:00
dependabot[bot]
33b1fdbf39 build(deps): Bump docker/bake-action from 2 to 3
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 2 to 3.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-18 18:10:42 +00:00
Justin Chadwell
de4cdab411 Merge pull request #1747 from jedevc/only-capture-result-for-controller
controller: only capture the result when invoked through a controller
2023-04-18 18:28:08 +01:00
Justin Chadwell
a7e471b7b3 controller: only capture the result when invoked through a controller
This ensures that the code used to capture and evaluated a result is
only executed when built through the controller. Otherwise, no build
result should be recorded.

This ensures that new code added to capture and store the build result
for debugging isn't used when BUILDX_EXPERIMENTAL is not set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 16:35:42 +01:00
Justin Chadwell
ba6e5cddb0 Merge pull request #1745 from jedevc/remote-unneccessary-exec-server-name
monitor: update check for exec args
2023-04-18 11:29:01 +01:00
Justin Chadwell
e4ff82f864 monitor: update exec failure message when no args are passed
"exec" does not take a server name, so we can rename the error check
here.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 11:16:12 +01:00
Justin Chadwell
48b733d6da Merge pull request #1740 from jedevc/resolve-paths-context-dockerfile
build: avoid resolution of dockerfile if context is remote
2023-04-18 10:54:41 +01:00
Justin Chadwell
0b432cc5f2 Merge pull request #1640 from ktock/monitor-invoke-mode-restore
monitor: add `debug-shell` and `on-error`
2023-04-18 10:41:04 +01:00
Justin Chadwell
f6cccefffc build: avoid resolution of dockerfile if context is remote
In 566f41b598, we added a check to ensure
that we avoid resolving http URLs for Dockerfile. However, we have
another circumstance we should not resolve the path in - if the context
is a remote context, the dockerfile is resolved in that context (see
build.go#LoadInputs for more information).

Therefore, we should only resolve the dockerfile to a local directory if
the context is also resolved to a local directory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-17 11:18:49 +01:00
Kohei Tokunaga
fd5d90c699 remove unused fields from ResultContext
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
06399630a2 remove ResultContextError
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
20693aa808 Fix monitor to be aware of Inspect API and invokations from errors
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
f373b91cc3 Add flags and subcommand
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
ce48b1ae84 Enable to restore build options from the server
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:05 +09:00
Kohei Tokunaga
b3340cc7ba Enable to create container from error
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:36:33 +09:00
Kohei Tokunaga
1303715aba Allow passing ResultContext from server to the client through grpcerror
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:11:32 +09:00
Justin Chadwell
b716e48926 Merge pull request #1734 from jedevc/more-ssh-context-fixes
Various context fixups
2023-04-14 09:27:36 +01:00
Justin Chadwell
7d35a3b8d8 Merge pull request #1733 from jedevc/use-dockerui-context-detection
Use dockerui context detection
2023-04-14 09:25:07 +01:00
CrazyMax
200058b505 build: support for saving local state by build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-12 21:37:03 -07:00
Justin Chadwell
566f41b598 build: avoid file resolution of dockerfile urls
Dockerfiles can be HTTP URLs as well as local paths 🤦

We just copy the same logic we use for resolving context paths, and
apply it here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
6c0547e7e6 bake: ensure remote files behind ssh expose agent
The updateContext function may make modifications to the build inputs,
creating either an SSH URL, or an SSH llb.State. In these cases, we need
to ensure that we appropriately expose the client's default agent.

Previously, we would only expose it if the remote context was a git URL,
however, we need to also ensure that if the input was used to override
the context (in the case of ReadRemoteFiles), that we expose the agent
here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
871f865ac8 bake: update ReadRemoteFiles to use buildkit api
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 11:15:25 +01:00
Justin Chadwell
62a21520ea vendor: update buildkit to master@333ee9158128
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 10:11:46 +01:00
CrazyMax
a597266a52 cli: add completion to list builder names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:45:59 +02:00
CrazyMax
14b66817fb build: add completion to list context folders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:35:50 +02:00
CrazyMax
af011d6ca3 bake: add completion to list targets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:34:40 +02:00
CrazyMax
8a02cf8717 cli: set default completion
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:32:48 +02:00
CrazyMax
672eeed9a6 Merge pull request #1731 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-04-11 11:26:32 +02:00
CrazyMax
1b816ff838 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:10:01 +02:00
CrazyMax
10365ddf22 Merge pull request #1723 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.0
build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
2023-04-11 09:39:17 +02:00
dependabot[bot]
a28cb1491d build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.4 to 5.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](38e0b6e68b...5b4a9f6a9e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-05 18:59:48 +00:00
Justin Chadwell
1e149bb84f Merge pull request #1722 from jedevc/bake-matrix-check-map-type
bake: fix map type checking to detect matrix property set to list
2023-04-05 16:03:50 +01:00
Justin Chadwell
9827abbf76 bake: fix map type checking to detect matrix property set to list
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 13:41:03 +01:00
Justin Chadwell
a3293cdaaa Merge pull request #1720 from jedevc/bake-replace-func
Add `replace` function to bake stdlib
2023-04-05 11:59:40 +01:00
Justin Chadwell
f7d8bd2055 Merge pull request #1690 from jedevc/bake-matrix
Implement matrix for bake targets
2023-04-05 11:58:59 +01:00
Justin Chadwell
5d33a3af22 bake: add replace func to stdlib
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
Justin Chadwell
87f900ce77 chore: sort bake stdlib map
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
CrazyMax
bb5c93cafc Merge pull request #1719 from dvdksn/docs/update-readme-links
docs: prune links in README toc, add external reference
2023-04-05 11:30:29 +02:00
David Karlsson
c6ce0964b9 docs: prune links in README toc, add external reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-04-05 11:00:06 +02:00
Justin Chadwell
5c21e80a83 Merge pull request #1711 from jedevc/bake-remote-file-git
bake: forward default ssh agent for remote bake file
2023-04-04 11:50:42 +01:00
CrazyMax
498cc9ba0a Merge pull request #1715 from crazy-max/ci-restrict-repo
ci: check repo origin on push tag event
2023-04-03 21:16:51 +02:00
CrazyMax
805f3a199d ci: check repo origin on push tag event
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-03 20:58:37 +02:00
Justin Chadwell
91fdb0423d Merge pull request #1713 from ktock/monitor-buildapi-options
controller: Extract nested CommonOptions on controller API
2023-04-03 14:48:06 +01:00
Kohei Tokunaga
8ba8659496 controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Justin Chadwell
16e41ba297 Merge pull request #1710 from jedevc/use-buildkit-gitutil-parsegitref
Use buildkit's gitutil package to detect remote git files
2023-04-03 11:14:08 +01:00
Justin Chadwell
387ce5be7c bake: forward default ssh agent for remote bake file
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:50:43 +01:00
Justin Chadwell
87a120e8e3 bake: use build.IsRemoteURL
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:48:31 +01:00
Justin Chadwell
589d4e4cf5 build: use buildkit's gitutil package to detect remote files
BuildKit's gitutil package behaves slightly differently than moby's
urlutil, so we should rely on BuildKit's gitutil when detecting URLs to
avoid cases of accidentally producing invalid build requests that can
confuse users.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:23:59 +01:00
CrazyMax
6535f16aec Merge pull request #1705 from docker/dependabot/go_modules/github.com/opencontainers/runc-1.1.5
build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
2023-04-03 10:53:26 +02:00
Justin Chadwell
a1520ea1b2 bake: additional validation for matrixes
This adds the following constraints to the new features:
- Explicit renaming with the `name` property is *only* permitted when
  used with the `matrix` property.
- Group does not support either `name` or `matrix` (we may choose to
  relax this constraint over time).
- All generated names must be unique.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 09:23:28 +01:00
Tõnis Tiigi
0844213897 Merge pull request #1707 from crazy-max/fix-ls-docs
docs: update ls command output sample
2023-03-30 19:33:55 -07:00
CrazyMax
989ba55d9a docs: update ls command output sample
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 17:54:23 +02:00
CrazyMax
33388d6ede Merge pull request #1552 from crazy-max/moby-buildkit-version
driver: resolve BuildKit version for docker driver
2023-03-30 17:31:47 +02:00
dependabot[bot]
bfadbecb96 build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 15:25:26 +00:00
CrazyMax
f815f4acf7 Merge pull request #1703 from jedevc/vendor-buildkit-master
Bump BuildKit to latest master
2023-03-30 17:23:38 +02:00
CrazyMax
81d7decd13 driver: resolve BuildKit version for docker driver
Buildx currently returns the Docker Engine version for the docker driver
and it can be confusing. This is because BuildKit before 0.11 version does
not support the Info API that returns the BuildKit version.

This change resolves the BuildKit version from the Docker engine one.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 15:52:29 +02:00
Justin Chadwell
d699d08399 bake: add additional tests for matrix behavior
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:39:38 +01:00
Justin Chadwell
9541457c54 vendor: update buildkit to master@8b7bcb900d3c
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:35:49 +01:00
Justin Chadwell
c6cdcb02cf build: replace removed buildkit BUILDINFO field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:24:46 +01:00
Justin Chadwell
799715ea24 Merge pull request #1701 from jedevc/fixup-resolve-paths-context-path
build: fixup resolvePaths for remote context path
2023-03-29 13:59:36 +01:00
Justin Chadwell
b5c6b3f10b build: fixup resolvePaths for remote context path
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-29 10:10:20 +01:00
Tõnis Tiigi
3f59b27cf4 Merge pull request #1698 from crazy-max/fix-issue-template
chore: fix issue template
2023-03-27 09:10:56 -07:00
CrazyMax
00b18558dd chore: fix issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-27 13:30:33 +02:00
Tõnis Tiigi
948414e1b2 Merge pull request #1693 from jedevc/remote-tls-missing-key-cert
remote: avoid tls error if both key and cert are not set
2023-03-23 10:58:12 -07:00
Justin Chadwell
56876ab825 remote: avoid tls error if both key and cert are not set
Previously, we would explicitly error if all TLS parameters were not
available. However, it is a perfectly valid use case to connect to a
buildkit server that only provides TLS in one direction to verify the
server (which is possible today with buildctl).

To support this use case, we only need to error if only one of key or
cert is set, and the other is not - if both are unspecified, the client
will not present a certificate to the server.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-23 10:43:10 +00:00
Justin Chadwell
0806870261 bake: generate implicit groups for matrixes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-22 17:07:04 +00:00
Tõnis Tiigi
fd8eaab2df Merge pull request #1675 from jedevc/source-date-epoch-fixes
SOURCE_DATE_EPOCH build arg injection fixes
2023-03-21 22:29:24 -07:00
Justin Chadwell
77252f161c bake: add matrix to target block
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-21 12:05:14 +00:00
Justin Chadwell
4437802e63 bake: allow overriding name property
Previously, the name property could not be set in the body of a bake
target and could only be set for a label. This patch allows the body to
override the values of label fields, though the default is still the
label.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 17:55:28 +00:00
Justin Chadwell
1613fde55c bake: allow interception to create derived contexts
This patch allows high level clients to define an EvalContext method
which can derive a new context given a block and the base parent
context.

This allows users of the package to intercept evaluation before it
begins, and define additional variables and functions that are bound to
a single block.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 12:18:36 +00:00
CrazyMax
624bc064d8 Merge pull request #1677 from crazy-max/bump-k8s
vendor: bump k8s to v0.25.4
2023-03-19 17:05:55 +00:00
CrazyMax
0c4a68555e Merge pull request #1682 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.4
build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
2023-03-16 09:50:44 +00:00
dependabot[bot]
476ac18d2c build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](2b011faafd...38e0b6e68b)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-15 19:00:53 +00:00
Justin Chadwell
780531425b bake: move SOURCE_DATE_EPOCH parsing to overrides
Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-15 10:06:53 +00:00
CrazyMax
92d2dc8263 k8s: corev1.Handler has been replaced by corev1.ProbeHandler
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
CrazyMax
cfa6b4f7c8 vendor: bump k8s to v0.25.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
Justin Chadwell
5d4223e4f8 build: move SOURCE_DATE_EPOCH parsing into option generation
This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-14 10:21:32 +00:00
Tõnis Tiigi
4a73abfd64 Merge pull request #1673 from crazy-max/fix-k8s-deploy-name
k8s: generate node name if not provided
2023-03-13 17:20:51 -07:00
Justin Chadwell
6f722da04d Merge pull request #1670 from jedevc/version-v-prefix
version: add "v" prefix to version for tagging convention consistency
2023-03-13 08:00:46 +00:00
CrazyMax
527d57540e k8s: generate node name if not provided
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-11 21:22:10 +01:00
Justin Chadwell
b65f49622e version: add "v" prefix to version for tagging convention consistency
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-08 16:25:00 +00:00
Tõnis Tiigi
c5ce08bf3c Merge pull request #1626 from ktock/monitor-exec
monitor: Enable to exec into the container
2023-03-07 17:46:22 +00:00
Tõnis Tiigi
71b35ae42e Merge pull request #1663 from crazy-max/stripcreds
build: strip credentials from remote url on collecting Git provenance info
2023-03-06 16:26:09 +00:00
CrazyMax
15eb6418e8 build: strip credentials from remote url on collecting Git provenance info
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 17:00:05 +01:00
Justin Chadwell
2a83723d57 Merge pull request #1581 from ktock/resolvepath
monitor: resolve paths arguments in client
2023-03-02 11:09:34 +00:00
Kohei Tokunaga
e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:30:29 +09:00
Justin Chadwell
b5ea989eee monitor: resolve oci layout path in client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-02 10:28:56 +00:00
Kohei Tokunaga
17105bfc50 monitor: resolve paths arguments in client
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:13:45 +09:00
CrazyMax
eefe27ff42 Merge pull request #1658 from crazy-max/remove-guides
docs: guides moved to docs website
2023-03-01 10:40:12 +01:00
CrazyMax
1ea71e358a docs: guides moved to docs website
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-01 10:21:02 +01:00
Tõnis Tiigi
14d8f95ec9 Merge pull request #1657 from ktock/argpath
controller: use os.Executable() for getting the exceutable path
2023-03-01 00:12:36 -08:00
Kohei Tokunaga
b0728c96d3 controller: use os.Executable() for getting the exceutable path
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-01 15:49:07 +09:00
Tõnis Tiigi
5e685c0e04 Merge pull request #1650 from crazy-max/dirty-optin
build: makes git dirty check opt-in
2023-02-28 10:49:17 -08:00
CrazyMax
f2ac30f431 Merge pull request #1651 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-02-24 15:43:32 +01:00
CrazyMax
6808c0e585 ci: update docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:13:10 +01:00
CrazyMax
9de12bb9c8 docs: do not generate ref for internal serve command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:12:15 +01:00
CrazyMax
0645acfd79 build: makes git dirty check opt-in
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 13:19:33 +01:00
CrazyMax
439d58ddbd Merge pull request #1037 from crazy-max/bug-template
chore: github issue template
2023-02-24 13:06:55 +01:00
CrazyMax
c0a9274d64 chore: github issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 12:33:24 +01:00
Justin Chadwell
f3a4cd5176 Merge pull request #1614 from jedevc/typed-controller-api
controller: strongly type the controller api
2023-02-24 09:44:45 +00:00
Justin Chadwell
c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:43:15 +00:00
Justin Chadwell
c4ad930e2a Merge pull request #1644 from crazy-max/update-frontend
Dockerfile: align frontend version
2023-02-23 11:00:13 +00:00
CrazyMax
3d0c88695e Merge pull request #1646 from docker/dependabot/go_modules/golang.org/x/net-0.7.0
build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
2023-02-23 11:36:36 +01:00
dependabot[bot]
7332140fdf build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-22 19:50:31 +00:00
CrazyMax
132fababb0 Dockerfile: align frontend version
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 17:47:48 +01:00
CrazyMax
71507c0b58 Merge pull request #1643 from crazy-max/go-1.20
update to go 1.20
2023-02-22 17:42:59 +01:00
CrazyMax
7888fdee58 fix lint issue
should drop = nil from declaration of var n; it is the zero value

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:57:29 +01:00
CrazyMax
fb61fde581 fix cannot set environment variables in parallel tests
> FAIL: TestReadTargets/ArgsOverrides/leaf (0.00s)
> panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:56:46 +01:00
CrazyMax
5258e44030 remove deprecated golangci-lint linters
Linters 'deadcode' and 'varcheck' are deprecated since v1.49.0
and had been replaced by 'unused'

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:40:14 +01:00
CrazyMax
e16c1b289b update golangci-lint to v1.51.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:38:17 +01:00
CrazyMax
376b73f078 update to go 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:37:50 +01:00
Justin Chadwell
1c6060f27d Merge pull request #1612 from cpuguy83/env_no_provenance
Add env var to disable default attestations
2023-02-22 11:36:35 +00:00
Justin Chadwell
ed4fd965ff Merge pull request #1620 from jedevc/remote-controller-fixes
Improvements for remote controller code
2023-02-21 10:52:58 +00:00
Brian Goff
bc9cb2c66a Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-02-16 17:25:40 +00:00
CrazyMax
aa05f4c207 Merge pull request #1584 from dvdksn/docs/build-platform-local
docs: added --platform=local example
2023-02-16 18:08:25 +01:00
CrazyMax
62fbef22d0 Merge pull request #1177 from crazy-max/build-display-builder
build: print instance being used
2023-02-16 18:07:30 +01:00
David Karlsson
2563685d27 docs: added --platform=local example
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-02-16 17:24:45 +01:00
CrazyMax
598f1f0a62 build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
8311b0963a vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
b1949b7388 Merge pull request #1634 from dgageot/fix-1633
build: fix git ambiguous argument
2023-02-16 14:05:32 +01:00
CrazyMax
3341bd1740 Merge pull request #1610 from thaJeztah/update_docker
vendor: github.com/docker/docker, github.com/docker/cli v23.0.0
2023-02-16 13:43:12 +01:00
Justin Chadwell
74f64f88a7 Merge pull request #1617 from jedevc/process-multiple-attestation-annotations
imagetools: process `com.docker.reference.*` annotations
2023-02-16 12:05:20 +00:00
David Gageot
d4a4aaf509 Remove git warning: buildx/1633
Signed-off-by: David Gageot <david.gageot@docker.com>
2023-02-16 07:36:24 +01:00
Justin Chadwell
1f73f4fd5d Merge pull request #1630 from jedevc/debug-help-message-typo
monitor: fix typo in help message
2023-02-16 00:06:06 +00:00
Justin Chadwell
77f83d4171 monitor: fix typo in help message
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-15 14:48:56 +00:00
Justin Chadwell
642f28f439 imagetools: process com.docker.reference.* annotations
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.

Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:24:40 +00:00
Justin Chadwell
54f4dc8f6e controller: set absolute path of server binary before execution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:21:18 +00:00
CrazyMax
89d99b1694 inspect: display labels and gc policies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:56 +01:00
CrazyMax
9753f63f57 builder: add worker gc policies and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:15 +01:00
CrazyMax
04804ff355 Merge pull request #1557 from crazy-max/gomod-1.19
go.mod: bump to Go 1.19
2023-02-11 22:47:51 +01:00
Justin Chadwell
ed9ea2476d controller: use unique files per buildx version
This ensures that we should never accidentally connect to a server with
a mismatched version, while also allowing us to run multiple buildx
servers at a time.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:23 +00:00
Justin Chadwell
d0d29168a5 controller: use grpc with contexts for improved timeouts
This patch improves timeout logic for testing and creating a buildx
server. Instead of needing to have a custom loop, we can just use the
primitives provided by go and grpc for easier handling.

Additionally, without the explicit time.Sleeps, we defer to GRPCs
exponential backoff algorithm which should provide substantially better
results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:19 +00:00
Justin Chadwell
abda257763 controller: exit cleanly on SIGTERM
This signal may be sent using an external tool such as pkill, and since
we can handle it neatly, we should.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:15 +00:00
Justin Chadwell
1b91bc2e02 controller: add more informative server exit messages
When exiting, we should ideally always print a message, and give details
as to exactly what error we received.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:26 +00:00
Justin Chadwell
56b9e785e5 build: don't kill remote controller after build
We don't know if other builds might be running, etc, so we should allow
the server to decide when to exit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:25 +00:00
Sebastiaan van Stijn
081447c9b1 vendor: github.com/docker/cli v23.0.0
full diff: https://github.com/docker/cli/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:32:57 +01:00
Sebastiaan van Stijn
260117289b vendor: github.com/docker/docker v23.0.0
full diff: https://github.com/docker/docker/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:30:17 +01:00
Sebastiaan van Stijn
73dca749ca vendor: golang.org/x/oauth2 v0.1.0
no changes in vendored code

full diff: https://github.com/golang/oauth2/compare/6fdb5e3db783...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:26:17 +01:00
Sebastiaan van Stijn
8ac380bfb3 vendor: golang.org/x/net v0.5.0
full diff: https://github.com/golang/net/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:21:05 +01:00
Sebastiaan van Stijn
aeac7e08f9 vendor: golang.org/x/term v0.4.0
no changes in vendored code

full diff: https://github.com/golang/term/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:20:00 +01:00
Sebastiaan van Stijn
7c9cdc4353 vendor: golang.org/x/text v0.6.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:19:53 +01:00
Sebastiaan van Stijn
67572785cf vendor: golang.org/x/sys v0.4.0
full diff: https://github.com/golang/sys/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:12:32 +01:00
Sebastiaan van Stijn
8a70e7634d vendor: github.com/google/certificate-transparency-go v1.1.4
full diff:

- https://github.com/google/certificate-transparency-go/compare/v1.0.21...v1.1.4
- https://github.com/kubernetes/klog/compare/v2.30.0...v2.80.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:03:26 +01:00
Sebastiaan van Stijn
6dd5589a9c vendor: github.com/containerd/containerd v1.6.16
full diff: https://github.com/containerd/containerd/compare/1709cfe273d9...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 14:55:44 +01:00
CrazyMax
78058ce5f3 Merge pull request #1592 from crazy-max/fix-build-git
build: check reachable git commits
2023-02-08 14:23:52 +01:00
CrazyMax
fd5884189c build: check reachable git commits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-08 12:46:04 +01:00
Justin Chadwell
ab7a9f008d Merge pull request #1607 from jedevc/bake-fix-error-traces
bake: avoid nesting error diagnostics
2023-02-07 13:20:07 +00:00
CrazyMax
a8eb2a7fbe gitutil: override the locale to ensure consistent output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-07 13:26:28 +01:00
Justin Chadwell
fbb4f4dec8 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-07 11:40:20 +00:00
Justin Chadwell
46fd0a61ba Merge pull request #1605 from jedevc/bake-fix-errors
bake: avoid early-exit for resolution failures
2023-02-07 10:36:05 +00:00
Justin Chadwell
6444c813dc Merge pull request #1601 from jedevc/buildx-document-attestation-booleans
build: add docs for boolean attestation flags
2023-02-06 15:15:01 +00:00
Justin Chadwell
dc8a2b0398 bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 15:05:45 +00:00
CrazyMax
d9780e27cd Merge pull request #1602 from crazy-max/rm-releases-json
ci: remove releases-json workflow
2023-02-06 13:44:57 +01:00
CrazyMax
ab44d03771 ci: remove releases-json workflow
This is now directly handled by our actions-toolkit. We keep
`.github/releases.json` file for now but will be removed on
next v0.11 release.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
crazy-max
b53cb256e5 github: update .github/releases.json
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
CrazyMax
c3075923f4 Merge pull request #1591 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.9.0
2023-02-06 13:28:24 +01:00
CrazyMax
a32881313b Merge pull request #1596 from crazy-max/builder-fix-validation
builder: fix docker context not validated
2023-02-06 12:31:04 +01:00
Justin Chadwell
07548bc898 build: add docs for boolean attestation flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 10:53:49 +00:00
CrazyMax
0e544fe835 builder: fix docker context not validated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 21:23:29 +01:00
CrazyMax
21ac4c34fb update github.com/compose-spec/compose-go to v1.9.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 03:19:59 +01:00
Justin Chadwell
d2fa4a5724 Merge pull request #1582 from jedevc/lint-fixups
Linting tidy-ups
2023-02-01 16:08:07 +00:00
Justin Chadwell
4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:31:01 +00:00
Justin Chadwell
5da09f0c23 lint: enable gosec
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:50 +00:00
Justin Chadwell
48357ee0c6 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:43 +00:00
Justin Chadwell
6506166f02 Merge pull request #1579 from jedevc/refactor-controllers
Refactor controllers, moving them into a separate package
2023-02-01 09:57:32 +00:00
Justin Chadwell
5f130b25ad hack: move syntax directive to top of generated-files
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
cb94298a02 Merge pull request #1296 from ktock/monitor-list
monitor: Enable to run build and invoke in background
2023-01-31 14:14:57 +00:00
CrazyMax
046084c0b8 Merge pull request #1574 from crazy-max/ci-fix-workflow
ci: fix pr branch name in releases-json workflow
2023-01-31 15:00:01 +01:00
CrazyMax
18760253b9 ci: fix pr branch name in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 13:51:30 +01:00
CrazyMax
ded6376ece Merge pull request #1573 from crazy-max/fix-releases-json
ci: add workflow_dispatch in releases-json workflow
2023-01-31 13:48:33 +01:00
CrazyMax
a4d60a451d ci: add workflow_dispatch in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 11:38:46 +01:00
Justin Chadwell
0f4030de5d Merge pull request #1572 from crazy-max/ci-update-ver
ci: update buildx and buildkit to latest
2023-01-31 10:22:58 +00:00
CrazyMax
f1a5a3ec50 ci: update buildx and buildkit to latest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 22:13:27 +01:00
CrazyMax
87beaefbb8 Merge pull request #1571 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-01-30 21:18:54 +01:00
CrazyMax
451847183d ci: fix docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 21:01:27 +01:00
CrazyMax
7625a3a4b0 Merge pull request #1568 from crazy-max/silent-git-no-upstream
build: silently fail if git remote not found
2023-01-30 20:32:12 +01:00
CrazyMax
6db696748b build: better message output for git provenance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 20:24:09 +01:00
Tõnis Tiigi
14f9ae679d Merge pull request #1566 from tonistiigi/fix-result-descriptor
build: fix multi-node merge to read descriptor from result
2023-01-30 09:04:40 -08:00
CrazyMax
4789d2219c build: silently fail if git remote not found
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 14:33:16 +01:00
CrazyMax
eacecf657c Merge pull request #1561 from crazy-max/fix-preferred-platform
build: fix preferred platform not taken into account
2023-01-30 12:22:08 +01:00
CrazyMax
1de0be240f Merge pull request #1563 from crazy-max/releases-json
ci: generate releases.json on release event
2023-01-30 11:51:08 +01:00
CrazyMax
ea4bec2bad github: update .github/releases.json
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
CrazyMax
36d95bd3b9 ci: generate releases.json on release event
We are currently using the GitHub API in our setup-buildx-action
to check for latest and tagged releases to make sure they exist
before download. But this requires using a token to avoid
rate-limit. It's fine for public runners but GHES runners don't
have the `github.token` populated automatically. They need to
create a PAT.

This PR will solve this issue by generating and pushing a
`releases.json` file in this repo when we publish a GitHub Release
that will then be fetched through `raw.githubusercontent.com`
endpoint on `setup-buildx-action` repo. This endpoint is better
served for our purpose with 5000 requests per hour compared to the
GitHub API endpoint that is limited to 60 requests per hour (unauth)
and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an
action in a workflow run can't trigger a new workflow run anyway.
See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
Tonis Tiigi
c33b310b48 build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-29 22:52:41 -08:00
Justin Chadwell
8af76c68a8 Merge pull request #1564 from AkihiroSuda/fix-1562
bake: SOURCE_DATE_EPOCH: fix `panic: assignment to entry in nil map`
2023-01-29 22:07:20 +00:00
Akihiro Suda
1f56f51740 bake: SOURCE_DATE_EPOCH: fix panic: assignment to entry in nil map
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-29 08:43:06 +09:00
CrazyMax
49b3c0dba5 build: fix preferred platform not taken account
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-27 20:02:13 +01:00
Tõnis Tiigi
a718d07f64 Merge pull request #1559 from tonistiigi/update-buildkit-v0.11.2
vendor: update buildkit to v0.11.2
2023-01-26 10:57:21 -08:00
Tonis Tiigi
f6da7ee135 vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-26 10:33:20 -08:00
CrazyMax
7eb266de69 go.mod: bump to Go 1.19
Go 1.20 will be there soon, I think it's time to move our go.mod to
latest stable. We can then remove the compat in our vendor.Dockerfile

Downstream projects like compose or github.com/linuxkit/linuxkit should
not be affected.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-26 16:08:09 +01:00
CrazyMax
9f821dabeb Merge pull request #1548 from crazy-max/git-ls-remote
build: set remote origin url
2023-01-26 11:03:18 +01:00
Kohei Tokunaga
a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-26 17:17:07 +09:00
Tõnis Tiigi
b1b4e64c97 Merge pull request #1504 from jedevc/fix-1497
Refactor BuildWithResultHandler to simplify concurrency usage
2023-01-25 10:52:17 -08:00
CrazyMax
c1058c17aa build: set remote origin url
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-25 18:15:05 +01:00
Tõnis Tiigi
059c347fc2 Merge pull request #1549 from jedevc/silence-intoto-warnings
imagetools: silence intoto warnings
2023-01-24 17:03:49 -08:00
Justin Chadwell
7145e021f9 imagetools: silence intoto warnings
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 16:49:28 +00:00
CrazyMax
9723f4f76c Merge pull request #1547 from crazy-max/fix-rm
rm: do not check for context builders when removing inactive
2023-01-24 13:39:36 +01:00
Justin Chadwell
db72d0cc05 Merge pull request #1543 from jedevc/bump-ci-buildkit
ci: update buildkit release version in build pipeline
2023-01-24 12:17:34 +00:00
Justin Chadwell
00b7d5b858 Merge pull request #1505 from jedevc/inspect-lazy-attestations
Lazily load attestation data in `imagetools inspect`
2023-01-24 12:10:11 +00:00
CrazyMax
6cd0c11ab1 rm: do not check for context builders when removing inactive
This change has been introduced in e7b5ee7518
but we should not check context builders when removing inactive
ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-24 13:08:18 +01:00
Justin Chadwell
c1ab55a3f2 ci: update buildkit release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 11:47:58 +00:00
Justin Chadwell
c756e3ba96 Merge pull request #1534 from jedevc/readme-minimum-docker-version
docs: add note about minimum docker version
2023-01-24 11:02:21 +00:00
Justin Chadwell
566f37b65b docs: add note about minimum docker version
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 10:25:58 +00:00
CrazyMax
6d1ff27410 Merge pull request #1538 from tonistiigi/caps-req-internal
build: mark capabilities request as internal
2023-01-24 11:12:37 +01:00
Tonis Tiigi
be55b41427 build: mark capabilities request as internal
So it doesn't show up in the History API.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-24 00:48:35 -08:00
Tõnis Tiigi
a4f01b41a4 Merge pull request #1529 from thaJeztah/update_buildkit
vendor: github.com/moby/buildkit v0.11.1
2023-01-18 12:56:26 -08:00
Sebastiaan van Stijn
01e1c28dd9 vendor: github.com/moby/buildkit v0.11.1
full diff: https://github.com/moby/buildkit/compare/v0.11.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 20:56:54 +01:00
Tõnis Tiigi
51e41b16db Merge pull request #1511 from jedevc/error-on-attestations-docker
build: error when using docker exporter and attestations
2023-01-17 09:57:48 -08:00
Justin Chadwell
9e9cdc2e6d Merge pull request #1525 from jedevc/ci-bump-buildx
ci: update buildx release version in build pipeline
2023-01-17 16:39:39 +00:00
Justin Chadwell
bc1d590ca7 ci: update buildx release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 15:21:52 +00:00
Justin Chadwell
900d9c294d Merge pull request #1523 from jedevc/docs-reference-attest
docs: add reference for new attest family of flags
2023-01-17 13:47:10 +00:00
CrazyMax
65aac16139 Merge pull request #1512 from nicks/patch-1
docs: small tweak to opentelemetry
2023-01-17 13:30:33 +01:00
Justin Chadwell
4903f462f6 docs: add reference for new attest family of flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 11:45:38 +00:00
Justin Chadwell
44b5a19c13 Merge pull request #1516 from dvdksn/docs/fix-broken-bake-link-in-cli-reference
docs: fix broken link in buildx_bake CLI reference
2023-01-17 11:11:54 +00:00
David Karlsson
ba8fa6c403 docs: fix broken link in buildx_bake CLI reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-01-13 15:33:31 +01:00
CrazyMax
5b3083e9e1 Merge pull request #1515 from crazy-max/fix-docs-release
ci: fix typo in docs-release workflow
2023-01-13 15:16:16 +01:00
CrazyMax
523a16aa35 ci: fix typo in docs-release workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-13 15:10:03 +01:00
Justin Chadwell
43a748fd15 build: error when using docker exporter and attestations
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:41:53 +00:00
Justin Chadwell
15a80b56b5 build: avoid compatability error when attestations disabled
We should avoid erroring with attestations support compatability errors
when a user has specified --provenance=false.

A user may wish to enable --provenance=false that works across buildkit
versions, but currently it will fail on old versions - this patch fixes
this, to silently ignore the provenance flag for this check if it's set
to disabled.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:36:55 +00:00
Nick Santos
b14bfb9fa2 docs: small tweak to opentelemetry
The buildx builder persists across Desktop restarts. So when we start up Jaeger, we should recommend that people configure it to persist across restarts as well.

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-01-11 12:09:05 -05:00
Justin Chadwell
56950ece69 inspect: lazily load attestation data
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.

With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 13:00:42 +00:00
Justin Chadwell
1d2ac78443 inspect: move attestation loading to struct methods
This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 12:40:42 +00:00
Justin Chadwell
8b7aa1a168 build: create error group per opt
Using the syncronization primitive, we can avoid needing to create a
separate wait group.

This allows us to sidestep the issue where the wait group could be
completed, but the build invocation functions had not terminated - if
one of the functions was to terminate with an error, then it was
possible to encounter a race condition, where the result handling code
would begin executing, despite an error.

The refactor to use a separate error group which more elegantly handles
the concept of function returns and errors, ensures that we can't
encounter this issue.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 11:02:27 +00:00
Justin Chadwell
1180d919f5 build: reorder error group funcs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
347417ee12 build: use copy for BuildWithResultHandler loop vars
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
fb27e3f919 Merge pull request #1502 from developer-guy/fix/readme
fix the directory of the buildx binary
2023-01-10 10:28:04 +00:00
Batuhan Apaydın
edb16f8aab fix the directory of the buildx binary
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2023-01-10 11:26:11 +03:00
Tõnis Tiigi
5c56e947fe Merge pull request #1500 from tonistiigi/update-buildkit-v0.11.0
github: update CI to buildkit v0.11
2023-01-09 15:59:25 -08:00
Tonis Tiigi
571871b084 github: update CI to buildkit v0.11
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-09 15:50:56 -08:00
Tõnis Tiigi
8340c40647 Merge pull request #1498 from jedevc/attestation-printing
Improved attestation inspect
2023-01-09 11:22:20 -08:00
Justin Chadwell
9818055b0e docs: update with new inspect output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:07:10 +00:00
Justin Chadwell
484823c97d inspect: change additional spdxs to not have duplicates
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:03:23 +00:00
Justin Chadwell
3ce17b01dc inspect: provide access to multiple spdx documents
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:42:26 +00:00
Justin Chadwell
e68c566c1c inspect: parse sbom and provenance into json structs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:43 +00:00
Justin Chadwell
19d16aa941 inspect: break after first matching attestation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:06 +00:00
CrazyMax
6852713121 Merge pull request #1494 from thaJeztah/docs_update_docs_tools
go.mod: update cli-docs-tool v0.5.1
2023-01-09 13:14:01 +01:00
Sebastiaan van Stijn
c97500b117 go.mod: update cli-docs-tool v0.5.1 and re-generate docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:03:01 +01:00
CrazyMax
85040a9067 Merge pull request #1493 from thaJeztah/docs_fix_anchors
docs: update anchor links
2023-01-09 12:19:58 +01:00
Sebastiaan van Stijn
b8285c17e6 docs: update anchor links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-07 15:12:08 +01:00
Tõnis Tiigi
332dfb4b92 Merge pull request #1444 from crazy-max/inspect-attest
imagetools inspect: handle provenance and sboms
2023-01-06 16:29:33 -08:00
Tõnis Tiigi
cb279bb14b Merge pull request #1491 from jedevc/ociindex-refactor
vendor: update buildkit to v0.11.0-rc4
2023-01-06 16:29:18 -08:00
Justin Chadwell
60c9cf74ce vendor: update buildkit to v0.11.0-rc4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-06 15:51:17 -08:00
Tõnis Tiigi
ff6754eb04 Merge pull request #1456 from jedevc/oci-layout-reference-parsing
build: refactor reference parsing for oci image layouts
2023-01-05 23:50:25 -08:00
CrazyMax
e6b9aba997 imagetools inspect: handle provenance and sbom
use stub structs for SLSA/SBOM while waiting for
go-imageinspect library to be public.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 17:34:30 +01:00
CrazyMax
0302894bfb Merge pull request #1463 from crazy-max/e2e-k3s
e2e: use native k3s installation script
2023-01-05 16:50:45 +01:00
CrazyMax
e46394c3be e2e: use native k3s installation script
debianmaster/actions-k3s action gives some warnings in our e2e
workflow. This commit brings https://github.com/debianmaster/actions-k3s/blob/master/index.js
directly in the workflow through actions/github-script with
some changes to properly wait for nodes to be up.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 15:37:19 +01:00
Justin Chadwell
1885e41789 docs: update oci layout with tag resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-05 13:47:42 +00:00
CrazyMax
2fb9db994b imagetools inspect: missing annotations key
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-04 15:52:16 +01:00
Tõnis Tiigi
287aaf1696 Merge pull request #1482 from AkihiroSuda/auto-propagate-source-date-epoch
Propagate SOURCE_DATE_EPOCH from the client env
2022-12-27 17:32:08 -08:00
Akihiro Suda
0e6f5a155e Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-24 01:32:15 +09:00
Justin Chadwell
70682b043e build: refactor reference parsing for image layouts
We allow any valid image reference format for the oci-layout, not just
limiting to name@digest, we additionally allow images of the form
name:tag@digest now.

The name of the reference is used to find the local directory to lookup
the store in, while the tag and digest are attached to a random identity
to generate the dummy reference sent to the oci-layout context.

This separation of the target to replace and the value to replace it
with ensures that any tag or digest set in the client is properly sent
across to the server. The tag is used when a digest was not specified,
and it is resolved in the context of the local directory before being
sent, using the same helpers as we use for the local cache expoter.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-14 16:53:10 +00:00
2766 changed files with 271355 additions and 68388 deletions

View File

@@ -116,6 +116,60 @@ commit automatically with `git commit -s`.
### Run the unit- and integration-tests
Running tests:
```bash
make test
```
This runs all unit and integration tests, in a containerized environment.
Locally, every package can be tested separately with standard Go tools, but
integration tests are skipped if local user doesn't have enough permissions or
worker binaries are not installed.
```bash
# run unit tests only
make test-unit
# run integration tests only
make test-integration
# test a specific package
TESTPKGS=./bake make test
# run all integration tests with a specific worker
TESTFLAGS="--run=//worker=remote -v" make test-integration
# run a specific integration test
TESTFLAGS="--run /TestBuild/worker=remote/ -v" make test-integration
# run a selection of integration tests using a regexp
TESTFLAGS="--run /TestBuild.*/worker=remote/ -v" make test-integration
```
> **Note**
>
> Set `TEST_KEEP_CACHE=1` for the test framework to keep external dependant
> images in a docker volume if you are repeatedly calling `make test`. This
> helps to avoid rate limiting on the remote registry side.
> **Note**
>
> Set `TEST_DOCKERD=1` for the test framework to enable the docker workers,
> specifically the `docker` and `docker-container` drivers.
>
> The docker tests cannot be run in parallel, so require passing `--parallel=1`
> in `TESTFLAGS`.
> **Note**
>
> If you are working behind a proxy, you can set some of or all
> `HTTP_PROXY=http://ip:port`, `HTTPS_PROXY=http://ip:port`, `NO_PROXY=http://ip:port`
> for the test framework to specify the proxy build args.
### Run the helper commands
To enter a demo container environment and experiment, you may run:
```

124
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,124 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Bug Report
description: Report a bug
labels:
- status/triage
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to report a bug!
If this is a security issue please report it to the [Docker Security team](mailto:security@docker.com).
- type: checkboxes
attributes:
label: Contributing guidelines
description: |
Please read the contributing guidelines before proceeding.
options:
- label: I've read the [contributing guidelines](https://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree
required: true
- type: checkboxes
attributes:
label: I've found a bug and checked that ...
description: |
Make sure that your request fulfills all of the following requirements.
If one requirement cannot be satisfied, explain in detail why.
options:
- label: ... the documentation does not mention anything about my problem
- label: ... there are no open or closed issues that are related to my problem
- type: textarea
attributes:
label: Description
description: |
Please provide a brief description of the bug in 1-2 sentences.
validations:
required: true
- type: textarea
attributes:
label: Expected behaviour
description: |
Please describe precisely what you'd expect to happen.
validations:
required: true
- type: textarea
attributes:
label: Actual behaviour
description: |
Please describe precisely what is actually happening.
validations:
required: true
- type: input
attributes:
label: Buildx version
description: |
Output of `docker buildx version` command.
Example: `github.com/docker/buildx v0.8.1 5fac64c2c49dae1320f2b51f1a899ca451935554`
validations:
required: true
- type: textarea
attributes:
label: Docker info
description: |
Output of `docker info` command.
render: text
- type: textarea
attributes:
label: Builders list
description: |
Output of `docker buildx ls` command.
render: text
validations:
required: true
- type: textarea
attributes:
label: Configuration
description: >
Please provide a minimal Dockerfile, bake definition (if applicable) and
invoked commands to help reproducing your issue.
placeholder: |
```dockerfile
FROM alpine
echo hello
```
```hcl
group "default" {
targets = ["app"]
}
target "app" {
dockerfile = "Dockerfile"
target = "build"
}
```
```console
$ docker buildx build .
$ docker buildx bake
```
validations:
required: true
- type: textarea
attributes:
label: Build logs
description: |
Please provide logs output (and/or BuildKit logs if applicable).
render: text
validations:
required: false
- type: textarea
attributes:
label: Additional info
description: |
Please provide any additional information that could be useful.

12
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
blank_issues_enabled: true
contact_links:
- name: Questions and Discussions
url: https://github.com/docker/buildx/discussions/new
about: Use Github Discussions to ask questions and/or open discussion topics.
- name: Command line reference
url: https://docs.docker.com/engine/reference/commandline/buildx/
about: Read the command line reference.
- name: Documentation
url: https://docs.docker.com/build/
about: Read the documentation.

15
.github/ISSUE_TEMPLATE/feature.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Feature request
description: Missing functionality? Come tell us about it!
labels:
- kind/enhancement
- status/triage
body:
- type: textarea
id: description
attributes:
label: Description
description: What is the feature you want to see?
validations:
required: true

12
.github/SECURITY.md vendored Normal file
View File

@@ -0,0 +1,12 @@
# Reporting security issues
The project maintainers take security seriously. If you discover a security
issue, please bring it to their attention right away!
**Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Security reports are greatly appreciated, and we will publicly thank you for it.
We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
ruling it out in the future.

735
.github/releases.json vendored Normal file
View File

@@ -0,0 +1,735 @@
{
"latest": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.2": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.1": {
"id": 90346950,
"tag_name": "v0.10.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/checksums.txt"
]
},
"v0.10.0": {
"id": 88388110,
"tag_name": "v0.10.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/checksums.txt"
]
},
"v0.10.0-rc3": {
"id": 88191592,
"tag_name": "v0.10.0-rc3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/checksums.txt"
]
},
"v0.10.0-rc2": {
"id": 86248476,
"tag_name": "v0.10.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/checksums.txt"
]
},
"v0.10.0-rc1": {
"id": 85963900,
"tag_name": "v0.10.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/checksums.txt"
]
},
"v0.9.1": {
"id": 74760068,
"tag_name": "v0.9.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/checksums.txt"
]
},
"v0.9.0": {
"id": 74546589,
"tag_name": "v0.9.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/checksums.txt"
]
},
"v0.9.0-rc2": {
"id": 74052235,
"tag_name": "v0.9.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/checksums.txt"
]
},
"v0.9.0-rc1": {
"id": 73389692,
"tag_name": "v0.9.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/checksums.txt"
]
},
"v0.8.2": {
"id": 63479740,
"tag_name": "v0.8.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/checksums.txt"
]
},
"v0.8.1": {
"id": 62289050,
"tag_name": "v0.8.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/checksums.txt"
]
},
"v0.8.0": {
"id": 61423774,
"tag_name": "v0.8.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/checksums.txt"
]
},
"v0.8.0-rc1": {
"id": 60513568,
"tag_name": "v0.8.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/checksums.txt"
]
},
"v0.7.1": {
"id": 54098347,
"tag_name": "v0.7.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/checksums.txt"
]
},
"v0.7.0": {
"id": 53109422,
"tag_name": "v0.7.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/checksums.txt"
]
},
"v0.7.0-rc1": {
"id": 52726324,
"tag_name": "v0.7.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/checksums.txt"
]
},
"v0.6.3": {
"id": 48691641,
"tag_name": "v0.6.3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-arm64.exe"
]
},
"v0.6.2": {
"id": 48207405,
"tag_name": "v0.6.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-arm64.exe"
]
},
"v0.6.1": {
"id": 47064772,
"tag_name": "v0.6.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-arm64.exe"
]
},
"v0.6.0": {
"id": 46343260,
"tag_name": "v0.6.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-arm64.exe"
]
},
"v0.6.0-rc1": {
"id": 46230351,
"tag_name": "v0.6.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-arm64.exe"
]
},
"v0.5.1": {
"id": 35276550,
"tag_name": "v0.5.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.windows-amd64.exe"
]
},
"v0.5.0": {
"id": 35268960,
"tag_name": "v0.5.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.windows-amd64.exe"
]
},
"v0.5.0-rc1": {
"id": 35015334,
"tag_name": "v0.5.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.windows-amd64.exe"
]
},
"v0.4.2": {
"id": 30007794,
"tag_name": "v0.4.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.windows-amd64.exe"
]
},
"v0.4.1": {
"id": 26067509,
"tag_name": "v0.4.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.windows-amd64.exe"
]
},
"v0.4.0": {
"id": 26028174,
"tag_name": "v0.4.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.windows-amd64.exe"
]
},
"v0.3.1": {
"id": 20316235,
"tag_name": "v0.3.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.windows-amd64.exe"
]
},
"v0.3.0": {
"id": 19029664,
"tag_name": "v0.3.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.windows-amd64.exe"
]
},
"v0.2.2": {
"id": 17671545,
"tag_name": "v0.2.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.windows-amd64.exe"
]
},
"v0.2.1": {
"id": 17582885,
"tag_name": "v0.2.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.windows-amd64.exe"
]
},
"v0.2.0": {
"id": 16965310,
"tag_name": "v0.2.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.windows-amd64.exe"
]
}
}

View File

@@ -13,26 +13,28 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
BUILDX_VERSION: "v0.10.0-rc1"
BUILDKIT_IMAGE: "moby/buildkit:v0.11.0-rc3"
BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:latest"
REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test"
jobs:
test:
prepare-test:
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
@@ -41,20 +43,89 @@ jobs:
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Test
uses: docker/bake-action@v2
name: Build
uses: docker/bake-action@v3
with:
targets: test
targets: integration-test-base
set: |
*.cache-from=type=gha,scope=test
*.cache-to=type=gha,scope=test
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.cache-to=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
test:
runs-on: ubuntu-22.04
needs:
- prepare-test
env:
TESTFLAGS: "-v --parallel=6 --timeout=30m"
TESTFLAGS_DOCKER: "-v --parallel=1 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose"
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildx-tests"
strategy:
fail-fast: false
matrix:
worker:
- docker
- docker-container
- remote
pkg:
- ./tests
include:
- pkg: ./...
skip-integration-tests: 1
steps:
-
name: Upload coverage
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build test image
uses: docker/bake-action@v3
with:
targets: integration-test
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |
export TEST_REPORT_SUFFIX=-${{ github.job }}-$(echo "${{ matrix.pkg }}-${{ matrix.skip-integration-tests }}-${{ matrix.worker }}" | tr -dc '[:alnum:]-\n\r' | tr '[:upper:]' '[:lower:]')
./hack/test
env:
TEST_DOCKERD: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker-container') && '1' || '0' }}"
TESTFLAGS: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker-container') && env.TESTFLAGS_DOCKER || env.TESTFLAGS }} --run=//worker=${{ matrix.worker }}$"
TESTPKGS: "${{ matrix.pkg }}"
SKIP_INTEGRATION_TESTS: "${{ matrix.skip-integration-tests }}"
-
name: Send to Codecov
if: always()
uses: codecov/codecov-action@v3
with:
directory: ${{ env.DESTDIR }}/coverage
directory: ./bin/testreports
-
name: Generate annotations
if: always()
uses: crazy-max/.github/.github/actions/gotest-annotations@1a64ea6d01db9a48aa61954cb20e265782c167d9
with:
directory: ./bin/testreports
-
name: Upload test reports
if: always()
uses: actions/upload-artifact@v3
with:
name: test-reports
path: ./bin/testreports
prepare:
prepare-binaries:
runs-on: ubuntu-22.04
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
@@ -75,11 +146,11 @@ jobs:
binaries:
runs-on: ubuntu-22.04
needs:
- prepare
- prepare-binaries
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.matrix) }}
platform: ${{ fromJson(needs.prepare-binaries.outputs.matrix) }}
steps:
-
name: Prepare
@@ -117,7 +188,9 @@ jobs:
bin-image:
runs-on: ubuntu-22.04
if: ${{ github.event_name != 'pull_request' }}
needs:
- test
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout
@@ -153,22 +226,22 @@ jobs:
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push image
uses: docker/bake-action@v2
uses: docker/bake-action@v3
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max
*.attest=type=sbom
*.attest=type=provenance,mode=max,builder-id=https://github.com/${{ env.GITHUB_REPOSITORY }}/actions/runs/${{ env.GITHUB_RUN_ID }}
release:
runs-on: ubuntu-22.04
needs:
- test
- binaries
steps:
-
@@ -221,6 +294,6 @@ jobs:
-
# Just run a bake target to check eveything runs fine
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v3
with:
targets: binaries

View File

@@ -3,12 +3,12 @@ name: docs-release
on:
release:
types:
- releases
- released
jobs:
open-pr:
runs-on: ubuntu-22.04
if: "!github.event.release.prerelease"
if: ${{ github.event.release.prerelease != true && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout docs repo
@@ -26,7 +26,7 @@ jobs:
uses: docker/setup-buildx-action@v2
-
name: Build docs
uses: docker/bake-action@v2
uses: docker/bake-action@v3
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ github.event.release.name }}
targets: update-docs
@@ -44,7 +44,7 @@ jobs:
git add -A .
-
name: Create PR on docs repo
uses: peter-evans/create-pull-request@2b011faafdcbc9ceb11414d64d0573f37c774b04
uses: peter-evans/create-pull-request@284f54f989303d2699d373481a0cfa13ad5a6666
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io

View File

@@ -34,7 +34,7 @@ jobs:
version: latest
-
name: Build reference YAML docs
uses: docker/bake-action@v2
uses: docker/bake-action@v3
with:
targets: update-docs
set: |
@@ -59,3 +59,4 @@ jobs:
repo: https://github.com/${{ github.repository }}
data-files-id: docs-yaml
data-files-folder: buildx
data-files-placeholder-folder: engine/reference/commandline

View File

@@ -11,15 +11,14 @@ on:
- 'master'
- 'v[0-9]*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
DESTDIR: "./bin"
K3S_VERSION: "v1.21.2-k3s1"
jobs:
build:
@@ -34,7 +33,7 @@ jobs:
version: latest
-
name: Build
uses: docker/bake-action@v2
uses: docker/bake-action@v3
with:
targets: binaries
set: |
@@ -133,20 +132,67 @@ jobs:
-
name: Install k3s
if: matrix.driver == 'kubernetes'
uses: debianmaster/actions-k3s@b9cf3f599fd118699a3c8a0d18a2f2bda6cf4ce4
id: k3s
uses: actions/github-script@v6
with:
version: v1.21.2-k3s1
script: |
const fs = require('fs');
let wait = function(milliseconds) {
return new Promise((resolve, reject) => {
if (typeof(milliseconds) !== 'number') {
throw new Error('milleseconds not a number');
}
setTimeout(() => resolve("done!"), milliseconds)
});
}
try {
const kubeconfig="/tmp/buildkit-k3s/kubeconfig.yaml";
core.info(`storing kubeconfig in ${kubeconfig}`);
await exec.exec('docker', ["run", "-d",
"--privileged",
"--name=buildkit-k3s",
"-e", "K3S_KUBECONFIG_OUTPUT="+kubeconfig,
"-e", "K3S_KUBECONFIG_MODE=666",
"-v", "/tmp/buildkit-k3s:/tmp/buildkit-k3s",
"-p", "6443:6443",
"-p", "80:80",
"-p", "443:443",
"-p", "8080:8080",
"rancher/k3s:${{ env.K3S_VERSION }}", "server"
]);
await wait(10000);
core.exportVariable('KUBECONFIG', kubeconfig);
let nodeName;
for (let count = 1; count <= 5; count++) {
try {
const nodeNameOutput = await exec.getExecOutput("kubectl get nodes --no-headers -oname");
nodeName = nodeNameOutput.stdout
} catch (error) {
core.info(`Unable to resolve node name (${error.message}). Attempt ${count} of 5.`)
} finally {
if (nodeName) {
break;
}
await wait(5000);
}
}
if (!nodeName) {
throw new Error(`Unable to resolve node name after 5 attempts.`);
}
await exec.exec(`kubectl wait --for=condition=Ready ${nodeName}`);
} catch (error) {
core.setFailed(error.message);
}
-
name: Config k3s
name: Print KUBECONFIG
if: matrix.driver == 'kubernetes'
run: |
(set -x ; cat ${{ steps.k3s.outputs.kubeconfig }})
-
name: Check k3s nodes
if: matrix.driver == 'kubernetes'
run: |
kubectl get nodes
yq ${{ env.KUBECONFIG }}
-
name: Launch remote buildkitd
if: matrix.driver == 'remote'

View File

@@ -13,9 +13,8 @@ on:
tags:
- 'v*'
pull_request:
branches:
- 'master'
- 'v[0-9]*'
paths-ignore:
- '.github/releases.json'
jobs:
validate:
@@ -27,6 +26,7 @@ jobs:
- lint
- validate-vendor
- validate-docs
- validate-generated-files
steps:
-
name: Checkout

View File

@@ -11,17 +11,17 @@ linters:
enable:
- gofmt
- govet
- deadcode
- depguard
- goimports
- ineffassign
- misspell
- unused
- varcheck
- revive
- staticcheck
- typecheck
- nolintlint
- gosec
- forbidigo
disable-all: true
linters-settings:
@@ -32,6 +32,15 @@ linters-settings:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- io/ioutil
forbidigo:
forbid:
- '^fmt\.Errorf(# use errors\.Errorf instead)?$'
gosec:
excludes:
- G204 # Audit use of command execution
- G402 # TLS MinVersion too low
config:
G306: "0644"
issues:
exclude-rules:

View File

@@ -1,8 +1,11 @@
# syntax=docker/dockerfile-upstream:master
# syntax=docker/dockerfile:1
ARG GO_VERSION=1.19
ARG XX_VERSION=1.1.2
ARG GO_VERSION=1.20
ARG XX_VERSION=1.2.1
ARG DOCKERD_VERSION=20.10.14
ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.11.6
FROM docker:$DOCKERD_VERSION AS dockerd-release
@@ -18,6 +21,17 @@ ENV GOFLAGS=-mod=vendor
ENV CGO_ENABLED=0
WORKDIR /src
FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM gobase AS gotestsum
ARG GOTESTSUM_VERSION
ENV GOFLAGS=
RUN --mount=target=/root/.cache,type=cache \
GOBIN=/out/ go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" && \
/out/gotestsum --version
FROM gobase AS buildx-version
RUN --mount=type=bind,target=. <<EOT
set -e
@@ -39,6 +53,7 @@ RUN --mount=type=bind,target=. \
EOT
FROM gobase AS test
ENV SKIP_INTEGRATION_TESTS=1
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
@@ -61,6 +76,17 @@ FROM binaries-$TARGETOS AS binaries
# enable scanning for this stage
ARG BUILDKIT_SBOM_SCAN_STAGE=true
FROM gobase AS integration-test-base
RUN apk add --no-cache docker runc containerd
COPY --link --from=gotestsum /out/gotestsum /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/
FROM integration-test-base AS integration-test
COPY . .
# Release
FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work

View File

@@ -37,7 +37,7 @@ release:
./hack/release
.PHONY: validate-all
validate-all: lint test validate-vendor validate-docs
validate-all: lint test validate-vendor validate-docs validate-generated-files
.PHONY: lint
lint:
@@ -45,7 +45,15 @@ lint:
.PHONY: test
test:
$(BUILDX_CMD) bake test
./hack/test
.PHONY: test-unit
test-unit:
TESTPKGS=./... SKIP_INTEGRATION_TESTS=1 ./hack/test
.PHONY: test
test-integration:
TESTPKGS=./tests ./hack/test
.PHONY: validate-vendor
validate-vendor:
@@ -59,6 +67,10 @@ validate-docs:
validate-authors:
$(BUILDX_CMD) bake validate-authors
.PHONY: validate-generated-files
validate-generated-files:
$(BUILDX_CMD) bake validate-generated-files
.PHONY: test-driver
test-driver:
./hack/test-driver
@@ -78,3 +90,7 @@ authors:
.PHONY: mod-outdated
mod-outdated:
$(BUILDX_CMD) bake mod-outdated
.PHONY: generated-files
generated-files:
$(BUILDX_CMD) bake update-generated-files

View File

@@ -32,19 +32,6 @@ Key features:
- [Building with buildx](#building-with-buildx)
- [Working with builder instances](#working-with-builder-instances)
- [Building multi-platform images](#building-multi-platform-images)
- [Manuals](docs/manuals)
- [High-level build options with Bake](docs/manuals/bake/index.md)
- [Drivers](docs/manuals/drivers/index.md)
- [Exporters](docs/manuals/exporters/index.md)
- [Cache backends](docs/manuals/cache/backends/index.md)
- [Guides](docs/guides)
- [CI/CD](docs/guides/cicd.md)
- [CNI networking](docs/guides/cni-networking.md)
- [Using a custom network](docs/guides/custom-network.md)
- [Using a custom registry configuration](docs/guides/custom-registry-config.md)
- [OpenTelemetry support](docs/guides/opentelemetry.md)
- [Registry mirror](docs/guides/registry-mirror.md)
- [Resource limiting](docs/guides/resource-limiting.md)
- [Reference](docs/reference/buildx.md)
- [`buildx bake`](docs/reference/buildx_bake.md)
- [`buildx build`](docs/reference/buildx_build.md)
@@ -64,11 +51,18 @@ Key features:
- [`buildx version`](docs/reference/buildx_version.md)
- [Contributing](#contributing)
For more information on how to use Buildx, see
[Docker Build docs](https://docs.docker.com/build/).
# Installing
Using `buildx` as a docker CLI plugin requires using Docker 19.03 or newer.
A limited set of functionality works with older versions of Docker when
invoking the binary directly.
Using `buildx` with Docker requires Docker engine 19.03 or newer.
> **Warning**
>
> Using an incompatible version of Docker may result in unexpected behavior,
> and will likely cause issues, especially when using Buildx builders with more
> recent versions of BuildKit.
## Windows and macOS
@@ -147,7 +141,7 @@ To remove this alias, run [`docker buildx uninstall`](docs/reference/buildx_unin
# Buildx 0.6+
$ docker buildx bake "https://github.com/docker/buildx.git"
$ mkdir -p ~/.docker/cli-plugins
$ mv ./bin/buildx ~/.docker/cli-plugins/docker-buildx
$ mv ./bin/build/buildx ~/.docker/cli-plugins/docker-buildx
# Docker 19.03+
$ DOCKER_BUILDKIT=1 docker build --platform=local -o . "https://github.com/docker/buildx.git"
@@ -316,7 +310,7 @@ cross-compilation helpers for more advanced use-cases.
## High-level build options
See [`docs/manuals/bake/index.md`](docs/manuals/bake/index.md) for more details.
See [High-level builds with Bake](https://docs.docker.com/build/bake/) for more details.
# Contributing

View File

@@ -3,7 +3,6 @@ package bake
import (
"context"
"encoding/csv"
"fmt"
"io"
"os"
"path"
@@ -13,22 +12,23 @@ import (
"strconv"
"strings"
composecli "github.com/compose-spec/compose-go/cli"
"github.com/docker/buildx/bake/hclparser"
"github.com/docker/buildx/build"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli/config"
"github.com/docker/docker/builder/remotecontext/urlutil"
hcl "github.com/hashicorp/hcl/v2"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/convert"
)
var (
httpPrefix = regexp.MustCompile(`^https?://`)
gitURLPathWithFragmentSuffix = regexp.MustCompile(`\.git(?:#.+)?$`)
validTargetNameChars = `[a-zA-Z0-9_-]+`
targetNamePattern = regexp.MustCompile(`^` + validTargetNameChars + `$`)
)
@@ -44,17 +44,18 @@ type Override struct {
}
func defaultFilenames() []string {
return []string{
"docker-compose.yml", // support app
"docker-compose.yaml", // support app
names := []string{}
names = append(names, composecli.DefaultFileNames...)
names = append(names, []string{
"docker-bake.json",
"docker-bake.override.json",
"docker-bake.hcl",
"docker-bake.override.hcl",
}
}...)
return names
}
func ReadLocalFiles(names []string) ([]File, error) {
func ReadLocalFiles(names []string, stdin io.Reader) ([]File, error) {
isDefault := false
if len(names) == 0 {
isDefault = true
@@ -66,7 +67,7 @@ func ReadLocalFiles(names []string) ([]File, error) {
var dt []byte
var err error
if n == "-" {
dt, err = io.ReadAll(os.Stdin)
dt, err = io.ReadAll(stdin)
if err != nil {
return nil, err
}
@@ -84,6 +85,21 @@ func ReadLocalFiles(names []string) ([]File, error) {
return out, nil
}
func ListTargets(files []File) ([]string, error) {
c, err := ParseFiles(files, nil)
if err != nil {
return nil, err
}
var targets []string
for _, g := range c.Groups {
targets = append(targets, g.Name)
}
for _, t := range c.Targets {
targets = append(targets, t.Name)
}
return dedupSlice(targets), nil
}
func ReadTargets(ctx context.Context, files []File, targets, overrides []string, defaults map[string]string) (map[string]*Target, map[string]*Group, error) {
c, err := ParseFiles(files, defaults)
if err != nil {
@@ -215,7 +231,7 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
hclFiles = append(hclFiles, hf)
} else if composeErr != nil {
return nil, fmt.Errorf("failed to parse %s: parsing yaml: %v, parsing hcl: %w", f.Name, composeErr, err)
return nil, errors.Wrapf(err, "failed to parse %s: parsing yaml: %v, parsing hcl", f.Name, composeErr)
} else {
return nil, err
}
@@ -232,13 +248,28 @@ func ParseFiles(files []File, defaults map[string]string) (_ *Config, err error)
}
if len(hclFiles) > 0 {
if err := hclparser.Parse(hcl.MergeFiles(hclFiles), hclparser.Opt{
renamed, err := hclparser.Parse(hcl.MergeFiles(hclFiles), hclparser.Opt{
LookupVar: os.LookupEnv,
Vars: defaults,
ValidateLabel: validateTargetName,
}, &c); err.HasErrors() {
}, &c)
if err.HasErrors() {
return nil, err
}
for _, renamed := range renamed {
for oldName, newNames := range renamed {
newNames = dedupSlice(newNames)
if len(newNames) == 1 && oldName == newNames[0] {
continue
}
c.Groups = append(c.Groups, &Group{
Name: oldName,
Targets: newNames,
})
}
}
c = dedupeConfig(c)
}
return &c, nil
@@ -577,14 +608,19 @@ type Target struct {
NoCache *bool `json:"no-cache,omitempty" hcl:"no-cache,optional" cty:"no-cache"`
NetworkMode *string `json:"-" hcl:"-" cty:"-"`
NoCacheFilter []string `json:"no-cache-filter,omitempty" hcl:"no-cache-filter,optional" cty:"no-cache-filter"`
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/manuals/bake/file-definition.md.
// IMPORTANT: if you add more fields here, do not forget to update newOverrides and docs/bake-reference.md.
// linked is a private field to mark a target used as a linked one
linked bool
}
var _ hclparser.WithEvalContexts = &Target{}
var _ hclparser.WithGetName = &Target{}
var _ hclparser.WithEvalContexts = &Group{}
var _ hclparser.WithGetName = &Group{}
func (t *Target) normalize() {
t.Attest = removeDupes(t.Attest)
t.Attest = removeAttestDupes(t.Attest)
t.Tags = removeDupes(t.Tags)
t.Secrets = removeDupes(t.Secrets)
t.SSH = removeDupes(t.SSH)
@@ -646,6 +682,7 @@ func (t *Target) Merge(t2 *Target) {
}
if t2.Attest != nil { // merge
t.Attest = append(t.Attest, t2.Attest...)
t.Attest = removeAttestDupes(t.Attest)
}
if t2.Secrets != nil { // merge
t.Secrets = append(t.Secrets, t2.Secrets...)
@@ -766,6 +803,114 @@ func (t *Target) AddOverrides(overrides map[string]Override) error {
return nil
}
func (g *Group) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
if _, ok := content.Attributes["matrix"]; ok {
return nil, errors.Errorf("matrix is not supported for groups")
}
return []*hcl.EvalContext{ectx}, nil
}
func (t *Target) GetEvalContexts(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error) {
content, _, err := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "matrix"}},
})
if err != nil {
return nil, err
}
attr, ok := content.Attributes["matrix"]
if !ok {
return []*hcl.EvalContext{ectx}, nil
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return nil, diags
}
value, err := attr.Expr.Value(ectx)
if err != nil {
return nil, err
}
if !value.Type().IsMapType() && !value.Type().IsObjectType() {
return nil, errors.Errorf("matrix must be a map")
}
matrix := value.AsValueMap()
ectxs := []*hcl.EvalContext{ectx}
for k, expr := range matrix {
if !expr.CanIterateElements() {
return nil, errors.Errorf("matrix values must be a list")
}
ectxs2 := []*hcl.EvalContext{}
for _, v := range expr.AsValueSlice() {
for _, e := range ectxs {
e2 := ectx.NewChild()
e2.Variables = make(map[string]cty.Value)
for k, v := range e.Variables {
e2.Variables[k] = v
}
e2.Variables[k] = v
ectxs2 = append(ectxs2, e2)
}
}
ectxs = ectxs2
}
return ectxs, nil
}
func (g *Group) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
if _, ok := content.Attributes["name"]; ok {
return "", errors.Errorf("name is not supported for groups")
}
if _, ok := content.Attributes["matrix"]; ok {
return "", errors.Errorf("matrix is not supported for groups")
}
return block.Labels[0], nil
}
func (t *Target) GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error) {
content, _, diags := block.Body.PartialContent(&hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: "name"}, {Name: "matrix"}},
})
if diags != nil {
return "", diags
}
attr, ok := content.Attributes["name"]
if !ok {
return block.Labels[0], nil
}
if _, ok := content.Attributes["matrix"]; !ok {
return "", errors.Errorf("name requires matrix")
}
if diags := loadDeps(attr.Expr); diags.HasErrors() {
return "", diags
}
value, diags := attr.Expr.Value(ectx)
if diags != nil {
return "", diags
}
value, err := convert.Convert(value, cty.String)
if err != nil {
return "", err
}
return value.AsString(), nil
}
func TargetsToBuildOpt(m map[string]*Target, inp *Input) (map[string]build.Options, error) {
m2 := make(map[string]build.Options, len(m))
for k, v := range m {
@@ -790,7 +935,7 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(v.Path, "cwd://") || strings.HasPrefix(v.Path, "target:") || strings.HasPrefix(v.Path, "docker-image:") {
continue
}
if IsRemoteURL(v.Path) {
if build.IsRemoteURL(v.Path) {
continue
}
st := llb.Scratch().File(llb.Copy(*inp.State, v.Path, "/"), llb.WithCustomNamef("set context %s to %s", k, v.Path))
@@ -804,10 +949,15 @@ func updateContext(t *build.Inputs, inp *Input) {
if strings.HasPrefix(t.ContextPath, "cwd://") {
return
}
if IsRemoteURL(t.ContextPath) {
if build.IsRemoteURL(t.ContextPath) {
return
}
st := llb.Scratch().File(llb.Copy(*inp.State, t.ContextPath, "/"), llb.WithCustomNamef("set context to %s", t.ContextPath))
st := llb.Scratch().File(
llb.Copy(*inp.State, t.ContextPath, "/", &llb.CopyInfo{
CopyDirContentsOnly: true,
}),
llb.WithCustomNamef("set context to %s", t.ContextPath),
)
t.ContextState = &st
}
@@ -840,7 +990,7 @@ func validateContextsEntitlements(t build.Inputs, inp *Input) error {
}
func checkPath(p string) error {
if IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
if build.IsRemoteURL(p) || strings.HasPrefix(p, "target:") || strings.HasPrefix(p, "docker-image:") {
return nil
}
p, err := filepath.EvalSymlinks(p)
@@ -850,6 +1000,10 @@ func checkPath(p string) error {
}
return err
}
p, err = filepath.Abs(p)
if err != nil {
return err
}
wd, err := os.Getwd()
if err != nil {
return err
@@ -858,7 +1012,8 @@ func checkPath(p string) error {
if err != nil {
return err
}
if strings.HasPrefix(rel, ".."+string(os.PathSeparator)) {
parts := strings.Split(rel, string(os.PathSeparator))
if parts[0] == ".." {
return errors.Errorf("path %s is outside of the working directory, please set BAKE_ALLOW_REMOTE_FS_ACCESS=1", p)
}
return nil
@@ -876,7 +1031,7 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if t.Context != nil {
contextPath = *t.Context
}
if !strings.HasPrefix(contextPath, "cwd://") && !IsRemoteURL(contextPath) {
if !strings.HasPrefix(contextPath, "cwd://") && !build.IsRemoteURL(contextPath) {
contextPath = path.Clean(contextPath)
}
dockerfilePath := "Dockerfile"
@@ -884,9 +1039,32 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
dockerfilePath = *t.Dockerfile
}
if !isRemoteResource(contextPath) && !path.IsAbs(dockerfilePath) {
dockerfilePath = path.Join(contextPath, dockerfilePath)
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if !build.IsRemoteURL(bi.ContextPath) && bi.ContextState == nil && !path.IsAbs(bi.DockerfilePath) {
bi.DockerfilePath = path.Join(bi.ContextPath, bi.DockerfilePath)
}
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
args := map[string]string{}
for k, v := range t.Args {
@@ -917,30 +1095,6 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
networkMode = *t.NetworkMode
}
bi := build.Inputs{
ContextPath: contextPath,
DockerfilePath: dockerfilePath,
NamedContexts: toNamedContexts(t.Contexts),
}
if t.DockerfileInline != nil {
bi.DockerfileInline = *t.DockerfileInline
}
updateContext(&bi, inp)
if strings.HasPrefix(bi.ContextPath, "cwd://") {
bi.ContextPath = path.Clean(strings.TrimPrefix(bi.ContextPath, "cwd://"))
}
for k, v := range bi.NamedContexts {
if strings.HasPrefix(v.Path, "cwd://") {
bi.NamedContexts[k] = build.NamedContext{Path: path.Clean(strings.TrimPrefix(v.Path, "cwd://"))}
}
}
if err := validateContextsEntitlements(bi, inp); err != nil {
return nil, err
}
t.Context = &bi.ContextPath
bo := &build.Options{
Inputs: bi,
Tags: t.Tags,
@@ -966,17 +1120,24 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, secrets)
sshSpecs := t.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(contextPath) {
sshSpecs = []string{"default"}
}
ssh, err := buildflags.ParseSSHSpecs(sshSpecs)
secretAttachment, err := controllerapi.CreateSecrets(secrets)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, ssh)
bo.Session = append(bo.Session, secretAttachment)
sshSpecs, err := buildflags.ParseSSHSpecs(t.SSH)
if err != nil {
return nil, err
}
if len(sshSpecs) == 0 && (buildflags.IsGitSSH(bi.ContextPath) || (inp != nil && buildflags.IsGitSSH(inp.URL))) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
sshAttachment, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, err
}
bo.Session = append(bo.Session, sshAttachment)
if t.Target != nil {
bo.Target = *t.Target
@@ -986,25 +1147,33 @@ func toBuildOpt(t *Target, inp *Input) (*build.Options, error) {
if err != nil {
return nil, err
}
bo.CacheFrom = cacheImports
bo.CacheFrom = controllerapi.CreateCaches(cacheImports)
cacheExports, err := buildflags.ParseCacheEntry(t.CacheTo)
if err != nil {
return nil, err
}
bo.CacheTo = cacheExports
bo.CacheTo = controllerapi.CreateCaches(cacheExports)
outputs, err := buildflags.ParseOutputs(t.Outputs)
outputs, err := buildflags.ParseExports(t.Outputs)
if err != nil {
return nil, err
}
bo.Exports, err = controllerapi.CreateExports(outputs)
if err != nil {
return nil, err
}
bo.Exports = outputs
attests, err := buildflags.ParseAttests(t.Attest)
if err != nil {
return nil, err
}
bo.Attests = attests
bo.Attests = controllerapi.CreateAttestations(attests)
bo.SourcePolicy, err = build.ReadSourcePolicy()
if err != nil {
return nil, err
}
return bo, nil
}
@@ -1030,8 +1199,24 @@ func removeDupes(s []string) []string {
return s[:i]
}
func isRemoteResource(str string) bool {
return urlutil.IsGitURL(str) || urlutil.IsURL(str)
func removeAttestDupes(s []string) []string {
res := []string{}
m := map[string]int{}
for _, v := range s {
att, err := buildflags.ParseAttest(v)
if err != nil {
res = append(res, v)
continue
}
if i, ok := m[att.Type]; ok {
res[i] = v
} else {
m[att.Type] = len(res)
res = append(res, v)
}
}
return res
}
func parseOutputType(str string) string {

View File

@@ -2,6 +2,7 @@ package bake
import (
"context"
"os"
"sort"
"strings"
"testing"
@@ -10,8 +11,6 @@ import (
)
func TestReadTargets(t *testing.T) {
t.Parallel()
fp := File{
Name: "config.hcl",
Data: []byte(`
@@ -35,6 +34,7 @@ target "webapp" {
ctx := context.TODO()
t.Run("NoOverrides", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, nil, nil)
require.NoError(t, err)
require.Equal(t, 1, len(m))
@@ -50,6 +50,7 @@ target "webapp" {
})
t.Run("InvalidTargetOverrides", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"nosuchtarget.context=foo"}, nil)
require.NotNil(t, err)
require.Equal(t, err.Error(), "could not find any target matching 'nosuchtarget'")
@@ -91,6 +92,7 @@ target "webapp" {
// building leaf but overriding parent fields
t.Run("parent", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{
"webDEP.args.VAR_INHERITED=override",
"webDEP.args.VAR_BOTH=override",
@@ -105,6 +107,7 @@ target "webapp" {
})
t.Run("ContextOverride", func(t *testing.T) {
t.Parallel()
_, _, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.context"}, nil)
require.NotNil(t, err)
@@ -116,6 +119,7 @@ target "webapp" {
})
t.Run("NoCacheOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.no-cache=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].NoCache)
@@ -124,6 +128,7 @@ target "webapp" {
})
t.Run("PullOverride", func(t *testing.T) {
t.Parallel()
m, g, err := ReadTargets(ctx, []File{fp}, []string{"webapp"}, []string{"webapp.pull=false"}, nil)
require.NoError(t, err)
require.Equal(t, false, *m["webapp"].Pull)
@@ -132,6 +137,7 @@ target "webapp" {
})
t.Run("PatternOverride", func(t *testing.T) {
t.Parallel()
// same check for two cases
multiTargetCheck := func(t *testing.T, m map[string]*Target, g map[string]*Group, err error) {
require.NoError(t, err)
@@ -1358,3 +1364,89 @@ func TestJSONNullVars(t *testing.T) {
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, m["default"].Args)
}
func TestReadLocalFilesDefault(t *testing.T) {
tests := []struct {
filenames []string
expected []string
}{
{
filenames: []string{"abc.yml", "docker-compose.yml"},
expected: []string{"docker-compose.yml"},
},
{
filenames: []string{"test.foo", "compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
expected: []string{"compose.yaml", "docker-compose.yml", "docker-bake.hcl"},
},
{
filenames: []string{"test.txt", "compsoe.yaml"}, // intentional misspell
expected: []string{},
},
}
pwd, err := os.Getwd()
require.NoError(t, err)
for _, tt := range tests {
t.Run(strings.Join(tt.filenames, "-"), func(t *testing.T) {
dir := t.TempDir()
t.Cleanup(func() { _ = os.Chdir(pwd) })
require.NoError(t, os.Chdir(dir))
for _, tf := range tt.filenames {
require.NoError(t, os.WriteFile(tf, []byte(tf), 0644))
}
files, err := ReadLocalFiles(nil, nil)
require.NoError(t, err)
if len(files) == 0 {
require.Equal(t, len(tt.expected), len(files))
} else {
found := false
for _, exp := range tt.expected {
for _, f := range files {
if f.Name == exp {
found = true
break
}
}
require.True(t, found, exp)
}
}
})
}
}
func TestAttestDuplicates(t *testing.T) {
fp := File{
Name: "docker-bake.hcl",
Data: []byte(
`target "default" {
attest = ["type=sbom", "type=sbom,generator=custom", "type=sbom,foo=bar", "type=provenance,mode=max"]
}`),
}
ctx := context.TODO()
m, _, err := ReadTargets(ctx, []File{fp}, []string{"default"}, nil, nil)
require.Equal(t, []string{"type=sbom,foo=bar", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err := TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": ptrstr("type=sbom,foo=bar"),
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
m, _, err = ReadTargets(ctx, []File{fp}, []string{"default"}, []string{"*.attest=type=sbom,disabled=true"}, nil)
require.Equal(t, []string{"type=sbom,disabled=true", "type=provenance,mode=max"}, m["default"].Attest)
require.NoError(t, err)
opts, err = TargetsToBuildOpt(m, &Input{})
require.NoError(t, err)
require.Equal(t, map[string]*string{
"sbom": nil,
"provenance": ptrstr("type=provenance,mode=max"),
}, opts["default"].Attests)
}

View File

@@ -28,10 +28,14 @@ func ParseComposeFiles(fs []File) (*Config, error) {
}
func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, error) {
if envs == nil {
envs = make(map[string]string)
}
cfg, err := loader.Load(compose.ConfigDetails{
ConfigFiles: cfgs,
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
})
if err != nil {
@@ -65,6 +69,19 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
dockerfilePath := s.Build.Dockerfile
dockerfilePathP = &dockerfilePath
}
var dockerfileInlineP *string
if s.Build.DockerfileInline != "" {
dockerfileInline := s.Build.DockerfileInline
dockerfileInlineP = &dockerfileInline
}
var additionalContexts map[string]string
if s.Build.AdditionalContexts != nil {
additionalContexts = map[string]string{}
for k, v := range s.Build.AdditionalContexts {
additionalContexts[k] = v
}
}
var secrets []string
for _, bs := range s.Build.Secrets {
@@ -78,16 +95,19 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
// compose does not support nil values for labels
labels := map[string]*string{}
for k, v := range s.Build.Labels {
v := v
labels[k] = &v
}
g.Targets = append(g.Targets, targetName)
t := &Target{
Name: targetName,
Context: contextPathP,
Dockerfile: dockerfilePathP,
Tags: s.Build.Tags,
Labels: labels,
Name: targetName,
Context: contextPathP,
Contexts: additionalContexts,
Dockerfile: dockerfilePathP,
DockerfileInline: dockerfileInlineP,
Tags: s.Build.Tags,
Labels: labels,
Args: flatten(s.Build.Args.Resolve(func(val string) (string, bool) {
if val, ok := s.Environment[val]; ok && val != nil {
return *val, true
@@ -144,6 +164,7 @@ func validateCompose(dt []byte, envs map[string]string) error {
},
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
// consistency is checked later in ParseCompose to ensure multiple
// compose files can be merged together
@@ -184,7 +205,7 @@ func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string,
return nil, err
}
envs, err := dotenv.UnmarshalBytes(dt)
envs, err := dotenv.UnmarshalBytesWithLookup(dt, nil)
if err != nil {
return nil, err
}

View File

@@ -21,6 +21,8 @@ services:
webapp:
build:
context: ./dir
additional_contexts:
foo: /bar
dockerfile: Dockerfile-alternate
network:
none
@@ -33,6 +35,11 @@ services:
secrets:
- token
- aws
webapp2:
build:
context: ./dir
dockerfile_inline: |
FROM alpine
secrets:
token:
environment: ENV_TOKEN
@@ -46,9 +53,9 @@ secrets:
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
sort.Strings(c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp", "webapp2"}, c.Groups[0].Targets)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, 3, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
@@ -58,6 +65,7 @@ secrets:
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, "./dir", *c.Targets[1].Context)
require.Equal(t, map[string]string{"foo": "/bar"}, c.Targets[1].Contexts)
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
@@ -68,6 +76,10 @@ secrets:
"id=token,env=ENV_TOKEN",
"id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, "./dir", *c.Targets[2].Context)
require.Equal(t, "FROM alpine\n", *c.Targets[2].DockerfileInline)
}
func TestNoBuildOutOfTreeService(t *testing.T) {

View File

@@ -634,6 +634,462 @@ func TestHCLMultiFileAttrs(t *testing.T) {
require.Equal(t, ptrstr("pre-ghi"), c.Targets[0].Args["v1"])
}
func TestHCLDuplicateTarget(t *testing.T) {
dt := []byte(`
target "app" {
dockerfile = "x"
}
target "app" {
dockerfile = "y"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, "app", c.Targets[0].Name)
require.Equal(t, "y", *c.Targets[0].Dockerfile)
}
func TestHCLRenameTarget(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
dockerfile = "foo"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "requires matrix")
}
func TestHCLRenameGroup(t *testing.T) {
dt := []byte(`
group "foo" {
name = "bar"
targets = ["x", "y"]
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
dt = []byte(`
group "foo" {
matrix = {
name = ["x", "y"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "not supported")
}
func TestHCLRenameTargetAttrs(t *testing.T) {
dt := []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.xyz.dockerfile
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "xyz", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "def", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "def" {
dockerfile = target.xyz.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
c, err = ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "def", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "xyz", c.Targets[1].Name)
require.Equal(t, "foo", *c.Targets[1].Dockerfile)
dt = []byte(`
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
target "def" {
dockerfile = target.abc.dockerfile
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
dt = []byte(`
target "def" {
dockerfile = target.abc.dockerfile
}
target "abc" {
name = "xyz"
matrix = {}
dockerfile = "foo"
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.ErrorContains(t, err, "abc")
}
func TestHCLRenameSplit(t *testing.T) {
dt := []byte(`
target "x" {
name = "y"
matrix = {}
dockerfile = "foo"
}
target "x" {
name = "z"
matrix = {}
dockerfile = "bar"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "y", c.Targets[0].Name)
require.Equal(t, "foo", *c.Targets[0].Dockerfile)
require.Equal(t, "z", c.Targets[1].Name)
require.Equal(t, "bar", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "x", c.Groups[0].Name)
require.Equal(t, []string{"y", "z"}, c.Groups[0].Targets)
}
func TestHCLRenameMultiFile(t *testing.T) {
dt := []byte(`
target "foo" {
name = "bar"
matrix = {}
dockerfile = "x"
}
`)
dt2 := []byte(`
target "foo" {
context = "y"
}
`)
dt3 := []byte(`
target "bar" {
target = "z"
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "c1.hcl"},
{Data: dt2, Name: "c2.hcl"},
{Data: dt3, Name: "c3.hcl"},
}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "bar")
require.Equal(t, *c.Targets[0].Dockerfile, "x")
require.Equal(t, *c.Targets[0].Target, "z")
require.Equal(t, c.Targets[1].Name, "foo")
require.Equal(t, *c.Targets[1].Context, "y")
}
func TestHCLMatrixBasic(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["x", "y"]
}
name = foo
dockerfile = "${foo}.Dockerfile"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "x")
require.Equal(t, c.Targets[1].Name, "y")
require.Equal(t, *c.Targets[0].Dockerfile, "x.Dockerfile")
require.Equal(t, *c.Targets[1].Dockerfile, "y.Dockerfile")
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.Equal(t, []string{"x", "y"}, c.Groups[0].Targets)
}
func TestHCLMatrixMultipleKeys(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a"]
bar = ["b", "c"]
baz = ["d", "e", "f"]
}
name = "${foo}-${bar}-${baz}"
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 6, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, names)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, "default", c.Groups[0].Name)
require.ElementsMatch(t, []string{"a-b-d", "a-b-e", "a-b-f", "a-c-d", "a-c-e", "a-c-f"}, c.Groups[0].Targets)
}
func TestHCLMatrixLists(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [["aa", "bb"], ["cc", "dd"]]
}
name = aa[0]
args = {
target = "val${aa[1]}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "aa", c.Targets[0].Name)
require.Equal(t, ptrstr("valbb"), c.Targets[0].Args["target"])
require.Equal(t, "cc", c.Targets[1].Name)
require.Equal(t, ptrstr("valdd"), c.Targets[1].Args["target"])
}
func TestHCLMatrixMaps(t *testing.T) {
dt := []byte(`
target "foo" {
matrix = {
aa = [
{
foo = "aa"
bar = "bb"
},
{
foo = "cc"
bar = "dd"
}
]
}
name = aa.foo
args = {
target = "val${aa.bar}"
}
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "aa")
require.Equal(t, c.Targets[0].Args["target"], ptrstr("valbb"))
require.Equal(t, c.Targets[1].Name, "cc")
require.Equal(t, c.Targets[1].Args["target"], ptrstr("valdd"))
}
func TestHCLMatrixMultipleTargets(t *testing.T) {
dt := []byte(`
target "x" {
matrix = {
foo = ["a", "b"]
}
name = foo
}
target "y" {
matrix = {
bar = ["c", "d"]
}
name = bar
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 4, len(c.Targets))
names := make([]string, len(c.Targets))
for i, t := range c.Targets {
names[i] = t.Name
}
require.ElementsMatch(t, []string{"a", "b", "c", "d"}, names)
require.Equal(t, 2, len(c.Groups))
names = make([]string, len(c.Groups))
for i, c := range c.Groups {
names[i] = c.Name
}
require.ElementsMatch(t, []string{"x", "y"}, names)
for _, g := range c.Groups {
switch g.Name {
case "x":
require.Equal(t, []string{"a", "b"}, g.Targets)
case "y":
require.Equal(t, []string{"c", "d"}, g.Targets)
}
}
}
func TestHCLMatrixDuplicateNames(t *testing.T) {
dt := []byte(`
target "default" {
matrix = {
foo = ["a", "b"]
}
name = "c"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestHCLMatrixArgs(t *testing.T) {
dt := []byte(`
a = 1
variable "b" {
default = 2
}
target "default" {
matrix = {
foo = [a, b]
}
name = foo
}
`)
c, err := ParseFile(dt, "docker-bake.hcl")
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, "1", c.Targets[0].Name)
require.Equal(t, "2", c.Targets[1].Name)
}
func TestHCLMatrixArgsOverride(t *testing.T) {
dt := []byte(`
variable "ABC" {
default = "def"
}
target "bar" {
matrix = {
aa = split(",", ABC)
}
name = "bar-${aa}"
args = {
foo = aa
}
}
`)
c, err := ParseFiles([]File{
{Data: dt, Name: "docker-bake.hcl"},
}, map[string]string{"ABC": "11,22,33"})
require.NoError(t, err)
require.Equal(t, 3, len(c.Targets))
require.Equal(t, "bar-11", c.Targets[0].Name)
require.Equal(t, "bar-22", c.Targets[1].Name)
require.Equal(t, "bar-33", c.Targets[2].Name)
require.Equal(t, ptrstr("11"), c.Targets[0].Args["foo"])
require.Equal(t, ptrstr("22"), c.Targets[1].Args["foo"])
require.Equal(t, ptrstr("33"), c.Targets[2].Args["foo"])
}
func TestHCLMatrixBadTypes(t *testing.T) {
dt := []byte(`
target "default" {
matrix = "test"
}
`)
_, err := ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = ["test"]
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
["a"] = ["b"]
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
1 = 2
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
dt = []byte(`
target "default" {
matrix = {
a = "b"
}
}
`)
_, err = ParseFile(dt, "docker-bake.hcl")
require.Error(t, err)
}
func TestJSONAttributes(t *testing.T) {
dt := []byte(`{"FOO": "abc", "variable": {"BAR": {"default": "def"}}, "target": { "app": { "args": {"v1": "pre-${FOO}-${BAR}"}} } }`)
@@ -670,6 +1126,24 @@ func TestJSONFunctions(t *testing.T) {
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
}
func TestJSONInvalidFunctions(t *testing.T) {
dt := []byte(`{
"target": {
"app": {
"args": {
"v1": "myfunc(\"foo\")"
}
}
}}`)
c, err := ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
}
func TestHCLFunctionInAttr(t *testing.T) {
dt := []byte(`
function "brace" {
@@ -928,7 +1402,7 @@ func TestVarUnsupportedType(t *testing.T) {
}
func ptrstr(s interface{}) *string {
var n *string = nil
var n *string
if reflect.ValueOf(s).Kind() == reflect.String {
ss := s.(string)
n = &ss

View File

@@ -14,15 +14,7 @@ func funcCalls(exp hcl.Expression) ([]string, hcl.Diagnostics) {
if !ok {
fns, err := jsonFuncCallsRecursive(exp)
if err != nil {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
return nil, wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
return fns, nil
}

View File

@@ -1,7 +1,9 @@
package hclparser
import (
"encoding/binary"
"fmt"
"hash/fnv"
"math"
"math/big"
"reflect"
@@ -49,36 +51,42 @@ type parser struct {
attrs map[string]*hcl.Attribute
funcs map[string]*functionDef
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block]reflect.Value
blockTypes map[string]reflect.Type
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block][]reflect.Value
blockEvalCtx map[*hcl.Block][]*hcl.EvalContext
blockNames map[*hcl.Block][]string
blockTypes map[string]reflect.Type
ectx *hcl.EvalContext
progress map[string]struct{}
progressF map[string]struct{}
progressB map[*hcl.Block]map[string]struct{}
doneF map[string]struct{}
doneB map[*hcl.Block]map[string]struct{}
progressV map[uint64]struct{}
progressF map[uint64]struct{}
progressB map[uint64]map[string]struct{}
doneB map[uint64]map[string]struct{}
}
func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.Diagnostics {
type WithEvalContexts interface {
GetEvalContexts(base *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error)
}
type WithGetName interface {
GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error)
}
var errUndefined = errors.New("undefined")
func (p *parser) loadDeps(ectx *hcl.EvalContext, exp hcl.Expression, exclude map[string]struct{}, allowMissing bool) hcl.Diagnostics {
fns, hcldiags := funcCalls(exp)
if hcldiags.HasErrors() {
return hcldiags
}
for _, fn := range fns {
if err := p.resolveFunction(fn); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
if err := p.resolveFunction(ectx, fn); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
@@ -127,28 +135,20 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
}
}
}
if err := p.resolveBlock(blocks[0], target); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
for _, block := range blocks {
if err := p.resolveBlock(block, target); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
} else {
if err := p.resolveValue(v.RootName()); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
if err := p.resolveValue(ectx, v.RootName()); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
}
@@ -158,21 +158,21 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
// resolveFunction forces evaluation of a function, storing the result into the
// parser.
func (p *parser) resolveFunction(name string) error {
if _, ok := p.doneF[name]; ok {
func (p *parser) resolveFunction(ectx *hcl.EvalContext, name string) error {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
if _, ok := ectx.Functions[name]; ok {
return nil
}
f, ok := p.funcs[name]
if !ok {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
return errors.Errorf("undefined function %s", name)
return errors.Wrapf(errUndefined, "function %q does not exist", name)
}
if _, ok := p.progressF[name]; ok {
if _, ok := p.progressF[key(ectx, name)]; ok {
return errors.Errorf("function cycle not allowed for %s", name)
}
p.progressF[name] = struct{}{}
p.progressF[key(ectx, name)] = struct{}{}
if f.Result == nil {
return errors.Errorf("empty result not allowed for %s", name)
@@ -217,7 +217,7 @@ func (p *parser) resolveFunction(name string) error {
return diags
}
if diags := p.loadDeps(f.Result.Expr, params); diags.HasErrors() {
if diags := p.loadDeps(p.ectx, f.Result.Expr, params, false); diags.HasErrors() {
return diags
}
@@ -227,7 +227,6 @@ func (p *parser) resolveFunction(name string) error {
if diags.HasErrors() {
return diags
}
p.doneF[name] = struct{}{}
p.ectx.Functions[name] = v
return nil
@@ -235,14 +234,17 @@ func (p *parser) resolveFunction(name string) error {
// resolveValue forces evaluation of a named value, storing the result into the
// parser.
func (p *parser) resolveValue(name string) (err error) {
func (p *parser) resolveValue(ectx *hcl.EvalContext, name string) (err error) {
if _, ok := p.ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progress[name]; ok {
if _, ok := ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progressV[key(ectx, name)]; ok {
return errors.Errorf("variable cycle not allowed for %s", name)
}
p.progress[name] = struct{}{}
p.progressV[key(ectx, name)] = struct{}{}
var v *cty.Value
defer func() {
@@ -255,9 +257,10 @@ func (p *parser) resolveValue(name string) (err error) {
if _, builtin := p.opt.Vars[name]; !ok && !builtin {
vr, ok := p.vars[name]
if !ok {
return errors.Errorf("undefined variable %q", name)
return errors.Wrapf(errUndefined, "variable %q does not exist", name)
}
def = vr.Default
ectx = p.ectx
}
if def == nil {
@@ -270,10 +273,10 @@ func (p *parser) resolveValue(name string) (err error) {
return
}
if diags := p.loadDeps(def.Expr, nil); diags.HasErrors() {
if diags := p.loadDeps(ectx, def.Expr, nil, true); diags.HasErrors() {
return diags
}
vv, diags := def.Expr.Value(p.ectx)
vv, diags := def.Expr.Value(ectx)
if diags.HasErrors() {
return diags
}
@@ -312,154 +315,226 @@ func (p *parser) resolveValue(name string) (err error) {
// target schema is provided, only the attributes and blocks present in the
// schema will be evaluated.
func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) {
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid name",
Detail: err.Error(),
Subject: &block.LabelRanges[0],
},
}
// prepare the variable map for this type
if _, ok := p.ectx.Variables[block.Type]; !ok {
p.ectx.Variables[block.Type] = cty.MapValEmpty(cty.Map(cty.String))
}
if _, ok := p.doneB[block]; !ok {
p.doneB[block] = map[string]struct{}{}
}
if _, ok := p.progressB[block]; !ok {
p.progressB[block] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[block][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[block][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[block][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[block][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[block][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[block][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[block] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
// prepare the output destination and evaluation context
t, ok := p.blockTypes[block.Type]
if !ok {
return nil
}
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
var outputs []reflect.Value
var ectxs []*hcl.EvalContext
if prev, ok := p.blockValues[block]; ok {
outputs = prev
ectxs = p.blockEvalCtx[block]
} else {
if v, ok := reflect.New(t).Interface().(WithEvalContexts); ok {
ectxs, err = v.GetEvalContexts(p.ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(p.ectx, expr, nil, true)
})
if err != nil {
return err
}
for _, ectx := range ectxs {
if ectx != p.ectx && ectx.Parent() != p.ectx {
return errors.Errorf("EvalContext must return a context with the correct parent")
}
}
} else {
ectxs = append([]*hcl.EvalContext{}, p.ectx)
}
for range ectxs {
outputs = append(outputs, reflect.New(t))
}
}
for _, a := range content.Attributes {
diag := p.loadDeps(a.Expr, nil)
p.blockValues[block] = outputs
p.blockEvalCtx[block] = ectxs
for i, output := range outputs {
target := target
ectx := ectxs[i]
name := block.Labels[0]
if names, ok := p.blockNames[block]; ok {
name = names[i]
}
if _, ok := p.doneB[key(block, ectx)]; !ok {
p.doneB[key(block, ectx)] = map[string]struct{}{}
}
if _, ok := p.progressB[key(block, ectx)]; !ok {
p.progressB[key(block, ectx)] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[key(block, ectx)][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[key(block, ectx)][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[key(block, ectx)][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[key(block, ectx)][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[key(block, ectx)][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[key(block, ectx)] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
for _, a := range content.Attributes {
diag := p.loadDeps(ectx, a.Expr, nil, true)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
if err != nil {
return err
}
}
// decode!
diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() {
return diag
}
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
}
// store the result into the evaluation context (so it can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
}
// decode!
var output reflect.Value
if prev, ok := p.blockValues[block]; ok {
output = prev
} else {
output = reflect.New(t)
setLabel(output, block.Labels[0]) // early attach labels, so we can reference them
}
diag = gohcl.DecodeBody(body(), p.ectx, output.Interface())
if diag.HasErrors() {
return diag
}
p.blockValues[block] = output
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[block][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[block][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[block][a.Name] = struct{}{}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
for _, b := range target.Blocks {
p.doneB[block][b.Type] = struct{}{}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
}
// store the result into the evaluation context (so if can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
return nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
// resolveBlockNames returns the names of the block, calling resolveBlock to
// evaluate any label fields to correctly resolve the name.
func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
if names, ok := p.blockNames[block]; ok {
return names, nil
}
if err := p.resolveBlock(block, &hcl.BodySchema{}); err != nil {
return nil, err
}
names := make([]string, 0, len(p.blockValues[block]))
for i, val := range p.blockValues[block] {
ectx := p.blockEvalCtx[block][i]
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
if v, ok := val.Interface().(WithGetName); ok {
var err error
name, err = v.GetName(ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(ectx, expr, nil, true)
})
if err != nil {
return nil, err
}
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
}
setName(val, name)
names = append(names, name)
}
found := map[string]struct{}{}
for _, name := range names {
if _, ok := found[name]; ok {
return nil, errors.Errorf("duplicate name %q", name)
}
found[name] = struct{}{}
}
p.blockNames[block] = names
return names, nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) (map[string]map[string][]string, hcl.Diagnostics) {
reserved := map[string]struct{}{}
schema, _ := gohcl.ImpliedBodySchema(val)
@@ -472,7 +547,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
var defs inputs
if err := gohcl.DecodeBody(b, nil, &defs); err != nil {
return err
return nil, err
}
defsSchema, _ := gohcl.ImpliedBodySchema(defs)
@@ -495,20 +570,20 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
attrs: map[string]*hcl.Attribute{},
funcs: map[string]*functionDef{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block]reflect.Value{},
blockTypes: map[string]reflect.Type{},
progress: map[string]struct{}{},
progressF: map[string]struct{}{},
progressB: map[*hcl.Block]map[string]struct{}{},
doneF: map[string]struct{}{},
doneB: map[*hcl.Block]map[string]struct{}{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block][]reflect.Value{},
blockEvalCtx: map[*hcl.Block][]*hcl.EvalContext{},
blockNames: map[*hcl.Block][]string{},
blockTypes: map[string]reflect.Type{},
ectx: &hcl.EvalContext{
Variables: map[string]cty.Value{},
Functions: stdlibFunctions,
Functions: Stdlib(),
},
progressV: map[uint64]struct{}{},
progressF: map[uint64]struct{}{},
progressB: map[uint64]map[string]struct{}{},
doneB: map[uint64]map[string]struct{}{},
}
for _, v := range defs.Variables {
@@ -528,18 +603,18 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
content, b, diags := b.PartialContent(schema)
if diags.HasErrors() {
return diags
return nil, diags
}
blocks, b, diags := b.PartialContent(defsSchema)
if diags.HasErrors() {
return diags
return nil, diags
}
attrs, diags := b.JustAttributes()
if diags.HasErrors() {
if d := removeAttributesDiags(diags, reserved, p.vars); len(d) > 0 {
return d
return nil, d
}
}
@@ -552,11 +627,11 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
delete(p.attrs, "function")
for k := range p.opt.Vars {
_ = p.resolveValue(k)
_ = p.resolveValue(p.ectx, k)
}
for _, a := range content.Attributes {
return hcl.Diagnostics{
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
@@ -568,27 +643,19 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
}
for k := range p.vars {
if err := p.resolveValue(k); err != nil {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
return nil, diags
}
r := p.vars[k].Body.MissingItemRange()
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid value",
Detail: err.Error(),
Subject: &r,
Context: &r,
},
}
return nil, wrapErrorDiagnostic("Invalid value", err, &r, &r)
}
}
for k := range p.funcs {
if err := p.resolveFunction(k); err != nil {
if err := p.resolveFunction(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
return nil, diags
}
var subject *hcl.Range
var context *hcl.Range
@@ -604,40 +671,10 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
}
}
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid function",
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
return nil, wrapErrorDiagnostic("Invalid function", err, subject, context)
}
}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
bm, ok := p.blocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
p.blocks[b.Type] = bm
}
lbl := b.Labels[0]
bm[lbl] = append(bm[lbl], b)
}
type value struct {
reflect.Value
idx int
@@ -648,7 +685,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
values map[string]value
}
types := map[string]field{}
renamed := map[string]map[string][]string{}
vt := reflect.ValueOf(val).Elem().Type()
for i := 0; i < vt.NumField(); i++ {
tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",")
@@ -659,8 +696,40 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
typ: vt.Field(i).Type,
values: make(map[string]value),
}
renamed[tags[0]] = map[string][]string{}
}
tmpBlocks := map[string]map[string][]*hcl.Block{}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
bm, ok := tmpBlocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
tmpBlocks[b.Type] = bm
}
names, err := p.resolveBlockNames(b)
if err != nil {
return nil, wrapErrorDiagnostic("Invalid name", err, &b.LabelRanges[0], &b.LabelRanges[0])
}
for _, name := range names {
bm[name] = append(bm[name], b)
renamed[b.Type][b.Labels[0]] = append(renamed[b.Type][b.Labels[0]], name)
}
}
p.blocks = tmpBlocks
diags = hcl.Diagnostics{}
for _, b := range content.Blocks {
v := reflect.ValueOf(val)
@@ -673,86 +742,116 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
continue
}
} else {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &b.LabelRanges[0],
Context: &b.DefRange,
},
}
return nil, wrapErrorDiagnostic("Invalid block", err, &b.LabelRanges[0], &b.DefRange)
}
}
vv := p.blockValues[b]
t := types[b.Type]
lblIndex := setLabel(vv, b.Labels[0])
oldValue, exists := t.values[b.Labels[0]]
if !exists && lblIndex != -1 {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if b.Labels[0] == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
vvs := p.blockValues[b]
for _, vv := range vvs {
t := types[b.Type]
lblIndex, lblExists := getNameIndex(vv)
lblName, _ := getName(vv)
oldValue, exists := t.values[lblName]
if !exists && lblExists {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
}
}
}
}
}
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
}
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[lblName] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
} else {
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[b.Labels[0]] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
}
if diags.HasErrors() {
return diags
return nil, diags
}
for k := range p.attrs {
if err := p.resolveValue(k); err != nil {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return diags
}
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &p.attrs[k].Range,
Context: &p.attrs[k].Range,
},
return nil, diags
}
return nil, wrapErrorDiagnostic("Invalid attribute", err, &p.attrs[k].Range, &p.attrs[k].Range)
}
}
return nil
return renamed, nil
}
func setLabel(v reflect.Value, lbl string) int {
// cache field index?
// wrapErrorDiagnostic wraps an error into a hcl.Diagnostics object.
// If the error is already an hcl.Diagnostics object, it is returned as is.
func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context *hcl.Range) hcl.Diagnostics {
switch err := err.(type) {
case *hcl.Diagnostic:
return hcl.Diagnostics{err}
case hcl.Diagnostics:
return err
default:
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: message,
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
}
func setName(v reflect.Value, name string) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
for _, t := range strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",") {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
v.Elem().Field(i).Set(reflect.ValueOf(lbl))
return i
v.Elem().Field(i).Set(reflect.ValueOf(name))
}
}
}
return -1
}
func getName(v reflect.Value) (string, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return v.Elem().Field(i).String(), true
}
}
}
return "", false
}
func getNameIndex(v reflect.Value) (int, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return i, true
}
}
}
return 0, false
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable) hcl.Diagnostics {
@@ -784,3 +883,21 @@ func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{},
}
return fdiags
}
// key returns a unique hash for the given values
func key(ks ...any) uint64 {
hash := fnv.New64a()
for _, k := range ks {
v := reflect.ValueOf(k)
switch v.Kind() {
case reflect.String:
hash.Write([]byte(v.String()))
case reflect.Pointer:
ptr := reflect.ValueOf(k).Pointer()
binary.Write(hash, binary.LittleEndian, uint64(ptr))
default:
panic(fmt.Sprintf("unknown key kind %s", v.Kind().String()))
}
}
return hash.Sum64()
}

View File

@@ -31,21 +31,21 @@ var stdlibFunctions = map[string]function.Function{
"cidrnetmask": cidr.NetmaskFunc,
"cidrsubnet": cidr.SubnetFunc,
"cidrsubnets": cidr.SubnetsFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"convert": typeexpr.ConvertFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"distinct": stdlib.DistinctFunc,
"divide": stdlib.DivideFunc,
"element": stdlib.ElementFunc,
"equal": stdlib.EqualFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"formatdate": stdlib.FormatDateFunc,
"format": stdlib.FormatFunc,
"formatdate": stdlib.FormatDateFunc,
"formatlist": stdlib.FormatListFunc,
"greaterthan": stdlib.GreaterThanFunc,
"greaterthanorequalto": stdlib.GreaterThanOrEqualToFunc,
@@ -53,10 +53,10 @@ var stdlibFunctions = map[string]function.Function{
"indent": stdlib.IndentFunc,
"index": stdlib.IndexFunc,
"int": stdlib.IntFunc,
"join": stdlib.JoinFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"join": stdlib.JoinFunc,
"length": stdlib.LengthFunc,
"lessthan": stdlib.LessThanFunc,
"lessthanorequalto": stdlib.LessThanOrEqualToFunc,
@@ -70,15 +70,16 @@ var stdlibFunctions = map[string]function.Function{
"modulo": stdlib.ModuloFunc,
"multiply": stdlib.MultiplyFunc,
"negate": stdlib.NegateFunc,
"notequal": stdlib.NotEqualFunc,
"not": stdlib.NotFunc,
"notequal": stdlib.NotEqualFunc,
"or": stdlib.OrFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regexall": stdlib.RegexAllFunc,
"regex": stdlib.RegexFunc,
"regex_replace": stdlib.RegexReplaceFunc,
"regex": stdlib.RegexFunc,
"regexall": stdlib.RegexAllFunc,
"replace": stdlib.ReplaceFunc,
"reverse": stdlib.ReverseFunc,
"reverselist": stdlib.ReverseListFunc,
"rsadecrypt": crypto.RsaDecryptFunc,
@@ -124,3 +125,11 @@ var timestampFunc = function.New(&function.Spec{
return cty.StringVal(time.Now().UTC().Format(time.RFC3339)), nil
},
})
func Stdlib() map[string]function.Function {
funcs := make(map[string]function.Function, len(stdlibFunctions))
for k, v := range stdlibFunctions {
funcs[k] = v
}
return funcs
}

View File

@@ -4,14 +4,16 @@ import (
"archive/tar"
"bytes"
"context"
"strings"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
)
@@ -21,10 +23,16 @@ type Input struct {
}
func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
var session []session.Attachable
var filename string
st, ok := detectGitContext(url)
if !ok {
st, filename, ok = detectHTTPContext(url)
st, ok := dockerui.DetectGitContext(url, false)
if ok {
ssh, err := controllerapi.CreateSSH([]*controllerapi.SSH{{ID: "default"}})
if err == nil {
session = append(session, ssh)
}
} else {
st, filename, ok = dockerui.DetectHTTPContext(url)
if !ok {
return nil, nil, errors.Errorf("not url context")
}
@@ -34,9 +42,9 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
var files []File
var node *builder.Node
for _, n := range nodes {
for i, n := range nodes {
if n.Err == nil {
node = &n
node = &nodes[i]
continue
}
}
@@ -51,7 +59,7 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Session: session}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
@@ -83,51 +91,6 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
return files, inp, nil
}
func IsRemoteURL(url string) bool {
if _, _, ok := detectHTTPContext(url); ok {
return true
}
if _, ok := detectGitContext(url); ok {
return true
}
return false
}
func detectHTTPContext(url string) (*llb.State, string, bool) {
if httpPrefix.MatchString(url) {
httpContext := llb.HTTP(url, llb.Filename("context"), llb.WithCustomName("[internal] load remote build context"))
return &httpContext, "context", true
}
return nil, "", false
}
func detectGitContext(ref string) (*llb.State, bool) {
found := false
if httpPrefix.MatchString(ref) && gitURLPathWithFragmentSuffix.MatchString(ref) {
found = true
}
for _, prefix := range []string{"git://", "github.com/", "git@"} {
if strings.HasPrefix(ref, prefix) {
found = true
break
}
}
if !found {
return nil, false
}
parts := strings.SplitN(ref, "#", 2)
branch := ""
if len(parts) > 1 {
branch = parts[1]
}
gitOpts := []llb.GitOption{llb.WithCustomName("[internal] load git source " + ref)}
st := llb.Git(parts[0], branch, gitOpts...)
return &st, true
}
func isArchive(header []byte) bool {
for _, m := range [][]byte{
{0x42, 0x5A, 0x68}, // bzip2

File diff suppressed because it is too large Load Diff

View File

@@ -52,22 +52,28 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd))
if err != nil {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("git was not found in the system. Current commit information was not captured by the build")
return res, errors.New("buildx: git was not found in the system. Current commit information was not captured by the build")
}
return
}
if !gitc.IsInsideWorkTree() {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("failed to read current commit information with git rev-parse --is-inside-work-tree")
return res, errors.New("buildx: failed to read current commit information with git rev-parse --is-inside-work-tree")
}
return res, nil
}
if sha, err := gitc.FullCommit(); err != nil {
return res, errors.Wrapf(err, "failed to get git commit")
if sha, err := gitc.FullCommit(); err != nil && !gitutil.IsUnknownRevision(err) {
return res, errors.Wrapf(err, "buildx: failed to get git commit")
} else if sha != "" {
if gitc.IsDirty() {
checkDirty := false
if v, ok := os.LookupEnv("BUILDX_GIT_CHECK_DIRTY"); ok {
if v, err := strconv.ParseBool(v); err == nil {
checkDirty = v
}
}
if checkDirty && gitc.IsDirty() {
sha += "-dirty"
}
if setGitLabels {
@@ -78,9 +84,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
}
}
if rurl, err := gitc.RemoteURL(); err != nil {
return res, errors.Wrapf(err, "failed to get git remote url")
} else if rurl != "" {
if rurl, err := gitc.RemoteURL(); err == nil && rurl != "" {
if setGitLabels {
res["label:"+specs.AnnotationSource] = rurl
}
@@ -91,7 +95,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
if setGitLabels {
if root, err := gitc.RootDir(); err != nil {
return res, errors.Wrapf(err, "failed to get git root dir")
return res, errors.Wrapf(err, "buildx: failed to get git root dir")
} else if root != "" {
if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile")

View File

@@ -26,7 +26,7 @@ func setupTest(tb testing.TB) {
gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit")
gitutil.GitSetRemote(c, tb, "git@github.com:docker/buildx.git")
gitutil.GitSetRemote(c, tb, "origin", "git@github.com:docker/buildx.git")
}
func TestGetGitAttributesNotGitRepo(t *testing.T) {
@@ -131,6 +131,7 @@ func TestGetGitAttributes(t *testing.T) {
func TestGetGitAttributesDirty(t *testing.T) {
setupTest(t)
t.Setenv("BUILDX_GIT_CHECK_DIRTY", "true")
// make a change to test dirty flag
df := []byte("FROM alpine:edge\n")

138
build/invoke.go Normal file
View File

@@ -0,0 +1,138 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"io"
"sync"
"sync/atomic"
"syscall"
controllerapi "github.com/docker/buildx/controller/pb"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type Container struct {
cancelOnce sync.Once
containerCancel func()
isUnavailable atomic.Bool
initStarted atomic.Bool
container gateway.Container
releaseCh chan struct{}
resultCtx *ResultHandle
}
func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig) (*Container, error) {
mainCtx := ctx
ctrCh := make(chan *Container)
errCh := make(chan error)
go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancel(ctx)
go func() {
<-mainCtx.Done()
cancel()
}()
containerCfg, err := resultCtx.getContainerConfig(ctx, c, cfg)
if err != nil {
return nil, err
}
containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil {
return nil, err
}
releaseCh := make(chan struct{})
container := &Container{
containerCancel: containerCancel,
container: bkContainer,
releaseCh: releaseCh,
resultCtx: resultCtx,
}
doneCh := make(chan struct{})
defer close(doneCh)
resultCtx.registerCleanup(func() {
container.Cancel()
<-doneCh
})
ctrCh <- container
<-container.releaseCh
return nil, bkContainer.Release(ctx)
})
if err != nil {
errCh <- err
}
}()
select {
case ctr := <-ctrCh:
return ctr, nil
case err := <-errCh:
return nil, err
case <-mainCtx.Done():
return nil, mainCtx.Err()
}
}
func (c *Container) Cancel() {
c.markUnavailable()
c.cancelOnce.Do(func() {
if c.containerCancel != nil {
c.containerCancel()
}
close(c.releaseCh)
})
}
func (c *Container) IsUnavailable() bool {
return c.isUnavailable.Load()
}
func (c *Container) markUnavailable() {
c.isUnavailable.Store(true)
}
func (c *Container) Exec(ctx context.Context, cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if isInit := c.initStarted.CompareAndSwap(false, true); isInit {
defer func() {
// container can't be used after init exits
c.markUnavailable()
}()
}
err := exec(ctx, c.resultCtx, cfg, c.container, stdin, stdout, stderr)
if err != nil {
// Container becomes unavailable if one of the processes fails in it.
c.markUnavailable()
}
return err
}
func exec(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig, ctr gateway.Container, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
processCfg, err := resultCtx.getProcessConfig(cfg, stdin, stdout, stderr)
if err != nil {
return err
}
proc, err := ctr.Start(ctx, processCfg)
if err != nil {
return errors.Errorf("failed to start container: %v", err)
}
doneCh := make(chan struct{})
defer close(doneCh)
go func() {
select {
case <-ctx.Done():
if err := proc.Signal(ctx, syscall.SIGKILL); err != nil {
logrus.Warnf("failed to kill process: %v", err)
}
case <-doneCh:
}
}()
return proc.Wait()
}

494
build/result.go Normal file
View File

@@ -0,0 +1,494 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"encoding/json"
"io"
"sync"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/solver/result"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
// NewResultHandle makes a call to client.Build, additionally returning a
// opaque ResultHandle alongside the standard response and error.
//
// This ResultHandle can be used to execute additional build steps in the same
// context as the build occurred, which can allow easy debugging of build
// failures and successes.
//
// If the returned ResultHandle is not nil, the caller must call Done() on it.
func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt, product string, buildFunc gateway.BuildFunc, ch chan *client.SolveStatus) (*ResultHandle, *client.SolveResponse, error) {
// Create a new context to wrap the original, and cancel it when the
// caller-provided context is cancelled.
//
// We derive the context from the background context so that we can forbid
// cancellation of the build request after <-done is closed (which we do
// before returning the ResultHandle).
baseCtx := ctx
ctx, cancel := context.WithCancelCause(context.Background())
done := make(chan struct{})
go func() {
select {
case <-baseCtx.Done():
cancel(baseCtx.Err())
case <-done:
// Once done is closed, we've recorded a ResultHandle, so we
// shouldn't allow cancelling the underlying build request anymore.
}
}()
// Create a new channel to forward status messages to the original.
//
// We do this so that we can discard status messages after the main portion
// of the build is complete. This is necessary for the solve error case,
// where the original gateway is kept open until the ResultHandle is
// closed - we don't want progress messages from operations in that
// ResultHandle to display after this function exits.
//
// Additionally, callers should wait for the progress channel to be closed.
// If we keep the session open and never close the progress channel, the
// caller will likely hang.
baseCh := ch
ch = make(chan *client.SolveStatus)
go func() {
for {
s, ok := <-ch
if !ok {
return
}
select {
case <-baseCh:
// base channel is closed, discard status messages
default:
baseCh <- s
}
}
}()
defer close(baseCh)
var resp *client.SolveResponse
var respErr error
var respHandle *ResultHandle
go func() {
defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result
var err error
resp, err = cc.Build(ctx, opt, product, func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
var err error
res, err = buildFunc(ctx, c)
if res != nil && err == nil {
// Force evaluation of the build result (otherwise, we likely
// won't get a solve error)
def, err2 := getDefinition(ctx, res)
if err2 != nil {
return nil, err2
}
res, err = evalDefinition(ctx, c, def)
}
if err != nil {
// Scenario 1: we failed to evaluate a node somewhere in the
// build graph.
//
// In this case, we construct a ResultHandle from this
// original Build session, and return it alongside the original
// build error. We then need to keep the gateway session open
// until the caller explicitly closes the ResultHandle.
var se *errdefs.SolveError
if errors.As(err, &se) {
respHandle = &ResultHandle{
done: make(chan struct{}),
solveErr: se,
gwClient: c,
gwCtx: ctx,
}
respErr = se
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
}
}
return res, err
}, ch)
if respHandle != nil {
return
}
if err != nil {
// Something unexpected failed during the build, we didn't succeed,
// but we also didn't make it far enough to create a ResultHandle.
respErr = err
close(done)
return
}
// Scenario 2: we successfully built the image with no errors.
//
// In this case, the original gateway session has now been closed
// since the Build has been completed. So, we need to create a new
// gateway session to populate the ResultHandle. To do this, we
// need to re-evaluate the target result, in this new session. This
// should be instantaneous since the result should be cached.
def, err := getDefinition(ctx, res)
if err != nil {
respErr = err
close(done)
return
}
// NOTE: ideally this second connection should be lazily opened
opt := opt
opt.Ref = ""
opt.Exports = nil
opt.CacheExports = nil
_, respErr = cc.Build(ctx, opt, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
res, err := evalDefinition(ctx, c, def)
if err != nil {
// This should probably not happen, since we've previously
// successfully evaluated the same result with no issues.
return nil, errors.Wrap(err, "inconsistent solve result")
}
respHandle = &ResultHandle{
done: make(chan struct{}),
res: res,
gwClient: c,
gwCtx: ctx,
}
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
return nil, ctx.Err()
}, nil)
if respHandle != nil {
return
}
close(done)
}()
// Block until the other thread signals that it's completed the build.
select {
case <-done:
case <-baseCtx.Done():
if respErr == nil {
respErr = baseCtx.Err()
}
}
return respHandle, resp, respErr
}
// getDefinition converts a gateway result into a collection of definitions for
// each ref in the result.
func getDefinition(ctx context.Context, res *gateway.Result) (*result.Result[*pb.Definition], error) {
return result.ConvertResult(res, func(ref gateway.Reference) (*pb.Definition, error) {
st, err := ref.ToState()
if err != nil {
return nil, err
}
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
}
return def.ToPB(), nil
})
}
// evalDefinition performs the reverse of getDefinition, converting a
// collection of definitions into a gateway result.
func evalDefinition(ctx context.Context, c gateway.Client, defs *result.Result[*pb.Definition]) (*gateway.Result, error) {
// force evaluation of all targets in parallel
results := make(map[*pb.Definition]*gateway.Result)
resultsMu := sync.Mutex{}
eg, egCtx := errgroup.WithContext(ctx)
defs.EachRef(func(def *pb.Definition) error {
eg.Go(func() error {
res, err := c.Solve(egCtx, gateway.SolveRequest{
Evaluate: true,
Definition: def,
})
if err != nil {
return err
}
resultsMu.Lock()
results[def] = res
resultsMu.Unlock()
return nil
})
return nil
})
if err := eg.Wait(); err != nil {
return nil, err
}
res, _ := result.ConvertResult(defs, func(def *pb.Definition) (gateway.Reference, error) {
if res, ok := results[def]; ok {
return res.Ref, nil
}
return nil, nil
})
return res, nil
}
// ResultHandle is a build result with the client that built it.
type ResultHandle struct {
res *gateway.Result
solveErr *errdefs.SolveError
done chan struct{}
doneOnce sync.Once
gwClient gateway.Client
gwCtx context.Context
cleanups []func()
cleanupsMu sync.Mutex
}
func (r *ResultHandle) Done() {
r.doneOnce.Do(func() {
r.cleanupsMu.Lock()
cleanups := r.cleanups
r.cleanups = nil
r.cleanupsMu.Unlock()
for _, f := range cleanups {
f()
}
close(r.done)
<-r.gwCtx.Done()
})
}
func (r *ResultHandle) registerCleanup(f func()) {
r.cleanupsMu.Lock()
r.cleanups = append(r.cleanups, f)
r.cleanupsMu.Unlock()
}
func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
_, err = buildFunc(r.gwCtx, r.gwClient)
return err
}
func (r *ResultHandle) getContainerConfig(ctx context.Context, c gateway.Client, cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(ctx, r.res, c, *cfg)
if err != nil {
return containerCfg, err
}
containerCfg = *ccfg
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available")
}
containerCfg = *ccfg
}
return containerCfg, nil
}
func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) (_ gateway.StartRequest, err error) {
processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err
}
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err
}
}
return processCfg, nil
}
func containerConfigFromResult(ctx context.Context, res *gateway.Result, c gateway.Client, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
}
ps, err := exptypes.ParsePlatforms(res.Metadata)
if err != nil {
return nil, err
}
ref, ok := res.FindRef(ps.Platforms[0].ID)
if !ok {
return nil, errors.Errorf("no reference found")
}
return &gateway.NewContainerRequest{
Mounts: []gateway.Mount{
{
Dest: "/",
MountType: pb.MountType_BIND,
Ref: ref,
},
},
}, nil
}
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image
if len(imgData) > 0 {
img = &specs.Image{}
if err := json.Unmarshal(imgData, img); err != nil {
return err
}
}
user := ""
if !cfg.NoUser {
user = cfg.User
} else if img != nil {
user = img.Config.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else if img != nil {
cwd = img.Config.WorkingDir
}
env := []string{}
if img != nil {
env = append(env, img.Config.Env...)
}
env = append(env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
} else if img != nil {
args = append(args, img.Config.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
} else if img != nil {
args = append(args, img.Config.Cmd...)
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr)
if err != nil {
return nil, err
}
var mounts []gateway.Mount
for i, mnt := range exec.Mounts {
rid := solveErr.Solve.MountIDs[i]
if cfg.Initial {
rid = solveErr.Solve.InputIDs[i]
}
mounts = append(mounts, gateway.Mount{
Selector: mnt.Selector,
Dest: mnt.Dest,
ResultID: rid,
Readonly: mnt.Readonly,
MountType: mnt.MountType,
CacheOpt: mnt.CacheOpt,
SecretOpt: mnt.SecretOpt,
SSHOpt: mnt.SSHOpt,
})
}
return &gateway.NewContainerRequest{
Mounts: mounts,
NetMode: exec.Network,
}, nil
}
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr)
if err != nil {
return err
}
meta := exec.Meta
user := ""
if !cfg.NoUser {
user = cfg.User
} else {
user = meta.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else {
cwd = meta.Cwd
}
env := append(meta.Env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
}
if len(args) == 0 {
args = meta.Args
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func execOpFromError(solveErr *errdefs.SolveError) (*pb.ExecOp, error) {
if solveErr == nil {
return nil, errors.Errorf("no error is available")
}
switch op := solveErr.Solve.Op.GetOp().(type) {
case *pb.Op_Exec:
return op.Exec, nil
default:
return nil, errors.Errorf("invoke: unsupported error type")
}
// TODO: support other ops
}
func newStartRequest(stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) gateway.StartRequest {
return gateway.StartRequest{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
}
}

View File

@@ -13,7 +13,7 @@ import (
"github.com/pkg/errors"
)
func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url string, pw progress.Writer) (string, error) {
func createTempDockerfileFromURL(ctx context.Context, d *driver.DriverHandle, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil {
return "", err

View File

@@ -8,6 +8,8 @@ import (
"strings"
"github.com/docker/cli/opts"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
"github.com/pkg/errors"
)
@@ -20,6 +22,16 @@ const (
mobyHostGatewayName = "host-gateway"
)
func IsRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}
func isLocalDir(c string) bool {
st, err := os.Stat(c)
return err == nil && st.IsDir()

View File

@@ -108,7 +108,7 @@ func New(dockerCli command.Cli, opts ...Option) (_ *Builder, err error) {
// Validate validates builder context
func (b *Builder) Validate() error {
if b.NodeGroup.DockerContext {
if b.NodeGroup != nil && b.NodeGroup.DockerContext {
list, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return err

View File

@@ -10,6 +10,7 @@ import (
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/platformutil"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
@@ -20,9 +21,12 @@ import (
type Node struct {
store.Node
Driver driver.Driver
Builder string
Driver *driver.DriverHandle
DriverInfo *driver.Info
Platforms []ocispecs.Platform
GCPolicy []client.PruneInfo
Labels map[string]string
ImageOpt imagetools.Opt
ProxyConfig map[string]string
Version string
@@ -62,6 +66,8 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
node := Node{
Node: n,
ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli),
Platforms: n.Platforms,
Builder: b.Name,
}
defer func() {
b.nodes[i] = node
@@ -181,8 +187,12 @@ func (n *Node) loadData(ctx context.Context) error {
if err != nil {
return errors.Wrap(err, "listing workers")
}
for _, w := range workers {
for idx, w := range workers {
n.Platforms = append(n.Platforms, w.Platforms...)
if idx == 0 {
n.GCPolicy = w.GCPolicy
n.Labels = w.Labels
}
}
n.Platforms = platformutil.Dedupe(n.Platforms)
inf, err := driverClient.Info(ctx)

View File

@@ -4,8 +4,8 @@ import (
"fmt"
"os"
"github.com/containerd/containerd/pkg/seed"
"github.com/docker/buildx/commands"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/manager"
@@ -16,10 +16,10 @@ import (
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack"
_ "k8s.io/client-go/plugin/pkg/client/auth/azure"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
@@ -28,7 +28,9 @@ import (
)
func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision)
}
@@ -85,6 +87,9 @@ func main() {
} else {
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
}
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
ebr.Print(cmd.Err())
}
os.Exit(1)
}

View File

@@ -6,12 +6,15 @@ import (
"fmt"
"os"
"github.com/containerd/console"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/bake"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/util/tracing"
@@ -22,13 +25,19 @@ import (
)
type bakeOptions struct {
files []string
overrides []string
printOnly bool
commonOptions
files []string
overrides []string
printOnly bool
sbom string
provenance string
builder string
metadataFile string
exportPush bool
exportLoad bool
}
func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error) {
func runBake(dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
ctx := appcontext.Context()
ctx, end, err := tracing.TraceCurrentCommand(ctx, "bake")
@@ -43,11 +52,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
cmdContext := "cwd://"
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
url = targets[0]
targets = targets[1:]
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
cmdContext = targets[0]
targets = targets[1:]
}
@@ -68,11 +77,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
} else if in.exportLoad {
overrides = append(overrides, "*.output=type=docker")
}
if in.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *in.noCache))
if cFlags.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *cFlags.noCache))
}
if in.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *in.pull))
if cFlags.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *cFlags.pull))
}
if in.sbom != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("sbom", in.sbom)))
@@ -84,23 +93,11 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
if err != nil {
return err
}
defer func() {
if printer != nil {
err1 := printer.Wait()
if err == nil {
err = err1
}
}
}()
var nodes []builder.Node
var files []bake.File
var inp *bake.Input
var progressConsoleDesc, progressTextDesc string
// instance only needed for reading remote bake files or building
if url != "" || !in.printOnly {
@@ -118,12 +115,38 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
if err != nil {
return err
}
progressConsoleDesc = fmt.Sprintf("%s:%s", b.Driver, b.Name)
progressTextDesc = fmt.Sprintf("building with %q instance using %s driver", b.Name, b.Driver)
}
var term bool
if _, err := console.ConsoleFromFile(os.Stderr); err == nil {
term = true
}
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, cFlags.progress,
progress.WithDesc(progressTextDesc, progressConsoleDesc),
)
if err != nil {
return err
}
defer func() {
if printer != nil {
err1 := printer.Wait()
if err == nil {
err = err1
}
if err == nil && cFlags.progress != progress.PrinterModeQuiet {
desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term)
}
}
}()
if url != "" {
files, inp, err = bake.ReadRemoteFiles(ctx, nodes, url, in.files, printer)
} else {
files, err = bake.ReadLocalFiles(in.files)
files, err = bake.ReadLocalFiles(in.files, dockerCli.In())
}
if err != nil {
return err
@@ -131,7 +154,7 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, map[string]string{
// don't forget to update documentation if you add a new
// built-in variable: docs/manuals/bake/file-definition.md#built-in-variables
// built-in variable: docs/bake-reference.md#built-in-variables
"BAKE_CMD_CONTEXT": cmdContext,
"BAKE_LOCAL_PLATFORM": platforms.DefaultString(),
})
@@ -139,6 +162,19 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return err
}
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
// TODO: extract env var parsing to a method easily usable by library consumers
for _, t := range tgts {
if _, ok := t.Args["SOURCE_DATE_EPOCH"]; ok {
continue
}
if t.Args == nil {
t.Args = map[string]*string{}
}
t.Args["SOURCE_DATE_EPOCH"] = &v
}
}
// this function can update target context string from the input so call before printOnly check
bo, err := bake.TargetsToBuildOpt(tgts, inp)
if err != nil {
@@ -185,6 +221,7 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
var options bakeOptions
var cFlags commonFlags
cmd := &cobra.Command{
Use: "bake [OPTIONS] [TARGET...]",
@@ -193,14 +230,17 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
// reset to nil to avoid override is unset
if !cmd.Flags().Lookup("no-cache").Changed {
options.noCache = nil
cFlags.noCache = nil
}
if !cmd.Flags().Lookup("pull").Changed {
options.pull = nil
cFlags.pull = nil
}
options.commonOptions.builder = rootOpts.builder
return runBake(dockerCli, args, options)
options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function.
return runBake(dockerCli, args, options, cFlags)
},
ValidArgsFunction: completion.BakeTargets(options.files),
}
flags := cmd.Flags()
@@ -213,7 +253,7 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
flags.StringArrayVar(&options.overrides, "set", nil, `Override target value (e.g., "targetpattern.key=value")`)
commonBuildFlags(&options.commonOptions, flags)
commonBuildFlags(&cFlags, flags)
return cmd
}

File diff suppressed because it is too large Load Diff

View File

@@ -12,10 +12,13 @@ import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
k8sutil "github.com/docker/buildx/driver/kubernetes/util"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
@@ -169,18 +172,34 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if err := ng.Leave(in.nodeName); err != nil {
return err
}
ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
if err := ls.RemoveBuilderNode(ng.Name, in.nodeName); err != nil {
return err
}
} else {
switch {
case driverName == "kubernetes":
if len(args) > 0 {
logrus.Warnf("kubernetes driver does not support endpoint args %q", args[0])
}
// generate node name if not provided to avoid duplicated endpoint
// error: https://github.com/docker/setup-buildx-action/issues/215
nodeName := in.nodeName
if nodeName == "" {
nodeName, err = k8sutil.GenerateNodeName(name, txn)
if err != nil {
return err
}
}
// naming endpoint to make --append works
ep = (&url.URL{
Scheme: driverName,
Path: "/" + in.name,
Path: "/" + name,
RawQuery: (&url.Values{
"deployment": {in.nodeName},
"deployment": {nodeName},
"kubeconfig": {os.Getenv("KUBECONFIG")},
}).Encode(),
}).String()
@@ -309,6 +328,7 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

70
commands/debug-shell.go Normal file
View File

@@ -0,0 +1,70 @@
package commands
import (
"context"
"os"
"runtime"
"github.com/containerd/console"
"github.com/docker/buildx/controller"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/monitor"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
func debugShellCmd(dockerCli command.Cli) *cobra.Command {
var options control.ControlOptions
var progressMode string
cmd := &cobra.Command{
Use: "debug-shell",
Short: "Start a monitor",
RunE: func(cmd *cobra.Command, args []string) error {
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, os.Stderr, progressMode)
if err != nil {
return err
}
ctx := context.TODO()
c, err := controller.NewController(ctx, options, dockerCli, printer)
if err != nil {
return err
}
defer func() {
if err := c.Close(); err != nil {
logrus.Warnf("failed to close server connection %v", err)
}
}()
con := console.Current()
if err := con.SetRaw(); err != nil {
return errors.Errorf("failed to configure terminal: %v", err)
}
err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true,
}, c, dockerCli.In(), os.Stdout, os.Stderr, printer)
con.Reset()
return err
},
}
flags := cmd.Flags()
flags.StringVar(&options.Root, "root", "", "Specify root directory of server to connect [experimental]")
flags.BoolVar(&options.Detach, "detach", runtime.GOOS == "linux", "Detach buildx server (supported only on linux) [experimental]")
flags.StringVar(&options.ServerConfig, "server-config", "", "Specify buildx server config file (used only when launching new server) [experimental]")
flags.StringVar(&progressMode, "progress", "auto", `Set type of progress output ("auto", "plain", "tty"). Use plain to show container output`)
return cmd
}
func addDebugShellCommand(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
debugShellCmd(dockerCli),
)
}

View File

@@ -9,6 +9,7 @@ import (
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
@@ -115,6 +116,7 @@ func duCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runDiskUsage(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -8,6 +8,7 @@ import (
"strings"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
@@ -273,6 +274,7 @@ func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
options.builder = *opts.Builder
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -2,6 +2,7 @@ package commands
import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
@@ -52,6 +53,7 @@ func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
options.builder = *rootOpts.Builder
return runInspect(dockerCli, options, args[0])
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -1,6 +1,7 @@
package commands
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
@@ -11,8 +12,9 @@ type RootOptions struct {
func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "imagetools",
Short: "Commands to work on images in registry",
Use: "imagetools",
Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(

View File

@@ -4,14 +4,19 @@ import (
"context"
"fmt"
"os"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/docker/go-units"
"github.com/moby/buildkit/util/appcontext"
"github.com/spf13/cobra"
)
@@ -89,6 +94,38 @@ func runInspect(dockerCli command.Cli, in inspectOptions) error {
fmt.Fprintf(w, "Buildkit:\t%s\n", nodes[i].Version)
}
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
if debug.IsEnabled() {
fmt.Fprintf(w, "Features:\n")
features := nodes[i].Driver.Features(ctx)
featKeys := make([]string, 0, len(features))
for k := range features {
featKeys = append(featKeys, string(k))
}
sort.Strings(featKeys)
for _, k := range featKeys {
fmt.Fprintf(w, "\t%s:\t%t\n", k, features[driver.Feature(k)])
}
}
if len(nodes[i].Labels) > 0 {
fmt.Fprintf(w, "Labels:\n")
for _, k := range sortedKeys(nodes[i].Labels) {
v := nodes[i].Labels[k]
fmt.Fprintf(w, "\t%s:\t%s\n", k, v)
}
}
for ri, rule := range nodes[i].GCPolicy {
fmt.Fprintf(w, "GC Policy rule#%d:\n", ri)
fmt.Fprintf(w, "\tAll:\t%v\n", rule.All)
if len(rule.Filter) > 0 {
fmt.Fprintf(w, "\tFilters:\t%s\n", strings.Join(rule.Filter, " "))
}
if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
}
if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
}
}
}
}
@@ -112,6 +149,7 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runInspect(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
@@ -119,3 +157,14 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
return cmd
}
func sortedKeys(m map[string]string) []string {
s := make([]string, len(m))
i := 0
for k := range m {
s[i] = k
i++
}
sort.Strings(s)
return s
}

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -46,7 +47,8 @@ func installCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runInstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -11,6 +11,7 @@ import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -126,6 +127,7 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runLs(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -1,48 +0,0 @@
package commands
import (
"fmt"
"io"
"log"
"os"
"github.com/docker/buildx/build"
"github.com/docker/docker/api/types/versions"
"github.com/moby/buildkit/frontend/subrequests"
"github.com/moby/buildkit/frontend/subrequests/outline"
"github.com/moby/buildkit/frontend/subrequests/targets"
)
func printResult(f *build.PrintFunc, res map[string]string) error {
switch f.Name {
case "outline":
return printValue(outline.PrintOutline, outline.SubrequestsOutlineDefinition.Version, f.Format, res)
case "targets":
return printValue(targets.PrintTargets, targets.SubrequestsTargetsDefinition.Version, f.Format, res)
case "subrequests.describe":
return printValue(subrequests.PrintDescribe, subrequests.SubrequestsDescribeDefinition.Version, f.Format, res)
default:
if dt, ok := res["result.txt"]; ok {
fmt.Print(dt)
} else {
log.Printf("%s %+v", f, res)
}
}
return nil
}
type printFunc func([]byte, io.Writer) error
func printValue(printer printFunc, version string, format string, res map[string]string) error {
if format == "json" {
fmt.Fprintln(os.Stdout, res["result.json"])
return nil
}
if res["version"] != "" && versions.LessThan(version, res["version"]) && res["result.txt"] != "" {
// structure is too new and we don't know how to print it
fmt.Fprint(os.Stdout, res["result.txt"])
return nil
}
return printer([]byte(res["result.json"]), os.Stdout)
}

View File

@@ -8,6 +8,7 @@ import (
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
@@ -139,6 +140,7 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runPrune(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()

View File

@@ -8,6 +8,7 @@ import (
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
@@ -92,6 +93,7 @@ func rmCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runRm(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
@@ -141,9 +143,6 @@ func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, i
if err != nil {
return errors.Wrapf(err, "cannot load %s", b.Name)
}
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
if b.Dynamic {
return nil
}

View File

@@ -4,6 +4,8 @@ import (
"os"
imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/logutil"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
@@ -22,6 +24,9 @@ func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Comman
Annotations: map[string]string{
annotation.CodeDelimiter: `"`,
},
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
}
if isPlugin {
cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
@@ -86,6 +91,15 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
)
if isExperimental() {
remote.AddControllerCommands(cmd, dockerCli)
addDebugShellCommand(cmd, dockerCli)
}
cmd.RegisterFlagCompletionFunc( //nolint:errcheck
"builder",
completion.BuilderNames(dockerCli),
)
}
func rootFlags(options *rootOptions, flags *pflag.FlagSet) {

View File

@@ -4,6 +4,7 @@ import (
"context"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
@@ -46,6 +47,7 @@ func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runStop(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
return cmd

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -52,7 +53,8 @@ func uninstallCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runUninstall(dockerCli, options)
},
Hidden: true,
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

View File

@@ -4,6 +4,7 @@ import (
"os"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -78,6 +79,7 @@ func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runUse(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -23,6 +24,7 @@ func versionCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runVersion(dockerCli)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command

266
controller/build/build.go Normal file
View File

@@ -0,0 +1,266 @@
package build
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"sync"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
)
const defaultTargetName = "default"
// RunBuild runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
}
contexts := map[string]build.NamedContext{}
for name, path := range in.NamedContexts {
contexts[name] = build.NamedContext{Path: path}
}
opts := build.Options{
Inputs: build.Inputs{
ContextPath: in.ContextPath,
DockerfilePath: in.DockerfileName,
InStream: inStream,
NamedContexts: contexts,
},
BuildArgs: in.BuildArgs,
ExtraHosts: in.ExtraHosts,
Labels: in.Labels,
NetworkMode: in.NetworkMode,
NoCache: in.NoCache,
NoCacheFilter: in.NoCacheFilter,
Pull: in.Pull,
ShmSize: dockeropts.MemBytes(in.ShmSize),
Tags: in.Tags,
Target: in.Target,
Ulimits: controllerUlimitOpt2DockerUlimit(in.Ulimits),
}
platforms, err := platformutil.Parse(in.Platforms)
if err != nil {
return nil, nil, err
}
opts.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(dockerConfig))
secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, secrets)
sshSpecs := in.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(in.ContextPath) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, ssh)
outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil {
return nil, nil, err
}
if in.ExportPush {
if in.ExportLoad {
return nil, nil, errors.Errorf("push and load may not be set together at the moment")
}
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
}}
} else {
switch outputs[0].Type {
case "image":
outputs[0].Attrs["push"] = "true"
default:
return nil, nil, errors.Errorf("push and %q output can't be used together", outputs[0].Type)
}
}
}
if in.ExportLoad {
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "docker",
Attrs: map[string]string{},
}}
} else {
switch outputs[0].Type {
case "docker":
default:
return nil, nil, errors.Errorf("load and %q output can't be used together", outputs[0].Type)
}
}
}
opts.Exports = outputs
opts.CacheFrom = controllerapi.CreateCaches(in.CacheFrom)
opts.CacheTo = controllerapi.CreateCaches(in.CacheTo)
opts.Attests = controllerapi.CreateAttestations(in.Attests)
opts.SourcePolicy = in.SourcePolicy
allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil {
return nil, nil, err
}
opts.Allow = allow
if in.PrintFunc != nil {
opts.PrintFunc = &build.PrintFunc{
Name: in.PrintFunc.Name,
Format: in.PrintFunc.Format,
}
}
// key string used for kubernetes "sticky" mode
contextPathHash, err := filepath.Abs(in.ContextPath)
if err != nil {
contextPathHash = in.ContextPath
}
// TODO: this should not be loaded this side of the controller api
b, err := builder.New(dockerCli,
builder.WithName(in.Builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return nil, nil, err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return nil, nil, err
}
resp, res, err := buildTargets(ctx, dockerCli, b.NodeGroup, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
err = wrapBuildError(err, false)
if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, err
}
return resp, res, nil
}
// buildTargets runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func buildTargets(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, nodes []builder.Node, opts map[string]build.Options, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
var res *build.ResultHandle
var resp map[string]*client.SolveResponse
var err error
if generateResult {
var mu sync.Mutex
var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock()
defer mu.Unlock()
if res == nil || driverIndex < idx {
idx, res = driverIndex, gotRes
}
})
} else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
}
if err != nil {
return nil, res, err
}
return resp[defaultTargetName], res, err
}
func wrapBuildError(err error, bake bool) error {
if err == nil {
return nil
}
st, ok := grpcerrors.AsGRPCStatus(err)
if ok {
if st.Code() == codes.Unimplemented && strings.Contains(st.Message(), "unsupported frontend capability moby.buildkit.frontend.contexts") {
msg := "current frontend does not support --build-context."
if bake {
msg = "current frontend does not support defining additional contexts for targets."
}
msg += " Named contexts are supported since Dockerfile v1.4. Use #syntax directive in Dockerfile or update to latest BuildKit."
return &wrapped{err, msg}
}
}
return err
}
type wrapped struct {
err error
msg string
}
func (w *wrapped) Error() string {
return w.msg
}
func (w *wrapped) Unwrap() error {
return w.err
}
func updateLastActivity(dockerCli command.Cli, ng *store.NodeGroup) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
return txn.UpdateLastActivity(ng)
}
func controllerUlimitOpt2DockerUlimit(u *controllerapi.UlimitOpt) *dockeropts.UlimitOpt {
if u == nil {
return nil
}
values := make(map[string]*units.Ulimit)
for k, v := range u.Values {
values[k] = &units.Ulimit{
Name: v.Name,
Hard: v.Hard,
Soft: v.Soft,
}
}
return dockeropts.NewUlimitOpt(&values)
}

View File

@@ -0,0 +1,32 @@
package control
import (
"context"
"io"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
)
type BuildxController interface {
Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process.
// If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error
Close() error
List(ctx context.Context) (refs []string, _ error)
Disconnect(ctx context.Context, ref string) error
ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error)
DisconnectProcess(ctx context.Context, ref, pid string) error
Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error)
}
type ControlOptions struct {
ServerConfig string
Root string
Detach bool
}

36
controller/controller.go Normal file
View File

@@ -0,0 +1,36 @@
package controller
import (
"context"
"fmt"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/controller/local"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
)
func NewController(ctx context.Context, opts control.ControlOptions, dockerCli command.Cli, pw progress.Writer) (control.BuildxController, error) {
var name string
if opts.Detach {
name = "remote"
} else {
name = "local"
}
var c control.BuildxController
err := progress.Wrap(fmt.Sprintf("[internal] connecting to %s controller", name), pw.Write, func(l progress.SubLogger) (err error) {
if opts.Detach {
c, err = remote.NewRemoteBuildxController(ctx, dockerCli, opts, l)
} else {
c = local.NewLocalBuildxController(ctx, dockerCli, l)
}
return err
})
if err != nil {
return nil, errors.Wrap(err, "failed to start buildx controller")
}
return c, nil
}

View File

@@ -0,0 +1,34 @@
package errdefs
import (
"github.com/containerd/typeurl/v2"
"github.com/moby/buildkit/util/grpcerrors"
)
func init() {
typeurl.Register((*Build)(nil), "github.com/docker/buildx", "errdefs.Build+json")
}
type BuildError struct {
Build
error
}
func (e *BuildError) Unwrap() error {
return e.error
}
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return &e.Build
}
func WrapBuild(err error, ref string) error {
if err == nil {
return nil
}
return &BuildError{Build: Build{Ref: ref}, error: err}
}
func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: *b}
}

View File

@@ -0,0 +1,77 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: errdefs.proto
package errdefs
import (
fmt "fmt"
proto "github.com/gogo/protobuf/proto"
_ "github.com/moby/buildkit/solver/pb"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct {
Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Build) Reset() { *m = Build{} }
func (m *Build) String() string { return proto.CompactTextString(m) }
func (*Build) ProtoMessage() {}
func (*Build) Descriptor() ([]byte, []int) {
return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
}
var xxx_messageInfo_Build proto.InternalMessageInfo
func (m *Build) GetRef() string {
if m != nil {
return m.Ref
}
return ""
}
func init() {
proto.RegisterType((*Build)(nil), "errdefs.Build")
}
func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var fileDescriptor_689dc58a5060aff5 = []byte{
// 111 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
}

View File

@@ -0,0 +1,9 @@
syntax = "proto3";
package errdefs;
import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build {
string Ref = 1;
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -0,0 +1,146 @@
package local
import (
"context"
"io"
"sync/atomic"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{
dockerCli: dockerCli,
ref: "local",
processes: processes.NewManager(),
}
}
type buildConfig struct {
// TODO: these two structs should be merged
// Discussion: https://github.com/docker/buildx/pull/1640#discussion_r1113279719
resultCtx *build.ResultHandle
buildOptions *controllerapi.BuildOptions
}
type localController struct {
dockerCli command.Cli
ref string
buildConfig buildConfig
processes *processes.Manager
buildOnGoing atomic.Bool
}
func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, errors.New("build ongoing")
}
defer b.buildOnGoing.Store(false)
resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil {
b.buildConfig = buildConfig{
resultCtx: res,
buildOptions: &options,
}
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, b.ref)
}
}
if buildErr != nil {
return "", nil, buildErr
}
return b.ref, resp, nil
}
func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return b.processes.ListProcesses(), nil
}
func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
return b.processes.DeleteProcess(pid)
}
func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses()
}
func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
proc, ok := b.processes.Get(pid)
if !ok {
// Start a new process.
if b.buildConfig.resultCtx == nil {
return errors.New("no build result is registered")
}
var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select {
case <-ioCancelledCh:
return errors.Errorf("io cancelled")
case err := <-proc.Done():
return err
case <-ctx.Done():
return ctx.Err()
}
}
func (b *localController) Kill(context.Context) error {
b.Close()
return nil
}
func (b *localController) Close() error {
b.cancelRunningProcesses()
if b.buildConfig.resultCtx != nil {
b.buildConfig.resultCtx.Done()
}
// TODO: cancel ongoing builds?
return nil
}
func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.ref}, nil
}
func (b *localController) Disconnect(ctx context.Context, key string) error {
b.Close()
return nil
}
func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
}

20
controller/pb/attest.go Normal file
View File

@@ -0,0 +1,20 @@
package pb
func CreateAttestations(attests []*Attest) map[string]*string {
result := map[string]*string{}
for _, attest := range attests {
// ignore duplicates
if _, ok := result[attest.Type]; ok {
continue
}
if attest.Disabled {
result[attest.Type] = nil
continue
}
attrs := attest.Attrs
result[attest.Type] = &attrs
}
return result
}

21
controller/pb/cache.go Normal file
View File

@@ -0,0 +1,21 @@
package pb
import "github.com/moby/buildkit/client"
func CreateCaches(entries []*CacheOptionsEntry) []client.CacheOptionsEntry {
var outs []client.CacheOptionsEntry
if len(entries) == 0 {
return nil
}
for _, entry := range entries {
out := client.CacheOptionsEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
outs = append(outs, out)
}
return outs
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,244 @@
syntax = "proto3";
package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "pb";
service Controller {
rpc Build(BuildRequest) returns (BuildResponse);
rpc Inspect(InspectRequest) returns (InspectResponse);
rpc Status(StatusRequest) returns (stream StatusResponse);
rpc Input(stream InputMessage) returns (InputResponse);
rpc Invoke(stream Message) returns (stream Message);
rpc List(ListRequest) returns (ListResponse);
rpc Disconnect(DisconnectRequest) returns (DisconnectResponse);
rpc Info(InfoRequest) returns (InfoResponse);
rpc ListProcesses(ListProcessesRequest) returns (ListProcessesResponse);
rpc DisconnectProcess(DisconnectProcessRequest) returns (DisconnectProcessResponse);
}
message ListProcessesRequest {
string Ref = 1;
}
message ListProcessesResponse {
repeated ProcessInfo Infos = 1;
}
message ProcessInfo {
string ProcessID = 1;
InvokeConfig InvokeConfig = 2;
}
message DisconnectProcessRequest {
string Ref = 1;
string ProcessID = 2;
}
message DisconnectProcessResponse {
}
message BuildRequest {
string Ref = 1;
BuildOptions Options = 2;
}
message BuildOptions {
string ContextPath = 1;
string DockerfileName = 2;
PrintFunc PrintFunc = 3;
map<string, string> NamedContexts = 4;
repeated string Allow = 5;
repeated Attest Attests = 6;
map<string, string> BuildArgs = 7;
repeated CacheOptionsEntry CacheFrom = 8;
repeated CacheOptionsEntry CacheTo = 9;
string CgroupParent = 10;
repeated ExportEntry Exports = 11;
repeated string ExtraHosts = 12;
map<string, string> Labels = 13;
string NetworkMode = 14;
repeated string NoCacheFilter = 15;
repeated string Platforms = 16;
repeated Secret Secrets = 17;
int64 ShmSize = 18;
repeated SSH SSH = 19;
repeated string Tags = 20;
string Target = 21;
UlimitOpt Ulimits = 22;
string Builder = 23;
bool NoCache = 24;
bool Pull = 25;
bool ExportPush = 26;
bool ExportLoad = 27;
moby.buildkit.v1.sourcepolicy.Policy SourcePolicy = 28;
}
message ExportEntry {
string Type = 1;
map<string, string> Attrs = 2;
string Destination = 3;
}
message CacheOptionsEntry {
string Type = 1;
map<string, string> Attrs = 2;
}
message Attest {
string Type = 1;
bool Disabled = 2;
string Attrs = 3;
}
message SSH {
string ID = 1;
repeated string Paths = 2;
}
message Secret {
string ID = 1;
string FilePath = 2;
string Env = 3;
}
message PrintFunc {
string Name = 1;
string Format = 2;
}
message InspectRequest {
string Ref = 1;
}
message InspectResponse {
BuildOptions Options = 1;
}
message UlimitOpt {
map<string, Ulimit> values = 1;
}
message Ulimit {
string Name = 1;
int64 Hard = 2;
int64 Soft = 3;
}
message BuildResponse {
map<string, string> ExporterResponse = 1;
}
message DisconnectRequest {
string Ref = 1;
}
message DisconnectResponse {}
message ListRequest {
string Ref = 1;
}
message ListResponse {
repeated string keys = 1;
}
message InputMessage {
oneof Input {
InputInitMessage Init = 1;
DataMessage Data = 2;
}
}
message InputInitMessage {
string Ref = 1;
}
message DataMessage {
bool EOF = 1; // true if eof was reached
bytes Data = 2; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message InputResponse {}
message Message {
oneof Input {
InitMessage Init = 1;
// FdMessage used from client to server for input (stdin) and
// from server to client for output (stdout, stderr)
FdMessage File = 2;
// ResizeMessage used from client to server for terminal resize events
ResizeMessage Resize = 3;
// SignalMessage is used from client to server to send signal events
SignalMessage Signal = 4;
}
}
message InitMessage {
string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored.
string ProcessID = 2;
InvokeConfig InvokeConfig = 3;
}
message InvokeConfig {
repeated string Entrypoint = 1;
repeated string Cmd = 2;
repeated string Env = 3;
string User = 4;
bool NoUser = 5; // Do not set user but use the image's default
string Cwd = 6;
bool NoCwd = 7; // Do not set cwd but use the image's default
bool Tty = 8;
bool Rollback = 9; // Kill all process in the container and recreate it.
bool Initial = 10; // Run container from the initial state of that stage (supported only on the failed step)
}
message FdMessage {
uint32 Fd = 1; // what fd the data was from
bool EOF = 2; // true if eof was reached
bytes Data = 3; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message ResizeMessage {
uint32 Rows = 1;
uint32 Cols = 2;
}
message SignalMessage {
// we only send name (ie HUP, INT) because the int values
// are platform dependent.
string Name = 1;
}
message StatusRequest {
string Ref = 1;
}
message StatusResponse {
repeated moby.buildkit.v1.Vertex vertexes = 1;
repeated moby.buildkit.v1.VertexStatus statuses = 2;
repeated moby.buildkit.v1.VertexLog logs = 3;
repeated moby.buildkit.v1.VertexWarning warnings = 4;
}
message InfoRequest {}
message InfoResponse {
BuildxVersion buildxVersion = 1;
}
message BuildxVersion {
string package = 1;
string version = 2;
string revision = 3;
}

View File

@@ -1,66 +1,31 @@
package buildflags
package pb
import (
"encoding/csv"
"io"
"os"
"strconv"
"strings"
"github.com/containerd/console"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func ParseOutputs(inp []string) ([]client.ExportEntry, error) {
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
if len(inp) == 0 {
if len(entries) == 0 {
return nil, nil
}
for _, s := range inp {
csvReader := csv.NewReader(strings.NewReader(s))
fields, err := csvReader.Read()
if err != nil {
return nil, err
for _, entry := range entries {
if entry.Type == "" {
return nil, errors.Errorf("type is required for output")
}
out := client.ExportEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
if len(fields) == 1 && fields[0] == s && !strings.HasPrefix(s, "type=") {
if s != "-" {
outs = append(outs, client.ExportEntry{
Type: client.ExporterLocal,
OutputDir: s,
})
continue
}
out = client.ExportEntry{
Type: client.ExporterTar,
Attrs: map[string]string{
"dest": s,
},
}
}
if out.Type == "" {
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) != 2 {
return nil, errors.Errorf("invalid value %s", field)
}
key := strings.TrimSpace(strings.ToLower(parts[0]))
value := parts[1]
switch key {
case "type":
out.Type = value
default:
out.Attrs[key] = value
}
}
}
if out.Type == "" {
return nil, errors.Errorf("type is required for output")
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
supportFile := false
@@ -79,56 +44,49 @@ func ParseOutputs(inp []string) ([]client.ExportEntry, error) {
supportDir = !tar
case "registry":
out.Type = client.ExporterImage
if _, ok := out.Attrs["push"]; !ok {
out.Attrs["push"] = "true"
}
}
dest, ok := out.Attrs["dest"]
if supportDir {
if !ok {
if entry.Destination == "" {
return nil, errors.Errorf("dest is required for %s exporter", out.Type)
}
if dest == "-" {
if entry.Destination == "-" {
return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
}
fi, err := os.Stat(dest)
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination directory: %s", dest)
return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
}
if err == nil && !fi.IsDir() {
return nil, errors.Errorf("destination directory %s is a file", dest)
return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
}
out.OutputDir = dest
out.OutputDir = entry.Destination
}
if supportFile {
if !ok && out.Type != client.ExporterDocker {
dest = "-"
if entry.Destination == "" && out.Type != client.ExporterDocker {
entry.Destination = "-"
}
if dest == "-" {
if entry.Destination == "-" {
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
} else if dest != "" {
fi, err := os.Stat(dest)
} else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination file: %s", dest)
return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
}
if err == nil && fi.IsDir() {
return nil, errors.Errorf("destination file %s is a directory", dest)
return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
}
f, err := os.Create(dest)
f, err := os.Create(entry.Destination)
if err != nil {
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
}
}
if supportFile || supportDir {
delete(out.Attrs, "dest")
}
outs = append(outs, out)
}

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

175
controller/pb/path.go Normal file
View File

@@ -0,0 +1,175 @@
package pb
import (
"path/filepath"
"strings"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
)
// ResolveOptionPaths resolves all paths contained in BuildOptions
// and replaces them to absolute paths.
func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
localContext := false
if options.ContextPath != "" && options.ContextPath != "-" {
if !isRemoteURL(options.ContextPath) {
localContext = true
options.ContextPath, err = filepath.Abs(options.ContextPath)
if err != nil {
return nil, err
}
}
}
if options.DockerfileName != "" && options.DockerfileName != "-" {
if localContext && !urlutil.IsURL(options.DockerfileName) {
options.DockerfileName, err = filepath.Abs(options.DockerfileName)
if err != nil {
return nil, err
}
}
}
var contexts map[string]string
for k, v := range options.NamedContexts {
if isRemoteURL(v) || strings.HasPrefix(v, "docker-image://") {
// url prefix, this is a remote path
} else if strings.HasPrefix(v, "oci-layout://") {
// oci layout prefix, this is a local path
p := strings.TrimPrefix(v, "oci-layout://")
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
v = "oci-layout://" + p
} else {
// no prefix, assume local path
v, err = filepath.Abs(v)
if err != nil {
return nil, err
}
}
if contexts == nil {
contexts = make(map[string]string)
}
contexts[k] = v
}
options.NamedContexts = contexts
var cacheFrom []*CacheOptionsEntry
for _, co := range options.CacheFrom {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "src":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheFrom = append(cacheFrom, co)
default:
cacheFrom = append(cacheFrom, co)
}
}
options.CacheFrom = cacheFrom
var cacheTo []*CacheOptionsEntry
for _, co := range options.CacheTo {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "dest":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheTo = append(cacheTo, co)
default:
cacheTo = append(cacheTo, co)
}
}
options.CacheTo = cacheTo
var exports []*ExportEntry
for _, e := range options.Exports {
if e.Destination != "" && e.Destination != "-" {
e.Destination, err = filepath.Abs(e.Destination)
if err != nil {
return nil, err
}
}
exports = append(exports, e)
}
options.Exports = exports
var secrets []*Secret
for _, s := range options.Secrets {
if s.FilePath != "" {
s.FilePath, err = filepath.Abs(s.FilePath)
if err != nil {
return nil, err
}
}
secrets = append(secrets, s)
}
options.Secrets = secrets
var ssh []*SSH
for _, s := range options.SSH {
var ps []string
for _, pt := range s.Paths {
p := pt
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
ps = append(ps, p)
}
s.Paths = ps
ssh = append(ssh, s)
}
options.SSH = ssh
return options, nil
}
func isRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}

247
controller/pb/path_test.go Normal file
View File

@@ -0,0 +1,247 @@
package pb
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/stretchr/testify/require"
)
func TestResolvePaths(t *testing.T) {
tmpwd, err := os.MkdirTemp("", "testresolvepaths")
require.NoError(t, err)
defer os.Remove(tmpwd)
require.NoError(t, os.Chdir(tmpwd))
tests := []struct {
name string
options BuildOptions
want BuildOptions
}{
{
name: "contextpath",
options: BuildOptions{ContextPath: "test"},
want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
},
{
name: "contextpath-cwd",
options: BuildOptions{ContextPath: "."},
want: BuildOptions{ContextPath: tmpwd},
},
{
name: "contextpath-dash",
options: BuildOptions{ContextPath: "-"},
want: BuildOptions{ContextPath: "-"},
},
{
name: "contextpath-ssh",
options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "dockerfilename",
options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
},
{
name: "dockerfilename-dash",
options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
},
{
name: "dockerfilename-remote",
options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "contexts",
options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
},
{
name: "cache-from",
options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "cache-to",
options: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "exports",
options: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: "test1",
},
{
Type: "tar",
Destination: "test3",
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: "test4",
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
want: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: filepath.Join(tmpwd, "test1"),
},
{
Type: "tar",
Destination: filepath.Join(tmpwd, "test3"),
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: filepath.Join(tmpwd, "test4"),
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
},
{
name: "secrets",
options: BuildOptions{
Secrets: []*Secret{
{
FilePath: "test1",
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: "test3",
},
},
},
want: BuildOptions{
Secrets: []*Secret{
{
FilePath: filepath.Join(tmpwd, "test1"),
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: filepath.Join(tmpwd, "test3"),
},
},
},
},
{
name: "ssh",
options: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{"test1", "test2"},
},
{
ID: "a",
Paths: []string{"test3"},
},
},
},
want: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{filepath.Join(tmpwd, "test1"), filepath.Join(tmpwd, "test2")},
},
{
ID: "a",
Paths: []string{filepath.Join(tmpwd, "test3")},
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err)
if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, *got)
}
})
}
}

126
controller/pb/progress.go Normal file
View File

@@ -0,0 +1,126 @@
package pb
import (
"github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest"
)
type writer struct {
ch chan<- *StatusResponse
}
func NewProgressWriter(ch chan<- *StatusResponse) progress.Writer {
return &writer{ch: ch}
}
func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status)
}
func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true
}
func (w *writer) ClearLogSource(interface{}) {}
func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{}
for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: v.Vertex,
Stream: int64(v.Stream),
Msg: v.Data,
Timestamp: v.Timestamp,
})
}
for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: v.Vertex,
Level: int64(v.Level),
Short: v.Short,
Detail: v.Detail,
Url: v.URL,
Info: v.SourceInfo,
Ranges: v.Range,
})
}
return &resp
}
func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{}
for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{
Vertex: v.Vertex,
Stream: int(v.Stream),
Data: v.Msg,
Timestamp: v.Timestamp,
})
}
for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: v.Vertex,
Level: int(v.Level),
Short: v.Short,
Detail: v.Detail,
URL: v.Url,
SourceInfo: v.Info,
Range: v.Ranges,
})
}
return &s
}

22
controller/pb/secrets.go Normal file
View File

@@ -0,0 +1,22 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/secrets/secretsprovider"
)
func CreateSecrets(secrets []*Secret) (session.Attachable, error) {
fs := make([]secretsprovider.Source, 0, len(secrets))
for _, secret := range secrets {
fs = append(fs, secretsprovider.Source{
ID: secret.ID,
FilePath: secret.FilePath,
Env: secret.Env,
})
}
store, err := secretsprovider.NewStore(fs)
if err != nil {
return nil, err
}
return secretsprovider.NewSecretProvider(store), nil
}

18
controller/pb/ssh.go Normal file
View File

@@ -0,0 +1,18 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider"
)
func CreateSSH(ssh []*SSH) (session.Attachable, error) {
configs := make([]sshprovider.AgentConfig, 0, len(ssh))
for _, ssh := range ssh {
cfg := sshprovider.AgentConfig{
ID: ssh.ID,
Paths: append([]string{}, ssh.Paths...),
}
configs = append(configs, cfg)
}
return sshprovider.NewSSHAgentProvider(configs)
}

View File

@@ -0,0 +1,149 @@
package processes
import (
"context"
"sync"
"sync/atomic"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/ioset"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// Process provides methods to control a process.
type Process struct {
inEnd *ioset.Forwarder
invokeConfig *pb.InvokeConfig
errCh chan error
processCancel func()
serveIOCancel func()
}
// ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil {
f()
}
p.serveIOCancel = ioCancelCallback
}
// Done returns a channel where error or nil will be sent
// when the process exits.
// TODO: change this to Wait()
func (p *Process) Done() <-chan error {
return p.errCh
}
// Manager manages a set of proceses.
type Manager struct {
container atomic.Value
processes sync.Map
}
// NewManager creates and returns a Manager.
func NewManager() *Manager {
return &Manager{}
}
// Get returns the specified process.
func (m *Manager) Get(id string) (*Process, bool) {
v, ok := m.processes.Load(id)
if !ok {
return nil, false
}
return v.(*Process), true
}
// CancelRunningProcesses cancels execution of all running processes.
func (m *Manager) CancelRunningProcesses() {
var funcs []func()
m.processes.Range(func(key, value any) bool {
funcs = append(funcs, value.(*Process).processCancel)
m.processes.Delete(key)
return true
})
for _, f := range funcs {
f()
}
}
// ListProcesses lists all running processes.
func (m *Manager) ListProcesses() (res []*pb.ProcessInfo) {
m.processes.Range(func(key, value any) bool {
res = append(res, &pb.ProcessInfo{
ProcessID: key.(string),
InvokeConfig: value.(*Process).invokeConfig,
})
return true
})
return res
}
// DeleteProcess deletes the specified process.
func (m *Manager) DeleteProcess(id string) error {
p, ok := m.processes.LoadAndDelete(id)
if !ok {
return errors.Errorf("unknown process %q", id)
}
p.(*Process).processCancel()
return nil
}
// StartProcess starts a process in the container.
// When a container isn't available (i.e. first time invoking or the container has exited) or cfg.Rollback is set,
// this method will start a new container and run the process in it. Otherwise, this method starts a new process in the
// existing container.
func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *pb.InvokeConfig) (*Process, error) {
// Get the target result to invoke a container from
var ctr *build.Container
if a := m.container.Load(); a != nil {
ctr = a.(*build.Container)
}
if cfg.Rollback || ctr == nil || ctr.IsUnavailable() {
go m.CancelRunningProcesses()
// (Re)create a new container if this is rollback or first time to invoke a process.
if ctr != nil {
go ctr.Cancel() // Finish the existing container
}
var err error
ctr, err = build.NewContainer(context.TODO(), resultCtx, cfg)
if err != nil {
return nil, errors.Errorf("failed to create container %v", err)
}
m.container.Store(ctr)
}
// [client(ForwardIO)] <-forwarder(switchable)-> [out] <-pipe-> [in] <- [process]
in, out := ioset.Pipe()
f := ioset.NewForwarder()
f.PropagateStdinClose = false
f.SetOut(&out)
// Register process
ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once
processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
p := &Process{
inEnd: f,
invokeConfig: cfg,
processCancel: processCancelFunc,
errCh: make(chan error),
}
m.processes.Store(pid, p)
go func() {
var err error
if err = ctr.Exec(ctx, cfg, in.Stdin, in.Stdout, in.Stderr); err != nil {
logrus.Errorf("failed to exec process: %v", err)
}
logrus.Debugf("finished process %s %v", pid, cfg.Entrypoint)
m.processes.Delete(pid)
processCancelFunc()
p.errCh <- err
}()
return p, nil
}

240
controller/remote/client.go Normal file
View File

@@ -0,0 +1,240 @@
package remote
import (
"context"
"io"
"sync"
"time"
"github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure"
)
func NewClient(ctx context.Context, addr string) (*Client, error) {
backoffConfig := backoff.DefaultConfig
backoffConfig.MaxDelay = 3 * time.Second
connParams := grpc.ConnectParams{
Backoff: backoffConfig,
}
gopts := []grpc.DialOption{
grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams),
grpc.WithContextDialer(dialer.ContextDialer),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(defaults.DefaultMaxRecvMsgSize)),
grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(defaults.DefaultMaxSendMsgSize)),
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
}
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil {
return nil, err
}
return &Client{conn: conn}, nil
}
type Client struct {
conn *grpc.ClientConn
closeOnce sync.Once
}
func (c *Client) Close() (err error) {
c.closeOnce.Do(func() {
err = c.conn.Close()
})
return
}
func (c *Client) Version(ctx context.Context) (string, string, string, error) {
res, err := c.client().Info(ctx, &pb.InfoRequest{})
if err != nil {
return "", "", "", err
}
v := res.BuildxVersion
return v.Package, v.Version, v.Revision, nil
}
func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
res, err := c.client().List(ctx, &pb.ListRequest{})
if err != nil {
return nil, err
}
return res.Keys, nil
}
func (c *Client) Disconnect(ctx context.Context, key string) error {
if key == "" {
return nil
}
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err
}
func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil {
return nil, err
}
return res.Infos, nil
}
func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err
}
func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if ref == "" || pid == "" {
return errors.New("build reference must be specified")
}
stream, err := c.client().Invoke(ctx)
if err != nil {
return err
}
return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in,
stdout: stdout,
stderr: stderr,
// TODO: Signal, Resize
})
}
func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
}
func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID()
statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx)
var resp *client.SolveResponse
eg.Go(func() error {
defer close(statusChan)
var err error
resp, err = c.build(egCtx, ref, options, in, statusChan)
return err
})
eg.Go(func() error {
for s := range statusChan {
st := s
progress.Write(st)
}
return nil
})
return ref, resp, eg.Wait()
}
func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{})
var resp *client.SolveResponse
eg.Go(func() error {
defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
Ref: ref,
Options: &options,
})
if err != nil {
return err
}
resp = &client.SolveResponse{
ExporterResponse: pbResp.ExporterResponse,
}
return nil
})
eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{
Ref: ref,
})
if err != nil {
return err
}
for {
resp, err := stream.Recv()
if err != nil {
if err == io.EOF {
return nil
}
return errors.Wrap(err, "failed to receive status")
}
statusChan <- pb.FromControlStatus(resp)
}
})
if in != nil {
eg.Go(func() error {
stream, err := c.client().Input(egCtx)
if err != nil {
return err
}
if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{
Ref: ref,
},
},
}); err != nil {
return errors.Wrap(err, "failed to init input")
}
inReader, inWriter := io.Pipe()
eg2, _ := errgroup.WithContext(ctx)
eg2.Go(func() error {
<-done
return inWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(inWriter, in)
inWriter.Close()
}()
eg2.Go(func() error {
for {
buf := make([]byte, 32*1024)
n, err := inReader.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
EOF: true,
},
},
})
})
return eg2.Wait()
})
}
return resp, eg.Wait()
}
func (c *Client) client() pb.ControllerClient {
return pb.NewControllerClient(c.conn)
}

View File

@@ -0,0 +1,333 @@
//go:build linux
package remote
import (
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"syscall"
"time"
"github.com/containerd/containerd/log"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pelletier/go-toml"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc"
)
const (
serveCommandName = "_INTERNAL_SERVE"
)
var (
defaultLogFilename = fmt.Sprintf("buildx.%s.log", version.Revision)
defaultSocketFilename = fmt.Sprintf("buildx.%s.sock", version.Revision)
defaultPIDFilename = fmt.Sprintf("buildx.%s.pid", version.Revision)
)
type serverConfig struct {
// Specify buildx server root
Root string `toml:"root"`
// LogLevel sets the logging level [trace, debug, info, warn, error, fatal, panic]
LogLevel string `toml:"log_level"`
// Specify file to output buildx server log
LogFile string `toml:"log_file"`
}
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
rootDir := opts.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running
ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server")
}
} else {
return &buildxController{c, serverRoot}, nil
}
// start buildx server via subcommand
err = logger.Wrap("no buildx server found; launching...", func() error {
launchFlags := []string{}
if opts.ServerConfig != "" {
launchFlags = append(launchFlags, "--config", opts.ServerConfig)
}
logFile, err := getLogFilePath(dockerCli, opts.ServerConfig)
if err != nil {
return err
}
wait, err := launch(ctx, logFile, append([]string{serveCommandName}, launchFlags...)...)
if err != nil {
return err
}
go wait()
// wait for buildx server to be ready
ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server")
}
return nil
})
if err != nil {
return nil, err
}
return &buildxController{c, serverRoot}, nil
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
serveCmd(dockerCli),
)
}
func serveCmd(dockerCli command.Cli) *cobra.Command {
var serverConfigPath string
cmd := &cobra.Command{
Use: fmt.Sprintf("%s [OPTIONS]", serveCommandName),
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Parse config
config, err := getConfig(dockerCli, serverConfigPath)
if err != nil {
return err
}
if config.LogLevel == "" {
logrus.SetLevel(logrus.InfoLevel)
} else {
lvl, err := logrus.ParseLevel(config.LogLevel)
if err != nil {
return errors.Wrap(err, "failed to prepare logger")
}
logrus.SetLevel(lvl)
}
logrus.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: log.RFC3339NanoFixed,
})
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return err
}
pidF := filepath.Join(root, defaultPIDFilename)
if err := os.WriteFile(pidF, []byte(fmt.Sprintf("%d", os.Getpid())), 0600); err != nil {
return err
}
defer func() {
if err := os.Remove(pidF); err != nil {
logrus.Errorf("failed to clean up info file %q: %v", pidF, err)
}
}()
// prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
})
defer b.Close()
// serve server
addr := filepath.Join(root, defaultSocketFilename)
if err := os.Remove(addr); err != nil && !os.IsNotExist(err) { // avoid EADDRINUSE
return err
}
defer func() {
if err := os.Remove(addr); err != nil {
logrus.Errorf("failed to clean up socket %q: %v", addr, err)
}
}()
logrus.Infof("starting server at %q", addr)
l, err := net.Listen("unix", addr)
if err != nil {
return err
}
rpc := grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
)
controllerapi.RegisterControllerServer(rpc, b)
doneCh := make(chan struct{})
errCh := make(chan error, 1)
go func() {
defer close(doneCh)
if err := rpc.Serve(l); err != nil {
errCh <- errors.Wrapf(err, "error on serving via socket %q", addr)
}
}()
var s os.Signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT)
signal.Notify(sigCh, syscall.SIGTERM)
select {
case err := <-errCh:
logrus.Errorf("got error %s, exiting", err)
return err
case s = <-sigCh:
logrus.Infof("got signal %s, exiting", s)
return nil
case <-doneCh:
logrus.Infof("rpc server done, exiting")
return nil
}
},
}
flags := cmd.Flags()
flags.StringVar(&serverConfigPath, "config", "", "Specify buildx server config file")
return cmd
}
func getLogFilePath(dockerCli command.Cli, configPath string) (string, error) {
config, err := getConfig(dockerCli, configPath)
if err != nil {
return "", err
}
if config.LogFile == "" {
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return "", err
}
return filepath.Join(root, defaultLogFilename), nil
}
return config.LogFile, nil
}
func getConfig(dockerCli command.Cli, configPath string) (*serverConfig, error) {
var defaultConfigPath bool
if configPath == "" {
defaultRoot := rootDataDir(dockerCli)
configPath = filepath.Join(defaultRoot, "config.toml")
defaultConfigPath = true
}
var config serverConfig
tree, err := toml.LoadFile(configPath)
if err != nil && !(os.IsNotExist(err) && defaultConfigPath) {
return nil, errors.Wrapf(err, "failed to read config %q", configPath)
} else if err == nil {
if err := tree.Unmarshal(&config); err != nil {
return nil, errors.Wrapf(err, "failed to unmarshal config %q", configPath)
}
}
return &config, nil
}
func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error) {
rootDir := config.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
if rootDir == "" {
return "", errors.New("buildx root dir must be determined")
}
if err := os.MkdirAll(rootDir, 0700); err != nil {
return "", err
}
serverRoot := filepath.Join(rootDir, "shared")
if err := os.MkdirAll(serverRoot, 0700); err != nil {
return "", err
}
return serverRoot, nil
}
func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
}
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {
c, err := NewClient(ctx, addr)
if err != nil {
return nil, err
}
p, v, r, err := c.Version(ctx)
if err != nil {
return nil, err
}
logrus.Debugf("connected to server (\"%v %v %v\")", p, v, r)
if !(p == version.Package && v == version.Version && r == version.Revision) {
return nil, errors.Errorf("version mismatch (client: \"%v %v %v\", server: \"%v %v %v\")", version.Package, version.Version, version.Revision, p, v, r)
}
return c, nil
}
type buildxController struct {
*Client
serverRoot string
}
func (c *buildxController) Kill(ctx context.Context) error {
pidB, err := os.ReadFile(filepath.Join(c.serverRoot, defaultPIDFilename))
if err != nil {
return err
}
pid, err := strconv.ParseInt(string(pidB), 10, 64)
if err != nil {
return err
}
if pid <= 0 {
return errors.New("no PID is recorded for buildx server")
}
p, err := os.FindProcess(int(pid))
if err != nil {
return err
}
if err := p.Signal(syscall.SIGINT); err != nil {
return err
}
// TODO: Should we send SIGKILL if process doesn't finish?
return nil
}
func launch(ctx context.Context, logFile string, args ...string) (func() error, error) {
// set absolute path of binary, since we set the working directory to the root
pathname, err := os.Executable()
if err != nil {
return nil, err
}
bCmd := exec.CommandContext(ctx, pathname, args...)
if logFile != "" {
f, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, err
}
defer f.Close()
bCmd.Stdout = f
bCmd.Stderr = f
}
bCmd.Stdin = nil
bCmd.Dir = "/"
bCmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
}
if err := bCmd.Start(); err != nil {
return nil, err
}
return bCmd.Wait, nil
}

View File

@@ -0,0 +1,19 @@
//go:build !linux
package remote
import (
"context"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
return nil, errors.New("remote buildx unsupported")
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {}

430
controller/remote/io.go Normal file
View File

@@ -0,0 +1,430 @@
package remote
import (
"context"
"io"
"syscall"
"time"
"github.com/docker/buildx/controller/pb"
"github.com/moby/sys/signal"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
type msgStream interface {
Send(*pb.Message) error
Recv() (*pb.Message, error)
}
type ioServerConfig struct {
stdin io.WriteCloser
stdout, stderr io.ReadCloser
// signalFn is a callback function called when a signal is reached to the client.
signalFn func(context.Context, syscall.Signal) error
// resizeFn is a callback function called when a resize event is reached to the client.
resizeFn func(context.Context, winSize) error
}
func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessage) error, ioConfig *ioServerConfig) (err error) {
stdin, stdout, stderr := ioConfig.stdin, ioConfig.stdout, ioConfig.stderr
stream := &debugStream{srv, "server=" + time.Now().String()}
eg, ctx := errgroup.WithContext(attachCtx)
done := make(chan struct{})
msg, err := receive(ctx, stream)
if err != nil {
return err
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
}
ref := init.Ref
if ref == "" {
return errors.New("no ref is provided")
}
if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server")
}
if stdout != nil {
stdoutReader, stdoutWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdoutWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdoutWriter, stdout)
stdoutWriter.Close()
}()
eg.Go(func() error {
defer stdoutReader.Close()
return copyToStream(1, stream, stdoutReader)
})
}
if stderr != nil {
stderrReader, stderrWriter := io.Pipe()
eg.Go(func() error {
<-done
return stderrWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stderrWriter, stderr)
stderrWriter.Close()
}()
eg.Go(func() error {
defer stderrReader.Close()
return copyToStream(2, stream, stderrReader)
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if file.Fd != 0 {
return errors.Errorf("unexpected fd: %v", file.Fd)
}
if stdin == nil {
continue // no stdin destination is specified so ignore the data
}
if len(file.Data) > 0 {
_, err := stdin.Write(file.Data)
if err != nil {
return err
}
}
if file.EOF {
stdin.Close()
}
} else if resize := msg.GetResize(); resize != nil {
if ioConfig.resizeFn != nil {
ioConfig.resizeFn(ctx, winSize{
cols: resize.Cols,
rows: resize.Rows,
})
}
} else if sig := msg.GetSignal(); sig != nil {
if ioConfig.signalFn != nil {
syscallSignal, ok := signal.SignalMap[sig.Name]
if !ok {
continue
}
ioConfig.signalFn(ctx, syscallSignal)
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
type ioAttachConfig struct {
stdin io.ReadCloser
stdout, stderr io.WriteCloser
signal <-chan syscall.Signal
resize <-chan winSize
}
type winSize struct {
rows uint32
cols uint32
}
func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage, cfg ioAttachConfig) (retErr error) {
eg, ctx := errgroup.WithContext(ctx)
done := make(chan struct{})
if err := stream.Send(&pb.Message{
Input: &pb.Message_Init{
Init: initMessage,
},
}); err != nil {
return errors.Wrap(err, "failed to init")
}
if cfg.stdin != nil {
stdinReader, stdinWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdinWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdinWriter, cfg.stdin)
stdinWriter.Close()
}()
eg.Go(func() error {
defer stdinReader.Close()
return copyToStream(0, stream, stdinReader)
})
}
if cfg.signal != nil {
eg.Go(func() error {
for {
var sig syscall.Signal
select {
case sig = <-cfg.signal:
case <-done:
return nil
case <-ctx.Done():
return nil
}
name := sigToName[sig]
if name == "" {
continue
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Signal{
Signal: &pb.SignalMessage{
Name: name,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send signal")
}
}
})
}
if cfg.resize != nil {
eg.Go(func() error {
for {
var win winSize
select {
case win = <-cfg.resize:
case <-done:
return nil
case <-ctx.Done():
return nil
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Resize{
Resize: &pb.ResizeMessage{
Rows: win.rows,
Cols: win.cols,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send resize")
}
}
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
eofs := make(map[uint32]struct{})
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if _, ok := eofs[file.Fd]; ok {
continue
}
var out io.WriteCloser
switch file.Fd {
case 1:
out = cfg.stdout
case 2:
out = cfg.stderr
default:
return errors.Errorf("unsupported fd %d", file.Fd)
}
if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
continue
}
if len(file.Data) > 0 {
if _, err := out.Write(file.Data); err != nil {
return err
}
}
if file.EOF {
eofs[file.Fd] = struct{}{}
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
msgCh := make(chan *pb.Message)
errCh := make(chan error)
go func() {
msg, err := stream.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
return
}
errCh <- err
return
}
msgCh <- msg
}()
select {
case msg := <-msgCh:
return msg, nil
case err := <-errCh:
return nil, err
case <-ctx.Done():
return nil, ctx.Err()
}
}
func copyToStream(fd uint32, snd msgStream, r io.Reader) error {
for {
buf := make([]byte, 32*1024)
n, err := r.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
EOF: true,
},
},
})
}
var sigToName = map[syscall.Signal]string{}
func init() {
for name, value := range signal.SignalMap {
sigToName[value] = name
}
}
type debugStream struct {
msgStream
prefix string
}
func (s *debugStream) Send(msg *pb.Message) error {
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|---> Resize Message (sender:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|---> Signal Message (sender:%v): %s", s.prefix, m.Signal.Name)
}
return s.msgStream.Send(msg)
}
func (s *debugStream) Recv() (*pb.Message, error) {
msg, err := s.msgStream.Recv()
if err != nil {
return nil, err
}
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|<--- Resize Message (receiver:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|<--- Signal Message (receiver:%v): %s", s.prefix, m.Signal.Name)
}
return msg, nil
}

441
controller/remote/server.go Normal file
View File

@@ -0,0 +1,441 @@
package remote
import (
"context"
"io"
"sync"
"sync/atomic"
"time"
"github.com/docker/buildx/build"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server {
return &Server{
buildFunc: buildFunc,
}
}
type Server struct {
buildFunc BuildFunc
session map[string]*session
sessionMu sync.Mutex
}
type session struct {
buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse
cancelBuild func()
buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter
result *build.ResultHandle
processes *processes.Manager
}
func (s *session) cancelRunningProcesses() {
s.processes.CancelRunningProcesses()
}
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
res = new(pb.ListProcessesResponse)
for _, p := range s.processes.ListProcesses() {
res.Infos = append(res.Infos, p)
}
return res, nil
}
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
return res, s.processes.DeleteProcess(req.ProcessID)
}
func (m *Server) Info(ctx context.Context, req *pb.InfoRequest) (res *pb.InfoResponse, err error) {
return &pb.InfoResponse{
BuildxVersion: &pb.BuildxVersion{
Package: version.Package,
Version: version.Version,
Revision: version.Revision,
},
}, nil
}
func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListResponse, err error) {
keys := make(map[string]struct{})
m.sessionMu.Lock()
for k := range m.session {
keys[k] = struct{}{}
}
m.sessionMu.Unlock()
var keysL []string
for k := range keys {
keysL = append(keysL, k)
}
return &pb.ListResponse{
Keys: keysL,
}, nil
}
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
key := req.Ref
if key == "" {
return nil, errors.New("disconnect: empty key")
}
m.sessionMu.Lock()
if s, ok := m.session[key]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
if s.result != nil {
s.result.Done()
}
}
delete(m.session, key)
m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil
}
func (m *Server) Close() error {
m.sessionMu.Lock()
for k := range m.session {
if s, ok := m.session[k]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
}
}
m.sessionMu.Unlock()
return nil
}
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("inspect: empty key")
}
var bo *pb.BuildOptions
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
bo = s.buildOptions
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", ref)
}
m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil
}
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("build: empty key")
}
// Prepare status channel and session
m.sessionMu.Lock()
if m.session == nil {
m.session = make(map[string]*session)
}
s, ok := m.session[ref]
if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock()
return &pb.BuildResponse{}, errors.New("build ongoing")
}
s.cancelRunningProcesses()
s.result = nil
} else {
s = &session{}
s.buildOnGoing.Store(true)
}
s.processes = processes.NewManager()
statusChan := make(chan *pb.StatusResponse)
s.statusChan = statusChan
inR, inW := io.Pipe()
defer inR.Close()
s.inputPipe = inW
m.session[ref] = s
m.sessionMu.Unlock()
defer func() {
close(statusChan)
m.sessionMu.Lock()
s, ok := m.session[ref]
if ok {
s.statusChan = nil
s.buildOnGoing.Store(false)
}
m.sessionMu.Unlock()
}()
pw := pb.NewProgressWriter(statusChan)
// Build the specified request
ctx, cancel := context.WithCancel(ctx)
defer cancel()
resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil {
s.result = res
s.cancelBuild = cancel
s.buildOptions = req.Options
m.session[ref] = s
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, ref)
}
}
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown key %v", ref)
}
m.sessionMu.Unlock()
if buildErr != nil {
return nil, buildErr
}
if resp == nil {
resp = &client.SolveResponse{}
}
return &pb.BuildResponse{
ExporterResponse: resp.ExporterResponse,
}, nil
}
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
ref := req.Ref
if ref == "" {
return errors.New("status: empty key")
}
// Wait and get status channel prepared by Build()
var statusChan <-chan *pb.StatusResponse
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
statusChan = m.session[ref].statusChan
m.sessionMu.Unlock()
break
}
// forward status
for ss := range statusChan {
if ss == nil {
break
}
if err := stream.Send(ss); err != nil {
return err
}
}
return nil
}
func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
// Get the target ref from init message
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
}
ref := init.Ref
if ref == "" {
return errors.New("input: no ref is provided")
}
// Wait and get input stream pipe prepared by Build()
var inputPipeW *io.PipeWriter
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock()
break
}
// Forward input stream
eg, ctx := errgroup.WithContext(context.TODO())
done := make(chan struct{})
msgCh := make(chan *pb.InputMessage)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() (retErr error) {
defer close(done)
defer func() {
if retErr != nil {
inputPipeW.CloseWithError(retErr)
return
}
inputPipeW.Close()
}()
for {
var msg *pb.InputMessage
select {
case msg = <-msgCh:
case <-ctx.Done():
return errors.Wrap(ctx.Err(), "canceled")
}
if msg == nil {
return nil
}
if data := msg.GetData(); data != nil {
if len(data.Data) > 0 {
_, err := inputPipeW.Write(data.Data)
if err != nil {
return err
}
}
if data.EOF {
return nil
}
}
}
})
return eg.Wait()
}
func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
containerIn, containerOut := ioset.Pipe()
defer func() { containerOut.Close(); containerIn.Close() }()
initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error {
defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() {
if retErr != nil {
initErrCh <- retErr
}
}()
ref := initMessage.Ref
cfg := initMessage.InvokeConfig
m.sessionMu.Lock()
s, ok := m.session[ref]
if !ok {
m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown key %v", ref)
}
m.sessionMu.Unlock()
pid := initMessage.ProcessID
if pid == "" {
return errors.Errorf("invoke: specify process ID")
}
proc, ok := s.processes.Get(pid)
if !ok {
// Start a new process.
if cfg == nil {
return errors.New("no container config is provided")
}
var err error
proc, err = s.processes.StartProcess(pid, s.result, cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
proc.ForwardIO(&containerIn, srvIOCancel)
initDoneCh <- proc
return nil
}, &ioServerConfig{
stdin: containerOut.Stdin,
stdout: containerOut.Stdout,
stderr: containerOut.Stderr,
// TODO: signal, resize
})
})
eg.Go(func() (rErr error) {
defer srvIOCancel()
// Wait for init done
var proc *processes.Process
select {
case p := <-initDoneCh:
proc = p
case err := <-initErrCh:
return err
case <-egCtx.Done():
return egCtx.Err()
}
// Wait for IO done
select {
case <-srvIOCtx.Done():
return srvIOCtx.Err()
case err := <-proc.Done():
return err
case <-egCtx.Done():
return egCtx.Err()
}
})
return eg.Wait()
}

View File

@@ -1,5 +1,5 @@
variable "GO_VERSION" {
default = "1.19"
default = "1.20"
}
variable "DOCS_FORMATS" {
default = "md"
@@ -59,6 +59,13 @@ target "validate-authors" {
output = ["type=cacheonly"]
}
target "validate-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "update-vendor" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -84,6 +91,13 @@ target "update-authors" {
output = ["."]
}
target "update-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "update"
output = ["."]
}
target "mod-outdated" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
@@ -142,3 +156,29 @@ target "image-local" {
inherits = ["image"]
output = ["type=docker"]
}
variable "HTTP_PROXY" {
default = ""
}
variable "HTTPS_PROXY" {
default = ""
}
variable "NO_PROXY" {
default = ""
}
target "integration-test-base" {
inherits = ["_common"]
args = {
HTTP_PROXY = HTTP_PROXY
HTTPS_PROXY = HTTPS_PROXY
NO_PROXY = NO_PROXY
}
target = "integration-test-base"
output = ["type=cacheonly"]
}
target "integration-test" {
inherits = ["integration-test-base"]
target = "integration-test"
}

952
docs/bake-reference.md Normal file
View File

@@ -0,0 +1,952 @@
# Bake file reference
The Bake file is a file for defining workflows that you run using `docker buildx bake`.
## File format
You can define your Bake file in the following file formats:
- HashiCorp Configuration Language (HCL)
- JSON
- YAML (Compose file)
By default, Bake uses the following lookup order to find the configuration file:
1. `docker-bake.override.hcl`
2. `docker-bake.hcl`
3. `docker-bake.override.json`
4. `docker-bake.json`
5. `docker-compose.yaml`
6. `docker-compose.yml`
Bake searches for the file in the current working directory.
You can specify the file location explicitly using the `--file` flag:
```console
$ docker buildx bake --file=../docker/bake.hcl --print
```
## Syntax
The Bake file supports the following property types:
- `target`: build targets
- `group`: collections of build targets
- `variable`: build arguments and variables
- `function`: custom Bake functions
You define properties as hierarchical blocks in the Bake file.
You can assign one or more attributes to a property.
The following snippet shows a JSON representation of a simple Bake file.
This Bake file defines three properties: a variable, a group, and a target.
```json
{
"variable": {
"TAG": {
"default": "latest"
}
},
"group": {
"default": {
"targets": ["webapp"]
}
},
"target": {
"webapp": {
"dockerfile": "Dockerfile",
"tags": ["docker.io/username/webapp:${TAG}"]
}
}
}
```
In the JSON representation of a Bake file, properties are objects,
and attributes are values assigned to those objects.
The following example shows the same Bake file in the HCL format:
```hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
dockerfile = "Dockerfile"
tags = ["docker.io/username/webapp:${TAG}"]
}
```
HCL is the preferred format for Bake files.
Aside from syntactic differences,
HCL lets you use features that the JSON and YAML formats don't support.
The examples in this document use the HCL format.
## Target
A target reflects a single `docker build` invocation.
Consider the following build command:
```console
$ docker build \
--file=Dockerfile.webapp \
--tag=docker.io/username/webapp:latest \
https://github.com/username/webapp
```
You can express this command in a Bake file as follows:
```hcl
target "webapp" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
context = "https://github.com/username/webapp"
}
```
The following table shows the complete list of attributes that you can assign to a target:
| Name | Type | Description |
| ----------------------------------------------- | ------- | -------------------------------------------------------------------- |
| [`args`](#targetargs) | Map | Build arguments |
| [`attest`](#targetattest) | List | Build attestations |
| [`cache-from`](#targetcache-from) | List | External cache sources |
| [`cache-to`](#targetcache-to) | List | External cache destinations |
| [`context`](#targetcontext) | String | Set of files located in the specified path or URL |
| [`contexts`](#targetcontexts) | Map | Additional build contexts |
| [`dockerfile-inline`](#targetdockerfile-inline) | String | Inline Dockerfile string |
| [`dockerfile`](#targetdockerfile) | String | Dockerfile location |
| [`inherits`](#targetinherits) | List | Inherit attributes from other targets |
| [`labels`](#targetlabels) | Map | Metadata for images |
| [`matrix`](#targetmatrix) | Map | Define a set of variables that forks a target into multiple targets. |
| [`name`](#targetname) | String | Override the target name when using a matrix. |
| [`no-cache-filter`](#targetno-cache-filter) | List | Disable build cache for specific stages |
| [`no-cache`](#targetno-cache) | Boolean | Disable build cache completely |
| [`output`](#targetoutput) | List | Output destinations |
| [`platforms`](#targetplatforms) | List | Target platforms |
| [`pull`](#targetpull) | Boolean | Always pull images |
| [`secret`](#targetsecret) | List | Secrets to expose to the build |
| [`ssh`](#targetssh) | List | SSH agent sockets or keys to expose to the build |
| [`tags`](#targettags) | List | Image names and tags |
| [`target`](#targettarget) | String | Target build stage |
### `target.args`
Use the `args` attribute to define build arguments for the target.
This has the same effect as passing a [`--build-arg`][build-arg] flag to the build command.
```hcl
target "default" {
args = {
VERSION = "0.0.0+unknown"
}
}
```
You can set `args` attributes to use `null` values.
Doing so forces the `target` to use the `ARG` value specified in the Dockerfile.
```hcl
variable "GO_VERSION" {
default = "1.20.3"
}
target "webapp" {
dockerfile = "webapp.Dockerfile"
tags = ["docker.io/username/webapp"]
}
target "db" {
args = {
GO_VERSION = null
}
dockerfile = "db.Dockerfile"
tags = ["docker.io/username/db"]
}
```
### `target.attest`
The `attest` attribute lets you apply [build attestations][attestations] to the target.
This attribute accepts the long-form CSV version of attestation parameters.
```hcl
target "default" {
attest = [
"type=provenance,mode=min",
"type=sbom"
]
}
```
### `target.cache-from`
Build cache sources.
The builder imports cache from the locations you specify.
It uses the [Buildx cache storage backends][cache-backends],
and it works the same way as the [`--cache-from`][cache-from] flag.
This takes a list value, so you can specify multiple cache sources.
```hcl
target "app" {
cache-from = [
"type=s3,region=eu-west-1,bucket=mybucket",
"user/repo:cache",
]
}
```
### `target.cache-to`
Build cache export destinations.
The builder exports its build cache to the locations you specify.
It uses the [Buildx cache storage backends][cache-backends],
and it works the same way as the [`--cache-to` flag][cache-to].
This takes a list value, so you can specify multiple cache export targets.
```hcl
target "app" {
cache-to = [
"type=s3,region=eu-west-1,bucket=mybucket",
"type=inline"
]
}
```
### `target.context`
Specifies the location of the build context to use for this target.
Accepts a URL or a directory path.
This is the same as the [build context][context] positional argument
that you pass to the build command.
```hcl
target "app" {
context = "./src/www"
}
```
This resolves to the current working directory (`"."`) by default.
```console
$ docker buildx bake --print -f - <<< 'target "default" {}'
[+] Building 0.0s (0/0)
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile"
}
}
}
```
### `target.contexts`
Additional build contexts.
This is the same as the [`--build-context` flag][build-context].
This attribute takes a map, where keys result in named contexts that you can
reference in your builds.
You can specify different types of contexts, such local directories, Git URLs,
and even other Bake targets. Bake automatically determines the type of
a context based on the pattern of the context value.
| Context type | Example |
| --------------- | ----------------------------------------- |
| Container image | `docker-image://alpine@sha256:0123456789` |
| Git URL | `https://github.com/user/proj.git` |
| HTTP URL | `https://example.com/files` |
| Local directory | `../path/to/src` |
| Bake target | `target:base` |
#### Pin an image version
```hcl
# docker-bake.hcl
target "app" {
contexts = {
alpine = "docker-image://alpine:3.13"
}
}
```
```Dockerfile
# Dockerfile
FROM alpine
RUN echo "Hello world"
```
#### Use a local directory
```hcl
# docker-bake.hcl
target "app" {
contexts = {
src = "../path/to/source"
}
}
```
```Dockerfile
# Dockerfile
FROM scratch AS src
FROM golang
COPY --from=src . .
```
#### Use another target as base
> **Note**
>
> You should prefer to use regular multi-stage builds over this option. You can
> Use this feature when you have multiple Dockerfiles that can't be easily
> merged into one.
```hcl
# docker-bake.hcl
target "base" {
dockerfile = "baseapp.Dockerfile"
}
target "app" {
contexts = {
baseapp = "target:base"
}
}
```
```Dockerfile
# Dockerfile
FROM baseapp
RUN echo "Hello world"
```
### `target.dockerfile-inline`
Uses the string value as an inline Dockerfile for the build target.
```hcl
target "default" {
dockerfile-inline = "FROM alpine\nENTRYPOINT [\"echo\", \"hello\"]"
}
```
The `dockerfile-inline` takes precedence over the `dockerfile` attribute.
If you specify both, Bake uses the inline version.
### `target.dockerfile`
Name of the Dockerfile to use for the build.
This is the same as the [`--file` flag][file] for the `docker build` command.
```hcl
target "default" {
dockerfile = "./src/www/Dockerfile"
}
```
Resolves to `"Dockerfile"` by default.
```console
$ docker buildx bake --print -f - <<< 'target "default" {}'
[+] Building 0.0s (0/0)
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile"
}
}
}
```
### `target.inherits`
A target can inherit attributes from other targets.
Use `inherits` to reference from one target to another.
In the following example,
the `app-dev` target specifies an image name and tag.
The `app-release` target uses `inherits` to reuse the tag name.
```hcl
variable "TAG" {
default = "latest"
}
target "app-dev" {
tags = ["docker.io/username/myapp:${TAG}"]
}
target "app-release" {
inherits = ["app-dev"]
platforms = ["linux/amd64", "linux/arm64"]
}
```
The `inherits` attribute is a list,
meaning you can reuse attributes from multiple other targets.
In the following example, the `app-release` target reuses attributes
from both the `app-dev` and `_release` targets.
```hcl
target "app-dev" {
args = {
GO_VERSION = "1.20"
BUILDX_EXPERIMENTAL = 1
}
tags = ["docker.io/username/myapp"]
dockerfile = "app.Dockerfile"
labels = {
"org.opencontainers.image.source" = "https://github.com/username/myapp"
}
}
target "_release" {
args = {
BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1
BUILDX_EXPERIMENTAL = 0
}
}
target "app-release" {
inherits = ["app-dev", "_release"]
platforms = ["linux/amd64", "linux/arm64"]
}
```
When inheriting attributes from multiple targets and there's a conflict,
the target that appears last in the `inherits` list takes precedence.
The previous example defines the `BUILDX_EXPERIMENTAL` argument twice for the `app-release` target.
It resolves to `0` because the `_release` target appears last in the inheritance chain:
```console
$ docker buildx bake --print app-release
[+] Building 0.0s (0/0)
{
"group": {
"default": {
"targets": [
"app-release"
]
}
},
"target": {
"app-release": {
"context": ".",
"dockerfile": "app.Dockerfile",
"args": {
"BUILDKIT_CONTEXT_KEEP_GIT_DIR": "1",
"BUILDX_EXPERIMENTAL": "0",
"GO_VERSION": "1.20"
},
"labels": {
"org.opencontainers.image.source": "https://github.com/username/myapp"
},
"tags": [
"docker.io/username/myapp"
],
"platforms": [
"linux/amd64",
"linux/arm64"
]
}
}
}
```
### `target.labels`
Assigns image labels to the build.
This is the same as the `--label` flag for `docker build`.
```hcl
target "default" {
labels = {
"org.opencontainers.image.source" = "https://github.com/username/myapp"
"com.docker.image.source.entrypoint" = "Dockerfile"
}
}
```
It's possible to use a `null` value for labels.
If you do, the builder uses the label value specified in the Dockerfile.
### `target.matrix`
A matrix strategy lets you fork a single target into multiple different
variants, based on parameters that you specify.
This works in a similar way to [Matrix strategies for GitHub Actions].
You can use this to reduce duplication in your bake definition.
The `matrix` attribute is a map of parameter names to lists of values.
Bake builds each possible combination of values as a separate target.
Each generated target **must** have a unique name.
To specify how target names should resolve, use the `name` attribute.
The following example resolves the `app` target to `app-foo` and `app-bar`.
It also uses the matrix value to define the [target build stage](#targettarget).
```hcl
target "app" {
name = "app-${tgt}"
matrix = {
tgt = ["foo", "bar"]
}
target = tgt
}
```
```console
$ docker buildx bake --print app
[+] Building 0.0s (0/0)
{
"group": {
"app": {
"targets": [
"app-foo",
"app-bar"
]
},
"default": {
"targets": [
"app"
]
}
},
"target": {
"app-bar": {
"context": ".",
"dockerfile": "Dockerfile",
"target": "bar"
},
"app-foo": {
"context": ".",
"dockerfile": "Dockerfile",
"target": "foo"
}
}
}
```
#### Multiple axes
You can specify multiple keys in your matrix to fork a target on multiple axes.
When using multiple matrix keys, Bake builds every possible variant.
The following example builds four targets:
- `app-foo-1-0`
- `app-foo-2-0`
- `app-bar-1-0`
- `app-bar-2-0`
```hcl
target "app" {
name = "app-${tgt}-${replace(version, ".", "-")}"
matrix = {
tgt = ["foo", "bar"]
version = ["1.0", "2.0"]
}
target = tgt
args = {
VERSION = version
}
}
```
#### Multiple values per matrix target
If you want to differentiate the matrix on more than just a single value,
you can use maps as matrix values. Bake creates a target for each map,
and you can access the nested values using dot notation.
The following example builds two targets:
- `app-foo-1-0`
- `app-bar-2-0`
```hcl
target "app" {
name = "app-${item.tgt}-${replace(item.version, ".", "-")}"
matrix = {
item = [
{
tgt = "foo"
version = "1.0"
},
{
tgt = "bar"
version = "2.0"
}
]
}
target = item.tgt
args = {
VERSION = item.version
}
}
```
### `target.name`
Specify name resolution for targets that use a matrix strategy.
The following example resolves the `app` target to `app-foo` and `app-bar`.
```hcl
target "app" {
name = "app-${tgt}"
matrix = {
tgt = ["foo", "bar"]
}
target = tgt
}
```
### `target.no-cache-filter`
Don't use build cache for the specified stages.
This is the same as the `--no-cache-filter` flag for `docker build`.
The following example avoids build cache for the `foo` build stage.
```hcl
target "default" {
no-cache-filter = ["foo"]
}
```
### `target.no-cache`
Don't use cache when building the image.
This is the same as the `--no-cache` flag for `docker build`.
```hcl
target "default" {
no-cache = 1
}
```
### `target.output`
Configuration for exporting the build output.
This is the same as the [`--output` flag][output].
The following example configures the target to use a cache-only output,
```hcl
target "default" {
output = ["type=cacheonly"]
}
```
### `target.platforms`
Set target platforms for the build target.
This is the same as the [`--platform` flag][platform].
The following example creates a multi-platform build for three architectures.
```hcl
target "default" {
platforms = ["linux/amd64", "linux/arm64", "linux/arm/v7"]
}
```
### `target.pull`
Configures whether the builder should attempt to pull images when building the target.
This is the same as the `--pull` flag for `docker build`.
The following example forces the builder to always pull all images referenced in the build target.
```hcl
target "default" {
pull = "always"
}
```
### `target.secret`
Defines secrets to expose to the build target.
This is the same as the [`--secret` flag][secret].
```hcl
variable "HOME" {
default = null
}
target "default" {
secret = [
"type=env,id=KUBECONFIG",
"type=file,id=aws,src=${HOME}/.aws/credentials"
]
}
```
This lets you [mount the secret][run_mount_secret] in your Dockerfile.
```dockerfile
RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
aws cloudfront create-invalidation ...
RUN --mount=type=secret,id=KUBECONFIG \
KUBECONFIG=$(cat /run/secrets/KUBECONFIG) helm upgrade --install
```
### `target.ssh`
Defines SSH agent sockets or keys to expose to the build.
This is the same as the [`--ssh` flag][ssh].
This can be useful if you need to access private repositories during a build.
```hcl
target "default" {
ssh = ["default"]
}
```
```dockerfile
FROM alpine
RUN --mount=type=ssh \
apk add git openssh-client \
&& install -m 0700 -d ~/.ssh \
&& ssh-keyscan github.com >> ~/.ssh/known_hosts \
&& git clone git@github.com:user/my-private-repo.git
```
### `target.tags`
Image names and tags to use for the build target.
This is the same as the [`--tag` flag][tag].
```hcl
target "default" {
tags = [
"org/repo:latest",
"myregistry.azurecr.io/team/image:v1"
]
}
```
### `target.target`
Set the target build stage to build.
This is the same as the [`--target` flag][target].
```hcl
target "default" {
target = "binaries"
}
```
## Group
Groups allow you to invoke multiple builds (targets) at once.
```hcl
group "default" {
targets = ["db", "webapp-dev"]
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
target "db" {
dockerfile = "Dockerfile.db"
tags = ["docker.io/username/db"]
}
```
Groups take precedence over targets, if both exist with the same name.
The following bake file builds the `default` group.
Bake ignores the `default` target.
```hcl
target "default" {
dockerfile-inline = "FROM ubuntu"
}
group "default" {
targets = ["alpine", "debian"]
}
target "alpine" {
dockerfile-inline = "FROM alpine"
}
target "debian" {
dockerfile-inline = "FROM debian"
}
```
## Variable
The HCL file format supports variable block definitions.
You can use variables as build arguments in your Dockerfile,
or interpolate them in attribute values in your Bake file.
```hcl
variable "TAG" {
default = "latest"
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:${TAG}"]
}
```
You can assign a default value for a variable in the Bake file,
or assign a `null` value to it. If you assign a `null` value,
Buildx uses the default value from the Dockerfile instead.
You can override variable defaults set in the Bake file using environment variables.
The following example sets the `TAG` variable to `dev`,
overriding the default `latest` value shown in the previous example.
```console
$ TAG=dev docker buildx bake webapp-dev
```
### Built-in variables
The following variables are built-ins that you can use with Bake without having
to define them.
| Variable | Description |
| --------------------- | ----------------------------------------------------------------------------------- |
| `BAKE_CMD_CONTEXT` | Holds the main context when building using a remote Bake file. |
| `BAKE_LOCAL_PLATFORM` | Returns the current platforms default platform specification (e.g. `linux/amd64`). |
### Use environment variable as default
You can set a Bake variable to use the value of an environment variable as a default value:
```hcl
variable "HOME" {
default = "$HOME"
}
```
### Interpolate variables into attributes
To interpolate a variable into an attribute string value,
you must use curly brackets.
The following doesn't work:
```hcl
variable "HOME" {
default = "$HOME"
}
target "default" {
ssh = ["default=$HOME/.ssh/id_rsa"]
}
```
Wrap the variable in curly brackets where you want to insert it:
```diff
variable "HOME" {
default = "$HOME"
}
target "default" {
- ssh = ["default=$HOME/.ssh/id_rsa"]
+ ssh = ["default=${HOME}/.ssh/id_rsa"]
}
```
Before you can interpolate a variable into an attribute,
first you must declare it in the bake file,
as demonstrated in the following example.
```console
$ cat docker-bake.hcl
target "default" {
dockerfile-inline = "FROM ${BASE_IMAGE}"
}
$ docker buildx bake
[+] Building 0.0s (0/0)
docker-bake.hcl:2
--------------------
1 | target "default" {
2 | >>> dockerfile-inline = "FROM ${BASE_IMAGE}"
3 | }
4 |
--------------------
ERROR: docker-bake.hcl:2,31-41: Unknown variable; There is no variable named "BASE_IMAGE"., and 1 other diagnostic(s)
$ cat >> docker-bake.hcl
variable "BASE_IMAGE" {
default = "alpine"
}
$ docker buildx bake
[+] Building 0.6s (5/5) FINISHED
```
## Function
A [set of general-purpose functions][bake_stdlib]
provided by [go-cty][go-cty]
are available for use in HCL files:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${add(123, 1)}"
}
}
```
In addition, [user defined functions][userfunc]
are also supported:
```hcl
# docker-bake.hcl
function "increment" {
params = [number]
result = number + 1
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${increment(123)}"
}
}
```
> **Note**
>
> See [User defined HCL functions][hcl-funcs] page for more details.
<!-- external links -->
[attestations]: https://docs.docker.com/build/attestations/
[bake_stdlib]: https://github.com/docker/buildx/blob/master/bake/hclparser/stdlib.go
[build-arg]: https://docs.docker.com/engine/reference/commandline/build/#build-arg
[build-context]: https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
[cache-backends]: https://docs.docker.com/build/cache/backends/
[cache-from]: https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-from
[cache-to]: https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to
[context]: https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
[file]: https://docs.docker.com/engine/reference/commandline/build/#file
[go-cty]: https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib
[hcl-funcs]: https://docs.docker.com/build/bake/hcl-funcs/
[output]: https://docs.docker.com/engine/reference/commandline/buildx_build/#output
[platform]: https://docs.docker.com/engine/reference/commandline/buildx_build/#platform
[run_mount_secret]: https://docs.docker.com/engine/reference/builder/#run---mounttypesecret
[secret]: https://docs.docker.com/engine/reference/commandline/buildx_build/#secret
[ssh]: https://docs.docker.com/engine/reference/commandline/buildx_build/#ssh
[tag]: https://docs.docker.com/engine/reference/commandline/build/#tag
[target]: https://docs.docker.com/engine/reference/commandline/build/#target
[userfunc]: https://github.com/hashicorp/hcl/tree/main/ext/userfunc

View File

@@ -1,48 +1,3 @@
# CI/CD
## GitHub Actions
Docker provides a [GitHub Action that will build and push your image](https://github.com/docker/build-push-action/#about)
using Buildx. Here is a simple workflow:
```yaml
name: ci
on:
push:
branches:
- 'main'
jobs:
docker:
runs-on: ubuntu-latest
steps:
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Login to DockerHub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Build and push
uses: docker/build-push-action@v2
with:
push: true
tags: user/app:latest
```
In this example we are also using 3 other actions:
* [`setup-buildx`](https://github.com/docker/setup-buildx-action) action will create and boot a builder using by
default the `docker-container` [builder driver](https://docs.docker.com/engine/reference/commandline/buildx_create/#driver).
This is **not required but recommended** using it to be able to build multi-platform images, export cache, etc.
* [`setup-qemu`](https://github.com/docker/setup-qemu-action) action can be useful if you want
to add emulation support with QEMU to be able to build against more platforms.
* [`login`](https://github.com/docker/login-action) action will take care to log
in against a Docker registry.
This page has moved to [Docker Docs website](https://docs.docker.com/build/ci/)

View File

@@ -1,23 +1,3 @@
# CNI networking
It can be useful to use a bridge network for your builder if for example you
encounter a network port contention during multiple builds. If you're using
the BuildKit image, CNI is not yet available in it, but you can create
[a custom BuildKit image with CNI support](https://github.com/moby/buildkit/blob/master/docs/cni-networking.md).
Now build this image:
```console
$ docker buildx build --tag buildkit-cni:local --load .
```
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) that
will use this image:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "image=buildkit-cni:local" \
--buildkitd-flags "--oci-worker-net=cni"
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#cni-networking)

View File

@@ -1,20 +1,3 @@
# Color output controls
Buildx has support for modifying the colors that are used to output information
to the terminal. You can set the environment variable `BUILDKIT_COLORS` to
something like `run=123,20,245:error=yellow:cancel=blue:warning=white` to set
the colors that you would like to use:
![Progress output custom colors](https://user-images.githubusercontent.com/1951866/180584033-24522385-cafd-4a54-a4a2-18f5ce74eb27.png)
Setting `NO_COLOR` to anything will disable any colorized output as recommended
by [no-color.org](https://no-color.org/):
![Progress output no color](https://user-images.githubusercontent.com/1951866/180584037-e28f9997-dd4c-49cf-8b26-04864815de19.png)
> **Note**
>
> Parsing errors will be reported but ignored. This will result in default
> color values being used where needed.
See also [the list of pre-defined colors](https://github.com/moby/buildkit/blob/master/util/progress/progressui/colors.go).
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/env-vars/#buildkit_colors)

View File

@@ -1,34 +1,3 @@
# Using a custom network
[Create a network](https://docs.docker.com/engine/reference/commandline/network_create/)
named `foonet`:
```console
$ docker network create foonet
```
[Create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
named `mybuilder` that will use this network:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "network=foonet"
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
[Inspect the builder container](https://docs.docker.com/engine/reference/commandline/inspect/)
and see what network is being used:
{% raw %}
```console
$ docker inspect buildx_buildkit_mybuilder0 --format={{.NetworkSettings.Networks}}
map[foonet:0xc00018c0c0]
```
{% endraw %}
This page has moved to [Docker Docs website](https://docs.docker.com/build/drivers/docker-container/#custom-network)

View File

@@ -1,63 +1,3 @@
# Using a custom registry configuration
If you [create a `docker-container` or `kubernetes` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/) and
have specified certificates for registries in the [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md),
the files will be copied into the container under `/etc/buildkit/certs` and
configuration will be updated to reflect that.
Take the following `buildkitd.toml` configuration that will be used for
pushing an image to this registry using self-signed certificates:
```toml
# /etc/buildkitd.toml
debug = true
[registry."myregistry.com"]
ca=["/etc/certs/myregistry.pem"]
[[registry."myregistry.com".keypair]]
key="/etc/certs/myregistry_key.pem"
cert="/etc/certs/myregistry_cert.pem"
```
Here we have configured a self-signed certificate for `myregistry.com` registry.
Now [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use this BuildKit configuration:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--config /etc/buildkitd.toml
```
Inspecting the builder container, you can see that buildkitd configuration
has changed:
```console
$ docker exec -it buildx_buildkit_mybuilder0 cat /etc/buildkit/buildkitd.toml
```
```toml
debug = true
[registry]
[registry."myregistry.com"]
ca = ["/etc/buildkit/certs/myregistry.com/myregistry.pem"]
[[registry."myregistry.com".keypair]]
cert = "/etc/buildkit/certs/myregistry.com/myregistry_cert.pem"
key = "/etc/buildkit/certs/myregistry.com/myregistry_key.pem"
```
And certificates copied inside the container:
```console
$ docker exec -it buildx_buildkit_mybuilder0 ls /etc/buildkit/certs/myregistry.com/
myregistry.pem myregistry_cert.pem myregistry_key.pem
```
Now you should be able to push to the registry with this builder:
```console
$ docker buildx build --push --tag myregistry.com/myimage:latest .
```
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#setting-registry-certificates)

164
docs/guides/debugging.md Normal file
View File

@@ -0,0 +1,164 @@
# Debug monitor
To assist with creating and debugging complex builds, Buildx provides a
debugger to help you step through the build process and easily inspect the
state of the build environment at any point.
> **Note**
>
> The debug monitor is a new experimental feature in recent versions of Buildx.
> There are rough edges, known bugs, and missing features. Please try it out
> and let us know what you think!
## Starting the debugger
To start the debugger, first, ensure that `BUILDX_EXPERIMENTAL=1` is set in
your environment.
```console
$ export BUILDX_EXPERIMENTAL=1
```
To start a debug session for a build, you can use the `--invoke` flag with the
build command to specify a command to launch in the resulting image.
```console
$ docker buildx build --invoke /bin/sh .
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
...
Launching interactive container. Press Ctrl-a-c to switch to monitor console
Interactive container was restarted with process "dzz7pjb4pk1mj29xqrx0ac3oj". Press Ctrl-a-c to switch to the new container
Switched IO
/ #
```
This launches a `/bin/sh` process in the final stage of the image, and allows
you to explore the contents of the image, without needing to export or load the
image outside of the builder.
For example, you can use `ls` to see the contents of the image:
```console
/ # ls
bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr work
```
Optional long form allows you specifying detailed configurations of the process.
It must be CSV-styled comma-separated key-value pairs.
Supported keys are `args` (can be JSON array format), `entrypoint` (can be JSON array format), `env` (can be JSON array format), `user`, `cwd` and `tty` (bool).
Example:
```
$ docker buildx build --invoke 'entrypoint=["sh"],"args=[""-c"", ""env | grep -e FOO -e AAA""]","env=[""FOO=bar"", ""AAA=bbb""]"' .
```
#### `on-error`
If you want to start a debug session when a build fails, you can use
`--invoke=on-error` to start a debug session when the build fails.
```console
$ docker buildx build --invoke on-error .
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
...
=> ERROR [shell 10/10] RUN bad-command
------
> [shell 10/10] RUN bad-command:
#0 0.049 /bin/sh: bad-command: not found
------
Launching interactive container. Press Ctrl-a-c to switch to monitor console
Interactive container was restarted with process "edmzor60nrag7rh1mbi4o9lm8". Press Ctrl-a-c to switch to the new container
/ #
```
This allows you to explore the state of the image when the build failed.
#### `debug-shell`
If you want to drop into a debug session without first starting the build, you
can use `--invoke=debug-shell` to start a debug session.
```
$ docker buildx build --invoke debug-shell .
[+] Building 4.2s (19/19) FINISHED
=> [internal] connecting to local controller 0.0s
(buildx)
```
You can then use the commands available in [monitor mode](#monitor-mode) to
start and observe the build.
## Monitor mode
By default, when debugging, you'll be dropped into a shell in the final stage.
When you're in a debug shell, you can use the `Ctrl-a-c` key combination (press
`Ctrl`+`a` together, lift, then press `c`) to toggle between the debug shell
and the monitor mode. In monitor mode, you can run commands that control the
debug environment.
```console
(buildx) help
Available commands are:
attach attach to a buildx server or a process in the container
disconnect disconnect a client from a buildx server. Specific session ID can be specified an arg
exec execute a process in the interactive container
exit exits monitor
help shows this message
kill kill buildx server
list list buildx sessions
ps list processes invoked by "exec". Use "attach" to attach IO to that process
reload reloads the context and build it
rollback re-runs the interactive container with initial rootfs contents
```
## Build controllers
Debugging is performed using a buildx "controller", which provides a high-level
abstraction to perform builds. By default, the local controller is used for a
more stable experience which runs all builds in-process. However, you can also
use the remote controller to detach the build process from the CLI.
To detach the build process from the CLI, you can use the `--detach=true` flag with
the build command.
```console
$ docker buildx build --detach=true --invoke /bin/sh .
```
If you start a debugging session using the `--invoke` flag with a detached
build, then you can attach to it using the `buildx debug-shell` subcommand to
immediately enter the monitor mode.
```console
$ docker buildx debug-shell
[+] Building 0.0s (1/1) FINISHED
=> [internal] connecting to remote controller
(buildx) list
ID CURRENT_SESSION
xfe1162ovd9def8yapb4ys66t false
(buildx) attach xfe1162ovd9def8yapb4ys66t
Attached to process "". Press Ctrl-a-c to switch to the new container
(buildx) ps
PID CURRENT_SESSION COMMAND
3ug8iqaufiwwnukimhqqt06jz false [sh]
(buildx) attach 3ug8iqaufiwwnukimhqqt06jz
Attached to process "3ug8iqaufiwwnukimhqqt06jz". Press Ctrl-a-c to switch to the new container
(buildx) Switched IO
/ # ls
bin etc lib mnt proc run srv tmp var
dev home media opt root sbin sys usr work
/ #
```

View File

@@ -1,31 +1,3 @@
# OpenTelemetry support
To capture the trace to [Jaeger](https://github.com/jaegertracing/jaeger), set
`JAEGER_TRACE` environment variable to the collection address using a `driver-opt`.
First create a Jaeger container:
```console
$ docker run -d --name jaeger -p "6831:6831/udp" -p "16686:16686" jaegertracing/all-in-one
```
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use the Jaeger instance via the `JAEGER_TRACE` env var:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--driver-opt "network=host" \
--driver-opt "env.JAEGER_TRACE=localhost:6831"
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
Buildx commands should be traced at `http://127.0.0.1:16686/`:
![OpenTelemetry Buildx Bake](https://user-images.githubusercontent.com/1951866/124468052-ef085400-dd98-11eb-84ab-7ac8e261dd52.png)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/opentelemetry/)

View File

@@ -1,62 +1,3 @@
# Registry mirror
You can define a registry mirror to use for your builds by providing a [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md)
while creating a builder with the [`--config` flags](https://docs.docker.com/engine/reference/commandline/buildx_create/#config).
```toml
# /etc/buildkitd.toml
debug = true
[registry."docker.io"]
mirrors = ["mirror.gcr.io"]
```
> **Note**
>
> `debug = true` has been added to be able to debug requests
> in the BuildKit daemon and see if the mirror is effectively used.
Then [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use this BuildKit configuration:
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--config /etc/buildkitd.toml
```
Boot and [inspect `mybuilder`](https://docs.docker.com/engine/reference/commandline/buildx_inspect/):
```console
$ docker buildx inspect --bootstrap
```
Build an image:
```console
$ docker buildx build --load . -f-<<EOF
FROM alpine
RUN echo "hello world"
EOF
```
Now let's check the BuildKit logs in the builder container:
```console
$ docker logs buildx_buildkit_mybuilder0
```
```text
...
time="2022-02-06T17:47:48Z" level=debug msg="do request" request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=containerd/1.5.8+unknown request.method=GET spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=1356 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=1469 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:25:17 GMT" response.header.etag="\"774380abda8f4eae9a149e5d5d3efc83\"" response.header.expires="Sun, 06 Feb 2022 18:25:17 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:57 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788077652182 response.header.x-goog-hash="crc32c=V3DSrg==" response.header.x-goog-hash.1="md5=d0OAq9qPTq6aFJ5dXT78gw==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=1469 response.header.x-guploader-uploadid=ADPycduqQipVAXc3tzXmTzKQ2gTT6CV736B2J628smtD1iDytEyiYCgvvdD8zz9BT1J1sASUq9pW_ctUyC4B-v2jvhIxnZTlKg response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=760 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=1471 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:35:13 GMT" response.header.etag="\"35d688bd15327daafcdb4d4395e616a8\"" response.header.expires="Sun, 06 Feb 2022 18:35:13 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:12 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788032100793 response.header.x-goog-hash="crc32c=aWgRjA==" response.header.x-goog-hash.1="md5=NdaIvRUyfar8201DleYWqA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=1471 response.header.x-guploader-uploadid=ADPycdtR-gJYwC7yHquIkJWFFG8FovDySvtmRnZBqlO3yVDanBXh_VqKYt400yhuf0XbQ3ZMB9IZV2vlcyHezn_Pu3a1SMMtiw response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg=fetch spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="do request" request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.5.8+unknown request.method=GET spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
time="2022-02-06T17:47:48Z" level=debug msg="fetch response received" response.header.accept-ranges=bytes response.header.age=1356 response.header.alt-svc="h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"" response.header.cache-control="public, max-age=3600" response.header.content-length=2818413 response.header.content-type=application/octet-stream response.header.date="Sun, 06 Feb 2022 17:25:17 GMT" response.header.etag="\"1d55e7be5a77c4a908ad11bc33ebea1c\"" response.header.expires="Sun, 06 Feb 2022 18:25:17 GMT" response.header.last-modified="Wed, 24 Nov 2021 21:07:06 GMT" response.header.server=UploadServer response.header.x-goog-generation=1637788026431708 response.header.x-goog-hash="crc32c=ZojF+g==" response.header.x-goog-hash.1="md5=HVXnvlp3xKkIrRG8M+vqHA==" response.header.x-goog-metageneration=1 response.header.x-goog-storage-class=STANDARD response.header.x-goog-stored-content-encoding=identity response.header.x-goog-stored-content-length=2818413 response.header.x-guploader-uploadid=ADPycdsebqxiTBJqZ0bv9zBigjFxgQydD2ESZSkKchpE0ILlN9Ibko3C5r4fJTJ4UR9ddp-UBd-2v_4eRpZ8Yo2llW_j4k8WhQ response.status="200 OK" spanID=9460e5b6e64cec91 traceID=b162d3040ddf86d6614e79c66a01a577
...
```
As you can see, requests come from the GCR registry mirror (`response.header.x-goog*`).
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#registry-mirror)

View File

@@ -1,33 +1,3 @@
# Resource limiting
## Max parallelism
You can limit the parallelism of the BuildKit solver, which is particularly useful
for low-powered machines, using a [BuildKit daemon configuration](https://github.com/moby/buildkit/blob/master/docs/buildkitd.toml.md)
while creating a builder with the [`--config` flags](https://docs.docker.com/engine/reference/commandline/buildx_create/#config).
```toml
# /etc/buildkitd.toml
[worker.oci]
max-parallelism = 4
```
Now you can [create a `docker-container` builder](https://docs.docker.com/engine/reference/commandline/buildx_create/)
that will use this BuildKit configuration to limit parallelism.
```console
$ docker buildx create --use \
--name mybuilder \
--driver docker-container \
--config /etc/buildkitd.toml
```
## Limit on TCP connections
We are also now limiting TCP connections to **4 per registry** with an additional
connection not used for layer pulls and pushes. This limitation will be able to
manage TCP connection per host to avoid your build being stuck while pulling
images. The additional connection is used for metadata requests
(image config retrieval) to enhance the overall build time.
More info: [moby/buildkit#2259](https://github.com/moby/buildkit/pull/2259)
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#resource-limiting)

View File

@@ -1,14 +0,0 @@
# Buildx manuals 📚
This directory contains a bunch of useful docs for how to use Buildx features.
> **Note**
>
> The markdown files in this directory (excluding this README) are reused
> downstream by the
> [Docker documentation repository](https://github.com/docker/docs).
>
> If you wish to contribute to these docs, be sure to first review the
> [documentation contribution guidelines](https://docs.docker.com/contribute/overview/).
>
> Thank you!

View File

@@ -1,3 +1,3 @@
# Defining additional build contexts and linking targets
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake/build-contexts)
This page has moved to [Docker Docs website](https://docs.docker.com/build/bake/build-contexts)

View File

@@ -1,3 +1,3 @@
# Building from Compose file
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake/compose-file)
This page has moved to [Docker Docs website](https://docs.docker.com/build/bake/compose-file)

View File

@@ -1,3 +1,3 @@
# Configuring builds
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake/configuring-build)
This page has moved to [Docker Docs website](https://docs.docker.com/build/bake/configuring-build)

View File

@@ -1,3 +1,3 @@
# Bake file definition
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake/file-definition)
This page has moved to [docs/bake-reference.md](../../bake-reference.md)

View File

@@ -1,3 +1,3 @@
# User defined HCL functions
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake/hcl-funcs)
This page has moved to [Docker Docs website](https://docs.docker.com/build/bake/hcl-funcs)

View File

@@ -1,3 +1,3 @@
# High-level build options with Bake
Moved to [docs.docker.com](https://docs.docker.com/build/customize/bake)
This page has moved to [Docker Docs website](https://docs.docker.com/build/bake)

View File

@@ -1,3 +1,3 @@
# Azure Blob Storage cache storage
Moved to [docs.docker.com](https://docs.docker.com/build/building/cache/backends/azblob)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/cache/backends/azblob)

View File

@@ -1,3 +1,3 @@
# GitHub Actions cache storage
Moved to [docs.docker.com](https://docs.docker.com/build/building/cache/backends/gha)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/cache/backends/gha)

View File

@@ -1,3 +1,3 @@
# Cache storage backends
Moved to [docs.docker.com](https://docs.docker.com/build/building/cache/backends)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/cache/backends)

View File

@@ -1,3 +1,3 @@
# Inline cache storage
Moved to [docs.docker.com](https://docs.docker.com/build/building/cache/backends/inline)
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/cache/backends/inline)

Some files were not shown because too many files have changed in this diff Show More