Compare commits

...

1296 Commits

Author SHA1 Message Date
Justin Chadwell
47e34f2684 Merge pull request #1813 from jedevc/oci-export-on-containerd 2023-06-07 15:19:54 +02:00
Justin Chadwell
3d981be4ad Merge pull request #1850 from ktock/monitor-longhelp 2023-06-07 15:09:35 +02:00
CrazyMax
5d94b0fcc7 Merge pull request #1866 from crazy-max/bump-compose
vendor: github.com/compose-spec/compose-go v1.14.0
2023-06-07 14:55:55 +02:00
CrazyMax
569c66fb62 Merge pull request #1867 from crazy-max/bump-testify
vendor: github.com/stretchr/testify v1.8.4
2023-06-07 12:19:33 +02:00
CrazyMax
93f7fbdd78 Merge pull request #1865 from crazy-max/bump-semver
vendor: github.com/Masterminds/semver/v3 v3.2.1
2023-06-07 12:19:17 +02:00
Tõnis Tiigi
ea06685c11 Merge pull request #1858 from potherca-contrib/fix/typo-Shortand
Fix typo "Shortand" -> "Shorthand"
2023-06-06 14:03:40 -07:00
Ben Peachey
eaba4fa9e6 Update docs with result from make docs command.
Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-06-06 21:43:41 +02:00
CrazyMax
99e3882e2a Merge pull request #1841 from crazy-max/desktop
build: display build details link
2023-06-06 19:06:19 +02:00
CrazyMax
0a2f35970c build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
CrazyMax
ab5f5e4169 build: fix quiet mode
We are using the quiet flag option and we are not taking
progress quiet mode into account

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:47:55 +02:00
Justin Chadwell
696770d29c Merge pull request #1840 from jedevc/fix-check-path-for-bake-cmd-context 2023-06-06 17:49:04 +02:00
CrazyMax
b47b4e5957 vendor: github.com/stretchr/testify v1.8.4
full diff https://github.com/stretchr/testify/compare/v1.8.2...v1.8.4

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:44:58 +02:00
CrazyMax
9a125afba0 vendor: github.com/compose-spec/compose-go v1.14.0
https: //github.com/compose-spec/compose-go/compare/v1.13.4...v1.14.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 17:34:34 +02:00
Justin Chadwell
d34103b0d9 bake: fix potential context entitlements escape
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
c820350b5e tests: refactor cmd helpers to allow configuring cwd, etc
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:17:03 +02:00
Justin Chadwell
61a7854659 tests: add bake test for remote subdir context
Fixed in 12b6a3ad9a, but now we have
regression tests! So we can add a check that we don't break this
behavior again.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
e859ebc12e tests: add bake test for remote cmd context override
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
ef997fd6d0 bake: fix BAKE_CMD_CONTEXT relative path resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
76c96347ff tests: add basic remote bake context test
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
48d7dafbd5 git: update gitutil test utilities
- Adds a new GitServeHTTP function to start an http server to serve a
  target git repository.
- Adds a new GitDir helper method to get the path to the .git
  directory
- Updates the GitAdd method to take a variable number of files

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
d03e93f6f1 test: tmpdir can be a test helper
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 17:15:19 +02:00
Justin Chadwell
fcb7810a38 Merge pull request #1854 from crazy-max/inspect-features 2023-06-06 16:59:58 +02:00
CrazyMax
459d94bdf1 vendor: github.com/Masterminds/semver/v3 v3.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 16:55:59 +02:00
Justin Chadwell
7cef021a8a Merge pull request #1804 from jedevc/fixup-solve 2023-06-06 09:12:10 +02:00
Justin Chadwell
c6db4cf342 build: clarify NewResultHandle comment
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-06 09:08:30 +02:00
CrazyMax
6c9436fbd5 inspect: display builder features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-05 23:11:58 +02:00
Justin Chadwell
a906149930 Merge pull request #1862 from jedevc/update-contributing-tests
docs: update CONTRIBUTING.md with specific test notes
2023-06-01 21:55:12 +01:00
Justin Chadwell
af328fe413 docs: update CONTRIBUTING.md with specific test notes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-06-01 11:34:20 +01:00
Justin Chadwell
183a73abae build: docker exporter should instead use oci with containerd
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:59:55 +01:00
Justin Chadwell
b7f0b3d763 build: clear exports for secondary solve request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:56 +01:00
Justin Chadwell
5b27d5a9f6 build: cleanup res if returned in basic build
In practice, this shouldn't happen, but the check is good to include
anyways.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 15:13:03 +01:00
Ben Peachey
8f24c58f4d Fix typo in commands/build.go and docs/reference/buildx_build.md
`Shortand` -> `Shorthand`

Signed-off-by: Ben Peachey <potherca@gmail.com>
2023-05-31 13:40:58 +02:00
Justin Chadwell
cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell
8d822fb06c build: move main solve request into main gateway call
Now, we always perform the full solve request in the main gateway call.
This ensures that progress works properly, and makes the lifetime
semantics much clearer.

NewResultContext abstracts the details of a successful/failed build, to
always return a single ResultContext, even though the details of how a
gateway is created is different:
- For a failed build, we can just keep the gateway open.
- For a successful build, we immediately open another gateway and
  re-evaluate the build definition in that gateway. This should give an
  instant cache hit (since the build was just successful).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:46:46 +01:00
CrazyMax
0758a9b75d Merge pull request #1856 from crazy-max/history-feature-update
driver: update history api check
2023-05-31 10:27:54 +02:00
CrazyMax
f8fa526678 driver: update history api check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-31 10:10:54 +02:00
Tõnis Tiigi
4abff3ce12 Merge pull request #1852 from jedevc/vendor-buildkit-master
vendor: update buildkit to master@c36941f4a10e
2023-05-30 16:14:26 -07:00
CrazyMax
e7034f66bc Merge pull request #1846 from crazy-max/history-capability
driver: check history capability
2023-05-30 20:25:31 +02:00
CrazyMax
8c65e4fc1d driver: add context to Features interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
d196ac347e driver: cache features
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:18 +02:00
CrazyMax
9b723ece46 driver: check history capability
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 20:13:17 +02:00
Justin Chadwell
5e2f8bd64a Merge pull request #1853 from jedevc/logrus-show-error 2023-05-30 10:29:49 +01:00
Justin Chadwell
5788ab33d2 logutil: print error from WithError if found
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 10:00:24 +01:00
CrazyMax
f1788002e1 Merge pull request #1851 from crazy-max/test-flow
test: update workflow
2023-05-30 10:58:17 +02:00
Justin Chadwell
6c62225d1b vendor: update buildkit to master@c36941f4a10e
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-30 09:10:52 +01:00
CrazyMax
38b4eef5c6 ci: make release and bin-image depend on test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
a4db138c5e docs: update CONTRIBUTING.md to run tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
55377b2b0f make: test-unit and test-integration targets
Also switch test target to run the test script.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
98dedd3225 ci: cache integration tests base and build in dedicated step
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
74b121be66 test: opt to avoid building the integration tests image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
b9cf46785b test: use trap to remove cache volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:08 +02:00
CrazyMax
ecf8dd0a26 test: type integration not needed
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
73c17ef4d2 ci: send test reports to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
CrazyMax
e762e46b4b ci: merge test workflow to build one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-30 02:51:07 +02:00
Kohei Tokunaga
cafeedba79 monitor: add long help for commands
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-29 23:23:56 +09:00
CrazyMax
17bdbbd3c3 Merge pull request #1831 from crazy-max/fix-ctn-certs-path
driver(docker-container): fix incorrect path when writing certs
2023-05-26 16:15:21 +02:00
Justin Chadwell
2dae553d18 build: update outline fallback image
Subrequests have been included in docker/dockerfile:1.5 labs, so we can
update the fallback to point to this release.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
Justin Chadwell
91c17f25fb build: tidy up print func
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-26 10:38:38 +01:00
CrazyMax
63fc01e08a Merge pull request #1842 from crazy-max/update-xx
update xx to 1.2.1
2023-05-26 11:22:33 +02:00
Justin Chadwell
354ccc9469 Merge pull request #1844 from tonistiigi/imageid-fixes 2023-05-26 10:21:53 +01:00
Tonis Tiigi
68ae67720a build: fix writing correct image ID with -q
Container driver wrote manifest digest that had a
mismatch with --iidfile output.

When --iidfile was set the --metadata-file was not
written.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:20:29 -07:00
Tonis Tiigi
b273db20c3 remote: allow docker exporter on remote driver
There is no reason why one can't make a docker tarball or
load to Docker instance from remote driver.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-26 00:18:57 -07:00
CrazyMax
0ae88ecc4d update xx to 1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-25 15:26:56 +02:00
Justin Chadwell
341fb65f6f Merge pull request #1830 from jedevc/fix-missing-driver-in-build 2023-05-25 11:35:01 +01:00
Justin Chadwell
69a9c6609a Merge pull request #1699 from jedevc/bake-attestation-override 2023-05-25 09:54:34 +01:00
CrazyMax
1c96fdaf03 driver(docker-container): fix incorrect path when writing certs
Replace invalid characters in the name of the base directory of
certificates using the name of the registry which can contain
for example a colon when the port is specified.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-23 15:36:38 +02:00
Justin Chadwell
c77bd8a578 build: fix missing "docker" driver name in build progress
This was missing, since the driver property can only be fully populated
after loading nodes from disk. So we add logic to load the nodes, and
check for an error, which ensures that the "docker" driver is always
correctly present in the progress description.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:37:22 +01:00
Justin Chadwell
e5f701351c Merge pull request #1829 from jedevc/vendor-buildkit-master 2023-05-23 10:23:24 +01:00
Justin Chadwell
09798cdebd vendor: update buildkit to master@212ab16a39b1
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-23 10:16:09 +01:00
CrazyMax
0dfc35d558 Merge pull request #1822 from thaJeztah/vendor_engine_24.0.1
vendor: github.com/docker/docker and github.com/docker/cli v24.0.1
2023-05-23 11:11:34 +02:00
CrazyMax
8085f57a3a Merge pull request #1825 from thaJeztah/remove_k8s_replace
go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
2023-05-23 11:10:31 +02:00
Sebastiaan van Stijn
d582a21acd go.mod: update k8s deps to v0.26.2 (remove "replace" rule)
Replace rules are not inherited by consumers of buildx as a module, and as
such would default to use the v0.26.2 version. Removing the replace rules
also removes various (indirect) dependencies (although brings in some new
packages from k8s itself).

The "azure" and "gcp" authentication packages in k8s.io/go-client are now
no longer functional, so removing those imports.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 16:10:49 +02:00
CrazyMax
580820a4de Merge pull request #1824 from thaJeztah/k8s_remove_openstack
cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
2023-05-22 16:09:30 +02:00
Sebastiaan van Stijn
b7e8afc61b cmd/buildx: non-functional k8s.io/client-go/plugin/pkg/client/auth/openstack
Kubernetes v1.26 removed the in-tree openstack cloud provider. While buildx
has a replace rule to pin the k8s dependencies to an older version, such
replace rules are non-transitive, which means that consumers of buildx as
a module could potentially end up with non-resolvable dependencies.

The used code was already non-functional since c41b006be1
(buildx v0.5.0-rc1);

a6b0959276/vendor/k8s.io/client-go/plugin/pkg/client/auth/openstack/openstack_stub.go (L26-L36)

See:

- 9bbf01bae9
- 98e81a7784

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 13:01:54 +02:00
Sebastiaan van Stijn
a8a637e19d vendor: github.com/docker/cli v24.0.1
no changes in vendored files

full diff: https://github.com/docker/cli/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:16:37 +02:00
Sebastiaan van Stijn
79632a4c4c vendor: github.com/docker/docker v24.0.1
no changes in vendored files

full diff: https://github.com/moby/moby/compare/v24.0.0...v24.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-22 12:08:30 +02:00
CrazyMax
a6b0959276 Merge pull request #1440 from crazy-max/gc-policies
builder: add worker gc policies and labels
2023-05-22 11:46:46 +02:00
CrazyMax
6d7142b057 Merge pull request #1820 from jedevc/use-docker-distribution
vendor: use docker/distribution instead of distribution/distribution
2023-05-22 11:17:01 +02:00
Justin Chadwell
7e39644f69 Merge pull request #1816 from bpaquet/fix_session_token 2023-05-22 09:34:07 +01:00
Bertrand Paquet
adc6349b28 Fix AWS Authentication when mixing static creds and IAM profile
When the user supply static creds, we must not enrich them with a
session token which is unrelated.

Signed-off-by: Bertrand Paquet <bertrand.paquet@gmail.com>
2023-05-22 09:26:28 +01:00
Justin Chadwell
f558fd8b22 vendor: use docker/distribution instead of distribution/distribution
This doesn't require pulling in a new direct dependency, we already use
docker/distribution throughout imagetools and build.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-22 09:24:41 +01:00
Justin Chadwell
432e16ef70 Merge pull request #1806 from thaJeztah/update_engine2 2023-05-22 09:19:35 +01:00
Sebastiaan van Stijn
8c86c2242a vendor: github.com/docker/cli v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:47:38 +02:00
Sebastiaan van Stijn
75ad5d732b vendor: github.com/docker/docker v24.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:23 +02:00
Sebastiaan van Stijn
9bd0202312 vendor: github.com/spf13/cobra v1.7.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:44:20 +02:00
Sebastiaan van Stijn
367f114cc7 vendor: github.com/opencontainers/runc v1.1.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:43:16 +02:00
Sebastiaan van Stijn
2959ce205e vendor: github.com/klauspost/compress v1.16.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:41:29 +02:00
Sebastiaan van Stijn
75b5c6560f vendor: github.com/moby/term v0.5.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 21:39:47 +02:00
Justin Chadwell
4429ccbcc2 Merge pull request #1817 from jedevc/docs-monitor-invoke-parts 2023-05-19 16:40:47 +01:00
Justin Chadwell
c59fc18325 Merge pull request #1770 from jedevc/integration-tests 2023-05-19 16:18:31 +01:00
Justin Chadwell
4ce80856b3 docs: add info for on-error and debug-shell values for invoke
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:51 +01:00
Justin Chadwell
af3feec4ea docs: fixup typo in buildkit console output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:51:30 +01:00
Justin Chadwell
90c849f5ef bake: merge attest entries
This ensures that `target.attest=["type=sbom,<value>"]` can be
appropriately merged when `--sbom=true` or `--set
target.attest=type=sbom`.

To merge, we simply naively take the last valid value.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-19 14:32:58 +01:00
Justin Chadwell
6024212ac8 ci: add integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
2d124e0ce9 test: add basic integration tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:26:12 +01:00
Justin Chadwell
e61a8cf637 vendor: update buildkit to master@31c870e82a48
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-18 14:25:31 +01:00
Justin Chadwell
167cd16acb Merge pull request #1797 from crazy-max/fix-image-create 2023-05-18 09:06:45 +01:00
Tõnis Tiigi
1dd31fefcb Merge pull request #1810 from nicks/patch-1
fix two bugs in bake-reference.md
2023-05-17 21:12:19 -07:00
Nick Santos
5a12b25bab Update bake-reference.md
Before this change, there were two bugs:
- the HCL was not valid. in hcl, argument names can't be quoted
- the target argument should be a real target

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-05-17 19:09:42 -04:00
CrazyMax
b702188b65 imagetools(create): set correct media type when combining manifests
When using imagetools create and combining multiple sources
we should check the media type of each manifest and set
the right media type for the manifest list.

If there is a mismatch we set OCI index as best effort.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-16 11:46:58 +02:00
Justin Chadwell
060ac842bb Merge pull request #1794 from jedevc/fix-printer-start-race 2023-05-12 09:43:58 +01:00
Justin Chadwell
31d1b778ff Merge pull request #1795 from thaJeztah/vendor_distribution_v2.8.2 2023-05-12 09:43:07 +01:00
Sebastiaan van Stijn
1cd4b54810 vendor: github.com/docker/distribution v2.8.2
CI

- Dockerfile: fix filenames of artifacts

Bugfixes

-  Fix panic in inmemory driver
-  Add code to handle pagination of parts. Fixes max layer size of 10GB bug
-  Parse http forbidden as denied
-  Revert "registry/client: set Accept: identity header when getting layers

Runtime

- Update to go1.19.9
- Dockerfile: update xx to v1.2.1 ([#3907](https://github.com/distribution/distribution/pull/3907))

Security

- Fix [CVE-2022-28391](https://www.cve.org/CVERecord?id=CVE-2022-28391) by bumping alpine from 3.14 to 3.16
- Fix [CVE-2023-2253](https://www.cve.org/CVERecord?id=CVE-2023-2253) runaway allocation on /v2/_catalog [`521ea3d9`](521ea3d973)

full diff: https://github.com/docker/distribution/compare/v2.8.1...v2.8.2-beta.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-11 19:08:16 +02:00
CrazyMax
c54926c5b2 Merge pull request #1767 from jedevc/issue-template-no-bug-feature
github: don't auto-add kind/ labels
2023-05-11 17:22:23 +02:00
Justin Chadwell
10aea8e970 Merge pull request #1791 from jedevc/docs/bake-reference-matrix
Document new bake matrix builds
2023-05-11 15:43:49 +01:00
Justin Chadwell
be6542911f progress: make printer ready only after pausing logrus
This fixes a possible race where messages printed directly after calls
to NewPrinter may appear before the printer starts. With this change, we
delay all of the logs until after.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 15:23:41 +01:00
CrazyMax
9b07f6510a Merge pull request #1793 from jedevc/move-path-resolution-to-controller
controller: move path resolution into controller package
2023-05-11 15:01:21 +02:00
Justin Chadwell
9ee19520dd controller: move path resolution into controller package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:56:05 +01:00
Justin Chadwell
878faae332 docs: add matrix docs for bake
Co-authored-by: David Karlsson <david.karlsson@docker.com
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:44:28 +01:00
Justin Chadwell
eaf38570e7 github: don't auto-add kind/bug labels
kind labels should only be manually added by maintainers after
confirming the issue. Otherwise, it can be tricky to sort through all
the issues when filtering for legitimate bugs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:43:25 +01:00
CrazyMax
167340df17 Merge pull request #1780 from techvelocity/elranshefer/add-support-for-proxy-url
kubernetes driver: add support for proxy-url
2023-05-11 14:40:23 +02:00
Justin Chadwell
e61a1da7fc Merge pull request #1748 from dvdksn/docs/bake-reference
docs: move and rewrite bake reference
2023-05-11 13:00:21 +01:00
David Karlsson
f8483d7243 docs: refactor bake file reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-05-11 12:54:08 +01:00
Justin Chadwell
2c8a9aad76 Merge pull request #1782 from jedevc/docs-monitor
docs: add initial guide about debugging monitor mode
2023-05-11 11:29:21 +01:00
Justin Chadwell
32009a701c Merge pull request #1786 from jedevc/debug-multi-platform
debug: evaluate all refs to allow multi-platform debugging
2023-05-11 09:54:13 +01:00
Justin Chadwell
0cbc316f76 docs: add initial guide about debugging monitor mode
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:53:21 +01:00
Justin Chadwell
45fccef3f3 debug: evaluate all refs to allow multi-platform debugging
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 09:44:50 +01:00
Justin Chadwell
fdcb4e2fb9 Merge pull request #1784 from jedevc/support-new-compose-features
Support new compose features in bake
2023-05-11 09:44:02 +01:00
Justin Chadwell
4a0a67d7a2 Merge pull request #1783 from jedevc/fix-bake-sub-dir-context
bake: copy remote context contents to root
2023-05-11 09:40:25 +01:00
CrazyMax
855d49ff58 Merge pull request #1788 from crazy-max/update-docker
vendor: github.com/docker/docker v23.0.6
2023-05-10 17:22:14 +02:00
CrazyMax
384e873db0 Merge pull request #1790 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-05-10 16:17:53 +02:00
CrazyMax
60e72ba989 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:43:24 +02:00
CrazyMax
45a2ae6762 vendor: github.com/docker/docker v23.0.6
full diff:
- https://github.com/docker/docker/compare/v23.0.1...v23.0.6
- https://github.com/docker/cli/compare/v23.0.1...v23.0.6

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-10 14:26:20 +02:00
Justin Chadwell
2eeef180ea Merge pull request #1746 from ktock/resultcleanup
controller: avoid "context canceled" error on cleanup
2023-05-10 13:24:12 +01:00
Kohei Tokunaga
8fd81f5cfd controller: avoid "context canceled" error on cleanup
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-10 12:56:33 +01:00
Justin Chadwell
1eb9ad979e Merge pull request #1785 from jedevc/default-to-local-controller
controller: default to using local controller
2023-05-10 11:18:55 +01:00
Justin Chadwell
77e0e860f8 controller: default to using local controller
--detach shouldn't be the default yet, since it still has the potential
to leak some sessions in odd edge cases, and is slightly more painful to
debug.

For now, we should set the local controller as the default, with the
idea that we can change it back in the future.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:51:58 +01:00
Justin Chadwell
e228c398f4 bake: support compose additional contexts
compose-go v1.13.0 supports the new additional_contexts to allow passing
additional build context during build, so we should map this to bake's
contexts property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:41:37 +01:00
Justin Chadwell
5d06406f26 bake: support compose inline dockerfile
compose-go v1.13.0 supports the new dockerfile_inline to allow including
a dockerfile inline in the compose definition, so we should map this to
bake's dockerfile-inline property.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-10 09:39:00 +01:00
Justin Chadwell
cb061b684c Merge pull request #1753 from nicksieger/compose-go-v1.13.4
compose go v1.13.4
2023-05-10 09:25:10 +01:00
Elran Shefer
29b427ce13 kubernetes driver: add support for proxy-url
Signed-off-by: Elran Shefer <elran.shefer@velocity.tech>
2023-05-10 08:36:39 +03:00
Tõnis Tiigi
4fa7cd1fc2 Merge pull request #1772 from tonistiigi/container-hang-fix
container-driver: avoid hanging on container panic
2023-05-09 10:27:11 -07:00
Justin Chadwell
12b6a3ad9a bake: copy remote context contents to root
When resolving remote contexts locally in bake, then we need to ensure
that we properly unpack the contents of that context to the root
directory, instead of leaving it in the subdirectory.

Otherwise, any files will be found in the wrong location. Along with
this change, we also need a change to the dockerfile location lookup to
ensure that it is found at the root instead of in the subdirectory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:19:26 +01:00
Justin Chadwell
22e1901581 bake: reorganize input creation together
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 17:15:44 +01:00
Justin Chadwell
e23c37fa96 Merge pull request #1771 from jedevc/unset-other-build-client-opt
debug: secondary client build should not reuse solve opts
2023-05-09 14:57:23 +01:00
Justin Chadwell
e5a0ed1149 debug: secondary client build should not reuse solve opt ref
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-09 14:23:41 +01:00
Justin Chadwell
c9c1303e31 Merge pull request #1777 from ktock/fixerrormsg
monitor: improve error messages
2023-05-09 13:49:07 +01:00
Justin Chadwell
ae3299d9d4 Merge pull request #1628 from cpuguy83/policy_file
Support for passing through raw buildkit policies
2023-05-09 11:50:10 +01:00
CrazyMax
a948cc14c5 Merge pull request #1769 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.1
build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
2023-05-09 11:41:08 +02:00
CrazyMax
621b07c799 Merge pull request #1727 from crazy-max/cmd-completion
cli: add shell completion
2023-05-09 10:59:32 +02:00
Justin Chadwell
7ad970f93a Merge pull request #1776 from tonistiigi/fix-conn-leak
docker: fix connection leak on features check
2023-05-09 09:52:18 +01:00
Kohei Tokunaga
437fe55104 monitor: improve error messages
Print more understandable messages on error:

- When ps fails because the monitor doesn't attach to any session, print "no
  attaching session" instead of "unknown ref".
- Avoid disconnect silently fails when the monitor doesn't attach to any
  session. Print "no attaching session" error instead.
- Fix error message of "attach"'s arguments. ("server name must be passed" ->
  "ID of session or process must be passed")

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-05-08 15:42:37 +09:00
Tonis Tiigi
bebd244e33 docker: fix connection leak on features check
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-07 21:58:19 -07:00
Tonis Tiigi
9f2143e3df container-driver: avoid hanging on container panic
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-05-03 17:11:08 -07:00
Brian Goff
98efe7af10 Support for passing through raw buildkit policies
This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-03 23:07:44 +00:00
Justin Chadwell
c7c37c3591 Merge pull request #1760 from ktock/monitor-commands
monitor: Move commands to a separated package
2023-05-03 14:33:08 +01:00
Justin Chadwell
a43837d26c monitor: extend monitor interface from controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-03 22:08:30 +09:00
dependabot[bot]
f115abb509 build(deps): Bump peter-evans/create-pull-request from 5.0.0 to 5.0.1
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 5.0.0 to 5.0.1.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](5b4a9f6a9e...284f54f989)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-02 18:59:56 +00:00
Justin Chadwell
43a07f3997 Merge pull request #1597 from robertlestak/k8s-sa
kubernetes driver: add serviceaccount opt
2023-05-02 12:56:07 +01:00
CrazyMax
41e1693be0 Merge pull request #1756 from crazy-max/fix-bin-image
ci: fix bin-image job
2023-04-27 12:20:16 +02:00
Justin Chadwell
9d5af461b2 Merge pull request #1755 from jedevc/move-across-controller-boundary
Move metadata and subrequest output outside of controller
2023-04-27 11:15:54 +01:00
CrazyMax
b38c9c7db4 Merge pull request #1759 from crazy-max/bump-gomod
bump go.mod to 1.20
2023-04-27 09:00:31 +02:00
Kohei Tokunaga
9f884edbbf monitor: use shlex
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:38 +09:00
Kohei Tokunaga
0a7a2b1882 monitor: Move commands to a separated package
This commit moves monitor commands to `monior/commands` package.
Commands still need access to the `monitor` object and buildx controller so this
commit enables this via `Monitor` interface stored in `monitor/types`.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-27 11:36:35 +09:00
Justin Chadwell
6bec8f6e00 docs: add serviceaccount driver-opt for k8s driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:44:03 +01:00
robertlestak
65037e4611 kubernetes driver: add serviceaccount opt
Signed-off-by: robertlestak <robert.lestak@umusic.com>
2023-04-26 16:40:07 +01:00
Justin Chadwell
ba92989a94 controller: print result outside of controller
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2bf996d9ad controller: don't write metadata file in controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
75ed3e296b Merge pull request #1737 from jedevc/improved-controller-progress
controller: refactor progress api
2023-04-26 10:17:00 +01:00
CrazyMax
e14e0521cf Merge pull request #1732 from milas/builder-init-panic
builder: extra init error handling
2023-04-26 09:23:01 +02:00
Tõnis Tiigi
28e6995f7c Merge pull request #1735 from tonistiigi/localstate
build: support for saving local state by build ref
2023-04-25 15:17:08 -07:00
CrazyMax
8f72fb353c bump go.mod to 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 21:45:40 +02:00
Milas Bowman
14f5d490ef builder: extra init error handling
* Return errors from creating the `NodeGroup`
* Ensure that `b.NodeGroup != nil` before reading from
  it during validation

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-04-25 11:32:54 -04:00
CrazyMax
c9095e8eab ci: fix bin-image job
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 13:39:00 +02:00
CrazyMax
0589f69206 build: null values support with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
b724a173a9 build: fix JSON formatting for --print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
e5ccb64617 fix: changed attest type from Map to List
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
08d114195f chore: format with prettier
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:04 +02:00
David Karlsson
caf7d2ec9b build: added attest field
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
David Karlsson
2dffed3f3a build: sort bake fields alphabetically
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
Justin Chadwell
784dc2223d lint: add empty line between console and json code blocks
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 12:05:03 +02:00
CrazyMax
c3fd1e8b79 build: fix links and formatting
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
6f0c550ee9 build: move bake under the build section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
5d551dbbc1 build: customizing builds section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:03 +02:00
CrazyMax
043cb3a0db build: add bake section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-24 12:05:02 +02:00
Justin Chadwell
16d5b38f2b debug: display build warnings after each build
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-24 09:58:02 +01:00
Nick Sieger
956a1be656 vendor: update to compose-go 1.13.4
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:29:20 -05:00
Justin Chadwell
afcaa8df5f Merge pull request #1752 from nicksieger/compose-go-default-filenames
bake: use default filenames from compose-go
2023-04-21 17:28:32 +01:00
Nick Sieger
12885c01ad bake: use default filenames from compose-go
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2023-04-21 11:11:37 -05:00
Justin Chadwell
2ab8749052 controller: replace logrus status messages with progress messages
logrus info messages aren't particularly in-theme with the rest of the
progress output (and are also frustratingly racy). The progress output
is a lot neater, so we refactor it into that.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 12:33:04 +01:00
Justin Chadwell
e826141af4 controller: refactor progress api
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.

This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
  they can be displayed correctly from the client side.

Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.

To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:17:43 +01:00
Justin Chadwell
0c1fd31226 build: refactor out common build command components
We had some duplicated code between the basic runBuild and
launchControllerAndRunBuild.

This patch refactors out the common logic (since it's only really like
to keep growing), and has runBuild call into either the controller or
directly start the build depending on whether BUILDX_EXPERIMENTAL is
set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:13:58 +01:00
Justin Chadwell
0e9804901b controller: refactor status message conversion to pb package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:59:16 +01:00
Justin Chadwell
2402607846 build: use gateway's solve context to allow cancelling getResultAt
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:52:03 +01:00
CrazyMax
3d49bbd23a Merge pull request #1749 from docker/dependabot/github_actions/docker/bake-action-3
build(deps): Bump docker/bake-action from 2 to 3
2023-04-19 11:07:06 +02:00
dependabot[bot]
33b1fdbf39 build(deps): Bump docker/bake-action from 2 to 3
Bumps [docker/bake-action](https://github.com/docker/bake-action) from 2 to 3.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-18 18:10:42 +00:00
Justin Chadwell
de4cdab411 Merge pull request #1747 from jedevc/only-capture-result-for-controller
controller: only capture the result when invoked through a controller
2023-04-18 18:28:08 +01:00
Justin Chadwell
a7e471b7b3 controller: only capture the result when invoked through a controller
This ensures that the code used to capture and evaluated a result is
only executed when built through the controller. Otherwise, no build
result should be recorded.

This ensures that new code added to capture and store the build result
for debugging isn't used when BUILDX_EXPERIMENTAL is not set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 16:35:42 +01:00
Justin Chadwell
ba6e5cddb0 Merge pull request #1745 from jedevc/remote-unneccessary-exec-server-name
monitor: update check for exec args
2023-04-18 11:29:01 +01:00
Justin Chadwell
e4ff82f864 monitor: update exec failure message when no args are passed
"exec" does not take a server name, so we can rename the error check
here.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 11:16:12 +01:00
Justin Chadwell
48b733d6da Merge pull request #1740 from jedevc/resolve-paths-context-dockerfile
build: avoid resolution of dockerfile if context is remote
2023-04-18 10:54:41 +01:00
Justin Chadwell
0b432cc5f2 Merge pull request #1640 from ktock/monitor-invoke-mode-restore
monitor: add `debug-shell` and `on-error`
2023-04-18 10:41:04 +01:00
Justin Chadwell
f6cccefffc build: avoid resolution of dockerfile if context is remote
In 566f41b598, we added a check to ensure
that we avoid resolving http URLs for Dockerfile. However, we have
another circumstance we should not resolve the path in - if the context
is a remote context, the dockerfile is resolved in that context (see
build.go#LoadInputs for more information).

Therefore, we should only resolve the dockerfile to a local directory if
the context is also resolved to a local directory.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-17 11:18:49 +01:00
Kohei Tokunaga
fd5d90c699 remove unused fields from ResultContext
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
06399630a2 remove ResultContextError
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
20693aa808 Fix monitor to be aware of Inspect API and invokations from errors
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
f373b91cc3 Add flags and subcommand
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:17 +09:00
Kohei Tokunaga
ce48b1ae84 Enable to restore build options from the server
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:05 +09:00
Kohei Tokunaga
b3340cc7ba Enable to create container from error
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:36:33 +09:00
Kohei Tokunaga
1303715aba Allow passing ResultContext from server to the client through grpcerror
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:11:32 +09:00
Justin Chadwell
b716e48926 Merge pull request #1734 from jedevc/more-ssh-context-fixes
Various context fixups
2023-04-14 09:27:36 +01:00
Justin Chadwell
7d35a3b8d8 Merge pull request #1733 from jedevc/use-dockerui-context-detection
Use dockerui context detection
2023-04-14 09:25:07 +01:00
CrazyMax
200058b505 build: support for saving local state by build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-04-12 21:37:03 -07:00
Justin Chadwell
566f41b598 build: avoid file resolution of dockerfile urls
Dockerfiles can be HTTP URLs as well as local paths 🤦

We just copy the same logic we use for resolving context paths, and
apply it here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
6c0547e7e6 bake: ensure remote files behind ssh expose agent
The updateContext function may make modifications to the build inputs,
creating either an SSH URL, or an SSH llb.State. In these cases, we need
to ensure that we appropriately expose the client's default agent.

Previously, we would only expose it if the remote context was a git URL,
however, we need to also ensure that if the input was used to override
the context (in the case of ReadRemoteFiles), that we expose the agent
here as well.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 12:11:20 +01:00
Justin Chadwell
871f865ac8 bake: update ReadRemoteFiles to use buildkit api
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 11:15:25 +01:00
Justin Chadwell
62a21520ea vendor: update buildkit to master@333ee9158128
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-12 10:11:46 +01:00
CrazyMax
a597266a52 cli: add completion to list builder names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:45:59 +02:00
CrazyMax
14b66817fb build: add completion to list context folders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:35:50 +02:00
CrazyMax
af011d6ca3 bake: add completion to list targets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:34:40 +02:00
CrazyMax
8a02cf8717 cli: set default completion
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:32:48 +02:00
CrazyMax
672eeed9a6 Merge pull request #1731 from crazy-max/update-moby-ver
driver: update BuildKit version constraint for docker driver
2023-04-11 11:26:32 +02:00
CrazyMax
1b816ff838 driver: update BuildKit version constraint for docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-11 11:10:01 +02:00
CrazyMax
10365ddf22 Merge pull request #1723 from docker/dependabot/github_actions/peter-evans/create-pull-request-5.0.0
build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
2023-04-11 09:39:17 +02:00
dependabot[bot]
a28cb1491d build(deps): Bump peter-evans/create-pull-request from 4.2.4 to 5.0.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.4 to 5.0.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](38e0b6e68b...5b4a9f6a9e)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-04-05 18:59:48 +00:00
Justin Chadwell
1e149bb84f Merge pull request #1722 from jedevc/bake-matrix-check-map-type
bake: fix map type checking to detect matrix property set to list
2023-04-05 16:03:50 +01:00
Justin Chadwell
9827abbf76 bake: fix map type checking to detect matrix property set to list
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 13:41:03 +01:00
Justin Chadwell
a3293cdaaa Merge pull request #1720 from jedevc/bake-replace-func
Add `replace` function to bake stdlib
2023-04-05 11:59:40 +01:00
Justin Chadwell
f7d8bd2055 Merge pull request #1690 from jedevc/bake-matrix
Implement matrix for bake targets
2023-04-05 11:58:59 +01:00
Justin Chadwell
5d33a3af22 bake: add replace func to stdlib
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
Justin Chadwell
87f900ce77 chore: sort bake stdlib map
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-05 11:48:35 +01:00
CrazyMax
bb5c93cafc Merge pull request #1719 from dvdksn/docs/update-readme-links
docs: prune links in README toc, add external reference
2023-04-05 11:30:29 +02:00
David Karlsson
c6ce0964b9 docs: prune links in README toc, add external reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-04-05 11:00:06 +02:00
Justin Chadwell
5c21e80a83 Merge pull request #1711 from jedevc/bake-remote-file-git
bake: forward default ssh agent for remote bake file
2023-04-04 11:50:42 +01:00
CrazyMax
498cc9ba0a Merge pull request #1715 from crazy-max/ci-restrict-repo
ci: check repo origin on push tag event
2023-04-03 21:16:51 +02:00
CrazyMax
805f3a199d ci: check repo origin on push tag event
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-03 20:58:37 +02:00
Justin Chadwell
91fdb0423d Merge pull request #1713 from ktock/monitor-buildapi-options
controller: Extract nested CommonOptions on controller API
2023-04-03 14:48:06 +01:00
Kohei Tokunaga
8ba8659496 controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Justin Chadwell
16e41ba297 Merge pull request #1710 from jedevc/use-buildkit-gitutil-parsegitref
Use buildkit's gitutil package to detect remote git files
2023-04-03 11:14:08 +01:00
Justin Chadwell
387ce5be7c bake: forward default ssh agent for remote bake file
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:50:43 +01:00
Justin Chadwell
87a120e8e3 bake: use build.IsRemoteURL
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:48:31 +01:00
Justin Chadwell
589d4e4cf5 build: use buildkit's gitutil package to detect remote files
BuildKit's gitutil package behaves slightly differently than moby's
urlutil, so we should rely on BuildKit's gitutil when detecting URLs to
avoid cases of accidentally producing invalid build requests that can
confuse users.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 10:23:59 +01:00
CrazyMax
6535f16aec Merge pull request #1705 from docker/dependabot/go_modules/github.com/opencontainers/runc-1.1.5
build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
2023-04-03 10:53:26 +02:00
Justin Chadwell
a1520ea1b2 bake: additional validation for matrixes
This adds the following constraints to the new features:
- Explicit renaming with the `name` property is *only* permitted when
  used with the `matrix` property.
- Group does not support either `name` or `matrix` (we may choose to
  relax this constraint over time).
- All generated names must be unique.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-03 09:23:28 +01:00
Tõnis Tiigi
0844213897 Merge pull request #1707 from crazy-max/fix-ls-docs
docs: update ls command output sample
2023-03-30 19:33:55 -07:00
CrazyMax
989ba55d9a docs: update ls command output sample
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 17:54:23 +02:00
CrazyMax
33388d6ede Merge pull request #1552 from crazy-max/moby-buildkit-version
driver: resolve BuildKit version for docker driver
2023-03-30 17:31:47 +02:00
dependabot[bot]
bfadbecb96 build(deps): Bump github.com/opencontainers/runc from 1.1.4 to 1.1.5
Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-30 15:25:26 +00:00
CrazyMax
f815f4acf7 Merge pull request #1703 from jedevc/vendor-buildkit-master
Bump BuildKit to latest master
2023-03-30 17:23:38 +02:00
CrazyMax
81d7decd13 driver: resolve BuildKit version for docker driver
Buildx currently returns the Docker Engine version for the docker driver
and it can be confusing. This is because BuildKit before 0.11 version does
not support the Info API that returns the BuildKit version.

This change resolves the BuildKit version from the Docker engine one.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-30 15:52:29 +02:00
Justin Chadwell
d699d08399 bake: add additional tests for matrix behavior
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:39:38 +01:00
Justin Chadwell
9541457c54 vendor: update buildkit to master@8b7bcb900d3c
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:35:49 +01:00
Justin Chadwell
c6cdcb02cf build: replace removed buildkit BUILDINFO field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-30 10:24:46 +01:00
Justin Chadwell
799715ea24 Merge pull request #1701 from jedevc/fixup-resolve-paths-context-path
build: fixup resolvePaths for remote context path
2023-03-29 13:59:36 +01:00
Justin Chadwell
b5c6b3f10b build: fixup resolvePaths for remote context path
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-29 10:10:20 +01:00
Tõnis Tiigi
3f59b27cf4 Merge pull request #1698 from crazy-max/fix-issue-template
chore: fix issue template
2023-03-27 09:10:56 -07:00
CrazyMax
00b18558dd chore: fix issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-27 13:30:33 +02:00
Tõnis Tiigi
948414e1b2 Merge pull request #1693 from jedevc/remote-tls-missing-key-cert
remote: avoid tls error if both key and cert are not set
2023-03-23 10:58:12 -07:00
Justin Chadwell
56876ab825 remote: avoid tls error if both key and cert are not set
Previously, we would explicitly error if all TLS parameters were not
available. However, it is a perfectly valid use case to connect to a
buildkit server that only provides TLS in one direction to verify the
server (which is possible today with buildctl).

To support this use case, we only need to error if only one of key or
cert is set, and the other is not - if both are unspecified, the client
will not present a certificate to the server.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-23 10:43:10 +00:00
Justin Chadwell
0806870261 bake: generate implicit groups for matrixes
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-22 17:07:04 +00:00
Tõnis Tiigi
fd8eaab2df Merge pull request #1675 from jedevc/source-date-epoch-fixes
SOURCE_DATE_EPOCH build arg injection fixes
2023-03-21 22:29:24 -07:00
Justin Chadwell
77252f161c bake: add matrix to target block
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-21 12:05:14 +00:00
Justin Chadwell
4437802e63 bake: allow overriding name property
Previously, the name property could not be set in the body of a bake
target and could only be set for a label. This patch allows the body to
override the values of label fields, though the default is still the
label.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 17:55:28 +00:00
Justin Chadwell
1613fde55c bake: allow interception to create derived contexts
This patch allows high level clients to define an EvalContext method
which can derive a new context given a block and the base parent
context.

This allows users of the package to intercept evaluation before it
begins, and define additional variables and functions that are bound to
a single block.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-20 12:18:36 +00:00
CrazyMax
624bc064d8 Merge pull request #1677 from crazy-max/bump-k8s
vendor: bump k8s to v0.25.4
2023-03-19 17:05:55 +00:00
CrazyMax
0c4a68555e Merge pull request #1682 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.4
build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
2023-03-16 09:50:44 +00:00
dependabot[bot]
476ac18d2c build(deps): Bump peter-evans/create-pull-request from 4.2.3 to 4.2.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.3 to 4.2.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](2b011faafd...38e0b6e68b)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-15 19:00:53 +00:00
Justin Chadwell
780531425b bake: move SOURCE_DATE_EPOCH parsing to overrides
Previously, when directly modifying the args map when reading targets,
we could end up in a scenario where bake tests that compare arg maps
would fail if SOURCE_DATE_EPOCH was set in the environment.

This patch prevents this failure by setting the SOURCE_DATE_EPOCH at the
command level (which isn't injected into tests as well), ensuring that
we test correctly even when SOURCE_DATE_EPOCH is set in the environment.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-15 10:06:53 +00:00
CrazyMax
92d2dc8263 k8s: corev1.Handler has been replaced by corev1.ProbeHandler
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
CrazyMax
cfa6b4f7c8 vendor: bump k8s to v0.25.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-14 18:44:13 +01:00
Justin Chadwell
5d4223e4f8 build: move SOURCE_DATE_EPOCH parsing into option generation
This allows the build package code to become more generic, and also
ensures that when the environment variables are not propogated (in the
case of the remote controller), that we can still correctly set
SOURCE_DATE_EPOCH.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-14 10:21:32 +00:00
Tõnis Tiigi
4a73abfd64 Merge pull request #1673 from crazy-max/fix-k8s-deploy-name
k8s: generate node name if not provided
2023-03-13 17:20:51 -07:00
Justin Chadwell
6f722da04d Merge pull request #1670 from jedevc/version-v-prefix
version: add "v" prefix to version for tagging convention consistency
2023-03-13 08:00:46 +00:00
CrazyMax
527d57540e k8s: generate node name if not provided
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-11 21:22:10 +01:00
Justin Chadwell
b65f49622e version: add "v" prefix to version for tagging convention consistency
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-08 16:25:00 +00:00
Tõnis Tiigi
c5ce08bf3c Merge pull request #1626 from ktock/monitor-exec
monitor: Enable to exec into the container
2023-03-07 17:46:22 +00:00
Tõnis Tiigi
71b35ae42e Merge pull request #1663 from crazy-max/stripcreds
build: strip credentials from remote url on collecting Git provenance info
2023-03-06 16:26:09 +00:00
CrazyMax
15eb6418e8 build: strip credentials from remote url on collecting Git provenance info
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 17:00:05 +01:00
Justin Chadwell
2a83723d57 Merge pull request #1581 from ktock/resolvepath
monitor: resolve paths arguments in client
2023-03-02 11:09:34 +00:00
Kohei Tokunaga
e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:30:29 +09:00
Justin Chadwell
b5ea989eee monitor: resolve oci layout path in client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-03-02 10:28:56 +00:00
Kohei Tokunaga
17105bfc50 monitor: resolve paths arguments in client
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:13:45 +09:00
CrazyMax
eefe27ff42 Merge pull request #1658 from crazy-max/remove-guides
docs: guides moved to docs website
2023-03-01 10:40:12 +01:00
CrazyMax
1ea71e358a docs: guides moved to docs website
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-01 10:21:02 +01:00
Tõnis Tiigi
14d8f95ec9 Merge pull request #1657 from ktock/argpath
controller: use os.Executable() for getting the exceutable path
2023-03-01 00:12:36 -08:00
Kohei Tokunaga
b0728c96d3 controller: use os.Executable() for getting the exceutable path
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-01 15:49:07 +09:00
Tõnis Tiigi
5e685c0e04 Merge pull request #1650 from crazy-max/dirty-optin
build: makes git dirty check opt-in
2023-02-28 10:49:17 -08:00
CrazyMax
f2ac30f431 Merge pull request #1651 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-02-24 15:43:32 +01:00
CrazyMax
6808c0e585 ci: update docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:13:10 +01:00
CrazyMax
9de12bb9c8 docs: do not generate ref for internal serve command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 15:12:15 +01:00
CrazyMax
0645acfd79 build: makes git dirty check opt-in
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 13:19:33 +01:00
CrazyMax
439d58ddbd Merge pull request #1037 from crazy-max/bug-template
chore: github issue template
2023-02-24 13:06:55 +01:00
CrazyMax
c0a9274d64 chore: github issue template
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-24 12:33:24 +01:00
Justin Chadwell
f3a4cd5176 Merge pull request #1614 from jedevc/typed-controller-api
controller: strongly type the controller api
2023-02-24 09:44:45 +00:00
Justin Chadwell
c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
0b8f0264b0 controller: move image id file writing to client
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
5c31d855fd controller: return solve response through api
Now clients can access the result of the solve, specifically the image
id output. This is a useful refactor, as well as being required if we
want to allow bake to invoke through the controller api.

This also allows us to remove the quiet option from the API, since we
can compute the required progress type outside of the controller, and
can print the image id from the result of the solve.

As a follow-up, we should also be able to remove the image id file
output from the controller api, now that the client has access to it.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00
Justin Chadwell
90d7fb5e77 controller: strongly type the controller api
Strongly typing the API allows us to perform all command line parsing
fully on the client-side, where we have access to the client local
directory and all the client environment variables, which may not be
available on the remote server.

Additionally, the controller api starts to look a lot like
build.Options, so at some point in the future there may be an
oppportunity to merge the two, which would allow both build and bake to
execute through the controller, instead of needing to maintain multiple
code paths.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:43:15 +00:00
Justin Chadwell
c4ad930e2a Merge pull request #1644 from crazy-max/update-frontend
Dockerfile: align frontend version
2023-02-23 11:00:13 +00:00
CrazyMax
3d0c88695e Merge pull request #1646 from docker/dependabot/go_modules/golang.org/x/net-0.7.0
build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
2023-02-23 11:36:36 +01:00
dependabot[bot]
7332140fdf build(deps): Bump golang.org/x/net from 0.5.0 to 0.7.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-02-22 19:50:31 +00:00
CrazyMax
132fababb0 Dockerfile: align frontend version
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 17:47:48 +01:00
CrazyMax
71507c0b58 Merge pull request #1643 from crazy-max/go-1.20
update to go 1.20
2023-02-22 17:42:59 +01:00
CrazyMax
7888fdee58 fix lint issue
should drop = nil from declaration of var n; it is the zero value

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:57:29 +01:00
CrazyMax
fb61fde581 fix cannot set environment variables in parallel tests
> FAIL: TestReadTargets/ArgsOverrides/leaf (0.00s)
> panic: testing: t.Setenv called after t.Parallel; cannot set environment variables in parallel tests

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:56:46 +01:00
CrazyMax
5258e44030 remove deprecated golangci-lint linters
Linters 'deadcode' and 'varcheck' are deprecated since v1.49.0
and had been replaced by 'unused'

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:40:14 +01:00
CrazyMax
e16c1b289b update golangci-lint to v1.51.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:38:17 +01:00
CrazyMax
376b73f078 update to go 1.20
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-22 14:37:50 +01:00
Justin Chadwell
1c6060f27d Merge pull request #1612 from cpuguy83/env_no_provenance
Add env var to disable default attestations
2023-02-22 11:36:35 +00:00
Justin Chadwell
ed4fd965ff Merge pull request #1620 from jedevc/remote-controller-fixes
Improvements for remote controller code
2023-02-21 10:52:58 +00:00
Brian Goff
bc9cb2c66a Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-02-16 17:25:40 +00:00
CrazyMax
aa05f4c207 Merge pull request #1584 from dvdksn/docs/build-platform-local
docs: added --platform=local example
2023-02-16 18:08:25 +01:00
CrazyMax
62fbef22d0 Merge pull request #1177 from crazy-max/build-display-builder
build: print instance being used
2023-02-16 18:07:30 +01:00
David Karlsson
2563685d27 docs: added --platform=local example
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-02-16 17:24:45 +01:00
CrazyMax
598f1f0a62 build: print instance being used
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
8311b0963a vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-16 14:56:46 +01:00
CrazyMax
b1949b7388 Merge pull request #1634 from dgageot/fix-1633
build: fix git ambiguous argument
2023-02-16 14:05:32 +01:00
CrazyMax
3341bd1740 Merge pull request #1610 from thaJeztah/update_docker
vendor: github.com/docker/docker, github.com/docker/cli v23.0.0
2023-02-16 13:43:12 +01:00
Justin Chadwell
74f64f88a7 Merge pull request #1617 from jedevc/process-multiple-attestation-annotations
imagetools: process `com.docker.reference.*` annotations
2023-02-16 12:05:20 +00:00
David Gageot
d4a4aaf509 Remove git warning: buildx/1633
Signed-off-by: David Gageot <david.gageot@docker.com>
2023-02-16 07:36:24 +01:00
Justin Chadwell
1f73f4fd5d Merge pull request #1630 from jedevc/debug-help-message-typo
monitor: fix typo in help message
2023-02-16 00:06:06 +00:00
Justin Chadwell
77f83d4171 monitor: fix typo in help message
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-15 14:48:56 +00:00
Justin Chadwell
642f28f439 imagetools: process com.docker.reference.* annotations
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.

Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:24:40 +00:00
Justin Chadwell
54f4dc8f6e controller: set absolute path of server binary before execution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-14 11:21:18 +00:00
CrazyMax
89d99b1694 inspect: display labels and gc policies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:56 +01:00
CrazyMax
9753f63f57 builder: add worker gc policies and labels
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-11 22:56:15 +01:00
CrazyMax
04804ff355 Merge pull request #1557 from crazy-max/gomod-1.19
go.mod: bump to Go 1.19
2023-02-11 22:47:51 +01:00
Justin Chadwell
ed9ea2476d controller: use unique files per buildx version
This ensures that we should never accidentally connect to a server with
a mismatched version, while also allowing us to run multiple buildx
servers at a time.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:23 +00:00
Justin Chadwell
d0d29168a5 controller: use grpc with contexts for improved timeouts
This patch improves timeout logic for testing and creating a buildx
server. Instead of needing to have a custom loop, we can just use the
primitives provided by go and grpc for easier handling.

Additionally, without the explicit time.Sleeps, we defer to GRPCs
exponential backoff algorithm which should provide substantially better
results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:19 +00:00
Justin Chadwell
abda257763 controller: exit cleanly on SIGTERM
This signal may be sent using an external tool such as pkill, and since
we can handle it neatly, we should.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:41:15 +00:00
Justin Chadwell
1b91bc2e02 controller: add more informative server exit messages
When exiting, we should ideally always print a message, and give details
as to exactly what error we received.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:26 +00:00
Justin Chadwell
56b9e785e5 build: don't kill remote controller after build
We don't know if other builds might be running, etc, so we should allow
the server to decide when to exit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-10 14:40:25 +00:00
Sebastiaan van Stijn
081447c9b1 vendor: github.com/docker/cli v23.0.0
full diff: https://github.com/docker/cli/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:32:57 +01:00
Sebastiaan van Stijn
260117289b vendor: github.com/docker/docker v23.0.0
full diff: https://github.com/docker/docker/compare/v23.0.0-rc.1...v23.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:30:17 +01:00
Sebastiaan van Stijn
73dca749ca vendor: golang.org/x/oauth2 v0.1.0
no changes in vendored code

full diff: https://github.com/golang/oauth2/compare/6fdb5e3db783...v0.1.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:26:17 +01:00
Sebastiaan van Stijn
8ac380bfb3 vendor: golang.org/x/net v0.5.0
full diff: https://github.com/golang/net/compare/v0.4.0...v0.5.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:21:05 +01:00
Sebastiaan van Stijn
aeac7e08f9 vendor: golang.org/x/term v0.4.0
no changes in vendored code

full diff: https://github.com/golang/term/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:20:00 +01:00
Sebastiaan van Stijn
7c9cdc4353 vendor: golang.org/x/text v0.6.0
no changes in vendored code

full diff: https://github.com/golang/text/compare/v0.5.0...v0.6.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:19:53 +01:00
Sebastiaan van Stijn
67572785cf vendor: golang.org/x/sys v0.4.0
full diff: https://github.com/golang/sys/compare/v0.3.0...v0.4.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:12:32 +01:00
Sebastiaan van Stijn
8a70e7634d vendor: github.com/google/certificate-transparency-go v1.1.4
full diff:

- https://github.com/google/certificate-transparency-go/compare/v1.0.21...v1.1.4
- https://github.com/kubernetes/klog/compare/v2.30.0...v2.80.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 15:03:26 +01:00
Sebastiaan van Stijn
6dd5589a9c vendor: github.com/containerd/containerd v1.6.16
full diff: https://github.com/containerd/containerd/compare/1709cfe273d9...v1.6.16

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-08 14:55:44 +01:00
CrazyMax
78058ce5f3 Merge pull request #1592 from crazy-max/fix-build-git
build: check reachable git commits
2023-02-08 14:23:52 +01:00
CrazyMax
fd5884189c build: check reachable git commits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-08 12:46:04 +01:00
Justin Chadwell
ab7a9f008d Merge pull request #1607 from jedevc/bake-fix-error-traces
bake: avoid nesting error diagnostics
2023-02-07 13:20:07 +00:00
CrazyMax
a8eb2a7fbe gitutil: override the locale to ensure consistent output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-07 13:26:28 +01:00
Justin Chadwell
fbb4f4dec8 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-07 11:40:20 +00:00
Justin Chadwell
46fd0a61ba Merge pull request #1605 from jedevc/bake-fix-errors
bake: avoid early-exit for resolution failures
2023-02-07 10:36:05 +00:00
Justin Chadwell
6444c813dc Merge pull request #1601 from jedevc/buildx-document-attestation-booleans
build: add docs for boolean attestation flags
2023-02-06 15:15:01 +00:00
Justin Chadwell
dc8a2b0398 bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 15:05:45 +00:00
CrazyMax
d9780e27cd Merge pull request #1602 from crazy-max/rm-releases-json
ci: remove releases-json workflow
2023-02-06 13:44:57 +01:00
CrazyMax
ab44d03771 ci: remove releases-json workflow
This is now directly handled by our actions-toolkit. We keep
`.github/releases.json` file for now but will be removed on
next v0.11 release.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
crazy-max
b53cb256e5 github: update .github/releases.json
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-06 13:37:00 +01:00
CrazyMax
c3075923f4 Merge pull request #1591 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.9.0
2023-02-06 13:28:24 +01:00
CrazyMax
a32881313b Merge pull request #1596 from crazy-max/builder-fix-validation
builder: fix docker context not validated
2023-02-06 12:31:04 +01:00
Justin Chadwell
07548bc898 build: add docs for boolean attestation flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-06 10:53:49 +00:00
CrazyMax
0e544fe835 builder: fix docker context not validated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 21:23:29 +01:00
CrazyMax
21ac4c34fb update github.com/compose-spec/compose-go to v1.9.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-02-03 03:19:59 +01:00
Justin Chadwell
d2fa4a5724 Merge pull request #1582 from jedevc/lint-fixups
Linting tidy-ups
2023-02-01 16:08:07 +00:00
Justin Chadwell
4bdf98cf20 lint: ban fmt.Errorf in preference of errors.Errorf
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:31:01 +00:00
Justin Chadwell
5da09f0c23 lint: enable gosec
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:50 +00:00
Justin Chadwell
48357ee0c6 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-01 10:13:43 +00:00
Justin Chadwell
6506166f02 Merge pull request #1579 from jedevc/refactor-controllers
Refactor controllers, moving them into a separate package
2023-02-01 09:57:32 +00:00
Justin Chadwell
5f130b25ad hack: move syntax directive to top of generated-files
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
a9fd128910 controller: move controllers out of commands into separate package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-31 17:51:33 +00:00
Justin Chadwell
cb94298a02 Merge pull request #1296 from ktock/monitor-list
monitor: Enable to run build and invoke in background
2023-01-31 14:14:57 +00:00
CrazyMax
046084c0b8 Merge pull request #1574 from crazy-max/ci-fix-workflow
ci: fix pr branch name in releases-json workflow
2023-01-31 15:00:01 +01:00
CrazyMax
18760253b9 ci: fix pr branch name in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 13:51:30 +01:00
CrazyMax
ded6376ece Merge pull request #1573 from crazy-max/fix-releases-json
ci: add workflow_dispatch in releases-json workflow
2023-01-31 13:48:33 +01:00
CrazyMax
a4d60a451d ci: add workflow_dispatch in releases-json workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-31 11:38:46 +01:00
Justin Chadwell
0f4030de5d Merge pull request #1572 from crazy-max/ci-update-ver
ci: update buildx and buildkit to latest
2023-01-31 10:22:58 +00:00
CrazyMax
f1a5a3ec50 ci: update buildx and buildkit to latest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 22:13:27 +01:00
CrazyMax
87beaefbb8 Merge pull request #1571 from crazy-max/fix-docs-upstream
ci: fix docs-upstream workflow
2023-01-30 21:18:54 +01:00
CrazyMax
451847183d ci: fix docs-upstream workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 21:01:27 +01:00
CrazyMax
7625a3a4b0 Merge pull request #1568 from crazy-max/silent-git-no-upstream
build: silently fail if git remote not found
2023-01-30 20:32:12 +01:00
CrazyMax
6db696748b build: better message output for git provenance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 20:24:09 +01:00
Tõnis Tiigi
14f9ae679d Merge pull request #1566 from tonistiigi/fix-result-descriptor
build: fix multi-node merge to read descriptor from result
2023-01-30 09:04:40 -08:00
CrazyMax
4789d2219c build: silently fail if git remote not found
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 14:33:16 +01:00
CrazyMax
eacecf657c Merge pull request #1561 from crazy-max/fix-preferred-platform
build: fix preferred platform not taken into account
2023-01-30 12:22:08 +01:00
CrazyMax
1de0be240f Merge pull request #1563 from crazy-max/releases-json
ci: generate releases.json on release event
2023-01-30 11:51:08 +01:00
CrazyMax
ea4bec2bad github: update .github/releases.json
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
CrazyMax
36d95bd3b9 ci: generate releases.json on release event
We are currently using the GitHub API in our setup-buildx-action
to check for latest and tagged releases to make sure they exist
before download. But this requires using a token to avoid
rate-limit. It's fine for public runners but GHES runners don't
have the `github.token` populated automatically. They need to
create a PAT.

This PR will solve this issue by generating and pushing a
`releases.json` file in this repo when we publish a GitHub Release
that will then be fetched through `raw.githubusercontent.com`
endpoint on `setup-buildx-action` repo. This endpoint is better
served for our purpose with 5000 requests per hour compared to the
GitHub API endpoint that is limited to 60 requests per hour (unauth)
and 1000 request per hour when authenticated.

Also ignore .github/releases.json file on pull request event as an
action in a workflow run can't trigger a new workflow run anyway.
See https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#example-using-more-than-one-event

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-30 11:46:47 +01:00
Tonis Tiigi
c33b310b48 build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-29 22:52:41 -08:00
Justin Chadwell
8af76c68a8 Merge pull request #1564 from AkihiroSuda/fix-1562
bake: SOURCE_DATE_EPOCH: fix `panic: assignment to entry in nil map`
2023-01-29 22:07:20 +00:00
Akihiro Suda
1f56f51740 bake: SOURCE_DATE_EPOCH: fix panic: assignment to entry in nil map
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-29 08:43:06 +09:00
CrazyMax
49b3c0dba5 build: fix preferred platform not taken account
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-27 20:02:13 +01:00
Tõnis Tiigi
a718d07f64 Merge pull request #1559 from tonistiigi/update-buildkit-v0.11.2
vendor: update buildkit to v0.11.2
2023-01-26 10:57:21 -08:00
Tonis Tiigi
f6da7ee135 vendor: update buildkit to v0.11.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-26 10:33:20 -08:00
CrazyMax
7eb266de69 go.mod: bump to Go 1.19
Go 1.20 will be there soon, I think it's time to move our go.mod to
latest stable. We can then remove the compat in our vendor.Dockerfile

Downstream projects like compose or github.com/linuxkit/linuxkit should
not be affected.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-26 16:08:09 +01:00
CrazyMax
9f821dabeb Merge pull request #1548 from crazy-max/git-ls-remote
build: set remote origin url
2023-01-26 11:03:18 +01:00
Kohei Tokunaga
a27b8395b1 Enable to run build and invoke in background
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-01-26 17:17:07 +09:00
Tõnis Tiigi
b1b4e64c97 Merge pull request #1504 from jedevc/fix-1497
Refactor BuildWithResultHandler to simplify concurrency usage
2023-01-25 10:52:17 -08:00
CrazyMax
c1058c17aa build: set remote origin url
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-25 18:15:05 +01:00
Tõnis Tiigi
059c347fc2 Merge pull request #1549 from jedevc/silence-intoto-warnings
imagetools: silence intoto warnings
2023-01-24 17:03:49 -08:00
Justin Chadwell
7145e021f9 imagetools: silence intoto warnings
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 16:49:28 +00:00
CrazyMax
9723f4f76c Merge pull request #1547 from crazy-max/fix-rm
rm: do not check for context builders when removing inactive
2023-01-24 13:39:36 +01:00
Justin Chadwell
db72d0cc05 Merge pull request #1543 from jedevc/bump-ci-buildkit
ci: update buildkit release version in build pipeline
2023-01-24 12:17:34 +00:00
Justin Chadwell
00b7d5b858 Merge pull request #1505 from jedevc/inspect-lazy-attestations
Lazily load attestation data in `imagetools inspect`
2023-01-24 12:10:11 +00:00
CrazyMax
6cd0c11ab1 rm: do not check for context builders when removing inactive
This change has been introduced in e7b5ee7518
but we should not check context builders when removing inactive
ones.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-24 13:08:18 +01:00
Justin Chadwell
c1ab55a3f2 ci: update buildkit release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 11:47:58 +00:00
Justin Chadwell
c756e3ba96 Merge pull request #1534 from jedevc/readme-minimum-docker-version
docs: add note about minimum docker version
2023-01-24 11:02:21 +00:00
Justin Chadwell
566f37b65b docs: add note about minimum docker version
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-24 10:25:58 +00:00
CrazyMax
6d1ff27410 Merge pull request #1538 from tonistiigi/caps-req-internal
build: mark capabilities request as internal
2023-01-24 11:12:37 +01:00
Tonis Tiigi
be55b41427 build: mark capabilities request as internal
So it doesn't show up in the History API.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-24 00:48:35 -08:00
Tõnis Tiigi
a4f01b41a4 Merge pull request #1529 from thaJeztah/update_buildkit
vendor: github.com/moby/buildkit v0.11.1
2023-01-18 12:56:26 -08:00
Sebastiaan van Stijn
01e1c28dd9 vendor: github.com/moby/buildkit v0.11.1
full diff: https://github.com/moby/buildkit/compare/v0.11.0...v0.11.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-18 20:56:54 +01:00
Tõnis Tiigi
51e41b16db Merge pull request #1511 from jedevc/error-on-attestations-docker
build: error when using docker exporter and attestations
2023-01-17 09:57:48 -08:00
Justin Chadwell
9e9cdc2e6d Merge pull request #1525 from jedevc/ci-bump-buildx
ci: update buildx release version in build pipeline
2023-01-17 16:39:39 +00:00
Justin Chadwell
bc1d590ca7 ci: update buildx release version in build pipeline
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 15:21:52 +00:00
Justin Chadwell
900d9c294d Merge pull request #1523 from jedevc/docs-reference-attest
docs: add reference for new attest family of flags
2023-01-17 13:47:10 +00:00
CrazyMax
65aac16139 Merge pull request #1512 from nicks/patch-1
docs: small tweak to opentelemetry
2023-01-17 13:30:33 +01:00
Justin Chadwell
4903f462f6 docs: add reference for new attest family of flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-17 11:45:38 +00:00
Justin Chadwell
44b5a19c13 Merge pull request #1516 from dvdksn/docs/fix-broken-bake-link-in-cli-reference
docs: fix broken link in buildx_bake CLI reference
2023-01-17 11:11:54 +00:00
David Karlsson
ba8fa6c403 docs: fix broken link in buildx_bake CLI reference
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-01-13 15:33:31 +01:00
CrazyMax
5b3083e9e1 Merge pull request #1515 from crazy-max/fix-docs-release
ci: fix typo in docs-release workflow
2023-01-13 15:16:16 +01:00
CrazyMax
523a16aa35 ci: fix typo in docs-release workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-13 15:10:03 +01:00
Justin Chadwell
43a748fd15 build: error when using docker exporter and attestations
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:41:53 +00:00
Justin Chadwell
15a80b56b5 build: avoid compatability error when attestations disabled
We should avoid erroring with attestations support compatability errors
when a user has specified --provenance=false.

A user may wish to enable --provenance=false that works across buildkit
versions, but currently it will fail on old versions - this patch fixes
this, to silently ignore the provenance flag for this check if it's set
to disabled.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-13 13:36:55 +00:00
Nick Santos
b14bfb9fa2 docs: small tweak to opentelemetry
The buildx builder persists across Desktop restarts. So when we start up Jaeger, we should recommend that people configure it to persist across restarts as well.

Signed-off-by: Nick Santos <nick.santos@docker.com>
2023-01-11 12:09:05 -05:00
Justin Chadwell
56950ece69 inspect: lazily load attestation data
Delay loading the attestation data immediately, and only compute it upon
request. We do this using a deferred function which allows to define the
computation in the same place as before, but perform the computation
later.

With this patch, we ensure that the attestation data is only pulled from
the remote if it is actually referenced in the format string -
otherwise, we can skip it, for improved performance.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 13:00:42 +00:00
Justin Chadwell
1d2ac78443 inspect: move attestation loading to struct methods
This refactor ensures that the attestations are not output in the JSON
output for "{{ json . }}", and additionally allows future refactors to
dynamically load the attestation contents, ensuring faster performance
when attestations are not used in the output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 12:40:42 +00:00
Justin Chadwell
8b7aa1a168 build: create error group per opt
Using the syncronization primitive, we can avoid needing to create a
separate wait group.

This allows us to sidestep the issue where the wait group could be
completed, but the build invocation functions had not terminated - if
one of the functions was to terminate with an error, then it was
possible to encounter a race condition, where the result handling code
would begin executing, despite an error.

The refactor to use a separate error group which more elegantly handles
the concept of function returns and errors, ensures that we can't
encounter this issue.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 11:02:27 +00:00
Justin Chadwell
1180d919f5 build: reorder error group funcs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
347417ee12 build: use copy for BuildWithResultHandler loop vars
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-10 10:50:15 +00:00
Justin Chadwell
fb27e3f919 Merge pull request #1502 from developer-guy/fix/readme
fix the directory of the buildx binary
2023-01-10 10:28:04 +00:00
Batuhan Apaydın
edb16f8aab fix the directory of the buildx binary
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2023-01-10 11:26:11 +03:00
Tõnis Tiigi
5c56e947fe Merge pull request #1500 from tonistiigi/update-buildkit-v0.11.0
github: update CI to buildkit v0.11
2023-01-09 15:59:25 -08:00
Tonis Tiigi
571871b084 github: update CI to buildkit v0.11
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-01-09 15:50:56 -08:00
Tõnis Tiigi
8340c40647 Merge pull request #1498 from jedevc/attestation-printing
Improved attestation inspect
2023-01-09 11:22:20 -08:00
Justin Chadwell
9818055b0e docs: update with new inspect output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:07:10 +00:00
Justin Chadwell
484823c97d inspect: change additional spdxs to not have duplicates
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 19:03:23 +00:00
Justin Chadwell
3ce17b01dc inspect: provide access to multiple spdx documents
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:42:26 +00:00
Justin Chadwell
e68c566c1c inspect: parse sbom and provenance into json structs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:43 +00:00
Justin Chadwell
19d16aa941 inspect: break after first matching attestation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-09 18:09:06 +00:00
CrazyMax
6852713121 Merge pull request #1494 from thaJeztah/docs_update_docs_tools
go.mod: update cli-docs-tool v0.5.1
2023-01-09 13:14:01 +01:00
Sebastiaan van Stijn
c97500b117 go.mod: update cli-docs-tool v0.5.1 and re-generate docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:03:01 +01:00
CrazyMax
85040a9067 Merge pull request #1493 from thaJeztah/docs_fix_anchors
docs: update anchor links
2023-01-09 12:19:58 +01:00
Sebastiaan van Stijn
b8285c17e6 docs: update anchor links
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-07 15:12:08 +01:00
Tõnis Tiigi
332dfb4b92 Merge pull request #1444 from crazy-max/inspect-attest
imagetools inspect: handle provenance and sboms
2023-01-06 16:29:33 -08:00
Tõnis Tiigi
cb279bb14b Merge pull request #1491 from jedevc/ociindex-refactor
vendor: update buildkit to v0.11.0-rc4
2023-01-06 16:29:18 -08:00
Justin Chadwell
60c9cf74ce vendor: update buildkit to v0.11.0-rc4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-06 15:51:17 -08:00
Tõnis Tiigi
ff6754eb04 Merge pull request #1456 from jedevc/oci-layout-reference-parsing
build: refactor reference parsing for oci image layouts
2023-01-05 23:50:25 -08:00
CrazyMax
e6b9aba997 imagetools inspect: handle provenance and sbom
use stub structs for SLSA/SBOM while waiting for
go-imageinspect library to be public.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 17:34:30 +01:00
CrazyMax
0302894bfb Merge pull request #1463 from crazy-max/e2e-k3s
e2e: use native k3s installation script
2023-01-05 16:50:45 +01:00
CrazyMax
e46394c3be e2e: use native k3s installation script
debianmaster/actions-k3s action gives some warnings in our e2e
workflow. This commit brings https://github.com/debianmaster/actions-k3s/blob/master/index.js
directly in the workflow through actions/github-script with
some changes to properly wait for nodes to be up.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 15:37:19 +01:00
Justin Chadwell
1885e41789 docs: update oci layout with tag resolution
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-01-05 13:47:42 +00:00
CrazyMax
2fb9db994b imagetools inspect: missing annotations key
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-04 15:52:16 +01:00
Tõnis Tiigi
287aaf1696 Merge pull request #1482 from AkihiroSuda/auto-propagate-source-date-epoch
Propagate SOURCE_DATE_EPOCH from the client env
2022-12-27 17:32:08 -08:00
Akihiro Suda
0e6f5a155e Propagate SOURCE_DATE_EPOCH from the client env
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-12-24 01:32:15 +09:00
Tõnis Tiigi
88852e2330 Merge pull request #1480 from crazy-max/fix-badge
disable buildx experimental in pipeline
2022-12-16 10:28:20 -08:00
CrazyMax
6369c50614 disable buildx experimental in pipeline
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-16 18:53:46 +01:00
CrazyMax
a22d0a35a4 readme: fix status badge
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-16 17:52:30 +01:00
CrazyMax
c93c02df85 Merge pull request #1479 from jedevc/fixup-git-err-check-order
build: check error from toSolveOpt before adding FrontendAttrs
2022-12-16 16:28:17 +01:00
Justin Chadwell
e584c6e1a7 build: check error from toSolveOpt before adding FrontendAttrs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-16 12:19:33 +00:00
Tõnis Tiigi
64e4c19971 Merge pull request #1477 from crazy-max/git-wsl
build: lookup the right git binary on WSL
2022-12-15 18:00:52 -08:00
Tonis Tiigi
551b8f6785 git: do not show warnings if project does not use git
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-15 17:51:46 -08:00
Tõnis Tiigi
fbbe1c1b91 Merge pull request #1472 from crazy-max/ci-attest
ci: opt-in sbom and provenance
2022-12-15 17:38:13 -08:00
Tonis Tiigi
1a85745bf1 github: update buildkit image to v0.11-rc3
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-15 16:54:32 -08:00
CrazyMax
0d1fea8134 build: warn if git operation fails
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 23:40:19 +01:00
CrazyMax
19417e76e7 build: lookup the right git binary on WSL
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 21:16:37 +01:00
Tõnis Tiigi
53d88a79ef Merge pull request #1475 from jedevc/attest-warn-duplicate
buildflags: error on duplicate attest field
2022-12-15 11:07:47 -08:00
Justin Chadwell
4c21b7e680 Merge pull request #1476 from jedevc/dont-filter-attestation-opts
build: forward all build opts everywhere
2022-12-15 19:05:39 +00:00
Justin Chadwell
a8f689c223 build: forward all build opts everywhere
All build options should be passed everywhere - the frontend and the
backend of buildkit should both be able to see all attestations, as well
as all other opts: e.g. epoch settings, and no-cache.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-15 18:19:47 +00:00
CrazyMax
ba8e3f9bc5 ci: generate provenance and sbom for bin image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 19:12:26 +01:00
CrazyMax
477200d1f9 ci: generate provenance and sbom for release binaries
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 19:12:25 +01:00
Tõnis Tiigi
662738a7e5 Merge pull request #1474 from crazy-max/fix-ci
ci: fix warnings
2022-12-15 09:29:25 -08:00
Justin Chadwell
f992b77535 buildflags: warn on duplicate attest field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-15 15:39:19 +00:00
CrazyMax
21b2f135b5 ci: update to ubuntu 22.04
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:34:03 +01:00
CrazyMax
71e6be5d99 ci: fix deprecated set-output syntax
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:24:02 +01:00
CrazyMax
df8e7d0a9a Merge pull request #1473 from crazy-max/fix-docs-prerelease
ci: do not publish docs on prerelease
2022-12-15 14:22:07 +01:00
CrazyMax
64422a48d9 Merge pull request #1471 from crazy-max/fix-builder-factory
builder: check endpoint from store when loading factory
2022-12-15 14:19:20 +01:00
CrazyMax
04f9c62772 ci: do not publish docs on prerelease
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 14:16:45 +01:00
CrazyMax
2185d07f05 builder: check endpoint from store when loading factory
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-15 13:32:54 +01:00
CrazyMax
a49d28e00e Merge pull request #1460 from crazy-max/docker-load
e2e: load to docker store
2022-12-15 10:35:29 +01:00
Tõnis Tiigi
629128c497 Merge pull request #1449 from crazy-max/bake-var-null
bake: support null arg and label value
2022-12-14 21:03:05 -08:00
Justin Chadwell
70682b043e build: refactor reference parsing for image layouts
We allow any valid image reference format for the oci-layout, not just
limiting to name@digest, we additionally allow images of the form
name:tag@digest now.

The name of the reference is used to find the local directory to lookup
the store in, while the tag and digest are attached to a random identity
to generate the dummy reference sent to the oci-layout context.

This separation of the target to replace and the value to replace it
with ensures that any tag or digest set in the client is properly sent
across to the server. The tag is used when a digest was not specified,
and it is resolved in the context of the local directory before being
sent, using the same helpers as we use for the local cache expoter.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-14 16:53:10 +00:00
Justin Chadwell
b741d26eb5 Merge pull request #1468 from crazy-max/fix-git-vcs-check
build: fix env vars check for vcs details
2022-12-14 12:51:29 +00:00
CrazyMax
cf8fa4a404 build: fix env vars check for vcs details
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 09:48:55 +01:00
CrazyMax
fe76a1b179 bake: support null label value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 08:30:58 +01:00
CrazyMax
df4957307f bake: support null arg value
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 08:29:27 +01:00
Tõnis Tiigi
e21f56e801 Merge pull request #1434 from jedevc/resource-interpolation
Resource interpolation support
2022-12-13 18:06:51 -08:00
Justin Chadwell
e51b55e03c bake: add tests for block interpolation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:39 -08:00
Justin Chadwell
296b8249cb bake: support block-based interpolation
This patch adds support for block-based interpolation, so that
properties of blocks can be referenced in the current block and across
other blocks.

Previously, order-of-evaluation did not matter for blocks, and could be
evaluated in any order. However, now that blocks can refer to each
other, we split out this dynamic evaluation order into a separate
resolveBlock function.

Additionally, we need to support partial block evaluations - if block A
refers to property X of block B, when we should only evaluate property
X, and not the entire block. This ensures that we can safely evaluate
blocks that refer to other properties within themselves, and allows
sequences that would otherwise be co-recursive. We take special care in
this logic to ensure that each property is evaluated once *and only*
once - this could otherwise present inconsistencies with stateful
functions, and could risk inconsistent results.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:39 -08:00
Justin Chadwell
7c6b840199 bake: add cty tags to hcl structures
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 17:27:36 -08:00
Tõnis Tiigi
2a6ff4cbfc Merge pull request #1462 from crazy-max/attest-vcs
build: set provenance vcs details
2022-12-13 17:23:20 -08:00
CrazyMax
6ad5e2fcf3 build: set provenance vcs details
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-14 00:09:38 +01:00
Tõnis Tiigi
37811320ef Merge pull request #1439 from crazy-max/last-activity
store: set nodegroup last activity
2022-12-13 15:00:47 -08:00
CrazyMax
99ac7f5f9e e2e: load to docker store
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-13 23:32:58 +01:00
Tõnis Tiigi
96aca741a2 Merge pull request #1467 from tonistiigi/update-buildkit-862b22
vendor: update buildkit to 862b22d7
2022-12-13 12:07:42 -08:00
Tonis Tiigi
12ec931237 vendor: update buildkit to 862b22d7
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-12-13 12:00:06 -08:00
Justin Chadwell
0e293a4ec9 Merge pull request #1464 from jedevc/vendor-buildkit-93b40706a007
vendor: update buildkit to 93b40706a007
2022-12-13 11:12:54 +00:00
Justin Chadwell
163712a23b vendor: update buildkit to 93b40706a007
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-13 11:03:53 +00:00
Tõnis Tiigi
5f4d463780 Merge pull request #1435 from thaJeztah/bump_buildkit_v0.11
vendor: github.com/moby/buildkit v0.11.0-rc1.0.20221207183946-5993b526de65
2022-12-08 19:51:10 -08:00
Tõnis Tiigi
abc8121aa8 Merge pull request #1457 from jedevc/add-hosts-length-check
build: don't set add-hosts option if empty
2022-12-08 19:48:09 -08:00
CrazyMax
8c47277141 store: set nodegroup last activity
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-08 19:49:51 +01:00
CrazyMax
36b5cd18e8 store: use constants for directory names
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-08 19:35:55 +01:00
Justin Chadwell
1e72e32ec3 Merge pull request #1412 from jedevc/attestations-cli
Attestations from buildx
2022-12-08 17:30:31 +00:00
Sebastiaan van Stijn
8e5e5a563d vendor: github.com/moby/buildkit v0.11.0-rc1.0.20221207183946-5993b526de65
- drops the replace-rule for github.com/aws/aws-sdk-go-v2/config (as it no longer was replacing anything)
- drops the replace-rules for docker/cli and docker/docker (at least as long as we continue using tagged releases)
- removes the github.com/docker/docker/pkg/stringid package (which was redundant)

full diff: 9624ab4710..5993b526de

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-07 23:27:29 +01:00
Justin Chadwell
98049e7eda build: don't set add-hosts option if empty
This looks like an oversight, all of the other options have similar
checks. This can interfere with generated provenance where "add-hosts"
will be marked as an argument to the build, even though it's not
actually being utilized.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 19:16:01 +00:00
Justin Chadwell
25aa893bad bake: add attests field
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:44:21 +00:00
Justin Chadwell
b270a20274 build: add attests flag and sbom/provenance shorthands
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:44:11 +00:00
Justin Chadwell
f0262dd10e build: add attestations to build options
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 18:43:36 +00:00
Justin Chadwell
f8b673eccd build: pass attestation attributes to build request
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 16:10:54 +00:00
Justin Chadwell
0c0c9a0030 chore: sort buildOptions alphabetically
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-12-07 16:10:54 +00:00
CrazyMax
d1f79317cf Merge pull request #1455 from crazy-max/nodegroup-exclude-field
store: skip DockerContext field from being saved
2022-12-07 16:07:30 +01:00
CrazyMax
fa58522242 store: skip DockerContext field from being saved
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-07 13:15:18 +01:00
CrazyMax
aa6fd3d888 Merge pull request #1454 from crazy-max/builder-fix-default
builder: fix default docker context behavior
2022-12-07 11:03:06 +01:00
CrazyMax
ebdd8834a9 builder: fix default docker context behavior
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-07 08:52:12 +01:00
CrazyMax
fe8d5627e0 Merge pull request #1433 from crazy-max/makefile-build-opts
hack: mutualize build opts in Makefile and Dockerfile
2022-12-07 05:04:44 +01:00
CrazyMax
b242e3280b Merge pull request #1430 from crazy-max/builder-pkg
Refactor builder and drivers info logic
2022-12-06 12:41:10 +01:00
CrazyMax
cc01caaecb builder: enhance driver factory logic when loading drivers
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
e7b5ee7518 mutualize builder logic
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
63073b65c0 dockerutil pkg to manage docker api client and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-06 12:13:41 +01:00
CrazyMax
47cf72b8ba Merge pull request #1451 from crazy-max/update-buildkit
vendor: update buildkit to master@9624ab4
2022-12-05 17:30:39 +01:00
CrazyMax
af24d72dd8 kubernetes: fix context load test
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 17:13:03 +01:00
CrazyMax
f451b455c4 vendor: update buildkit to master@9624ab4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 17:03:47 +01:00
Justin Chadwell
16f4dfafb1 Merge pull request #1450 from crazy-max/fix-hclparser-jsonfunc
hcl: SrcRange not checked when solving JSON func calls
2022-12-05 15:17:23 +00:00
CrazyMax
5b4e8b9d71 hcl: SrcRange not checked when solving JSON func calls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-05 15:05:17 +01:00
CrazyMax
b06eaffeeb Merge pull request #1442 from crazy-max/hcl-fix-panic
bake: fix panic for unsupported hcl variable type
2022-12-05 14:30:04 +01:00
CrazyMax
3d55540db1 Merge pull request #1445 from dvdksn/docs/move-manuals
docs: moved manual pages to docs repo, added link
2022-12-03 15:41:01 +01:00
Tõnis Tiigi
3c2b9aab96 Merge pull request #1446 from crazy-max/moby-host-gateway
build: skip "host-gateway" validation with moby driver
2022-12-02 20:24:40 -08:00
CrazyMax
49d46e71de build: skip "host-gateway" validation with moby driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-02 13:53:13 +01:00
David Karlsson
6c5168e1ec docs: moved manual pages to docs repo, added link
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-12-02 09:49:54 +01:00
Tõnis Tiigi
e91d5326fe Merge pull request #1441 from crazy-max/fix-nil-ref
imagetools: set default repo ref on creation if nil
2022-12-01 09:51:09 -05:00
CrazyMax
48b573e835 bake: fix panic for unsupported hcl variable type
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-12-01 00:52:05 +01:00
CrazyMax
4788eb24ab imagetools: set default repo ref on creation if nil
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-30 23:18:43 +01:00
CrazyMax
3ed2783f34 Merge pull request #1436 from thaJeztah/protobuf_extensions_fix
go.mod: golang_protobuf_extensions v1.0.4 - prevent incompat versions
2022-11-30 14:11:33 +01:00
Sebastiaan van Stijn
c0e8a41a6f go.mod: golang_protobuf_extensions v1.0.4 - prevent incompat versions
This module made a whoopsie, and updated to `google.golang.org/protobuf`
in a patch release, but `google.golang.org/protobuf` is not backward
compatible with `github.com/golang/protobuf`.

Updating the minimum version to v1.0.4 which corrects this, to prevent
users of buildx as a module from accidentally pulling in the wrong
version:

- v1.0.3 switched to use `google.golang.org/protobuf`; https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.2..v1.0.3
- This was reverted in v1.0.4 (which is the same as v1.0.2); https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.3..v1.0.4
- And a `v2` was created instead; https://github.com/matttproud/golang_protobuf_extensions/releases/tag/v2.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-30 13:31:42 +01:00
CrazyMax
23b217af24 hack: mutualize build opts in Makefile and Dockerfile
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-29 17:05:25 +01:00
CrazyMax
3dab19f933 Merge pull request #1432 from crazy-max/hack-hash-files
hack: improve hash-files script
2022-11-29 13:28:56 +01:00
CrazyMax
05efb6291f hack: improve hash-files script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-29 12:36:34 +01:00
CrazyMax
eba49fdefd Merge pull request #1255 from crazy-max/docker-api
docker api: use helper to parse context docker endpoint metadata
2022-11-29 12:16:52 +01:00
CrazyMax
29f2c49374 Merge pull request #1268 from crazy-max/hack-output
hack: use single output dir
2022-11-29 12:16:38 +01:00
CrazyMax
2245371696 Merge pull request #1420 from jedevc/oci-tar
Support new `tar` option for oci+docker exporters
2022-11-29 11:48:10 +01:00
CrazyMax
74631d5808 Merge pull request #1431 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.3
build(deps): Bump peter-evans/create-pull-request from 4.2.2 to 4.2.3
2022-11-28 20:11:01 +01:00
dependabot[bot]
9264b0ca09 build(deps): Bump peter-evans/create-pull-request from 4.2.2 to 4.2.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.2 to 4.2.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](331d02c7e2...2b011faafd)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-28 18:13:29 +00:00
CrazyMax
a96fb92939 Merge pull request #1429 from crazy-max/update-gh-release
ci: update softprops/action-gh-release to v0.1.15
2022-11-28 14:04:02 +01:00
CrazyMax
ae59e1f72e Merge pull request #1305 from jedevc/progress-group-prefixed-writer
progress: add prefix to vertex progress group
2022-11-28 13:32:27 +01:00
CrazyMax
47167a4e6f ci: update softprops/action-gh-release to v0.1.15
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-28 13:29:13 +01:00
CrazyMax
23cabd67fb Merge pull request #1427 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.2
build(deps): Bump peter-evans/create-pull-request from 4.2.0 to 4.2.2
2022-11-25 11:27:43 +01:00
CrazyMax
e66410b932 Merge pull request #1313 from jedevc/bake-group-recurse
bake: recursively resolve groups
2022-11-25 11:27:09 +01:00
dependabot[bot]
c3bba05770 build(deps): Bump peter-evans/create-pull-request from 4.2.0 to 4.2.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.2.0 to 4.2.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](b4d51739f9...331d02c7e2)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-11-24 18:07:15 +00:00
Justin Chadwell
69b91f2760 docs: add tar flag to oci+docker exporters
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
e6b09580b4 build: support tar flag for oci+docker exporters
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
36e663edda vendor: update buildkit to master@ae9d0f5
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-23 11:35:38 +00:00
Justin Chadwell
60e2029e70 Merge pull request #1419 from jedevc/docs-clarify-support
docs: clarify support for various sets of shared options
2022-11-23 10:25:14 +00:00
Justin Chadwell
5e1db43e34 docs: clarify support for various sets of shared options
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-22 11:10:50 +00:00
CrazyMax
6e9b743296 Merge pull request #1417 from accetto/patch-2
Correction in Synopsis.
2022-11-21 09:40:21 +01:00
accetto
ef9710d8e2 Correction in Synopsis.
Correction in Synopsis.

Signed-off-by: accetto <34798830+accetto@users.noreply.github.com>
2022-11-20 13:34:52 +00:00
CrazyMax
468b3b9c8c Merge pull request #1407 from AkihiroSuda/x-crypto-ssh
go.mod: golang.org/x/crypto v0.1.0 (Fix `ssh: parse error in message type 27` with OpenSSH >= 8.9)
2022-11-17 13:27:05 +01:00
CrazyMax
0d8c853917 Merge pull request #1394 from thaJeztah/update_cobra
go.mod: github.com/spf13/cobra v1.6.1
2022-11-17 13:22:29 +01:00
CrazyMax
df3b868fe7 Merge pull request #1414 from crazy-max/move-k8s-config
kubernetes: move context config logic to its own pkg
2022-11-17 11:22:40 +01:00
CrazyMax
3f6a5ab6ba kubernetes: move context config logic to its own pkg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-17 00:34:42 +01:00
Tõnis Tiigi
aa1f4389b1 Merge pull request #1396 from crazy-max/fix-indent
use double spaces with json marshal indent
2022-11-16 14:54:13 -08:00
Justin Chadwell
246cd2aee9 Merge pull request #1411 from jedevc/docker-container-volume-docs
docs: add cache persistence notes for docker-container driver
2022-11-16 12:29:01 +00:00
Justin Chadwell
0b6f8149d1 docs: add cache persistence notes for docker-container driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-16 11:56:42 +00:00
Akihiro Suda
4dda2ad58b go.mod: golang.org/x/crypto v0.1.0
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-11-16 07:43:29 +09:00
Justin Chadwell
15bb14fcf9 Merge pull request #1406 from felixdesouza/fds/fix-concurrent-map-write
Synchronise access to the map when printing.
2022-11-15 15:46:54 +00:00
Felix de Souza
b68114375f Address feedback
Signed-off-by: Felix de Souza <fdesouza@palantir.com>
2022-11-14 18:40:05 +00:00
Felix de Souza
83a09b3cf2 Synchronise access to the map when printing.
Signed-off-by: Felix de Souza <fdesouza@palantir.com>
2022-11-14 15:47:32 +00:00
Justin Chadwell
3690cb12e6 Merge pull request #1397 from markhildreth-gravity/patch-1
Correct documentation on gha scopes
2022-11-10 13:39:11 +00:00
Justin Chadwell
b4de4826c4 Merge pull request #1403 from jedevc/docs-fixups
Docs consistency fixes
2022-11-10 13:29:41 +00:00
Justin Chadwell
b06df637c7 docs: use consistent "Type" header across parameter tables
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-10 10:44:17 +00:00
Justin Chadwell
9bb9ae43f9 docs: add env parameter to docker-container driver
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-10 10:44:17 +00:00
Mark Hildreth
35e7172b89 Correct documentation on gha scopes
I believe the documentation is misleading in saying that specifying a scope in a gha cache declaration will `ensure each branch gets its own cache`. I believe this should say that each build or each image will get its own cache.

Signed-off-by: Mark Hildreth <mark.hildreth@gravityclimate.com>
2022-11-07 11:06:07 -05:00
CrazyMax
abebf4d955 use double spaces with json marshal indent
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-07 16:36:24 +01:00
Sebastiaan van Stijn
1c826d253b go.mod: github.com/spf13/cobra v1.6.1
full diff: https://github.com/spf13/cobra/compare/v1.5.0...v1.6.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-11-06 19:03:52 +01:00
CrazyMax
d1b454232d Merge pull request #1392 from dvdksn/refactor-docs
docs: refactored file and directory structure
2022-11-05 10:07:10 +01:00
David Karlsson
be3b41acc6 docs: refactored file and directory structure
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-11-04 19:57:50 +01:00
Justin Chadwell
2a3e51ebfe Merge pull request #1390 from dvdksn/update-docs-default-branch
docs: update git ref to docs repo
2022-11-04 16:54:44 +00:00
David Karlsson
1382fda1c9 docs: update git ref to docs repo
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-11-04 16:37:46 +01:00
Justin Chadwell
c658096c17 Merge pull request #1385 from jedevc/exporter-docs
Add exporter docs
2022-11-04 15:16:30 +00:00
Justin Chadwell
6097919958 docs: change cache storage links to exporter to docs site
Temporary fix while we update docs upstream.

Co-Authored-By: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-04 14:57:17 +00:00
Tõnis Tiigi
330bdde0a3 Merge pull request #1383 from jedevc/with-tracer-delegate-panic
driver: don't create tracer delegate opt if tracer is nil
2022-11-02 10:58:58 -07:00
CrazyMax
a55404fa2e Merge pull request #1388 from crazy-max/fix-docs-validate
ci(docs-upstream): fix ref on pull request event
2022-11-02 15:22:29 +01:00
CrazyMax
c8c7c9f376 Merge pull request #1387 from crazy-max/update-compose-go
update github.com/compose-spec/compose-go to v1.6.0
2022-11-02 15:22:12 +01:00
CrazyMax
df34c1ce45 ci(docs-upstream): switch to reusable workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-01 19:45:00 +01:00
CrazyMax
da1d66c938 update github.com/compose-spec/compose-go to v1.6.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-01 16:14:56 +01:00
Justin Chadwell
d32926a7e5 docs: add exporter docs
This patch adds more complete documentation for the various exporters
available to buildx.

Co-Authored-By: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-11-01 12:16:45 +00:00
Justin Chadwell
7f008a7d1e driver: don't create tracer delegate opt if tracer is nil
The error handling for the cast to client.TracerDelegate was incorrect,
and previously, a client would unconditionally append an opt.

This results in the scenario that while the ClientOpt was not nil, the
tracer delegate in the ClientOpt was, which isn't an error case
explicitly handled by buildkit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-31 11:19:47 +00:00
Tõnis Tiigi
eab3f704f5 Merge pull request #1371 from jedevc/progress-tty-fail
progress: explicitly fail if tty requested but not available
2022-10-25 11:07:50 -07:00
Justin Chadwell
a50e89c38e progress: explicitly fail if tty requested but not available
The NewPrinter function is mostly borrowed from buildkit. However, at
some point, it seems that the implementations drifted.

This patch updates buildx to be more similar in behavior to it's
buildkit counterpart, specifically, it will explicitly fail if a TTY
output is requested using "--progress=tty", but the output is not
available.

To gracefully fallback to plain progress in this scenario,
"--progress=plain" is required.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-25 10:55:36 +01:00
CrazyMax
85723a138f hack: lint scripts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-24 08:30:35 +02:00
CrazyMax
9c69ba6f6f hack: use single output dir
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-24 08:30:35 +02:00
CrazyMax
e84ed65525 Merge pull request #1356 from thaJeztah/protobuf_extensions_tag
go.mod: matttproud/golang_protobuf_extensions v1.0.2 (use tag)
2022-10-24 08:15:35 +02:00
CrazyMax
4060abd3aa Merge pull request #1363 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.2.0
build(deps): Bump peter-evans/create-pull-request from 4.1.4 to 4.2.0
2022-10-24 08:15:17 +02:00
CrazyMax
c924a0428d Merge pull request #1368 from jedevc/userns-to-driver
Userns to driver
2022-10-24 08:15:01 +02:00
Justin Chadwell
33ef1b3a30 docker-container: move userns detection into driver
This moves the detection of the docker daemon's security options into
the driver from the factory, handling them in a similar way to how we do
cgroups.

Because of recent changes that modify error detection in driver
creation, this attempt to contact the docker daemon during builder
creation meant that a docker-container builder could not be created
without access to the docker socket. This patch resolves this, by
defering the Info call to the driver, when the container is actually
created.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:44:44 +01:00
Justin Chadwell
a6caf4b948 chore: tidy up duplicate dockertypes import
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-21 13:40:17 +01:00
Justin Chadwell
cc7e11da99 Merge pull request #1366 from jedevc/remove-structcheck
lint: remove structcheck
2022-10-20 10:22:50 +01:00
Justin Chadwell
a4c3efe783 lint: add nolintlint and fix violations
We should be able to detect nolint comments that point to linters that
are disabled (such as with the removed structcheck).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-20 09:49:51 +01:00
Justin Chadwell
4e22846e95 lint: remove structcheck
structcheck is abandoned, and has been replaced by the unused linter.
See https://golangci-lint.run/usage/linters/#structcheck for more
information.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-19 14:19:06 +01:00
dependabot[bot]
ddbd0cd095 build(deps): Bump peter-evans/create-pull-request from 4.1.4 to 4.2.0
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.4 to 4.2.0.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](ad43dccb4d...b4d51739f9)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-18 18:56:21 +00:00
Justin Chadwell
255a3ec82c Merge pull request #1361 from dvdksn/docs-fix-cache-codeblocks
docs: add user sign for console blocks
2022-10-18 11:57:53 +01:00
David Karlsson
167c77baec docs: add user sign for console blocks
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-18 12:50:14 +02:00
CrazyMax
ca2718366e Merge pull request #1000 from tonistiigi/docs-named-context
docs: make sure syntax with latest stable dockerfile image is defined
2022-10-18 10:08:06 +02:00
CrazyMax
58d3a643b9 Merge pull request #1345 from crazy-max/bake-print
bake: do not fail printing definition if instance unavailable
2022-10-18 10:05:14 +02:00
CrazyMax
718b8085fa Merge pull request #1354 from dvdksn/fix-cache-cmds
docs: fix cache command examples
2022-10-17 22:32:29 +02:00
CrazyMax
64930d7440 Merge pull request #1359 from jedevc/cache-inline-typo
docs: fix typo in cache docs
2022-10-17 22:31:27 +02:00
CrazyMax
4d2f948869 Merge pull request #1360 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.4
build(deps): Bump peter-evans/create-pull-request from 4.1.3 to 4.1.4
2022-10-17 22:30:54 +02:00
dependabot[bot]
19c224cbe1 build(deps): Bump peter-evans/create-pull-request from 4.1.3 to 4.1.4
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](671dc9c9e0...ad43dccb4d)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-10-17 18:34:07 +00:00
Justin Chadwell
efd1581c01 docs: fix typo in cache docs
We should keep consistency with the rest of the list, and call it
"inline" instead of "inline-cache".

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-10-17 17:06:41 +01:00
Tõnis Tiigi
ac85f590ba Merge pull request #1324 from jedevc/invoke-defaults
invoke: load defaults from image config
2022-10-13 09:01:29 -07:00
Sebastiaan van Stijn
b0d3162875 go.mod: matttproud/golang_protobuf_extensions v1.0.2 (use tag)
it's the same commit: https://github.com/matttproud/golang_protobuf_extensions/compare/v1.0.2...c182affec369

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-13 16:12:19 +02:00
David Karlsson
4715a7e9e1 docs: fix cache command examples
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-12 17:15:55 +02:00
CrazyMax
c5aec243c9 Merge pull request #1353 from dvdksn/absolute-links-part2
docs: replaced broken relative links
2022-10-11 23:16:10 +02:00
David Karlsson
c76f3d3dba docs: replaced broken relative links
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-11 21:35:52 +02:00
Justin Chadwell
7add6e48b6 Merge pull request #1352 from dvdksn/remove-cache-opt-link
remove/update links to satisfy htmlproofer
2022-10-11 13:17:26 +01:00
David Karlsson
1267e0c076 changed from relative to absolute links
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-11 13:53:06 +02:00
Justin Chadwell
361c093a35 Merge pull request #1332 from dvdksn/refactor-cache-docs
added tables for cache parameters
2022-10-11 12:19:35 +01:00
CrazyMax
9ad39a29f7 Merge pull request #1334 from dvdksn/driver-docs-review
Driver docs update
2022-10-07 16:17:18 +02:00
David Karlsson
f5a1d8bff9 moved context arg to last
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-06 14:04:54 +02:00
CrazyMax
8c86afbd57 Merge pull request #1310 from AkihiroSuda/gcos-rootless
kubernetes: rootless: support Google Container-Optimized OS  (Fix ` Options:[rbind ro]}]: operation not permitted` errors)
2022-10-06 11:05:19 +02:00
David Karlsson
4d6e36df99 review comments, align style with driver docs
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:33:57 +02:00
David Karlsson
f51884e893 address review comments
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:18:42 +02:00
David Karlsson
4afd9ecf16 fix incorrect heading text
Co-authored-by: Justin Chadwell <github@jedevc.com>
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:55 +02:00
David Karlsson
ed3b311de4 improve wording on default buildkit image
Co-authored-by: Justin Chadwell <github@jedevc.com>
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:50 +02:00
David Karlsson
d030fcc076 updated prose and structure for driver docs
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-10-05 20:10:27 +02:00
CrazyMax
398da1f916 Merge pull request #1343 from dgageot/fix-1342
[1342] Fix assignment to nil map
2022-10-04 17:31:06 +02:00
CrazyMax
3a5741f534 Merge pull request #1341 from dgageot/fix-1340
[1340] Disable git labels if `BUILDX_GIT_LABELS` is not `1` or `full`
2022-10-04 17:04:35 +02:00
CrazyMax
c53b0b8a12 bake: do not fail printing definition if instance unavailable
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-10-04 15:38:45 +02:00
David Gageot
8fd34669ed Fix assignment to nil map
Signed-off-by: David Gageot <david.gageot@docker.com>
2022-10-04 08:30:47 +02:00
David Gageot
be7e91899b Support empty env var when it can't be unset
Signed-off-by: David Gageot <david.gageot@docker.com>
2022-10-03 18:36:06 +02:00
David Karlsson
74a822568e added tables for cache parameters
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2022-09-29 21:40:47 +02:00
CrazyMax
105c214d15 Merge pull request #1333 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.3
build(deps): Bump peter-evans/create-pull-request from 4.1.2 to 4.1.3
2022-09-29 17:25:43 +02:00
dependabot[bot]
2b6a51ed34 build(deps): Bump peter-evans/create-pull-request from 4.1.2 to 4.1.3
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](171dd555b9...671dc9c9e0)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-28 18:46:49 +00:00
CrazyMax
e98c252490 Merge pull request #1327 from docker/dependabot/github_actions/peter-evans/create-pull-request-4.1.2
build(deps): Bump peter-evans/create-pull-request from 4.0.4 to 4.1.2
2022-09-22 14:08:32 +02:00
CrazyMax
17f5d6309f Merge pull request #1329 from jedevc/ignore-workflows-on-docs-changes
ci: don't trigger build and e2e workflows on docs-only pull requests
2022-09-22 14:08:09 +02:00
CrazyMax
6a46ea04ab Merge pull request #1316 from jedevc/cache-docs
docs: add cache storage backend docs
2022-09-22 14:07:06 +02:00
CrazyMax
7bd97f6717 Merge pull request #1318 from crazy-max/new-docs-links
docs: update links to new docs
2022-09-22 14:06:47 +02:00
CrazyMax
2a9c98ae40 remove unrelated comment after upgrade
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-22 13:53:33 +02:00
Justin Chadwell
1adf80c613 ci: don't trigger build and e2e workflows on docs-only pull requests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 11:00:55 +01:00
Justin Chadwell
f823d3c73c docs: final cache storage fixups
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:52:13 +01:00
David Karlsson
91f0ed3fc3 docs: additions from editorial review
- editorial review
- address review comments, rework param sections
- added a common section for parameters
- remove liquid tags for notes

Signed-off-by: David Karlsson <david.karlsson@docker.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:51:16 +01:00
Justin Chadwell
04b56c7331 docs: add cache storage backend docs
This fleshes out the documentation to include information along with
examples and detailed use cases for the cache backends. The general
format and style follows from the style of the build driver docs.

Eventually, these docs will be included on docs.docker.com along with
the rest of the build docs.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-22 10:51:16 +01:00
dependabot[bot]
3c1a20097f build(deps): Bump peter-evans/create-pull-request from 4.0.4 to 4.1.2
Bumps [peter-evans/create-pull-request](https://github.com/peter-evans/create-pull-request) from 4.0.4 to 4.1.2.
- [Release notes](https://github.com/peter-evans/create-pull-request/releases)
- [Commits](923ad837f1...171dd555b9)

---
updated-dependencies:
- dependency-name: peter-evans/create-pull-request
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-09-21 18:48:36 +00:00
Justin Chadwell
966c4d4e14 invoke: load defaults from image config
If user does not specify image certain container parameters, we can load
them from the exporter metadata.

Additionally, we introduce a new "default" value for the --invoke flag,
that keeps all of the default parameters (since cobra does not have an
easy way of accepting an optional flag argument).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-21 11:43:52 +01:00
CrazyMax
6b8289d68e docs: update links to new docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-14 13:36:51 +02:00
CrazyMax
294421db9c Merge pull request #1317 from crazy-max/fix-link
docs: fix link in driver guide
2022-09-14 11:33:20 +02:00
CrazyMax
9fdf991c27 docs: update links in driver guide
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-13 22:11:47 +02:00
Justin Chadwell
77b33260f8 bake: recursively resolve groups
Groups that contained other groups were not recursively resolved by
ReadTargets, which prevented output from --print from being useable as a
self-contained bake file.

This patch ensures that all groups that are referenced inside the bake
file are actually defined under the groups field. This has required a
substantial refactor, as previously only a single group was returned
from ReadTargets, notably, returning a map of groups, instead of a
slice.

This does introduce a small behavior change to the behavior of --print -
while previously, passing a group name to bake would return all the
targets of that group back as the default group, now only the name of
that group will be inserted into the default group, keeping the original
group intact. The impact of this can be observed in some of the changes
to the bake_test.go file.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-12 13:51:34 +01:00
Akihiro Suda
33e5f47c6c kubernetes: rootless: support Google Container-Optimized OS
Tested with GKE Autopilot 1.24.3-gke.200 (kernel 5.10.123+, containerd 1.6.6).

ref: moby/buildkit PR 3097

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-09-09 17:16:35 +09:00
CrazyMax
25ceb90678 Merge pull request #1294 from crazy-max/s3-cache
s3 cache client-side support
2022-09-08 14:26:19 +02:00
CrazyMax
27e29055cb Merge pull request #1307 from zhyon404/master
return di.Err when it's not nil
2022-09-08 14:20:14 +02:00
Tõnis Tiigi
810ce31f4b Merge pull request #1297 from cdupuis/git-revision
Add git provenance labels
2022-09-06 10:23:50 -07:00
Christian Dupuis
e3c91c9d29 Add git provenance labels
as per #1290

Signed-off-by: Christian Dupuis <cd@atomist.com>
2022-09-06 19:11:55 +02:00
zhyon404
2f47838ea1 return di.Err when it's not nil
Signed-off-by: zhyon404 <zhyong4@gmail.com>
2022-09-05 18:05:03 +08:00
Justin Chadwell
0566e62995 progress: add prefix to vertex progress group
As buildkit now uses progress groups for the COPY --link instruction
we need to ensure that we additionally prefix the progress group name,
or the target name will be left off in bake commands with more than one
target.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-09-02 16:45:51 +01:00
CrazyMax
aeac42be47 Merge pull request #1299 from thaJeztah/store_cleanup
store: move regex to where it's used
2022-08-31 19:35:05 +02:00
Sebastiaan van Stijn
aa21ff7efd store: move regex to where it's used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-31 19:19:51 +02:00
CrazyMax
57d22a7bd1 s3 cache client-side support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-30 17:00:15 +02:00
CrazyMax
6804bcbf12 Merge pull request #1298 from nicksieger/compose-go-1.5.0
vendor: bump compose-go to v1.5.0
2022-08-30 15:31:19 +02:00
Nick Sieger
6d34cc0b60 vendor: bump compose-go to v1.5.0
Signed-off-by: Nick Sieger <nick@nicksieger.com>
2022-08-29 10:22:31 -05:00
Tõnis Tiigi
1bb375fe5c Merge pull request #1265 from crazy-max/go-1.19
Dockerfile: update to go 1.19
2022-08-24 21:19:51 -07:00
Tõnis Tiigi
ed00243a0c Merge pull request #1279 from jedevc/inspect-buildkit-version
inspect: add buildkit version information to command output
2022-08-18 09:44:29 -07:00
Tõnis Tiigi
1223e759a4 Merge pull request #1281 from jedevc/fixup-1273
Prevent duplicate "failed to find driver" message
2022-08-18 09:43:09 -07:00
Tõnis Tiigi
4fd3ec1a50 Merge pull request #1277 from crazy-max/fix-compose-merge
bake(compose): fix unskipped services without build context
2022-08-18 09:41:49 -07:00
Justin Chadwell
7f9cad1e4e buildx: prevent duplicate "failed to find driver" message
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-18 14:20:47 +01:00
Justin Chadwell
437b8b140f docs: update buildx inspect reference with buildkit version
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-18 11:27:38 +01:00
Justin Chadwell
8f0d9bd71f inspect: add buildkit version information to command output
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-18 11:03:03 +01:00
CrazyMax
1378c616d6 docs: missing syntax directive
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-18 12:01:20 +02:00
Tonis Tiigi
3b5dfb3fb4 docs: make sure all named context examples use 1.4
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-18 11:57:22 +02:00
CrazyMax
9c22be5d9c bake: test compose file validation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-18 11:34:41 +02:00
CrazyMax
42dea89247 bake: test for unknown extensions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-18 11:34:40 +02:00
CrazyMax
982a332679 bake(compose): fix unskipped services without build context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-18 11:34:40 +02:00
Justin Chadwell
441853f189 Merge pull request #1274 from jedevc/consistent-experimental-help
Consistent experimental help
2022-08-18 09:37:47 +01:00
Tõnis Tiigi
611329fc7f Merge pull request #1273 from jedevc/fix-1269
create: improve interface when attempting to create docker driver
2022-08-16 10:50:04 -07:00
Tõnis Tiigi
f3c135e583 Merge pull request #1275 from tonistiigi/update-buildx-220816
vendor: update buildkit to 55ba9d14
2022-08-16 10:47:40 -07:00
Tonis Tiigi
7f84582b37 vendor: update buildkit to 55ba9d14
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-16 10:22:36 -07:00
Justin Chadwell
297526c49d docs: add experimental options to build command reference
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-16 12:40:46 +01:00
Justin Chadwell
d01d394a2b build: ensure consistent help messages for experimental cli help
Append an [experimental] tag to the end of each experimental command to
highlight that these are experimental options. Square brackets are used
instead of parentheses as parentheses are already in use to highlight
examples and defaults.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-16 11:07:36 +01:00
Justin Chadwell
17d4369866 create: improve interface when attempting to create docker driver
Previously, the help information for buildx indicated that users could
create a new instance of the docker driver - which is explicitly
something we don't support, driver of this form are automatically
derived from the available list of docker contexts.

This patch ensures that don't have AllowsInstance set will not appear in
the help text, and additionally provide a new more specific error
message instead of the generic "failed to find driver". This should help
point users in the correct direction.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-16 10:58:23 +01:00
Tonis Tiigi
fb5e1393a4 commands: use buildx env for experimental opt-in
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-16 10:37:56 +01:00
Tonis Tiigi
18dbde9ed6 build: update outline fallback image
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-16 10:25:26 +01:00
CrazyMax
2a13491919 Dockerfile: update golangci-lint to 1.48.0 (go 1.19 support)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-11 02:10:33 +02:00
CrazyMax
3509a1a7ff Dockerfile: update to go 1.19
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-11 02:10:19 +02:00
Tõnis Tiigi
da1f4b8496 Merge pull request #1100 from tonistiigi/print-outline
Build: Support for printing outline/targets of the current build
2022-08-09 18:34:15 -07:00
Tõnis Tiigi
5b2e1d3ce4 Merge pull request #1261 from crazy-max/bake-env
bake: load .env file from working dir for compose files
2022-08-09 18:31:40 -07:00
CrazyMax
7d8a6bc1d7 bake: load .env file from working dir for compose files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-10 03:24:38 +02:00
CrazyMax
a378f8095e test: misplaced expected value in assert
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-10 02:18:35 +02:00
Tõnis Tiigi
005bc009e8 Merge pull request #1262 from tonistiigi/docker-driver-features
enable other exporters if docker driver uses containerd
2022-08-09 15:37:41 -07:00
Tonis Tiigi
3bc7d4bec6 enable other exporters if docker driver uses containerd
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-09 14:16:30 -07:00
Tõnis Tiigi
96c1b05238 Merge pull request #1257 from tonistiigi/invoke-release
build: fix issues with leaving invoke containers running
2022-08-09 12:29:41 -07:00
CrazyMax
98f9f806f3 Merge pull request #1260 from ndeloof/moby-containerd
detect moby worker supports multiplatform feature through containerd snapshotter
2022-08-09 18:05:50 +02:00
Tonis Tiigi
c834ba1389 add formatting support to print function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-09 08:24:59 -07:00
Tonis Tiigi
cab437adef build: add fallback to outline requests if not supported by frontend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-09 08:24:59 -07:00
Tonis Tiigi
eefa8188e1 build: add experimental support for print flag
Print flag can be used to make additional information
requests about the build and print their results.

Currently Dockerfile supports: outline, targets, subrequests.describe

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-09 08:24:52 -07:00
Nicolas De Loof
1d8db8a738 detect moby worker supports multiplatform feature through containerd snapshotter
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-08-09 14:41:08 +02:00
Tonis Tiigi
75ddc5b811 build: fix issues with leaving invoke containers running
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-08 23:16:59 -07:00
Tõnis Tiigi
17dc0e1108 Merge pull request #1259 from ktock/invoke-messages
invoke: add messages
2022-08-08 23:16:38 -07:00
Tõnis Tiigi
64ac6c9621 Merge pull request #1256 from crazy-max/x-bake-miss
bake: contexts support with x-bake
2022-08-08 11:43:05 -07:00
CrazyMax
a7753ea781 Merge pull request #1250 from jedevc/prune-all-help
docs: add correct definition of prune --all flag
2022-08-08 14:20:36 +02:00
Justin Chadwell
12a6eb5b22 docs: add correct definition of prune --all flag
The previous definition was the same as the docker images prune command
and referenced dangling images, which isn't what the command does. This
commit brings the command description more inline with the buildctl
definition.

Additionally, add some more description of what the various flags do in
our reference pages.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-08 11:31:53 +01:00
CrazyMax
74b21258b6 Merge pull request #1252 from jedevc/prune-filter-until
Fix prune `--filter until=<duration>` option
2022-08-08 12:00:55 +02:00
Kohei Tokunaga
2f9d46ce27 invoke: add messages
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2022-08-08 17:35:28 +09:00
Tõnis Tiigi
7b660c4e30 Merge pull request #1188 from jedevc/driver-opt-warnings
Introduce new errors for unsupported driver behaviors
2022-08-05 16:58:24 -07:00
Justin Chadwell
406799eb1c prune: cleanup variable names for clarity
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-05 17:20:52 +01:00
Justin Chadwell
ef0cbf20f4 buildx: warn on editing nodes
Previously, editing nodes to contain a new set of driver options or
config files was unsupported, and silently dropping them. In this patch,
we update with these, as well as add a new warning message that any new
options may not taken into account until the builder restarts (which
may apply to the flags, platforms and endpoints as well).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-05 17:18:02 +01:00
CrazyMax
7f572eb044 bake: contexts support with x-bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-05 15:21:26 +02:00
CrazyMax
0defb614a4 docker api: use helper to parse context docker endpoint metadata
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-04 15:38:04 +02:00
CrazyMax
18023d7f32 Merge pull request #1054 from thaJeztah/test_cli_update
vendor: v20.10.3-0.20220803220330-418ca3b4d46f (v22.06.0-dev)
2022-08-04 11:42:26 +02:00
Sebastiaan van Stijn
4983b98005 vendor: v20.10.3-0.20220803220330-418ca3b4d46f (v22.06.0-dev)
full diff: f1615facb1...418ca3b4d4

relevant changes;

- cli/command: remove unused args from ResolveDefaultContext()
- consider empty DOCKER_HOST and DOCKER_CONTEXT env-vars equivalent to "not set"
- cli: set timeout connection ping on sockets as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-04 00:45:51 +02:00
Tõnis Tiigi
8675e02cea Merge pull request #1253 from crazy-max/improve-ci
ci: enhanced build workflow
2022-08-03 12:03:48 -07:00
CrazyMax
45fc3bf842 ci: enhanced build workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-03 20:53:01 +02:00
Tõnis Tiigi
cf809aec47 Merge pull request #1218 from crazy-max/validate-docs-upstream
ci: upstream docs conformance validation
2022-08-03 10:53:05 -07:00
CrazyMax
cceb1acca8 docs: fix dead link to color output controls guide
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-02 21:22:09 +02:00
CrazyMax
e620c40a14 ci: upstream docs conformance validation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-02 21:22:08 +02:00
Tõnis Tiigi
e1590bf68b Merge pull request #1248 from tonistiigi/add-jedevc
add jedevc to maintainers
2022-08-02 08:54:45 -07:00
CrazyMax
bad07943b5 Merge pull request #1247 from crazy-max/docs-pr-on-released
ci: open pr on docs repo only on release
2022-08-02 15:29:09 +02:00
CrazyMax
603595559f Merge pull request #1251 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.4.0
2022-08-02 14:11:02 +02:00
Justin Chadwell
febcc25d1a prune: fix filter until option
Previously, when specifying the filter option with the until value, no
cache would be cleaned, preventing users from clearing by time. This bug
arises from passing the until field through into buildkit, where, on
filtering, a non-existent field returns false for a match.

The fix is simple, as we build up our list of filters to pass to
buildkit, we skip over the until key, so create a valid list of filters
for buildkit.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-02 12:15:28 +01:00
CrazyMax
e3c0e34b33 update github.com/compose-spec/compose-go to v1.4.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-02 13:00:48 +02:00
Justin Chadwell
3f5974b7f9 buildx: forbid mismatched drivers
This patch reorders+refactors the runCreate function to ensure that we
can detect and notify the user in the scenario that the user attempts to
combine multiple drivers in a single builder, which is an unsupported
scenario.

Previously, we would just overwrite the previous builder with the new
driver, potentially invalidating the already existing nodes.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-02 11:06:44 +01:00
Justin Chadwell
7ab3dc080b kubernetes: error about unused endpoint argument
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-08-02 11:06:44 +01:00
Tonis Tiigi
0883beac30 add jedevc to maintainers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-08-01 22:22:47 -07:00
CrazyMax
f9102a3295 ci: open pr on docs repo only on release
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-02 03:03:37 +02:00
Tõnis Tiigi
f360088ae7 Merge pull request #974 from crazy-max/k8s-azure-auth
kubernetes: enable azure auth
2022-08-01 17:16:16 -07:00
CrazyMax
dfc1b361a9 kubernetes: enable azure auth
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-01 16:51:48 -07:00
Tõnis Tiigi
19641ec8ca Merge pull request #1214 from jedevc/timestamp-func
bake: add timestamp function
2022-08-01 16:41:21 -07:00
Tõnis Tiigi
02f7d54aed Merge pull request #1246 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.3.0
2022-08-01 16:32:31 -07:00
Tõnis Tiigi
1f6612b118 Merge pull request #1137 from jedevc/imagetools-multiple-repositories
Imagetools multiple repositories
2022-08-01 16:30:53 -07:00
Tõnis Tiigi
c1fbebe73f Merge pull request #1206 from jedevc/init-builder-errors
buildx: log errors in initializing builders
2022-08-01 16:10:18 -07:00
Tõnis Tiigi
30d650862d Merge pull request #1011 from crazy-max/bake-target-name
bake(compose): allow dot in target name
2022-08-01 10:14:58 -07:00
CrazyMax
52fd555bdd update github.com/compose-spec/compose-go to v1.3.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-01 14:12:33 +02:00
CrazyMax
7b25e2cffc Merge pull request #1245 from crazy-max/fix-readme
chore: fix readme
2022-08-01 13:30:45 +02:00
CrazyMax
5eb1e40fea chore: fix readme
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-08-01 12:08:28 +02:00
CrazyMax
7ef679d945 bake(compose): allow dot in target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-31 16:53:44 +02:00
Tõnis Tiigi
480bf2e123 Merge pull request #1241 from jedevc/docker-driver-docs
docs: add minimal docker driver docs
2022-07-29 13:35:32 -07:00
CrazyMax
0078390934 Merge pull request #1129 from crazy-max/fix-docker-context
check context builder endpoint
2022-07-29 18:16:34 +02:00
Justin Chadwell
06c11ecb61 docs: add minimal docker driver docs
Mostly for completeness, this patch adds basic documentation for the
docker driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-29 16:54:01 +01:00
CrazyMax
e27a5966ef check context builder endpoint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-29 17:39:10 +02:00
Justin Chadwell
f1a9f91323 imagetools: support cross-repo mounting
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-29 14:29:29 +01:00
Justin Chadwell
4ecca34a42 imagetools: give imagetools create a progress bar
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-29 14:29:27 +01:00
Justin Chadwell
37ca8631f9 imagetools: copy manifests between repositories
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-29 14:29:05 +01:00
Justin Chadwell
d3412f1039 imagetools: refactor combining repository logic
This patch modifies the existing combining code in imagetools create to
provide better support for multiple repositories down the road.
Specifically, the code should no longer rely on a single repository
being used for all sources and tags, and should resolve descriptors in
their relevant repositories.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-29 14:21:27 +01:00
CrazyMax
8288ce96cc Merge pull request #1240 from crazy-max/docs-fix-links-2
docs: fix link to docs website
2022-07-29 13:32:10 +02:00
CrazyMax
0222b74ee1 docs: fix link to docs website
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-29 13:26:02 +02:00
CrazyMax
97bccc5ecf Merge pull request #1239 from crazy-max/docs-fix-links
docs: fix link
2022-07-29 12:22:25 +02:00
CrazyMax
47ea0c5b03 docs: fix link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-29 12:16:56 +02:00
Tõnis Tiigi
766653f7a6 Merge pull request #994 from corhere/local-dockerfile-remote-context
build: allow external Dockerfile on remote context
2022-07-28 22:32:25 -07:00
Tõnis Tiigi
264451ba18 Merge pull request #1233 from jedevc/drivers-guides
Enhanced driver guides
2022-07-28 18:27:21 -07:00
Justin Chadwell
a42eb73043 docs: add further reading section for drivers
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-28 11:21:33 +01:00
Justin Chadwell
f2b504b77d docs: add basic docker-container driver guide
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-28 11:21:33 +01:00
Justin Chadwell
68ef5b9c9b docs: create dedicated drivers section
Create a dedicated folder for information on drivers, and write a new
index.md with content adapted from the README, and a new feature
comparisons table.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-28 11:21:33 +01:00
CrazyMax
07992e66e0 Merge pull request #1236 from crazy-max/docs-experimental-note
docs(bake): set experimental note
2022-07-27 18:43:45 +02:00
CrazyMax
4522331229 docs(bake): set experimental note
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-27 13:48:37 +02:00
CrazyMax
ec1ba14f3e Merge pull request #1215 from crazy-max/docs-fix-bake-fields
docs(bake): fix target fields and show type
2022-07-27 13:42:39 +02:00
CrazyMax
0694efb566 Merge pull request #1235 from crazy-max/update-cli-docs-tool
docs: update cli-docs-tool to v0.5.0
2022-07-27 13:40:26 +02:00
CrazyMax
1324827cd5 docs(bake): fix target fields and show type
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-27 13:13:32 +02:00
Justin Chadwell
86825a95ce buildx: rollback configuration if create fails
This builds on the added warnings from initialized builders, now
erroring the command, and additionally attempting to revert to the
previous configuration.

To preserve the previous configuration, we add a deep Copy() function to
the NodeGroup and Node so that we can easily restore it later if we
encounter a failure.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-27 11:53:47 +01:00
CrazyMax
dd445e5f9b docs: update cli-docs-tool to v0.5.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-26 19:39:04 +02:00
Tõnis Tiigi
3075a5a8c1 Merge pull request #1184 from crazy-max/docs-workflow
ci: release workflow to open a PR on docs repo with latest changes
2022-07-26 10:26:20 -07:00
Tõnis Tiigi
9ff5fb0356 Merge pull request #1224 from crazy-max/update-docker-cli
vendor: update docker/cli to f1615fa
2022-07-25 09:10:03 -07:00
CrazyMax
bc19deb5d0 Merge pull request #1232 from crazy-max/docs-remove-frontmatter
docs: remove frontmatter section
2022-07-25 11:25:15 +02:00
CrazyMax
1c7088ee42 docs: remove frontmatter section
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-25 11:01:41 +02:00
CrazyMax
97d3841fbf Merge pull request #1230 from crazy-max/docs-color-ouput
docs(guide): color output controls
2022-07-23 12:29:00 +02:00
CrazyMax
20022fd441 docs(guide): color output controls
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-23 02:57:02 +02:00
Tõnis Tiigi
23455744ac Merge pull request #1226 from spkane/spkane/colorized-buildkit-output
Update buildkit w/ customizable output colors, etc.
2022-07-22 16:55:02 -07:00
Tõnis Tiigi
0ee14fb653 Merge pull request #1229 from crazy-max/docs-update-links
docs: replace links with ones from docs.docker.com
2022-07-22 16:54:21 -07:00
CrazyMax
ff57ae1705 docs: replace links with ones from docs.docker.com
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-22 17:05:07 +02:00
CrazyMax
8da133e34f ci: release workflow to open a PR on docs repo with latest changes
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-22 16:02:15 +02:00
CrazyMax
b0deb8bdd7 vendor: update docker/cli to f1615fa
also needs to update docker/docker to a60b458 (22.06 branch) otherwise
build breaks since docker/cli#3512 with:

    # github.com/docker/cli/cli/flags
    vendor/github.com/docker/cli/cli/flags/common.go:40:37: undefined: client.EnvOverrideCertPath
    vendor/github.com/docker/cli/cli/flags/common.go:41:37: undefined: client.EnvTLSVerify
    vendor/github.com/docker/cli/cli/flags/common.go:89:76: undefined: client.EnvOverrideHost

needs also to update github.com/spf13/cobra to v1.5.0 otherwise
build breaks with:

    # github.com/docker/cli/cli-plugins/plugin
    vendor/github.com/docker/cli/cli-plugins/plugin/plugin.go:130:4: unknown field 'HiddenDefaultCmd' in struct literal of type cobra.CompletionOptions

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-22 10:53:37 +02:00
Sean P. Kane
6583dd3aa2 Update buildkit w/ customizable output colors, etc.
Signed-off-by: Sean P. Kane <spkane00@gmail.com>
2022-07-21 15:45:02 -07:00
CrazyMax
701c548e46 Merge pull request #1223 from crazy-max/update-xx
Update xx to 1.1.2
2022-07-21 14:16:21 +02:00
CrazyMax
0db719af8a Update xx to 1.1.2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-21 13:17:51 +02:00
Tõnis Tiigi
7eb1235629 Merge pull request #1217 from crazy-max/fix-docs-links
docs(guides): fix links
2022-07-19 11:48:08 -07:00
Tõnis Tiigi
11c1e03e93 Merge pull request #1155 from crazy-max/compose-cacheto
bake: support compose build cache_to
2022-07-19 11:39:28 -07:00
Tõnis Tiigi
bea1ac296c Merge pull request #1173 from deitch/oci-layout-support
add support for oci-layout build-context
2022-07-19 10:13:12 -07:00
CrazyMax
2df799d331 docs(guides): fix links
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-07-19 18:56:41 +02:00
CrazyMax
fecc6958cb Merge pull request #1208 from jedevc/kubernetes-builder-docs
docs: add new kubernetes build driver docs
2022-07-19 18:21:15 +02:00
Avi Deitcher
02bae945c3 add support for oci-layout build-context
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2022-07-18 21:19:03 -04:00
Justin Chadwell
691723f9f9 bake: add timestamp function
Terraform includes a timestamp function to get the current time. go-cty
has imported a number of the timestamp functions to it's standard
library, however, this was one was not included.

This patch simply pulls in the TimestampFunc from Terraform's
internal/lang/funcs/datetime.go to allow easily fetching the current
time in bake.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-18 11:29:35 +01:00
Justin Chadwell
900f356df9 docs: add new kubernetes build driver docs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-18 09:54:12 +01:00
CrazyMax
724cb29042 Merge pull request #1207 from developer-guy/fix/doc
fix: correct determnistic word
2022-07-12 16:15:41 +02:00
Batuhan Apaydın
f69c62f07a fix: correct determnistic word
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-07-12 16:52:55 +03:00
Justin Chadwell
309c49413c buildx: log errors in initializing builders
Previously, errors within the driver config would not be reported to the
user until they tried to use the driver, even though they are easily
accessible from the node group info.

This patch reports these errors (but will not fail because of them,
since the data is already saved) - this should help improve
debuggability of some of the more complex drivers, and prevent error
messages being suppressed.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-12 13:10:05 +01:00
CrazyMax
6824cf4548 Merge pull request #1204 from jedevc/remote-builder-typos
docs: fixup remote builder typos
2022-07-11 09:56:52 +02:00
Justin Chadwell
881b48a3b6 docs: fixup remote builder typos
- Ensure that buildx is always used as a docker subcommand
- Correct invalid buildx ls output

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-08 15:33:35 +01:00
CrazyMax
5b452b72a2 Merge pull request #1198 from jedevc/bump-buildkit-f4eb826
Bump buildkit to master branch
2022-07-06 17:15:00 +02:00
Justin Chadwell
27fcb73c7c bake: add tests for missing attributes in userfuncs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-06 16:03:41 +01:00
Justin Chadwell
2aa22597f0 bake: forbid empty result and params in userfuncs
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-06 16:03:41 +01:00
Justin Chadwell
d9ef9bec34 kubernetes: add error when no pods available
This prevents the fall-through to the panic from division by zero in the
modulus below, and presents a neater error to the user.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-06 16:02:48 +01:00
Justin Chadwell
3b4780ef19 vendor: bump buildkit to master
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-07-06 09:47:29 +01:00
CrazyMax
12fde33d9b bake: dedup compose main and extension fields values
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-23 21:58:05 +02:00
CrazyMax
a0f92829a7 bake: merge cache-from field from compose and x-bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-23 21:58:05 +02:00
CrazyMax
b438032a60 bake: support compose build cache_to
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-23 21:58:04 +02:00
Tõnis Tiigi
3cf549a7f7 Merge pull request #1181 from crazy-max/compose-consistency
bake: fix compose consistency check
2022-06-23 09:51:14 -07:00
CrazyMax
f8884a58e9 Merge pull request #1182 from crazy-max/remove-yamllint
chore: remove yamllint
2022-06-23 13:26:50 +02:00
CrazyMax
5ce3909c48 bake: fix compose consistency check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-23 13:09:17 +02:00
CrazyMax
45fac6dee3 chore: remove yamllint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-23 11:15:01 +02:00
Tõnis Tiigi
a8bb25d1b5 Merge pull request #1168 from ktock/invoke
Add `--invoke` option to launch a container from the build result
2022-06-21 08:51:25 -07:00
Kohei Tokunaga
387e1ecca6 Add --invoke option to launch a container from the build result
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2022-06-21 10:32:07 +09:00
CrazyMax
ad7b077d13 Merge pull request #1167 from crazy-max/refactor-xbake
bake: better handling of compose extension interface
2022-06-16 23:17:52 +02:00
CrazyMax
432c2b2650 bake: better handling of compose extension interface
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-14 23:23:43 +02:00
Tõnis Tiigi
055e85f48f Merge pull request #1166 from crazy-max/ci-concurrency
ci: add concurrency check
2022-06-14 12:49:15 -07:00
Tõnis Tiigi
91fec23f5d Merge pull request #1153 from crazy-max/update-compose-go
bake: support compose build tags and secrets env
2022-06-14 12:45:33 -07:00
Tõnis Tiigi
0295555a5a Merge pull request #1109 from crazy-max/ls-progress
ls: move builder/node status error msg below table
2022-06-14 10:03:22 -07:00
CrazyMax
6cb1b85d7b bake: support compose build tags
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-14 18:45:57 +02:00
CrazyMax
e0350f671a bake: support compose secrets env
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-14 18:45:23 +02:00
CrazyMax
c1adfcb658 ci: add concurrency check
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-14 17:54:10 +02:00
CrazyMax
1343cdfc83 Merge pull request #1140 from crazy-max/bake-docs
docs: bake guides and refactor reference
2022-06-10 09:45:49 +02:00
CrazyMax
f40c2dbb86 docs: rework bake compose file definition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-09 12:53:01 +02:00
CrazyMax
50c23aa755 update github.com/compose-spec/compose-go to v1.2.7
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-08 22:53:15 +02:00
CrazyMax
ff9517cbf0 docs: rework bake "Configuring builds" page
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-08 16:45:08 +02:00
CrazyMax
824b0268d8 docs: bake specification intro
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-08 16:14:34 +02:00
CrazyMax
77ea999adb docs: guide page to configure bake builds
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-05 18:49:16 +02:00
CrazyMax
1807cfdd26 docs: changes to bake file definition guide
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-05 18:49:16 +02:00
CrazyMax
ebd7d062bf docs: bake guides and refactor reference
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-05 18:48:09 +02:00
CrazyMax
6cb026b766 Merge pull request #1148 from crazy-max/fix-no-output-warn
build: enhance warning message when no output specified
2022-06-05 17:30:46 +02:00
CrazyMax
1cb1ee018b build: enhance warning message when no output specified
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-03 19:26:21 +02:00
Tõnis Tiigi
71e4a39ae9 Merge pull request #1134 from jedevc/remote-driver-connhelpers
Remote driver connhelpers
2022-05-31 16:09:55 -07:00
CrazyMax
009730f5fd Merge pull request #1145 from docker/dependabot/github_actions/crazy-max/ghaction-github-runtime-2
build(deps): bump crazy-max/ghaction-github-runtime from 1.0.0 to 2
2022-05-27 07:01:42 +02:00
dependabot[bot]
36466c0744 build(deps): bump crazy-max/ghaction-github-runtime from 1.0.0 to 2
Bumps [crazy-max/ghaction-github-runtime](https://github.com/crazy-max/ghaction-github-runtime) from 1.0.0 to 2.
- [Release notes](https://github.com/crazy-max/ghaction-github-runtime/releases)
- [Changelog](https://github.com/crazy-max/ghaction-github-runtime/blob/master/CHANGELOG.md)
- [Commits](94085cef04...906832f62b)

---
updated-dependencies:
- dependency-name: crazy-max/ghaction-github-runtime
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-26 18:31:58 +00:00
Justin Chadwell
1406ff141b docs: add connhelpers info to remote-builder guide
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-25 10:39:16 +01:00
Justin Chadwell
1eff9310f8 remote: add additional connhelpers to buildx
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-25 10:39:16 +01:00
Justin Chadwell
22ac3271d2 vendor: update moby/buildkit
Update modules:

  go mod edit -require github.com/moby/buildkit@master
  go mod tidy -compat=1.17 && ./hack/update-vendor

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-25 10:20:57 +01:00
CrazyMax
064bd92583 ls: move builder/node status error msg below table
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-24 22:37:08 +02:00
Tõnis Tiigi
1beb3359a6 Merge pull request #1144 from tonistiigi/nil-platform
imagetools: handle manifest with nil platform
2022-05-24 10:05:59 -07:00
Tõnis Tiigi
35f4268081 Merge pull request #1128 from crazy-max/rm
rm: display removed builder and disallow removing context builders
2022-05-24 09:40:26 -07:00
CrazyMax
81ce766501 Merge pull request #1130 from photra/clarify-inspect-docs
Clarify inspect documentation
2022-05-24 14:11:22 +02:00
CrazyMax
66a764f9c1 Merge pull request #1143 from jedevc/revive-linter
Update golint to revive
2022-05-24 09:15:00 +02:00
Tonis Tiigi
e4137b2eea imagetools: handle manifest with nil platform
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-05-23 21:16:00 -07:00
Justin Chadwell
48067735fc Update golint to revive
Resolves the following message in golangci output:

> The linter 'golint' is deprecated (since v1.41.0) due to: The
repository of the linter has been archived by the owner.  Replaced by
revive.

Additionally, fix a minor linting issue discovered by revive.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-23 16:54:46 +01:00
CrazyMax
54a2a0c49f cli: uppercase level to match logrus one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-20 10:59:33 +02:00
CrazyMax
d611bbe609 rm: display name of removed builder
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-20 10:59:33 +02:00
CrazyMax
1e71a3ffa7 rm: disallow removing context builders
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-20 10:59:32 +02:00
Tõnis Tiigi
4a215a943b Merge pull request #1131 from crazy-max/ls-dedup
ls: dedup instances from store and context
2022-05-19 17:30:58 -07:00
CrazyMax
69d95cc847 create: warn if instance name already exists as context builder
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-19 17:17:44 +02:00
Tõnis Tiigi
cdd391e556 Merge pull request #1136 from jedevc/remote-builder-docs-typo
Fix typo in docker-container remote driver instructions
2022-05-18 19:51:55 -07:00
Justin Chadwell
d69fe6140d docs: fix typo in docker-container remote driver instructions
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-18 10:35:09 +01:00
Cory Snider
ca3507656d build: allow external Dockerfile on remote context
BuildKit has supported external Dockerfile on remote contexts since
v0.5.0, included in Moby v19.03.0. The client side was the only missing
piece.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-05-16 16:47:08 -04:00
CrazyMax
78ae826d74 ls: dedup instances from store and context
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-15 08:30:31 +02:00
Phong Tran
5a8060ea9f Clarify inspect documentation
Signed-off-by: Phong Tran <tran.pho@northeastern.edu>
2022-05-14 23:09:38 -07:00
Tõnis Tiigi
908ce2d206 Merge pull request #1097 from crazy-max/dockerfile-test-stage
dockerfile: enhance test stage
2022-05-13 13:56:11 -07:00
CrazyMax
69824a5d27 Merge pull request #1111 from crazy-max/default-conf
create: load default buildkit config if none specified
2022-05-13 14:23:01 +02:00
CrazyMax
5d38fff729 Merge pull request #1120 from docker/dependabot/github_actions/codecov/codecov-action-3
build(deps): bump codecov/codecov-action from 2 to 3
2022-05-13 13:46:47 +02:00
CrazyMax
31d12c89fa Merge pull request #1123 from crazy-max/update-gha
ci: update gha bundle
2022-05-13 13:45:33 +02:00
CrazyMax
8257a04a7d create: load default buildkit config if none specified
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-13 11:54:16 +02:00
CrazyMax
bdc41dd308 ci: pin external/untrusted github actions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-12 20:23:33 +02:00
CrazyMax
f6e00a609d ci: bump official actions to latest major
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-12 20:20:51 +02:00
CrazyMax
1845edd647 ci: remove godev workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-12 19:55:00 +02:00
CrazyMax
cab4cfe28f ci: bump docker actions to latest major
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-12 19:54:44 +02:00
dependabot[bot]
815c1dd05c build(deps): bump codecov/codecov-action from 2 to 3
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 2 to 3.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/master/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-12 17:26:02 +00:00
Tõnis Tiigi
bbfdaa4161 Merge pull request #1096 from crazy-max/ci-cache
ci: enable cache
2022-05-12 10:25:15 -07:00
Tõnis Tiigi
a9e62dfa83 Merge pull request #1098 from crazy-max/dependabot-gha
chore: dependabot to keep gha up to date
2022-05-12 10:22:39 -07:00
Tõnis Tiigi
b9a408017c Merge pull request #1093 from jedevc/remote-driver-guide
Add remote driver guide
2022-05-12 10:22:11 -07:00
Tõnis Tiigi
062cf29de2 Merge pull request #1103 from jedevc/remote-driver-bootstrap
Use --bootstrap to wait for remote to become active
2022-05-12 10:21:15 -07:00
Justin Chadwell
a2f1de6459 Add remote driver guide
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-12 10:44:27 +01:00
Tõnis Tiigi
98439f7f08 Merge pull request #1108 from crazy-max/ls-docs
docs: update ls output example
2022-05-11 11:21:37 -07:00
Tõnis Tiigi
6854eec48d Merge pull request #1057 from thaJeztah/vendor_update
vendor: vendor with -compat=1.17
2022-05-10 13:53:56 -07:00
CrazyMax
1edfb13ba8 docs: update ls output example
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-10 20:54:38 +02:00
Sebastiaan van Stijn
35b238ee82 vendor: vendor with -compat=1.17
This might break compatibility with projects using this module that
are still on go1.16, which is EOL, so probably ok to ignore:

    github.com/docker/buildx/store imports
        github.com/gofrs/flock tested by
        github.com/gofrs/flock.test imports
        gopkg.in/check.v1 loaded from gopkg.in/check.v1@v1.0.0-20200227125254-8fa46927fb4f,
        but go 1.16 would select v1.0.0-20201130134442-10cb98267c6c

    To upgrade to the versions selected by go 1.16:
        go mod tidy -go=1.16 && go mod tidy -go=1.17
    If reproducibility with go 1.16 is not needed:
        go mod tidy -compat=1.17
    For other options, see:
        https://golang.org/doc/modules/pruning

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-10 20:33:20 +02:00
Tõnis Tiigi
55b85f5bb2 Merge pull request #998 from crazy-max/ls-buildkit-version
ls: display buildkit version of the nodes
2022-05-10 08:37:38 -07:00
CrazyMax
57156ee95c ls: adds fallback if buildkit version info unimplemented
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-09 16:17:30 +02:00
Justin Chadwell
c245f30a94 remote: use --bootstrap to wait for remote to become active
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-09 13:51:07 +01:00
CrazyMax
6e3babc461 ls: display buildkit version of the nodes
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-09 14:12:31 +02:00
CrazyMax
4ee8b14f2a vendor: update buildkit to c78f696
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-09 14:12:31 +02:00
CrazyMax
21b41e580a chore: dependabot to keep gha up to date
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-06 12:51:43 +02:00
CrazyMax
cc90c5ca3c dockerfile: use gobase for test stage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-06 11:59:43 +02:00
CrazyMax
519aca3672 ci: enable cache
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-06 11:55:54 +02:00
CrazyMax
43968ffa68 Merge pull request #1094 from jedevc/remote-e2e-flake
Attempt to solve some flakiness in the remote driver ci
2022-05-06 10:40:40 +02:00
Justin Chadwell
79ba92b7f8 ci: add check remote buildkitd step in e2e tests
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-05-06 09:19:55 +01:00
CrazyMax
e0cffbdbdf Merge pull request #1095 from crazy-max/fix-checksum-file
release: fix checksum file
2022-05-05 16:15:59 +02:00
CrazyMax
df799b6a0f release: fix checksum file
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-05 14:29:54 +02:00
CrazyMax
27bdbea410 Merge pull request #1087 from crazy-max/fix-guides
docs: fix guides for docs.docker.com
2022-05-05 11:50:06 +02:00
CrazyMax
1e52c2107c Merge pull request #1049 from tonistiigi/update-dockerfile-deps
Dockerfile: update dependencies
2022-05-02 17:49:49 +02:00
CrazyMax
cf298ee01c docs: fix guides for docs.docker.com
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-05-02 13:10:48 +02:00
CrazyMax
e9d6501a4f Merge pull request #1086 from jedevc/remote-driver-docs
Add remote driver documentation
2022-05-02 09:51:32 +02:00
CrazyMax
92009ed03c Merge pull request #1085 from tonistiigi/closeread-ignore
root: ignore SSH CloseRead warning
2022-04-29 17:59:56 +02:00
Justin Chadwell
f2fc0e9eb5 Add remote driver documentation
Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-04-29 16:43:28 +01:00
Tõnis Tiigi
38f1138a45 Merge pull request #1078 from jedevc/remote-driver
Add remote driver
2022-04-29 08:25:25 -07:00
Tonis Tiigi
72758fef22 root: ignore SSH CloseRead warning
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-04-29 08:24:13 -07:00
Tõnis Tiigi
9cdd837f6b Merge pull request #1080 from jedevc/issue-1051
hclparser: strip out blocks for json files
2022-04-28 21:11:02 -07:00
Justin Chadwell
d7e4affe98 Complete remote driver
This patch completes the work started in creating a remote driver:

- Renames the env driver to the remote driver (an alternative suggestion
  that should be more user-friendly)
- Adds support for TLS to encrypt connections with buildkitd
- Fixes outstanding review comments
- Reworks the buildx create command endpoint construction to be clearer
  and include better support for this new driver.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-04-28 11:34:06 +01:00
Batuhan Apaydın
3dc83e5dd8 feat: env driver
Co-authored-by: Furkan Türkal <furkan.turkal@trendyol.com>
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-04-28 11:34:06 +01:00
CrazyMax
29f97f6762 Merge pull request #1083 from crazy-max/e2e-workflow
ci: enhance e2e workflow
2022-04-28 12:31:57 +02:00
CrazyMax
88a45cfb24 ci: enhance e2e workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-28 12:03:33 +02:00
Justin Chadwell
03885ec9f1 hclparser: strip out blocks for json files
Body.JustAttributes cannot distinguish between blocks and attributes for
JSON files, so the variable block could be included in the list of
attributes returned.

This patch ensures that JSON and HCL files behave the same way by
removing all known block types first, from the provided config schema
and then from a generated definitions schema.

Fixes #1051

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-04-28 10:22:20 +01:00
Tonis Tiigi
a648d58f63 hack: update linters
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-04-27 17:22:32 -07:00
Tonis Tiigi
0b9d426175 Dockerfile: update to go 1.18
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-04-27 17:22:31 -07:00
Tõnis Tiigi
1c23d1cef5 Merge pull request #1082 from crazy-max/fix-standalone
cli: fix standalone command behavior
2022-04-27 10:51:51 -07:00
CrazyMax
95086cf641 cli: fix standalone command behavior
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-27 17:33:16 +02:00
Tõnis Tiigi
6a702ebe5b Merge pull request #1025 from crazy-max/bake-merge-jsons
bake: merge targets and vars from multiple JSON files
2022-04-25 19:16:05 -07:00
Tõnis Tiigi
a6a1a362ad Merge pull request #1053 from szeber/kubernetes-tolerations-fix
Fix tolerations not parsing its options correctly, update docs regarding quoted driver options
2022-04-21 21:20:26 -07:00
Zsolt
4a226568a0 Fix tolerations not parsing its options correctly, add tests
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
2022-04-22 12:12:05 +12:00
CrazyMax
a2d5bc7cca Merge pull request #1069 from crazy-max/compose-build-secrets
bake: support compose build secrets
2022-04-14 12:05:58 +02:00
Tõnis Tiigi
951201ac1b Merge pull request #1067 from jedevc/fix-imagetools-builder
imagetools: respect --builder flag
2022-04-13 16:40:47 -07:00
CrazyMax
c0f8a8314b bake: support compose build secrets
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 01:28:00 +02:00
Tõnis Tiigi
d64428cd2a Merge pull request #1063 from dougborg/custom-rootless-image-fix
Only set default rootless image if it is not already customized
2022-04-13 11:05:43 -07:00
CrazyMax
3a90f99635 update github.com/compose-spec/compose-go to v1.2.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-13 16:39:32 +02:00
Justin Chadwell
04b44b3a89 imagetools: respect --builder flag
The --builder flag was being ignored by imagetools because of pointer
problems. Essentially, because the root cmds aren't parsed immediately,
we need to pass a pointer to the builder string so that it can be
updated before the RunE function gets called.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2022-04-13 14:36:48 +01:00
Tõnis Tiigi
b7c4fe5a3a Merge pull request #1066 from crazy-max/fix-inspect
inspect: fix printing of driver options
2022-04-12 18:05:36 -07:00
CrazyMax
082c83b825 inspect: fix printing of driver options
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-12 20:56:57 +02:00
Doug Borg
79de2c5d82 Only set default rootless image if it is not already customized
Only change the image to the default rootless image when using the
--rootless option if the image has not already customized with the
--image option.

Fix #938

Signed-off-by: Doug Borg <dougborg@apple.com>
2022-04-12 10:13:43 -06:00
Tõnis Tiigi
b8bcf1d810 Merge pull request #1052 from AkihiroSuda/update-kube-seccomp
kubernetes: replace deprecated seccomp annotations with securityContext
2022-04-08 20:50:00 -07:00
CrazyMax
28a4363672 Merge pull request #1055 from thaJeztah/update_dockerd_version
Dockerfile: update DOCKERD_VERSION to v20.10.14
2022-04-08 18:26:56 +02:00
CrazyMax
1e98de491d Merge pull request #1056 from thaJeztah/update_xx
Dockerfile: update to tonistiigi/xx:1.1.0
2022-04-08 18:26:36 +02:00
Sebastiaan van Stijn
b54a0aa37c Dockerfile: update to tonistiigi/xx:1.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-08 17:28:45 +02:00
Sebastiaan van Stijn
e10c385167 Dockerfile: update DOCKERD_VERSION to v20.10.14
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-08 17:26:40 +02:00
Akihiro Suda
add4301ed6 kubernetes: replace deprecated seccomp annotations with securityContext
Kubernetes added the official `securityContext.seccompProfile` support in Kubernetes 1.19.
Seccomp is still disabled by default.

The legacy `container.seccomp.security.alpha.kubernetes.io/<PODNAME>` annotation has been deprecated and will be unsupported in Kubernetes 1.25.
https://kubernetes.io/docs/tutorials/security/seccomp/

A test cluster can be created with the following minikube command:
```
minikube start --feature-gates SeccompDefault=true --extra-config kubelet.seccomp-default=true
```

Related to moby/buildkit PR 2782

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-04-07 17:23:42 +09:00
Tõnis Tiigi
a60150cbc6 Merge pull request #1045 from szeber/kubernetes-tolerations
feat: add tolerations handling to kubernetes driver
2022-04-06 21:50:17 -07:00
CrazyMax
cad7ed68be bake: merge vars from multiple JSON files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-06 22:17:23 +02:00
CrazyMax
c317ca1e95 bake: merge targets from multiple JSON files
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-06 22:17:23 +02:00
Zsolt
3f6517747e Add support for defining kubernetes tolerations
Signed-off-by: Zsolt <zsolt.szeberenyi@figured.com>
2022-04-06 17:10:21 +12:00
Tõnis Tiigi
adafbe0e65 Merge pull request #1041 from crazy-max/vendor-buildkit
vendor: update buildkit to 3e38a2d
2022-04-04 11:53:01 -07:00
CrazyMax
a49ad031a5 vendor: update buildkit to 3e38a2d
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-03 20:40:33 +02:00
Tõnis Tiigi
c3db06cda0 Merge pull request #970 from crazy-max/nocache-mod-outdated
chore: use no-cache-filter for outdated stage
2022-04-01 16:32:41 -07:00
Akihiro Suda
1201782a11 Merge pull request #1033 from crazy-max/update-compose-go
update github.com/compose-spec/compose-go to v1.2.1
2022-03-30 15:02:24 +09:00
CrazyMax
243b428a58 compose: add test for port mapping
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-30 00:37:55 +02:00
CrazyMax
785dc17f13 update github.com/compose-spec/compose-go to v1.2.1
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-29 21:24:16 +02:00
CrazyMax
cad87f54c5 Merge pull request #1021 from crazy-max/bake-fix-visited-group
bake: fix skipped group when already visited by another one
2022-03-28 21:16:01 +02:00
CrazyMax
0b8dde1071 bake: fix skipped group when already visited by another one
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-23 11:16:10 +01:00
CrazyMax
1ca30a58c2 Merge pull request #1003 from developer-guy/feature/1001
feat: printing driver options
2022-03-17 12:28:36 +01:00
CrazyMax
1246e8da3a Merge pull request #1008 from thaJeztah/bump_distribution
vendor: github.com/docker/distribution v2.8.1
2022-03-17 12:28:20 +01:00
Sebastiaan van Stijn
c0f31349a6 vendor: github.com/docker/distribution v2.8.1
no significant changes to code we use, but the v2.8.0 module was borked

full diff: https://github.com/docker/distribution/compare/v2.8.0...v2.8.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 11:31:59 +01:00
Tõnis Tiigi
5c2d2f294d Merge pull request #1004 from tonistiigi/update-fsutil-220315
vendor: update fsutil to 9ed61262
2022-03-16 09:40:37 -07:00
Batuhan Apaydın
da4c27e9af feat: printing driver options
Signed-off-by: Batuhan Apaydın <batuhan.apaydin@trendyol.com>
2022-03-16 11:13:19 +03:00
Tonis Tiigi
111ea95629 vendor: update fsutil to 9ed61262
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-15 17:09:12 -07:00
Tõnis Tiigi
3adca1c17d Merge pull request #995 from thaJeztah/replace_ioutil
Remove uses of deprecated io/ioutil
2022-03-14 19:13:07 -07:00
Tõnis Tiigi
6ffe22b843 Merge pull request #996 from crazy-max/buildinfo-use-helper
imagetools inspect: use buildinfo helper
2022-03-14 19:12:14 -07:00
CrazyMax
824cb42fe0 dockerfile: update frontend to 1.4
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-10 02:11:19 +01:00
CrazyMax
08bb626304 chore: use no-cache-filter for outdated stage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-10 02:11:19 +01:00
CrazyMax
38311a35f2 imagetools inspect: use buildinfo helper
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-10 01:52:51 +01:00
Sebastiaan van Stijn
fd62216cbc golangci-lint: prevent io/ioutil from being used
The package has been deprecated since Go 1.16:

https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 01:47:41 +01:00
Sebastiaan van Stijn
fc7ba75fd7 Remove uses of deprecated io/ioutil
The package has been deprecated since Go 1.16: https://go.dev/doc/go1.16#ioutil

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-10 01:46:06 +01:00
Tõnis Tiigi
c8f7c1e93f Merge pull request #993 from tonistiigi/update-buildkit-220308
vendor: update buildkit
2022-03-08 16:59:14 -08:00
Tõnis Tiigi
b78c680207 Merge pull request #989 from crazy-max/moby-imgdgst
build: set remote digest when pushed with docker driver
2022-03-08 10:55:18 -08:00
Tonis Tiigi
d7412c9420 vendor: update buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-08 10:53:06 -08:00
CrazyMax
a7fba7bf3a Merge pull request #992 from tonistiigi/bake-metadata-fix
bake: restore consistent output for metadata
2022-03-08 19:43:48 +01:00
CrazyMax
19ff7cdadc build: set remote digest when pushed with docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-08 19:31:13 +01:00
Tonis Tiigi
c255c04eed bake: restore consistent output for metadata
Metadata formatting should not depend on the number
of targets.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-08 10:06:03 -08:00
Tõnis Tiigi
9fcea76dea Merge pull request #977 from tonistiigi/logs-dupes
progress: avoid double logs when multiple targets build same step
2022-03-04 16:30:53 -08:00
Tõnis Tiigi
1416bc1d83 Merge pull request #972 from crazy-max/imagetools-inspect-order
imagetools inspect: keep platform order
2022-03-04 11:54:15 -08:00
CrazyMax
215a128fc1 imagetools inspect: missing manifest digest for manifest-list (json)
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-04 20:36:14 +01:00
CrazyMax
4e4eea7814 imagetools inspect: deterministic platform order
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-04 20:36:14 +01:00
Tõnis Tiigi
8079bd2841 Merge pull request #980 from crazy-max/imageid
build: return imageID when loading without docker driver
2022-03-04 10:50:53 -08:00
CrazyMax
2d5368cccc Merge pull request #981 from tonistiigi/target-context-remove
build: remove target context if platform specific used
2022-03-04 16:29:20 +01:00
CrazyMax
a1256c6bb2 Merge pull request #985 from tonistiigi/multi-node-platform
build: fix multi-node builds with mixed platforms
2022-03-04 15:35:55 +01:00
CrazyMax
e7863eb664 build: return imageID when loading without docker driver
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-03-04 15:27:09 +01:00
Tonis Tiigi
171c4375a1 build: fix multi-node builds with mixed platforms
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-03 13:15:13 -08:00
Tonis Tiigi
45844805ec build: remove target context if platform specific used
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-03-01 21:25:43 -08:00
Tonis Tiigi
b77d7864fa progress: avoid double logs when multiple targets build same step
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-28 23:32:54 -08:00
Tõnis Tiigi
6efcee28d5 Merge pull request #973 from crazy-max/vendor-docker-cli
vendor: update docker/cli to 8667ccd
2022-02-27 20:29:34 -08:00
Tõnis Tiigi
3ad24524c4 Merge pull request #971 from crazy-max/fix-docs
docs: small fixes
2022-02-27 20:26:27 -08:00
CrazyMax
971b5d2b73 vendor: update docker/cli to 8667ccd
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-27 00:35:39 +01:00
CrazyMax
94c5dde85a docs: small fixes
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-26 05:39:42 +01:00
Tõnis Tiigi
f62c02329e Merge pull request #969 from tonistiigi/update-buildkit-20220225
vendor: update buildkit to 0692ad79
2022-02-25 12:15:13 -08:00
Tonis Tiigi
d2e53f5e05 vendor: update buildkit to 0692ad79
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-25 11:52:45 -08:00
Tõnis Tiigi
7af29802d4 Merge pull request #854 from crazy-max/buildinfo-cmd
imagetools inspect: add --format flag
2022-02-25 11:52:27 -08:00
Tõnis Tiigi
6ac01ec9ac Merge pull request #965 from tonistiigi/bake-context-validation
bake: additional support for named context on remote inputs
2022-02-25 11:36:51 -08:00
CrazyMax
20a55e9184 imagetools inspect: multi-platform support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-25 20:30:08 +01:00
CrazyMax
6c56109083 imagetools inspect: add --format flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-25 17:55:34 +01:00
Tõnis Tiigi
dab3fe71bd Merge pull request #967 from crazy-max/update-clidocstool
docs: update cli-docs-tool to v0.4.0
2022-02-25 08:47:52 -08:00
CrazyMax
9867ca279a docs: update cli-docs-tool to v0.4.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-25 15:41:17 +01:00
Tonis Tiigi
91e550b715 bake: add path validation for remote bake invocations
This is a stopgap before proper entitlements support
is implemented.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-24 23:19:18 -08:00
Tonis Tiigi
280c008f81 bake: make named contexts relative to remote bake input
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-24 23:19:12 -08:00
Tõnis Tiigi
5939a23af6 Merge pull request #963 from tonistiigi/bake-contexts-error
bake: use better error in named contexts not supported
2022-02-24 20:30:55 -08:00
CrazyMax
7f1041164e Merge pull request #964 from tonistiigi/update-buildkit-022322
vendor: update buildkit to b124b0c3
2022-02-23 21:35:09 +01:00
Tonis Tiigi
64ce211ba4 vendor: update buildkit to b124b0c3
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-23 12:15:16 -08:00
Tonis Tiigi
b5bf28d722 bake: use better error in named contexts not supported
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-22 21:56:52 -08:00
Tõnis Tiigi
10debb577e Merge pull request #959 from tonistiigi/docker-proxy-config
set build-args from docker proxy configuration
2022-02-22 13:56:11 -08:00
CrazyMax
75cdea48e4 Merge pull request #962 from tonistiigi/bake-deps-error
build: fix deadlock on handling deps errors
2022-02-22 10:50:24 +01:00
CrazyMax
d96d7fb2dc Merge pull request #930 from tylerlwsmith/fix-readme-typo
Fix typo in readme
2022-02-21 10:53:05 +01:00
Tyler Smith
e3245a400a Fix typo in readme
Changed "note tha" to "note that"

Signed-off-by: Tyler Smith <tylerlwsmith@gmail.com>
2022-02-18 19:55:05 -08:00
Tõnis Tiigi
e871c39f05 Merge pull request #908 from crazy-max/inline-buildattrs
build: inline buildinfo attrs
2022-02-18 19:36:25 -08:00
Tõnis Tiigi
9c0a23996d Merge pull request #958 from crazy-max/buildinfo-deps
build: send buildinfo dependencies
2022-02-18 19:35:57 -08:00
Tonis Tiigi
3b2aeb2d5b build: fix deadlock on handling deps errors
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-18 18:41:52 -08:00
Tonis Tiigi
e98a476dc8 set build-args from docker proxy configuration
For backward compatibility with docker build.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-17 22:21:43 -08:00
CrazyMax
7677052cb7 build: send buildinfo dependencies
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-17 21:02:06 +01:00
Tõnis Tiigi
c273e0986c Merge pull request #953 from crazy-max/update-docs
docs: updates and guides
2022-02-17 10:54:31 -08:00
CrazyMax
9ee499ae27 docs: metadata-file usage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-17 17:52:50 +01:00
CrazyMax
230dfa96a3 docs: built-in build args
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-17 17:52:49 +01:00
CrazyMax
f1a8f54c83 docs: user guides
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-17 17:52:49 +01:00
CrazyMax
2bcf3524e5 docs: ssh usage example
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-16 15:14:45 +01:00
CrazyMax
26918513e3 docs: lint
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-16 15:14:45 +01:00
Tõnis Tiigi
893d505803 Merge pull request #955 from crazy-max/vendor-buildkit
vendor: update buildkit to 1e6032c
2022-02-15 11:32:04 -08:00
CrazyMax
22aaa260e7 vendor: update buildkit to 1e6032c
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-15 19:12:00 +01:00
CrazyMax
1bcc3556fc Merge pull request #949 from docker/dependabot/go_modules/github.com/docker/distribution-2.8.0incompatible
build(deps): bump github.com/docker/distribution from 2.7.1+incompatible to 2.8.0+incompatible
2022-02-14 11:48:21 +01:00
Tõnis Tiigi
eef6deb7c2 Merge pull request #860 from tonistiigi/no-cache-filter
build: add no-cache-filter
2022-02-11 20:09:40 -08:00
dependabot[bot]
542759ea31 build(deps): bump github.com/docker/distribution
Bumps [github.com/docker/distribution](https://github.com/docker/distribution) from 2.7.1+incompatible to 2.8.0+incompatible.
- [Release notes](https://github.com/docker/distribution/releases)
- [Commits](https://github.com/docker/distribution/compare/v2.7.1...v2.8.0)

---
updated-dependencies:
- dependency-name: github.com/docker/distribution
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-11 22:39:59 +00:00
Tõnis Tiigi
e5f590a7fa Merge pull request #946 from crazy-max/metadata-output
build: enhance metadata json output
2022-02-10 23:59:23 -08:00
CrazyMax
ecf215b927 e2e: add bake build and display metadata json
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-11 08:16:47 +01:00
CrazyMax
299fd19c49 build: enhance metadata json output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-11 08:16:42 +01:00
CrazyMax
4b633c3c7b docs: built-in build args
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-11 07:57:55 +01:00
Tonis Tiigi
eb8057e8e0 forbid setting no-cache and no-cache-filter together
Per review request.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-10 22:01:50 -08:00
Tonis Tiigi
32f6358d78 bake: add no-cache-filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-10 22:01:48 -08:00
Tonis Tiigi
3b47722032 build: add no-cache-filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-02-10 22:00:02 -08:00
Tõnis Tiigi
e60f0f2c4f Merge pull request #943 from crazy-max/secret-examples
docs: secret usage examples
2022-02-10 17:36:26 -08:00
CrazyMax
b39ebab666 docs: secret usage examples
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-10 15:37:11 +01:00
Akihiro Suda
f891187d8b Merge pull request #948 from crazy-max/vendor-buildkit
vendor: update buildkit to 2f99651
2022-02-10 12:15:39 +09:00
CrazyMax
307c94e5c7 vendor: update buildkit to 2f99651
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-09 21:53:40 +01:00
Tõnis Tiigi
60a025b227 Merge pull request #928 from tonistiigi/bake-named-contexts
bake: add named contexts keys
2022-02-07 10:52:01 -08:00
Tõnis Tiigi
fec415a8e0 Merge pull request #937 from hinshun/printer-writer
Relax to io.Writer requirement for NewPrinter
2022-02-06 23:24:45 -08:00
Edgar Lee
2d7540fb0a Separate io.Writer from console.File for NewPrinter
Signed-off-by: Edgar Lee <edgarl@netflix.com>
2022-02-06 09:11:47 -08:00
Tõnis Tiigi
595285736c Merge pull request #885 from crazy-max/rm-inactive
cli: add --all-inactive for rm command
2022-02-03 19:25:25 -08:00
CrazyMax
378f0b45c6 cli: add --all-inactive and --force flags for rm command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-02-03 13:39:53 +01:00
Tonis Tiigi
c3dab802d8 docs: add examples for bake named contexts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-27 18:46:12 -08:00
Tonis Tiigi
fa04611afc bake: connect results between build targets
Build context “target:<name>” will take the contents
from another bake target.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-27 18:46:12 -08:00
Tonis Tiigi
ffa062dc95 util: add waitmap for target synchronization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-27 18:46:12 -08:00
Tonis Tiigi
0fc2b5ca85 bake: add named contexts keys
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-27 18:46:02 -08:00
Tõnis Tiigi
9a1267cd02 Merge pull request #929 from crazy-max/bake-target-name
bake: restrict target name
2022-01-26 09:06:02 -08:00
CrazyMax
c74b2fe7a4 bake: restrict target name
This fix adds a restriction `[a-zA-Z0-9_-]+`
for target name. This is pretty much the same as the
container name restriction in moby.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-26 10:29:21 +01:00
Tõnis Tiigi
6c69d970f7 Merge pull request #924 from crazy-max/log
root: filter out useless commandConn.CloseWrite warning message
2022-01-25 08:54:21 -08:00
CrazyMax
d3e56ea9d9 root: simple output format on logrus for parity with cli
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-25 08:54:42 +01:00
Tõnis Tiigi
11b771c789 Merge pull request #904 from tonistiigi/named-contexts
build: support for named contexts(stages)
2022-01-20 19:58:40 -08:00
CrazyMax
278f94a8b6 root: filter out useless commandConn.CloseWrite warning message
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-21 00:36:06 +01:00
Tõnis Tiigi
14b38a9aa8 Merge pull request #914 from crazy-max/fix-target-inherit
bake: keep target inheritance
2022-01-13 18:35:13 -08:00
CrazyMax
0044c28b1f bake: keep target inheritance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-13 22:23:20 +01:00
Tõnis Tiigi
b568b219bc Merge pull request #920 from crazy-max/update-gocty
bump github.com/zclconf/go-cty from 1.7.1 to 1.10.0
2022-01-13 10:56:54 -08:00
Tõnis Tiigi
aabbe5a56a Merge pull request #919 from crazy-max/mod-outdated-bake
chore: invalidate cache for outdated run stage
2022-01-13 10:56:05 -08:00
Tõnis Tiigi
b038dd063e Merge pull request #921 from crazy-max/readme-install
Update install instructions
2022-01-13 10:55:07 -08:00
CrazyMax
f25d5ff02f Update install instructions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-13 14:57:25 +01:00
CrazyMax
b67bdedb23 bump github.com/zclconf/go-cty from 1.7.1 to 1.10.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-13 14:32:10 +01:00
CrazyMax
3ccb883d95 chore: invalidate cache for outdated run stage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-13 14:09:47 +01:00
Tõnis Tiigi
785c861233 Merge pull request #915 from crazy-max/ci-buildkit-image
ci: build with stable buildkit image
2022-01-11 22:37:55 -08:00
CrazyMax
1b69919313 ci: build with stable buildkit image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-12 07:16:58 +01:00
CrazyMax
24db7366ba build: inline buildinfo attrs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-08 17:53:24 +01:00
Tonis Tiigi
08547827db docs: add docs for -build-context
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-07 18:37:25 -08:00
Tonis Tiigi
f37c253ae4 commands: provide more helpful error when —build-context is not supported
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-07 18:32:37 -08:00
Tonis Tiigi
d77e2453da commands: rename context flag to build-context
Avoid conflicts with docker context

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-07 18:31:32 -08:00
Tõnis Tiigi
2b4d305c58 Merge pull request #905 from crazy-max/compose-go
compose: fix env
2022-01-06 09:02:26 -08:00
CrazyMax
5d715ada96 compose: resolve build args from service environment
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-06 11:59:49 +01:00
CrazyMax
3400fa5628 compose: test env_file
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-06 11:42:16 +01:00
CrazyMax
f04c8c8430 update github.com/compose-spec/compose-go to v1.0.8
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-01-06 11:41:45 +01:00
Tonis Tiigi
de6b04d726 build: add support for named contexts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2022-01-04 22:55:12 -08:00
Tõnis Tiigi
fe9f9bba87 Merge pull request #884 from crazy-max/fix-bake-resgroup
bake: fix group resolution
2022-01-04 10:50:12 -08:00
Tõnis Tiigi
e482ba2c73 Merge pull request #892 from tonistiigi/warnings-summary
Show summary of build warnings.
2022-01-04 10:48:57 -08:00
Tonis Tiigi
038727477c root: filter out useless debug logs from vendored packages
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-19 22:33:05 -08:00
Tonis Tiigi
ed4103ef52 commands: build summary of warnings on build
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-19 22:32:59 -08:00
Tõnis Tiigi
54286a0117 Merge pull request #889 from tonistiigi/update-buildkit-20211215
vendor: update buildkit to 539be170
2021-12-17 11:58:40 -08:00
Tonis Tiigi
9c3be32bc9 vendor: update buildkit to 539be170
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-12-16 11:42:02 -08:00
Tõnis Tiigi
59533bbb5c Merge pull request #887 from AkihiroSuda/avoid-unneeded-userns-host
docker-container: set UsernsMode only when needed
2021-12-15 21:00:12 -08:00
Akihiro Suda
5f8600f098 docker-container: set UsernsMode only when needed
Set `UsernsMode="host"` only when the daemon is running in userns-remapping mode.

Fix issue 561

The issue will be also fixed in moby/moby PR 43084 (Docker 20.10.13).
This buildx PR helps users of old releases of Docker.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-12-15 18:42:58 +09:00
CrazyMax
d95ebef55c bake: fix group resolution
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-12-14 11:17:37 +01:00
Tõnis Tiigi
33c121df01 Merge pull request #881 from crazy-max/fix-bake-print
bake: fix groups print
2021-12-13 16:09:24 -08:00
Tõnis Tiigi
1dde00c4bc Merge pull request #880 from crazy-max/bake-ignore-field
bake: ignore NetworkMode field for json and hcl
2021-12-13 16:06:41 -08:00
CrazyMax
4466a24f9e bake: fix groups print
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-12-13 19:21:33 +01:00
CrazyMax
ec9daba87e bake: ignore NetworkMode field for json and hcl
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-12-10 19:32:40 +01:00
Sebastiaan van Stijn
202e99695b Merge pull request #823 from crazy-max/docs-git-protoc
docs: fix git protocol
2021-12-09 12:26:17 +01:00
Tõnis Tiigi
7371dda7a2 Merge pull request #863 from zachary-povey/support_host_network_in_compose
Add NetworkMode to bake target
2021-12-04 21:04:05 -08:00
Tõnis Tiigi
62bdf4d85e Merge pull request #868 from crazy-max/discard-containerd-logger
imagetools resolver: discard containerd logger output
2021-12-04 21:03:42 -08:00
Zachary Povey
7f8dbf890d Remove support for network override in bake
Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
2021-12-01 09:45:32 +00:00
Tõnis Tiigi
bede4ab552 Merge pull request #869 from crazy-max/deprecaded-clientopt
imagetools resolver: fix deprecated client opt
2021-11-30 08:29:40 -08:00
CrazyMax
5c5125f30e imagetools resolver: fix deprecated client opt
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-30 15:34:29 +01:00
CrazyMax
e9cf2cbe32 imagetools resolver: discard containerd logger output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-30 15:17:56 +01:00
Zachary Povey
4ee7f70400 Remove NetworkMode support for HCL targets
Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
2021-11-30 11:33:32 +00:00
CrazyMax
0abda783bb Merge pull request #864 from eliottwiener/bake-read-from-stdin
bake: build definition file via stdin
2021-11-30 09:35:03 +01:00
Tõnis Tiigi
aadd118883 Merge pull request #867 from crazy-max/du-last-accessed
disk usage: last accessed not displayed
2021-11-29 09:49:12 -08:00
CrazyMax
9aff9301ce disk usage: last accessed not displayed
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-29 16:21:40 +01:00
CrazyMax
93d6b654ca Merge pull request #866 from Wojciechem/patch-1
add note about variable syntax in json
2021-11-29 09:12:40 +01:00
Wojciech M
42287815b5 add note about variable syntax in json
In addition to HCL, variables can also be defined in json.

Signed-off-by: wojciechem <wmiedzybrodzki@outlook.com>
2021-11-28 22:30:14 +01:00
Eliott Wiener
dcabc22072 bake: build definition file via stdin
closes #833

Accept bake build definition file from stdin with `-f -`.

Signed-off-by: Eliott Wiener <eliottwiener@gmail.com>
2021-11-25 12:43:38 -05:00
Zachary Povey
ae53101e89 Add NetworkMode to bake target
Allows specification of network mode in a bake target.

Fixes #848

Signed-off-by: Zachary Povey <zachary.povey@autotrader.co.uk>
2021-11-25 16:00:42 +00:00
CrazyMax
61627c2ece docs: fix git protocol
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-25 09:07:41 +01:00
Tõnis Tiigi
ab73275f58 Merge pull request #857 from crazy-max/bake-json
bake: fix print output
2021-11-24 15:59:29 -08:00
CrazyMax
316ca972b6 bake: fix print output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-24 21:54:57 +01:00
CrazyMax
5c2b9bbfc5 Merge pull request #861 from tonistiigi/cleanup
commands: clean up unnecessary code
2021-11-23 09:36:55 +01:00
Tonis Tiigi
cc2a879660 commands: clean up unnecessary code
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-22 23:57:42 -08:00
Tõnis Tiigi
89334a88a9 Merge pull request #853 from crazy-max/fix-flags
cli: fix flags usage
2021-11-22 16:21:48 -08:00
Tõnis Tiigi
1927dba42f Merge pull request #852 from DataDog/mayeul/docker-buildx-rm--keep-buildkitd
Add an flag to buildx rm to keep the buildkitd daemon running
2021-11-22 16:21:30 -08:00
Mayeul Blanzat
72dab552b5 Add an option to buildx rm to keep the buildkitd daemon running
Add --keep-daemon to the `rm` command option to preserve the buildkitd daemon after the buildx context is deleted.

Signed-off-by: Mayeul Blanzat <mayeul.blanzat@datadoghq.com>
2021-11-22 13:24:47 +01:00
CrazyMax
a0a7db127c cli: fix flags usage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-22 10:51:59 +01:00
Tõnis Tiigi
bcfd434829 Merge pull request #839 from crazy-max/xx-update
dockerfile: update xx to 1.0.0
2021-11-13 19:01:51 -08:00
CrazyMax
d1aaed7a77 dockerfile: update xx to 1.0.0
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-13 18:53:01 +01:00
CrazyMax
f0026081a7 Merge pull request #832 from crazy-max/update-compose
update github.com/compose-spec/compose-go to v1.0.5
2021-11-10 11:03:28 +01:00
CrazyMax
a18829f837 update github.com/compose-spec/compose-go to v1.0.5
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-09 10:16:17 +01:00
CrazyMax
da0eb138d0 Merge pull request #829 from crazy-max/typo
fix typo in docs
2021-11-05 09:17:23 +01:00
CrazyMax
a2c7d43e46 fix typo in docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-05 08:48:32 +01:00
Akihiro Suda
f7cba04f5e Merge pull request #828 from tonistiigi/strip-binary
Dockerfile: strip binary by default
2021-11-05 14:25:22 +09:00
Tonis Tiigi
12b5db70e2 Dockerfile: strip binary by default
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-04 18:11:26 -07:00
Tõnis Tiigi
5c4e3fc860 Merge pull request #818 from crazy-max/fix-builder-flag
cli: fix builder persistent flag
2021-11-04 12:00:43 -07:00
CrazyMax
eab0e6a8fe cli: fix builder persistent flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-04 18:52:17 +01:00
CrazyMax
4c938c77ba Merge pull request #825 from tonistiigi/multi-node-registry-conf
allow multi-node push and imagetools to use custom registry config
2021-11-04 18:43:31 +01:00
Tonis Tiigi
1cca41b81a build: support insecure export option for multi-node build
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-04 10:02:14 -07:00
Tonis Tiigi
c62472121b allow multi-node push and imagetools to use custom registry config
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-04 10:02:12 -07:00
Tonis Tiigi
88d0775692 refactor accessing registry configs via drivers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-04 10:01:23 -07:00
Tõnis Tiigi
8afc82b427 Merge pull request #817 from tonistiigi/project-sharedkey
build: set local sharedkey per project basename
2021-11-04 09:52:51 -07:00
Tõnis Tiigi
d311561a8b Merge pull request #824 from tonistiigi/config-files-store2
store snapshot of config files on create
2021-11-04 09:05:13 -07:00
Tõnis Tiigi
44e180b26e Merge pull request #826 from tonistiigi/multi-node-iidfile 2021-11-04 08:24:33 -07:00
Tonis Tiigi
02d29e0af5 build: fix setting iidfile with multi-node push
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-03 23:12:00 -07:00
Tonis Tiigi
40121c671c kubernetes: store config files for k8s
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-03 18:24:16 -07:00
Tonis Tiigi
4c1621cccd store snapshot of config files on create
Files can be reused when container needs to be booted again.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-11-03 16:34:43 -07:00
Tõnis Tiigi
7f0e37531c Merge pull request #822 from crazy-max/fix-bake-git-protoc
bake: fix protocol detection
2021-11-02 16:20:22 -07:00
CrazyMax
82b212bddf bake: fix protocol detection
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-11-02 23:12:27 +01:00
Tonis Tiigi
aa52a5a699 build: set local sharedkey per project basename
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-29 21:29:04 -07:00
Tõnis Tiigi
49342dd54d Merge pull request #787 from crazy-max/inject-certs
container driver: copy ca and user tls registries certs
2021-10-28 15:11:57 -07:00
CrazyMax
3f716f00fa container driver: copy ca and user tls registries certs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-28 22:30:11 +02:00
Tõnis Tiigi
5e25191cb6 Merge pull request #814 from crazy-max/cgroup-parent
build: add cgroup-parent support
2021-10-28 10:53:09 -07:00
CrazyMax
dd15969c93 build: add cgroup-parent support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-28 19:34:47 +02:00
Tõnis Tiigi
81cf2064c4 Merge pull request #815 from tonistiigi/multi-node-push-names
imagetools: fix pushing same image with multiple names
2021-10-28 10:21:30 -07:00
Tonis Tiigi
b497587f21 imagetools: fix pushing same image with multiple names
containerd pusher can’t handle this case atm so we
need to make sure we always create a new resolver
for each name.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-27 17:14:49 -07:00
CrazyMax
2890209a11 refactor: lexical order for build opts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-27 23:42:16 +02:00
Tõnis Tiigi
4690e14c40 Merge pull request #810 from crazy-max/warn-flags-depre
build: warning msg on deprecated flags
2021-10-26 16:25:19 -07:00
CrazyMax
25d2f73858 build: warning on deprecated flags
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-26 21:36:49 +02:00
CrazyMax
36a37a624e refactor: flags lexical order
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-26 13:11:28 +02:00
CrazyMax
e150d7bdd8 add long description to root command
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-26 13:11:28 +02:00
Tõnis Tiigi
be2c8f71fe Merge pull request #812 from cpuguy83/mark_spans_as_error
Mark span status as error when fatal error occurs.
2021-10-25 20:36:35 -07:00
Akihiro Suda
89f5c1ce51 Merge pull request #804 from tonistiigi/http-hijack-session
docker: dial session directly with http hijack
2021-10-22 13:10:26 +09:00
Brian Goff
b6474d43a9 Mark span status as error when fatal error occurs.
Before this only recorded errors instead of setting the span status,
which makes it harder to dig through.
Now an error that bubbles is reflected in the span status.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-10-21 22:24:51 +00:00
Tonis Tiigi
2644d56a6d docker: dial session directly with http hijack
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-10-20 18:54:31 -07:00
CrazyMax
084b6c0a95 Merge pull request #790 from crazy-max/shmsize
build: add shm-size support
2021-10-20 06:24:01 +02:00
CrazyMax
8e5595b7c7 build: add shm-size support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 21:06:01 +02:00
CrazyMax
22500c9929 vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-19 21:03:23 +02:00
Tõnis Tiigi
050f4f9219 Merge pull request #801 from crazy-max/bump-go
update go to 1.17.2
2021-10-18 13:26:07 -07:00
CrazyMax
1a56de8e68 update go to 1.17.2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-16 15:00:31 +02:00
Tõnis Tiigi
868610e0e9 Merge pull request #800 from crazy-max/ulimit
build: add ulimit support
2021-10-15 11:14:49 -07:00
CrazyMax
b89e2f35df build: add ulimit support
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-15 16:03:49 +02:00
CrazyMax
1b3068df7c vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-10-15 15:01:36 +02:00
Tõnis Tiigi
461369748c Merge pull request #782 from djs55/cgroup-parent
docker-container: place build containers in a separate cgroup
2021-09-30 09:06:29 -07:00
David Scott
d5908cdddf docker-container: use /docker/buildx cgroup by default
This allows resource limits to be applied to all builds on a host.
For example to limit the total amount of CPU used by builds:

https://medium.com/@asishrs/docker-limit-resource-utilization-using-cgroup-parent-72a646651f9d

Signed-off-by: David Scott <dave@recoil.org>
2021-09-29 19:58:22 +01:00
David Scott
b5bc754bad docker-container: support --driver-opt cgroup-parent=...
This allows the parent cgroup to be customised, which allows resource
limits to be imposed on build containers separately from "user"
containers.

Signed-off-by: David Scott <dave@recoil.org>
2021-09-29 19:57:46 +01:00
Tõnis Tiigi
dff7673afb Merge pull request #783 from tonistiigi/override-merge
bake: restore previous override merge behavior
2021-09-29 08:59:54 -07:00
Tõnis Tiigi
3e2fde5639 Merge pull request #785 from crazy-max/fix-docs
Fix --driver flag usage markdown output
2021-09-29 08:59:14 -07:00
Tonis Tiigi
7a7b73c043 bake: restore previous override merge behavior
For array fields, overrides are merged together
but override is not merged with the target. If merging
with target is desired we can add support for
overrides with += operator in the future.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-29 08:27:27 -07:00
Tõnis Tiigi
e50c9ae7be Merge pull request #784 from crazy-max/fix-e2e
Driver opt not used in e2e tests
2021-09-29 08:18:49 -07:00
CrazyMax
9e62c9f074 Fix --driver flag usage markdown output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-29 15:33:44 +02:00
CrazyMax
c82dbafaee Driver opt not used in e2e tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-29 10:42:27 +02:00
Tonis Tiigi
0e4d7aa7a9 bake: add test for merging overrides
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-28 23:21:07 -07:00
Tõnis Tiigi
c05a6eb2c1 Merge pull request #781 from crazy-max/vendor-buildkit
vendor: update buildkit
2021-09-27 19:01:49 -07:00
CrazyMax
eec1693f30 vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-27 21:54:35 +02:00
CrazyMax
c643c2ca95 Merge pull request #775 from crazy-max/vendor-buildkit
vendor: update buildkit
2021-09-25 18:48:36 +02:00
Tõnis Tiigi
761e22e395 Merge pull request #774 from crazy-max/driver-e2e
Driver e2e tests
2021-09-23 16:49:55 -07:00
CrazyMax
ef8c936b27 Driver e2e tests
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-23 10:49:30 +02:00
Tõnis Tiigi
0cea838344 Merge pull request #773 from tonistiigi/bake-push-fix
bake: fix using push override with output definition
2021-09-21 17:53:45 -07:00
Akihiro Suda
2b18a9b4a5 Merge pull request #761 from morlay/kubeconfig-enhance
util: support load Colon-separated KUBECONFIG
2021-09-21 16:45:55 +09:00
CrazyMax
45e4550c36 vendor: update buildkit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-21 07:49:45 +02:00
Tonis Tiigi
6fc906532b bake: fix using push override with output definition
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-09-20 16:22:27 -07:00
Tõnis Tiigi
06541ebd0f Merge pull request #765 from thaJeztah/fix_broken_anchors
docs: fix some broken anchors
2021-09-15 16:00:21 -07:00
Sebastiaan van Stijn
773fac9a73 docs: fix some broken anchors
- ./_site/engine/reference/commandline/buildx_build/index.html
      *  linking to internal hash #--buildkitd-flags-flags that does not exist (line 904)
         <a href="/engine/reference/commandline/buildx_create/#--buildkitd-flags-flags"></a>
    - ./_site/engine/reference/commandline/buildx_create/index.html
      *  linking to internal hash #--buildkitd-flags-flags that does not exist (line 350)
         <a href="#--buildkitd-flags-flags"></a>
      *  linking to internal hash #--config-file that does not exist (line 336)
         <a href="#--config-file"></a>
      *  linking to internal hash #--config-file that does not exist (line 336)
         <a href="/engine/reference/commandline/buildx_build/#--load"></a>
      *  linking to internal hash #--load that does not exist (line 369)
         <a href="/engine/reference/commandline/buildx_build/#--load"></a>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-09-13 17:06:18 +02:00
CrazyMax
7f0e05dfac Merge pull request #762 from crazy-max/remove-yaml
Remove generated YAML docs
2021-09-09 13:39:53 +02:00
CrazyMax
e59aecf034 Remove YAML docs from the repo
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-09 10:29:13 +02:00
CrazyMax
ac9a1612d2 Merge pull request #758 from crazy-max/mod-outdated
Mod outdated
2021-09-08 13:33:26 +02:00
Morlay
c83812144c util: support load Colon-separated KUBECONFIG
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-09-08 17:51:56 +08:00
CrazyMax
df521e4e96 Mod outdated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-04 18:57:48 +02:00
Tõnis Tiigi
00cb53d0ef Merge pull request #746 from crazy-max/bake-workflow
Bake workflow
2021-09-03 14:11:54 -07:00
CrazyMax
6cfef7fa36 Bake workflow
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-03 22:50:05 +02:00
Tõnis Tiigi
b05c313204 Merge pull request #740 from bossmc/support-quiet
Implement `--quiet` support
2021-09-03 11:00:26 -07:00
CrazyMax
3e8bbbc286 Merge pull request #757 from crazy-max/bump-clidocstool
Fix flags usage markdown output
2021-09-03 14:25:21 +02:00
CrazyMax
8a12884814 Fix flags usage markdown output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-03 11:52:49 +02:00
Tõnis Tiigi
6cf9fa8261 Merge pull request #739 from crazy-max/go117
Go 1.17
2021-09-02 16:01:45 -07:00
Tõnis Tiigi
fd94fc5fdf Merge pull request #748 from crazy-max/platform-func
Built-in variable `BAKE_LOCAL_PLATFORM`
2021-09-02 16:01:26 -07:00
CrazyMax
45c678ad26 Go 1.17
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-02 23:11:10 +02:00
CrazyMax
55a3ce606f Built-in variable BAKE_LOCAL_PLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-09-02 23:06:05 +02:00
CrazyMax
c1c414e4c9 Merge pull request #747 from crazy-max/yaml-docs
Generate YAML doc
2021-09-02 20:16:56 +02:00
Tõnis Tiigi
610601cec0 Merge pull request #753 from tonistiigi/ci-branches
github: fix running ci in version branches
2021-08-28 10:08:06 -07:00
Tonis Tiigi
9833420a03 github: fix running ci in version branches
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-27 21:19:11 -07:00
Tõnis Tiigi
7f322caa79 Merge pull request #751 from thaJeztah/fix_mount_path
container-driver: fix volume destination for cache
2021-08-27 21:02:09 -07:00
Sebastiaan van Stijn
93867d02f0 container-driver: fix volume destination for cache
The container-driver creates a Linux container (as there currently isn't a
Windows version of buildkitd). However, the defaults are platform specific.

Buildx was using the defaults from the buildkit `util/appdefault' package,
which resulted in Buildx running on a Windows client to create a Linux
container that used the Windows location, which causes it to fail:

    invalid mount config for type "volume": invalid mount path: 'C:/ProgramData/buildkitd/.buildstate' mount path must be absolute

This patch hard-codes the destination to the default Linux path.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 09:48:31 +02:00
CrazyMax
b8a602821c Generate YAML doc
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-25 19:10:03 +02:00
Tõnis Tiigi
a8a3b1738e Merge pull request #741 from tonistiigi/client-ctx
use long-running context for client initialization
2021-08-20 09:01:53 -07:00
Andy Caldwell
ef3e46fd62 Move printing to stdout up to the command itself
Signed-off-by: Andy Caldwell <andrew.caldwell@metaswitch.com>
2021-08-20 15:13:23 +01:00
Andy Caldwell
3ab0b6953a Regenerate docs now that --quiet is unmasked
Signed-off-by: Andy Caldwell <andrew.caldwell@metaswitch.com>
2021-08-20 15:13:19 +01:00
Andy Caldwell
c19c018a4c Implement --quiet support
Signed-off-by: Andy Caldwell <andrew.caldwell@metaswitch.com>
2021-08-20 15:13:13 +01:00
Tonis Tiigi
422ba60b04 use long-running context for client initialization
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-08-19 20:36:24 -07:00
Tõnis Tiigi
2d3763990c Merge pull request #731 from crazy-max/dockerfile-13
Update Dockerfile references to use 1.3
2021-08-18 09:05:53 -07:00
CrazyMax
dc6ada9b50 Update Dockerfile references to use 1.3
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-17 08:41:15 +02:00
Tõnis Tiigi
cb185f095f Merge pull request #721 from crazy-max/compose-ext
bake: `x-bake` extension field with compose
2021-08-13 07:51:23 -07:00
CrazyMax
89e126fa60 bake: x-bake extension field with compose
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-13 09:15:15 +02:00
Tõnis Tiigi
04bac63745 Merge pull request #692 from crazy-max/boostrap-cmd
Allow booting builder after creation
2021-08-12 10:35:33 -07:00
Tõnis Tiigi
3594851128 Merge pull request #720 from crazy-max/default-group
bake: print default group
2021-08-12 10:31:16 -07:00
Tõnis Tiigi
58e5a73389 Merge pull request #707 from crazy-max/update-readme
Enhance readme
2021-08-12 10:30:10 -07:00
CrazyMax
c685e46609 bake: print default group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-12 09:02:36 +02:00
CrazyMax
e3283e6169 Enhance readme
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-12 08:57:09 +02:00
CrazyMax
5d50bd7b43 Allow booting builder after creation
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-12 08:45:55 +02:00
Tõnis Tiigi
3dfbe2c184 Merge pull request #674 from crazy-max/checksum
Create checksums for artifacts
2021-08-10 13:16:13 -07:00
Tõnis Tiigi
06367a120b Merge pull request #682 from morlay/k8s-enhance
[kubernetes] Support --config to mount buildkit.toml and -driver-opt=qemu.install=true,qemu.image=tonistiigi/binfmt:latest for qemu installing
2021-08-10 13:15:32 -07:00
CrazyMax
6149507c7e Merge pull request #670 from crazy-max/cache-gha-doc
Example with gha cache
2021-08-09 13:39:59 +02:00
Morlay
c76b5eac03 feat(driver/kubernetes): support mount buildkit.toml and qemu installing
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-08-04 21:32:27 +08:00
CrazyMax
cd133cee25 Merge pull request #700 from crazy-max/bake-docs
Update bake docs
2021-08-03 14:53:43 +02:00
CrazyMax
eeab638476 Merge pull request #710 from crazy-max/update-ctnd
bump github.com/containerd/containerd from 1.5.4 to 1.5.5
2021-08-02 14:32:44 +02:00
CrazyMax
19b9b86af8 bump github.com/containerd/containerd from 1.5.4 to 1.5.5
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-02 13:47:27 +02:00
CrazyMax
0101c96532 Update cache docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-02 11:05:42 +02:00
CrazyMax
85dedf1aea Create checksums for artifacts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-02 03:37:36 +02:00
CrazyMax
5f05bd9a2b Update bake docs
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-08-01 03:21:20 +02:00
CrazyMax
260d07a9a1 Merge pull request #704 from crazy-max/use-compose-config-file
Set `ConfigFile` to parse compose files with bake
2021-07-30 18:35:00 +02:00
CrazyMax
9aa8f09f14 Set ConfigFile to parse compose files with bake
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-30 17:56:40 +02:00
CrazyMax
0363b676bc Merge pull request #697 from docker/dependabot/go_modules/github.com/containerd/containerd-1.5.4
build(deps): bump github.com/containerd/containerd from 1.5.2 to 1.5.4
2021-07-28 14:09:33 +02:00
dependabot[bot]
a10045e8cb build(deps): bump github.com/containerd/containerd from 1.5.2 to 1.5.4
Bumps [github.com/containerd/containerd](https://github.com/containerd/containerd) from 1.5.2 to 1.5.4.
- [Release notes](https://github.com/containerd/containerd/releases)
- [Changelog](https://github.com/containerd/containerd/blob/main/RELEASES.md)
- [Commits](https://github.com/containerd/containerd/compare/v1.5.2...v1.5.4)

---
updated-dependencies:
- dependency-name: github.com/containerd/containerd
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-07-26 21:37:28 +00:00
CrazyMax
0afcca221d Merge pull request #694 from crazy-max/codecov
Bump to codecov/codecov-action v2
2021-07-26 10:22:00 +02:00
CrazyMax
5daf176722 Merge pull request #693 from crazy-max/buildkit-progress
Duplicated progress env var
2021-07-25 22:18:29 +02:00
CrazyMax
3d1ab82dc6 Duplicated progress env var
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-25 12:40:13 +02:00
CrazyMax
872430d2d3 Bump to codecov/codecov-action v2
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-24 23:24:12 +02:00
CrazyMax
7d312eaa0a Merge pull request #686 from morlay/nil-client-fix
fix: should ignore nil client
2021-07-23 20:12:32 +02:00
CrazyMax
a6bc4ed21e Merge pull request #679 from akvadrako/patch-1
Fix link to generally useful functions
2021-07-23 08:19:53 +02:00
Morlay
3768ab268b fix: should ignore nil client
Signed-off-by: Morlay <morlay.null@gmail.com>
2021-07-21 15:47:55 +08:00
Devin Bayer
4c2daeb852 Fix link to generally useful functions
Signed-off-by: Devin Bayer <dev@doubly.so>
2021-07-20 18:09:53 +02:00
Tõnis Tiigi
d9ee3b134c Merge pull request #676 from tonistiigi/vol-delete
don't error on deleting old build containers without state volume
2021-07-16 12:37:37 -07:00
Tonis Tiigi
0b6ba1cd32 don't error on deleting old build containers without state volume
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-16 12:00:27 -07:00
Tõnis Tiigi
65a6955db8 Merge pull request #672 from crazy-max/keep-buildkit-state
Keep BuildKit state in a volume
2021-07-13 10:21:51 -07:00
CrazyMax
258d12b2e7 Keep BuildKit state in a volume
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-13 18:09:35 +02:00
Tõnis Tiigi
6e3a319a9d Merge pull request #671 from tonistiigi/remote-context
bake: allow BAKE_CMD_CONTEXT builtin var
2021-07-13 08:50:27 -07:00
Tonis Tiigi
1bb425a882 bake: allow BAKE_CMD_CONTEXT builtin var
Allows accessing the main context for bake command from bake
file that has been imported remotely.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 21:01:41 -07:00
Tõnis Tiigi
5f6ad50df4 Merge pull request #635 from tonistiigi/otel
OpenTelemetry support
2021-07-12 14:27:32 -07:00
Tonis Tiigi
9d88450118 enable opentelemetry support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 13:42:52 -07:00
Tonis Tiigi
334c93fbbe vendor: update buildkit to opentelemetry support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-07-12 13:42:45 -07:00
Tõnis Tiigi
6ba080d337 Merge pull request #669 from crazy-max/compose-spec
Use compose-spec parser
2021-07-12 09:46:13 -07:00
CrazyMax
ba443811e4 Use compose-spec parser
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-12 11:21:04 +02:00
CrazyMax
67bd6f4dc8 Merge pull request #659 from crazy-max/release-out
Ignore release-out folder
2021-07-02 11:12:46 +02:00
CrazyMax
9f50eccbd7 Ignore release-out folder
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-07-02 07:18:18 +02:00
Tõnis Tiigi
12db50748b Merge pull request #645 from tonistiigi/new-bake-parser
New bake parser
2021-07-01 17:09:45 -07:00
CrazyMax
9b4937f062 Merge pull request #656 from crazy-max/buildx-image
Create buildx image
2021-06-30 20:22:38 +02:00
CrazyMax
3d48359e95 Create buildx image
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-06-30 16:01:28 +02:00
CrazyMax
70002ebbc7 Merge pull request #657 from crazy-max/moby-ioutils
Use `ioutils.AtomicWriteFile` from moby
2021-06-30 09:20:38 +02:00
CrazyMax
ef95f8135b Use ioutils.AtomicWriteFile from moby
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-06-30 08:34:45 +02:00
CrazyMax
9215fc56a3 Merge pull request #605 from crazy-max/bake-iidfile
Add `metadata-file` flag
2021-06-30 08:25:33 +02:00
CrazyMax
1253020b3d Add metadata-file flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-06-30 07:41:26 +02:00
Tõnis Tiigi
621c55066c Merge pull request #654 from tonistiigi/enable-win-arm64
Dockerfile: enable windows/arm64
2021-06-29 10:52:29 -07:00
Tonis Tiigi
77632ac15f Dockerfile: enable windows/arm64
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-29 08:55:05 -07:00
Tõnis Tiigi
db6aa34252 Merge pull request #653 from tonistiigi/xx-v2
Dockerfile: update xx
2021-06-29 08:53:25 -07:00
CrazyMax
7ecfd3d298 Merge pull request #652 from tonistiigi/enable-riscv64
enable linux/riscv64 builds
2021-06-29 09:52:44 +02:00
Tonis Tiigi
9a8c287629 Dockerfile: update xx
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-28 18:30:57 -07:00
Tonis Tiigi
591099a4b8 enable linux/riscv64 builds
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-28 18:16:53 -07:00
Tõnis Tiigi
31309b9205 Merge pull request #582 from rumpl/feat-fail-fast
Fail fast on multi platform build with load
2021-06-28 09:29:12 -07:00
CrazyMax
8c0cefcd89 Merge pull request #649 from admackin/patch-3
note that buildx not in all distros
2021-06-28 03:45:27 +02:00
Andy MacKinlay
a07f5cdf42 note that buildx not in all distros
Signed-off-by: Andy MacKinlay <admackin@gmail.com>
2021-06-28 09:41:19 +10:00
Djordje Lukic
a1d899d400 Fail fast on multi platform build with load
Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
2021-06-27 16:18:08 +02:00
CrazyMax
886e1a378c Merge pull request #639 from eitsupi/fix-link
Minor fix to links in the document
2021-06-25 14:51:31 +02:00
SHIMA Tatsuya
47b7ba4e79 change the default branch name of the linked repo
Signed-off-by: SHIMA Tatsuya <ts1s1andn@gmail.com>
2021-06-25 19:17:40 +09:00
Tonis Tiigi
79433cef7a bake: fix target merge between compose and hcl
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-24 22:40:32 -07:00
Tonis Tiigi
c5eb8f58b4 bake: new hclparser package
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-06-24 22:40:25 -07:00
Tõnis Tiigi
03b7128b60 Merge pull request #575 from tonistiigi/user-func-vars
bake: allow user functions in variables and vice-versa
2021-06-23 11:08:56 -07:00
Tõnis Tiigi
15b358bec6 Merge pull request #618 from MichalAugustyn/handle-resources
feat: add resources handling to kubernetes driver
2021-05-29 22:25:40 -07:00
Michal Augustyn
a53e392afb feat: add resources handling to kubernetes driver
Signed-off-by: Michal Augustyn <michal.augustyn@mail.com>
2021-05-29 13:45:52 +02:00
Akihiro Suda
4fec647b9d Merge pull request #613 from tonistiigi/docs-update
readme: update installation instructions
2021-05-19 15:50:28 +09:00
Tonis Tiigi
d7b28fb4d3 docs: update bake notes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-17 22:53:56 -07:00
Tonis Tiigi
9bc9291fc9 readme: update installation instructions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-17 22:33:08 -07:00
Tonis Tiigi
df7a318ec0 bake: allow user functions in variables and vice-versa
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-05-05 22:08:31 -07:00
Tõnis Tiigi
908a856079 Merge pull request #590 from AkihiroSuda/split-flagparser
build: split buildflags package
2021-04-22 11:50:57 -07:00
Akihiro Suda
8d64b6484f Merge pull request #592 from tonistiigi/descriptor-merge
imagetools: fix merging JSON descriptor with old one
2021-04-15 21:23:56 +09:00
Akihiro Suda
399df854ea build: split buildflags package
Planned to be imported by nerdctl in future.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-12 14:36:56 +09:00
Tonis Tiigi
328441cdc6 imagetools: fix merging JSON descriptor with old one
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-04-09 11:02:16 -07:00
Tõnis Tiigi
5ca0cbff8e Merge pull request #535 from tonistiigi/github-actions-cache
allow exporting to github cache backend
2021-04-09 10:30:46 -07:00
Tõnis Tiigi
ab09846df7 Merge pull request #583 from thaJeztah/docs_fixes
docs: address some review comments from the docs repository
2021-04-07 08:07:49 -07:00
Sebastiaan van Stijn
cd3a9ad38d docs: address some review comments from the docs repository
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-07 16:04:11 +02:00
Tõnis Tiigi
adc5f35237 Merge pull request #581 from alexcb/use-default-ssh-when-missing-v2
include default ssh socket when given an ssh-based git url
2021-04-01 19:32:18 -07:00
Alex Couture-Beil
0b984e429b Update buildkit
- updated buildkit to current code in master via:

  go mod edit -require github.com/moby/buildkit@master && go mod tidy && ./hack/update-vendor

Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2021-04-01 16:38:16 -07:00
Alex Couture-Beil
eec843a325 include default ssh socket when given an ssh-based git url
Signed-off-by: Alex Couture-Beil <alex@earthly.dev>
2021-04-01 11:25:18 -07:00
Tonis Tiigi
83868a48b7 temp local copy of userfunc
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-26 17:06:38 -07:00
Tõnis Tiigi
98d337af21 Merge pull request #541 from tonistiigi/json-vars-attrs
bake: allow attributes in global scope
2021-03-26 17:06:19 -07:00
Tõnis Tiigi
b2c7dc00cc Merge pull request #570 from thaJeztah/fix_trailing_whitespace
remove trailing whitespace in command descriptions
2021-03-26 16:47:29 -07:00
Sebastiaan van Stijn
44ddc5a02b remove trailing whitespace in command descriptions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 23:19:15 +01:00
Tonis Tiigi
f036bba48c bake: add test for json vars and attributes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-25 11:00:17 -07:00
Tonis Tiigi
0fe2ce7fac bake: allow attributes in global scope
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-25 11:00:17 -07:00
Tõnis Tiigi
0147b92230 Merge pull request #506 from thaJeztah/split_docs
Split docs to separate files
2021-03-24 22:14:43 -07:00
Tonis Tiigi
4047bccf6c docs: add external docs links support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-25 00:44:29 +01:00
Tonis Tiigi
363c0fdf4b hack: add docs generation/validation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-25 00:44:26 +01:00
Tonis Tiigi
c46407b2d3 docs: reference docs updates
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:44:17 +01:00
Tonis Tiigi
ca0f5dabea docs: add md generation
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-25 00:38:04 +01:00
Sebastiaan van Stijn
17d4106e1b docs: fix markdown formatting and highlighting
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:38:02 +01:00
Sebastiaan van Stijn
442d38080e docs: use "examples" section, and rephrase headings
Put the flag descriptions/examples under an "examples"
section (used at docs.docker.com), and rephrase the
headings to be more consistent with other pages in the
docker documentation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:38:01 +01:00
Sebastiaan van Stijn
87ec3af5bb docs: add stubs for all commands
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:37:59 +01:00
Sebastiaan van Stijn
1a8af33ff6 docs: standardize format for usage
Use the usage output of `--help` for each subcommand, to make
sure all flags/options are included on the page, and to make
it easier to keep docs in sync.

Note that the usage output is only used when reading these
docs on GitHub; docs.docker.com only consumes the "description"
and "example" sections (when present), and generates flag information
and usage output from source

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:37:57 +01:00
Sebastiaan van Stijn
ff749d8863 docs: split reference docs to separate files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:37:53 +01:00
Tõnis Tiigi
2d86ddd37f Merge pull request #569 from thaJeztah/dont_discard_manifest_errors
imagetools inspect: don't discard errors from PrintManifestList
2021-03-24 09:23:43 -07:00
Sebastiaan van Stijn
e1bbb9d8de imagetools inspect: don't discard errors from PrintManifestList
Looks like this function may return an error, which we currently discard.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-24 15:54:32 +01:00
Tonis Tiigi
d7964be29c gha cache caps detection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-23 08:58:50 -07:00
Tonis Tiigi
3fef64f584 allow exporting to github cache backend
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-23 08:58:50 -07:00
Akihiro Suda
319b6503a5 Merge pull request #565 from tonistiigi/buildkit-update-0322
vendor: update buildkit to 8effd45b
2021-03-23 18:32:30 +09:00
Tonis Tiigi
d40a6082fa vendor: update buildkit to 8effd45b
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-22 15:54:07 -07:00
Akihiro Suda
28809b82a2 Merge pull request #539 from tonistiigi/vars-cross-refs
bake: allow variables to reference each other
2021-03-19 18:34:29 +09:00
Tõnis Tiigi
c9f02c32d4 Merge pull request #558 from ulyssessouza/add-quiet-mode-to-progress-printer
Add quiet mode to progress printer
2021-03-09 13:07:48 -08:00
Ulysses Souza
55d5b80dfe Add quiet mode to progress printer
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2021-03-09 17:12:46 -03:00
Tonis Tiigi
33f25acb08 bake: allow variables to reference each other
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-07 21:27:30 -08:00
Tonis Tiigi
0e9066f6ed bake: fix hcl tests layout
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-03-01 12:08:09 -08:00
Akihiro Suda
7d2e30096b Merge pull request #538 from tonistiigi/vars-multi-file 2021-02-27 18:11:36 +09:00
Tonis Tiigi
0e9d6460db bake: allow variables across files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-25 16:30:27 -08:00
Tõnis Tiigi
927163bf13 Merge pull request #481 from fauust/patch-1
Update README.md
2021-02-25 16:05:16 -08:00
Tibor Vass
8ac1cf6e45 Merge pull request #545 from tonistiigi/update-go116
update to go1.16
2021-02-23 08:41:34 -08:00
Tonis Tiigi
dba79ba223 update lint to go1.16/golangci
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-16 23:43:06 -08:00
Tonis Tiigi
905be6431b Dockerfile: update to go1.16
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-16 23:42:59 -08:00
Tõnis Tiigi
ad95d6ba04 Merge pull request #532 from tonistiigi/vars-in-func
bake: allow variables in user functions
2021-02-10 10:06:13 -08:00
Tõnis Tiigi
b77690a373 Merge pull request #540 from tonistiigi/maintainers-update
add akihiro and crazy-max to maintainers
2021-02-10 09:52:50 -08:00
Akihiro Suda
84a734dc87 Merge pull request #531 from tonistiigi/net-none 2021-02-11 02:34:46 +09:00
Akihiro Suda
5079b64ab5 Merge pull request #536 from tonistiigi/hcl-update 2021-02-11 02:32:09 +09:00
Tonis Tiigi
6a343488d2 add akihiro and crazy-max to maintainers
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-10 09:18:05 -08:00
Tonis Tiigi
98c3ef60e6 vendor: update hcl
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-08 10:18:14 -08:00
faust
73fa351b1c Update README.md
typo

Signed-off-by: Faustin Lammler <faustin@fala.red>
2021-02-08 12:26:18 +01:00
Tibor Vass
c88f7fc307 Merge pull request #529 from tonistiigi/darwin-arm-ci
ci: enable building darwin/arm64 in CI
2021-02-05 08:54:06 -08:00
Tõnis Tiigi
55b8712268 Merge pull request #530 from HollowMan6/patch-1
Fix typos in README.md
2021-02-05 00:51:02 -08:00
Tonis Tiigi
7878f0c514 bake: allow variables in user functions
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-04 23:27:09 -08:00
Tonis Tiigi
0f09e2ecfe don't require entitlement for network none
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-04 22:45:20 -08:00
Hollow Man
bea3acd4b6 Fix typos
defintion -> definition
avaialble -> available
registed -> registered

Signed-off-by: Hollow Man <hollowman@hollowman.ml>
2021-02-05 14:26:26 +08:00
Tonis Tiigi
fb9004d6b2 ci: enable building darwin/arm64 in CI
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-04 19:00:41 -08:00
Tibor Vass
42b7e7bc56 Merge pull request #526 from tonistiigi/darwin-arm64
Dockerfile: add darwin/arm64 support
2021-02-04 17:07:07 -08:00
Tibor Vass
4b2ddd5b6e Merge pull request #527 from tonistiigi/avoid-cp
avoid cp in install target
2021-02-04 17:04:23 -08:00
Tonis Tiigi
b3006221f1 avoid cp in install target
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-04 13:29:16 -08:00
Tonis Tiigi
e57108e7c9 Dockerfile: add darwin/arm64 support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2021-02-04 12:23:23 -08:00
Tõnis Tiigi
6b3dc6687b Merge pull request #517 from tiborvass/execabs
Use golang.org/x/sys/execabs
2021-01-28 15:47:19 -08:00
Tibor Vass
92f6f9f973 Use golang.org/x/sys/execabs
Signed-off-by: Tibor Vass <tibor@docker.com>
2021-01-27 21:09:53 +00:00
Tõnis Tiigi
a56a4c00dd Merge pull request #503 from felipecrs/patch-1
Add setproduct function to the bake HCL
2021-01-19 16:54:35 -08:00
Tõnis Tiigi
ee4a115d4c Merge pull request #505 from thaJeztah/bump_cobra
vendor: github.com/spf13/cobra v1.1.1
2021-01-19 16:53:11 -08:00
Sebastiaan van Stijn
976a58c918 vendor: github.com/spf13/cobra v1.1.1
v1.1.1:

- Fix: yaml.v2 2.3.0 contained a unintended breaking change. This release reverts
  to yaml.v2 v2.2.8 which has recent critical CVE fixes, but does not have the
  breaking changes.
- Fix: correct internal formatting for go-md2man v2 (which caused man page
  generation to be broken).

v1.1.0:

- Extend Go completions and revamp zsh comp
- Add completion for help command
- Complete subcommands when TraverseChildren is set
- Fix stderr printing functions
- fix: fish output redirection

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-13 11:18:25 +01:00
Tõnis Tiigi
db82aa1b77 Merge pull request #504 from crazy-max/default-progress
Allow to set default progress through env var
2021-01-11 09:33:33 -08:00
CrazyMax
d05504c50f Allow to set default progress through env var
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2021-01-11 01:41:44 +01:00
Felipe Santos
f1f464e364 Add setproduct function to the bake HCL
As explained in the following link, it's a very useful function.

https://www.terraform.io/docs/configuration/functions/setproduct.html#finding-combinations-for-for_each
Signed-off-by: Felipe Santos <felipecassiors@gmail.com>
2021-01-10 06:11:14 +00:00
Tõnis Tiigi
57b875a955 Merge pull request #491 from crazy-max/hcl2-funcs
Extend hcl2 support with more functions
2021-01-04 10:44:23 -08:00
Tõnis Tiigi
ea5d32ddff Merge pull request #488 from crazy-max/secrets-env
Allow secrets with env
2021-01-04 10:40:09 -08:00
Tõnis Tiigi
da8c8ccaf5 Merge pull request #497 from morlay/k8s-driver-configuration-bind
feat: store kube config file to make buildx builder switchable
2021-01-04 10:39:14 -08:00
Tõnis Tiigi
dcbe4b3e1a Merge pull request #477 from morlay/master
should list real pod nodes when all drivers are kubernetes
2021-01-04 10:37:55 -08:00
Wang
68cebffe13 feat: store kube config file to make buildx builder switchable
Signed-off-by: Wang <morlay.null@gmail.com>
2020-12-30 17:28:14 +08:00
CrazyMax
96e7f3224a Extend hcl2 support with more functions
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2020-12-23 12:33:55 +01:00
CrazyMax
f6d83c97bb Allow secrets with env
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2020-12-19 04:18:51 +01:00
Tõnis Tiigi
74f76cf4e9 Merge pull request #480 from AkihiroSuda/bkimage-rootless
set DefaultRootlessImage to "moby/buildkit:buildx-stable-1-rootless"
2020-12-16 11:59:56 -08:00
Akihiro Suda
8b8725d1fd set DefaultRootlessImage to "moby/buildkit:buildx-stable-1-rootless"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-12-16 17:35:38 +09:00
Tõnis Tiigi
20494f799d Merge pull request #470 from tonistiigi/dockerfile-1.2
update Dockerfile to v1.2
2020-12-15 10:35:55 -08:00
Wang Jinglei
dd13e16bc7 should list real pod nodes when all drivers are kubernetes
Signed-off-by: Wang <morlay.null@gmail.com>
2020-12-15 17:09:03 +08:00
Tonis Tiigi
f349ba8750 update Dockerfile to v1.2
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-12-08 15:44:08 -08:00
5489 changed files with 762350 additions and 174079 deletions

View File

@@ -1,2 +1 @@
bin/
cross-out/

View File

@@ -116,6 +116,60 @@ commit automatically with `git commit -s`.
### Run the unit- and integration-tests
Running tests:
```bash
make test
```
This runs all unit and integration tests, in a containerized environment.
Locally, every package can be tested separately with standard Go tools, but
integration tests are skipped if local user doesn't have enough permissions or
worker binaries are not installed.
```bash
# run unit tests only
make test-unit
# run integration tests only
make test-integration
# test a specific package
TESTPKGS=./bake make test
# run all integration tests with a specific worker
TESTFLAGS="--run=//worker=remote -v" make test-integration
# run a specific integration test
TESTFLAGS="--run /TestBuild/worker=remote/ -v" make test-integration
# run a selection of integration tests using a regexp
TESTFLAGS="--run /TestBuild.*/worker=remote/ -v" make test-integration
```
> **Note**
>
> Set `TEST_KEEP_CACHE=1` for the test framework to keep external dependant
> images in a docker volume if you are repeatedly calling `make test`. This
> helps to avoid rate limiting on the remote registry side.
> **Note**
>
> Set `TEST_DOCKERD=1` for the test framework to enable the docker workers,
> specifically the `docker` and `docker-container` drivers.
>
> The docker tests cannot be run in parallel, so require passing `--parallel=1`
> in `TESTFLAGS`.
> **Note**
>
> If you are working behind a proxy, you can set some of or all
> `HTTP_PROXY=http://ip:port`, `HTTPS_PROXY=http://ip:port`, `NO_PROXY=http://ip:port`
> for the test framework to specify the proxy build args.
### Run the helper commands
To enter a demo container environment and experiment, you may run:
```

124
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,124 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Bug Report
description: Report a bug
labels:
- status/triage
body:
- type: markdown
attributes:
value: |
Thank you for taking the time to report a bug!
If this is a security issue please report it to the [Docker Security team](mailto:security@docker.com).
- type: checkboxes
attributes:
label: Contributing guidelines
description: |
Please read the contributing guidelines before proceeding.
options:
- label: I've read the [contributing guidelines](https://github.com/docker/buildx/blob/master/.github/CONTRIBUTING.md) and wholeheartedly agree
required: true
- type: checkboxes
attributes:
label: I've found a bug and checked that ...
description: |
Make sure that your request fulfills all of the following requirements.
If one requirement cannot be satisfied, explain in detail why.
options:
- label: ... the documentation does not mention anything about my problem
- label: ... there are no open or closed issues that are related to my problem
- type: textarea
attributes:
label: Description
description: |
Please provide a brief description of the bug in 1-2 sentences.
validations:
required: true
- type: textarea
attributes:
label: Expected behaviour
description: |
Please describe precisely what you'd expect to happen.
validations:
required: true
- type: textarea
attributes:
label: Actual behaviour
description: |
Please describe precisely what is actually happening.
validations:
required: true
- type: input
attributes:
label: Buildx version
description: |
Output of `docker buildx version` command.
Example: `github.com/docker/buildx v0.8.1 5fac64c2c49dae1320f2b51f1a899ca451935554`
validations:
required: true
- type: textarea
attributes:
label: Docker info
description: |
Output of `docker info` command.
render: text
- type: textarea
attributes:
label: Builders list
description: |
Output of `docker buildx ls` command.
render: text
validations:
required: true
- type: textarea
attributes:
label: Configuration
description: >
Please provide a minimal Dockerfile, bake definition (if applicable) and
invoked commands to help reproducing your issue.
placeholder: |
```dockerfile
FROM alpine
echo hello
```
```hcl
group "default" {
targets = ["app"]
}
target "app" {
dockerfile = "Dockerfile"
target = "build"
}
```
```console
$ docker buildx build .
$ docker buildx bake
```
validations:
required: true
- type: textarea
attributes:
label: Build logs
description: |
Please provide logs output (and/or BuildKit logs if applicable).
render: text
validations:
required: false
- type: textarea
attributes:
label: Additional info
description: |
Please provide any additional information that could be useful.

12
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
blank_issues_enabled: true
contact_links:
- name: Questions and Discussions
url: https://github.com/docker/buildx/discussions/new
about: Use Github Discussions to ask questions and/or open discussion topics.
- name: Command line reference
url: https://docs.docker.com/engine/reference/commandline/buildx/
about: Read the command line reference.
- name: Documentation
url: https://docs.docker.com/build/
about: Read the documentation.

15
.github/ISSUE_TEMPLATE/feature.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
# https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
name: Feature request
description: Missing functionality? Come tell us about it!
labels:
- kind/enhancement
- status/triage
body:
- type: textarea
id: description
attributes:
label: Description
description: What is the feature you want to see?
validations:
required: true

12
.github/SECURITY.md vendored Normal file
View File

@@ -0,0 +1,12 @@
# Reporting security issues
The project maintainers take security seriously. If you discover a security
issue, please bring it to their attention right away!
**Please _DO NOT_ file a public issue**, instead send your report privately to
[security@docker.com](mailto:security@docker.com).
Security reports are greatly appreciated, and we will publicly thank you for it.
We also like to send gifts&mdash;if you're into schwag, make sure to let
us know. We currently do not offer a paid security bounty program, but are not
ruling it out in the future.

10
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: "github-actions"
open-pull-requests-limit: 10
directory: "/"
schedule:
interval: "daily"
labels:
- "dependencies"
- "bot"

735
.github/releases.json vendored Normal file
View File

@@ -0,0 +1,735 @@
{
"latest": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.2": {
"id": 90741208,
"tag_name": "v0.10.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/buildx-v0.10.2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.2/checksums.txt"
]
},
"v0.10.1": {
"id": 90346950,
"tag_name": "v0.10.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/buildx-v0.10.1.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.1/checksums.txt"
]
},
"v0.10.0": {
"id": 88388110,
"tag_name": "v0.10.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/buildx-v0.10.0.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0/checksums.txt"
]
},
"v0.10.0-rc3": {
"id": 88191592,
"tag_name": "v0.10.0-rc3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/buildx-v0.10.0-rc3.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc3/checksums.txt"
]
},
"v0.10.0-rc2": {
"id": 86248476,
"tag_name": "v0.10.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.darwin-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v6.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm-v7.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-ppc64le.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-riscv64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.linux-s390x.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-amd64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.provenance.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/buildx-v0.10.0-rc2.windows-arm64.sbom.json",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc2/checksums.txt"
]
},
"v0.10.0-rc1": {
"id": 85963900,
"tag_name": "v0.10.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.10.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/buildx-v0.10.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.10.0-rc1/checksums.txt"
]
},
"v0.9.1": {
"id": 74760068,
"tag_name": "v0.9.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/buildx-v0.9.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.1/checksums.txt"
]
},
"v0.9.0": {
"id": 74546589,
"tag_name": "v0.9.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/buildx-v0.9.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0/checksums.txt"
]
},
"v0.9.0-rc2": {
"id": 74052235,
"tag_name": "v0.9.0-rc2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/buildx-v0.9.0-rc2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc2/checksums.txt"
]
},
"v0.9.0-rc1": {
"id": 73389692,
"tag_name": "v0.9.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.9.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/buildx-v0.9.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.9.0-rc1/checksums.txt"
]
},
"v0.8.2": {
"id": 63479740,
"tag_name": "v0.8.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/buildx-v0.8.2.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.2/checksums.txt"
]
},
"v0.8.1": {
"id": 62289050,
"tag_name": "v0.8.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/buildx-v0.8.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.1/checksums.txt"
]
},
"v0.8.0": {
"id": 61423774,
"tag_name": "v0.8.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/buildx-v0.8.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0/checksums.txt"
]
},
"v0.8.0-rc1": {
"id": 60513568,
"tag_name": "v0.8.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.8.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/buildx-v0.8.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.8.0-rc1/checksums.txt"
]
},
"v0.7.1": {
"id": 54098347,
"tag_name": "v0.7.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/buildx-v0.7.1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.1/checksums.txt"
]
},
"v0.7.0": {
"id": 53109422,
"tag_name": "v0.7.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/buildx-v0.7.0.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0/checksums.txt"
]
},
"v0.7.0-rc1": {
"id": 52726324,
"tag_name": "v0.7.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.7.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/buildx-v0.7.0-rc1.windows-arm64.exe",
"https://github.com/docker/buildx/releases/download/v0.7.0-rc1/checksums.txt"
]
},
"v0.6.3": {
"id": 48691641,
"tag_name": "v0.6.3",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.3",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.3/buildx-v0.6.3.windows-arm64.exe"
]
},
"v0.6.2": {
"id": 48207405,
"tag_name": "v0.6.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.2/buildx-v0.6.2.windows-arm64.exe"
]
},
"v0.6.1": {
"id": 47064772,
"tag_name": "v0.6.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.1/buildx-v0.6.1.windows-arm64.exe"
]
},
"v0.6.0": {
"id": 46343260,
"tag_name": "v0.6.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0/buildx-v0.6.0.windows-arm64.exe"
]
},
"v0.6.0-rc1": {
"id": 46230351,
"tag_name": "v0.6.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.6.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-riscv64",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-amd64.exe",
"https://github.com/docker/buildx/releases/download/v0.6.0-rc1/buildx-v0.6.0-rc1.windows-arm64.exe"
]
},
"v0.5.1": {
"id": 35276550,
"tag_name": "v0.5.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.1/buildx-v0.5.1.windows-amd64.exe"
]
},
"v0.5.0": {
"id": 35268960,
"tag_name": "v0.5.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.darwin-universal",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0/buildx-v0.5.0.windows-amd64.exe"
]
},
"v0.5.0-rc1": {
"id": 35015334,
"tag_name": "v0.5.0-rc1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.5.0-rc1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.5.0-rc1/buildx-v0.5.0-rc1.windows-amd64.exe"
]
},
"v0.4.2": {
"id": 30007794,
"tag_name": "v0.4.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.2/buildx-v0.4.2.windows-amd64.exe"
]
},
"v0.4.1": {
"id": 26067509,
"tag_name": "v0.4.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.1/buildx-v0.4.1.windows-amd64.exe"
]
},
"v0.4.0": {
"id": 26028174,
"tag_name": "v0.4.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.4.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.4.0/buildx-v0.4.0.windows-amd64.exe"
]
},
"v0.3.1": {
"id": 20316235,
"tag_name": "v0.3.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.1/buildx-v0.3.1.windows-amd64.exe"
]
},
"v0.3.0": {
"id": 19029664,
"tag_name": "v0.3.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.3.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.windows-amd64.exe"
]
},
"v0.2.2": {
"id": 17671545,
"tag_name": "v0.2.2",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.2",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.2/buildx-v0.2.2.windows-amd64.exe"
]
},
"v0.2.1": {
"id": 17582885,
"tag_name": "v0.2.1",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.1",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.1/buildx-v0.2.1.windows-amd64.exe"
]
},
"v0.2.0": {
"id": 16965310,
"tag_name": "v0.2.0",
"html_url": "https://github.com/docker/buildx/releases/tag/v0.2.0",
"assets": [
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.darwin-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-amd64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v6",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm-v7",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-arm64",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-ppc64le",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.linux-s390x",
"https://github.com/docker/buildx/releases/download/v0.2.0/buildx-v0.2.0.windows-amd64.exe"
]
}
}

View File

@@ -1,185 +1,299 @@
name: build
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- 'v[0-9]*'
tags:
- 'v*'
pull_request:
branches:
- 'master'
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
REPO_SLUG_ORIGIN: "moby/buildkit:master"
CACHEKEY_BINARIES: "binaries"
PLATFORMS: "linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64,linux/s390x,linux/ppc64le"
BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:latest"
REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin"
TEST_CACHE_SCOPE: "test"
jobs:
base:
runs-on: ubuntu-latest
prepare-test:
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Cache ${{ env.CACHEKEY_BINARIES }}
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
key: ${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
with:
driver-opts: image=${{ env.REPO_SLUG_ORIGIN }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build ${{ env.CACHEKEY_BINARIES }}
run: |
./hack/build_ci_first_pass binaries
env:
CACHEDIR_FROM: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
CACHEDIR_TO: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}-new
-
# FIXME: Temp fix for https://github.com/moby/buildkit/issues/1850
name: Move cache
run: |
rm -rf /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
mv /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}-new /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
name: Build
uses: docker/bake-action@v3
with:
targets: integration-test-base
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.cache-to=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
test:
runs-on: ubuntu-latest
needs: [base]
runs-on: ubuntu-22.04
needs:
- prepare-test
env:
TESTFLAGS: "-v --parallel=6 --timeout=30m"
TESTFLAGS_DOCKER: "-v --parallel=1 --timeout=30m"
GOTESTSUM_FORMAT: "standard-verbose"
TEST_IMAGE_BUILD: "0"
TEST_IMAGE_ID: "buildx-tests"
strategy:
fail-fast: false
matrix:
worker:
- docker
- docker-container
- remote
pkg:
- ./tests
include:
- pkg: ./...
skip-integration-tests: 1
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Cache ${{ env.CACHEKEY_BINARIES }}
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
key: ${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
with:
driver-opts: image=${{ env.REPO_SLUG_ORIGIN }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build test image
uses: docker/bake-action@v3
with:
targets: integration-test
set: |
*.cache-from=type=gha,scope=${{ env.TEST_CACHE_SCOPE }}
*.output=type=docker,name=${{ env.TEST_IMAGE_ID }}
-
name: Test
run: |
make test
export TEST_REPORT_SUFFIX=-${{ github.job }}-$(echo "${{ matrix.pkg }}-${{ matrix.skip-integration-tests }}-${{ matrix.worker }}" | tr -dc '[:alnum:]-\n\r' | tr '[:upper:]' '[:lower:]')
./hack/test
env:
TEST_COVERAGE: 1
TESTFLAGS: -v --parallel=6 --timeout=20m
CACHEDIR_FROM: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
TEST_DOCKERD: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker-container') && '1' || '0' }}"
TESTFLAGS: "${{ (matrix.worker == 'docker' || matrix.worker == 'docker-container') && env.TESTFLAGS_DOCKER || env.TESTFLAGS }} --run=//worker=${{ matrix.worker }}$"
TESTPKGS: "${{ matrix.pkg }}"
SKIP_INTEGRATION_TESTS: "${{ matrix.skip-integration-tests }}"
-
name: Send to Codecov
uses: codecov/codecov-action@v1
if: always()
uses: codecov/codecov-action@v3
with:
file: ./coverage/coverage.txt
directory: ./bin/testreports
-
name: Generate annotations
if: always()
uses: crazy-max/.github/.github/actions/gotest-annotations@1a64ea6d01db9a48aa61954cb20e265782c167d9
with:
directory: ./bin/testreports
-
name: Upload test reports
if: always()
uses: actions/upload-artifact@v3
with:
name: test-reports
path: ./bin/testreports
cross:
runs-on: ubuntu-latest
needs: [base]
prepare-binaries:
runs-on: ubuntu-22.04
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
steps:
-
name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
-
name: Cache ${{ env.CACHEKEY_BINARIES }}
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
key: ${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
driver-opts: image=${{ env.REPO_SLUG_ORIGIN }}
-
name: Cross
name: Create matrix
id: platforms
run: |
make cross
env:
TARGETPLATFORM: ${{ env.PLATFORMS }},darwin/amd64,windows/amd64
CACHEDIR_FROM: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
echo "matrix=$(docker buildx bake binaries-cross --print | jq -cr '.target."binaries-cross".platforms')" >>${GITHUB_OUTPUT}
-
name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
binaries:
runs-on: ubuntu-latest
needs: [test, cross]
runs-on: ubuntu-22.04
needs:
- prepare-binaries
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare-binaries.outputs.matrix) }}
steps:
-
name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build
run: |
make release
env:
PLATFORMS: ${{ matrix.platform }}
CACHE_FROM: type=gha,scope=binaries-${{ env.PLATFORM_PAIR }}
CACHE_TO: type=gha,scope=binaries-${{ env.PLATFORM_PAIR }},mode=max
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: buildx
path: ${{ env.DESTDIR }}/*
if-no-files-found: error
bin-image:
runs-on: ubuntu-22.04
needs:
- test
if: ${{ github.event_name != 'pull_request' && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Prepare
id: prep
run: |
TAG=pr
if [[ $GITHUB_REF == refs/tags/v* ]]; then
TAG=${GITHUB_REF#refs/tags/}
elif [[ $GITHUB_REF == refs/heads/* ]]; then
TAG=$(echo ${GITHUB_REF#refs/heads/} | sed -r 's#/+#-#g')
fi
echo ::set-output name=tag::${TAG}
-
name: Cache ${{ env.CACHEKEY_BINARIES }}
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
key: ${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-${{ env.CACHEKEY_BINARIES }}-
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v1
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
with:
driver-opts: image=${{ env.REPO_SLUG_ORIGIN }}
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=${{ env.BUILDKIT_IMAGE }}
buildkitd-flags: --debug
-
name: Build ${{ steps.prep.outputs.tag }}
run: |
./hack/release "${{ steps.prep.outputs.tag }}" release-out
env:
PLATFORMS: ${{ env.PLATFORMS }},darwin/amd64,windows/amd64
CACHEDIR_FROM: /tmp/.buildx-cache/${{ env.CACHEKEY_BINARIES }}
name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
images: |
${{ env.REPO_SLUG }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
bake-target: meta-helper
-
name: Move artifacts
run: |
mv ./release-out/**/* ./release-out/
name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Upload artifacts
uses: actions/upload-artifact@v2
name: Build and push image
uses: docker/bake-action@v3
with:
files: |
./docker-bake.hcl
${{ steps.meta.outputs.bake-file }}
targets: image-cross
push: ${{ github.event_name != 'pull_request' }}
sbom: true
set: |
*.cache-from=type=gha,scope=bin-image
*.cache-to=type=gha,scope=bin-image,mode=max
release:
runs-on: ubuntu-22.04
needs:
- test
- binaries
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Download binaries
uses: actions/download-artifact@v3
with:
name: buildx
path: ./release-out/*
if-no-files-found: error
path: ${{ env.DESTDIR }}
-
name: Create checksums
run: ./hack/hash-files
-
name: List artifacts
run: |
tree -nh ${{ env.DESTDIR }}
-
name: Check artifacts
run: |
find ${{ env.DESTDIR }} -type f -exec file -e ascii -- {} +
-
name: GitHub Release
if: startsWith(github.ref, 'refs/tags/v')
uses: softprops/action-gh-release@v1
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844 # v0.1.15
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
draft: true
files: ./release-out/*
name: ${{ steps.prep.outputs.tag }}
files: ${{ env.DESTDIR }}/*
buildkit-edge:
runs-on: ubuntu-22.04
continue-on-error: true
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: ${{ env.BUILDX_VERSION }}
driver-opts: image=moby/buildkit:master
buildkitd-flags: --debug
-
# Just run a bake target to check eveything runs fine
name: Build
uses: docker/bake-action@v3
with:
targets: binaries

58
.github/workflows/docs-release.yml vendored Normal file
View File

@@ -0,0 +1,58 @@
name: docs-release
on:
release:
types:
- released
jobs:
open-pr:
runs-on: ubuntu-22.04
if: ${{ github.event.release.prerelease != true && github.repository == 'docker/buildx' }}
steps:
-
name: Checkout docs repo
uses: actions/checkout@v3
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
repository: docker/docs
ref: main
-
name: Prepare
run: |
rm -rf ./_data/buildx/*
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
-
name: Build docs
uses: docker/bake-action@v3
with:
source: ${{ github.server_url }}/${{ github.repository }}.git#${{ github.event.release.name }}
targets: update-docs
set: |
*.output=/tmp/buildx-docs
env:
DOCS_FORMATS: yaml
-
name: Copy files
run: |
cp /tmp/buildx-docs/out/reference/*.yaml ./_data/buildx/
-
name: Commit changes
run: |
git add -A .
-
name: Create PR on docs repo
uses: peter-evans/create-pull-request@284f54f989303d2699d373481a0cfa13ad5a6666
with:
token: ${{ secrets.GHPAT_DOCS_DISPATCH }}
push-to-fork: docker-tools-robot/docker.github.io
commit-message: "build: update buildx reference to ${{ github.event.release.name }}"
signoff: true
branch: dispatch/buildx-ref-${{ github.event.release.name }}
delete-branch: true
title: Update buildx reference to ${{ github.event.release.name }}
body: |
Update the buildx reference documentation to keep in sync with the latest release `${{ github.event.release.name }}`
draft: false

62
.github/workflows/docs-upstream.yml vendored Normal file
View File

@@ -0,0 +1,62 @@
# this workflow runs the remote validate bake target from docker/docker.github.io
# to check if yaml reference docs and markdown files used in this repo are still valid
# https://github.com/docker/docker.github.io/blob/98c7c9535063ae4cd2cd0a31478a21d16d2f07a3/docker-bake.hcl#L34-L36
name: docs-upstream
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
push:
branches:
- 'master'
- 'v[0-9]*'
paths:
- '.github/workflows/docs-upstream.yml'
- 'docs/**'
pull_request:
paths:
- '.github/workflows/docs-upstream.yml'
- 'docs/**'
jobs:
docs-yaml:
runs-on: ubuntu-22.04
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: latest
-
name: Build reference YAML docs
uses: docker/bake-action@v3
with:
targets: update-docs
set: |
*.output=/tmp/buildx-docs
*.cache-from=type=gha,scope=docs-yaml
*.cache-to=type=gha,scope=docs-yaml,mode=max
env:
DOCS_FORMATS: yaml
-
name: Upload reference YAML docs
uses: actions/upload-artifact@v3
with:
name: docs-yaml
path: /tmp/buildx-docs/out/reference
retention-days: 1
validate:
uses: docker/docs/.github/workflows/validate-upstream.yml@main
needs:
- docs-yaml
with:
repo: https://github.com/${{ github.repository }}
data-files-id: docs-yaml
data-files-folder: buildx
data-files-placeholder-folder: engine/reference/commandline

216
.github/workflows/e2e.yml vendored Normal file
View File

@@ -0,0 +1,216 @@
name: e2e
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- 'v[0-9]*'
pull_request:
paths-ignore:
- '.github/releases.json'
- 'README.md'
- 'docs/**'
env:
DESTDIR: "./bin"
K3S_VERSION: "v1.21.2-k3s1"
jobs:
build:
runs-on: ubuntu-22.04
steps:
- name: Checkout
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
version: latest
-
name: Build
uses: docker/bake-action@v3
with:
targets: binaries
set: |
*.cache-from=type=gha,scope=release
*.cache-from=type=gha,scope=binaries
*.cache-to=type=gha,scope=binaries
-
name: Rename binary
run: |
mv ${{ env.DESTDIR }}/build/buildx ${{ env.DESTDIR }}/build/docker-buildx
-
name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: binary
path: ${{ env.DESTDIR }}/build
if-no-files-found: error
retention-days: 7
driver:
runs-on: ubuntu-20.04
needs:
- build
strategy:
fail-fast: false
matrix:
driver:
- docker
- docker-container
- kubernetes
- remote
buildkit:
- moby/buildkit:buildx-stable-1
- moby/buildkit:master
buildkit-cfg:
- bkcfg-false
- bkcfg-true
multi-node:
- mnode-false
- mnode-true
platforms:
- linux/amd64
- linux/amd64,linux/arm64
include:
- driver: kubernetes
driver-opt: qemu.install=true
- driver: remote
endpoint: tcp://localhost:1234
exclude:
- driver: docker
multi-node: mnode-true
- driver: docker
buildkit-cfg: bkcfg-true
- driver: docker-container
multi-node: mnode-true
- driver: remote
multi-node: mnode-true
- driver: remote
buildkit-cfg: bkcfg-true
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Set up QEMU
uses: docker/setup-qemu-action@v2
if: matrix.driver == 'docker' || matrix.driver == 'docker-container'
-
name: Install buildx
uses: actions/download-artifact@v3
with:
name: binary
path: /home/runner/.docker/cli-plugins
-
name: Fix perms and check
run: |
chmod +x /home/runner/.docker/cli-plugins/docker-buildx
docker buildx version
-
name: Init env vars
run: |
# BuildKit cfg
if [ "${{ matrix.buildkit-cfg }}" = "bkcfg-true" ]; then
cat > "/tmp/buildkitd.toml" <<EOL
[worker.oci]
max-parallelism = 2
EOL
echo "BUILDKIT_CFG=/tmp/buildkitd.toml" >> $GITHUB_ENV
fi
# Multi node
if [ "${{ matrix.multi-node }}" = "mnode-true" ]; then
echo "MULTI_NODE=1" >> $GITHUB_ENV
else
echo "MULTI_NODE=0" >> $GITHUB_ENV
fi
-
name: Install k3s
if: matrix.driver == 'kubernetes'
uses: actions/github-script@v6
with:
script: |
const fs = require('fs');
let wait = function(milliseconds) {
return new Promise((resolve, reject) => {
if (typeof(milliseconds) !== 'number') {
throw new Error('milleseconds not a number');
}
setTimeout(() => resolve("done!"), milliseconds)
});
}
try {
const kubeconfig="/tmp/buildkit-k3s/kubeconfig.yaml";
core.info(`storing kubeconfig in ${kubeconfig}`);
await exec.exec('docker', ["run", "-d",
"--privileged",
"--name=buildkit-k3s",
"-e", "K3S_KUBECONFIG_OUTPUT="+kubeconfig,
"-e", "K3S_KUBECONFIG_MODE=666",
"-v", "/tmp/buildkit-k3s:/tmp/buildkit-k3s",
"-p", "6443:6443",
"-p", "80:80",
"-p", "443:443",
"-p", "8080:8080",
"rancher/k3s:${{ env.K3S_VERSION }}", "server"
]);
await wait(10000);
core.exportVariable('KUBECONFIG', kubeconfig);
let nodeName;
for (let count = 1; count <= 5; count++) {
try {
const nodeNameOutput = await exec.getExecOutput("kubectl get nodes --no-headers -oname");
nodeName = nodeNameOutput.stdout
} catch (error) {
core.info(`Unable to resolve node name (${error.message}). Attempt ${count} of 5.`)
} finally {
if (nodeName) {
break;
}
await wait(5000);
}
}
if (!nodeName) {
throw new Error(`Unable to resolve node name after 5 attempts.`);
}
await exec.exec(`kubectl wait --for=condition=Ready ${nodeName}`);
} catch (error) {
core.setFailed(error.message);
}
-
name: Print KUBECONFIG
if: matrix.driver == 'kubernetes'
run: |
yq ${{ env.KUBECONFIG }}
-
name: Launch remote buildkitd
if: matrix.driver == 'remote'
run: |
docker run -d \
--privileged \
--name=remote-buildkit \
-p 1234:1234 \
${{ matrix.buildkit }} \
--addr unix:///run/buildkit/buildkitd.sock \
--addr tcp://0.0.0.0:1234
-
name: Test
run: |
make test-driver
env:
BUILDKIT_IMAGE: ${{ matrix.buildkit }}
DRIVER: ${{ matrix.driver }}
DRIVER_OPT: ${{ matrix.driver-opt }}
ENDPOINT: ${{ matrix.endpoint }}
PLATFORMS: ${{ matrix.platforms }}

View File

@@ -1,25 +0,0 @@
# Workflow used to make a request to proxy.golang.org to refresh cache on https://pkg.go.dev/github.com/docker/buildx
# when a released of buildx is produced
name: godev
on:
push:
tags:
- 'v*'
jobs:
update:
runs-on: ubuntu-latest
steps:
-
name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.13
-
name: Call pkg.go.dev
run: |
go get github.com/${GITHUB_REPOSITORY}@${GITHUB_REF#refs/tags/}
env:
GO111MODULE: on
GOPROXY: https://proxy.golang.org

View File

@@ -1,37 +1,41 @@
name: validate
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
on:
workflow_dispatch:
push:
branches:
- 'master'
- 'v[0-9]*'
tags:
- 'v*'
pull_request:
branches:
- 'master'
env:
REPO_SLUG_ORIGIN: "moby/buildkit:master"
paths-ignore:
- '.github/releases.json'
jobs:
validate:
runs-on: ubuntu-latest
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
target:
- lint
- validate-vendor
- validate-docs
- validate-generated-files
steps:
-
name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v3
-
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
uses: docker/setup-buildx-action@v2
with:
driver-opts: image=${{ env.REPO_SLUG_ORIGIN }}
version: latest
-
name: Run
run: |

4
.gitignore vendored
View File

@@ -1,3 +1 @@
bin
coverage
cross-out
/bin

49
.golangci.yml Normal file
View File

@@ -0,0 +1,49 @@
run:
timeout: 10m
skip-files:
- ".*\\.pb\\.go$"
modules-download-mode: vendor
build-tags:
linters:
enable:
- gofmt
- govet
- depguard
- goimports
- ineffassign
- misspell
- unused
- revive
- staticcheck
- typecheck
- nolintlint
- gosec
- forbidigo
disable-all: true
linters-settings:
depguard:
list-type: blacklist
include-go-root: true
packages:
# The io/ioutil package has been deprecated.
# https://go.dev/doc/go1.16#ioutil
- io/ioutil
forbidigo:
forbid:
- '^fmt\.Errorf(# use errors\.Errorf instead)?$'
gosec:
excludes:
- G204 # Audit use of command execution
- G402 # TLS MinVersion too low
config:
G306: "0644"
issues:
exclude-rules:
- linters:
- revive
text: "stutters"

View File

@@ -1,6 +1,13 @@
# This file lists all individuals having contributed content to the repository.
# For how it is generated, see `hack/generate-authors`.
# For how it is generated, see hack/dockerfiles/authors.Dockerfile.
CrazyMax <github@crazymax.dev>
CrazyMax <github@crazymax.dev> <1951866+crazy-max@users.noreply.github.com>
CrazyMax <github@crazymax.dev> <crazy-max@users.noreply.github.com>
Sebastiaan van Stijn <github@gone.nl>
Sebastiaan van Stijn <github@gone.nl> <thaJeztah@users.noreply.github.com>
Tibor Vass <tibor@docker.com>
Tibor Vass <tibor@docker.com> <tiborvass@users.noreply.github.com>
Tõnis Tiigi <tonistiigi@gmail.com>
Ulysses Souza <ulyssessouza@gmail.com>
Wang Jinglei <morlay.null@gmail.com>

View File

@@ -1,13 +0,0 @@
ignore: |
/vendor
extends: default
yaml-files:
- '*.yaml'
- '*.yml'
rules:
truthy: disable
line-length: disable
document-start: disable

40
AUTHORS
View File

@@ -1,7 +1,45 @@
# This file lists all individuals having contributed content to the repository.
# For how it is generated, see `scripts/generate-authors.sh`.
# For how it is generated, see hack/dockerfiles/authors.Dockerfile.
Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Alex Couture-Beil <alex@earthly.dev>
Andrew Haines <andrew.haines@zencargo.com>
Andy MacKinlay <admackin@users.noreply.github.com>
Anthony Poschen <zanven42@gmail.com>
Artur Klauser <Artur.Klauser@computer.org>
Batuhan Apaydın <developerguy2@gmail.com>
Bin Du <bindu@microsoft.com>
Brandon Philips <brandon@ifup.org>
Brian Goff <cpuguy83@gmail.com>
CrazyMax <github@crazymax.dev>
dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Devin Bayer <dev@doubly.so>
Djordje Lukic <djordje.lukic@docker.com>
Dmytro Makovey <dmytro.makovey@docker.com>
Donghui Wang <977675308@qq.com>
faust <faustin@fala.red>
Felipe Santos <felipecassiors@gmail.com>
Fernando Miguel <github@FernandoMiguel.net>
gfrancesco <gfrancesco@users.noreply.github.com>
gracenoah <gracenoahgh@gmail.com>
Hollow Man <hollowman@hollowman.ml>
Ilya Dmitrichenko <errordeveloper@gmail.com>
Jack Laxson <jackjrabbit@gmail.com>
Jean-Yves Gastaud <jygastaud@gmail.com>
khs1994 <khs1994@khs1994.com>
Kotaro Adachi <k33asby@gmail.com>
l00397676 <lujingxiao@huawei.com>
Michal Augustyn <michal.augustyn@mail.com>
Patrick Van Stee <patrick@vanstee.me>
Saul Shanabrook <s.shanabrook@gmail.com>
Sebastiaan van Stijn <github@gone.nl>
SHIMA Tatsuya <ts1s1andn@gmail.com>
Silvin Lubecki <silvin.lubecki@docker.com>
Solomon Hykes <sh.github.6811@hykes.org>
Sune Keller <absukl@almbrand.dk>
Tibor Vass <tibor@docker.com>
Tõnis Tiigi <tonistiigi@gmail.com>
Ulysses Souza <ulyssessouza@gmail.com>
Wang Jinglei <morlay.null@gmail.com>
Xiang Dai <764524258@qq.com>
zelahi <elahi.zuhayr@gmail.com>

View File

@@ -1,76 +1,114 @@
# syntax=docker/dockerfile:1.1-experimental
# syntax=docker/dockerfile:1
ARG DOCKERD_VERSION=19.03-rc
ARG CLI_VERSION=19.03
ARG GO_VERSION=1.20
ARG XX_VERSION=1.2.1
ARG DOCKERD_VERSION=20.10.14
ARG GOTESTSUM_VERSION=v1.9.0
ARG REGISTRY_VERSION=2.8.0
ARG BUILDKIT_VERSION=v0.11.6
FROM docker:$DOCKERD_VERSION AS dockerd-release
# xgo is a helper for golang cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:golang@sha256:6f7d999551dd471b58f70716754290495690efa8421e0a1fcf18eb11d0c0a537 AS xgo
# xx is a helper for cross-compilation
FROM --platform=$BUILDPLATFORM tonistiigi/xx:${XX_VERSION} AS xx
FROM --platform=$BUILDPLATFORM golang:1.13-alpine AS gobase
COPY --from=xgo / /
FROM --platform=$BUILDPLATFORM golang:${GO_VERSION}-alpine AS golatest
FROM golatest AS gobase
COPY --from=xx / /
RUN apk add --no-cache file git
ENV GOFLAGS=-mod=vendor
ENV CGO_ENABLED=0
WORKDIR /src
FROM registry:$REGISTRY_VERSION AS registry
FROM moby/buildkit:$BUILDKIT_VERSION AS buildkit
FROM gobase AS gotestsum
ARG GOTESTSUM_VERSION
ENV GOFLAGS=
RUN --mount=target=/root/.cache,type=cache \
GOBIN=/out/ go install "gotest.tools/gotestsum@${GOTESTSUM_VERSION}" && \
/out/gotestsum --version
FROM gobase AS buildx-version
RUN --mount=target=. \
PKG=github.com/docker/buildx VERSION=$(git describe --match 'v[0-9]*' --dirty='.m' --always --tags) REVISION=$(git rev-parse HEAD)$(if ! git diff --no-ext-diff --quiet --exit-code; then echo .m; fi); \
echo "-X ${PKG}/version.Version=${VERSION} -X ${PKG}/version.Revision=${REVISION} -X ${PKG}/version.Package=${PKG}" | tee /tmp/.ldflags; \
echo -n "${VERSION}" | tee /tmp/.version;
RUN --mount=type=bind,target=. <<EOT
set -e
mkdir /buildx-version
echo -n "$(./hack/git-meta version)" | tee /buildx-version/version
echo -n "$(./hack/git-meta revision)" | tee /buildx-version/revision
EOT
FROM gobase AS buildx-build
ENV CGO_ENABLED=0
ARG TARGETPLATFORM
RUN --mount=target=. --mount=target=/root/.cache,type=cache \
--mount=target=/go/pkg/mod,type=cache \
--mount=source=/tmp/.ldflags,target=/tmp/.ldflags,from=buildx-version \
set -x; go build -ldflags "$(cat /tmp/.ldflags)" -o /usr/bin/buildx ./cmd/buildx && \
file /usr/bin/buildx && file /usr/bin/buildx | egrep "statically linked|Mach-O|Windows"
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
--mount=type=bind,from=buildx-version,source=/buildx-version,target=/buildx-version <<EOT
set -e
xx-go --wrap
DESTDIR=/usr/bin VERSION=$(cat /buildx-version/version) REVISION=$(cat /buildx-version/revision) GO_EXTRA_LDFLAGS="-s -w" ./hack/build
xx-verify --static /usr/bin/docker-buildx
EOT
FROM buildx-build AS integration-tests
COPY . .
FROM gobase AS test
ENV SKIP_INTEGRATION_TESTS=1
RUN --mount=type=bind,target=. \
--mount=type=cache,target=/root/.cache \
--mount=type=cache,target=/go/pkg/mod \
go test -v -coverprofile=/tmp/coverage.txt -covermode=atomic ./... && \
go tool cover -func=/tmp/coverage.txt
# FROM golang:1.12-alpine AS docker-cli-build
# RUN apk add -U git bash coreutils gcc musl-dev
# ENV CGO_ENABLED=0
# ARG REPO=github.com/tiborvass/cli
# ARG BRANCH=cli-plugin-aliases
# ARG CLI_VERSION
# WORKDIR /go/src/github.com/docker/cli
# RUN git clone git://$REPO . && git checkout $BRANCH
# RUN ./scripts/build/binary
FROM scratch AS test-coverage
COPY --from=test /tmp/coverage.txt /coverage.txt
FROM scratch AS binaries-unix
COPY --from=buildx-build /usr/bin/buildx /
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx
FROM binaries-unix AS binaries-darwin
FROM binaries-unix AS binaries-linux
FROM scratch AS binaries-windows
COPY --from=buildx-build /usr/bin/buildx /buildx.exe
COPY --link --from=buildx-build /usr/bin/docker-buildx /buildx.exe
FROM binaries-$TARGETOS AS binaries
# enable scanning for this stage
ARG BUILDKIT_SBOM_SCAN_STAGE=true
FROM gobase AS integration-test-base
RUN apk add --no-cache docker runc containerd
COPY --link --from=gotestsum /out/gotestsum /usr/bin/
COPY --link --from=registry /bin/registry /usr/bin/
COPY --link --from=buildkit /usr/bin/buildkitd /usr/bin/
COPY --link --from=buildkit /usr/bin/buildctl /usr/bin/
COPY --link --from=binaries /buildx /usr/bin/
FROM integration-test-base AS integration-test
COPY . .
# Release
FROM --platform=$BUILDPLATFORM alpine AS releaser
WORKDIR /work
ARG TARGETPLATFORM
RUN --mount=from=binaries \
--mount=source=/tmp/.version,target=/tmp/.version,from=buildx-version \
mkdir -p /out && cp buildx* "/out/buildx-$(cat /tmp/.version).$(echo $TARGETPLATFORM | sed 's/\//-/g')$(ls buildx* | sed -e 's/^buildx//')"
--mount=type=bind,from=buildx-version,source=/buildx-version,target=/buildx-version <<EOT
set -e
mkdir -p /out
cp buildx* "/out/buildx-$(cat /buildx-version/version).$(echo $TARGETPLATFORM | sed 's/\//-/g')$(ls buildx* | sed -e 's/^buildx//')"
EOT
FROM scratch AS release
COPY --from=releaser /out/ /
FROM alpine AS demo-env
# Shell
FROM docker:$DOCKERD_VERSION AS dockerd-release
FROM alpine AS shell
RUN apk add --no-cache iptables tmux git vim less openssh
RUN mkdir -p /usr/local/lib/docker/cli-plugins && ln -s /usr/local/bin/buildx /usr/local/lib/docker/cli-plugins/docker-buildx
COPY ./hack/demo-env/entrypoint.sh /usr/local/bin
COPY ./hack/demo-env/tmux.conf /root/.tmux.conf
COPY --from=dockerd-release /usr/local/bin /usr/local/bin
#COPY --from=docker-cli-build /go/src/github.com/docker/cli/build/docker /usr/local/bin
WORKDIR /work
COPY ./hack/demo-env/examples .
COPY --from=binaries / /usr/local/bin/

View File

@@ -150,6 +150,9 @@ made through a pull request.
[Org.Maintainers]
people = [
"akihirosuda",
"crazy-max",
"jedevc",
"tiborvass",
"tonistiigi",
]
@@ -176,6 +179,21 @@ made through a pull request.
# All other sections should refer to people by their canonical key
# in the people section.
[people.akihirosuda]
Name = "Akihiro Suda"
Email = "akihiro.suda.cz@hco.ntt.co.jp"
GitHub = "AkihiroSuda"
[people.crazy-max]
Name = "Kevin Alvarez"
Email = "contact@crazymax.dev"
GitHub = "crazy-max"
[people.jedevc]
Name = "Justin Chadwell"
Email = "me@jedevc.com"
GitHub = "jedevc"
[people.thajeztah]
Name = "Sebastiaan van Stijn"
Email = "github@gone.nl"

View File

@@ -1,34 +1,96 @@
ifneq (, $(BUILDX_BIN))
export BUILDX_CMD = $(BUILDX_BIN)
else ifneq (, $(shell docker buildx version))
export BUILDX_CMD = docker buildx
else ifneq (, $(shell which buildx))
export BUILDX_CMD = $(which buildx)
endif
export BUILDX_CMD ?= docker buildx
.PHONY: all
all: binaries
.PHONY: build
build:
./hack/build
.PHONY: shell
shell:
./hack/shell
.PHONY: binaries
binaries:
./hack/binaries
$(BUILDX_CMD) bake binaries
.PHONY: binaries-cross
binaries-cross:
EXPORT_LOCAL=cross-out ./hack/cross
cross:
./hack/cross
$(BUILDX_CMD) bake binaries-cross
.PHONY: install
install: binaries
mkdir -p ~/.docker/cli-plugins
cp bin/buildx ~/.docker/cli-plugins/docker-buildx
install bin/build/buildx ~/.docker/cli-plugins/docker-buildx
.PHONY: release
release:
./hack/release
.PHONY: validate-all
validate-all: lint test validate-vendor validate-docs validate-generated-files
.PHONY: lint
lint:
./hack/lint
$(BUILDX_CMD) bake lint
.PHONY: test
test:
./hack/test
.PHONY: test-unit
test-unit:
TESTPKGS=./... SKIP_INTEGRATION_TESTS=1 ./hack/test
.PHONY: test
test-integration:
TESTPKGS=./tests ./hack/test
.PHONY: validate-vendor
validate-vendor:
./hack/validate-vendor
$(BUILDX_CMD) bake validate-vendor
validate-all: lint test validate-vendor
.PHONY: validate-docs
validate-docs:
$(BUILDX_CMD) bake validate-docs
.PHONY: validate-authors
validate-authors:
$(BUILDX_CMD) bake validate-authors
.PHONY: validate-generated-files
validate-generated-files:
$(BUILDX_CMD) bake validate-generated-files
.PHONY: test-driver
test-driver:
./hack/test-driver
.PHONY: vendor
vendor:
./hack/update-vendor
generate-authors:
./hack/generate-authors
.PHONY: docs
docs:
./hack/update-docs
.PHONY: vendor lint shell binaries install binaries-cross validate-all generate-authors
.PHONY: authors
authors:
$(BUILDX_CMD) bake update-authors
.PHONY: mod-outdated
mod-outdated:
$(BUILDX_CMD) bake mod-outdated
.PHONY: generated-files
generated-files:
$(BUILDX_CMD) bake update-generated-files

995
README.md

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,50 +1,48 @@
package bake
import (
"fmt"
"os"
"reflect"
"path/filepath"
"strings"
"github.com/docker/cli/cli/compose/loader"
composetypes "github.com/docker/cli/cli/compose/types"
"github.com/compose-spec/compose-go/dotenv"
"github.com/compose-spec/compose-go/loader"
compose "github.com/compose-spec/compose-go/types"
"github.com/pkg/errors"
"gopkg.in/yaml.v3"
)
func parseCompose(dt []byte) (*composetypes.Config, error) {
parsed, err := loader.ParseYAML([]byte(dt))
func ParseComposeFiles(fs []File) (*Config, error) {
envs, err := composeEnv()
if err != nil {
return nil, err
}
return loader.Load(composetypes.ConfigDetails{
ConfigFiles: []composetypes.ConfigFile{
{
Config: parsed,
},
},
Environment: envMap(os.Environ()),
var cfgs []compose.ConfigFile
for _, f := range fs {
cfgs = append(cfgs, compose.ConfigFile{
Filename: f.Name,
Content: f.Data,
})
}
func envMap(env []string) map[string]string {
result := make(map[string]string, len(env))
for _, s := range env {
kv := strings.SplitN(s, "=", 2)
if len(kv) != 2 {
continue
}
result[kv[0]] = kv[1]
}
return result
return ParseCompose(cfgs, envs)
}
func ParseCompose(dt []byte) (*Config, error) {
cfg, err := parseCompose(dt)
func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, error) {
if envs == nil {
envs = make(map[string]string)
}
cfg, err := loader.Load(compose.ConfigDetails{
ConfigFiles: cfgs,
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
})
if err != nil {
return nil, err
}
var c Config
var zeroBuildConfig composetypes.BuildConfig
if len(cfg.Services) > 0 {
c.Groups = []*Group{}
c.Targets = []*Target{}
@@ -52,15 +50,15 @@ func ParseCompose(dt []byte) (*Config, error) {
g := &Group{Name: "default"}
for _, s := range cfg.Services {
if reflect.DeepEqual(s.Build, zeroBuildConfig) {
// if not make sure they're setting an image or it's invalid d-c.yml
if s.Image == "" {
return nil, fmt.Errorf("compose file invalid: service %s has neither an image nor a build context specified. At least one must be provided.", s.Name)
}
if s.Build == nil {
continue
}
targetName := sanitizeTargetName(s.Name)
if err = validateTargetName(targetName); err != nil {
return nil, errors.Wrapf(err, "invalid service name %q", targetName)
}
var contextPathP *string
if s.Build.Context != "" {
contextPath := s.Build.Context
@@ -71,21 +69,65 @@ func ParseCompose(dt []byte) (*Config, error) {
dockerfilePath := s.Build.Dockerfile
dockerfilePathP = &dockerfilePath
}
g.Targets = append(g.Targets, s.Name)
var dockerfileInlineP *string
if s.Build.DockerfileInline != "" {
dockerfileInline := s.Build.DockerfileInline
dockerfileInlineP = &dockerfileInline
}
var additionalContexts map[string]string
if s.Build.AdditionalContexts != nil {
additionalContexts = map[string]string{}
for k, v := range s.Build.AdditionalContexts {
additionalContexts[k] = v
}
}
var secrets []string
for _, bs := range s.Build.Secrets {
secret, err := composeToBuildkitSecret(bs, cfg.Secrets[bs.Source])
if err != nil {
return nil, err
}
secrets = append(secrets, secret)
}
// compose does not support nil values for labels
labels := map[string]*string{}
for k, v := range s.Build.Labels {
v := v
labels[k] = &v
}
g.Targets = append(g.Targets, targetName)
t := &Target{
Name: s.Name,
Name: targetName,
Context: contextPathP,
Contexts: additionalContexts,
Dockerfile: dockerfilePathP,
Labels: s.Build.Labels,
Args: toMap(s.Build.Args),
DockerfileInline: dockerfileInlineP,
Tags: s.Build.Tags,
Labels: labels,
Args: flatten(s.Build.Args.Resolve(func(val string) (string, bool) {
if val, ok := s.Environment[val]; ok && val != nil {
return *val, true
}
val, ok := cfg.Environment[val]
return val, ok
})),
CacheFrom: s.Build.CacheFrom,
// TODO: add platforms
CacheTo: s.Build.CacheTo,
NetworkMode: &s.Build.Network,
Secrets: secrets,
}
if err = t.composeExtTarget(s.Build.Extensions); err != nil {
return nil, err
}
if s.Build.Target != "" {
target := s.Build.Target
t.Target = &target
}
if s.Image != "" {
if len(t.Tags) == 0 && s.Image != "" {
t.Tags = []string{s.Image}
}
c.Targets = append(c.Targets, t)
@@ -97,14 +139,206 @@ func ParseCompose(dt []byte) (*Config, error) {
return &c, nil
}
func toMap(in composetypes.MappingWithEquals) map[string]string {
m := map[string]string{}
func validateComposeFile(dt []byte, fn string) (bool, error) {
envs, err := composeEnv()
if err != nil {
return true, err
}
fnl := strings.ToLower(fn)
if strings.HasSuffix(fnl, ".yml") || strings.HasSuffix(fnl, ".yaml") {
return true, validateCompose(dt, envs)
}
if strings.HasSuffix(fnl, ".json") || strings.HasSuffix(fnl, ".hcl") {
return false, nil
}
err = validateCompose(dt, envs)
return err == nil, err
}
func validateCompose(dt []byte, envs map[string]string) error {
_, err := loader.Load(compose.ConfigDetails{
ConfigFiles: []compose.ConfigFile{
{
Content: dt,
},
},
Environment: envs,
}, func(options *loader.Options) {
options.SetProjectName("bake", false)
options.SkipNormalization = true
// consistency is checked later in ParseCompose to ensure multiple
// compose files can be merged together
options.SkipConsistencyCheck = true
})
return err
}
func composeEnv() (map[string]string, error) {
envs := sliceToMap(os.Environ())
if wd, err := os.Getwd(); err == nil {
envs, err = loadDotEnv(envs, wd)
if err != nil {
return nil, err
}
}
return envs, nil
}
func loadDotEnv(curenv map[string]string, workingDir string) (map[string]string, error) {
if curenv == nil {
curenv = make(map[string]string)
}
ef, err := filepath.Abs(filepath.Join(workingDir, ".env"))
if err != nil {
return nil, err
}
if _, err = os.Stat(ef); os.IsNotExist(err) {
return curenv, nil
} else if err != nil {
return nil, err
}
dt, err := os.ReadFile(ef)
if err != nil {
return nil, err
}
envs, err := dotenv.UnmarshalBytesWithLookup(dt, nil)
if err != nil {
return nil, err
}
for k, v := range envs {
if _, set := curenv[k]; set {
continue
}
curenv[k] = v
}
return curenv, nil
}
func flatten(in compose.MappingWithEquals) map[string]*string {
if len(in) == 0 {
return nil
}
out := map[string]*string{}
for k, v := range in {
if v != nil {
m[k] = *v
if v == nil {
continue
}
out[k] = v
}
return out
}
// xbake Compose build extension provides fields not (yet) available in
// Compose build specification: https://github.com/compose-spec/compose-spec/blob/master/build.md
type xbake struct {
Tags stringArray `yaml:"tags,omitempty"`
CacheFrom stringArray `yaml:"cache-from,omitempty"`
CacheTo stringArray `yaml:"cache-to,omitempty"`
Secrets stringArray `yaml:"secret,omitempty"`
SSH stringArray `yaml:"ssh,omitempty"`
Platforms stringArray `yaml:"platforms,omitempty"`
Outputs stringArray `yaml:"output,omitempty"`
Pull *bool `yaml:"pull,omitempty"`
NoCache *bool `yaml:"no-cache,omitempty"`
NoCacheFilter stringArray `yaml:"no-cache-filter,omitempty"`
Contexts stringMap `yaml:"contexts,omitempty"`
// don't forget to update documentation if you add a new field:
// docs/manuals/bake/compose-file.md#extension-field-with-x-bake
}
type stringMap map[string]string
type stringArray []string
func (sa *stringArray) UnmarshalYAML(unmarshal func(interface{}) error) error {
var multi []string
err := unmarshal(&multi)
if err != nil {
var single string
if err := unmarshal(&single); err != nil {
return err
}
*sa = strings.Fields(single)
} else {
m[k] = os.Getenv(k)
*sa = multi
}
return nil
}
return m
// composeExtTarget converts Compose build extension x-bake to bake Target
// https://github.com/compose-spec/compose-spec/blob/master/spec.md#extension
func (t *Target) composeExtTarget(exts map[string]interface{}) error {
var xb xbake
ext, ok := exts["x-bake"]
if !ok || ext == nil {
return nil
}
yb, _ := yaml.Marshal(ext)
if err := yaml.Unmarshal(yb, &xb); err != nil {
return err
}
if len(xb.Tags) > 0 {
t.Tags = dedupSlice(append(t.Tags, xb.Tags...))
}
if len(xb.CacheFrom) > 0 {
t.CacheFrom = dedupSlice(append(t.CacheFrom, xb.CacheFrom...))
}
if len(xb.CacheTo) > 0 {
t.CacheTo = dedupSlice(append(t.CacheTo, xb.CacheTo...))
}
if len(xb.Secrets) > 0 {
t.Secrets = dedupSlice(append(t.Secrets, xb.Secrets...))
}
if len(xb.SSH) > 0 {
t.SSH = dedupSlice(append(t.SSH, xb.SSH...))
}
if len(xb.Platforms) > 0 {
t.Platforms = dedupSlice(append(t.Platforms, xb.Platforms...))
}
if len(xb.Outputs) > 0 {
t.Outputs = dedupSlice(append(t.Outputs, xb.Outputs...))
}
if xb.Pull != nil {
t.Pull = xb.Pull
}
if xb.NoCache != nil {
t.NoCache = xb.NoCache
}
if len(xb.NoCacheFilter) > 0 {
t.NoCacheFilter = dedupSlice(append(t.NoCacheFilter, xb.NoCacheFilter...))
}
if len(xb.Contexts) > 0 {
t.Contexts = dedupMap(t.Contexts, xb.Contexts)
}
return nil
}
// composeToBuildkitSecret converts secret from compose format to buildkit's
// csv format.
func composeToBuildkitSecret(inp compose.ServiceSecretConfig, psecret compose.SecretConfig) (string, error) {
if psecret.External.External {
return "", errors.Errorf("unsupported external secret %s", psecret.Name)
}
var bkattrs []string
if inp.Source != "" {
bkattrs = append(bkattrs, "id="+inp.Source)
}
if psecret.File != "" {
bkattrs = append(bkattrs, "src="+psecret.File)
}
if psecret.Environment != "" {
bkattrs = append(bkattrs, "env="+psecret.Environment)
}
return strings.Join(bkattrs, ","), nil
}

View File

@@ -1,16 +1,18 @@
package bake
import (
"os"
"path/filepath"
"sort"
"testing"
compose "github.com/compose-spec/compose-go/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestParseCompose(t *testing.T) {
var dt = []byte(`
version: "3"
services:
db:
build: ./db
@@ -19,52 +21,83 @@ services:
webapp:
build:
context: ./dir
additional_contexts:
foo: /bar
dockerfile: Dockerfile-alternate
network:
none
args:
buildno: 123
cache_from:
- type=local,src=path/to/cache
cache_to:
- type=local,dest=path/to/cache
secrets:
- token
- aws
webapp2:
build:
context: ./dir
dockerfile_inline: |
FROM alpine
secrets:
token:
environment: ENV_TOKEN
aws:
file: /root/.aws/credentials
`)
c, err := ParseCompose(dt)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, c.Groups[0].Name, "default")
require.Equal(t, "default", c.Groups[0].Name)
sort.Strings(c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp"}, c.Groups[0].Targets)
require.Equal(t, []string{"db", "webapp", "webapp2"}, c.Groups[0].Targets)
require.Equal(t, 2, len(c.Targets))
require.Equal(t, 3, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, "./db", *c.Targets[0].Context)
require.Equal(t, []string{"docker.io/tonistiigi/db"}, c.Targets[0].Tags)
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, "./dir", *c.Targets[1].Context)
require.Equal(t, map[string]string{"foo": "/bar"}, c.Targets[1].Contexts)
require.Equal(t, "Dockerfile-alternate", *c.Targets[1].Dockerfile)
require.Equal(t, 1, len(c.Targets[1].Args))
require.Equal(t, "123", c.Targets[1].Args["buildno"])
require.Equal(t, ptrstr("123"), c.Targets[1].Args["buildno"])
require.Equal(t, []string{"type=local,src=path/to/cache"}, c.Targets[1].CacheFrom)
require.Equal(t, []string{"type=local,dest=path/to/cache"}, c.Targets[1].CacheTo)
require.Equal(t, "none", *c.Targets[1].NetworkMode)
require.Equal(t, []string{
"id=token,env=ENV_TOKEN",
"id=aws,src=/root/.aws/credentials",
}, c.Targets[1].Secrets)
require.Equal(t, "webapp2", c.Targets[2].Name)
require.Equal(t, "./dir", *c.Targets[2].Context)
require.Equal(t, "FROM alpine\n", *c.Targets[2].DockerfileInline)
}
func TestNoBuildOutOfTreeService(t *testing.T) {
var dt = []byte(`
version: "3.7"
services:
external:
image: "verycooldb:1337"
webapp:
build: ./db
`)
c, err := ParseCompose(dt)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Groups))
require.Equal(t, 1, len(c.Targets))
}
func TestParseComposeTarget(t *testing.T) {
var dt = []byte(`
version: "3.7"
services:
db:
build:
@@ -76,7 +109,7 @@ services:
target: webapp
`)
c, err := ParseCompose(dt)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
@@ -91,8 +124,6 @@ services:
func TestComposeBuildWithoutContext(t *testing.T) {
var dt = []byte(`
version: "3.7"
services:
db:
build:
@@ -103,33 +134,540 @@ services:
target: webapp
`)
c, err := ParseCompose(dt)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, c.Targets[0].Name, "db")
require.Equal(t, "db", c.Targets[0].Name)
require.Equal(t, "db", *c.Targets[0].Target)
require.Equal(t, c.Targets[1].Name, "webapp")
require.Equal(t, "webapp", c.Targets[1].Name)
require.Equal(t, "webapp", *c.Targets[1].Target)
}
func TestBogusCompose(t *testing.T) {
func TestBuildArgEnvCompose(t *testing.T) {
var dt = []byte(`
version: "3.7"
version: "3.8"
services:
db:
labels:
- "foo"
webapp:
example:
image: example
build:
context: .
target: webapp
dockerfile: Dockerfile
args:
FOO:
BAR: $ZZZ_BAR
BRB: FOO
`)
_, err := ParseCompose(dt)
require.Error(t, err)
require.Contains(t, err.Error(), "has neither an image nor a build context specified. At least one must be provided")
t.Setenv("FOO", "bar")
t.Setenv("BAR", "foo")
t.Setenv("ZZZ_BAR", "zzz_foo")
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, sliceToMap(os.Environ()))
require.NoError(t, err)
require.Equal(t, ptrstr("bar"), c.Targets[0].Args["FOO"])
require.Equal(t, ptrstr("zzz_foo"), c.Targets[0].Args["BAR"])
require.Equal(t, ptrstr("FOO"), c.Targets[0].Args["BRB"])
}
func TestInconsistentComposeFile(t *testing.T) {
var dt = []byte(`
services:
webapp:
entrypoint: echo 1
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.Error(t, err)
}
func TestAdvancedNetwork(t *testing.T) {
var dt = []byte(`
services:
db:
networks:
- example.com
build:
context: ./db
target: db
networks:
example.com:
name: example.com
driver: bridge
ipam:
config:
- subnet: 10.5.0.0/24
ip_range: 10.5.0.0/24
gateway: 10.5.0.254
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
func TestTags(t *testing.T) {
var dt = []byte(`
services:
example:
image: example
build:
context: .
dockerfile: Dockerfile
tags:
- foo
- bar
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, []string{"foo", "bar"}, c.Targets[0].Tags)
}
func TestDependsOnList(t *testing.T) {
var dt = []byte(`
version: "3.8"
services:
example-container:
image: example/fails:latest
build:
context: .
dockerfile: Dockerfile
depends_on:
other-container:
condition: service_healthy
networks:
default:
aliases:
- integration-tests
other-container:
image: example/other:latest
healthcheck:
test: ["CMD", "echo", "success"]
retries: 5
interval: 5s
timeout: 10s
start_period: 5s
networks:
default:
name: test-net
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
func TestComposeExt(t *testing.T) {
var dt = []byte(`
services:
addon:
image: ct-addon:bar
build:
context: .
dockerfile: ./Dockerfile
cache_from:
- user/app:cache
cache_to:
- user/app:cache
tags:
- ct-addon:baz
args:
CT_ECR: foo
CT_TAG: bar
x-bake:
contexts:
alpine: docker-image://alpine:3.13
tags:
- ct-addon:foo
- ct-addon:alp
platforms:
- linux/amd64
- linux/arm64
cache-from:
- type=local,src=path/to/cache
cache-to:
- type=local,dest=path/to/cache
pull: true
aws:
image: ct-fake-aws:bar
build:
dockerfile: ./aws.Dockerfile
args:
CT_ECR: foo
CT_TAG: bar
x-bake:
secret:
- id=mysecret,src=/local/secret
- id=mysecret2,src=/local/secret2
ssh: default
platforms: linux/arm64
output: type=docker
no-cache: true
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 2, len(c.Targets))
sort.Slice(c.Targets, func(i, j int) bool {
return c.Targets[i].Name < c.Targets[j].Name
})
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "CT_TAG": ptrstr("bar")}, c.Targets[0].Args)
require.Equal(t, []string{"ct-addon:baz", "ct-addon:foo", "ct-addon:alp"}, c.Targets[0].Tags)
require.Equal(t, []string{"linux/amd64", "linux/arm64"}, c.Targets[0].Platforms)
require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
require.Equal(t, newBool(true), c.Targets[0].Pull)
require.Equal(t, map[string]string{"alpine": "docker-image://alpine:3.13"}, c.Targets[0].Contexts)
require.Equal(t, []string{"ct-fake-aws:bar"}, c.Targets[1].Tags)
require.Equal(t, []string{"id=mysecret,src=/local/secret", "id=mysecret2,src=/local/secret2"}, c.Targets[1].Secrets)
require.Equal(t, []string{"default"}, c.Targets[1].SSH)
require.Equal(t, []string{"linux/arm64"}, c.Targets[1].Platforms)
require.Equal(t, []string{"type=docker"}, c.Targets[1].Outputs)
require.Equal(t, newBool(true), c.Targets[1].NoCache)
}
func TestComposeExtDedup(t *testing.T) {
var dt = []byte(`
services:
webapp:
image: app:bar
build:
cache_from:
- user/app:cache
cache_to:
- user/app:cache
tags:
- ct-addon:foo
x-bake:
tags:
- ct-addon:foo
- ct-addon:baz
cache-from:
- user/app:cache
- type=local,src=path/to/cache
cache-to:
- type=local,dest=path/to/cache
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, []string{"ct-addon:foo", "ct-addon:baz"}, c.Targets[0].Tags)
require.Equal(t, []string{"user/app:cache", "type=local,src=path/to/cache"}, c.Targets[0].CacheFrom)
require.Equal(t, []string{"user/app:cache", "type=local,dest=path/to/cache"}, c.Targets[0].CacheTo)
}
func TestEnv(t *testing.T) {
envf, err := os.CreateTemp("", "env")
require.NoError(t, err)
defer os.Remove(envf.Name())
_, err = envf.WriteString("FOO=bsdf -csdf\n")
require.NoError(t, err)
var dt = []byte(`
services:
scratch:
build:
context: .
args:
CT_ECR: foo
FOO:
NODE_ENV:
environment:
- NODE_ENV=test
- AWS_ACCESS_KEY_ID=dummy
- AWS_SECRET_ACCESS_KEY=dummy
env_file:
- ` + envf.Name() + `
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]*string{"CT_ECR": ptrstr("foo"), "FOO": ptrstr("bsdf -csdf"), "NODE_ENV": ptrstr("test")}, c.Targets[0].Args)
}
func TestDotEnv(t *testing.T) {
tmpdir := t.TempDir()
err := os.WriteFile(filepath.Join(tmpdir, ".env"), []byte("FOO=bar"), 0644)
require.NoError(t, err)
var dt = []byte(`
services:
scratch:
build:
context: .
args:
FOO:
`)
chdir(t, tmpdir)
c, err := ParseComposeFiles([]File{{
Name: "docker-compose.yml",
Data: dt,
}})
require.NoError(t, err)
require.Equal(t, map[string]*string{"FOO": ptrstr("bar")}, c.Targets[0].Args)
}
func TestPorts(t *testing.T) {
var dt = []byte(`
services:
foo:
build:
context: .
ports:
- 3306:3306
bar:
build:
context: .
ports:
- mode: ingress
target: 3306
published: "3306"
protocol: tcp
`)
_, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
}
func newBool(val bool) *bool {
b := val
return &b
}
func TestServiceName(t *testing.T) {
cases := []struct {
svc string
wantErr bool
}{
{
svc: "a",
wantErr: false,
},
{
svc: "abc",
wantErr: false,
},
{
svc: "a.b",
wantErr: false,
},
{
svc: "_a",
wantErr: false,
},
{
svc: "a_b",
wantErr: false,
},
{
svc: "AbC",
wantErr: false,
},
{
svc: "AbC-0123",
wantErr: false,
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.svc, func(t *testing.T) {
_, err := ParseCompose([]compose.ConfigFile{{Content: []byte(`
services:
` + tt.svc + `:
build:
context: .
`)}}, nil)
if tt.wantErr {
require.Error(t, err)
} else {
require.NoError(t, err)
}
})
}
}
func TestValidateComposeSecret(t *testing.T) {
cases := []struct {
name string
dt []byte
wantErr bool
}{
{
name: "secret set by file",
dt: []byte(`
secrets:
foo:
file: .secret
`),
wantErr: false,
},
{
name: "secret set by environment",
dt: []byte(`
secrets:
foo:
environment: TOKEN
`),
wantErr: false,
},
{
name: "external secret",
dt: []byte(`
secrets:
foo:
external: true
`),
wantErr: false,
},
{
name: "unset secret",
dt: []byte(`
secrets:
foo: {}
`),
wantErr: true,
},
{
name: "undefined secret",
dt: []byte(`
services:
foo:
build:
secrets:
- token
`),
wantErr: true,
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
_, err := ParseCompose([]compose.ConfigFile{{Content: tt.dt}}, nil)
if tt.wantErr {
require.Error(t, err)
} else {
require.NoError(t, err)
}
})
}
}
func TestValidateComposeFile(t *testing.T) {
cases := []struct {
name string
fn string
dt []byte
isCompose bool
wantErr bool
}{
{
name: "empty service",
fn: "docker-compose.yml",
dt: []byte(`
services:
foo:
`),
isCompose: true,
wantErr: true,
},
{
name: "build",
fn: "docker-compose.yml",
dt: []byte(`
services:
foo:
build: .
`),
isCompose: true,
wantErr: false,
},
{
name: "image",
fn: "docker-compose.yml",
dt: []byte(`
services:
simple:
image: nginx
`),
isCompose: true,
wantErr: false,
},
{
name: "unknown ext",
fn: "docker-compose.foo",
dt: []byte(`
services:
simple:
image: nginx
`),
isCompose: true,
wantErr: false,
},
{
name: "hcl",
fn: "docker-bake.hcl",
dt: []byte(`
target "default" {
dockerfile = "test"
}
`),
isCompose: false,
wantErr: false,
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
isCompose, err := validateComposeFile(tt.dt, tt.fn)
assert.Equal(t, tt.isCompose, isCompose)
if tt.wantErr {
require.Error(t, err)
} else {
require.NoError(t, err)
}
})
}
}
func TestComposeNullArgs(t *testing.T) {
var dt = []byte(`
services:
scratch:
build:
context: .
args:
FOO: null
bar: "baz"
`)
c, err := ParseCompose([]compose.ConfigFile{{Content: dt}}, nil)
require.NoError(t, err)
require.Equal(t, map[string]*string{"bar": ptrstr("baz")}, c.Targets[0].Args)
}
// chdir changes the current working directory to the named directory,
// and then restore the original working directory at the end of the test.
func chdir(t *testing.T, dir string) {
olddir, err := os.Getwd()
if err != nil {
t.Fatalf("chdir: %v", err)
}
if err := os.Chdir(dir); err != nil {
t.Fatalf("chdir %s: %v", dir, err)
}
t.Cleanup(func() {
if err := os.Chdir(olddir); err != nil {
t.Errorf("chdir to original working directory %s: %v", olddir, err)
os.Exit(1)
}
})
}

View File

@@ -1,200 +1,42 @@
package bake
import (
"os"
"strings"
hcl "github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/ext/userfunc"
"github.com/hashicorp/hcl/v2/hclsimple"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/hashicorp/hcl/v2/json"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclparse"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/function"
"github.com/zclconf/go-cty/cty/function/stdlib"
)
// Collection of generally useful functions in cty-using applications, which
// HCL supports. These functions are available for use in HCL files.
var (
stdlibFunctions = map[string]function.Function{
"absolute": stdlib.AbsoluteFunc,
"add": stdlib.AddFunc,
"and": stdlib.AndFunc,
"byteslen": stdlib.BytesLenFunc,
"bytesslice": stdlib.BytesSliceFunc,
"chomp": stdlib.ChompFunc,
"chunklist": stdlib.ChunklistFunc,
"ceil": stdlib.CeilFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"distinct": stdlib.DistinctFunc,
"divide": stdlib.DivideFunc,
"element": stdlib.ElementFunc,
"equal": stdlib.EqualFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"formatdate": stdlib.FormatDateFunc,
"format": stdlib.FormatFunc,
"formatlist": stdlib.FormatListFunc,
"greaterthan": stdlib.GreaterThanFunc,
"greaterthanorequalto": stdlib.GreaterThanOrEqualToFunc,
"hasindex": stdlib.HasIndexFunc,
"indent": stdlib.IndentFunc,
"index": stdlib.IndexFunc,
"int": stdlib.IntFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"join": stdlib.JoinFunc,
"length": stdlib.LengthFunc,
"lessthan": stdlib.LessThanFunc,
"lessthanorequalto": stdlib.LessThanOrEqualToFunc,
"log": stdlib.LogFunc,
"lookup": stdlib.LookupFunc,
"lower": stdlib.LowerFunc,
"max": stdlib.MaxFunc,
"merge": stdlib.MergeFunc,
"min": stdlib.MinFunc,
"modulo": stdlib.ModuloFunc,
"multiply": stdlib.MultiplyFunc,
"negate": stdlib.NegateFunc,
"notequal": stdlib.NotEqualFunc,
"not": stdlib.NotFunc,
"or": stdlib.OrFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regexall": stdlib.RegexAllFunc,
"regex": stdlib.RegexFunc,
"reverse": stdlib.ReverseFunc,
"reverselist": stdlib.ReverseListFunc,
"sethaselement": stdlib.SetHasElementFunc,
"setintersection": stdlib.SetIntersectionFunc,
"setsubtract": stdlib.SetSubtractFunc,
"setsymmetricdifference": stdlib.SetSymmetricDifferenceFunc,
"setunion": stdlib.SetUnionFunc,
"signum": stdlib.SignumFunc,
"slice": stdlib.SliceFunc,
"sort": stdlib.SortFunc,
"split": stdlib.SplitFunc,
"strlen": stdlib.StrlenFunc,
"substr": stdlib.SubstrFunc,
"subtract": stdlib.SubtractFunc,
"timeadd": stdlib.TimeAddFunc,
"title": stdlib.TitleFunc,
"trim": stdlib.TrimFunc,
"trimprefix": stdlib.TrimPrefixFunc,
"trimspace": stdlib.TrimSpaceFunc,
"trimsuffix": stdlib.TrimSuffixFunc,
"upper": stdlib.UpperFunc,
"values": stdlib.ValuesFunc,
"zipmap": stdlib.ZipmapFunc,
}
)
// Used in the first pass of decoding instead of the Config struct to disallow
// interpolation while parsing variable blocks.
type staticConfig struct {
Variables []*Variable `hcl:"variable,block"`
Remain hcl.Body `hcl:",remain"`
}
func ParseHCL(dt []byte, fn string) (_ *Config, err error) {
if strings.HasSuffix(fn, ".json") || strings.HasSuffix(fn, ".hcl") {
return parseHCL(dt, fn)
}
cfg, err := parseHCL(dt, fn+".hcl")
if err != nil {
cfg2, err2 := parseHCL(dt, fn+".json")
if err2 == nil {
return cfg2, nil
}
}
return cfg, err
}
func parseHCL(dt []byte, fn string) (_ *Config, err error) {
defer func() {
err = formatHCLError(dt, err)
}()
// Decode user defined functions, first parsing as hcl and falling back to
// json, returning errors based on the file suffix.
file, hcldiags := hclsyntax.ParseConfig(dt, fn, hcl.Pos{Line: 1, Column: 1})
if hcldiags.HasErrors() {
var jsondiags hcl.Diagnostics
file, jsondiags = json.Parse(dt, fn)
if jsondiags.HasErrors() {
fnl := strings.ToLower(fn)
if strings.HasSuffix(fnl, ".json") {
return nil, jsondiags
} else {
return nil, hcldiags
}
}
}
userFunctions, _, diags := userfunc.DecodeUserFunctions(file.Body, "function", func() *hcl.EvalContext {
return &hcl.EvalContext{
Functions: stdlibFunctions,
}
})
func ParseHCLFile(dt []byte, fn string) (*hcl.File, bool, error) {
var err error
if strings.HasSuffix(fn, ".json") {
f, diags := hclparse.NewParser().ParseJSON(dt, fn)
if diags.HasErrors() {
return nil, diags
err = diags
}
return f, true, err
}
if strings.HasSuffix(fn, ".hcl") {
f, diags := hclparse.NewParser().ParseHCL(dt, fn)
if diags.HasErrors() {
err = diags
}
return f, true, err
}
f, diags := hclparse.NewParser().ParseHCL(dt, fn+".hcl")
if diags.HasErrors() {
f, diags2 := hclparse.NewParser().ParseJSON(dt, fn+".json")
if !diags2.HasErrors() {
return f, true, nil
}
return nil, false, diags
}
return f, true, nil
}
var sc staticConfig
// Decode only variable blocks without interpolation.
if err := hclsimple.Decode(fn, dt, nil, &sc); err != nil {
return nil, err
}
// Set all variables to their default value if defined.
variables := make(map[string]cty.Value)
for _, variable := range sc.Variables {
variables[variable.Name] = cty.StringVal(variable.Default)
}
// Override default with values from environment.
for _, env := range os.Environ() {
parts := strings.SplitN(env, "=", 2)
name, value := parts[0], parts[1]
if _, ok := variables[name]; ok {
variables[name] = cty.StringVal(value)
}
}
functions := make(map[string]function.Function)
for k, v := range stdlibFunctions {
functions[k] = v
}
for k, v := range userFunctions {
functions[k] = v
}
ctx := &hcl.EvalContext{
Variables: variables,
Functions: functions,
}
var c Config
// Decode with variables and functions.
if err := hclsimple.Decode(fn, dt, ctx, &c); err != nil {
return nil, err
}
return &c, nil
}
func formatHCLError(dt []byte, err error) error {
func formatHCLError(err error, files []File) error {
if err == nil {
return nil
}
@@ -207,6 +49,13 @@ func formatHCLError(dt []byte, err error) error {
continue
}
if d.Subject != nil {
var dt []byte
for _, f := range files {
if d.Subject.Filename == f.Name {
dt = f.Data
break
}
}
src := errdefs.Source{
Info: &pb.SourceInfo{
Filename: d.Subject.Filename,

File diff suppressed because it is too large Load Diff

103
bake/hclparser/body.go Normal file
View File

@@ -0,0 +1,103 @@
package hclparser
import (
"github.com/hashicorp/hcl/v2"
)
type filterBody struct {
body hcl.Body
schema *hcl.BodySchema
exclude bool
}
func FilterIncludeBody(body hcl.Body, schema *hcl.BodySchema) hcl.Body {
return &filterBody{
body: body,
schema: schema,
}
}
func FilterExcludeBody(body hcl.Body, schema *hcl.BodySchema) hcl.Body {
return &filterBody{
body: body,
schema: schema,
exclude: true,
}
}
func (b *filterBody) Content(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Diagnostics) {
if b.exclude {
schema = subtractSchemas(schema, b.schema)
} else {
schema = intersectSchemas(schema, b.schema)
}
content, _, diag := b.body.PartialContent(schema)
return content, diag
}
func (b *filterBody) PartialContent(schema *hcl.BodySchema) (*hcl.BodyContent, hcl.Body, hcl.Diagnostics) {
if b.exclude {
schema = subtractSchemas(schema, b.schema)
} else {
schema = intersectSchemas(schema, b.schema)
}
return b.body.PartialContent(schema)
}
func (b *filterBody) JustAttributes() (hcl.Attributes, hcl.Diagnostics) {
return b.body.JustAttributes()
}
func (b *filterBody) MissingItemRange() hcl.Range {
return b.body.MissingItemRange()
}
func intersectSchemas(a, b *hcl.BodySchema) *hcl.BodySchema {
result := &hcl.BodySchema{}
for _, blockA := range a.Blocks {
for _, blockB := range b.Blocks {
if blockA.Type == blockB.Type {
result.Blocks = append(result.Blocks, blockA)
break
}
}
}
for _, attrA := range a.Attributes {
for _, attrB := range b.Attributes {
if attrA.Name == attrB.Name {
result.Attributes = append(result.Attributes, attrA)
break
}
}
}
return result
}
func subtractSchemas(a, b *hcl.BodySchema) *hcl.BodySchema {
result := &hcl.BodySchema{}
for _, blockA := range a.Blocks {
found := false
for _, blockB := range b.Blocks {
if blockA.Type == blockB.Type {
found = true
break
}
}
if !found {
result.Blocks = append(result.Blocks, blockA)
}
}
for _, attrA := range a.Attributes {
found := false
for _, attrB := range b.Attributes {
if attrA.Name == attrB.Name {
found = true
break
}
}
if !found {
result.Attributes = append(result.Attributes, attrA)
}
}
return result
}

145
bake/hclparser/expr.go Normal file
View File

@@ -0,0 +1,145 @@
package hclparser
import (
"reflect"
"unsafe"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/hclsyntax"
"github.com/pkg/errors"
)
func funcCalls(exp hcl.Expression) ([]string, hcl.Diagnostics) {
node, ok := exp.(hclsyntax.Node)
if !ok {
fns, err := jsonFuncCallsRecursive(exp)
if err != nil {
return nil, wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
return fns, nil
}
var funcnames []string
hcldiags := hclsyntax.VisitAll(node, func(n hclsyntax.Node) hcl.Diagnostics {
if fe, ok := n.(*hclsyntax.FunctionCallExpr); ok {
funcnames = append(funcnames, fe.Name)
}
return nil
})
if hcldiags.HasErrors() {
return nil, hcldiags
}
return funcnames, nil
}
func jsonFuncCallsRecursive(exp hcl.Expression) ([]string, error) {
je, ok := exp.(jsonExp)
if !ok {
return nil, errors.Errorf("invalid expression type %T", exp)
}
m := map[string]struct{}{}
for _, e := range elementExpressions(je, exp) {
if err := appendJSONFuncCalls(e, m); err != nil {
return nil, err
}
}
arr := make([]string, 0, len(m))
for n := range m {
arr = append(arr, n)
}
return arr, nil
}
func appendJSONFuncCalls(exp hcl.Expression, m map[string]struct{}) error {
v := reflect.ValueOf(exp)
if v.Kind() != reflect.Ptr || v.IsNil() {
return errors.Errorf("invalid json expression kind %T %v", exp, v.Kind())
}
src := v.Elem().FieldByName("src")
if src.IsZero() {
return errors.Errorf("%v has no property src", v.Elem().Type())
}
if src.Kind() != reflect.Interface {
return errors.Errorf("%v src is not interface: %v", src.Type(), src.Kind())
}
src = src.Elem()
if src.IsNil() {
return nil
}
if src.Kind() == reflect.Ptr {
src = src.Elem()
}
if src.Kind() != reflect.Struct {
return errors.Errorf("%v is not struct: %v", src.Type(), src.Kind())
}
// hcl/v2/json/ast#stringVal
val := src.FieldByName("Value")
if !val.IsValid() || val.IsZero() {
return nil
}
rng := src.FieldByName("SrcRange")
if rng.IsZero() {
return nil
}
var stringVal struct {
Value string
SrcRange hcl.Range
}
if !val.Type().AssignableTo(reflect.ValueOf(stringVal.Value).Type()) {
return nil
}
if !rng.Type().AssignableTo(reflect.ValueOf(stringVal.SrcRange).Type()) {
return nil
}
// reflect.Set does not work for unexported fields
stringVal.Value = *(*string)(unsafe.Pointer(val.UnsafeAddr()))
stringVal.SrcRange = *(*hcl.Range)(unsafe.Pointer(rng.UnsafeAddr()))
expr, diags := hclsyntax.ParseExpression([]byte(stringVal.Value), stringVal.SrcRange.Filename, stringVal.SrcRange.Start)
if diags.HasErrors() {
return nil
}
fns, err := funcCalls(expr)
if err != nil {
return err
}
for _, fn := range fns {
m[fn] = struct{}{}
}
return nil
}
type jsonExp interface {
ExprList() []hcl.Expression
ExprMap() []hcl.KeyValuePair
}
func elementExpressions(je jsonExp, exp hcl.Expression) []hcl.Expression {
list := je.ExprList()
if len(list) != 0 {
exp := make([]hcl.Expression, 0, len(list))
for _, e := range list {
if je, ok := e.(jsonExp); ok {
exp = append(exp, elementExpressions(je, e)...)
}
}
return exp
}
kvlist := je.ExprMap()
if len(kvlist) != 0 {
exp := make([]hcl.Expression, 0, len(kvlist)*2)
for _, p := range kvlist {
exp = append(exp, p.Key)
if je, ok := p.Value.(jsonExp); ok {
exp = append(exp, elementExpressions(je, p.Value)...)
}
}
return exp
}
return []hcl.Expression{exp}
}

903
bake/hclparser/hclparser.go Normal file
View File

@@ -0,0 +1,903 @@
package hclparser
import (
"encoding/binary"
"fmt"
"hash/fnv"
"math"
"math/big"
"reflect"
"strconv"
"strings"
"github.com/docker/buildx/util/userfunc"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/pkg/errors"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/gocty"
)
type Opt struct {
LookupVar func(string) (string, bool)
Vars map[string]string
ValidateLabel func(string) error
}
type variable struct {
Name string `json:"-" hcl:"name,label"`
Default *hcl.Attribute `json:"default,omitempty" hcl:"default,optional"`
Body hcl.Body `json:"-" hcl:",body"`
}
type functionDef struct {
Name string `json:"-" hcl:"name,label"`
Params *hcl.Attribute `json:"params,omitempty" hcl:"params"`
Variadic *hcl.Attribute `json:"variadic_param,omitempty" hcl:"variadic_params"`
Result *hcl.Attribute `json:"result,omitempty" hcl:"result"`
}
type inputs struct {
Variables []*variable `hcl:"variable,block"`
Functions []*functionDef `hcl:"function,block"`
Remain hcl.Body `json:"-" hcl:",remain"`
}
type parser struct {
opt Opt
vars map[string]*variable
attrs map[string]*hcl.Attribute
funcs map[string]*functionDef
blocks map[string]map[string][]*hcl.Block
blockValues map[*hcl.Block][]reflect.Value
blockEvalCtx map[*hcl.Block][]*hcl.EvalContext
blockNames map[*hcl.Block][]string
blockTypes map[string]reflect.Type
ectx *hcl.EvalContext
progressV map[uint64]struct{}
progressF map[uint64]struct{}
progressB map[uint64]map[string]struct{}
doneB map[uint64]map[string]struct{}
}
type WithEvalContexts interface {
GetEvalContexts(base *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) ([]*hcl.EvalContext, error)
}
type WithGetName interface {
GetName(ectx *hcl.EvalContext, block *hcl.Block, loadDeps func(hcl.Expression) hcl.Diagnostics) (string, error)
}
var errUndefined = errors.New("undefined")
func (p *parser) loadDeps(ectx *hcl.EvalContext, exp hcl.Expression, exclude map[string]struct{}, allowMissing bool) hcl.Diagnostics {
fns, hcldiags := funcCalls(exp)
if hcldiags.HasErrors() {
return hcldiags
}
for _, fn := range fns {
if err := p.resolveFunction(ectx, fn); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
for _, v := range exp.Variables() {
if _, ok := exclude[v.RootName()]; ok {
continue
}
if _, ok := p.blockTypes[v.RootName()]; ok {
blockType := v.RootName()
split := v.SimpleSplit().Rel
if len(split) == 0 {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: fmt.Sprintf("cannot access %s as a variable", blockType),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
}
blockName, ok := split[0].(hcl.TraverseAttr)
if !ok {
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: fmt.Sprintf("cannot traverse %s without attribute", blockType),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
}
blocks := p.blocks[blockType][blockName.Name]
if len(blocks) == 0 {
continue
}
var target *hcl.BodySchema
if len(split) > 1 {
if attr, ok := split[1].(hcl.TraverseAttr); ok {
target = &hcl.BodySchema{
Attributes: []hcl.AttributeSchema{{Name: attr.Name}},
Blocks: []hcl.BlockHeaderSchema{{Type: attr.Name}},
}
}
}
for _, block := range blocks {
if err := p.resolveBlock(block, target); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
} else {
if err := p.resolveValue(ectx, v.RootName()); err != nil {
if allowMissing && errors.Is(err, errUndefined) {
continue
}
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
}
}
}
return nil
}
// resolveFunction forces evaluation of a function, storing the result into the
// parser.
func (p *parser) resolveFunction(ectx *hcl.EvalContext, name string) error {
if _, ok := p.ectx.Functions[name]; ok {
return nil
}
if _, ok := ectx.Functions[name]; ok {
return nil
}
f, ok := p.funcs[name]
if !ok {
return errors.Wrapf(errUndefined, "function %q does not exist", name)
}
if _, ok := p.progressF[key(ectx, name)]; ok {
return errors.Errorf("function cycle not allowed for %s", name)
}
p.progressF[key(ectx, name)] = struct{}{}
if f.Result == nil {
return errors.Errorf("empty result not allowed for %s", name)
}
if f.Params == nil {
return errors.Errorf("empty params not allowed for %s", name)
}
paramExprs, paramsDiags := hcl.ExprList(f.Params.Expr)
if paramsDiags.HasErrors() {
return paramsDiags
}
var diags hcl.Diagnostics
params := map[string]struct{}{}
for _, paramExpr := range paramExprs {
param := hcl.ExprAsKeyword(paramExpr)
if param == "" {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid param element",
Detail: "Each parameter name must be an identifier.",
Subject: paramExpr.Range().Ptr(),
})
}
params[param] = struct{}{}
}
var variadic hcl.Expression
if f.Variadic != nil {
variadic = f.Variadic.Expr
param := hcl.ExprAsKeyword(variadic)
if param == "" {
diags = append(diags, &hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid param element",
Detail: "Each parameter name must be an identifier.",
Subject: f.Variadic.Range.Ptr(),
})
}
params[param] = struct{}{}
}
if diags.HasErrors() {
return diags
}
if diags := p.loadDeps(p.ectx, f.Result.Expr, params, false); diags.HasErrors() {
return diags
}
v, diags := userfunc.NewFunction(f.Params.Expr, variadic, f.Result.Expr, func() *hcl.EvalContext {
return p.ectx
})
if diags.HasErrors() {
return diags
}
p.ectx.Functions[name] = v
return nil
}
// resolveValue forces evaluation of a named value, storing the result into the
// parser.
func (p *parser) resolveValue(ectx *hcl.EvalContext, name string) (err error) {
if _, ok := p.ectx.Variables[name]; ok {
return nil
}
if _, ok := ectx.Variables[name]; ok {
return nil
}
if _, ok := p.progressV[key(ectx, name)]; ok {
return errors.Errorf("variable cycle not allowed for %s", name)
}
p.progressV[key(ectx, name)] = struct{}{}
var v *cty.Value
defer func() {
if v != nil {
p.ectx.Variables[name] = *v
}
}()
def, ok := p.attrs[name]
if _, builtin := p.opt.Vars[name]; !ok && !builtin {
vr, ok := p.vars[name]
if !ok {
return errors.Wrapf(errUndefined, "variable %q does not exist", name)
}
def = vr.Default
ectx = p.ectx
}
if def == nil {
val, ok := p.opt.Vars[name]
if !ok {
val, _ = p.opt.LookupVar(name)
}
vv := cty.StringVal(val)
v = &vv
return
}
if diags := p.loadDeps(ectx, def.Expr, nil, true); diags.HasErrors() {
return diags
}
vv, diags := def.Expr.Value(ectx)
if diags.HasErrors() {
return diags
}
_, isVar := p.vars[name]
if envv, ok := p.opt.LookupVar(name); ok && isVar {
switch {
case vv.Type().Equals(cty.Bool):
b, err := strconv.ParseBool(envv)
if err != nil {
return errors.Wrapf(err, "failed to parse %s as bool", name)
}
vv = cty.BoolVal(b)
case vv.Type().Equals(cty.String), vv.Type().Equals(cty.DynamicPseudoType):
vv = cty.StringVal(envv)
case vv.Type().Equals(cty.Number):
n, err := strconv.ParseFloat(envv, 64)
if err == nil && (math.IsNaN(n) || math.IsInf(n, 0)) {
err = errors.Errorf("invalid number value")
}
if err != nil {
return errors.Wrapf(err, "failed to parse %s as number", name)
}
vv = cty.NumberVal(big.NewFloat(n))
default:
// TODO: support lists with csv values
return errors.Errorf("unsupported type %s for variable %s", vv.Type().FriendlyName(), name)
}
}
v = &vv
return nil
}
// resolveBlock force evaluates a block, storing the result in the parser. If a
// target schema is provided, only the attributes and blocks present in the
// schema will be evaluated.
func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) {
// prepare the variable map for this type
if _, ok := p.ectx.Variables[block.Type]; !ok {
p.ectx.Variables[block.Type] = cty.MapValEmpty(cty.Map(cty.String))
}
// prepare the output destination and evaluation context
t, ok := p.blockTypes[block.Type]
if !ok {
return nil
}
var outputs []reflect.Value
var ectxs []*hcl.EvalContext
if prev, ok := p.blockValues[block]; ok {
outputs = prev
ectxs = p.blockEvalCtx[block]
} else {
if v, ok := reflect.New(t).Interface().(WithEvalContexts); ok {
ectxs, err = v.GetEvalContexts(p.ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(p.ectx, expr, nil, true)
})
if err != nil {
return err
}
for _, ectx := range ectxs {
if ectx != p.ectx && ectx.Parent() != p.ectx {
return errors.Errorf("EvalContext must return a context with the correct parent")
}
}
} else {
ectxs = append([]*hcl.EvalContext{}, p.ectx)
}
for range ectxs {
outputs = append(outputs, reflect.New(t))
}
}
p.blockValues[block] = outputs
p.blockEvalCtx[block] = ectxs
for i, output := range outputs {
target := target
ectx := ectxs[i]
name := block.Labels[0]
if names, ok := p.blockNames[block]; ok {
name = names[i]
}
if _, ok := p.doneB[key(block, ectx)]; !ok {
p.doneB[key(block, ectx)] = map[string]struct{}{}
}
if _, ok := p.progressB[key(block, ectx)]; !ok {
p.progressB[key(block, ectx)] = map[string]struct{}{}
}
if target != nil {
// filter out attributes and blocks that are already evaluated
original := target
target = &hcl.BodySchema{}
for _, a := range original.Attributes {
if _, ok := p.doneB[key(block, ectx)][a.Name]; !ok {
target.Attributes = append(target.Attributes, a)
}
}
for _, b := range original.Blocks {
if _, ok := p.doneB[key(block, ectx)][b.Type]; !ok {
target.Blocks = append(target.Blocks, b)
}
}
if len(target.Attributes) == 0 && len(target.Blocks) == 0 {
return nil
}
}
if target != nil {
// detect reference cycles
for _, a := range target.Attributes {
if _, ok := p.progressB[key(block, ectx)][a.Name]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, a.Name)
}
}
for _, b := range target.Blocks {
if _, ok := p.progressB[key(block, ectx)][b.Type]; ok {
return errors.Errorf("reference cycle not allowed for %s.%s.%s", block.Type, name, b.Type)
}
}
for _, a := range target.Attributes {
p.progressB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.progressB[key(block, ectx)][b.Type] = struct{}{}
}
}
// create a filtered body that contains only the target properties
body := func() hcl.Body {
if target != nil {
return FilterIncludeBody(block.Body, target)
}
filter := &hcl.BodySchema{}
for k := range p.doneB[key(block, ectx)] {
filter.Attributes = append(filter.Attributes, hcl.AttributeSchema{Name: k})
filter.Blocks = append(filter.Blocks, hcl.BlockHeaderSchema{Type: k})
}
return FilterExcludeBody(block.Body, filter)
}
// load dependencies from all targeted properties
schema, _ := gohcl.ImpliedBodySchema(reflect.New(t).Interface())
content, _, diag := body().PartialContent(schema)
if diag.HasErrors() {
return diag
}
for _, a := range content.Attributes {
diag := p.loadDeps(ectx, a.Expr, nil, true)
if diag.HasErrors() {
return diag
}
}
for _, b := range content.Blocks {
err := p.resolveBlock(b, nil)
if err != nil {
return err
}
}
// decode!
diag = gohcl.DecodeBody(body(), ectx, output.Interface())
if diag.HasErrors() {
return diag
}
// mark all targeted properties as done
for _, a := range content.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range content.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
if target != nil {
for _, a := range target.Attributes {
p.doneB[key(block, ectx)][a.Name] = struct{}{}
}
for _, b := range target.Blocks {
p.doneB[key(block, ectx)][b.Type] = struct{}{}
}
}
// store the result into the evaluation context (so it can be referenced)
outputType, err := gocty.ImpliedType(output.Interface())
if err != nil {
return err
}
outputValue, err := gocty.ToCtyValue(output.Interface(), outputType)
if err != nil {
return err
}
var m map[string]cty.Value
if m2, ok := p.ectx.Variables[block.Type]; ok {
m = m2.AsValueMap()
}
if m == nil {
m = map[string]cty.Value{}
}
m[name] = outputValue
p.ectx.Variables[block.Type] = cty.MapVal(m)
}
return nil
}
// resolveBlockNames returns the names of the block, calling resolveBlock to
// evaluate any label fields to correctly resolve the name.
func (p *parser) resolveBlockNames(block *hcl.Block) ([]string, error) {
if names, ok := p.blockNames[block]; ok {
return names, nil
}
if err := p.resolveBlock(block, &hcl.BodySchema{}); err != nil {
return nil, err
}
names := make([]string, 0, len(p.blockValues[block]))
for i, val := range p.blockValues[block] {
ectx := p.blockEvalCtx[block][i]
name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
if v, ok := val.Interface().(WithGetName); ok {
var err error
name, err = v.GetName(ectx, block, func(expr hcl.Expression) hcl.Diagnostics {
return p.loadDeps(ectx, expr, nil, true)
})
if err != nil {
return nil, err
}
if err := p.opt.ValidateLabel(name); err != nil {
return nil, err
}
}
setName(val, name)
names = append(names, name)
}
found := map[string]struct{}{}
for _, name := range names {
if _, ok := found[name]; ok {
return nil, errors.Errorf("duplicate name %q", name)
}
found[name] = struct{}{}
}
p.blockNames[block] = names
return names, nil
}
func Parse(b hcl.Body, opt Opt, val interface{}) (map[string]map[string][]string, hcl.Diagnostics) {
reserved := map[string]struct{}{}
schema, _ := gohcl.ImpliedBodySchema(val)
for _, bs := range schema.Blocks {
reserved[bs.Type] = struct{}{}
}
for k := range opt.Vars {
reserved[k] = struct{}{}
}
var defs inputs
if err := gohcl.DecodeBody(b, nil, &defs); err != nil {
return nil, err
}
defsSchema, _ := gohcl.ImpliedBodySchema(defs)
if opt.LookupVar == nil {
opt.LookupVar = func(string) (string, bool) {
return "", false
}
}
if opt.ValidateLabel == nil {
opt.ValidateLabel = func(string) error {
return nil
}
}
p := &parser{
opt: opt,
vars: map[string]*variable{},
attrs: map[string]*hcl.Attribute{},
funcs: map[string]*functionDef{},
blocks: map[string]map[string][]*hcl.Block{},
blockValues: map[*hcl.Block][]reflect.Value{},
blockEvalCtx: map[*hcl.Block][]*hcl.EvalContext{},
blockNames: map[*hcl.Block][]string{},
blockTypes: map[string]reflect.Type{},
ectx: &hcl.EvalContext{
Variables: map[string]cty.Value{},
Functions: Stdlib(),
},
progressV: map[uint64]struct{}{},
progressF: map[uint64]struct{}{},
progressB: map[uint64]map[string]struct{}{},
doneB: map[uint64]map[string]struct{}{},
}
for _, v := range defs.Variables {
// TODO: validate name
if _, ok := reserved[v.Name]; ok {
continue
}
p.vars[v.Name] = v
}
for _, v := range defs.Functions {
// TODO: validate name
if _, ok := reserved[v.Name]; ok {
continue
}
p.funcs[v.Name] = v
}
content, b, diags := b.PartialContent(schema)
if diags.HasErrors() {
return nil, diags
}
blocks, b, diags := b.PartialContent(defsSchema)
if diags.HasErrors() {
return nil, diags
}
attrs, diags := b.JustAttributes()
if diags.HasErrors() {
if d := removeAttributesDiags(diags, reserved, p.vars); len(d) > 0 {
return nil, d
}
}
for _, v := range attrs {
if _, ok := reserved[v.Name]; ok {
continue
}
p.attrs[v.Name] = v
}
delete(p.attrs, "function")
for k := range p.opt.Vars {
_ = p.resolveValue(p.ectx, k)
}
for _, a := range content.Attributes {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: "global attributes currently not supported",
Subject: &a.Range,
Context: &a.Range,
},
}
}
for k := range p.vars {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
r := p.vars[k].Body.MissingItemRange()
return nil, wrapErrorDiagnostic("Invalid value", err, &r, &r)
}
}
for k := range p.funcs {
if err := p.resolveFunction(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
var subject *hcl.Range
var context *hcl.Range
if p.funcs[k].Params != nil {
subject = &p.funcs[k].Params.Range
context = subject
} else {
for _, block := range blocks.Blocks {
if block.Type == "function" && len(block.Labels) == 1 && block.Labels[0] == k {
subject = &block.LabelRanges[0]
context = &block.DefRange
break
}
}
}
return nil, wrapErrorDiagnostic("Invalid function", err, subject, context)
}
}
type value struct {
reflect.Value
idx int
}
type field struct {
idx int
typ reflect.Type
values map[string]value
}
types := map[string]field{}
renamed := map[string]map[string][]string{}
vt := reflect.ValueOf(val).Elem().Type()
for i := 0; i < vt.NumField(); i++ {
tags := strings.Split(vt.Field(i).Tag.Get("hcl"), ",")
p.blockTypes[tags[0]] = vt.Field(i).Type.Elem().Elem()
types[tags[0]] = field{
idx: i,
typ: vt.Field(i).Type,
values: make(map[string]value),
}
renamed[tags[0]] = map[string][]string{}
}
tmpBlocks := map[string]map[string][]*hcl.Block{}
for _, b := range content.Blocks {
if len(b.Labels) == 0 || len(b.Labels) > 1 {
return nil, hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid block",
Detail: fmt.Sprintf("invalid block label: %v", b.Labels),
Subject: &b.LabelRanges[0],
Context: &b.LabelRanges[0],
},
}
}
bm, ok := tmpBlocks[b.Type]
if !ok {
bm = map[string][]*hcl.Block{}
tmpBlocks[b.Type] = bm
}
names, err := p.resolveBlockNames(b)
if err != nil {
return nil, wrapErrorDiagnostic("Invalid name", err, &b.LabelRanges[0], &b.LabelRanges[0])
}
for _, name := range names {
bm[name] = append(bm[name], b)
renamed[b.Type][b.Labels[0]] = append(renamed[b.Type][b.Labels[0]], name)
}
}
p.blocks = tmpBlocks
diags = hcl.Diagnostics{}
for _, b := range content.Blocks {
v := reflect.ValueOf(val)
err := p.resolveBlock(b, nil)
if err != nil {
if diag, ok := err.(hcl.Diagnostics); ok {
if diag.HasErrors() {
diags = append(diags, diag...)
continue
}
} else {
return nil, wrapErrorDiagnostic("Invalid block", err, &b.LabelRanges[0], &b.DefRange)
}
}
vvs := p.blockValues[b]
for _, vv := range vvs {
t := types[b.Type]
lblIndex, lblExists := getNameIndex(vv)
lblName, _ := getName(vv)
oldValue, exists := t.values[lblName]
if !exists && lblExists {
if v.Elem().Field(t.idx).Type().Kind() == reflect.Slice {
for i := 0; i < v.Elem().Field(t.idx).Len(); i++ {
if lblName == v.Elem().Field(t.idx).Index(i).Elem().Field(lblIndex).String() {
exists = true
oldValue = value{Value: v.Elem().Field(t.idx).Index(i), idx: i}
break
}
}
}
}
if exists {
if m := oldValue.Value.MethodByName("Merge"); m.IsValid() {
m.Call([]reflect.Value{vv})
} else {
v.Elem().Field(t.idx).Index(oldValue.idx).Set(vv)
}
} else {
slice := v.Elem().Field(t.idx)
if slice.IsNil() {
slice = reflect.New(t.typ).Elem()
}
t.values[lblName] = value{Value: vv, idx: slice.Len()}
v.Elem().Field(t.idx).Set(reflect.Append(slice, vv))
}
}
}
if diags.HasErrors() {
return nil, diags
}
for k := range p.attrs {
if err := p.resolveValue(p.ectx, k); err != nil {
if diags, ok := err.(hcl.Diagnostics); ok {
return nil, diags
}
return nil, wrapErrorDiagnostic("Invalid attribute", err, &p.attrs[k].Range, &p.attrs[k].Range)
}
}
return renamed, nil
}
// wrapErrorDiagnostic wraps an error into a hcl.Diagnostics object.
// If the error is already an hcl.Diagnostics object, it is returned as is.
func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context *hcl.Range) hcl.Diagnostics {
switch err := err.(type) {
case *hcl.Diagnostic:
return hcl.Diagnostics{err}
case hcl.Diagnostics:
return err
default:
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: message,
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
}
func setName(v reflect.Value, name string) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
v.Elem().Field(i).Set(reflect.ValueOf(name))
}
}
}
}
func getName(v reflect.Value) (string, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return v.Elem().Field(i).String(), true
}
}
}
return "", false
}
func getNameIndex(v reflect.Value) (int, bool) {
numFields := v.Elem().Type().NumField()
for i := 0; i < numFields; i++ {
parts := strings.Split(v.Elem().Type().Field(i).Tag.Get("hcl"), ",")
for _, t := range parts[1:] {
if t == "label" {
return i, true
}
}
}
return 0, false
}
func removeAttributesDiags(diags hcl.Diagnostics, reserved map[string]struct{}, vars map[string]*variable) hcl.Diagnostics {
var fdiags hcl.Diagnostics
for _, d := range diags {
if fout := func(d *hcl.Diagnostic) bool {
// https://github.com/docker/buildx/pull/541
if d.Detail == "Blocks are not allowed here." {
return true
}
for r := range reserved {
// JSON body objects don't handle repeated blocks like HCL but
// reserved name attributes should be allowed when multi bodies are merged.
// https://github.com/hashicorp/hcl/blob/main/json/spec.md#blocks
if strings.HasPrefix(d.Detail, fmt.Sprintf(`Argument "%s" was already set at `, r)) {
return true
}
}
for v := range vars {
// Do the same for global variables
if strings.HasPrefix(d.Detail, fmt.Sprintf(`Argument "%s" was already set at `, v)) {
return true
}
}
return false
}(d); !fout {
fdiags = append(fdiags, d)
}
}
return fdiags
}
// key returns a unique hash for the given values
func key(ks ...any) uint64 {
hash := fnv.New64a()
for _, k := range ks {
v := reflect.ValueOf(k)
switch v.Kind() {
case reflect.String:
hash.Write([]byte(v.String()))
case reflect.Pointer:
ptr := reflect.ValueOf(k).Pointer()
binary.Write(hash, binary.LittleEndian, uint64(ptr))
default:
panic(fmt.Sprintf("unknown key kind %s", v.Kind().String()))
}
}
return hash.Sum64()
}

135
bake/hclparser/stdlib.go Normal file
View File

@@ -0,0 +1,135 @@
package hclparser
import (
"time"
"github.com/hashicorp/go-cty-funcs/cidr"
"github.com/hashicorp/go-cty-funcs/crypto"
"github.com/hashicorp/go-cty-funcs/encoding"
"github.com/hashicorp/go-cty-funcs/uuid"
"github.com/hashicorp/hcl/v2/ext/tryfunc"
"github.com/hashicorp/hcl/v2/ext/typeexpr"
"github.com/zclconf/go-cty/cty"
"github.com/zclconf/go-cty/cty/function"
"github.com/zclconf/go-cty/cty/function/stdlib"
)
var stdlibFunctions = map[string]function.Function{
"absolute": stdlib.AbsoluteFunc,
"add": stdlib.AddFunc,
"and": stdlib.AndFunc,
"base64decode": encoding.Base64DecodeFunc,
"base64encode": encoding.Base64EncodeFunc,
"bcrypt": crypto.BcryptFunc,
"byteslen": stdlib.BytesLenFunc,
"bytesslice": stdlib.BytesSliceFunc,
"can": tryfunc.CanFunc,
"ceil": stdlib.CeilFunc,
"chomp": stdlib.ChompFunc,
"chunklist": stdlib.ChunklistFunc,
"cidrhost": cidr.HostFunc,
"cidrnetmask": cidr.NetmaskFunc,
"cidrsubnet": cidr.SubnetFunc,
"cidrsubnets": cidr.SubnetsFunc,
"coalesce": stdlib.CoalesceFunc,
"coalescelist": stdlib.CoalesceListFunc,
"compact": stdlib.CompactFunc,
"concat": stdlib.ConcatFunc,
"contains": stdlib.ContainsFunc,
"convert": typeexpr.ConvertFunc,
"csvdecode": stdlib.CSVDecodeFunc,
"distinct": stdlib.DistinctFunc,
"divide": stdlib.DivideFunc,
"element": stdlib.ElementFunc,
"equal": stdlib.EqualFunc,
"flatten": stdlib.FlattenFunc,
"floor": stdlib.FloorFunc,
"format": stdlib.FormatFunc,
"formatdate": stdlib.FormatDateFunc,
"formatlist": stdlib.FormatListFunc,
"greaterthan": stdlib.GreaterThanFunc,
"greaterthanorequalto": stdlib.GreaterThanOrEqualToFunc,
"hasindex": stdlib.HasIndexFunc,
"indent": stdlib.IndentFunc,
"index": stdlib.IndexFunc,
"int": stdlib.IntFunc,
"join": stdlib.JoinFunc,
"jsondecode": stdlib.JSONDecodeFunc,
"jsonencode": stdlib.JSONEncodeFunc,
"keys": stdlib.KeysFunc,
"length": stdlib.LengthFunc,
"lessthan": stdlib.LessThanFunc,
"lessthanorequalto": stdlib.LessThanOrEqualToFunc,
"log": stdlib.LogFunc,
"lookup": stdlib.LookupFunc,
"lower": stdlib.LowerFunc,
"max": stdlib.MaxFunc,
"md5": crypto.Md5Func,
"merge": stdlib.MergeFunc,
"min": stdlib.MinFunc,
"modulo": stdlib.ModuloFunc,
"multiply": stdlib.MultiplyFunc,
"negate": stdlib.NegateFunc,
"not": stdlib.NotFunc,
"notequal": stdlib.NotEqualFunc,
"or": stdlib.OrFunc,
"parseint": stdlib.ParseIntFunc,
"pow": stdlib.PowFunc,
"range": stdlib.RangeFunc,
"regex_replace": stdlib.RegexReplaceFunc,
"regex": stdlib.RegexFunc,
"regexall": stdlib.RegexAllFunc,
"replace": stdlib.ReplaceFunc,
"reverse": stdlib.ReverseFunc,
"reverselist": stdlib.ReverseListFunc,
"rsadecrypt": crypto.RsaDecryptFunc,
"sethaselement": stdlib.SetHasElementFunc,
"setintersection": stdlib.SetIntersectionFunc,
"setproduct": stdlib.SetProductFunc,
"setsubtract": stdlib.SetSubtractFunc,
"setsymmetricdifference": stdlib.SetSymmetricDifferenceFunc,
"setunion": stdlib.SetUnionFunc,
"sha1": crypto.Sha1Func,
"sha256": crypto.Sha256Func,
"sha512": crypto.Sha512Func,
"signum": stdlib.SignumFunc,
"slice": stdlib.SliceFunc,
"sort": stdlib.SortFunc,
"split": stdlib.SplitFunc,
"strlen": stdlib.StrlenFunc,
"substr": stdlib.SubstrFunc,
"subtract": stdlib.SubtractFunc,
"timeadd": stdlib.TimeAddFunc,
"timestamp": timestampFunc,
"title": stdlib.TitleFunc,
"trim": stdlib.TrimFunc,
"trimprefix": stdlib.TrimPrefixFunc,
"trimspace": stdlib.TrimSpaceFunc,
"trimsuffix": stdlib.TrimSuffixFunc,
"try": tryfunc.TryFunc,
"upper": stdlib.UpperFunc,
"urlencode": encoding.URLEncodeFunc,
"uuidv4": uuid.V4Func,
"uuidv5": uuid.V5Func,
"values": stdlib.ValuesFunc,
"zipmap": stdlib.ZipmapFunc,
}
// timestampFunc constructs a function that returns a string representation of the current date and time.
//
// This function was imported from terraform's datetime utilities.
var timestampFunc = function.New(&function.Spec{
Params: []function.Parameter{},
Type: function.StaticReturnType(cty.String),
Impl: func(args []cty.Value, retType cty.Type) (cty.Value, error) {
return cty.StringVal(time.Now().UTC().Format(time.RFC3339)), nil
},
})
func Stdlib() map[string]function.Function {
funcs := make(map[string]function.Function, len(stdlibFunctions))
for k, v := range stdlibFunctions {
funcs[k] = v
}
return funcs
}

View File

@@ -4,14 +4,16 @@ import (
"archive/tar"
"bytes"
"context"
"strings"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/client/llb"
"github.com/moby/buildkit/frontend/dockerui"
gwclient "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/session"
"github.com/pkg/errors"
)
@@ -20,10 +22,17 @@ type Input struct {
URL string
}
func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
st, filename, ok := detectHttpContext(url)
if !ok {
st, ok = detectGitContext(url)
func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, names []string, pw progress.Writer) ([]File, *Input, error) {
var session []session.Attachable
var filename string
st, ok := dockerui.DetectGitContext(url, false)
if ok {
ssh, err := controllerapi.CreateSSH([]*controllerapi.SSH{{ID: "default"}})
if err == nil {
session = append(session, ssh)
}
} else {
st, filename, ok = dockerui.DetectHTTPContext(url)
if !ok {
return nil, nil, errors.Errorf("not url context")
}
@@ -32,25 +41,25 @@ func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, na
inp := &Input{State: st, URL: url}
var files []File
var di *build.DriverInfo
for _, d := range dis {
if d.Err == nil {
di = &d
var node *builder.Node
for i, n := range nodes {
if n.Err == nil {
node = &nodes[i]
continue
}
}
if di == nil {
if node == nil {
return nil, nil, nil
}
c, err := driver.Boot(ctx, di.Driver, pw)
c, err := driver.Boot(ctx, ctx, node.Driver, pw)
if err != nil {
return nil, nil, err
}
ch, done := progress.NewChannel(pw)
defer func() { <-done }()
_, err = c.Build(ctx, client.SolveOpt{}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
_, err = c.Build(ctx, client.SolveOpt{Session: session}, "buildx", func(ctx context.Context, c gwclient.Client) (*gwclient.Result, error) {
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
@@ -82,51 +91,6 @@ func ReadRemoteFiles(ctx context.Context, dis []build.DriverInfo, url string, na
return files, inp, nil
}
func IsRemoteURL(url string) bool {
if _, _, ok := detectHttpContext(url); ok {
return true
}
if _, ok := detectGitContext(url); ok {
return true
}
return false
}
func detectHttpContext(url string) (*llb.State, string, bool) {
if httpPrefix.MatchString(url) {
httpContext := llb.HTTP(url, llb.Filename("context"), llb.WithCustomName("[internal] load remote build context"))
return &httpContext, "context", true
}
return nil, "", false
}
func detectGitContext(ref string) (*llb.State, bool) {
found := false
if httpPrefix.MatchString(ref) && gitURLPathWithFragmentSuffix.MatchString(ref) {
found = true
}
for _, prefix := range []string{"git://", "github.com/", "git@"} {
if strings.HasPrefix(ref, prefix) {
found = true
break
}
}
if !found {
return nil, false
}
parts := strings.SplitN(ref, "#", 2)
branch := ""
if len(parts) > 1 {
branch = parts[1]
}
gitOpts := []llb.GitOption{llb.WithCustomName("[internal] load git source " + ref)}
st := llb.Git(parts[0], branch, gitOpts...)
return &st, true
}
func isArchive(header []byte) bool {
for _, m := range [][]byte{
{0x42, 0x5A, 0x68}, // bzip2

File diff suppressed because it is too large Load Diff

View File

@@ -1,60 +0,0 @@
package build
import (
"encoding/csv"
"strings"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func ParseCacheEntry(in []string) ([]client.CacheOptionsEntry, error) {
imports := make([]client.CacheOptionsEntry, 0, len(in))
for _, in := range in {
csvReader := csv.NewReader(strings.NewReader(in))
fields, err := csvReader.Read()
if err != nil {
return nil, err
}
if isRefOnlyFormat(fields) {
for _, field := range fields {
imports = append(imports, client.CacheOptionsEntry{
Type: "registry",
Attrs: map[string]string{"ref": field},
})
}
continue
}
im := client.CacheOptionsEntry{
Attrs: map[string]string{},
}
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) != 2 {
return nil, errors.Errorf("invalid value %s", field)
}
key := strings.ToLower(parts[0])
value := parts[1]
switch key {
case "type":
im.Type = value
default:
im.Attrs[key] = value
}
}
if im.Type == "" {
return nil, errors.Errorf("type required form> %q", in)
}
imports = append(imports, im)
}
return imports, nil
}
func isRefOnlyFormat(in []string) bool {
for _, v := range in {
if strings.Contains(v, "=") {
return false
}
}
return true
}

115
build/git.go Normal file
View File

@@ -0,0 +1,115 @@
package build
import (
"context"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"github.com/docker/buildx/util/gitutil"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
)
const DockerfileLabel = "com.docker.image.source.entrypoint"
func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath string) (res map[string]string, _ error) {
res = make(map[string]string)
if contextPath == "" {
return
}
setGitLabels := false
if v, ok := os.LookupEnv("BUILDX_GIT_LABELS"); ok {
if v == "full" { // backward compatibility with old "full" mode
setGitLabels = true
} else if v, err := strconv.ParseBool(v); err == nil {
setGitLabels = v
}
}
setGitInfo := true
if v, ok := os.LookupEnv("BUILDX_GIT_INFO"); ok {
if v, err := strconv.ParseBool(v); err == nil {
setGitInfo = v
}
}
if !setGitLabels && !setGitInfo {
return
}
// figure out in which directory the git command needs to run in
var wd string
if filepath.IsAbs(contextPath) {
wd = contextPath
} else {
cwd, _ := os.Getwd()
wd, _ = filepath.Abs(filepath.Join(cwd, contextPath))
}
gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd))
if err != nil {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("buildx: git was not found in the system. Current commit information was not captured by the build")
}
return
}
if !gitc.IsInsideWorkTree() {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("buildx: failed to read current commit information with git rev-parse --is-inside-work-tree")
}
return res, nil
}
if sha, err := gitc.FullCommit(); err != nil && !gitutil.IsUnknownRevision(err) {
return res, errors.Wrapf(err, "buildx: failed to get git commit")
} else if sha != "" {
checkDirty := false
if v, ok := os.LookupEnv("BUILDX_GIT_CHECK_DIRTY"); ok {
if v, err := strconv.ParseBool(v); err == nil {
checkDirty = v
}
}
if checkDirty && gitc.IsDirty() {
sha += "-dirty"
}
if setGitLabels {
res["label:"+specs.AnnotationRevision] = sha
}
if setGitInfo {
res["vcs:revision"] = sha
}
}
if rurl, err := gitc.RemoteURL(); err == nil && rurl != "" {
if setGitLabels {
res["label:"+specs.AnnotationSource] = rurl
}
if setGitInfo {
res["vcs:source"] = rurl
}
}
if setGitLabels {
if root, err := gitc.RootDir(); err != nil {
return res, errors.Wrapf(err, "buildx: failed to get git root dir")
} else if root != "" {
if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile")
}
if !filepath.IsAbs(dockerfilePath) {
cwd, _ := os.Getwd()
dockerfilePath = filepath.Join(cwd, dockerfilePath)
}
dockerfilePath, _ = filepath.Rel(root, dockerfilePath)
if !strings.HasPrefix(dockerfilePath, "..") {
res["label:"+DockerfileLabel] = dockerfilePath
}
}
}
return
}

156
build/git_test.go Normal file
View File

@@ -0,0 +1,156 @@
package build
import (
"context"
"os"
"path"
"path/filepath"
"strings"
"testing"
"github.com/docker/buildx/util/gitutil"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func setupTest(tb testing.TB) {
gitutil.Mktmp(tb)
c, err := gitutil.New()
require.NoError(tb, err)
gitutil.GitInit(c, tb)
df := []byte("FROM alpine:latest\n")
assert.NoError(tb, os.WriteFile("Dockerfile", df, 0644))
gitutil.GitAdd(c, tb, "Dockerfile")
gitutil.GitCommit(c, tb, "initial commit")
gitutil.GitSetRemote(c, tb, "origin", "git@github.com:docker/buildx.git")
}
func TestGetGitAttributesNotGitRepo(t *testing.T) {
_, err := getGitAttributes(context.Background(), t.TempDir(), "Dockerfile")
assert.NoError(t, err)
}
func TestGetGitAttributesBadGitRepo(t *testing.T) {
tmp := t.TempDir()
require.NoError(t, os.MkdirAll(path.Join(tmp, ".git"), 0755))
_, err := getGitAttributes(context.Background(), tmp, "Dockerfile")
assert.Error(t, err)
}
func TestGetGitAttributesNoContext(t *testing.T) {
setupTest(t)
gitattrs, err := getGitAttributes(context.Background(), "", "Dockerfile")
assert.NoError(t, err)
assert.Empty(t, gitattrs)
}
func TestGetGitAttributes(t *testing.T) {
cases := []struct {
name string
envGitLabels string
envGitInfo string
expected []string
}{
{
name: "default",
envGitLabels: "",
envGitInfo: "",
expected: []string{
"vcs:revision",
"vcs:source",
},
},
{
name: "none",
envGitLabels: "false",
envGitInfo: "false",
expected: []string{},
},
{
name: "gitinfo",
envGitLabels: "false",
envGitInfo: "true",
expected: []string{
"vcs:revision",
"vcs:source",
},
},
{
name: "gitlabels",
envGitLabels: "true",
envGitInfo: "false",
expected: []string{
"label:" + DockerfileLabel,
"label:" + specs.AnnotationRevision,
"label:" + specs.AnnotationSource,
},
},
{
name: "both",
envGitLabels: "true",
envGitInfo: "",
expected: []string{
"label:" + DockerfileLabel,
"label:" + specs.AnnotationRevision,
"label:" + specs.AnnotationSource,
"vcs:revision",
"vcs:source",
},
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
setupTest(t)
if tt.envGitLabels != "" {
t.Setenv("BUILDX_GIT_LABELS", tt.envGitLabels)
}
if tt.envGitInfo != "" {
t.Setenv("BUILDX_GIT_INFO", tt.envGitInfo)
}
gitattrs, err := getGitAttributes(context.Background(), ".", "Dockerfile")
require.NoError(t, err)
for _, e := range tt.expected {
assert.Contains(t, gitattrs, e)
assert.NotEmpty(t, gitattrs[e])
if e == "label:"+DockerfileLabel {
assert.Equal(t, "Dockerfile", gitattrs[e])
} else if e == "label:"+specs.AnnotationSource || e == "vcs:source" {
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs[e])
}
}
})
}
}
func TestGetGitAttributesDirty(t *testing.T) {
setupTest(t)
t.Setenv("BUILDX_GIT_CHECK_DIRTY", "true")
// make a change to test dirty flag
df := []byte("FROM alpine:edge\n")
require.NoError(t, os.Mkdir("dir", 0755))
require.NoError(t, os.WriteFile(filepath.Join("dir", "Dockerfile"), df, 0644))
t.Setenv("BUILDX_GIT_LABELS", "true")
gitattrs, _ := getGitAttributes(context.Background(), ".", "Dockerfile")
assert.Equal(t, 5, len(gitattrs))
assert.Contains(t, gitattrs, "label:"+DockerfileLabel)
assert.Equal(t, "Dockerfile", gitattrs["label:"+DockerfileLabel])
assert.Contains(t, gitattrs, "label:"+specs.AnnotationSource)
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs["label:"+specs.AnnotationSource])
assert.Contains(t, gitattrs, "label:"+specs.AnnotationRevision)
assert.True(t, strings.HasSuffix(gitattrs["label:"+specs.AnnotationRevision], "-dirty"))
assert.Contains(t, gitattrs, "vcs:source")
assert.Equal(t, "git@github.com:docker/buildx.git", gitattrs["vcs:source"])
assert.Contains(t, gitattrs, "vcs:revision")
assert.True(t, strings.HasSuffix(gitattrs["vcs:revision"], "-dirty"))
}

138
build/invoke.go Normal file
View File

@@ -0,0 +1,138 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"io"
"sync"
"sync/atomic"
"syscall"
controllerapi "github.com/docker/buildx/controller/pb"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type Container struct {
cancelOnce sync.Once
containerCancel func()
isUnavailable atomic.Bool
initStarted atomic.Bool
container gateway.Container
releaseCh chan struct{}
resultCtx *ResultHandle
}
func NewContainer(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig) (*Container, error) {
mainCtx := ctx
ctrCh := make(chan *Container)
errCh := make(chan error)
go func() {
err := resultCtx.build(func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
ctx, cancel := context.WithCancel(ctx)
go func() {
<-mainCtx.Done()
cancel()
}()
containerCfg, err := resultCtx.getContainerConfig(ctx, c, cfg)
if err != nil {
return nil, err
}
containerCtx, containerCancel := context.WithCancel(ctx)
defer containerCancel()
bkContainer, err := c.NewContainer(containerCtx, containerCfg)
if err != nil {
return nil, err
}
releaseCh := make(chan struct{})
container := &Container{
containerCancel: containerCancel,
container: bkContainer,
releaseCh: releaseCh,
resultCtx: resultCtx,
}
doneCh := make(chan struct{})
defer close(doneCh)
resultCtx.registerCleanup(func() {
container.Cancel()
<-doneCh
})
ctrCh <- container
<-container.releaseCh
return nil, bkContainer.Release(ctx)
})
if err != nil {
errCh <- err
}
}()
select {
case ctr := <-ctrCh:
return ctr, nil
case err := <-errCh:
return nil, err
case <-mainCtx.Done():
return nil, mainCtx.Err()
}
}
func (c *Container) Cancel() {
c.markUnavailable()
c.cancelOnce.Do(func() {
if c.containerCancel != nil {
c.containerCancel()
}
close(c.releaseCh)
})
}
func (c *Container) IsUnavailable() bool {
return c.isUnavailable.Load()
}
func (c *Container) markUnavailable() {
c.isUnavailable.Store(true)
}
func (c *Container) Exec(ctx context.Context, cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if isInit := c.initStarted.CompareAndSwap(false, true); isInit {
defer func() {
// container can't be used after init exits
c.markUnavailable()
}()
}
err := exec(ctx, c.resultCtx, cfg, c.container, stdin, stdout, stderr)
if err != nil {
// Container becomes unavailable if one of the processes fails in it.
c.markUnavailable()
}
return err
}
func exec(ctx context.Context, resultCtx *ResultHandle, cfg *controllerapi.InvokeConfig, ctr gateway.Container, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
processCfg, err := resultCtx.getProcessConfig(cfg, stdin, stdout, stderr)
if err != nil {
return err
}
proc, err := ctr.Start(ctx, processCfg)
if err != nil {
return errors.Errorf("failed to start container: %v", err)
}
doneCh := make(chan struct{})
defer close(doneCh)
go func() {
select {
case <-ctx.Done():
if err := proc.Signal(ctx, syscall.SIGKILL); err != nil {
logrus.Warnf("failed to kill process: %v", err)
}
case <-doneCh:
}
}()
return proc.Wait()
}

View File

@@ -1,115 +0,0 @@
package build
import (
"encoding/csv"
"io"
"os"
"strings"
"github.com/containerd/console"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func ParseOutputs(inp []string) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
if len(inp) == 0 {
return nil, nil
}
for _, s := range inp {
csvReader := csv.NewReader(strings.NewReader(s))
fields, err := csvReader.Read()
if err != nil {
return nil, err
}
out := client.ExportEntry{
Attrs: map[string]string{},
}
if len(fields) == 1 && fields[0] == s && !strings.HasPrefix(s, "type=") {
if s != "-" {
outs = append(outs, client.ExportEntry{
Type: client.ExporterLocal,
OutputDir: s,
})
continue
}
out = client.ExportEntry{
Type: client.ExporterTar,
Attrs: map[string]string{
"dest": s,
},
}
}
if out.Type == "" {
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
if len(parts) != 2 {
return nil, errors.Errorf("invalid value %s", field)
}
key := strings.TrimSpace(strings.ToLower(parts[0]))
value := parts[1]
switch key {
case "type":
out.Type = value
default:
out.Attrs[key] = value
}
}
}
if out.Type == "" {
return nil, errors.Errorf("type is required for output")
}
// handle client side
switch out.Type {
case client.ExporterLocal:
dest, ok := out.Attrs["dest"]
if !ok {
return nil, errors.Errorf("dest is required for local output")
}
out.OutputDir = dest
delete(out.Attrs, "dest")
case client.ExporterOCI, client.ExporterDocker, client.ExporterTar:
dest, ok := out.Attrs["dest"]
if !ok {
if out.Type != client.ExporterDocker {
dest = "-"
}
}
if dest == "-" {
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, errors.Errorf("output file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
} else if dest != "" {
fi, err := os.Stat(dest)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination file: %s", dest)
}
if err == nil && fi.IsDir() {
return nil, errors.Errorf("destination file %s is a directory", dest)
}
f, err := os.Create(dest)
if err != nil {
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
}
delete(out.Attrs, "dest")
case "registry":
out.Type = client.ExporterImage
out.Attrs["push"] = "true"
}
outs = append(outs, out)
}
return outs, nil
}
func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) {
return func(map[string]string) (io.WriteCloser, error) {
return wc, nil
}
}

494
build/result.go Normal file
View File

@@ -0,0 +1,494 @@
package build
import (
"context"
_ "crypto/sha256" // ensure digests can be computed
"encoding/json"
"io"
"sync"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/exporter/containerimage/exptypes"
gateway "github.com/moby/buildkit/frontend/gateway/client"
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/solver/pb"
"github.com/moby/buildkit/solver/result"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
// NewResultHandle makes a call to client.Build, additionally returning a
// opaque ResultHandle alongside the standard response and error.
//
// This ResultHandle can be used to execute additional build steps in the same
// context as the build occurred, which can allow easy debugging of build
// failures and successes.
//
// If the returned ResultHandle is not nil, the caller must call Done() on it.
func NewResultHandle(ctx context.Context, cc *client.Client, opt client.SolveOpt, product string, buildFunc gateway.BuildFunc, ch chan *client.SolveStatus) (*ResultHandle, *client.SolveResponse, error) {
// Create a new context to wrap the original, and cancel it when the
// caller-provided context is cancelled.
//
// We derive the context from the background context so that we can forbid
// cancellation of the build request after <-done is closed (which we do
// before returning the ResultHandle).
baseCtx := ctx
ctx, cancel := context.WithCancelCause(context.Background())
done := make(chan struct{})
go func() {
select {
case <-baseCtx.Done():
cancel(baseCtx.Err())
case <-done:
// Once done is closed, we've recorded a ResultHandle, so we
// shouldn't allow cancelling the underlying build request anymore.
}
}()
// Create a new channel to forward status messages to the original.
//
// We do this so that we can discard status messages after the main portion
// of the build is complete. This is necessary for the solve error case,
// where the original gateway is kept open until the ResultHandle is
// closed - we don't want progress messages from operations in that
// ResultHandle to display after this function exits.
//
// Additionally, callers should wait for the progress channel to be closed.
// If we keep the session open and never close the progress channel, the
// caller will likely hang.
baseCh := ch
ch = make(chan *client.SolveStatus)
go func() {
for {
s, ok := <-ch
if !ok {
return
}
select {
case <-baseCh:
// base channel is closed, discard status messages
default:
baseCh <- s
}
}
}()
defer close(baseCh)
var resp *client.SolveResponse
var respErr error
var respHandle *ResultHandle
go func() {
defer cancel(context.Canceled) // ensure no dangling processes
var res *gateway.Result
var err error
resp, err = cc.Build(ctx, opt, product, func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
var err error
res, err = buildFunc(ctx, c)
if res != nil && err == nil {
// Force evaluation of the build result (otherwise, we likely
// won't get a solve error)
def, err2 := getDefinition(ctx, res)
if err2 != nil {
return nil, err2
}
res, err = evalDefinition(ctx, c, def)
}
if err != nil {
// Scenario 1: we failed to evaluate a node somewhere in the
// build graph.
//
// In this case, we construct a ResultHandle from this
// original Build session, and return it alongside the original
// build error. We then need to keep the gateway session open
// until the caller explicitly closes the ResultHandle.
var se *errdefs.SolveError
if errors.As(err, &se) {
respHandle = &ResultHandle{
done: make(chan struct{}),
solveErr: se,
gwClient: c,
gwCtx: ctx,
}
respErr = se
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
}
}
return res, err
}, ch)
if respHandle != nil {
return
}
if err != nil {
// Something unexpected failed during the build, we didn't succeed,
// but we also didn't make it far enough to create a ResultHandle.
respErr = err
close(done)
return
}
// Scenario 2: we successfully built the image with no errors.
//
// In this case, the original gateway session has now been closed
// since the Build has been completed. So, we need to create a new
// gateway session to populate the ResultHandle. To do this, we
// need to re-evaluate the target result, in this new session. This
// should be instantaneous since the result should be cached.
def, err := getDefinition(ctx, res)
if err != nil {
respErr = err
close(done)
return
}
// NOTE: ideally this second connection should be lazily opened
opt := opt
opt.Ref = ""
opt.Exports = nil
opt.CacheExports = nil
_, respErr = cc.Build(ctx, opt, "buildx", func(ctx context.Context, c gateway.Client) (*gateway.Result, error) {
res, err := evalDefinition(ctx, c, def)
if err != nil {
// This should probably not happen, since we've previously
// successfully evaluated the same result with no issues.
return nil, errors.Wrap(err, "inconsistent solve result")
}
respHandle = &ResultHandle{
done: make(chan struct{}),
res: res,
gwClient: c,
gwCtx: ctx,
}
close(done)
// Block until the caller closes the ResultHandle.
select {
case <-respHandle.done:
case <-ctx.Done():
}
return nil, ctx.Err()
}, nil)
if respHandle != nil {
return
}
close(done)
}()
// Block until the other thread signals that it's completed the build.
select {
case <-done:
case <-baseCtx.Done():
if respErr == nil {
respErr = baseCtx.Err()
}
}
return respHandle, resp, respErr
}
// getDefinition converts a gateway result into a collection of definitions for
// each ref in the result.
func getDefinition(ctx context.Context, res *gateway.Result) (*result.Result[*pb.Definition], error) {
return result.ConvertResult(res, func(ref gateway.Reference) (*pb.Definition, error) {
st, err := ref.ToState()
if err != nil {
return nil, err
}
def, err := st.Marshal(ctx)
if err != nil {
return nil, err
}
return def.ToPB(), nil
})
}
// evalDefinition performs the reverse of getDefinition, converting a
// collection of definitions into a gateway result.
func evalDefinition(ctx context.Context, c gateway.Client, defs *result.Result[*pb.Definition]) (*gateway.Result, error) {
// force evaluation of all targets in parallel
results := make(map[*pb.Definition]*gateway.Result)
resultsMu := sync.Mutex{}
eg, egCtx := errgroup.WithContext(ctx)
defs.EachRef(func(def *pb.Definition) error {
eg.Go(func() error {
res, err := c.Solve(egCtx, gateway.SolveRequest{
Evaluate: true,
Definition: def,
})
if err != nil {
return err
}
resultsMu.Lock()
results[def] = res
resultsMu.Unlock()
return nil
})
return nil
})
if err := eg.Wait(); err != nil {
return nil, err
}
res, _ := result.ConvertResult(defs, func(def *pb.Definition) (gateway.Reference, error) {
if res, ok := results[def]; ok {
return res.Ref, nil
}
return nil, nil
})
return res, nil
}
// ResultHandle is a build result with the client that built it.
type ResultHandle struct {
res *gateway.Result
solveErr *errdefs.SolveError
done chan struct{}
doneOnce sync.Once
gwClient gateway.Client
gwCtx context.Context
cleanups []func()
cleanupsMu sync.Mutex
}
func (r *ResultHandle) Done() {
r.doneOnce.Do(func() {
r.cleanupsMu.Lock()
cleanups := r.cleanups
r.cleanups = nil
r.cleanupsMu.Unlock()
for _, f := range cleanups {
f()
}
close(r.done)
<-r.gwCtx.Done()
})
}
func (r *ResultHandle) registerCleanup(f func()) {
r.cleanupsMu.Lock()
r.cleanups = append(r.cleanups, f)
r.cleanupsMu.Unlock()
}
func (r *ResultHandle) build(buildFunc gateway.BuildFunc) (err error) {
_, err = buildFunc(r.gwCtx, r.gwClient)
return err
}
func (r *ResultHandle) getContainerConfig(ctx context.Context, c gateway.Client, cfg *controllerapi.InvokeConfig) (containerCfg gateway.NewContainerRequest, _ error) {
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
ccfg, err := containerConfigFromResult(ctx, r.res, c, *cfg)
if err != nil {
return containerCfg, err
}
containerCfg = *ccfg
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
ccfg, err := containerConfigFromError(r.solveErr, *cfg)
if err != nil {
return containerCfg, errors.Wrapf(err, "no result nor error is available")
}
containerCfg = *ccfg
}
return containerCfg, nil
}
func (r *ResultHandle) getProcessConfig(cfg *controllerapi.InvokeConfig, stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) (_ gateway.StartRequest, err error) {
processCfg := newStartRequest(stdin, stdout, stderr)
if r.res != nil && r.solveErr == nil {
logrus.Debugf("creating container from successful build")
if err := populateProcessConfigFromResult(&processCfg, r.res, *cfg); err != nil {
return processCfg, err
}
} else {
logrus.Debugf("creating container from failed build %+v", cfg)
if err := populateProcessConfigFromError(&processCfg, r.solveErr, *cfg); err != nil {
return processCfg, err
}
}
return processCfg, nil
}
func containerConfigFromResult(ctx context.Context, res *gateway.Result, c gateway.Client, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
if cfg.Initial {
return nil, errors.Errorf("starting from the container from the initial state of the step is supported only on the failed steps")
}
ps, err := exptypes.ParsePlatforms(res.Metadata)
if err != nil {
return nil, err
}
ref, ok := res.FindRef(ps.Platforms[0].ID)
if !ok {
return nil, errors.Errorf("no reference found")
}
return &gateway.NewContainerRequest{
Mounts: []gateway.Mount{
{
Dest: "/",
MountType: pb.MountType_BIND,
Ref: ref,
},
},
}, nil
}
func populateProcessConfigFromResult(req *gateway.StartRequest, res *gateway.Result, cfg controllerapi.InvokeConfig) error {
imgData := res.Metadata[exptypes.ExporterImageConfigKey]
var img *specs.Image
if len(imgData) > 0 {
img = &specs.Image{}
if err := json.Unmarshal(imgData, img); err != nil {
return err
}
}
user := ""
if !cfg.NoUser {
user = cfg.User
} else if img != nil {
user = img.Config.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else if img != nil {
cwd = img.Config.WorkingDir
}
env := []string{}
if img != nil {
env = append(env, img.Config.Env...)
}
env = append(env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
} else if img != nil {
args = append(args, img.Config.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
} else if img != nil {
args = append(args, img.Config.Cmd...)
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func containerConfigFromError(solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) (*gateway.NewContainerRequest, error) {
exec, err := execOpFromError(solveErr)
if err != nil {
return nil, err
}
var mounts []gateway.Mount
for i, mnt := range exec.Mounts {
rid := solveErr.Solve.MountIDs[i]
if cfg.Initial {
rid = solveErr.Solve.InputIDs[i]
}
mounts = append(mounts, gateway.Mount{
Selector: mnt.Selector,
Dest: mnt.Dest,
ResultID: rid,
Readonly: mnt.Readonly,
MountType: mnt.MountType,
CacheOpt: mnt.CacheOpt,
SecretOpt: mnt.SecretOpt,
SSHOpt: mnt.SSHOpt,
})
}
return &gateway.NewContainerRequest{
Mounts: mounts,
NetMode: exec.Network,
}, nil
}
func populateProcessConfigFromError(req *gateway.StartRequest, solveErr *errdefs.SolveError, cfg controllerapi.InvokeConfig) error {
exec, err := execOpFromError(solveErr)
if err != nil {
return err
}
meta := exec.Meta
user := ""
if !cfg.NoUser {
user = cfg.User
} else {
user = meta.User
}
cwd := ""
if !cfg.NoCwd {
cwd = cfg.Cwd
} else {
cwd = meta.Cwd
}
env := append(meta.Env, cfg.Env...)
args := []string{}
if cfg.Entrypoint != nil {
args = append(args, cfg.Entrypoint...)
}
if cfg.Cmd != nil {
args = append(args, cfg.Cmd...)
}
if len(args) == 0 {
args = meta.Args
}
req.Args = args
req.Env = env
req.User = user
req.Cwd = cwd
req.Tty = cfg.Tty
return nil
}
func execOpFromError(solveErr *errdefs.SolveError) (*pb.ExecOp, error) {
if solveErr == nil {
return nil, errors.Errorf("no error is available")
}
switch op := solveErr.Solve.Op.GetOp().(type) {
case *pb.Op_Exec:
return op.Exec, nil
default:
return nil, errors.Errorf("invoke: unsupported error type")
}
// TODO: support other ops
}
func newStartRequest(stdin io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) gateway.StartRequest {
return gateway.StartRequest{
Stdin: stdin,
Stdout: stdout,
Stderr: stderr,
}
}

View File

@@ -1,60 +0,0 @@
package build
import (
"encoding/csv"
"strings"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/secrets/secretsprovider"
"github.com/pkg/errors"
)
func ParseSecretSpecs(sl []string) (session.Attachable, error) {
fs := make([]secretsprovider.Source, 0, len(sl))
for _, v := range sl {
s, err := parseSecret(v)
if err != nil {
return nil, err
}
fs = append(fs, *s)
}
store, err := secretsprovider.NewStore(fs)
if err != nil {
return nil, err
}
return secretsprovider.NewSecretProvider(store), nil
}
func parseSecret(value string) (*secretsprovider.Source, error) {
csvReader := csv.NewReader(strings.NewReader(value))
fields, err := csvReader.Read()
if err != nil {
return nil, errors.Wrap(err, "failed to parse csv secret")
}
fs := secretsprovider.Source{}
for _, field := range fields {
parts := strings.SplitN(field, "=", 2)
key := strings.ToLower(parts[0])
if len(parts) != 2 {
return nil, errors.Errorf("invalid field '%s' must be a key=value pair", field)
}
value := parts[1]
switch key {
case "type":
if value != "file" {
return nil, errors.Errorf("unsupported secret type %q", value)
}
case "id":
fs.ID = value
case "source", "src":
fs.FilePath = value
default:
return nil, errors.Errorf("unexpected key '%s' in '%s'", key, field)
}
}
return &fs, nil
}

View File

@@ -1,31 +0,0 @@
package build
import (
"strings"
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider"
)
func ParseSSHSpecs(sl []string) (session.Attachable, error) {
configs := make([]sshprovider.AgentConfig, 0, len(sl))
for _, v := range sl {
c, err := parseSSH(v)
if err != nil {
return nil, err
}
configs = append(configs, *c)
}
return sshprovider.NewSSHAgentProvider(configs)
}
func parseSSH(value string) (*sshprovider.AgentConfig, error) {
parts := strings.SplitN(value, "=", 2)
cfg := sshprovider.AgentConfig{
ID: parts[0],
}
if len(parts) > 1 {
cfg.Paths = strings.Split(parts[1], ",")
}
return &cfg, nil
}

View File

@@ -2,7 +2,7 @@ package build
import (
"context"
"io/ioutil"
"os"
"path/filepath"
"github.com/docker/buildx/driver"
@@ -14,7 +14,7 @@ import (
)
func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url string, pw progress.Writer) (string, error) {
c, err := driver.Boot(ctx, d, pw)
c, err := driver.Boot(ctx, ctx, d, pw)
if err != nil {
return "", err
}
@@ -53,11 +53,11 @@ func createTempDockerfileFromURL(ctx context.Context, d driver.Driver, url strin
if err != nil {
return nil, err
}
dir, err := ioutil.TempDir("", "buildx")
dir, err := os.MkdirTemp("", "buildx")
if err != nil {
return nil, err
}
if err := ioutil.WriteFile(filepath.Join(dir, "Dockerfile"), dt, 0600); err != nil {
if err := os.WriteFile(filepath.Join(dir, "Dockerfile"), dt, 0600); err != nil {
return nil, err
}
out = dir

View File

@@ -7,11 +7,30 @@ import (
"os"
"strings"
"github.com/docker/cli/opts"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
"github.com/pkg/errors"
)
const (
// archiveHeaderSize is the number of bytes in an archive header
const archiveHeaderSize = 512
archiveHeaderSize = 512
// mobyHostGatewayName defines a special string which users can append to
// --add-host to add an extra entry in /etc/hosts that maps
// host.docker.internal to the host IP
mobyHostGatewayName = "host-gateway"
)
func IsRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}
func isLocalDir(c string) bool {
st, err := os.Stat(c)
@@ -38,18 +57,35 @@ func isArchive(header []byte) bool {
}
// toBuildkitExtraHosts converts hosts from docker key:value format to buildkit's csv format
func toBuildkitExtraHosts(inp []string) (string, error) {
func toBuildkitExtraHosts(inp []string, mobyDriver bool) (string, error) {
if len(inp) == 0 {
return "", nil
}
hosts := make([]string, 0, len(inp))
for _, h := range inp {
parts := strings.Split(h, ":")
if len(parts) != 2 || parts[0] == "" || net.ParseIP(parts[1]) == nil {
host, ip, ok := strings.Cut(h, ":")
if !ok || host == "" || ip == "" {
return "", errors.Errorf("invalid host %s", h)
}
hosts = append(hosts, parts[0]+"="+parts[1])
// Skip IP address validation for "host-gateway" string with moby driver
if !mobyDriver || ip != mobyHostGatewayName {
if net.ParseIP(ip) == nil {
return "", errors.Errorf("invalid host %s", h)
}
}
hosts = append(hosts, host+"="+ip)
}
return strings.Join(hosts, ","), nil
}
// toBuildkitUlimits converts ulimits from docker type=soft:hard format to buildkit's csv format
func toBuildkitUlimits(inp *opts.UlimitOpt) (string, error) {
if inp == nil || len(inp.GetList()) == 0 {
return "", nil
}
ulimits := make([]string, 0, len(inp.GetList()))
for _, ulimit := range inp.GetList() {
ulimits = append(ulimits, ulimit.String())
}
return strings.Join(ulimits, ","), nil
}

292
builder/builder.go Normal file
View File

@@ -0,0 +1,292 @@
package builder
import (
"context"
"os"
"sort"
"sync"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
// Builder represents an active builder object
type Builder struct {
*store.NodeGroup
driverFactory driverFactory
nodes []Node
opts builderOpts
err error
}
type builderOpts struct {
dockerCli command.Cli
name string
txn *store.Txn
contextPathHash string
validate bool
}
// Option provides a variadic option for configuring the builder.
type Option func(b *Builder)
// WithName sets builder name.
func WithName(name string) Option {
return func(b *Builder) {
b.opts.name = name
}
}
// WithStore sets a store instance used at init.
func WithStore(txn *store.Txn) Option {
return func(b *Builder) {
b.opts.txn = txn
}
}
// WithContextPathHash is used for determining pods in k8s driver instance.
func WithContextPathHash(contextPathHash string) Option {
return func(b *Builder) {
b.opts.contextPathHash = contextPathHash
}
}
// WithSkippedValidation skips builder context validation.
func WithSkippedValidation() Option {
return func(b *Builder) {
b.opts.validate = false
}
}
// New initializes a new builder client
func New(dockerCli command.Cli, opts ...Option) (_ *Builder, err error) {
b := &Builder{
opts: builderOpts{
dockerCli: dockerCli,
validate: true,
},
}
for _, opt := range opts {
opt(b)
}
if b.opts.txn == nil {
// if store instance is nil we create a short-lived one using the
// default store and ensure we release it on completion
var release func()
b.opts.txn, release, err = storeutil.GetStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
}
if b.opts.name != "" {
if b.NodeGroup, err = storeutil.GetNodeGroup(b.opts.txn, dockerCli, b.opts.name); err != nil {
return nil, err
}
} else {
if b.NodeGroup, err = storeutil.GetCurrentInstance(b.opts.txn, dockerCli); err != nil {
return nil, err
}
}
if b.opts.validate {
if err = b.Validate(); err != nil {
return nil, err
}
}
return b, nil
}
// Validate validates builder context
func (b *Builder) Validate() error {
if b.NodeGroup != nil && b.NodeGroup.DockerContext {
list, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return err
}
currentContext := b.opts.dockerCli.CurrentContext()
for _, l := range list {
if l.Name == b.Name && l.Name != currentContext {
return errors.Errorf("use `docker --context=%s buildx` to switch to context %q", l.Name, l.Name)
}
}
}
return nil
}
// ContextName returns builder context name if available.
func (b *Builder) ContextName() string {
ctxbuilders, err := b.opts.dockerCli.ContextStore().List()
if err != nil {
return ""
}
for _, cb := range ctxbuilders {
if b.NodeGroup.Driver == "docker" && len(b.NodeGroup.Nodes) == 1 && b.NodeGroup.Nodes[0].Endpoint == cb.Name {
return cb.Name
}
}
return ""
}
// ImageOpt returns registry auth configuration
func (b *Builder) ImageOpt() (imagetools.Opt, error) {
return storeutil.GetImageConfig(b.opts.dockerCli, b.NodeGroup)
}
// Boot bootstrap a builder
func (b *Builder) Boot(ctx context.Context) (bool, error) {
toBoot := make([]int, 0, len(b.nodes))
for idx, d := range b.nodes {
if d.Err != nil || d.Driver == nil || d.DriverInfo == nil {
continue
}
if d.DriverInfo.Status != driver.Running {
toBoot = append(toBoot, idx)
}
}
if len(toBoot) == 0 {
return false, nil
}
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, os.Stderr, progress.PrinterModeAuto)
if err != nil {
return false, err
}
baseCtx := ctx
eg, _ := errgroup.WithContext(ctx)
for _, idx := range toBoot {
func(idx int) {
eg.Go(func() error {
pw := progress.WithPrefix(printer, b.NodeGroup.Nodes[idx].Name, len(toBoot) > 1)
_, err := driver.Boot(ctx, baseCtx, b.nodes[idx].Driver, pw)
if err != nil {
b.nodes[idx].Err = err
}
return nil
})
}(idx)
}
err = eg.Wait()
err1 := printer.Wait()
if err == nil {
err = err1
}
return true, err
}
// Inactive checks if all nodes are inactive for this builder.
func (b *Builder) Inactive() bool {
for _, d := range b.nodes {
if d.DriverInfo != nil && d.DriverInfo.Status == driver.Running {
return false
}
}
return true
}
// Err returns error if any.
func (b *Builder) Err() error {
return b.err
}
type driverFactory struct {
driver.Factory
once sync.Once
}
// Factory returns the driver factory.
func (b *Builder) Factory(ctx context.Context) (_ driver.Factory, err error) {
b.driverFactory.once.Do(func() {
if b.Driver != "" {
b.driverFactory.Factory, err = driver.GetFactory(b.Driver, true)
if err != nil {
return
}
} else {
// empty driver means nodegroup was implicitly created as a default
// driver for a docker context and allows falling back to a
// docker-container driver for older daemon that doesn't support
// buildkit (< 18.06).
ep := b.NodeGroup.Nodes[0].Endpoint
var dockerapi *dockerutil.ClientAPI
dockerapi, err = dockerutil.NewClientAPI(b.opts.dockerCli, b.NodeGroup.Nodes[0].Endpoint)
if err != nil {
return
}
// check if endpoint is healthy is needed to determine the driver type.
// if this fails then can't continue with driver selection.
if _, err = dockerapi.Ping(ctx); err != nil {
return
}
b.driverFactory.Factory, err = driver.GetDefaultFactory(ctx, ep, dockerapi, false)
if err != nil {
return
}
b.Driver = b.driverFactory.Factory.Name()
}
})
return b.driverFactory.Factory, err
}
// GetBuilders returns all builders
func GetBuilders(dockerCli command.Cli, txn *store.Txn) ([]*Builder, error) {
storeng, err := txn.List()
if err != nil {
return nil, err
}
builders := make([]*Builder, len(storeng))
seen := make(map[string]struct{})
for i, ng := range storeng {
b, err := New(dockerCli,
WithName(ng.Name),
WithStore(txn),
WithSkippedValidation(),
)
if err != nil {
return nil, err
}
builders[i] = b
seen[b.NodeGroup.Name] = struct{}{}
}
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
sort.Slice(contexts, func(i, j int) bool {
return contexts[i].Name < contexts[j].Name
})
for _, c := range contexts {
// if a context has the same name as an instance from the store, do not
// add it to the builders list. An instance from the store takes
// precedence over context builders.
if _, ok := seen[c.Name]; ok {
continue
}
b, err := New(dockerCli,
WithName(c.Name),
WithStore(txn),
WithSkippedValidation(),
)
if err != nil {
return nil, err
}
builders = append(builders, b)
}
return builders, nil
}

211
builder/node.go Normal file
View File

@@ -0,0 +1,211 @@
package builder
import (
"context"
"github.com/docker/buildx/driver"
ctxkube "github.com/docker/buildx/driver/kubernetes/context"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/platformutil"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
ocispecs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc/codes"
)
type Node struct {
store.Node
Builder string
Driver driver.Driver
DriverInfo *driver.Info
Platforms []ocispecs.Platform
GCPolicy []client.PruneInfo
Labels map[string]string
ImageOpt imagetools.Opt
ProxyConfig map[string]string
Version string
Err error
}
// Nodes returns nodes for this builder.
func (b *Builder) Nodes() []Node {
return b.nodes
}
// LoadNodes loads and returns nodes for this builder.
// TODO: this should be a method on a Node object and lazy load data for each driver.
func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err error) {
eg, _ := errgroup.WithContext(ctx)
b.nodes = make([]Node, len(b.NodeGroup.Nodes))
defer func() {
if b.err == nil && err != nil {
b.err = err
}
}()
factory, err := b.Factory(ctx)
if err != nil {
return nil, err
}
imageopt, err := b.ImageOpt()
if err != nil {
return nil, err
}
for i, n := range b.NodeGroup.Nodes {
func(i int, n store.Node) {
eg.Go(func() error {
node := Node{
Node: n,
ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli),
Platforms: n.Platforms,
Builder: b.Name,
}
defer func() {
b.nodes[i] = node
}()
dockerapi, err := dockerutil.NewClientAPI(b.opts.dockerCli, n.Endpoint)
if err != nil {
node.Err = err
return nil
}
contextStore := b.opts.dockerCli.ContextStore()
var kcc driver.KubeClientConfig
kcc, err = ctxkube.ConfigFromContext(n.Endpoint, contextStore)
if err != nil {
// err is returned if n.Endpoint is non-context name like "unix:///var/run/docker.sock".
// try again with name="default".
// FIXME(@AkihiroSuda): n should retain real context name.
kcc, err = ctxkube.ConfigFromContext("default", contextStore)
if err != nil {
logrus.Error(err)
}
}
tryToUseKubeConfigInCluster := false
if kcc == nil {
tryToUseKubeConfigInCluster = true
} else {
if _, err := kcc.ClientConfig(); err != nil {
tryToUseKubeConfigInCluster = true
}
}
if tryToUseKubeConfigInCluster {
kccInCluster := driver.KubeClientConfigInCluster{}
if _, err := kccInCluster.ClientConfig(); err == nil {
logrus.Debug("using kube config in cluster")
kcc = kccInCluster
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, factory, n.Endpoint, dockerapi, imageopt.Auth, kcc, n.Flags, n.Files, n.DriverOpts, n.Platforms, b.opts.contextPathHash)
if err != nil {
node.Err = err
return nil
}
node.Driver = d
node.ImageOpt = imageopt
if withData {
if err := node.loadData(ctx); err != nil {
node.Err = err
}
}
return nil
})
}(i, n)
}
if err := eg.Wait(); err != nil {
return nil, err
}
// TODO: This should be done in the routine loading driver data
if withData {
kubernetesDriverCount := 0
for _, d := range b.nodes {
if d.DriverInfo != nil && len(d.DriverInfo.DynamicNodes) > 0 {
kubernetesDriverCount++
}
}
isAllKubernetesDrivers := len(b.nodes) == kubernetesDriverCount
if isAllKubernetesDrivers {
var nodes []Node
var dynamicNodes []store.Node
for _, di := range b.nodes {
// dynamic nodes are used in Kubernetes driver.
// Kubernetes' pods are dynamically mapped to BuildKit Nodes.
if di.DriverInfo != nil && len(di.DriverInfo.DynamicNodes) > 0 {
for i := 0; i < len(di.DriverInfo.DynamicNodes); i++ {
diClone := di
if pl := di.DriverInfo.DynamicNodes[i].Platforms; len(pl) > 0 {
diClone.Platforms = pl
}
nodes = append(nodes, di)
}
dynamicNodes = append(dynamicNodes, di.DriverInfo.DynamicNodes...)
}
}
// not append (remove the static nodes in the store)
b.NodeGroup.Nodes = dynamicNodes
b.nodes = nodes
b.NodeGroup.Dynamic = true
}
}
return b.nodes, nil
}
func (n *Node) loadData(ctx context.Context) error {
if n.Driver == nil {
return nil
}
info, err := n.Driver.Info(ctx)
if err != nil {
return err
}
n.DriverInfo = info
if n.DriverInfo.Status == driver.Running {
driverClient, err := n.Driver.Client(ctx)
if err != nil {
return err
}
workers, err := driverClient.ListWorkers(ctx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
for idx, w := range workers {
n.Platforms = append(n.Platforms, w.Platforms...)
if idx == 0 {
n.GCPolicy = w.GCPolicy
n.Labels = w.Labels
}
}
n.Platforms = platformutil.Dedupe(n.Platforms)
inf, err := driverClient.Info(ctx)
if err != nil {
if st, ok := grpcerrors.AsGRPCStatus(err); ok && st.Code() == codes.Unimplemented {
n.Version, err = n.Driver.Version(ctx)
if err != nil {
return errors.Wrap(err, "getting version")
}
}
} else {
n.Version = inf.BuildkitVersion.Version
}
}
return nil
}

View File

@@ -4,8 +4,8 @@ import (
"fmt"
"os"
"github.com/containerd/containerd/pkg/seed"
"github.com/docker/buildx/commands"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/manager"
@@ -16,59 +16,60 @@ import (
"github.com/moby/buildkit/solver/errdefs"
"github.com/moby/buildkit/util/stack"
// FIXME: "k8s.io/client-go/plugin/pkg/client/auth/azure" is excluded because of compilation error
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
//nolint:staticcheck // vendored dependencies may still use this
"github.com/containerd/containerd/pkg/seed"
_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
_ "k8s.io/client-go/plugin/pkg/client/auth/openstack"
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
_ "github.com/docker/buildx/driver/kubernetes"
_ "github.com/docker/buildx/driver/remote"
)
var experimental string
func init() {
//nolint:staticcheck
seed.WithTimeAndRand()
stack.SetVersionInfo(version.Version, version.Revision)
}
func main() {
if os.Getenv("DOCKER_CLI_PLUGIN_ORIGINAL_CLI_COMMAND") == "" {
if len(os.Args) < 2 || os.Args[1] != manager.MetadataSubcommandName {
dockerCli, err := command.NewDockerCli()
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
opts := cliflags.NewClientOptions()
dockerCli.Initialize(opts)
rootCmd := commands.NewRootCmd(os.Args[0], false, dockerCli)
if err := rootCmd.Execute(); err != nil {
os.Exit(1)
}
os.Exit(0)
func runStandalone(cmd *command.DockerCli) error {
if err := cmd.Initialize(cliflags.NewClientOptions()); err != nil {
return err
}
rootCmd := commands.NewRootCmd(os.Args[0], false, cmd)
return rootCmd.Execute()
}
dockerCli, err := command.NewDockerCli()
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
p := commands.NewRootCmd("buildx", true, dockerCli)
meta := manager.Metadata{
func runPlugin(cmd *command.DockerCli) error {
rootCmd := commands.NewRootCmd("buildx", true, cmd)
return plugin.RunPlugin(cmd, rootCmd, manager.Metadata{
SchemaVersion: "0.1.0",
Vendor: "Docker Inc.",
Version: version.Version,
Experimental: experimental != "",
})
}
func main() {
cmd, err := command.NewDockerCli()
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if plugin.RunningStandalone() {
err = runStandalone(cmd)
} else {
err = runPlugin(cmd)
}
if err == nil {
return
}
if err := plugin.RunPlugin(dockerCli, p, meta); err != nil {
if sterr, ok := err.(cli.StatusError); ok {
if sterr.Status != "" {
fmt.Fprintln(dockerCli.Err(), sterr.Status)
fmt.Fprintln(cmd.Err(), sterr.Status)
}
// StatusError should only be used for errors, and all errors should
// have a non-zero exit status, so never exit with 0
@@ -77,16 +78,18 @@ func main() {
}
os.Exit(sterr.StatusCode)
}
for _, s := range errdefs.Sources(err) {
s.Print(dockerCli.Err())
}
for _, s := range errdefs.Sources(err) {
s.Print(cmd.Err())
}
if debug.IsEnabled() {
fmt.Fprintf(dockerCli.Err(), "error: %+v", stack.Formatter(err))
fmt.Fprintf(cmd.Err(), "ERROR: %+v", stack.Formatter(err))
} else {
fmt.Fprintf(dockerCli.Err(), "error: %v\n", err)
fmt.Fprintf(cmd.Err(), "ERROR: %v\n", err)
}
if ebr, ok := err.(*desktop.ErrorWithBuildRef); ok {
ebr.Print(cmd.Err())
}
os.Exit(1)
}
}

19
cmd/buildx/tracing.go Normal file
View File

@@ -0,0 +1,19 @@
package main
import (
"github.com/moby/buildkit/util/tracing/detect"
"go.opentelemetry.io/otel"
_ "github.com/moby/buildkit/util/tracing/detect/delegated"
_ "github.com/moby/buildkit/util/tracing/env"
)
func init() {
detect.ServiceName = "buildx"
// do not log tracing errors to stdio
otel.SetErrorHandler(skipErrors{})
}
type skipErrors struct{}
func (skipErrors) Handle(err error) {}

View File

@@ -6,9 +6,18 @@ import (
"fmt"
"os"
"github.com/containerd/console"
"github.com/containerd/containerd/platforms"
"github.com/docker/buildx/bake"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/desktop"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/util/tracing"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
@@ -17,20 +26,41 @@ import (
type bakeOptions struct {
files []string
printOnly bool
overrides []string
commonOptions
printOnly bool
sbom string
provenance string
builder string
metadataFile string
exportPush bool
exportLoad bool
}
func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error) {
func runBake(dockerCli command.Cli, targets []string, in bakeOptions, cFlags commonFlags) (err error) {
ctx := appcontext.Context()
ctx, end, err := tracing.TraceCurrentCommand(ctx, "bake")
if err != nil {
return err
}
defer func() {
end(err)
}()
var url string
cmdContext := "cwd://"
if len(targets) > 0 {
if bake.IsRemoteURL(targets[0]) {
if build.IsRemoteURL(targets[0]) {
url = targets[0]
targets = targets[1:]
if len(targets) > 0 {
if build.IsRemoteURL(targets[0]) {
cmdContext = targets[0]
targets = targets[1:]
}
}
}
}
@@ -43,21 +73,63 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
if in.exportLoad {
return errors.Errorf("push and load may not be set together at the moment")
}
overrides = append(overrides, "*.output=type=registry")
overrides = append(overrides, "*.push=true")
} else if in.exportLoad {
overrides = append(overrides, "*.output=type=docker")
}
if in.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *in.noCache))
if cFlags.noCache != nil {
overrides = append(overrides, fmt.Sprintf("*.no-cache=%t", *cFlags.noCache))
}
if in.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *in.pull))
if cFlags.pull != nil {
overrides = append(overrides, fmt.Sprintf("*.pull=%t", *cFlags.pull))
}
if in.sbom != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("sbom", in.sbom)))
}
if in.provenance != "" {
overrides = append(overrides, fmt.Sprintf("*.attest=%s", buildflags.CanonicalizeAttest("provenance", in.provenance)))
}
contextPathHash, _ := os.Getwd()
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer := progress.NewPrinter(ctx2, os.Stderr, in.progress)
var nodes []builder.Node
var files []bake.File
var inp *bake.Input
var progressConsoleDesc, progressTextDesc string
// instance only needed for reading remote bake files or building
if url != "" || !in.printOnly {
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err = b.LoadNodes(ctx, false)
if err != nil {
return err
}
progressConsoleDesc = fmt.Sprintf("%s:%s", b.Driver, b.Name)
progressTextDesc = fmt.Sprintf("building with %q instance using %s driver", b.Name, b.Driver)
}
var term bool
if _, err := console.ConsoleFromFile(os.Stderr); err == nil {
term = true
}
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, cFlags.progress,
progress.WithDesc(progressTextDesc, progressConsoleDesc),
)
if err != nil {
return err
}
defer func() {
if printer != nil {
@@ -65,18 +137,14 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
if err == nil {
err = err1
}
if err == nil && cFlags.progress != progress.PrinterModeQuiet {
desktop.PrintBuildDetails(os.Stderr, printer.BuildRefs(), term)
}
}
}()
dis, err := getInstanceOrDefault(ctx, dockerCli, in.builder, contextPathHash)
if err != nil {
return err
}
var files []bake.File
var inp *bake.Input
if url != "" {
files, inp, err = bake.ReadRemoteFiles(ctx, dis, url, in.files, printer)
files, inp, err = bake.ReadRemoteFiles(ctx, nodes, url, in.files, printer)
} else {
files, err = bake.ReadLocalFiles(in.files)
}
@@ -84,13 +152,43 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return err
}
m, err := bake.ReadTargets(ctx, files, targets, overrides)
tgts, grps, err := bake.ReadTargets(ctx, files, targets, overrides, map[string]string{
// don't forget to update documentation if you add a new
// built-in variable: docs/bake-reference.md#built-in-variables
"BAKE_CMD_CONTEXT": cmdContext,
"BAKE_LOCAL_PLATFORM": platforms.DefaultString(),
})
if err != nil {
return err
}
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
// TODO: extract env var parsing to a method easily usable by library consumers
for _, t := range tgts {
if _, ok := t.Args["SOURCE_DATE_EPOCH"]; ok {
continue
}
if t.Args == nil {
t.Args = map[string]*string{}
}
t.Args["SOURCE_DATE_EPOCH"] = &v
}
}
// this function can update target context string from the input so call before printOnly check
bo, err := bake.TargetsToBuildOpt(tgts, inp)
if err != nil {
return err
}
if in.printOnly {
dt, err := json.MarshalIndent(map[string]map[string]*bake.Target{"target": m}, "", " ")
dt, err := json.MarshalIndent(struct {
Group map[string]*bake.Group `json:"group,omitempty"`
Target map[string]*bake.Target `json:"target"`
}{
grps,
tgts,
}, "", " ")
if err != nil {
return err
}
@@ -103,17 +201,27 @@ func runBake(dockerCli command.Cli, targets []string, in bakeOptions) (err error
return nil
}
bo, err := bake.TargetsToBuildOpt(m, inp)
resp, err := build.Build(ctx, nodes, bo, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), printer)
if err != nil {
return err
return wrapBuildError(err, true)
}
if len(in.metadataFile) > 0 {
dt := make(map[string]interface{})
for t, r := range resp {
dt[t] = decodeExporterResponse(r.ExporterResponse)
}
if err := writeMetadataFile(in.metadataFile, dt); err != nil {
return err
}
}
_, err = build.Build(ctx, dis, bo, dockerAPI(dockerCli), dockerCli.ConfigFile(), printer)
return err
}
func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
var options bakeOptions
var cFlags commonFlags
cmd := &cobra.Command{
Use: "bake [OPTIONS] [TARGET...]",
@@ -122,25 +230,30 @@ func bakeCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
// reset to nil to avoid override is unset
if !cmd.Flags().Lookup("no-cache").Changed {
options.noCache = nil
cFlags.noCache = nil
}
if !cmd.Flags().Lookup("pull").Changed {
options.pull = nil
cFlags.pull = nil
}
options.commonOptions.builder = rootOpts.builder
return runBake(dockerCli, args, options)
options.builder = rootOpts.builder
options.metadataFile = cFlags.metadataFile
// Other common flags (noCache, pull and progress) are processed in runBake function.
return runBake(dockerCli, args, options, cFlags)
},
ValidArgsFunction: completion.BakeTargets(options.files),
}
flags := cmd.Flags()
flags.StringArrayVarP(&options.files, "file", "f", []string{}, "Build definition file")
flags.BoolVar(&options.exportLoad, "load", false, `Shorthand for "--set=*.output=type=docker"`)
flags.BoolVar(&options.printOnly, "print", false, "Print the options without building")
flags.StringArrayVar(&options.overrides, "set", nil, "Override target value (eg: targetpattern.key=value)")
flags.BoolVar(&options.exportPush, "push", false, "Shorthand for --set=*.output=type=registry")
flags.BoolVar(&options.exportLoad, "load", false, "Shorthand for --set=*.output=type=docker")
flags.BoolVar(&options.exportPush, "push", false, `Shorthand for "--set=*.output=type=registry"`)
flags.StringVar(&options.sbom, "sbom", "", `Shorthand for "--set=*.attest=type=sbom"`)
flags.StringVar(&options.provenance, "provenance", "", `Shorthand for "--set=*.attest=type=provenance"`)
flags.StringArrayVar(&options.overrides, "set", nil, `Override target value (e.g., "targetpattern.key=value")`)
commonBuildFlags(&options.commonOptions, flags)
commonBuildFlags(&cFlags, flags)
return cmd
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,15 +1,29 @@
package commands
import (
"bytes"
"context"
"encoding/csv"
"fmt"
"net/url"
"os"
"strings"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
k8sutil "github.com/docker/buildx/driver/kubernetes/util"
remoteutil "github.com/docker/buildx/driver/remote/util"
"github.com/docker/buildx/localstate"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
dopts "github.com/docker/cli/opts"
"github.com/google/shlex"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
@@ -28,6 +42,7 @@ type createOptions struct {
flags string
configFile string
driverOpts []string
bootstrap bool
// upgrade bool // perform upgrade of the driver
}
@@ -53,23 +68,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
driverName := in.driver
if driverName == "" {
f, err := driver.GetDefaultFactory(ctx, dockerCli.Client(), true)
if err != nil {
return err
}
if f == nil {
return errors.Errorf("no valid drivers found")
}
driverName = f.Name()
}
if driver.GetFactory(driverName, true) == nil {
return errors.Errorf("failed to find driver %q", in.driver)
}
txn, release, err := getStore(dockerCli)
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
@@ -83,6 +82,19 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
if !in.actionLeave && !in.actionAppend {
contexts, err := dockerCli.ContextStore().List()
if err != nil {
return err
}
for _, c := range contexts {
if c.Name == name {
logrus.Warnf("instance name %q already exists as context builder", name)
break
}
}
}
ng, err := txn.NodeGroupByName(name)
if err != nil {
if os.IsNotExist(errors.Cause(err)) {
@@ -90,29 +102,62 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
logrus.Warnf("failed to find %q for append, creating a new instance instead", in.name)
}
if in.actionLeave {
return errors.Errorf("failed to find instance %q for leave", name)
return errors.Errorf("failed to find instance %q for leave", in.name)
}
} else {
return err
}
}
buildkitHost := os.Getenv("BUILDKIT_HOST")
driverName := in.driver
if driverName == "" {
if ng != nil {
driverName = ng.Driver
} else if len(args) == 0 && buildkitHost != "" {
driverName = "remote"
} else {
var arg string
if len(args) > 0 {
arg = args[0]
}
f, err := driver.GetDefaultFactory(ctx, arg, dockerCli.Client(), true)
if err != nil {
return err
}
if f == nil {
return errors.Errorf("no valid drivers found")
}
driverName = f.Name()
}
}
if ng != nil {
if in.nodeName == "" && !in.actionAppend {
return errors.Errorf("existing instance for %s but no append mode, specify --node to make changes for existing instances", name)
return errors.Errorf("existing instance for %q but no append mode, specify --node to make changes for existing instances", name)
}
if driverName != ng.Driver {
return errors.Errorf("existing instance for %q but has mismatched driver %q", name, ng.Driver)
}
}
if _, err := driver.GetFactory(driverName, true); err != nil {
return err
}
ngOriginal := ng
if ngOriginal != nil {
ngOriginal = ngOriginal.Copy()
}
if ng == nil {
ng = &store.NodeGroup{
Name: name,
Driver: driverName,
}
}
if ng.Driver == "" || in.driver != "" {
ng.Driver = driverName
}
var flags []string
if in.flags != "" {
flags, err = shlex.Split(in.flags)
@@ -122,37 +167,88 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
var ep string
var setEp bool
if in.actionLeave {
if err := ng.Leave(in.nodeName); err != nil {
return err
}
ls, err := localstate.New(confutil.ConfigDir(dockerCli))
if err != nil {
return err
}
if err := ls.RemoveBuilderNode(ng.Name, in.nodeName); err != nil {
return err
}
} else {
switch {
case driverName == "kubernetes":
if len(args) > 0 {
logrus.Warnf("kubernetes driver does not support endpoint args %q", args[0])
}
// generate node name if not provided to avoid duplicated endpoint
// error: https://github.com/docker/setup-buildx-action/issues/215
nodeName := in.nodeName
if nodeName == "" {
nodeName, err = k8sutil.GenerateNodeName(name, txn)
if err != nil {
return err
}
}
// naming endpoint to make --append works
ep = (&url.URL{
Scheme: driverName,
Path: "/" + name,
RawQuery: (&url.Values{
"deployment": {nodeName},
"kubeconfig": {os.Getenv("KUBECONFIG")},
}).Encode(),
}).String()
setEp = false
case driverName == "remote":
if len(args) > 0 {
ep = args[0]
} else if buildkitHost != "" {
ep = buildkitHost
} else {
return errors.Errorf("no remote endpoint provided")
}
ep, err = validateBuildkitEndpoint(ep)
if err != nil {
return err
}
setEp = true
case len(args) > 0:
ep, err = validateEndpoint(dockerCli, args[0])
if err != nil {
return err
}
} else {
setEp = true
default:
if dockerCli.CurrentContext() == "default" && dockerCli.DockerEndpoint().TLSData != nil {
return errors.Errorf("could not create a builder instance with TLS data loaded from environment. Please use `docker context create <context-name>` to create a context for current environment and then create a builder instance with `docker buildx create <context-name>`")
}
ep, err = getCurrentEndpoint(dockerCli)
ep, err = dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
}
if in.driver == "kubernetes" {
// naming endpoint to make --append works
ep = fmt.Sprintf("%s://%s?deployment=%s", in.driver, in.name, in.nodeName)
setEp = false
}
m, err := csvToMap(in.driverOpts)
if err != nil {
return err
}
if err := ng.Update(in.nodeName, ep, in.platform, len(args) > 0, in.actionAppend, flags, in.configFile, m); err != nil {
if in.configFile == "" {
// if buildkit config is not provided, check if the default one is
// available and use it
if f, ok := confutil.DefaultConfigFile(dockerCli); ok {
logrus.Warnf("Using default BuildKit config in %s", f)
in.configFile = f
}
}
if err := ng.Update(in.nodeName, ep, in.platform, setEp, in.actionAppend, flags, in.configFile, m); err != nil {
return err
}
}
@@ -161,8 +257,41 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
return err
}
b, err := builder.New(dockerCli,
builder.WithName(ng.Name),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
nodes, err := b.LoadNodes(timeoutCtx, true)
if err != nil {
return err
}
for _, node := range nodes {
if err := node.Err; err != nil {
err := errors.Errorf("failed to initialize builder %s (%s): %s", ng.Name, node.Name, err)
var err2 error
if ngOriginal == nil {
err2 = txn.Remove(ng.Name)
} else {
err2 = txn.Save(ngOriginal)
}
if err2 != nil {
logrus.Warnf("Could not rollback to previous state: %s", err2)
}
return err
}
}
if in.use && ep != "" {
current, err := getCurrentEndpoint(dockerCli)
current, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
@@ -171,6 +300,12 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
if in.bootstrap {
if _, err = b.Boot(ctx); err != nil {
return err
}
}
fmt.Printf("%s\n", ng.Name)
return nil
}
@@ -178,9 +313,12 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
func createCmd(dockerCli command.Cli) *cobra.Command {
var options createOptions
var drivers []string
for s := range driver.GetFactories() {
drivers = append(drivers, s)
var drivers bytes.Buffer
for _, d := range driver.GetFactories(true) {
if len(drivers.String()) > 0 {
drivers.WriteString(", ")
}
drivers.WriteString(fmt.Sprintf(`"%s"`, d.Name()))
}
cmd := &cobra.Command{
@@ -190,28 +328,34 @@ func createCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringVar(&options.name, "name", "", "Builder instance name")
flags.StringVar(&options.driver, "driver", "", fmt.Sprintf("Driver to use (available: %v)", drivers))
flags.StringVar(&options.driver, "driver", "", fmt.Sprintf("Driver to use (available: %s)", drivers.String()))
flags.StringVar(&options.nodeName, "node", "", "Create/modify node with given name")
flags.StringVar(&options.flags, "buildkitd-flags", "", "Flags for buildkitd daemon")
flags.StringVar(&options.configFile, "config", "", "BuildKit config file")
flags.StringArrayVar(&options.platform, "platform", []string{}, "Fixed platforms for current node")
flags.StringArrayVar(&options.driverOpts, "driver-opt", []string{}, "Options for the driver")
flags.BoolVar(&options.bootstrap, "bootstrap", false, "Boot builder after creation")
flags.BoolVar(&options.actionAppend, "append", false, "Append a node to builder instead of changing it")
flags.BoolVar(&options.actionLeave, "leave", false, "Remove a node from builder instead of changing it")
flags.BoolVar(&options.use, "use", false, "Set the current builder instance")
_ = flags
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}
func csvToMap(in []string) (map[string]string, error) {
if len(in) == 0 {
return nil, nil
}
m := make(map[string]string, len(in))
for _, s := range in {
csvReader := csv.NewReader(strings.NewReader(s))
@@ -229,3 +373,27 @@ func csvToMap(in []string) (map[string]string, error) {
}
return m, nil
}
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
dem, err := dockerutil.GetDockerEndpoint(dockerCli, ep)
if err == nil && dem != nil {
if ep == "default" {
return dem.Host, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// validateBuildkitEndpoint validates that endpoint is a valid buildkit host
func validateBuildkitEndpoint(ep string) (string, error) {
if err := remoteutil.IsValidEndpoint(ep); err != nil {
return "", err
}
return ep, nil
}

26
commands/create_test.go Normal file
View File

@@ -0,0 +1,26 @@
package commands
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestCsvToMap(t *testing.T) {
d := []string{
"\"tolerations=key=foo,value=bar;key=foo2,value=bar2\",replicas=1",
"namespace=default",
}
r, err := csvToMap(d)
require.NoError(t, err)
require.Contains(t, r, "tolerations")
require.Equal(t, r["tolerations"], "key=foo,value=bar;key=foo2,value=bar2")
require.Contains(t, r, "replicas")
require.Equal(t, r["replicas"], "1")
require.Contains(t, r, "namespace")
require.Equal(t, r["namespace"], "default")
}

70
commands/debug-shell.go Normal file
View File

@@ -0,0 +1,70 @@
package commands
import (
"context"
"os"
"runtime"
"github.com/containerd/console"
"github.com/docker/buildx/controller"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/monitor"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
func debugShellCmd(dockerCli command.Cli) *cobra.Command {
var options control.ControlOptions
var progressMode string
cmd := &cobra.Command{
Use: "debug-shell",
Short: "Start a monitor",
RunE: func(cmd *cobra.Command, args []string) error {
printer, err := progress.NewPrinter(context.TODO(), os.Stderr, os.Stderr, progressMode)
if err != nil {
return err
}
ctx := context.TODO()
c, err := controller.NewController(ctx, options, dockerCli, printer)
if err != nil {
return err
}
defer func() {
if err := c.Close(); err != nil {
logrus.Warnf("failed to close server connection %v", err)
}
}()
con := console.Current()
if err := con.SetRaw(); err != nil {
return errors.Errorf("failed to configure terminal: %v", err)
}
err = monitor.RunMonitor(ctx, "", nil, controllerapi.InvokeConfig{
Tty: true,
}, c, os.Stdin, os.Stdout, os.Stderr, printer)
con.Reset()
return err
},
}
flags := cmd.Flags()
flags.StringVar(&options.Root, "root", "", "Specify root directory of server to connect [experimental]")
flags.BoolVar(&options.Detach, "detach", runtime.GOOS == "linux", "Detach buildx server (supported only on linux) [experimental]")
flags.StringVar(&options.ServerConfig, "server-config", "", "Specify buildx server config file (used only when launching new server) [experimental]")
flags.StringVar(&progressMode, "progress", "auto", `Set type of progress output ("auto", "plain", "tty"). Use plain to show container output`)
return cmd
}
func addDebugShellCommand(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
debugShellCmd(dockerCli),
)
}

View File

@@ -4,16 +4,19 @@ import (
"fmt"
"io"
"os"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/appcontext"
"github.com/spf13/cobra"
"github.com/tonistiigi/units"
"golang.org/x/sync/errgroup"
)
@@ -31,25 +34,29 @@ func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
return err
}
dis, err := getInstanceOrDefault(ctx, dockerCli, opts.builder, "")
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
for _, di := range dis {
if di.Err != nil {
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
out := make([][]*client.UsageInfo, len(dis))
out := make([][]*client.UsageInfo, len(nodes))
eg, ctx := errgroup.WithContext(ctx)
for i, di := range dis {
func(i int, di build.DriverInfo) {
for i, node := range nodes {
func(i int, node builder.Node) {
eg.Go(func() error {
if di.Driver != nil {
c, err := di.Driver.Client(ctx)
if node.Driver != nil {
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
@@ -62,7 +69,7 @@ func runDiskUsage(dockerCli command.Cli, opts duOptions) error {
}
return nil
})
}(i, di)
}(i, node)
}
if err := eg.Wait(); err != nil {
@@ -109,6 +116,7 @@ func duCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runDiskUsage(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
@@ -125,20 +133,20 @@ func printKV(w io.Writer, k string, v interface{}) {
func printVerbose(tw *tabwriter.Writer, du []*client.UsageInfo) {
for _, di := range du {
printKV(tw, "ID", di.ID)
if di.Parent != "" {
printKV(tw, "Parent", di.Parent)
if len(di.Parents) != 0 {
printKV(tw, "Parent", strings.Join(di.Parents, ","))
}
printKV(tw, "Created at", di.CreatedAt)
printKV(tw, "Mutable", di.Mutable)
printKV(tw, "Reclaimable", !di.InUse)
printKV(tw, "Shared", di.Shared)
printKV(tw, "Size", fmt.Sprintf("%.2f", units.Bytes(di.Size)))
printKV(tw, "Size", units.HumanSize(float64(di.Size)))
if di.Description != "" {
printKV(tw, "Description", di.Description)
}
printKV(tw, "Usage count", di.UsageCount)
if di.LastUsedAt != nil {
printKV(tw, "Last used", di.LastUsedAt)
printKV(tw, "Last used", units.HumanDuration(time.Since(*di.LastUsedAt))+" ago")
}
if di.RecordType != "" {
printKV(tw, "Type", di.RecordType)
@@ -159,11 +167,15 @@ func printTableRow(tw *tabwriter.Writer, di *client.UsageInfo) {
if di.Mutable {
id += "*"
}
size := fmt.Sprintf("%.2f", units.Bytes(di.Size))
size := units.HumanSize(float64(di.Size))
if di.Shared {
size += "*"
}
fmt.Fprintf(tw, "%-71s\t%-11v\t%s\t\n", id, !di.InUse, size)
lastAccessed := ""
if di.LastUsedAt != nil {
lastAccessed = units.HumanDuration(time.Since(*di.LastUsedAt)) + " ago"
}
fmt.Fprintf(tw, "%-40s\t%-5v\t%-10s\t%s\n", id, !di.InUse, size, lastAccessed)
}
func printSummary(tw *tabwriter.Writer, dus [][]*client.UsageInfo) {
@@ -185,14 +197,12 @@ func printSummary(tw *tabwriter.Writer, dus [][]*client.UsageInfo) {
}
}
tw = tabwriter.NewWriter(os.Stdout, 1, 8, 1, '\t', 0)
if shared > 0 {
fmt.Fprintf(tw, "Shared:\t%.2f\n", units.Bytes(shared))
fmt.Fprintf(tw, "Private:\t%.2f\n", units.Bytes(total-shared))
fmt.Fprintf(tw, "Shared:\t%s\n", units.HumanSize(float64(shared)))
fmt.Fprintf(tw, "Private:\t%s\n", units.HumanSize(float64(total-shared)))
}
fmt.Fprintf(tw, "Reclaimable:\t%.2f\n", units.Bytes(reclaimable))
fmt.Fprintf(tw, "Total:\t%.2f\n", units.Bytes(total))
fmt.Fprintf(tw, "Reclaimable:\t%s\n", units.HumanSize(float64(reclaimable)))
fmt.Fprintf(tw, "Total:\t%s\n", units.HumanSize(float64(total)))
tw.Flush()
}

View File

@@ -1,12 +1,16 @@
package commands
import (
"context"
"encoding/json"
"fmt"
"io/ioutil"
"os"
"strings"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/distribution/reference"
"github.com/moby/buildkit/util/appcontext"
@@ -18,10 +22,12 @@ import (
)
type createOptions struct {
builder string
files []string
tags []string
dryrun bool
actionAppend bool
progress string
}
func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
@@ -35,7 +41,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
fileArgs := make([]string, len(in.files))
for i, f := range in.files {
dt, err := ioutil.ReadFile(f)
dt, err := os.ReadFile(f)
if err != nil {
return err
}
@@ -75,35 +81,48 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if len(repos) == 0 {
return errors.Errorf("no repositories specified, please set a reference in tag or source")
}
if len(repos) > 1 {
return errors.Errorf("multiple repositories currently not supported, found %v", repos)
}
var repo string
for r := range repos {
repo = r
var defaultRepo *string
if len(repos) == 1 {
for repo := range repos {
defaultRepo = &repo
}
}
for i, s := range srcs {
if s.Ref == nil && s.Desc.MediaType == "" && s.Desc.Digest != "" {
n, err := reference.ParseNormalizedNamed(repo)
if s.Ref == nil {
if defaultRepo == nil {
return errors.Errorf("multiple repositories specified, cannot infer repository for %q", args[i])
}
n, err := reference.ParseNormalizedNamed(*defaultRepo)
if err != nil {
return err
}
if s.Desc.MediaType == "" && s.Desc.Digest != "" {
r, err := reference.WithDigest(n, s.Desc.Digest)
if err != nil {
return err
}
srcs[i].Ref = r
sourceRefs = true
} else {
srcs[i].Ref = reference.TagNameOnly(n)
}
}
}
ctx := appcontext.Context()
r := imagetools.New(imagetools.Opt{
Auth: dockerCli.ConfigFile(),
})
b, err := builder.New(dockerCli, builder.WithName(in.builder))
if err != nil {
return err
}
imageopt, err := b.ImageOpt()
if err != nil {
return err
}
r := imagetools.New(imageopt)
if sourceRefs {
eg, ctx2 := errgroup.WithContext(ctx)
@@ -117,8 +136,15 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
if err != nil {
return err
}
srcs[i].Ref = nil
if srcs[i].Desc.Digest == "" {
srcs[i].Desc = desc
} else {
var err error
srcs[i].Desc, err = mergeDesc(desc, srcs[i].Desc)
if err != nil {
return err
}
}
return nil
})
}(i)
@@ -128,12 +154,7 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
}
descs := make([]ocispec.Descriptor, len(srcs))
for i := range descs {
descs[i] = srcs[i].Desc
}
dt, desc, err := r.Combine(ctx, repo, descs)
dt, desc, err := r.Combine(ctx, srcs)
if err != nil {
return err
}
@@ -144,31 +165,58 @@ func runCreate(dockerCli command.Cli, in createOptions, args []string) error {
}
// new resolver cause need new auth
r = imagetools.New(imagetools.Opt{
Auth: dockerCli.ConfigFile(),
})
r = imagetools.New(imageopt)
for _, t := range tags {
if err := r.Push(ctx, t, desc, dt); err != nil {
ctx2, cancel := context.WithCancel(context.TODO())
defer cancel()
printer, err := progress.NewPrinter(ctx2, os.Stderr, os.Stderr, in.progress)
if err != nil {
return err
}
fmt.Println(t.String())
eg, _ := errgroup.WithContext(ctx)
pw := progress.WithPrefix(printer, "internal", true)
for _, t := range tags {
t := t
eg.Go(func() error {
return progress.Wrap(fmt.Sprintf("pushing %s", t.String()), pw.Write, func(sub progress.SubLogger) error {
eg2, _ := errgroup.WithContext(ctx)
for _, s := range srcs {
if reference.Domain(s.Ref) == reference.Domain(t) && reference.Path(s.Ref) == reference.Path(t) {
continue
}
s := s
eg2.Go(func() error {
sub.Log(1, []byte(fmt.Sprintf("copying %s from %s to %s\n", s.Desc.Digest.String(), s.Ref.String(), t.String())))
return r.Copy(ctx, s, t)
})
}
return nil
if err := eg2.Wait(); err != nil {
return err
}
sub.Log(1, []byte(fmt.Sprintf("pushing %s to %s\n", desc.Digest.String(), t.String())))
return r.Push(ctx, t, desc, dt)
})
})
}
type src struct {
Desc ocispec.Descriptor
Ref reference.Named
err = eg.Wait()
err1 := printer.Wait()
if err == nil {
err = err1
}
func parseSources(in []string) ([]*src, error) {
out := make([]*src, len(in))
return err
}
func parseSources(in []string) ([]*imagetools.Source, error) {
out := make([]*imagetools.Source, len(in))
for i, in := range in {
s, err := parseSource(in)
if err != nil {
return nil, errors.Wrapf(err, "failed to parse source %q, valid sources are digests, refereces and descriptors", in)
return nil, errors.Wrapf(err, "failed to parse source %q, valid sources are digests, references and descriptors", in)
}
out[i] = s
}
@@ -187,11 +235,11 @@ func parseRefs(in []string) ([]reference.Named, error) {
return refs, nil
}
func parseSource(in string) (*src, error) {
func parseSource(in string) (*imagetools.Source, error) {
// source can be a digest, reference or a descriptor JSON
dgst, err := digest.Parse(in)
if err == nil {
return &src{
return &imagetools.Source{
Desc: ocispec.Descriptor{
Digest: dgst,
},
@@ -202,39 +250,55 @@ func parseSource(in string) (*src, error) {
ref, err := reference.ParseNormalizedNamed(in)
if err == nil {
return &src{
return &imagetools.Source{
Ref: ref,
}, nil
} else if !strings.HasPrefix(in, "{") {
return nil, err
}
var s src
var s imagetools.Source
if err := json.Unmarshal([]byte(in), &s.Desc); err != nil {
return nil, errors.WithStack(err)
}
return &s, nil
}
func createCmd(dockerCli command.Cli) *cobra.Command {
func createCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
var options createOptions
cmd := &cobra.Command{
Use: "create [OPTIONS] [SOURCE] [SOURCE...]",
Short: "Create a new image based on source images",
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *opts.Builder
return runCreate(dockerCli, options, args)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.StringArrayVarP(&options.files, "file", "f", []string{}, "Read source descriptor from file")
flags.StringArrayVarP(&options.tags, "tag", "t", []string{}, "Set reference for new image")
flags.BoolVar(&options.dryrun, "dry-run", false, "Show final image instead of pushing")
flags.BoolVar(&options.actionAppend, "append", false, "Append to existing manifest")
_ = flags
flags.StringVar(&options.progress, "progress", "auto", `Set type of progress output ("auto", "plain", "tty"). Use plain to show container output`)
return cmd
}
func mergeDesc(d1, d2 ocispec.Descriptor) (ocispec.Descriptor, error) {
if d2.Size != 0 && d1.Size != d2.Size {
return ocispec.Descriptor{}, errors.Errorf("invalid size mismatch for %s, %d != %d", d1.Digest, d2.Size, d1.Size)
}
if d2.MediaType != "" {
d1.MediaType = d2.MediaType
}
if len(d2.Annotations) != 0 {
d1.Annotations = d2.Annotations // no merge so support removes
}
if d2.Platform != nil {
d1.Platform = d2.Platform // missing items filled in later from image config
}
return d1, nil
}

View File

@@ -1,68 +1,67 @@
package commands
import (
"fmt"
"os"
"github.com/containerd/containerd/images"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/imagetools"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
type inspectOptions struct {
builder string
format string
raw bool
}
func runInspect(dockerCli command.Cli, in inspectOptions, name string) error {
ctx := appcontext.Context()
r := imagetools.New(imagetools.Opt{
Auth: dockerCli.ConfigFile(),
})
if in.format != "" && in.raw {
return errors.Errorf("format and raw cannot be used together")
}
dt, desc, err := r.Get(ctx, name)
b, err := builder.New(dockerCli, builder.WithName(in.builder))
if err != nil {
return err
}
imageopt, err := b.ImageOpt()
if err != nil {
return err
}
if in.raw {
fmt.Printf("%s", dt) // avoid newline to keep digest
return nil
p, err := imagetools.NewPrinter(ctx, imageopt, name, in.format)
if err != nil {
return err
}
switch desc.MediaType {
// case images.MediaTypeDockerSchema2Manifest, specs.MediaTypeImageManifest:
// TODO: handle distribution manifest and schema1
case images.MediaTypeDockerSchema2ManifestList, ocispec.MediaTypeImageIndex:
imagetools.PrintManifestList(dt, desc, name, os.Stdout)
default:
fmt.Printf("%s\n", dt)
return p.Print(in.raw, dockerCli.Out())
}
return nil
}
func inspectCmd(dockerCli command.Cli) *cobra.Command {
func inspectCmd(dockerCli command.Cli, rootOpts RootOptions) *cobra.Command {
var options inspectOptions
cmd := &cobra.Command{
Use: "inspect [OPTIONS] NAME",
Short: "Show details of image in the registry",
Short: "Show details of an image in the registry",
Args: cli.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = *rootOpts.Builder
return runInspect(dockerCli, options, args[0])
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.BoolVar(&options.raw, "raw", false, "Show original JSON manifest")
flags.StringVar(&options.format, "format", "", "Format the output using the given Go template")
flags.SetAnnotation("format", annotation.DefaultValue, []string{`"{{.Manifest}}"`})
_ = flags
flags.BoolVar(&options.raw, "raw", false, "Show original, unformatted JSON manifest")
return cmd
}

View File

@@ -1,19 +1,25 @@
package commands
import (
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli/command"
"github.com/spf13/cobra"
)
func RootCmd(dockerCli command.Cli) *cobra.Command {
type RootOptions struct {
Builder *string
}
func RootCmd(dockerCli command.Cli, opts RootOptions) *cobra.Command {
cmd := &cobra.Command{
Use: "imagetools",
Short: "Commands to work on images in registry",
ValidArgsFunction: completion.Disable,
}
cmd.AddCommand(
inspectCmd(dockerCli),
createCmd(dockerCli),
createCmd(dockerCli, opts),
inspectCmd(dockerCli, opts),
)
return cmd

View File

@@ -4,21 +4,21 @@ import (
"context"
"fmt"
"os"
"sort"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/store"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/debug"
"github.com/docker/go-units"
"github.com/moby/buildkit/util/appcontext"
specs "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type inspectOptions struct {
@@ -26,103 +26,106 @@ type inspectOptions struct {
builder string
}
type dinfo struct {
di *build.DriverInfo
info *driver.Info
platforms []specs.Platform
err error
}
type nginfo struct {
ng *store.NodeGroup
drivers []dinfo
err error
}
func runInspect(dockerCli command.Cli, in inspectOptions) error {
ctx := appcontext.Context()
txn, release, err := getStore(dockerCli)
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
defer release()
var ng *store.NodeGroup
if in.builder != "" {
ng, err = getNodeGroup(txn, dockerCli, in.builder)
if err != nil {
return err
}
} else {
ng, err = getCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
}
if ng == nil {
ng = &store.NodeGroup{
Name: "default",
Nodes: []store.Node{{
Name: "default",
Endpoint: "default",
}},
}
}
ngi := &nginfo{ng: ng}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
err = loadNodeGroupData(timeoutCtx, dockerCli, ngi)
var bootNgi *nginfo
nodes, err := b.LoadNodes(timeoutCtx, true)
if in.bootstrap {
var ok bool
ok, err = boot(ctx, ngi, dockerCli)
ok, err = b.Boot(ctx)
if err != nil {
return err
}
bootNgi = ngi
if ok {
ngi = &nginfo{ng: ng}
err = loadNodeGroupData(ctx, dockerCli, ngi)
nodes, err = b.LoadNodes(timeoutCtx, true)
}
}
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "Name:\t%s\n", ngi.ng.Name)
fmt.Fprintf(w, "Driver:\t%s\n", ngi.ng.Driver)
fmt.Fprintf(w, "Name:\t%s\n", b.Name)
fmt.Fprintf(w, "Driver:\t%s\n", b.Driver)
if !b.NodeGroup.LastActivity.IsZero() {
fmt.Fprintf(w, "Last Activity:\t%v\n", b.NodeGroup.LastActivity)
}
if err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if ngi.err != nil {
fmt.Fprintf(w, "Error:\t%s\n", ngi.err.Error())
} else if b.Err() != nil {
fmt.Fprintf(w, "Error:\t%s\n", b.Err().Error())
}
if err == nil {
fmt.Fprintln(w, "")
fmt.Fprintln(w, "Nodes:")
for i, n := range ngi.ng.Nodes {
for i, n := range nodes {
if i != 0 {
fmt.Fprintln(w, "")
}
fmt.Fprintf(w, "Name:\t%s\n", n.Name)
fmt.Fprintf(w, "Endpoint:\t%s\n", n.Endpoint)
if err := ngi.drivers[i].di.Err; err != nil {
var driverOpts []string
for k, v := range n.DriverOpts {
driverOpts = append(driverOpts, fmt.Sprintf("%s=%q", k, v))
}
if len(driverOpts) > 0 {
fmt.Fprintf(w, "Driver Options:\t%s\n", strings.Join(driverOpts, " "))
}
if err := n.Err; err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if err := ngi.drivers[i].err; err != nil {
fmt.Fprintf(w, "Error:\t%s\n", err.Error())
} else if bootNgi != nil && len(bootNgi.drivers) > i && bootNgi.drivers[i].err != nil {
fmt.Fprintf(w, "Error:\t%s\n", bootNgi.drivers[i].err.Error())
} else {
fmt.Fprintf(w, "Status:\t%s\n", ngi.drivers[i].info.Status)
fmt.Fprintf(w, "Status:\t%s\n", nodes[i].DriverInfo.Status)
if len(n.Flags) > 0 {
fmt.Fprintf(w, "Flags:\t%s\n", strings.Join(n.Flags, " "))
}
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.FormatInGroups(n.Platforms, ngi.drivers[i].platforms), ", "))
if nodes[i].Version != "" {
fmt.Fprintf(w, "Buildkit:\t%s\n", nodes[i].Version)
}
fmt.Fprintf(w, "Platforms:\t%s\n", strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
if debug.IsEnabled() {
fmt.Fprintf(w, "Features:\n")
features := nodes[i].Driver.Features(ctx)
featKeys := make([]string, 0, len(features))
for k := range features {
featKeys = append(featKeys, string(k))
}
sort.Strings(featKeys)
for _, k := range featKeys {
fmt.Fprintf(w, "\t%s:\t%t\n", k, features[driver.Feature(k)])
}
}
if len(nodes[i].Labels) > 0 {
fmt.Fprintf(w, "Labels:\n")
for _, k := range sortedKeys(nodes[i].Labels) {
v := nodes[i].Labels[k]
fmt.Fprintf(w, "\t%s:\t%s\n", k, v)
}
}
for ri, rule := range nodes[i].GCPolicy {
fmt.Fprintf(w, "GC Policy rule#%d:\n", ri)
fmt.Fprintf(w, "\tAll:\t%v\n", rule.All)
if len(rule.Filter) > 0 {
fmt.Fprintf(w, "\tFilters:\t%s\n", strings.Join(rule.Filter, " "))
}
if rule.KeepDuration > 0 {
fmt.Fprintf(w, "\tKeep Duration:\t%v\n", rule.KeepDuration.String())
}
if rule.KeepBytes > 0 {
fmt.Fprintf(w, "\tKeep Bytes:\t%s\n", units.BytesSize(float64(rule.KeepBytes)))
}
}
}
}
}
@@ -146,52 +149,22 @@ func inspectCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runInspect(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
flags.BoolVar(&options.bootstrap, "bootstrap", false, "Ensure builder has booted before inspecting")
_ = flags
return cmd
}
func boot(ctx context.Context, ngi *nginfo, dockerCli command.Cli) (bool, error) {
toBoot := make([]int, 0, len(ngi.drivers))
for i, d := range ngi.drivers {
if d.err != nil || d.di.Err != nil || d.di.Driver == nil || d.info == nil {
continue
func sortedKeys(m map[string]string) []string {
s := make([]string, len(m))
i := 0
for k := range m {
s[i] = k
i++
}
if d.info.Status != driver.Running {
toBoot = append(toBoot, i)
}
}
if len(toBoot) == 0 {
return false, nil
}
printer := progress.NewPrinter(context.TODO(), os.Stderr, "auto")
eg, _ := errgroup.WithContext(ctx)
for _, idx := range toBoot {
func(idx int) {
eg.Go(func() error {
pw := progress.WithPrefix(printer, ngi.ng.Nodes[idx].Name, len(toBoot) > 1)
_, err := driver.Boot(ctx, ngi.drivers[idx].di.Driver, pw)
if err != nil {
ngi.drivers[idx].err = err
}
return nil
})
}(idx)
}
err := eg.Wait()
err1 := printer.Wait()
if err == nil {
err = err1
}
return true, err
sort.Strings(s)
return s
}

View File

@@ -3,6 +3,8 @@ package commands
import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -46,7 +48,11 @@ func installCmd(dockerCli command.Cli) *cobra.Command {
return runInstall(dockerCli, options)
},
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}

View File

@@ -4,12 +4,14 @@ import (
"context"
"fmt"
"io"
"os"
"strings"
"text/tabwriter"
"time"
"github.com/docker/buildx/store"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -24,51 +26,30 @@ type lsOptions struct {
func runLs(dockerCli command.Cli, in lsOptions) error {
ctx := appcontext.Context()
txn, release, err := getStore(dockerCli)
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
ctx, cancel := context.WithTimeout(ctx, 20*time.Second)
current, err := storeutil.GetCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
builders, err := builder.GetBuilders(dockerCli, txn)
if err != nil {
return err
}
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
ll, err := txn.List()
if err != nil {
return err
}
builders := make([]*nginfo, len(ll))
for i, ng := range ll {
builders[i] = &nginfo{ng: ng}
}
list, err := dockerCli.ContextStore().List()
if err != nil {
return err
}
ctxbuilders := make([]*nginfo, len(list))
for i, l := range list {
ctxbuilders[i] = &nginfo{ng: &store.NodeGroup{
Name: l.Name,
Nodes: []store.Node{{
Name: l.Name,
Endpoint: l.Name,
}},
}}
}
builders = append(builders, ctxbuilders...)
eg, _ := errgroup.WithContext(ctx)
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {
func(b *nginfo) {
func(b *builder.Builder) {
eg.Go(func() error {
err = loadNodeGroupData(ctx, dockerCli, b)
if b.err == nil && err != nil {
b.err = err
}
_, _ = b.LoadNodes(timeoutCtx, true)
return nil
})
}(b)
@@ -78,61 +59,62 @@ func runLs(dockerCli command.Cli, in lsOptions) error {
return err
}
currentName := "default"
current, err := getCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
if current != nil {
currentName = current.Name
if current.Name == "default" {
currentName = current.Nodes[0].Endpoint
}
}
w := tabwriter.NewWriter(dockerCli.Out(), 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "NAME/NODE\tDRIVER/ENDPOINT\tSTATUS\tBUILDKIT\tPLATFORMS\n")
w := tabwriter.NewWriter(os.Stdout, 0, 0, 1, ' ', 0)
fmt.Fprintf(w, "NAME/NODE\tDRIVER/ENDPOINT\tSTATUS\tPLATFORMS\n")
currentSet := false
printErr := false
for _, b := range builders {
if !currentSet && b.ng.Name == currentName {
b.ng.Name += " *"
currentSet = true
if current.Name == b.Name {
b.Name += " *"
}
if ok := printBuilder(w, b); !ok {
printErr = true
}
printngi(w, b)
}
w.Flush()
if printErr {
_, _ = fmt.Fprintf(dockerCli.Err(), "\n")
for _, b := range builders {
if b.Err() != nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "Cannot load builder %s: %s\n", b.Name, strings.TrimSpace(b.Err().Error()))
} else {
for _, d := range b.Nodes() {
if d.Err != nil {
_, _ = fmt.Fprintf(dockerCli.Err(), "Failed to get status for %s (%s): %s\n", b.Name, d.Name, strings.TrimSpace(d.Err.Error()))
}
}
}
}
}
return nil
}
func printngi(w io.Writer, ngi *nginfo) {
func printBuilder(w io.Writer, b *builder.Builder) (ok bool) {
ok = true
var err string
if ngi.err != nil {
err = ngi.err.Error()
}
fmt.Fprintf(w, "%s\t%s\t%s\t\n", ngi.ng.Name, ngi.ng.Driver, err)
if ngi.err == nil {
for idx, n := range ngi.ng.Nodes {
d := ngi.drivers[idx]
var err string
if d.err != nil {
err = d.err.Error()
} else if d.di.Err != nil {
err = d.di.Err.Error()
if b.Err() != nil {
ok = false
err = "error"
}
fmt.Fprintf(w, "%s\t%s\t%s\t\t\n", b.Name, b.Driver, err)
if b.Err() == nil {
for _, n := range b.Nodes() {
var status string
if d.info != nil {
status = d.info.Status.String()
if n.DriverInfo != nil {
status = n.DriverInfo.Status.String()
}
if err != "" {
fmt.Fprintf(w, " %s\t%s\t%s\n", n.Name, n.Endpoint, err)
if n.Err != nil {
ok = false
fmt.Fprintf(w, " %s\t%s\t%s\t\t\n", n.Name, n.Endpoint, "error")
} else {
fmt.Fprintf(w, " %s\t%s\t%s\t%s\n", n.Name, n.Endpoint, status, strings.Join(platformutil.FormatInGroups(n.Platforms, d.platforms), ", "))
fmt.Fprintf(w, " %s\t%s\t%s\t%s\t%s\n", n.Name, n.Endpoint, status, n.Version, strings.Join(platformutil.FormatInGroups(n.Node.Platforms, n.Platforms), ", "))
}
}
}
return
}
func lsCmd(dockerCli command.Cli) *cobra.Command {
@@ -145,7 +127,11 @@ func lsCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runLs(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}

View File

@@ -7,16 +7,17 @@ import (
"text/tabwriter"
"time"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/opts"
"github.com/docker/docker/api/types/filters"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/tonistiigi/units"
"golang.org/x/sync/errgroup"
)
@@ -54,15 +55,19 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
return nil
}
dis, err := getInstanceOrDefault(ctx, dockerCli, opts.builder, "")
b, err := builder.New(dockerCli, builder.WithName(opts.builder))
if err != nil {
return err
}
for _, di := range dis {
if di.Err != nil {
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return err
}
for _, node := range nodes {
if node.Err != nil {
return node.Err
}
}
ch := make(chan client.UsageInfo)
@@ -90,11 +95,11 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
}()
eg, ctx := errgroup.WithContext(ctx)
for _, di := range dis {
func(di build.DriverInfo) {
for _, node := range nodes {
func(node builder.Node) {
eg.Go(func() error {
if di.Driver != nil {
c, err := di.Driver.Client(ctx)
if node.Driver != nil {
c, err := node.Driver.Client(ctx)
if err != nil {
return err
}
@@ -109,7 +114,7 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
}
return nil
})
}(di)
}(node)
}
if err := eg.Wait(); err != nil {
@@ -119,7 +124,7 @@ func runPrune(dockerCli command.Cli, opts pruneOptions) error {
<-printed
tw = tabwriter.NewWriter(os.Stdout, 1, 8, 1, '\t', 0)
fmt.Fprintf(tw, "Total:\t%.2f\n", units.Bytes(total))
fmt.Fprintf(tw, "Total:\t%s\n", units.HumanSize(float64(total)))
tw.Flush()
return nil
}
@@ -135,11 +140,12 @@ func pruneCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
options.builder = rootOpts.builder
return runPrune(dockerCli, options)
},
ValidArgsFunction: completion.Disable,
}
flags := cmd.Flags()
flags.BoolVarP(&options.all, "all", "a", false, "Remove all unused images, not just dangling ones")
flags.Var(&options.filter, "filter", "Provide filter values (e.g. 'until=24h')")
flags.BoolVarP(&options.all, "all", "a", false, "Include internal/frontend images")
flags.Var(&options.filter, "filter", `Provide filter values (e.g., "until=24h")`)
flags.Var(&options.keepStorage, "keep-storage", "Amount of disk space to keep for cache")
flags.BoolVar(&options.verbose, "verbose", false, "Provide a more verbose output")
flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation")
@@ -155,9 +161,9 @@ func toBuildkitPruneInfo(f filters.Args) (*client.PruneInfo, error) {
if len(untilValues) > 0 && len(unusedForValues) > 0 {
return nil, errors.Errorf("conflicting filters %q and %q", "until", "unused-for")
}
filterKey := "until"
untilKey := "until"
if len(unusedForValues) > 0 {
filterKey = "unused-for"
untilKey = "unused-for"
}
untilValues = append(untilValues, unusedForValues...)
@@ -168,23 +174,27 @@ func toBuildkitPruneInfo(f filters.Args) (*client.PruneInfo, error) {
var err error
until, err = time.ParseDuration(untilValues[0])
if err != nil {
return nil, errors.Wrapf(err, "%q filter expects a duration (e.g., '24h')", filterKey)
return nil, errors.Wrapf(err, "%q filter expects a duration (e.g., '24h')", untilKey)
}
default:
return nil, errors.Errorf("filters expect only one value")
}
bkFilter := make([]string, 0, f.Len())
for _, field := range f.Keys() {
values := f.Get(field)
filters := make([]string, 0, f.Len())
for _, filterKey := range f.Keys() {
if filterKey == untilKey {
continue
}
values := f.Get(filterKey)
switch len(values) {
case 0:
bkFilter = append(bkFilter, field)
filters = append(filters, filterKey)
case 1:
if field == "id" {
bkFilter = append(bkFilter, field+"~="+values[0])
if filterKey == "id" {
filters = append(filters, filterKey+"~="+values[0])
} else {
bkFilter = append(bkFilter, field+"=="+values[0])
filters = append(filters, filterKey+"=="+values[0])
}
default:
return nil, errors.Errorf("filters expect only one value")
@@ -192,6 +202,6 @@ func toBuildkitPruneInfo(f filters.Args) (*client.PruneInfo, error) {
}
return &client.PruneInfo{
KeepDuration: until,
Filter: []string{strings.Join(bkFilter, ",")},
Filter: []string{strings.Join(filters, ",")},
}, nil
}

View File

@@ -2,51 +2,77 @@ package commands
import (
"context"
"fmt"
"time"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"golang.org/x/sync/errgroup"
)
type rmOptions struct {
builder string
keepState bool
keepDaemon bool
allInactive bool
force bool
}
const (
rmInactiveWarning = `WARNING! This will remove all builders that are not in running state. Are you sure you want to continue?`
)
func runRm(dockerCli command.Cli, in rmOptions) error {
ctx := appcontext.Context()
txn, release, err := getStore(dockerCli)
if in.allInactive && !in.force && !command.PromptForConfirmation(dockerCli.In(), dockerCli.Out(), rmInactiveWarning) {
return nil
}
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
if in.builder != "" {
ng, err := getNodeGroup(txn, dockerCli, in.builder)
if in.allInactive {
return rmAllInactive(ctx, txn, dockerCli, in)
}
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithStore(txn),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
err1 := stop(ctx, dockerCli, ng, true)
if err := txn.Remove(ng.Name); err != nil {
return err
}
return err1
}
ng, err := getCurrentInstance(txn, dockerCli)
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return err
}
if ng != nil {
err1 := stop(ctx, dockerCli, ng, true)
if err := txn.Remove(ng.Name); err != nil {
if cb := b.ContextName(); cb != "" {
return errors.Errorf("context builder cannot be removed, run `docker context rm %s` to remove this context", cb)
}
err1 := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
if err1 != nil {
return err1
}
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.Name)
return nil
}
@@ -60,57 +86,78 @@ func rmCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
options.builder = rootOpts.builder
if len(args) > 0 {
if options.allInactive {
return errors.New("cannot specify builder name when --all-inactive is set")
}
options.builder = args[0]
}
return runRm(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
flags.BoolVar(&options.keepState, "keep-state", false, "Keep BuildKit state")
flags.BoolVar(&options.keepDaemon, "keep-daemon", false, "Keep the buildkitd daemon running")
flags.BoolVar(&options.allInactive, "all-inactive", false, "Remove all inactive builders")
flags.BoolVarP(&options.force, "force", "f", false, "Do not prompt for confirmation")
return cmd
}
func stop(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, rm bool) error {
dis, err := driversForNodeGroup(ctx, dockerCli, ng, "")
if err != nil {
return err
func rm(ctx context.Context, nodes []builder.Node, in rmOptions) (err error) {
for _, node := range nodes {
if node.Driver == nil {
continue
}
for _, di := range dis {
if di.Driver != nil {
if err := di.Driver.Stop(ctx, true); err != nil {
return err
}
if rm {
if err := di.Driver.Rm(ctx, true); err != nil {
// Do not stop the buildkitd daemon when --keep-daemon is provided
if !in.keepDaemon {
if err := node.Driver.Stop(ctx, true); err != nil {
return err
}
}
if err := node.Driver.Rm(ctx, true, !in.keepState, !in.keepDaemon); err != nil {
return err
}
if di.Err != nil {
err = di.Err
if node.Err != nil {
err = node.Err
}
}
return err
}
func stopCurrent(ctx context.Context, dockerCli command.Cli, rm bool) error {
dis, err := getDefaultDrivers(ctx, dockerCli, false, "")
func rmAllInactive(ctx context.Context, txn *store.Txn, dockerCli command.Cli, in rmOptions) error {
builders, err := builder.GetBuilders(dockerCli, txn)
if err != nil {
return err
}
for _, di := range dis {
if di.Driver != nil {
if err := di.Driver.Stop(ctx, true); err != nil {
timeoutCtx, cancel := context.WithTimeout(ctx, 20*time.Second)
defer cancel()
eg, _ := errgroup.WithContext(timeoutCtx)
for _, b := range builders {
func(b *builder.Builder) {
eg.Go(func() error {
nodes, err := b.LoadNodes(timeoutCtx, true)
if err != nil {
return errors.Wrapf(err, "cannot load %s", b.Name)
}
if b.Dynamic {
return nil
}
if b.Inactive() {
rmerr := rm(ctx, nodes, in)
if err := txn.Remove(b.Name); err != nil {
return err
}
if rm {
if err := di.Driver.Rm(ctx, true); err != nil {
return err
_, _ = fmt.Fprintf(dockerCli.Err(), "%s removed\n", b.Name)
return rmerr
}
return nil
})
}(b)
}
}
if di.Err != nil {
err = di.Err
}
}
return err
return eg.Wait()
}

View File

@@ -4,23 +4,65 @@ import (
"os"
imagetoolscmd "github.com/docker/buildx/commands/imagetools"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/logutil"
"github.com/docker/cli-docs-tool/annotation"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli-plugins/plugin"
"github.com/docker/cli/cli/command"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
)
func NewRootCmd(name string, isPlugin bool, dockerCli command.Cli) *cobra.Command {
cmd := &cobra.Command{
Short: "Build with BuildKit",
Short: "Docker Buildx",
Long: `Extended build capabilities with BuildKit`,
Use: name,
Annotations: map[string]string{
annotation.CodeDelimiter: `"`,
},
CompletionOptions: cobra.CompletionOptions{
HiddenDefaultCmd: true,
},
}
if isPlugin {
cmd.PersistentPreRunE = func(cmd *cobra.Command, args []string) error {
return plugin.PersistentPreRunE(cmd, args)
}
} else {
// match plugin behavior for standalone mode
// https://github.com/docker/cli/blob/6c9eb708fa6d17765d71965f90e1c59cea686ee9/cli-plugins/plugin/plugin.go#L117-L127
cmd.SilenceUsage = true
cmd.SilenceErrors = true
cmd.TraverseChildren = true
cmd.DisableFlagsInUseLine = true
cli.DisableFlagsInUseLine(cmd)
}
logrus.SetFormatter(&logutil.Formatter{})
logrus.AddHook(logutil.NewFilter([]logrus.Level{
logrus.DebugLevel,
},
"serving grpc connection",
"stopping session",
"using default config store",
))
// filter out useless commandConn.CloseWrite warning message that can occur
// when listing builder instances with "buildx ls" for those that are
// unreachable: "commandConn.CloseWrite: commandconn: failed to wait: signal: killed"
// https://github.com/docker/cli/blob/3fb4fb83dfb5db0c0753a8316f21aea54dab32c5/cli/connhelper/commandconn/commandconn.go#L203-L214
logrus.AddHook(logutil.NewFilter([]logrus.Level{
logrus.WarnLevel,
},
"commandConn.CloseWrite:",
"commandConn.CloseRead:",
))
addCommands(cmd, dockerCli)
return cmd
}
@@ -47,7 +89,16 @@ func addCommands(cmd *cobra.Command, dockerCli command.Cli) {
versionCmd(dockerCli),
pruneCmd(dockerCli, opts),
duCmd(dockerCli, opts),
imagetoolscmd.RootCmd(dockerCli),
imagetoolscmd.RootCmd(dockerCli, imagetoolscmd.RootOptions{Builder: &opts.builder}),
)
if isExperimental() {
remote.AddControllerCommands(cmd, dockerCli)
addDebugShellCommand(cmd, dockerCli)
}
cmd.RegisterFlagCompletionFunc( //nolint:errcheck
"builder",
completion.BuilderNames(dockerCli),
)
}

View File

@@ -1,6 +1,10 @@
package commands
import (
"context"
"github.com/docker/buildx/builder"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/util/appcontext"
@@ -14,32 +18,19 @@ type stopOptions struct {
func runStop(dockerCli command.Cli, in stopOptions) error {
ctx := appcontext.Context()
txn, release, err := getStore(dockerCli)
b, err := builder.New(dockerCli,
builder.WithName(in.builder),
builder.WithSkippedValidation(),
)
if err != nil {
return err
}
defer release()
if in.builder != "" {
ng, err := getNodeGroup(txn, dockerCli, in.builder)
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return err
}
if err := stop(ctx, dockerCli, ng, false); err != nil {
return err
}
return nil
}
ng, err := getCurrentInstance(txn, dockerCli)
if err != nil {
return err
}
if ng != nil {
return stop(ctx, dockerCli, ng, false)
}
return stopCurrent(ctx, dockerCli, false)
return stop(ctx, nodes)
}
func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
@@ -56,13 +47,22 @@ func stopCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runStop(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
// flags.StringArrayVarP(&options.outputs, "output", "o", []string{}, "Output destination (format: type=local,dest=path)")
_ = flags
return cmd
}
func stop(ctx context.Context, nodes []builder.Node) (err error) {
for _, node := range nodes {
if node.Driver != nil {
if err := node.Driver.Stop(ctx, true); err != nil {
return err
}
}
if node.Err != nil {
err = node.Err
}
}
return err
}

View File

@@ -3,6 +3,8 @@ package commands
import (
"os"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
@@ -52,7 +54,11 @@ func uninstallCmd(dockerCli command.Cli) *cobra.Command {
return runUninstall(dockerCli, options)
},
Hidden: true,
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}

View File

@@ -3,6 +3,9 @@ package commands
import (
"os"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
@@ -16,7 +19,7 @@ type useOptions struct {
}
func runUse(dockerCli command.Cli, in useOptions) error {
txn, release, err := getStore(dockerCli)
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
@@ -28,7 +31,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Errorf("run `docker context use default` to switch to default context")
}
if in.builder == "default" || in.builder == dockerCli.CurrentContext() {
ep, err := getCurrentEndpoint(dockerCli)
ep, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
@@ -51,7 +54,7 @@ func runUse(dockerCli command.Cli, in useOptions) error {
return errors.Wrapf(err, "failed to find instance %q", in.builder)
}
ep, err := getCurrentEndpoint(dockerCli)
ep, err := dockerutil.GetCurrentEndpoint(dockerCli)
if err != nil {
return err
}
@@ -76,14 +79,12 @@ func useCmd(dockerCli command.Cli, rootOpts *rootOptions) *cobra.Command {
}
return runUse(dockerCli, options)
},
ValidArgsFunction: completion.BuilderNames(dockerCli),
}
flags := cmd.Flags()
flags.BoolVar(&options.isGlobal, "global", false, "Builder persists context changes")
flags.BoolVar(&options.isDefault, "default", false, "Set builder as default for current context")
_ = flags
return cmd
}

View File

@@ -1,445 +0,0 @@
package commands
import (
"context"
"os"
"path/filepath"
"github.com/docker/buildx/build"
"github.com/docker/buildx/driver"
"github.com/docker/buildx/store"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/context/docker"
"github.com/docker/cli/cli/context/kubernetes"
dopts "github.com/docker/cli/opts"
dockerclient "github.com/docker/docker/client"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
// getStore returns current builder instance store
func getStore(dockerCli command.Cli) (*store.Txn, func(), error) {
s, err := store.New(getConfigStorePath(dockerCli))
if err != nil {
return nil, nil, err
}
return s.Txn()
}
// getConfigStorePath will look for correct configuration store path;
// if `$BUILDX_CONFIG` is set - use it, otherwise use parent directory
// of Docker config file (i.e. `${DOCKER_CONFIG}/buildx`)
func getConfigStorePath(dockerCli command.Cli) string {
if buildxConfig := os.Getenv("BUILDX_CONFIG"); buildxConfig != "" {
logrus.Debugf("using config store %q based in \"$BUILDX_CONFIG\" environment variable", buildxConfig)
return buildxConfig
}
buildxConfig := filepath.Join(filepath.Dir(dockerCli.ConfigFile().Filename), "buildx")
logrus.Debugf("using default config store %q", buildxConfig)
return buildxConfig
}
// getCurrentEndpoint returns the current default endpoint value
func getCurrentEndpoint(dockerCli command.Cli) (string, error) {
name := dockerCli.CurrentContext()
if name != "default" {
return name, nil
}
de, err := getDockerEndpoint(dockerCli, name)
if err != nil {
return "", errors.Errorf("docker endpoint for %q not found", name)
}
return de, nil
}
// getDockerEndpoint returns docker endpoint string for given context
func getDockerEndpoint(dockerCli command.Cli, name string) (string, error) {
list, err := dockerCli.ContextStore().List()
if err != nil {
return "", err
}
for _, l := range list {
if l.Name == name {
ep, ok := l.Endpoints["docker"]
if !ok {
return "", errors.Errorf("context %q does not have a Docker endpoint", name)
}
typed, ok := ep.(docker.EndpointMeta)
if !ok {
return "", errors.Errorf("endpoint %q is not of type EndpointMeta, %T", ep, ep)
}
return typed.Host, nil
}
}
return "", nil
}
// validateEndpoint validates that endpoint is either a context or a docker host
func validateEndpoint(dockerCli command.Cli, ep string) (string, error) {
de, err := getDockerEndpoint(dockerCli, ep)
if err == nil && de != "" {
if ep == "default" {
return de, nil
}
return ep, nil
}
h, err := dopts.ParseHost(true, ep)
if err != nil {
return "", errors.Wrapf(err, "failed to parse endpoint %s", ep)
}
return h, nil
}
// getCurrentInstance finds the current builder instance
func getCurrentInstance(txn *store.Txn, dockerCli command.Cli) (*store.NodeGroup, error) {
ep, err := getCurrentEndpoint(dockerCli)
if err != nil {
return nil, err
}
ng, err := txn.Current(ep)
if err != nil {
return nil, err
}
if ng == nil {
ng, _ = getNodeGroup(txn, dockerCli, dockerCli.CurrentContext())
}
return ng, nil
}
// getNodeGroup returns nodegroup based on the name
func getNodeGroup(txn *store.Txn, dockerCli command.Cli, name string) (*store.NodeGroup, error) {
ng, err := txn.NodeGroupByName(name)
if err != nil {
if !os.IsNotExist(errors.Cause(err)) {
return nil, err
}
}
if ng != nil {
return ng, nil
}
if name == "default" {
name = dockerCli.CurrentContext()
}
list, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, l := range list {
if l.Name == name {
return &store.NodeGroup{
Name: "default",
Nodes: []store.Node{
{
Name: "default",
Endpoint: name,
},
},
}, nil
}
}
return nil, errors.Errorf("no builder %q found", name)
}
// driversForNodeGroup returns drivers for a nodegroup instance
func driversForNodeGroup(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, contextPathHash string) ([]build.DriverInfo, error) {
eg, _ := errgroup.WithContext(ctx)
dis := make([]build.DriverInfo, len(ng.Nodes))
var f driver.Factory
if ng.Driver != "" {
f = driver.GetFactory(ng.Driver, true)
if f == nil {
return nil, errors.Errorf("failed to find driver %q", f)
}
} else {
dockerapi, err := clientForEndpoint(dockerCli, ng.Nodes[0].Endpoint)
if err != nil {
return nil, err
}
f, err = driver.GetDefaultFactory(ctx, dockerapi, false)
if err != nil {
return nil, err
}
ng.Driver = f.Name()
}
for i, n := range ng.Nodes {
func(i int, n store.Node) {
eg.Go(func() error {
di := build.DriverInfo{
Name: n.Name,
Platform: n.Platforms,
}
defer func() {
dis[i] = di
}()
dockerapi, err := clientForEndpoint(dockerCli, n.Endpoint)
if err != nil {
di.Err = err
return nil
}
// TODO: replace the following line with dockerclient.WithAPIVersionNegotiation option in clientForEndpoint
dockerapi.NegotiateAPIVersion(ctx)
contextStore := dockerCli.ContextStore()
var kcc driver.KubeClientConfig
kcc, err = kubernetes.ConfigFromContext(n.Endpoint, contextStore)
if err != nil {
// err is returned if n.Endpoint is non-context name like "unix:///var/run/docker.sock".
// try again with name="default".
// FIXME: n should retain real context name.
kcc, err = kubernetes.ConfigFromContext("default", contextStore)
if err != nil {
logrus.Error(err)
}
}
tryToUseKubeConfigInCluster := false
if kcc == nil {
tryToUseKubeConfigInCluster = true
} else {
if _, err := kcc.ClientConfig(); err != nil {
tryToUseKubeConfigInCluster = true
}
}
if tryToUseKubeConfigInCluster {
kccInCluster := driver.KubeClientConfigInCluster{}
if _, err := kccInCluster.ClientConfig(); err == nil {
logrus.Debug("using kube config in cluster")
kcc = kccInCluster
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_"+n.Name, f, dockerapi, dockerCli.ConfigFile(), kcc, n.Flags, n.ConfigFile, n.DriverOpts, n.Platforms, contextPathHash)
if err != nil {
di.Err = err
return nil
}
di.Driver = d
return nil
})
}(i, n)
}
if err := eg.Wait(); err != nil {
return nil, err
}
return dis, nil
}
// clientForEndpoint returns a docker client for an endpoint
func clientForEndpoint(dockerCli command.Cli, name string) (dockerclient.APIClient, error) {
list, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, l := range list {
if l.Name == name {
dep, ok := l.Endpoints["docker"]
if !ok {
return nil, errors.Errorf("context %q does not have a Docker endpoint", name)
}
epm, ok := dep.(docker.EndpointMeta)
if !ok {
return nil, errors.Errorf("endpoint %q is not of type EndpointMeta, %T", dep, dep)
}
ep, err := docker.WithTLSData(dockerCli.ContextStore(), name, epm)
if err != nil {
return nil, err
}
clientOpts, err := ep.ClientOpts()
if err != nil {
return nil, err
}
return dockerclient.NewClientWithOpts(clientOpts...)
}
}
ep := docker.Endpoint{
EndpointMeta: docker.EndpointMeta{
Host: name,
},
}
clientOpts, err := ep.ClientOpts()
if err != nil {
return nil, err
}
return dockerclient.NewClientWithOpts(clientOpts...)
}
func getInstanceOrDefault(ctx context.Context, dockerCli command.Cli, instance, contextPathHash string) ([]build.DriverInfo, error) {
var defaultOnly bool
if instance == "default" && instance != dockerCli.CurrentContext() {
return nil, errors.Errorf("use `docker --context=default buildx` to switch to default context")
}
if instance == "default" || instance == dockerCli.CurrentContext() {
instance = ""
defaultOnly = true
}
list, err := dockerCli.ContextStore().List()
if err != nil {
return nil, err
}
for _, l := range list {
if l.Name == instance {
return nil, errors.Errorf("use `docker --context=%s buildx` to switch to context %s", instance, instance)
}
}
if instance != "" {
return getInstanceByName(ctx, dockerCli, instance, contextPathHash)
}
return getDefaultDrivers(ctx, dockerCli, defaultOnly, contextPathHash)
}
func getInstanceByName(ctx context.Context, dockerCli command.Cli, instance, contextPathHash string) ([]build.DriverInfo, error) {
txn, release, err := getStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
ng, err := txn.NodeGroupByName(instance)
if err != nil {
return nil, err
}
return driversForNodeGroup(ctx, dockerCli, ng, contextPathHash)
}
// getDefaultDrivers returns drivers based on current cli config
func getDefaultDrivers(ctx context.Context, dockerCli command.Cli, defaultOnly bool, contextPathHash string) ([]build.DriverInfo, error) {
txn, release, err := getStore(dockerCli)
if err != nil {
return nil, err
}
defer release()
if !defaultOnly {
ng, err := getCurrentInstance(txn, dockerCli)
if err != nil {
return nil, err
}
if ng != nil {
return driversForNodeGroup(ctx, dockerCli, ng, contextPathHash)
}
}
d, err := driver.GetDriver(ctx, "buildx_buildkit_default", nil, dockerCli.Client(), dockerCli.ConfigFile(), nil, nil, "", nil, nil, contextPathHash)
if err != nil {
return nil, err
}
return []build.DriverInfo{
{
Name: "default",
Driver: d,
},
}, nil
}
func loadInfoData(ctx context.Context, d *dinfo) error {
if d.di.Driver == nil {
return nil
}
info, err := d.di.Driver.Info(ctx)
if err != nil {
return err
}
d.info = info
if info.Status == driver.Running {
c, err := d.di.Driver.Client(ctx)
if err != nil {
return err
}
workers, err := c.ListWorkers(ctx)
if err != nil {
return errors.Wrap(err, "listing workers")
}
for _, w := range workers {
for _, p := range w.Platforms {
d.platforms = append(d.platforms, p)
}
}
d.platforms = platformutil.Dedupe(d.platforms)
}
return nil
}
func loadNodeGroupData(ctx context.Context, dockerCli command.Cli, ngi *nginfo) error {
eg, _ := errgroup.WithContext(ctx)
dis, err := driversForNodeGroup(ctx, dockerCli, ngi.ng, "")
if err != nil {
return err
}
ngi.drivers = make([]dinfo, len(dis))
for i, di := range dis {
d := di
ngi.drivers[i].di = &d
func(d *dinfo) {
eg.Go(func() error {
if err := loadInfoData(ctx, d); err != nil {
d.err = err
}
return nil
})
}(&ngi.drivers[i])
}
if eg.Wait(); err != nil {
return err
}
// skip when multi drivers
if len(ngi.drivers) == 1 {
for _, di := range ngi.drivers {
// dynamic nodes are used in Kubernetes driver.
// Kubernetes pods are dynamically mapped to BuildKit Nodes.
if di.info != nil && len(di.info.DynamicNodes) > 0 {
var drivers []dinfo
for i := 0; i < len(di.info.DynamicNodes); i++ {
// all []dinfo share *build.DriverInfo and *driver.Info
diClone := di
if pl := di.info.DynamicNodes[i].Platforms; len(pl) > 0 {
diClone.platforms = pl
}
drivers = append(drivers, di)
}
// not append (remove the static nodes in the store)
ngi.ng.Nodes = di.info.DynamicNodes
ngi.ng.Dynamic = true
ngi.drivers = drivers
return nil
}
}
}
return nil
}
func dockerAPI(dockerCli command.Cli) *api {
return &api{dockerCli: dockerCli}
}
type api struct {
dockerCli command.Cli
}
func (a *api) DockerAPI(name string) (dockerclient.APIClient, error) {
if name == "" {
name = a.dockerCli.CurrentContext()
}
return clientForEndpoint(a.dockerCli, name)
}

View File

@@ -3,6 +3,8 @@ package commands
import (
"fmt"
"github.com/docker/buildx/util/cobrautil"
"github.com/docker/buildx/util/cobrautil/completion"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli"
"github.com/docker/cli/cli/command"
@@ -22,6 +24,11 @@ func versionCmd(dockerCli command.Cli) *cobra.Command {
RunE: func(cmd *cobra.Command, args []string) error {
return runVersion(dockerCli)
},
ValidArgsFunction: completion.Disable,
}
// hide builder persistent flag for this command
cobrautil.HideInheritedFlags(cmd, "builder")
return cmd
}

266
controller/build/build.go Normal file
View File

@@ -0,0 +1,266 @@
package build
import (
"context"
"io"
"os"
"path/filepath"
"strings"
"sync"
"github.com/docker/buildx/build"
"github.com/docker/buildx/builder"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/store"
"github.com/docker/buildx/store/storeutil"
"github.com/docker/buildx/util/buildflags"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/dockerutil"
"github.com/docker/buildx/util/platformutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/docker/cli/cli/config"
dockeropts "github.com/docker/cli/opts"
"github.com/docker/go-units"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/session/auth/authprovider"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"google.golang.org/grpc/codes"
)
const defaultTargetName = "default"
// RunBuild runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func RunBuild(ctx context.Context, dockerCli command.Cli, in controllerapi.BuildOptions, inStream io.Reader, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
if in.NoCache && len(in.NoCacheFilter) > 0 {
return nil, nil, errors.Errorf("--no-cache and --no-cache-filter cannot currently be used together")
}
contexts := map[string]build.NamedContext{}
for name, path := range in.NamedContexts {
contexts[name] = build.NamedContext{Path: path}
}
opts := build.Options{
Inputs: build.Inputs{
ContextPath: in.ContextPath,
DockerfilePath: in.DockerfileName,
InStream: inStream,
NamedContexts: contexts,
},
BuildArgs: in.BuildArgs,
ExtraHosts: in.ExtraHosts,
Labels: in.Labels,
NetworkMode: in.NetworkMode,
NoCache: in.NoCache,
NoCacheFilter: in.NoCacheFilter,
Pull: in.Pull,
ShmSize: dockeropts.MemBytes(in.ShmSize),
Tags: in.Tags,
Target: in.Target,
Ulimits: controllerUlimitOpt2DockerUlimit(in.Ulimits),
}
platforms, err := platformutil.Parse(in.Platforms)
if err != nil {
return nil, nil, err
}
opts.Platforms = platforms
dockerConfig := config.LoadDefaultConfigFile(os.Stderr)
opts.Session = append(opts.Session, authprovider.NewDockerAuthProvider(dockerConfig))
secrets, err := controllerapi.CreateSecrets(in.Secrets)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, secrets)
sshSpecs := in.SSH
if len(sshSpecs) == 0 && buildflags.IsGitSSH(in.ContextPath) {
sshSpecs = append(sshSpecs, &controllerapi.SSH{ID: "default"})
}
ssh, err := controllerapi.CreateSSH(sshSpecs)
if err != nil {
return nil, nil, err
}
opts.Session = append(opts.Session, ssh)
outputs, err := controllerapi.CreateExports(in.Exports)
if err != nil {
return nil, nil, err
}
if in.ExportPush {
if in.ExportLoad {
return nil, nil, errors.Errorf("push and load may not be set together at the moment")
}
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "image",
Attrs: map[string]string{
"push": "true",
},
}}
} else {
switch outputs[0].Type {
case "image":
outputs[0].Attrs["push"] = "true"
default:
return nil, nil, errors.Errorf("push and %q output can't be used together", outputs[0].Type)
}
}
}
if in.ExportLoad {
if len(outputs) == 0 {
outputs = []client.ExportEntry{{
Type: "docker",
Attrs: map[string]string{},
}}
} else {
switch outputs[0].Type {
case "docker":
default:
return nil, nil, errors.Errorf("load and %q output can't be used together", outputs[0].Type)
}
}
}
opts.Exports = outputs
opts.CacheFrom = controllerapi.CreateCaches(in.CacheFrom)
opts.CacheTo = controllerapi.CreateCaches(in.CacheTo)
opts.Attests = controllerapi.CreateAttestations(in.Attests)
opts.SourcePolicy = in.SourcePolicy
allow, err := buildflags.ParseEntitlements(in.Allow)
if err != nil {
return nil, nil, err
}
opts.Allow = allow
if in.PrintFunc != nil {
opts.PrintFunc = &build.PrintFunc{
Name: in.PrintFunc.Name,
Format: in.PrintFunc.Format,
}
}
// key string used for kubernetes "sticky" mode
contextPathHash, err := filepath.Abs(in.ContextPath)
if err != nil {
contextPathHash = in.ContextPath
}
// TODO: this should not be loaded this side of the controller api
b, err := builder.New(dockerCli,
builder.WithName(in.Builder),
builder.WithContextPathHash(contextPathHash),
)
if err != nil {
return nil, nil, err
}
if err = updateLastActivity(dockerCli, b.NodeGroup); err != nil {
return nil, nil, errors.Wrapf(err, "failed to update builder last activity time")
}
nodes, err := b.LoadNodes(ctx, false)
if err != nil {
return nil, nil, err
}
resp, res, err := buildTargets(ctx, dockerCli, b.NodeGroup, nodes, map[string]build.Options{defaultTargetName: opts}, progress, generateResult)
err = wrapBuildError(err, false)
if err != nil {
// NOTE: buildTargets can return *build.ResultHandle even on error.
return nil, res, err
}
return resp, res, nil
}
// buildTargets runs the specified build and returns the result.
//
// NOTE: When an error happens during the build and this function acquires the debuggable *build.ResultHandle,
// this function returns it in addition to the error (i.e. it does "return nil, res, err"). The caller can
// inspect the result and debug the cause of that error.
func buildTargets(ctx context.Context, dockerCli command.Cli, ng *store.NodeGroup, nodes []builder.Node, opts map[string]build.Options, progress progress.Writer, generateResult bool) (*client.SolveResponse, *build.ResultHandle, error) {
var res *build.ResultHandle
var resp map[string]*client.SolveResponse
var err error
if generateResult {
var mu sync.Mutex
var idx int
resp, err = build.BuildWithResultHandler(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress, func(driverIndex int, gotRes *build.ResultHandle) {
mu.Lock()
defer mu.Unlock()
if res == nil || driverIndex < idx {
idx, res = driverIndex, gotRes
}
})
} else {
resp, err = build.Build(ctx, nodes, opts, dockerutil.NewClient(dockerCli), confutil.ConfigDir(dockerCli), progress)
}
if err != nil {
return nil, res, err
}
return resp[defaultTargetName], res, err
}
func wrapBuildError(err error, bake bool) error {
if err == nil {
return nil
}
st, ok := grpcerrors.AsGRPCStatus(err)
if ok {
if st.Code() == codes.Unimplemented && strings.Contains(st.Message(), "unsupported frontend capability moby.buildkit.frontend.contexts") {
msg := "current frontend does not support --build-context."
if bake {
msg = "current frontend does not support defining additional contexts for targets."
}
msg += " Named contexts are supported since Dockerfile v1.4. Use #syntax directive in Dockerfile or update to latest BuildKit."
return &wrapped{err, msg}
}
}
return err
}
type wrapped struct {
err error
msg string
}
func (w *wrapped) Error() string {
return w.msg
}
func (w *wrapped) Unwrap() error {
return w.err
}
func updateLastActivity(dockerCli command.Cli, ng *store.NodeGroup) error {
txn, release, err := storeutil.GetStore(dockerCli)
if err != nil {
return err
}
defer release()
return txn.UpdateLastActivity(ng)
}
func controllerUlimitOpt2DockerUlimit(u *controllerapi.UlimitOpt) *dockeropts.UlimitOpt {
if u == nil {
return nil
}
values := make(map[string]*units.Ulimit)
for k, v := range u.Values {
values[k] = &units.Ulimit{
Name: v.Name,
Hard: v.Hard,
Soft: v.Soft,
}
}
return dockeropts.NewUlimitOpt(&values)
}

View File

@@ -0,0 +1,32 @@
package control
import (
"context"
"io"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
)
type BuildxController interface {
Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (ref string, resp *client.SolveResponse, err error)
// Invoke starts an IO session into the specified process.
// If pid doesn't matche to any running processes, it starts a new process with the specified config.
// If there is no container running or InvokeConfig.Rollback is speicfied, the process will start in a newly created container.
// NOTE: If needed, in the future, we can split this API into three APIs (NewContainer, NewProcess and Attach).
Invoke(ctx context.Context, ref, pid string, options controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error
Kill(ctx context.Context) error
Close() error
List(ctx context.Context) (refs []string, _ error)
Disconnect(ctx context.Context, ref string) error
ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error)
DisconnectProcess(ctx context.Context, ref, pid string) error
Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error)
}
type ControlOptions struct {
ServerConfig string
Root string
Detach bool
}

36
controller/controller.go Normal file
View File

@@ -0,0 +1,36 @@
package controller
import (
"context"
"fmt"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/controller/local"
"github.com/docker/buildx/controller/remote"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
)
func NewController(ctx context.Context, opts control.ControlOptions, dockerCli command.Cli, pw progress.Writer) (control.BuildxController, error) {
var name string
if opts.Detach {
name = "remote"
} else {
name = "local"
}
var c control.BuildxController
err := progress.Wrap(fmt.Sprintf("[internal] connecting to %s controller", name), pw.Write, func(l progress.SubLogger) (err error) {
if opts.Detach {
c, err = remote.NewRemoteBuildxController(ctx, dockerCli, opts, l)
} else {
c = local.NewLocalBuildxController(ctx, dockerCli, l)
}
return err
})
if err != nil {
return nil, errors.Wrap(err, "failed to start buildx controller")
}
return c, nil
}

View File

@@ -0,0 +1,34 @@
package errdefs
import (
"github.com/containerd/typeurl/v2"
"github.com/moby/buildkit/util/grpcerrors"
)
func init() {
typeurl.Register((*Build)(nil), "github.com/docker/buildx", "errdefs.Build+json")
}
type BuildError struct {
Build
error
}
func (e *BuildError) Unwrap() error {
return e.error
}
func (e *BuildError) ToProto() grpcerrors.TypedErrorProto {
return &e.Build
}
func WrapBuild(err error, ref string) error {
if err == nil {
return nil
}
return &BuildError{Build: Build{Ref: ref}, error: err}
}
func (b *Build) WrapError(err error) error {
return &BuildError{error: err, Build: *b}
}

View File

@@ -0,0 +1,77 @@
// Code generated by protoc-gen-gogo. DO NOT EDIT.
// source: errdefs.proto
package errdefs
import (
fmt "fmt"
proto "github.com/gogo/protobuf/proto"
_ "github.com/moby/buildkit/solver/pb"
math "math"
)
// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf
// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.GoGoProtoPackageIsVersion3 // please upgrade the proto package
type Build struct {
Ref string `protobuf:"bytes,1,opt,name=Ref,proto3" json:"Ref,omitempty"`
XXX_NoUnkeyedLiteral struct{} `json:"-"`
XXX_unrecognized []byte `json:"-"`
XXX_sizecache int32 `json:"-"`
}
func (m *Build) Reset() { *m = Build{} }
func (m *Build) String() string { return proto.CompactTextString(m) }
func (*Build) ProtoMessage() {}
func (*Build) Descriptor() ([]byte, []int) {
return fileDescriptor_689dc58a5060aff5, []int{0}
}
func (m *Build) XXX_Unmarshal(b []byte) error {
return xxx_messageInfo_Build.Unmarshal(m, b)
}
func (m *Build) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
return xxx_messageInfo_Build.Marshal(b, m, deterministic)
}
func (m *Build) XXX_Merge(src proto.Message) {
xxx_messageInfo_Build.Merge(m, src)
}
func (m *Build) XXX_Size() int {
return xxx_messageInfo_Build.Size(m)
}
func (m *Build) XXX_DiscardUnknown() {
xxx_messageInfo_Build.DiscardUnknown(m)
}
var xxx_messageInfo_Build proto.InternalMessageInfo
func (m *Build) GetRef() string {
if m != nil {
return m.Ref
}
return ""
}
func init() {
proto.RegisterType((*Build)(nil), "errdefs.Build")
}
func init() { proto.RegisterFile("errdefs.proto", fileDescriptor_689dc58a5060aff5) }
var fileDescriptor_689dc58a5060aff5 = []byte{
// 111 bytes of a gzipped FileDescriptorProto
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0xe2, 0x4d, 0x2d, 0x2a, 0x4a,
0x49, 0x4d, 0x2b, 0xd6, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x87, 0x72, 0xa5, 0x74, 0xd2,
0x33, 0x4b, 0x32, 0x4a, 0x93, 0xf4, 0x92, 0xf3, 0x73, 0xf5, 0x73, 0xf3, 0x93, 0x2a, 0xf5, 0x93,
0x4a, 0x33, 0x73, 0x52, 0xb2, 0x33, 0x4b, 0xf4, 0x8b, 0xf3, 0x73, 0xca, 0x52, 0x8b, 0xf4, 0x0b,
0x92, 0xf4, 0xf3, 0x0b, 0xa0, 0xda, 0x94, 0x24, 0xb9, 0x58, 0x9d, 0x40, 0xf2, 0x42, 0x02, 0x5c,
0xcc, 0x41, 0xa9, 0x69, 0x12, 0x8c, 0x0a, 0x8c, 0x1a, 0x9c, 0x41, 0x20, 0x66, 0x12, 0x1b, 0x58,
0x85, 0x31, 0x20, 0x00, 0x00, 0xff, 0xff, 0x56, 0x52, 0x41, 0x91, 0x69, 0x00, 0x00, 0x00,
}

View File

@@ -0,0 +1,9 @@
syntax = "proto3";
package errdefs;
import "github.com/moby/buildkit/solver/pb/ops.proto";
message Build {
string Ref = 1;
}

View File

@@ -0,0 +1,3 @@
package errdefs
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. errdefs.proto

View File

@@ -0,0 +1,146 @@
package local
import (
"context"
"io"
"sync/atomic"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllererrors "github.com/docker/buildx/controller/errdefs"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func NewLocalBuildxController(ctx context.Context, dockerCli command.Cli, logger progress.SubLogger) control.BuildxController {
return &localController{
dockerCli: dockerCli,
ref: "local",
processes: processes.NewManager(),
}
}
type buildConfig struct {
// TODO: these two structs should be merged
// Discussion: https://github.com/docker/buildx/pull/1640#discussion_r1113279719
resultCtx *build.ResultHandle
buildOptions *controllerapi.BuildOptions
}
type localController struct {
dockerCli command.Cli
ref string
buildConfig buildConfig
processes *processes.Manager
buildOnGoing atomic.Bool
}
func (b *localController) Build(ctx context.Context, options controllerapi.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
if !b.buildOnGoing.CompareAndSwap(false, true) {
return "", nil, errors.New("build ongoing")
}
defer b.buildOnGoing.Store(false)
resp, res, buildErr := cbuild.RunBuild(ctx, b.dockerCli, options, in, progress, true)
// NOTE: RunBuild can return *build.ResultHandle even on error.
if res != nil {
b.buildConfig = buildConfig{
resultCtx: res,
buildOptions: &options,
}
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, b.ref)
}
}
if buildErr != nil {
return "", nil, buildErr
}
return b.ref, resp, nil
}
func (b *localController) ListProcesses(ctx context.Context, ref string) (infos []*controllerapi.ProcessInfo, retErr error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return b.processes.ListProcesses(), nil
}
func (b *localController) DisconnectProcess(ctx context.Context, ref, pid string) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
return b.processes.DeleteProcess(pid)
}
func (b *localController) cancelRunningProcesses() {
b.processes.CancelRunningProcesses()
}
func (b *localController) Invoke(ctx context.Context, ref string, pid string, cfg controllerapi.InvokeConfig, ioIn io.ReadCloser, ioOut io.WriteCloser, ioErr io.WriteCloser) error {
if ref != b.ref {
return errors.Errorf("unknown ref %q", ref)
}
proc, ok := b.processes.Get(pid)
if !ok {
// Start a new process.
if b.buildConfig.resultCtx == nil {
return errors.New("no build result is registered")
}
var err error
proc, err = b.processes.StartProcess(pid, b.buildConfig.resultCtx, &cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
ioCancelledCh := make(chan struct{})
proc.ForwardIO(&ioset.In{Stdin: ioIn, Stdout: ioOut, Stderr: ioErr}, func() { close(ioCancelledCh) })
select {
case <-ioCancelledCh:
return errors.Errorf("io cancelled")
case err := <-proc.Done():
return err
case <-ctx.Done():
return ctx.Err()
}
}
func (b *localController) Kill(context.Context) error {
b.Close()
return nil
}
func (b *localController) Close() error {
b.cancelRunningProcesses()
if b.buildConfig.resultCtx != nil {
b.buildConfig.resultCtx.Done()
}
// TODO: cancel ongoing builds?
return nil
}
func (b *localController) List(ctx context.Context) (res []string, _ error) {
return []string{b.ref}, nil
}
func (b *localController) Disconnect(ctx context.Context, key string) error {
b.Close()
return nil
}
func (b *localController) Inspect(ctx context.Context, ref string) (*controllerapi.InspectResponse, error) {
if ref != b.ref {
return nil, errors.Errorf("unknown ref %q", ref)
}
return &controllerapi.InspectResponse{Options: b.buildConfig.buildOptions}, nil
}

20
controller/pb/attest.go Normal file
View File

@@ -0,0 +1,20 @@
package pb
func CreateAttestations(attests []*Attest) map[string]*string {
result := map[string]*string{}
for _, attest := range attests {
// ignore duplicates
if _, ok := result[attest.Type]; ok {
continue
}
if attest.Disabled {
result[attest.Type] = nil
continue
}
attrs := attest.Attrs
result[attest.Type] = &attrs
}
return result
}

21
controller/pb/cache.go Normal file
View File

@@ -0,0 +1,21 @@
package pb
import "github.com/moby/buildkit/client"
func CreateCaches(entries []*CacheOptionsEntry) []client.CacheOptionsEntry {
var outs []client.CacheOptionsEntry
if len(entries) == 0 {
return nil
}
for _, entry := range entries {
out := client.CacheOptionsEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
outs = append(outs, out)
}
return outs
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,244 @@
syntax = "proto3";
package buildx.controller.v1;
import "github.com/moby/buildkit/api/services/control/control.proto";
import "github.com/moby/buildkit/sourcepolicy/pb/policy.proto";
option go_package = "pb";
service Controller {
rpc Build(BuildRequest) returns (BuildResponse);
rpc Inspect(InspectRequest) returns (InspectResponse);
rpc Status(StatusRequest) returns (stream StatusResponse);
rpc Input(stream InputMessage) returns (InputResponse);
rpc Invoke(stream Message) returns (stream Message);
rpc List(ListRequest) returns (ListResponse);
rpc Disconnect(DisconnectRequest) returns (DisconnectResponse);
rpc Info(InfoRequest) returns (InfoResponse);
rpc ListProcesses(ListProcessesRequest) returns (ListProcessesResponse);
rpc DisconnectProcess(DisconnectProcessRequest) returns (DisconnectProcessResponse);
}
message ListProcessesRequest {
string Ref = 1;
}
message ListProcessesResponse {
repeated ProcessInfo Infos = 1;
}
message ProcessInfo {
string ProcessID = 1;
InvokeConfig InvokeConfig = 2;
}
message DisconnectProcessRequest {
string Ref = 1;
string ProcessID = 2;
}
message DisconnectProcessResponse {
}
message BuildRequest {
string Ref = 1;
BuildOptions Options = 2;
}
message BuildOptions {
string ContextPath = 1;
string DockerfileName = 2;
PrintFunc PrintFunc = 3;
map<string, string> NamedContexts = 4;
repeated string Allow = 5;
repeated Attest Attests = 6;
map<string, string> BuildArgs = 7;
repeated CacheOptionsEntry CacheFrom = 8;
repeated CacheOptionsEntry CacheTo = 9;
string CgroupParent = 10;
repeated ExportEntry Exports = 11;
repeated string ExtraHosts = 12;
map<string, string> Labels = 13;
string NetworkMode = 14;
repeated string NoCacheFilter = 15;
repeated string Platforms = 16;
repeated Secret Secrets = 17;
int64 ShmSize = 18;
repeated SSH SSH = 19;
repeated string Tags = 20;
string Target = 21;
UlimitOpt Ulimits = 22;
string Builder = 23;
bool NoCache = 24;
bool Pull = 25;
bool ExportPush = 26;
bool ExportLoad = 27;
moby.buildkit.v1.sourcepolicy.Policy SourcePolicy = 28;
}
message ExportEntry {
string Type = 1;
map<string, string> Attrs = 2;
string Destination = 3;
}
message CacheOptionsEntry {
string Type = 1;
map<string, string> Attrs = 2;
}
message Attest {
string Type = 1;
bool Disabled = 2;
string Attrs = 3;
}
message SSH {
string ID = 1;
repeated string Paths = 2;
}
message Secret {
string ID = 1;
string FilePath = 2;
string Env = 3;
}
message PrintFunc {
string Name = 1;
string Format = 2;
}
message InspectRequest {
string Ref = 1;
}
message InspectResponse {
BuildOptions Options = 1;
}
message UlimitOpt {
map<string, Ulimit> values = 1;
}
message Ulimit {
string Name = 1;
int64 Hard = 2;
int64 Soft = 3;
}
message BuildResponse {
map<string, string> ExporterResponse = 1;
}
message DisconnectRequest {
string Ref = 1;
}
message DisconnectResponse {}
message ListRequest {
string Ref = 1;
}
message ListResponse {
repeated string keys = 1;
}
message InputMessage {
oneof Input {
InputInitMessage Init = 1;
DataMessage Data = 2;
}
}
message InputInitMessage {
string Ref = 1;
}
message DataMessage {
bool EOF = 1; // true if eof was reached
bytes Data = 2; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message InputResponse {}
message Message {
oneof Input {
InitMessage Init = 1;
// FdMessage used from client to server for input (stdin) and
// from server to client for output (stdout, stderr)
FdMessage File = 2;
// ResizeMessage used from client to server for terminal resize events
ResizeMessage Resize = 3;
// SignalMessage is used from client to server to send signal events
SignalMessage Signal = 4;
}
}
message InitMessage {
string Ref = 1;
// If ProcessID already exists in the server, it tries to connect to it
// instead of invoking the new one. In this case, InvokeConfig will be ignored.
string ProcessID = 2;
InvokeConfig InvokeConfig = 3;
}
message InvokeConfig {
repeated string Entrypoint = 1;
repeated string Cmd = 2;
repeated string Env = 3;
string User = 4;
bool NoUser = 5; // Do not set user but use the image's default
string Cwd = 6;
bool NoCwd = 7; // Do not set cwd but use the image's default
bool Tty = 8;
bool Rollback = 9; // Kill all process in the container and recreate it.
bool Initial = 10; // Run container from the initial state of that stage (supported only on the failed step)
}
message FdMessage {
uint32 Fd = 1; // what fd the data was from
bool EOF = 2; // true if eof was reached
bytes Data = 3; // should be chunked smaller than 4MB:
// https://pkg.go.dev/google.golang.org/grpc#MaxRecvMsgSize
}
message ResizeMessage {
uint32 Rows = 1;
uint32 Cols = 2;
}
message SignalMessage {
// we only send name (ie HUP, INT) because the int values
// are platform dependent.
string Name = 1;
}
message StatusRequest {
string Ref = 1;
}
message StatusResponse {
repeated moby.buildkit.v1.Vertex vertexes = 1;
repeated moby.buildkit.v1.VertexStatus statuses = 2;
repeated moby.buildkit.v1.VertexLog logs = 3;
repeated moby.buildkit.v1.VertexWarning warnings = 4;
}
message InfoRequest {}
message InfoResponse {
BuildxVersion buildxVersion = 1;
}
message BuildxVersion {
string package = 1;
string version = 2;
string revision = 3;
}

100
controller/pb/export.go Normal file
View File

@@ -0,0 +1,100 @@
package pb
import (
"io"
"os"
"strconv"
"github.com/containerd/console"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
)
func CreateExports(entries []*ExportEntry) ([]client.ExportEntry, error) {
var outs []client.ExportEntry
if len(entries) == 0 {
return nil, nil
}
for _, entry := range entries {
if entry.Type == "" {
return nil, errors.Errorf("type is required for output")
}
out := client.ExportEntry{
Type: entry.Type,
Attrs: map[string]string{},
}
for k, v := range entry.Attrs {
out.Attrs[k] = v
}
supportFile := false
supportDir := false
switch out.Type {
case client.ExporterLocal:
supportDir = true
case client.ExporterTar:
supportFile = true
case client.ExporterOCI, client.ExporterDocker:
tar, err := strconv.ParseBool(out.Attrs["tar"])
if err != nil {
tar = true
}
supportFile = tar
supportDir = !tar
case "registry":
out.Type = client.ExporterImage
}
if supportDir {
if entry.Destination == "" {
return nil, errors.Errorf("dest is required for %s exporter", out.Type)
}
if entry.Destination == "-" {
return nil, errors.Errorf("dest cannot be stdout for %s exporter", out.Type)
}
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination directory: %s", entry.Destination)
}
if err == nil && !fi.IsDir() {
return nil, errors.Errorf("destination directory %s is a file", entry.Destination)
}
out.OutputDir = entry.Destination
}
if supportFile {
if entry.Destination == "" && out.Type != client.ExporterDocker {
entry.Destination = "-"
}
if entry.Destination == "-" {
if _, err := console.ConsoleFromFile(os.Stdout); err == nil {
return nil, errors.Errorf("dest file is required for %s exporter. refusing to write to console", out.Type)
}
out.Output = wrapWriteCloser(os.Stdout)
} else if entry.Destination != "" {
fi, err := os.Stat(entry.Destination)
if err != nil && !os.IsNotExist(err) {
return nil, errors.Wrapf(err, "invalid destination file: %s", entry.Destination)
}
if err == nil && fi.IsDir() {
return nil, errors.Errorf("destination file %s is a directory", entry.Destination)
}
f, err := os.Create(entry.Destination)
if err != nil {
return nil, errors.Errorf("failed to open %s", err)
}
out.Output = wrapWriteCloser(f)
}
}
outs = append(outs, out)
}
return outs, nil
}
func wrapWriteCloser(wc io.WriteCloser) func(map[string]string) (io.WriteCloser, error) {
return func(map[string]string) (io.WriteCloser, error) {
return wc, nil
}
}

View File

@@ -0,0 +1,3 @@
package pb
//go:generate protoc -I=. -I=../../vendor/ --gogo_out=plugins=grpc:. controller.proto

175
controller/pb/path.go Normal file
View File

@@ -0,0 +1,175 @@
package pb
import (
"path/filepath"
"strings"
"github.com/docker/docker/builder/remotecontext/urlutil"
"github.com/moby/buildkit/util/gitutil"
)
// ResolveOptionPaths resolves all paths contained in BuildOptions
// and replaces them to absolute paths.
func ResolveOptionPaths(options *BuildOptions) (_ *BuildOptions, err error) {
localContext := false
if options.ContextPath != "" && options.ContextPath != "-" {
if !isRemoteURL(options.ContextPath) {
localContext = true
options.ContextPath, err = filepath.Abs(options.ContextPath)
if err != nil {
return nil, err
}
}
}
if options.DockerfileName != "" && options.DockerfileName != "-" {
if localContext && !urlutil.IsURL(options.DockerfileName) {
options.DockerfileName, err = filepath.Abs(options.DockerfileName)
if err != nil {
return nil, err
}
}
}
var contexts map[string]string
for k, v := range options.NamedContexts {
if isRemoteURL(v) || strings.HasPrefix(v, "docker-image://") {
// url prefix, this is a remote path
} else if strings.HasPrefix(v, "oci-layout://") {
// oci layout prefix, this is a local path
p := strings.TrimPrefix(v, "oci-layout://")
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
v = "oci-layout://" + p
} else {
// no prefix, assume local path
v, err = filepath.Abs(v)
if err != nil {
return nil, err
}
}
if contexts == nil {
contexts = make(map[string]string)
}
contexts[k] = v
}
options.NamedContexts = contexts
var cacheFrom []*CacheOptionsEntry
for _, co := range options.CacheFrom {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "src":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheFrom = append(cacheFrom, co)
default:
cacheFrom = append(cacheFrom, co)
}
}
options.CacheFrom = cacheFrom
var cacheTo []*CacheOptionsEntry
for _, co := range options.CacheTo {
switch co.Type {
case "local":
var attrs map[string]string
for k, v := range co.Attrs {
if attrs == nil {
attrs = make(map[string]string)
}
switch k {
case "dest":
p := v
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
attrs[k] = p
default:
attrs[k] = v
}
}
co.Attrs = attrs
cacheTo = append(cacheTo, co)
default:
cacheTo = append(cacheTo, co)
}
}
options.CacheTo = cacheTo
var exports []*ExportEntry
for _, e := range options.Exports {
if e.Destination != "" && e.Destination != "-" {
e.Destination, err = filepath.Abs(e.Destination)
if err != nil {
return nil, err
}
}
exports = append(exports, e)
}
options.Exports = exports
var secrets []*Secret
for _, s := range options.Secrets {
if s.FilePath != "" {
s.FilePath, err = filepath.Abs(s.FilePath)
if err != nil {
return nil, err
}
}
secrets = append(secrets, s)
}
options.Secrets = secrets
var ssh []*SSH
for _, s := range options.SSH {
var ps []string
for _, pt := range s.Paths {
p := pt
if p != "" {
p, err = filepath.Abs(p)
if err != nil {
return nil, err
}
}
ps = append(ps, p)
}
s.Paths = ps
ssh = append(ssh, s)
}
options.SSH = ssh
return options, nil
}
func isRemoteURL(c string) bool {
if urlutil.IsURL(c) {
return true
}
if _, err := gitutil.ParseGitRef(c); err == nil {
return true
}
return false
}

247
controller/pb/path_test.go Normal file
View File

@@ -0,0 +1,247 @@
package pb
import (
"os"
"path/filepath"
"reflect"
"testing"
"github.com/stretchr/testify/require"
)
func TestResolvePaths(t *testing.T) {
tmpwd, err := os.MkdirTemp("", "testresolvepaths")
require.NoError(t, err)
defer os.Remove(tmpwd)
require.NoError(t, os.Chdir(tmpwd))
tests := []struct {
name string
options BuildOptions
want BuildOptions
}{
{
name: "contextpath",
options: BuildOptions{ContextPath: "test"},
want: BuildOptions{ContextPath: filepath.Join(tmpwd, "test")},
},
{
name: "contextpath-cwd",
options: BuildOptions{ContextPath: "."},
want: BuildOptions{ContextPath: tmpwd},
},
{
name: "contextpath-dash",
options: BuildOptions{ContextPath: "-"},
want: BuildOptions{ContextPath: "-"},
},
{
name: "contextpath-ssh",
options: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "dockerfilename",
options: BuildOptions{DockerfileName: "test", ContextPath: "."},
want: BuildOptions{DockerfileName: filepath.Join(tmpwd, "test"), ContextPath: tmpwd},
},
{
name: "dockerfilename-dash",
options: BuildOptions{DockerfileName: "-", ContextPath: "."},
want: BuildOptions{DockerfileName: "-", ContextPath: tmpwd},
},
{
name: "dockerfilename-remote",
options: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
want: BuildOptions{DockerfileName: "test", ContextPath: "git@github.com:docker/buildx.git"},
},
{
name: "contexts",
options: BuildOptions{NamedContexts: map[string]string{"a": "test1", "b": "test2",
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
want: BuildOptions{NamedContexts: map[string]string{"a": filepath.Join(tmpwd, "test1"), "b": filepath.Join(tmpwd, "test2"),
"alpine": "docker-image://alpine@sha256:0123456789", "project": "https://github.com/myuser/project.git"}},
},
{
name: "cache-from",
options: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheFrom: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"src": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "cache-to",
options: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": "test"},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
want: BuildOptions{
CacheTo: []*CacheOptionsEntry{
{
Type: "local",
Attrs: map[string]string{"dest": filepath.Join(tmpwd, "test")},
},
{
Type: "registry",
Attrs: map[string]string{"ref": "user/app"},
},
},
},
},
{
name: "exports",
options: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: "test1",
},
{
Type: "tar",
Destination: "test3",
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: "test4",
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
want: BuildOptions{
Exports: []*ExportEntry{
{
Type: "local",
Destination: "-",
},
{
Type: "local",
Destination: filepath.Join(tmpwd, "test1"),
},
{
Type: "tar",
Destination: filepath.Join(tmpwd, "test3"),
},
{
Type: "oci",
Destination: "-",
},
{
Type: "docker",
Destination: filepath.Join(tmpwd, "test4"),
},
{
Type: "image",
Attrs: map[string]string{"push": "true"},
},
},
},
},
{
name: "secrets",
options: BuildOptions{
Secrets: []*Secret{
{
FilePath: "test1",
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: "test3",
},
},
},
want: BuildOptions{
Secrets: []*Secret{
{
FilePath: filepath.Join(tmpwd, "test1"),
},
{
ID: "val",
Env: "a",
},
{
ID: "test",
FilePath: filepath.Join(tmpwd, "test3"),
},
},
},
},
{
name: "ssh",
options: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{"test1", "test2"},
},
{
ID: "a",
Paths: []string{"test3"},
},
},
},
want: BuildOptions{
SSH: []*SSH{
{
ID: "default",
Paths: []string{filepath.Join(tmpwd, "test1"), filepath.Join(tmpwd, "test2")},
},
{
ID: "a",
Paths: []string{filepath.Join(tmpwd, "test3")},
},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got, err := ResolveOptionPaths(&tt.options)
require.NoError(t, err)
if !reflect.DeepEqual(tt.want, *got) {
t.Fatalf("expected %#v, got %#v", tt.want, *got)
}
})
}
}

126
controller/pb/progress.go Normal file
View File

@@ -0,0 +1,126 @@
package pb
import (
"github.com/docker/buildx/util/progress"
control "github.com/moby/buildkit/api/services/control"
"github.com/moby/buildkit/client"
"github.com/opencontainers/go-digest"
)
type writer struct {
ch chan<- *StatusResponse
}
func NewProgressWriter(ch chan<- *StatusResponse) progress.Writer {
return &writer{ch: ch}
}
func (w *writer) Write(status *client.SolveStatus) {
w.ch <- ToControlStatus(status)
}
func (w *writer) WriteBuildRef(target string, ref string) {
return
}
func (w *writer) ValidateLogSource(digest.Digest, interface{}) bool {
return true
}
func (w *writer) ClearLogSource(interface{}) {}
func ToControlStatus(s *client.SolveStatus) *StatusResponse {
resp := StatusResponse{}
for _, v := range s.Vertexes {
resp.Vertexes = append(resp.Vertexes, &control.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range s.Statuses {
resp.Statuses = append(resp.Statuses, &control.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range s.Logs {
resp.Logs = append(resp.Logs, &control.VertexLog{
Vertex: v.Vertex,
Stream: int64(v.Stream),
Msg: v.Data,
Timestamp: v.Timestamp,
})
}
for _, v := range s.Warnings {
resp.Warnings = append(resp.Warnings, &control.VertexWarning{
Vertex: v.Vertex,
Level: int64(v.Level),
Short: v.Short,
Detail: v.Detail,
Url: v.URL,
Info: v.SourceInfo,
Ranges: v.Range,
})
}
return &resp
}
func FromControlStatus(resp *StatusResponse) *client.SolveStatus {
s := client.SolveStatus{}
for _, v := range resp.Vertexes {
s.Vertexes = append(s.Vertexes, &client.Vertex{
Digest: v.Digest,
Inputs: v.Inputs,
Name: v.Name,
Started: v.Started,
Completed: v.Completed,
Error: v.Error,
Cached: v.Cached,
ProgressGroup: v.ProgressGroup,
})
}
for _, v := range resp.Statuses {
s.Statuses = append(s.Statuses, &client.VertexStatus{
ID: v.ID,
Vertex: v.Vertex,
Name: v.Name,
Total: v.Total,
Current: v.Current,
Timestamp: v.Timestamp,
Started: v.Started,
Completed: v.Completed,
})
}
for _, v := range resp.Logs {
s.Logs = append(s.Logs, &client.VertexLog{
Vertex: v.Vertex,
Stream: int(v.Stream),
Data: v.Msg,
Timestamp: v.Timestamp,
})
}
for _, v := range resp.Warnings {
s.Warnings = append(s.Warnings, &client.VertexWarning{
Vertex: v.Vertex,
Level: int(v.Level),
Short: v.Short,
Detail: v.Detail,
URL: v.Url,
SourceInfo: v.Info,
Range: v.Ranges,
})
}
return &s
}

22
controller/pb/secrets.go Normal file
View File

@@ -0,0 +1,22 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/secrets/secretsprovider"
)
func CreateSecrets(secrets []*Secret) (session.Attachable, error) {
fs := make([]secretsprovider.Source, 0, len(secrets))
for _, secret := range secrets {
fs = append(fs, secretsprovider.Source{
ID: secret.ID,
FilePath: secret.FilePath,
Env: secret.Env,
})
}
store, err := secretsprovider.NewStore(fs)
if err != nil {
return nil, err
}
return secretsprovider.NewSecretProvider(store), nil
}

18
controller/pb/ssh.go Normal file
View File

@@ -0,0 +1,18 @@
package pb
import (
"github.com/moby/buildkit/session"
"github.com/moby/buildkit/session/sshforward/sshprovider"
)
func CreateSSH(ssh []*SSH) (session.Attachable, error) {
configs := make([]sshprovider.AgentConfig, 0, len(ssh))
for _, ssh := range ssh {
cfg := sshprovider.AgentConfig{
ID: ssh.ID,
Paths: append([]string{}, ssh.Paths...),
}
configs = append(configs, cfg)
}
return sshprovider.NewSSHAgentProvider(configs)
}

View File

@@ -0,0 +1,149 @@
package processes
import (
"context"
"sync"
"sync/atomic"
"github.com/docker/buildx/build"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/ioset"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// Process provides methods to control a process.
type Process struct {
inEnd *ioset.Forwarder
invokeConfig *pb.InvokeConfig
errCh chan error
processCancel func()
serveIOCancel func()
}
// ForwardIO forwards process's io to the specified reader/writer.
// Optionally specify ioCancelCallback which will be called when
// the process closes the specified IO. This will be useful for additional cleanup.
func (p *Process) ForwardIO(in *ioset.In, ioCancelCallback func()) {
p.inEnd.SetIn(in)
if f := p.serveIOCancel; f != nil {
f()
}
p.serveIOCancel = ioCancelCallback
}
// Done returns a channel where error or nil will be sent
// when the process exits.
// TODO: change this to Wait()
func (p *Process) Done() <-chan error {
return p.errCh
}
// Manager manages a set of proceses.
type Manager struct {
container atomic.Value
processes sync.Map
}
// NewManager creates and returns a Manager.
func NewManager() *Manager {
return &Manager{}
}
// Get returns the specified process.
func (m *Manager) Get(id string) (*Process, bool) {
v, ok := m.processes.Load(id)
if !ok {
return nil, false
}
return v.(*Process), true
}
// CancelRunningProcesses cancels execution of all running processes.
func (m *Manager) CancelRunningProcesses() {
var funcs []func()
m.processes.Range(func(key, value any) bool {
funcs = append(funcs, value.(*Process).processCancel)
m.processes.Delete(key)
return true
})
for _, f := range funcs {
f()
}
}
// ListProcesses lists all running processes.
func (m *Manager) ListProcesses() (res []*pb.ProcessInfo) {
m.processes.Range(func(key, value any) bool {
res = append(res, &pb.ProcessInfo{
ProcessID: key.(string),
InvokeConfig: value.(*Process).invokeConfig,
})
return true
})
return res
}
// DeleteProcess deletes the specified process.
func (m *Manager) DeleteProcess(id string) error {
p, ok := m.processes.LoadAndDelete(id)
if !ok {
return errors.Errorf("unknown process %q", id)
}
p.(*Process).processCancel()
return nil
}
// StartProcess starts a process in the container.
// When a container isn't available (i.e. first time invoking or the container has exited) or cfg.Rollback is set,
// this method will start a new container and run the process in it. Otherwise, this method starts a new process in the
// existing container.
func (m *Manager) StartProcess(pid string, resultCtx *build.ResultHandle, cfg *pb.InvokeConfig) (*Process, error) {
// Get the target result to invoke a container from
var ctr *build.Container
if a := m.container.Load(); a != nil {
ctr = a.(*build.Container)
}
if cfg.Rollback || ctr == nil || ctr.IsUnavailable() {
go m.CancelRunningProcesses()
// (Re)create a new container if this is rollback or first time to invoke a process.
if ctr != nil {
go ctr.Cancel() // Finish the existing container
}
var err error
ctr, err = build.NewContainer(context.TODO(), resultCtx, cfg)
if err != nil {
return nil, errors.Errorf("failed to create container %v", err)
}
m.container.Store(ctr)
}
// [client(ForwardIO)] <-forwarder(switchable)-> [out] <-pipe-> [in] <- [process]
in, out := ioset.Pipe()
f := ioset.NewForwarder()
f.PropagateStdinClose = false
f.SetOut(&out)
// Register process
ctx, cancel := context.WithCancel(context.TODO())
var cancelOnce sync.Once
processCancelFunc := func() { cancelOnce.Do(func() { cancel(); f.Close(); in.Close(); out.Close() }) }
p := &Process{
inEnd: f,
invokeConfig: cfg,
processCancel: processCancelFunc,
errCh: make(chan error),
}
m.processes.Store(pid, p)
go func() {
var err error
if err = ctr.Exec(ctx, cfg, in.Stdin, in.Stdout, in.Stderr); err != nil {
logrus.Errorf("failed to exec process: %v", err)
}
logrus.Debugf("finished process %s %v", pid, cfg.Entrypoint)
m.processes.Delete(pid)
processCancelFunc()
p.errCh <- err
}()
return p, nil
}

240
controller/remote/client.go Normal file
View File

@@ -0,0 +1,240 @@
package remote
import (
"context"
"io"
"sync"
"time"
"github.com/containerd/containerd/defaults"
"github.com/containerd/containerd/pkg/dialer"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/progress"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/identity"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
"google.golang.org/grpc"
"google.golang.org/grpc/backoff"
"google.golang.org/grpc/credentials/insecure"
)
func NewClient(ctx context.Context, addr string) (*Client, error) {
backoffConfig := backoff.DefaultConfig
backoffConfig.MaxDelay = 3 * time.Second
connParams := grpc.ConnectParams{
Backoff: backoffConfig,
}
gopts := []grpc.DialOption{
grpc.WithBlock(),
grpc.WithTransportCredentials(insecure.NewCredentials()),
grpc.WithConnectParams(connParams),
grpc.WithContextDialer(dialer.ContextDialer),
grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(defaults.DefaultMaxRecvMsgSize)),
grpc.WithDefaultCallOptions(grpc.MaxCallSendMsgSize(defaults.DefaultMaxSendMsgSize)),
grpc.WithUnaryInterceptor(grpcerrors.UnaryClientInterceptor),
grpc.WithStreamInterceptor(grpcerrors.StreamClientInterceptor),
}
conn, err := grpc.DialContext(ctx, dialer.DialAddress(addr), gopts...)
if err != nil {
return nil, err
}
return &Client{conn: conn}, nil
}
type Client struct {
conn *grpc.ClientConn
closeOnce sync.Once
}
func (c *Client) Close() (err error) {
c.closeOnce.Do(func() {
err = c.conn.Close()
})
return
}
func (c *Client) Version(ctx context.Context) (string, string, string, error) {
res, err := c.client().Info(ctx, &pb.InfoRequest{})
if err != nil {
return "", "", "", err
}
v := res.BuildxVersion
return v.Package, v.Version, v.Revision, nil
}
func (c *Client) List(ctx context.Context) (keys []string, retErr error) {
res, err := c.client().List(ctx, &pb.ListRequest{})
if err != nil {
return nil, err
}
return res.Keys, nil
}
func (c *Client) Disconnect(ctx context.Context, key string) error {
if key == "" {
return nil
}
_, err := c.client().Disconnect(ctx, &pb.DisconnectRequest{Ref: key})
return err
}
func (c *Client) ListProcesses(ctx context.Context, ref string) (infos []*pb.ProcessInfo, retErr error) {
res, err := c.client().ListProcesses(ctx, &pb.ListProcessesRequest{Ref: ref})
if err != nil {
return nil, err
}
return res.Infos, nil
}
func (c *Client) DisconnectProcess(ctx context.Context, ref, pid string) error {
_, err := c.client().DisconnectProcess(ctx, &pb.DisconnectProcessRequest{Ref: ref, ProcessID: pid})
return err
}
func (c *Client) Invoke(ctx context.Context, ref string, pid string, invokeConfig pb.InvokeConfig, in io.ReadCloser, stdout io.WriteCloser, stderr io.WriteCloser) error {
if ref == "" || pid == "" {
return errors.New("build reference must be specified")
}
stream, err := c.client().Invoke(ctx)
if err != nil {
return err
}
return attachIO(ctx, stream, &pb.InitMessage{Ref: ref, ProcessID: pid, InvokeConfig: &invokeConfig}, ioAttachConfig{
stdin: in,
stdout: stdout,
stderr: stderr,
// TODO: Signal, Resize
})
}
func (c *Client) Inspect(ctx context.Context, ref string) (*pb.InspectResponse, error) {
return c.client().Inspect(ctx, &pb.InspectRequest{Ref: ref})
}
func (c *Client) Build(ctx context.Context, options pb.BuildOptions, in io.ReadCloser, progress progress.Writer) (string, *client.SolveResponse, error) {
ref := identity.NewID()
statusChan := make(chan *client.SolveStatus)
eg, egCtx := errgroup.WithContext(ctx)
var resp *client.SolveResponse
eg.Go(func() error {
defer close(statusChan)
var err error
resp, err = c.build(egCtx, ref, options, in, statusChan)
return err
})
eg.Go(func() error {
for s := range statusChan {
st := s
progress.Write(st)
}
return nil
})
return ref, resp, eg.Wait()
}
func (c *Client) build(ctx context.Context, ref string, options pb.BuildOptions, in io.ReadCloser, statusChan chan *client.SolveStatus) (*client.SolveResponse, error) {
eg, egCtx := errgroup.WithContext(ctx)
done := make(chan struct{})
var resp *client.SolveResponse
eg.Go(func() error {
defer close(done)
pbResp, err := c.client().Build(egCtx, &pb.BuildRequest{
Ref: ref,
Options: &options,
})
if err != nil {
return err
}
resp = &client.SolveResponse{
ExporterResponse: pbResp.ExporterResponse,
}
return nil
})
eg.Go(func() error {
stream, err := c.client().Status(egCtx, &pb.StatusRequest{
Ref: ref,
})
if err != nil {
return err
}
for {
resp, err := stream.Recv()
if err != nil {
if err == io.EOF {
return nil
}
return errors.Wrap(err, "failed to receive status")
}
statusChan <- pb.FromControlStatus(resp)
}
})
if in != nil {
eg.Go(func() error {
stream, err := c.client().Input(egCtx)
if err != nil {
return err
}
if err := stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Init{
Init: &pb.InputInitMessage{
Ref: ref,
},
},
}); err != nil {
return errors.Wrap(err, "failed to init input")
}
inReader, inWriter := io.Pipe()
eg2, _ := errgroup.WithContext(ctx)
eg2.Go(func() error {
<-done
return inWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(inWriter, in)
inWriter.Close()
}()
eg2.Go(func() error {
for {
buf := make([]byte, 32*1024)
n, err := inReader.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return stream.Send(&pb.InputMessage{
Input: &pb.InputMessage_Data{
Data: &pb.DataMessage{
EOF: true,
},
},
})
})
return eg2.Wait()
})
}
return resp, eg.Wait()
}
func (c *Client) client() pb.ControllerClient {
return pb.NewControllerClient(c.conn)
}

View File

@@ -0,0 +1,333 @@
//go:build linux
package remote
import (
"context"
"fmt"
"io"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"strconv"
"syscall"
"time"
"github.com/containerd/containerd/log"
"github.com/docker/buildx/build"
cbuild "github.com/docker/buildx/controller/build"
"github.com/docker/buildx/controller/control"
controllerapi "github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/util/confutil"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/docker/cli/cli/command"
"github.com/moby/buildkit/client"
"github.com/moby/buildkit/util/grpcerrors"
"github.com/pelletier/go-toml"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"google.golang.org/grpc"
)
const (
serveCommandName = "_INTERNAL_SERVE"
)
var (
defaultLogFilename = fmt.Sprintf("buildx.%s.log", version.Revision)
defaultSocketFilename = fmt.Sprintf("buildx.%s.sock", version.Revision)
defaultPIDFilename = fmt.Sprintf("buildx.%s.pid", version.Revision)
)
type serverConfig struct {
// Specify buildx server root
Root string `toml:"root"`
// LogLevel sets the logging level [trace, debug, info, warn, error, fatal, panic]
LogLevel string `toml:"log_level"`
// Specify file to output buildx server log
LogFile string `toml:"log_file"`
}
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
rootDir := opts.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
serverRoot := filepath.Join(rootDir, "shared")
// connect to buildx server if it is already running
ctx2, cancel := context.WithTimeout(ctx, 1*time.Second)
c, err := newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
if !errors.Is(err, context.DeadlineExceeded) {
return nil, errors.Wrap(err, "cannot connect to the buildx server")
}
} else {
return &buildxController{c, serverRoot}, nil
}
// start buildx server via subcommand
err = logger.Wrap("no buildx server found; launching...", func() error {
launchFlags := []string{}
if opts.ServerConfig != "" {
launchFlags = append(launchFlags, "--config", opts.ServerConfig)
}
logFile, err := getLogFilePath(dockerCli, opts.ServerConfig)
if err != nil {
return err
}
wait, err := launch(ctx, logFile, append([]string{serveCommandName}, launchFlags...)...)
if err != nil {
return err
}
go wait()
// wait for buildx server to be ready
ctx2, cancel = context.WithTimeout(ctx, 10*time.Second)
c, err = newBuildxClientAndCheck(ctx2, filepath.Join(serverRoot, defaultSocketFilename))
cancel()
if err != nil {
return errors.Wrap(err, "cannot connect to the buildx server")
}
return nil
})
if err != nil {
return nil, err
}
return &buildxController{c, serverRoot}, nil
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {
cmd.AddCommand(
serveCmd(dockerCli),
)
}
func serveCmd(dockerCli command.Cli) *cobra.Command {
var serverConfigPath string
cmd := &cobra.Command{
Use: fmt.Sprintf("%s [OPTIONS]", serveCommandName),
Hidden: true,
RunE: func(cmd *cobra.Command, args []string) error {
// Parse config
config, err := getConfig(dockerCli, serverConfigPath)
if err != nil {
return err
}
if config.LogLevel == "" {
logrus.SetLevel(logrus.InfoLevel)
} else {
lvl, err := logrus.ParseLevel(config.LogLevel)
if err != nil {
return errors.Wrap(err, "failed to prepare logger")
}
logrus.SetLevel(lvl)
}
logrus.SetFormatter(&logrus.JSONFormatter{
TimestampFormat: log.RFC3339NanoFixed,
})
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return err
}
pidF := filepath.Join(root, defaultPIDFilename)
if err := os.WriteFile(pidF, []byte(fmt.Sprintf("%d", os.Getpid())), 0600); err != nil {
return err
}
defer func() {
if err := os.Remove(pidF); err != nil {
logrus.Errorf("failed to clean up info file %q: %v", pidF, err)
}
}()
// prepare server
b := NewServer(func(ctx context.Context, options *controllerapi.BuildOptions, stdin io.Reader, progress progress.Writer) (*client.SolveResponse, *build.ResultHandle, error) {
return cbuild.RunBuild(ctx, dockerCli, *options, stdin, progress, true)
})
defer b.Close()
// serve server
addr := filepath.Join(root, defaultSocketFilename)
if err := os.Remove(addr); err != nil && !os.IsNotExist(err) { // avoid EADDRINUSE
return err
}
defer func() {
if err := os.Remove(addr); err != nil {
logrus.Errorf("failed to clean up socket %q: %v", addr, err)
}
}()
logrus.Infof("starting server at %q", addr)
l, err := net.Listen("unix", addr)
if err != nil {
return err
}
rpc := grpc.NewServer(
grpc.UnaryInterceptor(grpcerrors.UnaryServerInterceptor),
grpc.StreamInterceptor(grpcerrors.StreamServerInterceptor),
)
controllerapi.RegisterControllerServer(rpc, b)
doneCh := make(chan struct{})
errCh := make(chan error, 1)
go func() {
defer close(doneCh)
if err := rpc.Serve(l); err != nil {
errCh <- errors.Wrapf(err, "error on serving via socket %q", addr)
}
}()
var s os.Signal
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT)
signal.Notify(sigCh, syscall.SIGTERM)
select {
case err := <-errCh:
logrus.Errorf("got error %s, exiting", err)
return err
case s = <-sigCh:
logrus.Infof("got signal %s, exiting", s)
return nil
case <-doneCh:
logrus.Infof("rpc server done, exiting")
return nil
}
},
}
flags := cmd.Flags()
flags.StringVar(&serverConfigPath, "config", "", "Specify buildx server config file")
return cmd
}
func getLogFilePath(dockerCli command.Cli, configPath string) (string, error) {
config, err := getConfig(dockerCli, configPath)
if err != nil {
return "", err
}
if config.LogFile == "" {
root, err := prepareRootDir(dockerCli, config)
if err != nil {
return "", err
}
return filepath.Join(root, defaultLogFilename), nil
}
return config.LogFile, nil
}
func getConfig(dockerCli command.Cli, configPath string) (*serverConfig, error) {
var defaultConfigPath bool
if configPath == "" {
defaultRoot := rootDataDir(dockerCli)
configPath = filepath.Join(defaultRoot, "config.toml")
defaultConfigPath = true
}
var config serverConfig
tree, err := toml.LoadFile(configPath)
if err != nil && !(os.IsNotExist(err) && defaultConfigPath) {
return nil, errors.Wrapf(err, "failed to read config %q", configPath)
} else if err == nil {
if err := tree.Unmarshal(&config); err != nil {
return nil, errors.Wrapf(err, "failed to unmarshal config %q", configPath)
}
}
return &config, nil
}
func prepareRootDir(dockerCli command.Cli, config *serverConfig) (string, error) {
rootDir := config.Root
if rootDir == "" {
rootDir = rootDataDir(dockerCli)
}
if rootDir == "" {
return "", errors.New("buildx root dir must be determined")
}
if err := os.MkdirAll(rootDir, 0700); err != nil {
return "", err
}
serverRoot := filepath.Join(rootDir, "shared")
if err := os.MkdirAll(serverRoot, 0700); err != nil {
return "", err
}
return serverRoot, nil
}
func rootDataDir(dockerCli command.Cli) string {
return filepath.Join(confutil.ConfigDir(dockerCli), "controller")
}
func newBuildxClientAndCheck(ctx context.Context, addr string) (*Client, error) {
c, err := NewClient(ctx, addr)
if err != nil {
return nil, err
}
p, v, r, err := c.Version(ctx)
if err != nil {
return nil, err
}
logrus.Debugf("connected to server (\"%v %v %v\")", p, v, r)
if !(p == version.Package && v == version.Version && r == version.Revision) {
return nil, errors.Errorf("version mismatch (client: \"%v %v %v\", server: \"%v %v %v\")", version.Package, version.Version, version.Revision, p, v, r)
}
return c, nil
}
type buildxController struct {
*Client
serverRoot string
}
func (c *buildxController) Kill(ctx context.Context) error {
pidB, err := os.ReadFile(filepath.Join(c.serverRoot, defaultPIDFilename))
if err != nil {
return err
}
pid, err := strconv.ParseInt(string(pidB), 10, 64)
if err != nil {
return err
}
if pid <= 0 {
return errors.New("no PID is recorded for buildx server")
}
p, err := os.FindProcess(int(pid))
if err != nil {
return err
}
if err := p.Signal(syscall.SIGINT); err != nil {
return err
}
// TODO: Should we send SIGKILL if process doesn't finish?
return nil
}
func launch(ctx context.Context, logFile string, args ...string) (func() error, error) {
// set absolute path of binary, since we set the working directory to the root
pathname, err := os.Executable()
if err != nil {
return nil, err
}
bCmd := exec.CommandContext(ctx, pathname, args...)
if logFile != "" {
f, err := os.OpenFile(logFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
return nil, err
}
defer f.Close()
bCmd.Stdout = f
bCmd.Stderr = f
}
bCmd.Stdin = nil
bCmd.Dir = "/"
bCmd.SysProcAttr = &syscall.SysProcAttr{
Setsid: true,
}
if err := bCmd.Start(); err != nil {
return nil, err
}
return bCmd.Wait, nil
}

View File

@@ -0,0 +1,19 @@
//go:build !linux
package remote
import (
"context"
"github.com/docker/buildx/controller/control"
"github.com/docker/buildx/util/progress"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/spf13/cobra"
)
func NewRemoteBuildxController(ctx context.Context, dockerCli command.Cli, opts control.ControlOptions, logger progress.SubLogger) (control.BuildxController, error) {
return nil, errors.New("remote buildx unsupported")
}
func AddControllerCommands(cmd *cobra.Command, dockerCli command.Cli) {}

430
controller/remote/io.go Normal file
View File

@@ -0,0 +1,430 @@
package remote
import (
"context"
"io"
"syscall"
"time"
"github.com/docker/buildx/controller/pb"
"github.com/moby/sys/signal"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sync/errgroup"
)
type msgStream interface {
Send(*pb.Message) error
Recv() (*pb.Message, error)
}
type ioServerConfig struct {
stdin io.WriteCloser
stdout, stderr io.ReadCloser
// signalFn is a callback function called when a signal is reached to the client.
signalFn func(context.Context, syscall.Signal) error
// resizeFn is a callback function called when a resize event is reached to the client.
resizeFn func(context.Context, winSize) error
}
func serveIO(attachCtx context.Context, srv msgStream, initFn func(*pb.InitMessage) error, ioConfig *ioServerConfig) (err error) {
stdin, stdout, stderr := ioConfig.stdin, ioConfig.stdout, ioConfig.stderr
stream := &debugStream{srv, "server=" + time.Now().String()}
eg, ctx := errgroup.WithContext(attachCtx)
done := make(chan struct{})
msg, err := receive(ctx, stream)
if err != nil {
return err
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInput())
}
ref := init.Ref
if ref == "" {
return errors.New("no ref is provided")
}
if err := initFn(init); err != nil {
return errors.Wrap(err, "failed to initialize IO server")
}
if stdout != nil {
stdoutReader, stdoutWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdoutWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdoutWriter, stdout)
stdoutWriter.Close()
}()
eg.Go(func() error {
defer stdoutReader.Close()
return copyToStream(1, stream, stdoutReader)
})
}
if stderr != nil {
stderrReader, stderrWriter := io.Pipe()
eg.Go(func() error {
<-done
return stderrWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stderrWriter, stderr)
stderrWriter.Close()
}()
eg.Go(func() error {
defer stderrReader.Close()
return copyToStream(2, stream, stderrReader)
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if file.Fd != 0 {
return errors.Errorf("unexpected fd: %v", file.Fd)
}
if stdin == nil {
continue // no stdin destination is specified so ignore the data
}
if len(file.Data) > 0 {
_, err := stdin.Write(file.Data)
if err != nil {
return err
}
}
if file.EOF {
stdin.Close()
}
} else if resize := msg.GetResize(); resize != nil {
if ioConfig.resizeFn != nil {
ioConfig.resizeFn(ctx, winSize{
cols: resize.Cols,
rows: resize.Rows,
})
}
} else if sig := msg.GetSignal(); sig != nil {
if ioConfig.signalFn != nil {
syscallSignal, ok := signal.SignalMap[sig.Name]
if !ok {
continue
}
ioConfig.signalFn(ctx, syscallSignal)
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
type ioAttachConfig struct {
stdin io.ReadCloser
stdout, stderr io.WriteCloser
signal <-chan syscall.Signal
resize <-chan winSize
}
type winSize struct {
rows uint32
cols uint32
}
func attachIO(ctx context.Context, stream msgStream, initMessage *pb.InitMessage, cfg ioAttachConfig) (retErr error) {
eg, ctx := errgroup.WithContext(ctx)
done := make(chan struct{})
if err := stream.Send(&pb.Message{
Input: &pb.Message_Init{
Init: initMessage,
},
}); err != nil {
return errors.Wrap(err, "failed to init")
}
if cfg.stdin != nil {
stdinReader, stdinWriter := io.Pipe()
eg.Go(func() error {
<-done
return stdinWriter.Close()
})
go func() {
// do not wait for read completion but return here and let the caller send EOF
// this allows us to return on ctx.Done() without being blocked by this reader.
io.Copy(stdinWriter, cfg.stdin)
stdinWriter.Close()
}()
eg.Go(func() error {
defer stdinReader.Close()
return copyToStream(0, stream, stdinReader)
})
}
if cfg.signal != nil {
eg.Go(func() error {
for {
var sig syscall.Signal
select {
case sig = <-cfg.signal:
case <-done:
return nil
case <-ctx.Done():
return nil
}
name := sigToName[sig]
if name == "" {
continue
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Signal{
Signal: &pb.SignalMessage{
Name: name,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send signal")
}
}
})
}
if cfg.resize != nil {
eg.Go(func() error {
for {
var win winSize
select {
case win = <-cfg.resize:
case <-done:
return nil
case <-ctx.Done():
return nil
}
if err := stream.Send(&pb.Message{
Input: &pb.Message_Resize{
Resize: &pb.ResizeMessage{
Rows: win.rows,
Cols: win.cols,
},
},
}); err != nil {
return errors.Wrap(err, "failed to send resize")
}
}
})
}
msgCh := make(chan *pb.Message)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := receive(ctx, stream)
if err != nil {
return err
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() error {
eofs := make(map[uint32]struct{})
defer close(done)
for {
var msg *pb.Message
select {
case msg = <-msgCh:
case <-ctx.Done():
return nil
}
if msg == nil {
return nil
}
if file := msg.GetFile(); file != nil {
if _, ok := eofs[file.Fd]; ok {
continue
}
var out io.WriteCloser
switch file.Fd {
case 1:
out = cfg.stdout
case 2:
out = cfg.stderr
default:
return errors.Errorf("unsupported fd %d", file.Fd)
}
if out == nil {
logrus.Warnf("attachIO: no writer for fd %d", file.Fd)
continue
}
if len(file.Data) > 0 {
if _, err := out.Write(file.Data); err != nil {
return err
}
}
if file.EOF {
eofs[file.Fd] = struct{}{}
}
} else {
return errors.Errorf("unexpected message: %T", msg.GetInput())
}
}
})
return eg.Wait()
}
func receive(ctx context.Context, stream msgStream) (*pb.Message, error) {
msgCh := make(chan *pb.Message)
errCh := make(chan error)
go func() {
msg, err := stream.Recv()
if err != nil {
if errors.Is(err, io.EOF) {
return
}
errCh <- err
return
}
msgCh <- msg
}()
select {
case msg := <-msgCh:
return msg, nil
case err := <-errCh:
return nil, err
case <-ctx.Done():
return nil, ctx.Err()
}
}
func copyToStream(fd uint32, snd msgStream, r io.Reader) error {
for {
buf := make([]byte, 32*1024)
n, err := r.Read(buf)
if err != nil {
if err == io.EOF {
break // break loop and send EOF
}
return err
} else if n > 0 {
if snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
Data: buf[:n],
},
},
}); err != nil {
return err
}
}
}
return snd.Send(&pb.Message{
Input: &pb.Message_File{
File: &pb.FdMessage{
Fd: fd,
EOF: true,
},
},
})
}
var sigToName = map[syscall.Signal]string{}
func init() {
for name, value := range signal.SignalMap {
sigToName[value] = name
}
}
type debugStream struct {
msgStream
prefix string
}
func (s *debugStream) Send(msg *pb.Message) error {
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|---> File Message (sender:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|---> Resize Message (sender:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|---> Signal Message (sender:%v): %s", s.prefix, m.Signal.Name)
}
return s.msgStream.Send(msg)
}
func (s *debugStream) Recv() (*pb.Message, error) {
msg, err := s.msgStream.Recv()
if err != nil {
return nil, err
}
switch m := msg.GetInput().(type) {
case *pb.Message_File:
if m.File.EOF {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, EOF", s.prefix, m.File.Fd)
} else {
logrus.Debugf("|<--- File Message (receiver:%v) fd=%d, %d bytes", s.prefix, m.File.Fd, len(m.File.Data))
}
case *pb.Message_Resize:
logrus.Debugf("|<--- Resize Message (receiver:%v): %+v", s.prefix, m.Resize)
case *pb.Message_Signal:
logrus.Debugf("|<--- Signal Message (receiver:%v): %s", s.prefix, m.Signal.Name)
}
return msg, nil
}

441
controller/remote/server.go Normal file
View File

@@ -0,0 +1,441 @@
package remote
import (
"context"
"io"
"sync"
"sync/atomic"
"time"
"github.com/docker/buildx/build"
controllererrors "github.com/docker/buildx/controller/errdefs"
"github.com/docker/buildx/controller/pb"
"github.com/docker/buildx/controller/processes"
"github.com/docker/buildx/util/ioset"
"github.com/docker/buildx/util/progress"
"github.com/docker/buildx/version"
"github.com/moby/buildkit/client"
"github.com/pkg/errors"
"golang.org/x/sync/errgroup"
)
type BuildFunc func(ctx context.Context, options *pb.BuildOptions, stdin io.Reader, progress progress.Writer) (resp *client.SolveResponse, res *build.ResultHandle, err error)
func NewServer(buildFunc BuildFunc) *Server {
return &Server{
buildFunc: buildFunc,
}
}
type Server struct {
buildFunc BuildFunc
session map[string]*session
sessionMu sync.Mutex
}
type session struct {
buildOnGoing atomic.Bool
statusChan chan *pb.StatusResponse
cancelBuild func()
buildOptions *pb.BuildOptions
inputPipe *io.PipeWriter
result *build.ResultHandle
processes *processes.Manager
}
func (s *session) cancelRunningProcesses() {
s.processes.CancelRunningProcesses()
}
func (m *Server) ListProcesses(ctx context.Context, req *pb.ListProcessesRequest) (res *pb.ListProcessesResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
res = new(pb.ListProcessesResponse)
for _, p := range s.processes.ListProcesses() {
res.Infos = append(res.Infos, p)
}
return res, nil
}
func (m *Server) DisconnectProcess(ctx context.Context, req *pb.DisconnectProcessRequest) (res *pb.DisconnectProcessResponse, err error) {
m.sessionMu.Lock()
defer m.sessionMu.Unlock()
s, ok := m.session[req.Ref]
if !ok {
return nil, errors.Errorf("unknown ref %q", req.Ref)
}
return res, s.processes.DeleteProcess(req.ProcessID)
}
func (m *Server) Info(ctx context.Context, req *pb.InfoRequest) (res *pb.InfoResponse, err error) {
return &pb.InfoResponse{
BuildxVersion: &pb.BuildxVersion{
Package: version.Package,
Version: version.Version,
Revision: version.Revision,
},
}, nil
}
func (m *Server) List(ctx context.Context, req *pb.ListRequest) (res *pb.ListResponse, err error) {
keys := make(map[string]struct{})
m.sessionMu.Lock()
for k := range m.session {
keys[k] = struct{}{}
}
m.sessionMu.Unlock()
var keysL []string
for k := range keys {
keysL = append(keysL, k)
}
return &pb.ListResponse{
Keys: keysL,
}, nil
}
func (m *Server) Disconnect(ctx context.Context, req *pb.DisconnectRequest) (res *pb.DisconnectResponse, err error) {
key := req.Ref
if key == "" {
return nil, errors.New("disconnect: empty key")
}
m.sessionMu.Lock()
if s, ok := m.session[key]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
if s.result != nil {
s.result.Done()
}
}
delete(m.session, key)
m.sessionMu.Unlock()
return &pb.DisconnectResponse{}, nil
}
func (m *Server) Close() error {
m.sessionMu.Lock()
for k := range m.session {
if s, ok := m.session[k]; ok {
if s.cancelBuild != nil {
s.cancelBuild()
}
s.cancelRunningProcesses()
}
}
m.sessionMu.Unlock()
return nil
}
func (m *Server) Inspect(ctx context.Context, req *pb.InspectRequest) (*pb.InspectResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("inspect: empty key")
}
var bo *pb.BuildOptions
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
bo = s.buildOptions
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("inspect: unknown key %v", ref)
}
m.sessionMu.Unlock()
return &pb.InspectResponse{Options: bo}, nil
}
func (m *Server) Build(ctx context.Context, req *pb.BuildRequest) (*pb.BuildResponse, error) {
ref := req.Ref
if ref == "" {
return nil, errors.New("build: empty key")
}
// Prepare status channel and session
m.sessionMu.Lock()
if m.session == nil {
m.session = make(map[string]*session)
}
s, ok := m.session[ref]
if ok {
if !s.buildOnGoing.CompareAndSwap(false, true) {
m.sessionMu.Unlock()
return &pb.BuildResponse{}, errors.New("build ongoing")
}
s.cancelRunningProcesses()
s.result = nil
} else {
s = &session{}
s.buildOnGoing.Store(true)
}
s.processes = processes.NewManager()
statusChan := make(chan *pb.StatusResponse)
s.statusChan = statusChan
inR, inW := io.Pipe()
defer inR.Close()
s.inputPipe = inW
m.session[ref] = s
m.sessionMu.Unlock()
defer func() {
close(statusChan)
m.sessionMu.Lock()
s, ok := m.session[ref]
if ok {
s.statusChan = nil
s.buildOnGoing.Store(false)
}
m.sessionMu.Unlock()
}()
pw := pb.NewProgressWriter(statusChan)
// Build the specified request
ctx, cancel := context.WithCancel(ctx)
defer cancel()
resp, res, buildErr := m.buildFunc(ctx, req.Options, inR, pw)
m.sessionMu.Lock()
if s, ok := m.session[ref]; ok {
// NOTE: buildFunc can return *build.ResultHandle even on error (e.g. when it's implemented using (github.com/docker/buildx/controller/build).RunBuild).
if res != nil {
s.result = res
s.cancelBuild = cancel
s.buildOptions = req.Options
m.session[ref] = s
if buildErr != nil {
buildErr = controllererrors.WrapBuild(buildErr, ref)
}
}
} else {
m.sessionMu.Unlock()
return nil, errors.Errorf("build: unknown key %v", ref)
}
m.sessionMu.Unlock()
if buildErr != nil {
return nil, buildErr
}
if resp == nil {
resp = &client.SolveResponse{}
}
return &pb.BuildResponse{
ExporterResponse: resp.ExporterResponse,
}, nil
}
func (m *Server) Status(req *pb.StatusRequest, stream pb.Controller_StatusServer) error {
ref := req.Ref
if ref == "" {
return errors.New("status: empty key")
}
// Wait and get status channel prepared by Build()
var statusChan <-chan *pb.StatusResponse
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].statusChan == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
statusChan = m.session[ref].statusChan
m.sessionMu.Unlock()
break
}
// forward status
for ss := range statusChan {
if ss == nil {
break
}
if err := stream.Send(ss); err != nil {
return err
}
}
return nil
}
func (m *Server) Input(stream pb.Controller_InputServer) (err error) {
// Get the target ref from init message
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
init := msg.GetInit()
if init == nil {
return errors.Errorf("unexpected message: %T; wanted init", msg.GetInit())
}
ref := init.Ref
if ref == "" {
return errors.New("input: no ref is provided")
}
// Wait and get input stream pipe prepared by Build()
var inputPipeW *io.PipeWriter
for {
// TODO: timeout?
m.sessionMu.Lock()
if _, ok := m.session[ref]; !ok || m.session[ref].inputPipe == nil {
m.sessionMu.Unlock()
time.Sleep(time.Millisecond) // TODO: wait Build without busy loop and make it cancellable
continue
}
inputPipeW = m.session[ref].inputPipe
m.sessionMu.Unlock()
break
}
// Forward input stream
eg, ctx := errgroup.WithContext(context.TODO())
done := make(chan struct{})
msgCh := make(chan *pb.InputMessage)
eg.Go(func() error {
defer close(msgCh)
for {
msg, err := stream.Recv()
if err != nil {
if !errors.Is(err, io.EOF) {
return err
}
return nil
}
select {
case msgCh <- msg:
case <-done:
return nil
case <-ctx.Done():
return nil
}
}
})
eg.Go(func() (retErr error) {
defer close(done)
defer func() {
if retErr != nil {
inputPipeW.CloseWithError(retErr)
return
}
inputPipeW.Close()
}()
for {
var msg *pb.InputMessage
select {
case msg = <-msgCh:
case <-ctx.Done():
return errors.Wrap(ctx.Err(), "canceled")
}
if msg == nil {
return nil
}
if data := msg.GetData(); data != nil {
if len(data.Data) > 0 {
_, err := inputPipeW.Write(data.Data)
if err != nil {
return err
}
}
if data.EOF {
return nil
}
}
}
})
return eg.Wait()
}
func (m *Server) Invoke(srv pb.Controller_InvokeServer) error {
containerIn, containerOut := ioset.Pipe()
defer func() { containerOut.Close(); containerIn.Close() }()
initDoneCh := make(chan *processes.Process)
initErrCh := make(chan error)
eg, egCtx := errgroup.WithContext(context.TODO())
srvIOCtx, srvIOCancel := context.WithCancel(egCtx)
eg.Go(func() error {
defer srvIOCancel()
return serveIO(srvIOCtx, srv, func(initMessage *pb.InitMessage) (retErr error) {
defer func() {
if retErr != nil {
initErrCh <- retErr
}
}()
ref := initMessage.Ref
cfg := initMessage.InvokeConfig
m.sessionMu.Lock()
s, ok := m.session[ref]
if !ok {
m.sessionMu.Unlock()
return errors.Errorf("invoke: unknown key %v", ref)
}
m.sessionMu.Unlock()
pid := initMessage.ProcessID
if pid == "" {
return errors.Errorf("invoke: specify process ID")
}
proc, ok := s.processes.Get(pid)
if !ok {
// Start a new process.
if cfg == nil {
return errors.New("no container config is provided")
}
var err error
proc, err = s.processes.StartProcess(pid, s.result, cfg)
if err != nil {
return err
}
}
// Attach containerIn to this process
proc.ForwardIO(&containerIn, srvIOCancel)
initDoneCh <- proc
return nil
}, &ioServerConfig{
stdin: containerOut.Stdin,
stdout: containerOut.Stdout,
stderr: containerOut.Stderr,
// TODO: signal, resize
})
})
eg.Go(func() (rErr error) {
defer srvIOCancel()
// Wait for init done
var proc *processes.Process
select {
case p := <-initDoneCh:
proc = p
case err := <-initErrCh:
return err
case <-egCtx.Done():
return egCtx.Err()
}
// Wait for IO done
select {
case <-srvIOCtx.Done():
return srvIOCtx.Err()
case err := <-proc.Done():
return err
case <-egCtx.Done():
return egCtx.Err()
}
})
return eg.Wait()
}

184
docker-bake.hcl Normal file
View File

@@ -0,0 +1,184 @@
variable "GO_VERSION" {
default = "1.20"
}
variable "DOCS_FORMATS" {
default = "md"
}
variable "DESTDIR" {
default = "./bin"
}
# Special target: https://github.com/docker/metadata-action#bake-definition
target "meta-helper" {
tags = ["docker/buildx-bin:local"]
}
target "_common" {
args = {
GO_VERSION = GO_VERSION
BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1
}
}
group "default" {
targets = ["binaries"]
}
group "validate" {
targets = ["lint", "validate-vendor", "validate-docs"]
}
target "lint" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/lint.Dockerfile"
output = ["type=cacheonly"]
}
target "validate-vendor" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "validate-docs" {
inherits = ["_common"]
args = {
FORMATS = DOCS_FORMATS
BUILDX_EXPERIMENTAL = 1 // enables experimental cmds/flags for docs generation
}
dockerfile = "./hack/dockerfiles/docs.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "validate-authors" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/authors.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "validate-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "validate"
output = ["type=cacheonly"]
}
target "update-vendor" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
target = "update"
output = ["."]
}
target "update-docs" {
inherits = ["_common"]
args = {
FORMATS = DOCS_FORMATS
BUILDX_EXPERIMENTAL = 1 // enables experimental cmds/flags for docs generation
}
dockerfile = "./hack/dockerfiles/docs.Dockerfile"
target = "update"
output = ["./docs/reference"]
}
target "update-authors" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/authors.Dockerfile"
target = "update"
output = ["."]
}
target "update-generated-files" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/generated-files.Dockerfile"
target = "update"
output = ["."]
}
target "mod-outdated" {
inherits = ["_common"]
dockerfile = "./hack/dockerfiles/vendor.Dockerfile"
target = "outdated"
no-cache-filter = ["outdated"]
output = ["type=cacheonly"]
}
target "test" {
inherits = ["_common"]
target = "test-coverage"
output = ["${DESTDIR}/coverage"]
}
target "binaries" {
inherits = ["_common"]
target = "binaries"
output = ["${DESTDIR}/build"]
platforms = ["local"]
}
target "binaries-cross" {
inherits = ["binaries"]
platforms = [
"darwin/amd64",
"darwin/arm64",
"linux/amd64",
"linux/arm/v6",
"linux/arm/v7",
"linux/arm64",
"linux/ppc64le",
"linux/riscv64",
"linux/s390x",
"windows/amd64",
"windows/arm64"
]
}
target "release" {
inherits = ["binaries-cross"]
target = "release"
output = ["${DESTDIR}/release"]
}
target "image" {
inherits = ["meta-helper", "binaries"]
output = ["type=image"]
}
target "image-cross" {
inherits = ["meta-helper", "binaries-cross"]
output = ["type=image"]
}
target "image-local" {
inherits = ["image"]
output = ["type=docker"]
}
variable "HTTP_PROXY" {
default = ""
}
variable "HTTPS_PROXY" {
default = ""
}
variable "NO_PROXY" {
default = ""
}
target "integration-test-base" {
inherits = ["_common"]
args = {
HTTP_PROXY = HTTP_PROXY
HTTPS_PROXY = HTTPS_PROXY
NO_PROXY = NO_PROXY
}
target = "integration-test-base"
output = ["type=cacheonly"]
}
target "integration-test" {
inherits = ["integration-test-base"]
target = "integration-test"
}

952
docs/bake-reference.md Normal file
View File

@@ -0,0 +1,952 @@
# Bake file reference
The Bake file is a file for defining workflows that you run using `docker buildx bake`.
## File format
You can define your Bake file in the following file formats:
- HashiCorp Configuration Language (HCL)
- JSON
- YAML (Compose file)
By default, Bake uses the following lookup order to find the configuration file:
1. `docker-bake.override.hcl`
2. `docker-bake.hcl`
3. `docker-bake.override.json`
4. `docker-bake.json`
5. `docker-compose.yaml`
6. `docker-compose.yml`
Bake searches for the file in the current working directory.
You can specify the file location explicitly using the `--file` flag:
```console
$ docker buildx bake --file=../docker/bake.hcl --print
```
## Syntax
The Bake file supports the following property types:
- `target`: build targets
- `group`: collections of build targets
- `variable`: build arguments and variables
- `function`: custom Bake functions
You define properties as hierarchical blocks in the Bake file.
You can assign one or more attributes to a property.
The following snippet shows a JSON representation of a simple Bake file.
This Bake file defines three properties: a variable, a group, and a target.
```json
{
"variable": {
"TAG": {
"default": "latest"
}
},
"group": {
"default": {
"targets": ["webapp"]
}
},
"target": {
"webapp": {
"dockerfile": "Dockerfile",
"tags": ["docker.io/username/webapp:${TAG}"]
}
}
}
```
In the JSON representation of a Bake file, properties are objects,
and attributes are values assigned to those objects.
The following example shows the same Bake file in the HCL format:
```hcl
variable "TAG" {
default = "latest"
}
group "default" {
targets = ["webapp"]
}
target "webapp" {
dockerfile = "Dockerfile"
tags = ["docker.io/username/webapp:${TAG}"]
}
```
HCL is the preferred format for Bake files.
Aside from syntactic differences,
HCL lets you use features that the JSON and YAML formats don't support.
The examples in this document use the HCL format.
## Target
A target reflects a single `docker build` invocation.
Consider the following build command:
```console
$ docker build \
--file=Dockerfile.webapp \
--tag=docker.io/username/webapp:latest \
https://github.com/username/webapp
```
You can express this command in a Bake file as follows:
```hcl
target "webapp" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
context = "https://github.com/username/webapp"
}
```
The following table shows the complete list of attributes that you can assign to a target:
| Name | Type | Description |
| ----------------------------------------------- | ------- | -------------------------------------------------------------------- |
| [`args`](#targetargs) | Map | Build arguments |
| [`attest`](#targetattest) | List | Build attestations |
| [`cache-from`](#targetcache-from) | List | External cache sources |
| [`cache-to`](#targetcache-to) | List | External cache destinations |
| [`context`](#targetcontext) | String | Set of files located in the specified path or URL |
| [`contexts`](#targetcontexts) | Map | Additional build contexts |
| [`dockerfile-inline`](#targetdockerfile-inline) | String | Inline Dockerfile string |
| [`dockerfile`](#targetdockerfile) | String | Dockerfile location |
| [`inherits`](#targetinherits) | List | Inherit attributes from other targets |
| [`labels`](#targetlabels) | Map | Metadata for images |
| [`matrix`](#targetmatrix) | Map | Define a set of variables that forks a target into multiple targets. |
| [`name`](#targetname) | String | Override the target name when using a matrix. |
| [`no-cache-filter`](#targetno-cache-filter) | List | Disable build cache for specific stages |
| [`no-cache`](#targetno-cache) | Boolean | Disable build cache completely |
| [`output`](#targetoutput) | List | Output destinations |
| [`platforms`](#targetplatforms) | List | Target platforms |
| [`pull`](#targetpull) | Boolean | Always pull images |
| [`secret`](#targetsecret) | List | Secrets to expose to the build |
| [`ssh`](#targetssh) | List | SSH agent sockets or keys to expose to the build |
| [`tags`](#targettags) | List | Image names and tags |
| [`target`](#targettarget) | String | Target build stage |
### `target.args`
Use the `args` attribute to define build arguments for the target.
This has the same effect as passing a [`--build-arg`][build-arg] flag to the build command.
```hcl
target "default" {
args = {
VERSION = "0.0.0+unknown"
}
}
```
You can set `args` attributes to use `null` values.
Doing so forces the `target` to use the `ARG` value specified in the Dockerfile.
```hcl
variable "GO_VERSION" {
default = "1.20.3"
}
target "webapp" {
dockerfile = "webapp.Dockerfile"
tags = ["docker.io/username/webapp"]
}
target "db" {
args = {
GO_VERSION = null
}
dockerfile = "db.Dockerfile"
tags = ["docker.io/username/db"]
}
```
### `target.attest`
The `attest` attribute lets you apply [build attestations][attestations] to the target.
This attribute accepts the long-form CSV version of attestation parameters.
```hcl
target "default" {
attest = [
"type=provenance,mode=min",
"type=sbom"
]
}
```
### `target.cache-from`
Build cache sources.
The builder imports cache from the locations you specify.
It uses the [Buildx cache storage backends][cache-backends],
and it works the same way as the [`--cache-from`][cache-from] flag.
This takes a list value, so you can specify multiple cache sources.
```hcl
target "app" {
cache-from = [
"type=s3,region=eu-west-1,bucket=mybucket",
"user/repo:cache",
]
}
```
### `target.cache-to`
Build cache export destinations.
The builder exports its build cache to the locations you specify.
It uses the [Buildx cache storage backends][cache-backends],
and it works the same way as the [`--cache-to` flag][cache-to].
This takes a list value, so you can specify multiple cache export targets.
```hcl
target "app" {
cache-to = [
"type=s3,region=eu-west-1,bucket=mybucket",
"type=inline"
]
}
```
### `target.context`
Specifies the location of the build context to use for this target.
Accepts a URL or a directory path.
This is the same as the [build context][context] positional argument
that you pass to the build command.
```hcl
target "app" {
context = "./src/www"
}
```
This resolves to the current working directory (`"."`) by default.
```console
$ docker buildx bake --print -f - <<< 'target "default" {}'
[+] Building 0.0s (0/0)
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile"
}
}
}
```
### `target.contexts`
Additional build contexts.
This is the same as the [`--build-context` flag][build-context].
This attribute takes a map, where keys result in named contexts that you can
reference in your builds.
You can specify different types of contexts, such local directories, Git URLs,
and even other Bake targets. Bake automatically determines the type of
a context based on the pattern of the context value.
| Context type | Example |
| --------------- | ----------------------------------------- |
| Container image | `docker-image://alpine@sha256:0123456789` |
| Git URL | `https://github.com/user/proj.git` |
| HTTP URL | `https://example.com/files` |
| Local directory | `../path/to/src` |
| Bake target | `target:base` |
#### Pin an image version
```hcl
# docker-bake.hcl
target "app" {
contexts = {
alpine = "docker-image://alpine:3.13"
}
}
```
```Dockerfile
# Dockerfile
FROM alpine
RUN echo "Hello world"
```
#### Use a local directory
```hcl
# docker-bake.hcl
target "app" {
contexts = {
src = "../path/to/source"
}
}
```
```Dockerfile
# Dockerfile
FROM scratch AS src
FROM golang
COPY --from=src . .
```
#### Use another target as base
> **Note**
>
> You should prefer to use regular multi-stage builds over this option. You can
> Use this feature when you have multiple Dockerfiles that can't be easily
> merged into one.
```hcl
# docker-bake.hcl
target "base" {
dockerfile = "baseapp.Dockerfile"
}
target "app" {
contexts = {
baseapp = "target:base"
}
}
```
```Dockerfile
# Dockerfile
FROM baseapp
RUN echo "Hello world"
```
### `target.dockerfile-inline`
Uses the string value as an inline Dockerfile for the build target.
```hcl
target "default" {
dockerfile-inline = "FROM alpine\nENTRYPOINT [\"echo\", \"hello\"]"
}
```
The `dockerfile-inline` takes precedence over the `dockerfile` attribute.
If you specify both, Bake uses the inline version.
### `target.dockerfile`
Name of the Dockerfile to use for the build.
This is the same as the [`--file` flag][file] for the `docker build` command.
```hcl
target "default" {
dockerfile = "./src/www/Dockerfile"
}
```
Resolves to `"Dockerfile"` by default.
```console
$ docker buildx bake --print -f - <<< 'target "default" {}'
[+] Building 0.0s (0/0)
{
"target": {
"default": {
"context": ".",
"dockerfile": "Dockerfile"
}
}
}
```
### `target.inherits`
A target can inherit attributes from other targets.
Use `inherits` to reference from one target to another.
In the following example,
the `app-dev` target specifies an image name and tag.
The `app-release` target uses `inherits` to reuse the tag name.
```hcl
variable "TAG" {
default = "latest"
}
target "app-dev" {
tags = ["docker.io/username/myapp:${TAG}"]
}
target "app-release" {
inherits = ["app-dev"]
platforms = ["linux/amd64", "linux/arm64"]
}
```
The `inherits` attribute is a list,
meaning you can reuse attributes from multiple other targets.
In the following example, the `app-release` target reuses attributes
from both the `app-dev` and `_release` targets.
```hcl
target "app-dev" {
args = {
GO_VERSION = "1.20"
BUILDX_EXPERIMENTAL = 1
}
tags = ["docker.io/username/myapp"]
dockerfile = "app.Dockerfile"
labels = {
"org.opencontainers.image.source" = "https://github.com/username/myapp"
}
}
target "_release" {
args = {
BUILDKIT_CONTEXT_KEEP_GIT_DIR = 1
BUILDX_EXPERIMENTAL = 0
}
}
target "app-release" {
inherits = ["app-dev", "_release"]
platforms = ["linux/amd64", "linux/arm64"]
}
```
When inheriting attributes from multiple targets and there's a conflict,
the target that appears last in the `inherits` list takes precedence.
The previous example defines the `BUILDX_EXPERIMENTAL` argument twice for the `app-release` target.
It resolves to `0` because the `_release` target appears last in the inheritance chain:
```console
$ docker buildx bake --print app-release
[+] Building 0.0s (0/0)
{
"group": {
"default": {
"targets": [
"app-release"
]
}
},
"target": {
"app-release": {
"context": ".",
"dockerfile": "app.Dockerfile",
"args": {
"BUILDKIT_CONTEXT_KEEP_GIT_DIR": "1",
"BUILDX_EXPERIMENTAL": "0",
"GO_VERSION": "1.20"
},
"labels": {
"org.opencontainers.image.source": "https://github.com/username/myapp"
},
"tags": [
"docker.io/username/myapp"
],
"platforms": [
"linux/amd64",
"linux/arm64"
]
}
}
}
```
### `target.labels`
Assigns image labels to the build.
This is the same as the `--label` flag for `docker build`.
```hcl
target "default" {
labels = {
"org.opencontainers.image.source" = "https://github.com/username/myapp"
"com.docker.image.source.entrypoint" = "Dockerfile"
}
}
```
It's possible to use a `null` value for labels.
If you do, the builder uses the label value specified in the Dockerfile.
### `target.matrix`
A matrix strategy lets you fork a single target into multiple different
variants, based on parameters that you specify.
This works in a similar way to [Matrix strategies for GitHub Actions].
You can use this to reduce duplication in your bake definition.
The `matrix` attribute is a map of parameter names to lists of values.
Bake builds each possible combination of values as a separate target.
Each generated target **must** have a unique name.
To specify how target names should resolve, use the `name` attribute.
The following example resolves the `app` target to `app-foo` and `app-bar`.
It also uses the matrix value to define the [target build stage](#targettarget).
```hcl
target "app" {
name = "app-${tgt}"
matrix = {
tgt = ["foo", "bar"]
}
target = tgt
}
```
```console
$ docker buildx bake --print app
[+] Building 0.0s (0/0)
{
"group": {
"app": {
"targets": [
"app-foo",
"app-bar"
]
},
"default": {
"targets": [
"app"
]
}
},
"target": {
"app-bar": {
"context": ".",
"dockerfile": "Dockerfile",
"target": "bar"
},
"app-foo": {
"context": ".",
"dockerfile": "Dockerfile",
"target": "foo"
}
}
}
```
#### Multiple axes
You can specify multiple keys in your matrix to fork a target on multiple axes.
When using multiple matrix keys, Bake builds every possible variant.
The following example builds four targets:
- `app-foo-1-0`
- `app-foo-2-0`
- `app-bar-1-0`
- `app-bar-2-0`
```hcl
target "app" {
name = "app-${tgt}-${replace(version, ".", "-")}"
matrix = {
tgt = ["foo", "bar"]
version = ["1.0", "2.0"]
}
target = tgt
args = {
VERSION = version
}
}
```
#### Multiple values per matrix target
If you want to differentiate the matrix on more than just a single value,
you can use maps as matrix values. Bake creates a target for each map,
and you can access the nested values using dot notation.
The following example builds two targets:
- `app-foo-1-0`
- `app-bar-2-0`
```hcl
target "app" {
name = "app-${item.tgt}-${replace(item.version, ".", "-")}"
matrix = {
item = [
{
tgt = "foo"
version = "1.0"
},
{
tgt = "bar"
version = "2.0"
}
]
}
target = item.tgt
args = {
VERSION = item.version
}
}
```
### `target.name`
Specify name resolution for targets that use a matrix strategy.
The following example resolves the `app` target to `app-foo` and `app-bar`.
```hcl
target "app" {
name = "app-${tgt}"
matrix = {
tgt = ["foo", "bar"]
}
target = tgt
}
```
### `target.no-cache-filter`
Don't use build cache for the specified stages.
This is the same as the `--no-cache-filter` flag for `docker build`.
The following example avoids build cache for the `foo` build stage.
```hcl
target "default" {
no-cache-filter = ["foo"]
}
```
### `target.no-cache`
Don't use cache when building the image.
This is the same as the `--no-cache` flag for `docker build`.
```hcl
target "default" {
no-cache = 1
}
```
### `target.output`
Configuration for exporting the build output.
This is the same as the [`--output` flag][output].
The following example configures the target to use a cache-only output,
```hcl
target "default" {
output = ["type=cacheonly"]
}
```
### `target.platforms`
Set target platforms for the build target.
This is the same as the [`--platform` flag][platform].
The following example creates a multi-platform build for three architectures.
```hcl
target "default" {
platforms = ["linux/amd64", "linux/arm64", "linux/arm/v7"]
}
```
### `target.pull`
Configures whether the builder should attempt to pull images when building the target.
This is the same as the `--pull` flag for `docker build`.
The following example forces the builder to always pull all images referenced in the build target.
```hcl
target "default" {
pull = "always"
}
```
### `target.secret`
Defines secrets to expose to the build target.
This is the same as the [`--secret` flag][secret].
```hcl
variable "HOME" {
default = null
}
target "default" {
secret = [
"type=env,id=KUBECONFIG",
"type=file,id=aws,src=${HOME}/.aws/credentials"
]
}
```
This lets you [mount the secret][run_mount_secret] in your Dockerfile.
```dockerfile
RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
aws cloudfront create-invalidation ...
RUN --mount=type=secret,id=KUBECONFIG \
KUBECONFIG=$(cat /run/secrets/KUBECONFIG) helm upgrade --install
```
### `target.ssh`
Defines SSH agent sockets or keys to expose to the build.
This is the same as the [`--ssh` flag][ssh].
This can be useful if you need to access private repositories during a build.
```hcl
target "default" {
ssh = ["default"]
}
```
```dockerfile
FROM alpine
RUN --mount=type=ssh \
apk add git openssh-client \
&& install -m 0700 -d ~/.ssh \
&& ssh-keyscan github.com >> ~/.ssh/known_hosts \
&& git clone git@github.com:user/my-private-repo.git
```
### `target.tags`
Image names and tags to use for the build target.
This is the same as the [`--tag` flag][tag].
```hcl
target "default" {
tags = [
"org/repo:latest",
"myregistry.azurecr.io/team/image:v1"
]
}
```
### `target.target`
Set the target build stage to build.
This is the same as the [`--target` flag][target].
```hcl
target "default" {
target = "binaries"
}
```
## Group
Groups allow you to invoke multiple builds (targets) at once.
```hcl
group "default" {
targets = ["db", "webapp-dev"]
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
}
target "db" {
dockerfile = "Dockerfile.db"
tags = ["docker.io/username/db"]
}
```
Groups take precedence over targets, if both exist with the same name.
The following bake file builds the `default` group.
Bake ignores the `default` target.
```hcl
target "default" {
dockerfile-inline = "FROM ubuntu"
}
group "default" {
targets = ["alpine", "debian"]
}
target "alpine" {
dockerfile-inline = "FROM alpine"
}
target "debian" {
dockerfile-inline = "FROM debian"
}
```
## Variable
The HCL file format supports variable block definitions.
You can use variables as build arguments in your Dockerfile,
or interpolate them in attribute values in your Bake file.
```hcl
variable "TAG" {
default = "latest"
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:${TAG}"]
}
```
You can assign a default value for a variable in the Bake file,
or assign a `null` value to it. If you assign a `null` value,
Buildx uses the default value from the Dockerfile instead.
You can override variable defaults set in the Bake file using environment variables.
The following example sets the `TAG` variable to `dev`,
overriding the default `latest` value shown in the previous example.
```console
$ TAG=dev docker buildx bake webapp-dev
```
### Built-in variables
The following variables are built-ins that you can use with Bake without having
to define them.
| Variable | Description |
| --------------------- | ----------------------------------------------------------------------------------- |
| `BAKE_CMD_CONTEXT` | Holds the main context when building using a remote Bake file. |
| `BAKE_LOCAL_PLATFORM` | Returns the current platforms default platform specification (e.g. `linux/amd64`). |
### Use environment variable as default
You can set a Bake variable to use the value of an environment variable as a default value:
```hcl
variable "HOME" {
default = "$HOME"
}
```
### Interpolate variables into attributes
To interpolate a variable into an attribute string value,
you must use curly brackets.
The following doesn't work:
```hcl
variable "HOME" {
default = "$HOME"
}
target "default" {
ssh = ["default=$HOME/.ssh/id_rsa"]
}
```
Wrap the variable in curly brackets where you want to insert it:
```diff
variable "HOME" {
default = "$HOME"
}
target "default" {
- ssh = ["default=$HOME/.ssh/id_rsa"]
+ ssh = ["default=${HOME}/.ssh/id_rsa"]
}
```
Before you can interpolate a variable into an attribute,
first you must declare it in the bake file,
as demonstrated in the following example.
```console
$ cat docker-bake.hcl
target "default" {
dockerfile-inline = "FROM ${BASE_IMAGE}"
}
$ docker buildx bake
[+] Building 0.0s (0/0)
docker-bake.hcl:2
--------------------
1 | target "default" {
2 | >>> dockerfile-inline = "FROM ${BASE_IMAGE}"
3 | }
4 |
--------------------
ERROR: docker-bake.hcl:2,31-41: Unknown variable; There is no variable named "BASE_IMAGE"., and 1 other diagnostic(s)
$ cat >> docker-bake.hcl
variable "BASE_IMAGE" {
default = "alpine"
}
$ docker buildx bake
[+] Building 0.6s (5/5) FINISHED
```
## Function
A [set of general-purpose functions][bake_stdlib]
provided by [go-cty][go-cty]
are available for use in HCL files:
```hcl
# docker-bake.hcl
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${add(123, 1)}"
}
}
```
In addition, [user defined functions][userfunc]
are also supported:
```hcl
# docker-bake.hcl
function "increment" {
params = [number]
result = number + 1
}
target "webapp-dev" {
dockerfile = "Dockerfile.webapp"
tags = ["docker.io/username/webapp:latest"]
args = {
buildno = "${increment(123)}"
}
}
```
> **Note**
>
> See [User defined HCL functions][hcl-funcs] page for more details.
<!-- external links -->
[attestations]: https://docs.docker.com/build/attestations/
[bake_stdlib]: https://github.com/docker/buildx/blob/master/bake/hclparser/stdlib.go
[build-arg]: https://docs.docker.com/engine/reference/commandline/build/#build-arg
[build-context]: https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
[cache-backends]: https://docs.docker.com/build/cache/backends/
[cache-from]: https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-from
[cache-to]: https://docs.docker.com/engine/reference/commandline/buildx_build/#cache-to
[context]: https://docs.docker.com/engine/reference/commandline/buildx_build/#build-context
[file]: https://docs.docker.com/engine/reference/commandline/build/#file
[go-cty]: https://github.com/zclconf/go-cty/tree/main/cty/function/stdlib
[hcl-funcs]: https://docs.docker.com/build/bake/hcl-funcs/
[output]: https://docs.docker.com/engine/reference/commandline/buildx_build/#output
[platform]: https://docs.docker.com/engine/reference/commandline/buildx_build/#platform
[run_mount_secret]: https://docs.docker.com/engine/reference/builder/#run---mounttypesecret
[secret]: https://docs.docker.com/engine/reference/commandline/buildx_build/#secret
[ssh]: https://docs.docker.com/engine/reference/commandline/buildx_build/#ssh
[tag]: https://docs.docker.com/engine/reference/commandline/build/#tag
[target]: https://docs.docker.com/engine/reference/commandline/build/#target
[userfunc]: https://github.com/hashicorp/hcl/tree/main/ext/userfunc

90
docs/generate.go Normal file
View File

@@ -0,0 +1,90 @@
package main
import (
"log"
"os"
"github.com/docker/buildx/commands"
clidocstool "github.com/docker/cli-docs-tool"
"github.com/docker/cli/cli/command"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
// import drivers otherwise factories are empty
// for --driver output flag usage
_ "github.com/docker/buildx/driver/docker"
_ "github.com/docker/buildx/driver/docker-container"
_ "github.com/docker/buildx/driver/kubernetes"
_ "github.com/docker/buildx/driver/remote"
)
const defaultSourcePath = "docs/reference/"
type options struct {
source string
formats []string
}
func gen(opts *options) error {
log.SetFlags(0)
dockerCLI, err := command.NewDockerCli()
if err != nil {
return err
}
cmd := &cobra.Command{
Use: "docker [OPTIONS] COMMAND [ARG...]",
Short: "The base command for the Docker CLI.",
DisableAutoGenTag: true,
}
cmd.AddCommand(commands.NewRootCmd("buildx", true, dockerCLI))
c, err := clidocstool.New(clidocstool.Options{
Root: cmd,
SourceDir: opts.source,
Plugin: true,
})
if err != nil {
return err
}
for _, format := range opts.formats {
switch format {
case "md":
if err = c.GenMarkdownTree(cmd); err != nil {
return err
}
case "yaml":
if err = c.GenYamlTree(cmd); err != nil {
return err
}
default:
return errors.Errorf("unknown format %q", format)
}
}
return nil
}
func run() error {
opts := &options{}
flags := pflag.NewFlagSet(os.Args[0], pflag.ContinueOnError)
flags.StringVar(&opts.source, "source", defaultSourcePath, "Docs source folder")
flags.StringSliceVar(&opts.formats, "formats", []string{}, "Format (md, yaml)")
if err := flags.Parse(os.Args[1:]); err != nil {
return err
}
if len(opts.formats) == 0 {
return errors.New("Docs format required")
}
return gen(opts)
}
func main() {
if err := run(); err != nil {
log.Printf("ERROR: %+v", err)
os.Exit(1)
}
}

3
docs/guides/cicd.md Normal file
View File

@@ -0,0 +1,3 @@
# CI/CD
This page has moved to [Docker Docs website](https://docs.docker.com/build/ci/)

View File

@@ -0,0 +1,3 @@
# CNI networking
This page has moved to [Docker Docs website](https://docs.docker.com/build/buildkit/configure/#cni-networking)

View File

@@ -0,0 +1,3 @@
# Color output controls
This page has moved to [Docker Docs website](https://docs.docker.com/build/building/env-vars/#buildkit_colors)

Some files were not shown because too many files have changed in this diff Show More