62 Commits

Author SHA1 Message Date
Tonis Tiigi
adbcc2225e
build: allow builds from stdin for multi-node builders
When building from same stream all nodes need to read
data from the same stream. In order to achive that there
is a new SyncMultiReader wrapper that sends the stream
concurrently to all readers. Readers must read at similar
speed or pauses will happen while they wait for each other.

Dockerfiles were already written to disk before sent. Now
the file written by first node is reused for others.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-08-14 13:13:18 +03:00
CrazyMax
d03c13b947
chore: rename PrintFunc to CallFunc
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-08-13 14:13:32 +02:00
Tõnis Tiigi
3d542f3d31
Merge pull request #2601 from tonistiigi/init-fixes
Improvements based on inittrace
2024-07-24 10:52:25 -07:00
Tonis Tiigi
96509847b9
remote: avoid signal names map on init
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-07-15 16:40:06 -07:00
Bryce Lampe
6c929a45c7 build: don't force default configuration
Signed-off-by: Bryce Lampe <bryce@pulumi.com>
2024-07-15 12:32:23 -07:00
Tõnis Tiigi
59a0ffcf83
Merge pull request #2546 from treuherz/multinode-annotations
Pass in index annotations from builds on multiple nodes
2024-06-28 16:46:20 -07:00
Eli Treuherz
6f45b0ea06 Get annotations from exports
Signed-off-by: Eli Treuherz <et@arenko.group>
2024-06-27 13:26:07 +01:00
Eli Treuherz
3971361ed2 Pass in index annotations from builds on multiple nodes
Fixes #2540

Signed-off-by: Eli Treuherz <et@arenko.group>
2024-06-27 13:26:07 +01:00
Sebastiaan van Stijn
9358f84668
vendor: buildkit, docker/docker and docker/cli v27.0.1
diffs:

- https://github.com/docker/cli/compare/v26.1.4..v27.0.1
- https://github.com/docker/docker/compare/v26.1.4..v27.0.1
- https://github.com/moby/buildkit/compare/v0.14.1...aaaf86e5470bffbb395f5c15ad4a1c152642ea30

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-06-26 15:31:47 +02:00
CrazyMax
d155747029
build: read provenance response mode in commands pkg
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-06-24 14:32:26 +02:00
Sebastiaan van Stijn
b948b07e2d
remove uses of github.com/docker/docker/builder/remotecontext package
This package is part of the classic builder, and was currently only used
for the IsURL utility, which is a very rudimentary check for a string having
a "https://" or "http://" scheme.

This patch copies the code as non-exported functions where they're used to
remove the dependency.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-05-26 11:06:02 +02:00
Tonis Tiigi
b30566438b
lint: gopls fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-04-24 17:58:17 -07:00
Tonis Tiigi
6112c41637
lint: nilness fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-04-24 17:19:32 -07:00
Tonis Tiigi
46c44c58ae
build: support statuscode response for print requests
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-04-15 10:38:54 -07:00
CrazyMax
2e2f9f571f
build: set record provenance in response
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-04-11 10:11:27 +02:00
CrazyMax
a03263acf8
build: handle push/load shorthands for multi exporters
Signed-off-by: CrazyMax <1951866+crazy-max@users.noreply.github.com>
2024-03-05 17:05:59 +01:00
Tonis Tiigi
fcbe2803c8
build: allow multiple exports if supported by buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2024-02-28 13:16:15 -08:00
CrazyMax
f3871b158f
build: make annotations work with push flag
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-26 12:01:10 +02:00
Tõnis Tiigi
6f71ea8904
Merge pull request #2086 from ktock/debug-errormessage
debug: Following-up patches for error messages
2023-10-25 09:27:44 -07:00
Justin Chadwell
adc839aa40 vendor: update buildkit to master@d5c1d785b042
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-10-23 14:48:50 +01:00
Kohei Tokunaga
14834e6085
debug: print error message on monitor instead of processes pkg
This can catch and print more errors on monitor side not only process-related
errors.

Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-20 14:09:32 +09:00
CrazyMax
503d483731
fix lint issues after update
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-19 12:31:24 +02:00
Tõnis Tiigi
ac93a7fbfb
Merge pull request #2006 from ktock/debug-command
debug: Add `buildx debug` command
2023-10-17 18:20:33 -07:00
CrazyMax
fd251d2a7b
builder: use variadic options in LoadNodes func
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-10-17 00:45:50 +02:00
Kohei Tokunaga
6db8569f09
process: Do not print error log when process is canceled
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-10-13 12:09:30 +09:00
CrazyMax
6c77b76b7b
Merge pull request #2038 from crazy-max/localstate-group
build: support local state group
2023-09-30 06:01:14 +02:00
CrazyMax
3dcb03452c
build: support local state group
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:08:06 +02:00
CrazyMax
9b7d30c9a0
build: opt to set build ref
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-09-30 01:03:00 +02:00
Justin Chadwell
e2ebab5f26 vendor: update buildkit to master@cbfd4023383d
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-09-27 11:06:18 +01:00
guoguangwu
a71a24c0f4 chore: slice loop replace
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-08-31 12:24:02 +08:00
Kohei Tokunaga
3eb490153d
remote controller: Fix entrypoint interaction bugs
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-07-27 09:12:52 +09:00
Jhan S. Álvarez
e65f6b8c8b controller: include CgroupParent in build.Options
Signed-off-by: Jhan S. Álvarez <alvarezpcuser@gmail.com>
2023-06-26 23:18:53 -05:00
CrazyMax
0a2f35970c
build: display build details link
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-06 18:54:31 +02:00
Justin Chadwell
cd1648192e build: rename ResultContext to ResultHandle
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-31 09:50:46 +01:00
Justin Chadwell
9ee19520dd controller: move path resolution into controller package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-05-11 13:56:05 +01:00
Brian Goff
98efe7af10 Support for passing through raw buildkit policies
This adds an env var which can be used to pass in a path to a file to
read a buildkit source poliy from.

This is applied to any build is executed with the env set.
It is also applied to bakes (which are calling build behind the scenes).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-03 23:07:44 +00:00
Justin Chadwell
ba92989a94 controller: print result outside of controller
This will allow result printing to work with the remote controller
(though this currently causes a panic, to be fixed in a follow-up).

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2bf996d9ad controller: don't write metadata file in controller
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-26 16:34:36 +01:00
Justin Chadwell
2ab8749052 controller: replace logrus status messages with progress messages
logrus info messages aren't particularly in-theme with the rest of the
progress output (and are also frustratingly racy). The progress output
is a lot neater, so we refactor it into that.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 12:33:04 +01:00
Justin Chadwell
e826141af4 controller: refactor progress api
Refactor the progress printer creation to the caller-side of the
controller api. Then, instead of passing around status channels (and
progressMode strings), we can simply pass around the higher level
interface progress.Writer.

This has a couple of benefits:
- A simplified interface to the controller
- Allows us to correctly extract warnings out of the controller, so that
  they can be displayed correctly from the client side.

Some extra work is required to make sure that we can pass a
progress.Printer into the debug monitor. If we want to keep it
persistent, then we need a way to temporarily suspend output from it,
otherwise it will continue printing as the monitor is prompting for
input from the user, and forwarding output from debug containers.

To handle this, we add two methods to the printer, `Pause` and
`Unpause`. `Pause` acts similarly to `Wait`, closing the printer, and
cleanly shutting down the display - however, the printer does not
terminate, and can later be resumed by a call to `Unpause`. This
provides a neater interface to the caller, instead of needing to
continually reconstruct printers for every single time we want to
produce progress output.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 11:17:43 +01:00
Justin Chadwell
0e9804901b controller: refactor status message conversion to pb package
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-21 10:59:16 +01:00
Justin Chadwell
a7e471b7b3 controller: only capture the result when invoked through a controller
This ensures that the code used to capture and evaluated a result is
only executed when built through the controller. Otherwise, no build
result should be recorded.

This ensures that new code added to capture and store the build result
for debugging isn't used when BUILDX_EXPERIMENTAL is not set.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-04-18 16:35:42 +01:00
Kohei Tokunaga
06399630a2
remove ResultContextError
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:18 +09:00
Kohei Tokunaga
ce48b1ae84
Enable to restore build options from the server
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:38:05 +09:00
Kohei Tokunaga
b3340cc7ba
Enable to create container from error
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:36:33 +09:00
Kohei Tokunaga
1303715aba
Allow passing ResultContext from server to the client through grpcerror
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-15 15:11:32 +09:00
Kohei Tokunaga
8ba8659496
controller: Extract nested CommonOptions on controller API
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-04-03 21:35:04 +09:00
Kohei Tokunaga
e8f55a3cf7 monitor: Enable to exec into the container
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-02 19:30:29 +09:00
Kohei Tokunaga
b0728c96d3 controller: use os.Executable() for getting the exceutable path
Signed-off-by: Kohei Tokunaga <ktokunaga.mail@gmail.com>
2023-03-01 15:49:07 +09:00
Justin Chadwell
c2e11196dd controller: handle attestation options across api
We can perform all attestation processing, handling how the sbom and
provenance arguments interact on the client, while applying defaults on
the server.

Additionally, this allows us to start pulling fields out of CommonOpts.

Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-02-23 15:46:37 +00:00