Compare commits

...

26 Commits

Author SHA1 Message Date
Tõnis Tiigi
c513d34049 Merge pull request #1664 from crazy-max/v0.10_backport_stripcreds
[v0.10 backport] build: strip credentials from remote url on collecting Git provenance info
2023-03-06 16:25:59 +00:00
CrazyMax
d455c07331 build: strip credentials from remote url on collecting Git provenance info
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 17:14:40 +01:00
Tõnis Tiigi
5ac3b4c4b6 Merge pull request #1662 from crazy-max/v0.10.4_picks
[v0.10] cherry-picks for v0.10.4
2023-03-06 14:37:30 +00:00
CrazyMax
b1440b07f2 build: makes git dirty check opt-in
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-03-06 10:56:54 +01:00
David Karlsson
a3286a0ab1 docs: added --platform=local example
Signed-off-by: David Karlsson <david.karlsson@docker.com>
2023-03-06 10:54:42 +01:00
Tõnis Tiigi
b79345c63e Merge pull request #1645 from cpuguy83/0.10_env_no_provenance
[0.10] Add env var to disable default attestations
2023-02-22 10:28:01 -08:00
Brian Goff
23eb3c3ccd Add env var to disable default attestations
For certain cases we need to build with `--provenance=false`.
However not all build envs (especially in the OSS ethos) have the latest
buildx so just blanket setting `--provenance=false` will fail in these
cases.

Having an env var allows people to set the value without having to worry
about if the buildx version has the `--provenance` flag.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit bc9cb2c66a)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-02-22 18:20:34 +00:00
CrazyMax
79e156beb1 Merge pull request #1636 from crazy-max/v0.10_backport_ci-update-ver
[v0.10 backport] ci: update buildx and buildkit to latest
2023-02-16 14:22:20 +01:00
CrazyMax
c960d16da5 ci: update buildx and buildkit to latest
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit f1a5a3ec50)
2023-02-16 14:16:36 +01:00
CrazyMax
b5b9de69d9 Merge pull request #1635 from crazy-max/v0.10_backport_fix-git-ambiguous
[v0.10 backport] build: fix git ambiguous argument
2023-02-16 14:14:11 +01:00
David Gageot
45863c4f16 Remove git warning: buildx/1633
Signed-off-by: David Gageot <david.gageot@docker.com>
(cherry picked from commit d4a4aaf509)
2023-02-16 14:07:28 +01:00
CrazyMax
f2feea8bed Merge pull request #1609 from crazy-max/0.10.3_cherry_picks
[v0.10] cherry-picks for v0.10.3
2023-02-16 13:48:46 +01:00
Justin Chadwell
a73d07ff7a imagetools: process com.docker.reference.* annotations
To give us the option later down the road of producing recommended OCI
names in BuildKit (using com instead of vnd, woops), we need to update
Buildx to be able to process both.

Ideally, if a Buildx/BuildKit release hadn't been made we could just
switch over, but since we have, we'd need to support both (at least for
a while, eventually we could consider deprecating+removing the vnd
variant).

Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit 642f28f439)
2023-02-16 13:21:41 +01:00
Justin Chadwell
0fad89c3b9 bake: avoid nesting error diagnostics
With changes to the lazy evaluation, the evaluation order is no longer
fixed - this means that we can follow long and confusing paths to get to
an error.

Because of the co-recursive nature of the lazy evaluation, we need to
take special care that the original HCL diagnostics are not discarded
and are preserved so that the original source of the error can be
detected. Preserving the full trace is not necessary, and probably not
useful to the user - all of the file that is not lazily loaded will be
eagerly loaded after all struct blocks are loaded - so the error would
be found regardless.

Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit fbb4f4dec8)
2023-02-09 22:23:02 +01:00
CrazyMax
661af29d46 build: check reachable git commits
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit fd5884189c)
2023-02-08 14:34:23 +01:00
CrazyMax
02cf539a08 gitutil: override the locale to ensure consistent output
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a8eb2a7fbe)
2023-02-08 14:34:14 +01:00
Justin Chadwell
cc87bd104e bake: avoid early-exit for resolution failures
With changes made to allow lazy evaluation, we were early exiting if an
undefined name was detected, either for a variable or a function.

This had two key implications:

1. The error messages changed, and became significantly less
   informative.

   For example, we went from:

   > Unknown variable; There is no variable named "FO". Did you mean "FOO"?, and 1 other diagnostic(s)

   To

   > Invalid expression; undefined variable "FO"

2. Any issues in our function detection from funcCalls which cause JSON
   functions to be erroneously detected cause invalid functions to be
   resolved, which causes new name resolution errors.

To avoid the above problems, we can defer the error from an undefined
name until HCL evaluation - which produces the more informative errors,
and does not suffer from incorrectly detecting JSON functions.

Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit dc8a2b0398)
2023-02-08 14:33:53 +01:00
Justin Chadwell
582cc04be6 build: add docs for boolean attestation flags
Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit 07548bc898)
2023-02-08 14:33:35 +01:00
CrazyMax
ae278ce450 builder: fix docker context not validated
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 0e544fe835)
2023-02-08 14:31:43 +01:00
Justin Chadwell
b66988c824 bake: fix loop references
Signed-off-by: Justin Chadwell <me@jedevc.com>
(cherry picked from commit 48357ee0c6)
2023-02-08 14:29:45 +01:00
Tõnis Tiigi
00ed17df6d Merge pull request #1569 from tonistiigi/v0.10.2-picks
[v0.10] cherry-picks for v0.10.2
2023-01-30 11:57:04 -08:00
CrazyMax
cfb71fab97 build: better message output for git provenance
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6db696748b)
2023-01-30 11:46:51 -08:00
CrazyMax
f62342768b build: silently fail if git remote not found
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 4789d2219c)
2023-01-30 11:46:42 -08:00
Tonis Tiigi
7776652a4d build: fix multi-node merge to read descriptor from result
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit c33b310b48)
2023-01-30 11:46:12 -08:00
Akihiro Suda
5a4f80f3ce bake: SOURCE_DATE_EPOCH: fix panic: assignment to entry in nil map
Fix issue 1562

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 1f56f51740)
2023-01-30 11:45:50 -08:00
CrazyMax
b5ea79e277 build: fix preferred platform not taken account
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 49b3c0dba5)
2023-01-30 11:45:15 -08:00
17 changed files with 275 additions and 113 deletions

View File

@@ -21,8 +21,8 @@ on:
- 'docs/**' - 'docs/**'
env: env:
BUILDX_VERSION: "v0.10.0" BUILDX_VERSION: "latest"
BUILDKIT_IMAGE: "moby/buildkit:v0.11.1" BUILDKIT_IMAGE: "moby/buildkit:latest"
REPO_SLUG: "docker/buildx-bin" REPO_SLUG: "docker/buildx-bin"
DESTDIR: "./bin" DESTDIR: "./bin"

View File

@@ -144,6 +144,9 @@ func ReadTargets(ctx context.Context, files []File, targets, overrides []string,
// The logic is purposely duplicated from `build/build`.go for keeping this visible in `bake --print`. // The logic is purposely duplicated from `build/build`.go for keeping this visible in `bake --print`.
if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" { if v := os.Getenv("SOURCE_DATE_EPOCH"); v != "" {
for _, f := range m { for _, f := range m {
if f.Args == nil {
f.Args = make(map[string]*string)
}
if _, ok := f.Args["SOURCE_DATE_EPOCH"]; !ok { if _, ok := f.Args["SOURCE_DATE_EPOCH"]; !ok {
f.Args["SOURCE_DATE_EPOCH"] = &v f.Args["SOURCE_DATE_EPOCH"] = &v
} }

View File

@@ -78,6 +78,7 @@ func ParseCompose(cfgs []compose.ConfigFile, envs map[string]string) (*Config, e
// compose does not support nil values for labels // compose does not support nil values for labels
labels := map[string]*string{} labels := map[string]*string{}
for k, v := range s.Build.Labels { for k, v := range s.Build.Labels {
v := v
labels[k] = &v labels[k] = &v
} }

View File

@@ -670,6 +670,24 @@ func TestJSONFunctions(t *testing.T) {
require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"]) require.Equal(t, ptrstr("pre-<FOO-abc>"), c.Targets[0].Args["v1"])
} }
func TestJSONInvalidFunctions(t *testing.T) {
dt := []byte(`{
"target": {
"app": {
"args": {
"v1": "myfunc(\"foo\")"
}
}
}}`)
c, err := ParseFile(dt, "docker-bake.json")
require.NoError(t, err)
require.Equal(t, 1, len(c.Targets))
require.Equal(t, c.Targets[0].Name, "app")
require.Equal(t, ptrstr(`myfunc("foo")`), c.Targets[0].Args["v1"])
}
func TestHCLFunctionInAttr(t *testing.T) { func TestHCLFunctionInAttr(t *testing.T) {
dt := []byte(` dt := []byte(`
function "brace" { function "brace" {

View File

@@ -14,15 +14,7 @@ func funcCalls(exp hcl.Expression) ([]string, hcl.Diagnostics) {
if !ok { if !ok {
fns, err := jsonFuncCallsRecursive(exp) fns, err := jsonFuncCallsRecursive(exp)
if err != nil { if err != nil {
return nil, hcl.Diagnostics{ return nil, wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
}
} }
return fns, nil return fns, nil
} }

View File

@@ -62,7 +62,9 @@ type parser struct {
doneB map[*hcl.Block]map[string]struct{} doneB map[*hcl.Block]map[string]struct{}
} }
func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.Diagnostics { var errUndefined = errors.New("undefined")
func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}, allowMissing bool) hcl.Diagnostics {
fns, hcldiags := funcCalls(exp) fns, hcldiags := funcCalls(exp)
if hcldiags.HasErrors() { if hcldiags.HasErrors() {
return hcldiags return hcldiags
@@ -70,15 +72,10 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
for _, fn := range fns { for _, fn := range fns {
if err := p.resolveFunction(fn); err != nil { if err := p.resolveFunction(fn); err != nil {
return hcl.Diagnostics{ if allowMissing && errors.Is(err, errUndefined) {
&hcl.Diagnostic{ continue
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: exp.Range().Ptr(),
Context: exp.Range().Ptr(),
},
} }
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
} }
} }
@@ -128,27 +125,17 @@ func (p *parser) loadDeps(exp hcl.Expression, exclude map[string]struct{}) hcl.D
} }
} }
if err := p.resolveBlock(blocks[0], target); err != nil { if err := p.resolveBlock(blocks[0], target); err != nil {
return hcl.Diagnostics{ if allowMissing && errors.Is(err, errUndefined) {
&hcl.Diagnostic{ continue
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
} }
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
} }
} else { } else {
if err := p.resolveValue(v.RootName()); err != nil { if err := p.resolveValue(v.RootName()); err != nil {
return hcl.Diagnostics{ if allowMissing && errors.Is(err, errUndefined) {
&hcl.Diagnostic{ continue
Severity: hcl.DiagError,
Summary: "Invalid expression",
Detail: err.Error(),
Subject: v.SourceRange().Ptr(),
Context: v.SourceRange().Ptr(),
},
} }
return wrapErrorDiagnostic("Invalid expression", err, exp.Range().Ptr(), exp.Range().Ptr())
} }
} }
} }
@@ -167,7 +154,7 @@ func (p *parser) resolveFunction(name string) error {
if _, ok := p.ectx.Functions[name]; ok { if _, ok := p.ectx.Functions[name]; ok {
return nil return nil
} }
return errors.Errorf("undefined function %s", name) return errors.Wrapf(errUndefined, "function %q does not exit", name)
} }
if _, ok := p.progressF[name]; ok { if _, ok := p.progressF[name]; ok {
return errors.Errorf("function cycle not allowed for %s", name) return errors.Errorf("function cycle not allowed for %s", name)
@@ -217,7 +204,7 @@ func (p *parser) resolveFunction(name string) error {
return diags return diags
} }
if diags := p.loadDeps(f.Result.Expr, params); diags.HasErrors() { if diags := p.loadDeps(f.Result.Expr, params, false); diags.HasErrors() {
return diags return diags
} }
@@ -255,7 +242,7 @@ func (p *parser) resolveValue(name string) (err error) {
if _, builtin := p.opt.Vars[name]; !ok && !builtin { if _, builtin := p.opt.Vars[name]; !ok && !builtin {
vr, ok := p.vars[name] vr, ok := p.vars[name]
if !ok { if !ok {
return errors.Errorf("undefined variable %q", name) return errors.Wrapf(errUndefined, "variable %q does not exit", name)
} }
def = vr.Default def = vr.Default
} }
@@ -270,7 +257,7 @@ func (p *parser) resolveValue(name string) (err error) {
return return
} }
if diags := p.loadDeps(def.Expr, nil); diags.HasErrors() { if diags := p.loadDeps(def.Expr, nil, true); diags.HasErrors() {
return diags return diags
} }
vv, diags := def.Expr.Value(p.ectx) vv, diags := def.Expr.Value(p.ectx)
@@ -314,14 +301,7 @@ func (p *parser) resolveValue(name string) (err error) {
func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) { func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err error) {
name := block.Labels[0] name := block.Labels[0]
if err := p.opt.ValidateLabel(name); err != nil { if err := p.opt.ValidateLabel(name); err != nil {
return hcl.Diagnostics{ return wrapErrorDiagnostic("Invalid name", err, &block.LabelRanges[0], &block.LabelRanges[0])
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid name",
Detail: err.Error(),
Subject: &block.LabelRanges[0],
},
}
} }
if _, ok := p.doneB[block]; !ok { if _, ok := p.doneB[block]; !ok {
@@ -395,7 +375,7 @@ func (p *parser) resolveBlock(block *hcl.Block, target *hcl.BodySchema) (err err
return diag return diag
} }
for _, a := range content.Attributes { for _, a := range content.Attributes {
diag := p.loadDeps(a.Expr, nil) diag := p.loadDeps(a.Expr, nil, true)
if diag.HasErrors() { if diag.HasErrors() {
return diag return diag
} }
@@ -573,15 +553,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
return diags return diags
} }
r := p.vars[k].Body.MissingItemRange() r := p.vars[k].Body.MissingItemRange()
return hcl.Diagnostics{ return wrapErrorDiagnostic("Invalid value", err, &r, &r)
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid value",
Detail: err.Error(),
Subject: &r,
Context: &r,
},
}
} }
} }
@@ -604,15 +576,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
} }
} }
} }
return hcl.Diagnostics{ return wrapErrorDiagnostic("Invalid function", err, subject, context)
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid function",
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
} }
} }
@@ -673,15 +637,7 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
continue continue
} }
} else { } else {
return hcl.Diagnostics{ return wrapErrorDiagnostic("Invalid block", err, &b.LabelRanges[0], &b.DefRange)
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &b.LabelRanges[0],
Context: &b.DefRange,
},
}
} }
} }
@@ -726,21 +682,34 @@ func Parse(b hcl.Body, opt Opt, val interface{}) hcl.Diagnostics {
if diags, ok := err.(hcl.Diagnostics); ok { if diags, ok := err.(hcl.Diagnostics); ok {
return diags return diags
} }
return hcl.Diagnostics{ return wrapErrorDiagnostic("Invalid attribute", err, &p.attrs[k].Range, &p.attrs[k].Range)
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: "Invalid attribute",
Detail: err.Error(),
Subject: &p.attrs[k].Range,
Context: &p.attrs[k].Range,
},
}
} }
} }
return nil return nil
} }
// wrapErrorDiagnostic wraps an error into a hcl.Diagnostics object.
// If the error is already an hcl.Diagnostics object, it is returned as is.
func wrapErrorDiagnostic(message string, err error, subject *hcl.Range, context *hcl.Range) hcl.Diagnostics {
switch err := err.(type) {
case *hcl.Diagnostic:
return hcl.Diagnostics{err}
case hcl.Diagnostics:
return err
default:
return hcl.Diagnostics{
&hcl.Diagnostic{
Severity: hcl.DiagError,
Summary: message,
Detail: err.Error(),
Subject: subject,
Context: context,
},
}
}
}
func setLabel(v reflect.Value, lbl string) int { func setLabel(v reflect.Value, lbl string) int {
// cache field index? // cache field index?
numFields := v.Elem().Type().NumField() numFields := v.Elem().Type().NumField()

View File

@@ -34,9 +34,9 @@ func ReadRemoteFiles(ctx context.Context, nodes []builder.Node, url string, name
var files []File var files []File
var node *builder.Node var node *builder.Node
for _, n := range nodes { for i, n := range nodes {
if n.Err == nil { if n.Err == nil {
node = &n node = &nodes[i]
continue continue
} }
} }

View File

@@ -6,6 +6,7 @@ import (
"context" "context"
"crypto/rand" "crypto/rand"
_ "crypto/sha256" // ensure digests can be computed _ "crypto/sha256" // ensure digests can be computed
"encoding/base64"
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"fmt" "fmt"
@@ -464,8 +465,19 @@ func toSolveOpt(ctx context.Context, node builder.Node, multiDriver bool, opt Op
so.FrontendAttrs[k] = v so.FrontendAttrs[k] = v
} }
} }
if _, ok := opt.Attests["attest:provenance"]; !ok && supportsAttestations { if _, ok := opt.Attests["attest:provenance"]; !ok && supportsAttestations {
so.FrontendAttrs["attest:provenance"] = "mode=min,inline-only=true" const noAttestEnv = "BUILDX_NO_DEFAULT_ATTESTATIONS"
var noProv bool
if v, ok := os.LookupEnv(noAttestEnv); ok {
noProv, err = strconv.ParseBool(v)
if err != nil {
return nil, nil, errors.Wrap(err, "invalid "+noAttestEnv)
}
}
if !noProv {
so.FrontendAttrs["attest:provenance"] = "mode=min,inline-only=true"
}
} }
switch len(opt.Exports) { switch len(opt.Exports) {
@@ -1157,7 +1169,24 @@ func BuildWithResultHandler(ctx context.Context, nodes []builder.Node, opt map[s
descs := make([]specs.Descriptor, 0, len(res)) descs := make([]specs.Descriptor, 0, len(res))
for _, r := range res { for _, r := range res {
s, ok := r.ExporterResponse[exptypes.ExporterImageDigestKey] s, ok := r.ExporterResponse[exptypes.ExporterImageDescriptorKey]
if ok {
dt, err := base64.StdEncoding.DecodeString(s)
if err != nil {
return err
}
var desc specs.Descriptor
if err := json.Unmarshal(dt, &desc); err != nil {
return errors.Wrapf(err, "failed to unmarshal descriptor %s", s)
}
descs = append(descs, desc)
continue
}
// This is fallback for some very old buildkit versions.
// Note that the mediatype isn't really correct as most of the time it is image manifest and
// not manifest list but actually both are handled because for Docker mediatypes the
// mediatype value in the Accpet header does not seem to matter.
s, ok = r.ExporterResponse[exptypes.ExporterImageDigestKey]
if ok { if ok {
descs = append(descs, specs.Descriptor{ descs = append(descs, specs.Descriptor{
Digest: digest.Digest(s), Digest: digest.Digest(s),

View File

@@ -52,22 +52,28 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd)) gitc, err := gitutil.New(gitutil.WithContext(ctx), gitutil.WithWorkingDir(wd))
if err != nil { if err != nil {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() { if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("git was not found in the system. Current commit information was not captured by the build") return res, errors.New("buildx: git was not found in the system. Current commit information was not captured by the build")
} }
return return
} }
if !gitc.IsInsideWorkTree() { if !gitc.IsInsideWorkTree() {
if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() { if st, err := os.Stat(path.Join(wd, ".git")); err == nil && st.IsDir() {
return res, errors.New("failed to read current commit information with git rev-parse --is-inside-work-tree") return res, errors.New("buildx: failed to read current commit information with git rev-parse --is-inside-work-tree")
} }
return res, nil return res, nil
} }
if sha, err := gitc.FullCommit(); err != nil { if sha, err := gitc.FullCommit(); err != nil && !gitutil.IsUnknownRevision(err) {
return res, errors.Wrapf(err, "failed to get git commit") return res, errors.Wrapf(err, "buildx: failed to get git commit")
} else if sha != "" { } else if sha != "" {
if gitc.IsDirty() { checkDirty := false
if v, ok := os.LookupEnv("BUILDX_GIT_CHECK_DIRTY"); ok {
if v, err := strconv.ParseBool(v); err == nil {
checkDirty = v
}
}
if checkDirty && gitc.IsDirty() {
sha += "-dirty" sha += "-dirty"
} }
if setGitLabels { if setGitLabels {
@@ -78,9 +84,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
} }
} }
if rurl, err := gitc.RemoteURL(); err != nil { if rurl, err := gitc.RemoteURL(); err == nil && rurl != "" {
return res, errors.Wrapf(err, "failed to get git remote url")
} else if rurl != "" {
if setGitLabels { if setGitLabels {
res["label:"+specs.AnnotationSource] = rurl res["label:"+specs.AnnotationSource] = rurl
} }
@@ -91,7 +95,7 @@ func getGitAttributes(ctx context.Context, contextPath string, dockerfilePath st
if setGitLabels { if setGitLabels {
if root, err := gitc.RootDir(); err != nil { if root, err := gitc.RootDir(); err != nil {
return res, errors.Wrapf(err, "failed to get git root dir") return res, errors.Wrapf(err, "buildx: failed to get git root dir")
} else if root != "" { } else if root != "" {
if dockerfilePath == "" { if dockerfilePath == "" {
dockerfilePath = filepath.Join(wd, "Dockerfile") dockerfilePath = filepath.Join(wd, "Dockerfile")

View File

@@ -131,6 +131,7 @@ func TestGetGitAttributes(t *testing.T) {
func TestGetGitAttributesDirty(t *testing.T) { func TestGetGitAttributesDirty(t *testing.T) {
setupTest(t) setupTest(t)
t.Setenv("BUILDX_GIT_CHECK_DIRTY", "true")
// make a change to test dirty flag // make a change to test dirty flag
df := []byte("FROM alpine:edge\n") df := []byte("FROM alpine:edge\n")

View File

@@ -62,6 +62,7 @@ func (b *Builder) LoadNodes(ctx context.Context, withData bool) (_ []Node, err e
node := Node{ node := Node{
Node: n, Node: n,
ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli), ProxyConfig: storeutil.GetProxyConfig(b.opts.dockerCli),
Platforms: n.Platforms,
} }
defer func() { defer func() {
b.nodes[i] = node b.nodes[i] = node

View File

@@ -88,6 +88,9 @@ BuildKit currently supports:
Use `--attest=type=provenance` to generate provenance for an image at Use `--attest=type=provenance` to generate provenance for an image at
build-time. Alternatively, you can use the [`--provenance` shorthand](#provenance). build-time. Alternatively, you can use the [`--provenance` shorthand](#provenance).
By default, a minimal provenance attestation will be created for the build
result, which will only be attached for images pushed to registries.
For more information, see [here](https://docs.docker.com/build/attestations/slsa-provenance/). For more information, see [here](https://docs.docker.com/build/attestations/slsa-provenance/).
### <a name="allow"></a> Allow extra privileged entitlement (--allow) ### <a name="allow"></a> Allow extra privileged entitlement (--allow)
@@ -411,8 +414,13 @@ The `registry` exporter is a shortcut for `type=image,push=true`.
Set the target platform for the build. All `FROM` commands inside the Dockerfile Set the target platform for the build. All `FROM` commands inside the Dockerfile
without their own `--platform` flag will pull base images for this platform and without their own `--platform` flag will pull base images for this platform and
this value will also be the platform of the resulting image. The default value this value will also be the platform of the resulting image.
will be the current platform of the buildkit daemon.
The default value is the platform of the BuildKit daemon where the build runs.
The value takes the form of `os/arch` or `os/arch/variant`. For example,
`linux/amd64` or `linux/arm/v7`. Additionally, the `--platform` flag also supports
a special `local` value, which tells BuildKit to use the platform of the BuildKit
client that invokes the build.
When using `docker-container` driver with `buildx`, this flag can accept multiple When using `docker-container` driver with `buildx`, this flag can accept multiple
values as an input separated by a comma. With multiple values the result will be values as an input separated by a comma. With multiple values the result will be
@@ -477,8 +485,20 @@ $ docker buildx build --load --progress=plain .
### <a name="provenance"></a> Create provenance attestations (--provenance) ### <a name="provenance"></a> Create provenance attestations (--provenance)
Shorthand for [`--attest=type=provenance`](#attest). Enables provenance Shorthand for [`--attest=type=provenance`](#attest), used to configure
attestations for the build result. provenance attestations for the build result. For example,
`--provenance=mode=max` can be used as an abbreviation for
`--attest=type=provenance,mode=max`.
Additionally, `--provenance` can be used with boolean values to broadly enable
or disable provenance attestations. For example, `--provenance=false` can be
used to disable all provenance attestations, while `--provenance=true` can be
used to enable all provenance attestations.
By default, a minimal provenance attestation will be created for the build
result, which will only be attached for images pushed to registries.
For more information, see [here](https://docs.docker.com/build/attestations/slsa-provenance/).
### <a name="push"></a> Push the build result to a registry (--push) ### <a name="push"></a> Push the build result to a registry (--push)
@@ -487,8 +507,16 @@ build result to registry.
### <a name="sbom"></a> Create SBOM attestations (--sbom) ### <a name="sbom"></a> Create SBOM attestations (--sbom)
Shorthand for [`--attest=type=sbom`](#attest). Enables SBOM attestations for Shorthand for [`--attest=type=sbom`](#attest), used to configure SBOM
the build result. attestations for the build result. For example,
`--sbom=generator=<user>/<generator-image>` can be used as an abbreviation for
`--attest=type=sbom,generator=<user>/<generator-image>`.
Additionally, `--sbom` can be used with boolean values to broadly enable or
disable SBOM attestations. For example, `--sbom=false` can be used to disable
all SBOM attestations.
For more information, see [here](https://docs.docker.com/build/attestations/sbom/).
### <a name="secret"></a> Secret to expose to the build (--secret) ### <a name="secret"></a> Secret to expose to the build (--secret)

View File

@@ -93,6 +93,7 @@ func GetNodeGroup(txn *store.Txn, dockerCli command.Cli, name string) (*store.No
Endpoint: name, Endpoint: name,
}, },
}, },
DockerContext: true,
} }
if ng.LastActivity, err = txn.GetLastActivity(ng); err != nil { if ng.LastActivity, err = txn.GetLastActivity(ng); err != nil {
return nil, err return nil, err

View File

@@ -3,6 +3,8 @@ package gitutil
import ( import (
"bytes" "bytes"
"context" "context"
"net/url"
"os"
"os/exec" "os/exec"
"strings" "strings"
@@ -69,21 +71,21 @@ func (c *Git) RootDir() (string, error) {
func (c *Git) RemoteURL() (string, error) { func (c *Git) RemoteURL() (string, error) {
// Try to get the remote URL from the origin remote first // Try to get the remote URL from the origin remote first
if ru, err := c.clean(c.run("remote", "get-url", "origin")); err == nil && ru != "" { if ru, err := c.clean(c.run("remote", "get-url", "origin")); err == nil && ru != "" {
return ru, nil return stripCredentials(ru), nil
} }
// If that fails, try to get the remote URL from the upstream remote // If that fails, try to get the remote URL from the upstream remote
if ru, err := c.clean(c.run("remote", "get-url", "upstream")); err == nil && ru != "" { if ru, err := c.clean(c.run("remote", "get-url", "upstream")); err == nil && ru != "" {
return ru, nil return stripCredentials(ru), nil
} }
return "", errors.New("no remote URL found for either origin or upstream") return "", errors.New("no remote URL found for either origin or upstream")
} }
func (c *Git) FullCommit() (string, error) { func (c *Git) FullCommit() (string, error) {
return c.clean(c.run("show", "--format=%H", "HEAD", "--quiet")) return c.clean(c.run("show", "--format=%H", "HEAD", "--quiet", "--"))
} }
func (c *Git) ShortCommit() (string, error) { func (c *Git) ShortCommit() (string, error) {
return c.clean(c.run("show", "--format=%h", "HEAD", "--quiet")) return c.clean(c.run("show", "--format=%h", "HEAD", "--quiet", "--"))
} }
func (c *Git) Tag() (string, error) { func (c *Git) Tag() (string, error) {
@@ -116,6 +118,9 @@ func (c *Git) run(args ...string) (string, error) {
cmd.Dir = c.wd cmd.Dir = c.wd
} }
// Override the locale to ensure consistent output
cmd.Env = append(os.Environ(), "LC_ALL=C")
stdout := bytes.Buffer{} stdout := bytes.Buffer{}
stderr := bytes.Buffer{} stderr := bytes.Buffer{}
cmd.Stdout = &stdout cmd.Stdout = &stdout
@@ -134,3 +139,25 @@ func (c *Git) clean(out string, err error) (string, error) {
} }
return out, err return out, err
} }
func IsUnknownRevision(err error) bool {
if err == nil {
return false
}
// https://github.com/git/git/blob/a6a323b31e2bcbac2518bddec71ea7ad558870eb/setup.c#L204
errMsg := strings.ToLower(err.Error())
return strings.Contains(errMsg, "unknown revision or path not in the working tree") || strings.Contains(errMsg, "bad revision")
}
// stripCredentials takes a URL and strips username and password from it.
// e.g. "https://user:password@host.tld/path.git" will be changed to
// "https://host.tld/path.git".
// TODO: remove this function once fix from BuildKit is vendored here
func stripCredentials(s string) string {
ru, err := url.Parse(s)
if err != nil {
return s // string is not a URL, just return it
}
ru.User = nil
return ru.String()
}

View File

@@ -46,6 +46,32 @@ func TestGitShortCommit(t *testing.T) {
require.Equal(t, 7, len(out)) require.Equal(t, 7, len(out))
} }
func TestGitFullCommitErr(t *testing.T) {
Mktmp(t)
c, err := New()
require.NoError(t, err)
GitInit(c, t)
_, err = c.FullCommit()
require.Error(t, err)
require.True(t, IsUnknownRevision(err))
require.False(t, IsAmbiguousArgument(err))
}
func TestGitShortCommitErr(t *testing.T) {
Mktmp(t)
c, err := New()
require.NoError(t, err)
GitInit(c, t)
_, err = c.ShortCommit()
require.Error(t, err)
require.True(t, IsUnknownRevision(err))
require.False(t, IsAmbiguousArgument(err))
}
func TestGitTagsPointsAt(t *testing.T) { func TestGitTagsPointsAt(t *testing.T) {
Mktmp(t) Mktmp(t)
c, err := New() c, err := New()
@@ -163,3 +189,45 @@ func TestGitRemoteURL(t *testing.T) {
}) })
} }
} }
func TestStripCredentials(t *testing.T) {
cases := []struct {
name string
url string
want string
}{
{
name: "non-blank Password",
url: "https://user:password@host.tld/this:that",
want: "https://host.tld/this:that",
},
{
name: "blank Password",
url: "https://user@host.tld/this:that",
want: "https://host.tld/this:that",
},
{
name: "blank Username",
url: "https://:password@host.tld/this:that",
want: "https://host.tld/this:that",
},
{
name: "blank Username, blank Password",
url: "https://host.tld/this:that",
want: "https://host.tld/this:that",
},
{
name: "invalid URL",
url: "1https://foo.com",
want: "1https://foo.com",
},
}
for _, tt := range cases {
tt := tt
t.Run(tt.name, func(t *testing.T) {
if g, w := stripCredentials(tt.url), tt.want; g != w {
t.Fatalf("got: %q\nwant: %q", g, w)
}
})
}
}

View File

@@ -2,6 +2,7 @@ package gitutil
import ( import (
"os" "os"
"strings"
"testing" "testing"
"github.com/stretchr/testify/require" "github.com/stretchr/testify/require"
@@ -73,3 +74,11 @@ func fakeGit(c *Git, args ...string) (string, error) {
allArgs = append(allArgs, args...) allArgs = append(allArgs, args...)
return c.clean(c.run(allArgs...)) return c.clean(c.run(allArgs...))
} }
func IsAmbiguousArgument(err error) bool {
if err == nil {
return false
}
errMsg := strings.ToLower(err.Error())
return strings.Contains(errMsg, "use '--' to separate paths from revisions")
}

View File

@@ -21,8 +21,11 @@ import (
"golang.org/x/sync/errgroup" "golang.org/x/sync/errgroup"
) )
const ( var (
annotationReference = "vnd.docker.reference.digest" annotationReferences = []string{
"com.docker.reference.digest",
"vnd.docker.reference.digest", // TODO: deprecate/remove after migration to new annotation
}
) )
type contentCache interface { type contentCache interface {
@@ -167,8 +170,13 @@ func (l *loader) fetch(ctx context.Context, fetcher remotes.Fetcher, desc ocispe
} }
r.mu.Unlock() r.mu.Unlock()
ref, ok := desc.Annotations[annotationReference] found := false
if ok { for _, annotationReference := range annotationReferences {
ref, ok := desc.Annotations[annotationReference]
if !ok {
continue
}
refdgst, err := digest.Parse(ref) refdgst, err := digest.Parse(ref)
if err != nil { if err != nil {
return err return err
@@ -176,7 +184,10 @@ func (l *loader) fetch(ctx context.Context, fetcher remotes.Fetcher, desc ocispe
r.mu.Lock() r.mu.Lock()
r.refs[refdgst] = append(r.refs[refdgst], desc.Digest) r.refs[refdgst] = append(r.refs[refdgst], desc.Digest)
r.mu.Unlock() r.mu.Unlock()
} else { found = true
break
}
if !found {
p := desc.Platform p := desc.Platform
if p == nil { if p == nil {
p, err = l.readPlatformFromConfig(ctx, fetcher, mfst.Config) p, err = l.readPlatformFromConfig(ctx, fetcher, mfst.Config)