Go maintainers started to unconditionally update the minimum go version
for golang.org/x/ dependencies to go1.23, which means that we'll no longer
be able to support any version below that when updating those dependencies;
> all: upgrade go directive to at least 1.23.0 [generated]
>
> By now Go 1.24.0 has been released, and Go 1.22 is no longer supported
> per the Go Release Policy (https://go.dev/doc/devel/release#policy).
>
> For golang/go#69095.
This updates our minimum version to go1.23, as we won't be able to maintain
compatibility with older versions because of the above.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
In most situations, the CLI is created through the `NewDockerCli` constructor,
however, it's possible to construct a CLI manually (`&DockerCli{}`). We
should probably prevent this (and un-export the `DockerCli` implementation),
but currently have some code-paths that depend on the type being exported.
When constructing the CLI with this approach, the CLI would not be fully
initialized and not have the context-store configuration set up.
Using the default context store without a config set will result in Endpoints
from contexts not being type-mapped correctly, and used as a generic
`map[string]any`, instead of a [docker.EndpointMeta].
When looking up the API endpoint (using [EndpointFromContext]), no endpoint
will be found, and a default, empty endpoint will be used instead which in
its turn, causes [newAPIClientFromEndpoint] to be initialized with the default
config instead of settings for the current context (which may mean; connecting
with the wrong endpoint and/or TLS Config to be missing).
I'm not sure if this situation could happen in practice, but it caused some
of our unit-tests ([TestInitializeFromClient] among others) to fail when
running outside of the dev-container on a host that used Docker Desktop's
"desktop-linux" context. In that situation, the test would produce the wrong
"Ping" results (using defaults, instead of the results produced in the test).
This patch:
- updates the contextStoreConfig field to be a pointer, so that we are
able to detect if a config was already set.
- updates the `Initialize` function to set the default context-store config
if no config was found (technically the field is mostly immutable, and
can only set through `WithDefaultContextStoreConfig`, so this may be
slightly redundant).
We should update this code to be less error-prone to use; the combination
of an exported type (`DockerCli`), a constructor `NewDockerCli` and a
`Initialize` function (as well as some internal contructors to allow
lazy initialization) make constructing the "CLI" hard to use, and there's
various codepaths where it can be in a partially initialized state. The
same applies to the default context store, which also requires too much
"domain" knowledge to use properly.
I'm leaving improvements around that for a follow-up.
[EndpointFromContext]: 33494921b8/cli/context/docker/load.go (L139-L149)
[docker.EndpointMeta]: 33494921b8/cli/context/docker/load.go (L19-L21)
[newAPIClientFromEndpoint]: 33494921b8/cli/command/cli.go (L295-L305)
[TestInitializeFromClient]: 33494921b8/cli/command/cli_test.go (L157-L205)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method was a shallow wrapper around registryclient.NewRegistryClient but
due to its signature resulted in various dependencies becoming a dependency
of the "command" package. Consequence of this was that cli-plugins, which
need the cli/command package, would also get those dependencies. It is no
longer used, and was deprecated in 8ad07217dc92fa81806d4eb64a4eb04cf5c99b15.
This patch removes the RegistryClient method from the interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method is a shallow wrapper around trust.GetNotaryRepository, but
due to its signature resulted in the trust package, and notary dependencies
to become a dependency of the CLI. Consequence of this was that cli-plugins,
which need the cli/command package, would also get notary and its
dependencies as a dependency. It is no longer used, and was deprecated
in 9bc16bbde085c618d3bddfb633560ab9af83a58d.
This patch removes the NotaryClient method from the interface
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's hard-coded to the API defaultversion, so we can use
that const directly. Ultimately, this should be something
returned by the API client configured on the CLI, not the
CLI itself.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The DOCKER_CLI_HINTS env-var is replaced by DOCKER_CLI_HOOKS; check the
new env-var first, and only fall back to checking the legacy env-var
if it's not set.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method was a shallow wrapper around registryclient.NewRegistryClient but
due to its signature resulted in various dependencies becoming a dependency
of the "command" package. Consequence of this was that cli-plugins, which
need the cli/command package, would also get those dependencies. It is no
longer used in our code, which constructs the client in packages that need it,
so we can deprecate this method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method is a shallow wrapper around manifeststore.NewStore, but
due to its signature resulted in various dependencies becoming a dependency
of the "command" package. Consequence of this was that cli-plugins, which
need the cli/command package, would also get those dependencies. It is no
longer used in our code, which constructs the client in packages that need it,
so we can deprecate this method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This method is a shallow wrapper around trust.GetNotaryRepository, but
due to its signature resulted in the trust package, and notary dependencies
to become a dependency of the CLI. Consequence of this was that cli-plugins,
which need the cli/command package, would also get notary and its
dependencies as a dependency. It is no longer used in our code, which
constructs the client in packages that need it, so we can deprecate this
method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These utility functions were added in 8890a1c9292ed5f2bf45df44a0e57d345f563d31,
and are all related to OTEL. The ResourceAttributesEnvvar const defines
the "OTEL_RESOURCE_ATTRIBUTES" environment-variable to use, which is part
of the [OpenTelemetry specification], so should be considered a well-known
env-var, and not up to us to define a const for. These code-changes were not
yet included in a release, so we don't have to deprecate.
This patch:
- Moves the utility functions to the telemetry files, so that all code related
to OpenTelemetry is together.
- Un-exports the ResourceAttributesEnvvar to reduce our public API.
- Un-exports the DockerCliAttributePrefix to reduce depdency on cli/command
in CLI-plugins, but adds a TODO to move telemetry-related code to a common
(internal) package.
- Deprecates the cli-plugins/manager.ResourceAttributesEnvvar const. This
const has no known consumers, so we could skip deprecation, but just in
case some codebase uses this.
[OpenTelemetry specification]: https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Remove the `docker.cli` prefixed attributes from
`OTEL_RESOURCE_ATTRIBUTES` after the telemetry provider has been created
within a plugin. This prevents accidentally sending the attributes to
something downstream for the user.
This also fixes an issue with compose where the self-injected `OTEL_RESOURCE_ATTRIBUTES`
would override an existing attribute in the environment file because the
"user environment" overrode the environment file, but the "user
environment" was created by the `docker` tool rather than by the user's
environment.
When `OTEL_RESOURCE_ATTRIBUTES` is empty after pruning, the environment
variable is unset.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
cli/command/cli.go💯7: unused-receiver: method receiver 'cli' is not referenced in method's body, consider removing or renaming it as _ (revive)
func (cli *DockerCli) DefaultVersion() string {
^
cli/command/cli.go:234:7: unused-receiver: method receiver 'cli' is not referenced in method's body, consider removing or renaming it as _ (revive)
func (cli *DockerCli) ManifestStore() manifeststore.Store {
^
cli/command/telemetry.go:57:7: unused-receiver: method receiver 'cli' is not referenced in method's body, consider removing or renaming it as _ (revive)
func (cli *DockerCli) TracerProvider() trace.TracerProvider {
^
cli/command/telemetry.go:61:7: unused-receiver: method receiver 'cli' is not referenced in method's body, consider removing or renaming it as _ (revive)
func (cli *DockerCli) MeterProvider() metric.MeterProvider {
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- use Println to print newline instead of custom format
- suppress some errors to make my IDE and linters happier
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
commit 4a7b04d4127c6082a9e8de95cfce6f34744d8fc1 configured golangci-lint
to use go1.23 semantics, which enabled the copyloopvar linter.
go1.22 now creates a copy of variables when assigned in a loop; make sure we
don't have files that may downgrade semantics to go1.21 in case that also means
disabling that feature; https://go.dev/ref/spec#Go_1.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This environment variable allows for setting additional headers
to be sent by the client. Headers set through this environment
variable are added to headers set through the config-file (through
the HttpHeaders field).
This environment variable can be used in situations where headers
must be set for a specific invocation of the CLI, but should not
be set by default, and therefore cannot be set in the config-file.
WARNING: If both config and environment-variable are set, the environment
variable currently overrides all headers set in the configuration file.
This behavior may change in a future update, as we are considering the
environment variable to be appending to existing headers (and to only
override headers with the same name).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 6fef143dbcf5fdcb436d05cf8e67d56c24bc4e6e switched the CLI to use
BuildKit by default, but as part of that removed the use of the
BuildkitVersion field as returned by Ping.
Some follow-up changes in commits e38e6c51ffb55405321038dfcf3a477f and
e7a8748b939fbf7798352cff1e0b87ef updated the logic for detecting whether
BuildKit should be used or the legacy builder, but hard-coded using the
legacy builder for Windows daemons.
While Windows / WCOW does not yet support BuildKit by default, there is
work in progress to implement it, so we should not hard-code the assumption
that a Windows daemon cannot support BuildKit.
On the daemon-side, [moby@7b153b9] (Docker v23.0) changed the default as
advertised by the daemon to be BuildKit for Linux daemons. That change
still hardcoded BuildKit to be unsupported for Windows daemons (and does
not yet allow overriding the config), but this may change for future
versions of the daemon, or test-builds.
This patch:
- Re-introduces checks for the BuildkitVersion field in the "Ping" response.
- If the Ping response from the daemon advertises that it supports BuildKit,
the CLI will now use BuildKit as builder.
- If we didn't get a Ping response, or the Ping response did NOT advertise
that the daemon supported BuildKit, we continue to use the current
defaults (BuildKit for Linux daemons, and the legacy builder for Windows)
- Handling of the DOCKER_BUILDKIT environment variable is unchanged; for
CLI.BuildKitEnabled, DOCKER_BUILDKIT always takes precedence, and for
processBuilder the value is taken into account, but will print a warning
when BuildKit is disabled and a Linux daemon is used. For Windows daemons,
no warning is printed.
[moby@7b153b9]: 7b153b9e28
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This wraps the cli stderr stream the same way as stdin and stdout, which
extends the stream with TTY-related methods.
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
internal/test/cli.go:175:14: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("no notary client available unless defined")
^
cli/command/cli.go:318:29: fmt.Errorf can be replaced with errors.New (perfsprint)
return docker.Endpoint{}, fmt.Errorf("no context store initialized")
^
cli/command/container/attach.go:161:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf(result.Error.Message)
^
cli/command/container/opts.go:577:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("--health-start-period cannot be negative")
^
cli/command/container/opts.go:580:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("--health-start-interval cannot be negative")
^
cli/command/container/stats.go:221:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("filtering is not supported when specifying a list of containers")
^
cli/command/container/attach_test.go:82:17: fmt.Errorf can be replaced with errors.New (perfsprint)
expectedErr = fmt.Errorf("unexpected error")
^
cli/command/container/create_test.go:234:40: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.CreateResponse{}, fmt.Errorf("shouldn't try to pull image")
^
cli/command/container/list_test.go:150:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("error listing containers")
^
cli/command/container/rm_test.go:40:31: fmt.Errorf can be replaced with errors.New (perfsprint)
return errdefs.NotFound(fmt.Errorf("Error: no such container: " + container))
^
cli/command/container/run_test.go:138:40: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.CreateResponse{}, fmt.Errorf("shouldn't try to pull image")
^
cli/command/image/pull_test.go:115:49: fmt.Errorf can be replaced with errors.New (perfsprint)
return io.NopCloser(strings.NewReader("")), fmt.Errorf("shouldn't try to pull image")
^
cli/command/network/connect.go:88:16: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("invalid key/value pair format in driver options")
^
cli/command/plugin/create_test.go:96:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error creating plugin")
^
cli/command/plugin/disable_test.go:32:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error disabling plugin")
^
cli/command/plugin/enable_test.go:32:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("failed to enable plugin")
^
cli/command/plugin/inspect_test.go:55:22: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, nil, fmt.Errorf("error inspecting plugin")
^
cli/command/plugin/install_test.go:43:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("Error installing plugin")
^
cli/command/plugin/install_test.go:51:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("(image) when fetching")
^
cli/command/plugin/install_test.go:95:17: fmt.Errorf can be replaced with errors.New (perfsprint)
return nil, fmt.Errorf("should not try to install plugin")
^
cli/command/plugin/list_test.go:35:41: fmt.Errorf can be replaced with errors.New (perfsprint)
return types.PluginsListResponse{}, fmt.Errorf("error listing plugins")
^
cli/command/plugin/remove_test.go:27:12: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("Error removing plugin")
^
cli/command/registry/login_test.go:36:46: fmt.Errorf can be replaced with errors.New (perfsprint)
return registrytypes.AuthenticateOKBody{}, fmt.Errorf("Invalid Username or Password")
^
cli/command/registry/login_test.go:44:46: fmt.Errorf can be replaced with errors.New (perfsprint)
return registrytypes.AuthenticateOKBody{}, fmt.Errorf(errUnknownUser)
^
cli/command/system/info.go:190:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("errors pretty printing info")
^
cli/command/system/prune.go:77:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf(`ERROR: The "until" filter is not supported with "--volumes"`)
^
cli/command/system/version_test.go:19:28: fmt.Errorf can be replaced with errors.New (perfsprint)
return types.Version{}, fmt.Errorf("no server")
^
cli/command/trust/key_load.go:112:22: fmt.Errorf can be replaced with errors.New (perfsprint)
return []byte{}, fmt.Errorf("could not decrypt key")
^
cli/command/trust/revoke.go:44:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("cannot use a digest reference for IMAGE:TAG")
^
cli/command/trust/revoke.go:105:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("no signed tags to remove")
^
cli/command/trust/signer_add.go:56:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("releases is a reserved keyword, please use a different signer name")
^
cli/command/trust/signer_add.go:60:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("path to a public key must be provided using the `--key` flag")
^
opts/config.go:71:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("source is required")
^
opts/mount.go:168:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("type is required")
^
opts/mount.go:172:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("target is required")
^
opts/network.go:90:11: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("network name/id is not specified")
^
opts/network.go:129:18: fmt.Errorf can be replaced with errors.New (perfsprint)
return "", "", fmt.Errorf("invalid key value pair format in driver options")
^
opts/opts.go:404:13: fmt.Errorf can be replaced with errors.New (perfsprint)
return 0, fmt.Errorf("value is too precise")
^
opts/opts.go:412:18: fmt.Errorf can be replaced with errors.New (perfsprint)
return "", "", fmt.Errorf("empty string specified for links")
^
opts/parse.go:84:37: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.RestartPolicy{}, fmt.Errorf("invalid restart policy format: no policy provided before colon")
^
opts/parse.go:89:38: fmt.Errorf can be replaced with errors.New (perfsprint)
return container.RestartPolicy{}, fmt.Errorf("invalid restart policy format: maximum retry count must be an integer")
^
opts/port.go:105:13: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("hostip is not supported")
^
opts/secret.go:70:10: fmt.Errorf can be replaced with errors.New (perfsprint)
return fmt.Errorf("source is required")
^
opts/env_test.go:57:11: fmt.Errorf can be replaced with errors.New (perfsprint)
err: fmt.Errorf("invalid environment variable: =a"),
^
opts/env_test.go:93:11: fmt.Errorf can be replaced with errors.New (perfsprint)
err: fmt.Errorf("invalid environment variable: ="),
^
cli-plugins/manager/error_test.go:16:11: fmt.Errorf can be replaced with errors.New (perfsprint)
inner := fmt.Errorf("testing")
^
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This adds the code used by buildx and compose into the default CLI
program to help normalize the usage of these APIs and allow code reuse
between projects. It also allows these projects to benefit from
improvements or changes that may be made by another team.
At the moment, these APIs are a pretty thin layer on the OTEL SDK. It
configures an additional exporter to a docker endpoint that's used for
usage collection and is only active if the option is configured in
docker desktop.
This also upgrades the OTEL version to v1.19 which is the one being used
by buildkit, buildx, compose, etc.
Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>
This is a follow-up to 0e73168b7e6d1d029d76d05b843b1aaec46739a8
This repository is not yet a module (i.e., does not have a `go.mod`). This
is not problematic when building the code in GOPATH or "vendor" mode, but
when using the code as a module-dependency (in module-mode), different semantics
are applied since Go1.21, which switches Go _language versions_ on a per-module,
per-package, or even per-file base.
A condensed summary of that logic [is as follows][1]:
- For modules that have a go.mod containing a go version directive; that
version is considered a minimum _required_ version (starting with the
go1.19.13 and go1.20.8 patch releases: before those, it was only a
recommendation).
- For dependencies that don't have a go.mod (not a module), go language
version go1.16 is assumed.
- Likewise, for modules that have a go.mod, but the file does not have a
go version directive, go language version go1.16 is assumed.
- If a go.work file is present, but does not have a go version directive,
language version go1.17 is assumed.
When switching language versions, Go _downgrades_ the language version,
which means that language features (such as generics, and `any`) are not
available, and compilation fails. For example:
# github.com/docker/cli/cli/context/store
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/storeconfig.go:6:24: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
/go/pkg/mod/github.com/docker/cli@v25.0.0-beta.2+incompatible/cli/context/store/store.go:74:12: predeclared any requires go1.18 or later (-lang was set to go1.16; check go.mod)
Note that these fallbacks are per-module, per-package, and can even be
per-file, so _(indirect) dependencies_ can still use modern language
features, as long as their respective go.mod has a version specified.
Unfortunately, these failures do not occur when building locally (using
vendor / GOPATH mode), but will affect consumers of the module.
Obviously, this situation is not ideal, and the ultimate solution is to
move to go modules (add a go.mod), but this comes with a non-insignificant
risk in other areas (due to our complex dependency tree).
We can revert to using go1.16 language features only, but this may be
limiting, and may still be problematic when (e.g.) matching signatures
of dependencies.
There is an escape hatch: adding a `//go:build` directive to files that
make use of go language features. From the [go toolchain docs][2]:
> The go line for each module sets the language version the compiler enforces
> when compiling packages in that module. The language version can be changed
> on a per-file basis by using a build constraint.
>
> For example, a module containing code that uses the Go 1.21 language version
> should have a `go.mod` file with a go line such as `go 1.21` or `go 1.21.3`.
> If a specific source file should be compiled only when using a newer Go
> toolchain, adding `//go:build go1.22` to that source file both ensures that
> only Go 1.22 and newer toolchains will compile the file and also changes
> the language version in that file to Go 1.22.
This patch adds `//go:build` directives to those files using recent additions
to the language. It's currently using go1.19 as version to match the version
in our "vendor.mod", but we can consider being more permissive ("any" requires
go1.18 or up), or more "optimistic" (force go1.21, which is the version we
currently use to build).
For completeness sake, note that any file _without_ a `//go:build` directive
will continue to use go1.16 language version when used as a module.
[1]: 58c28ba286/src/cmd/go/internal/gover/version.go (L9-L56)
[2]; https://go.dev/doc/toolchain#:~:text=The%20go%20line%20for,file%20to%20Go%201.22
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The cli/command package defined two option-types with the same signature.
This patch creates a new type instead (CLIOption), and makes the existing
types an alias for this (deprecating their old names).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Both these functions took the whole DockerCLI as argument, but only needed
the ConfigFile. ResolveAuthConfig also had an unused context.Context as
argument.
This patch updates both functions to accept a ConfigFile, and removes the
unused context.Context.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
More things to be done after this, to allow passing a custom user-agent,
but let's start with just using this utility.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
---
commandconn: fix race on `Close()`
During normal operation, if a `Read()` or `Write()` call results
in an EOF, we call `onEOF()` to handle the terminating command,
and store it's exit value.
However, if a Read/Write call was blocked while `Close()` is called
the in/out pipes are immediately closed which causes an EOF to be
returned. Here, we shouldn't call `onEOF()`, since the reason why
we got an EOF is because we're already terminating the connection.
This also prevents a race between two calls to the commands `Wait()`,
in the `Close()` call and `onEOF()`
---
Add CLI init timeout to SSH connections
---
connhelper: add 30s ssh default dialer timeout
(same as non-ssh dialer)
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
Add a const for the name of the environment-variable we accept, so
that we can document its purpose in code.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fixes the cli erroring out if the variable is set to an empty
value.
```
$ export DOCKER_BUILDKIT=
$ docker version
DOCKER_BUILDKIT environment variable expects boolean value: strconv.ParseBool: parsing "": invalid syntax
```
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
The DockerCLI interface was repeating the Streams interface. Embed
the interface to make it more transparent that they're the same.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
