Skaffold’s verbose logging isn’t just about seeing more output; it’s about unlocking a real-time, step-by-step execution trace of Skaffold’s entire build and deploy lifecycle, revealing exactly where and why your containerized application is failing.
Let’s see this in action. Imagine you’re trying to build your Docker image with Skaffold, and it’s failing. You’ve tried skaffold build and it just hangs or gives a cryptic error.
skaffold build
Now, let’s add the verbose flag:
skaffold build -v debug
Suddenly, your terminal explodes with information. You’re not just seeing "build failed." You’re seeing Skaffold communicate with your Docker daemon, execute docker build, and critically, you’ll see the exact output from the docker build command itself, including any build errors from your Dockerfile. This level of detail is what lets you pinpoint if the issue is with your Dockerfile syntax, a missing dependency, or a misconfigured Docker build context.
The mental model Skaffold operates on is a pipeline: Build -> (Test) -> Deploy. Verbose logging lets you inspect each stage.
Build Stage:
When you run skaffold build, Skaffold first looks at your skaffold.yaml to determine your build strategy (e.g., Docker, Jib, Bazel). If it’s Docker, it generates the docker build command.
-
What to look for in verbose logs:
- The exact
docker buildcommand Skaffold is attempting to run. This includes thecontextdirectory, theDockerfilepath, and thetags. - The output stream from the
docker buildcommand. This is where you’ll see the line-by-line progress of your image build and any specific errors.
- The exact
-
Common Issues and Fixes:
-
Docker Daemon Not Running or Unreachable:
- Diagnosis: You’ll see messages like
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?or similar network errors. - Fix: Ensure your Docker daemon is running. On Linux,
sudo systemctl status dockerandsudo systemctl start docker. On macOS or Windows, check the Docker Desktop application. - Why it works: Skaffold (or the underlying build tool like
docker build) needs to communicate with the Docker daemon to build images. If it can’t reach it, the build will fail.
- Diagnosis: You’ll see messages like
-
Incorrect Docker Build Context:
- Diagnosis: Errors in the
docker buildoutput likeopen /path/to/your/app/Dockerfile: no such file or directoryorCOPY failed: no such file or directory. - Fix: In your
skaffold.yaml, ensure thecontextfield for your Docker builder points to the correct directory containing yourDockerfileand application code. For example:build: docker: context: ./backend # This should be the directory with your Dockerfile - Why it works: The
contexttells Docker where to look for theDockerfileand what directory to consider the root forCOPYandADDcommands.
- Diagnosis: Errors in the
-
Dockerfile Syntax Errors:
- Diagnosis: The
docker buildoutput will explicitly state syntax errors, e.g.,Error parsing reference: "my-image:latest" not a valid repository nameorunknown instruction: RUNN echo hello. - Fix: Correct the syntax errors directly in your
Dockerfile. - Why it works: Dockerfile instructions have specific formats. Incorrect syntax prevents the image from being built.
- Diagnosis: The
-
Missing Build Dependencies (e.g.,
RUN apt-get updatefailed):- Diagnosis: The
docker buildoutput will show a failedRUNcommand, often with an error code from the package manager (e.g.,E: Unable to locate package ...orThe command '/bin/sh -c apt-get update && apt-get install -y some-package' returned a non-zero code: 100). - Fix: Ensure your
RUNcommands that install dependencies are correct and that the package repositories are accessible. Often, aRUN apt-get updateorRUN apk updatebefore installing packages resolves this.RUN apt-get update && apt-get install -y --no-install-recommends \ some-package \ another-package \ && rm -rf /var/lib/apt/lists/* - Why it works: The
RUNcommand executes commands within the image build environment. If those commands fail, the build stops.
- Diagnosis: The
-
Tagging Issues:
- Diagnosis: Errors related to image tagging, such as
Error: Invalid tag name "my-app:latest.1"ortagging failed: invalid reference format. - Fix: Ensure your image tags conform to Docker’s naming conventions (lowercase alphanumeric characters, periods, underscores, and hyphens). Skaffold can also auto-tag for you; if you’re manually specifying tags in
skaffold.yamlor via CLI flags, double-check their validity.build: artifacts: - image: my-app tagPolicy: git: {} # Skaffold will use git commit SHA - Why it works: Docker requires specific formats for image names and tags.
- Diagnosis: Errors related to image tagging, such as
-
Deploy Stage:
After a successful build, Skaffold moves to deployment. Verbose logging here shows Skaffold interacting with your Kubernetes cluster.
-
What to look for in verbose logs:
- The Kubernetes manifests Skaffold is applying.
- The
kubectl applyor equivalent commands being executed. - The responses from the Kubernetes API server.
-
Common Issues and Fixes:
-
Kubernetes Context Not Set or Incorrect:
- Diagnosis:
Error: no configuration found at ...orUnable to connect to the server: dial tcp ...: connection refusedwhen Skaffold tries to communicate with the cluster. - Fix: Ensure your
kubectlcontext is correctly set to the cluster you intend to deploy to. Runkubectl config current-contextto check andkubectl config use-context <your-context-name>to switch. - Why it works: Skaffold uses
kubectl(or its own internal client) to interact with the Kubernetes API. Ifkubectlisn’t configured to point to a valid, running cluster, Skaffold can’t deploy.
- Diagnosis:
-
RBAC Permissions Issues:
- Diagnosis:
Error from server (Forbidden): ...messages from the Kubernetes API server, indicating the service account Skaffold is using (or your currentkubectluser) lacks necessary permissions to create, update, or patch resources. - Fix: Grant the appropriate Role-Based Access Control (RBAC) permissions. This typically involves creating a
RoleorClusterRoleand binding it to the user or service account using aRoleBindingorClusterRoleBinding. For example, to allow creating Deployments and Services:apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: namespace: default name: skaffold-deployer rules: - apiGroups: ["apps"] resources: ["deployments"] verbs: ["create", "get", "list", "watch", "update", "patch", "delete"] - apiGroups: [""] resources: ["services"] verbs: ["create", "get", "list", "watch", "update", "patch", "delete"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: skaffold-deploy-binding namespace: default subjects: - kind: User # Or ServiceAccount if using one name: your-user-name # Or service account name apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: skaffold-deployer apiGroup: rbac.authorization.k8s.io - Why it works: Kubernetes secures its API with RBAC. If the identity Skaffold uses doesn’t have permission to perform the requested operation (like creating a Deployment), the API server will reject it with a
Forbiddenerror.
- Diagnosis:
-
Invalid Kubernetes Manifests:
- Diagnosis: Skaffold will report that
kubectl applyfailed, and the detailed error from Kubernetes will indicate invalid YAML syntax, missing required fields (likeapiVersionorkind), or incorrect field values (e.g., invalid labels). - Fix: Correct your Kubernetes manifest files (
.yamlor.json). Usekubectl apply --dry-run=client -f your-manifest.yamlto validate syntax before Skaffold runs. - Why it works: Kubernetes expects manifests to adhere to a strict schema. Errors in the manifest prevent the API server from creating or updating the resources.
- Diagnosis: Skaffold will report that
-
Resource Conflicts (e.g., Service with same name):
- Diagnosis:
Error from server (AlreadyExists): ...when trying to create a resource that already exists (and isn’t being updated). - Fix: If you intend to re-deploy, ensure your manifests are idempotent (Skaffold’s
kubectl buildanddeployusually handle this well by default usingapply). If a resource truly shouldn’t exist, you might need to manually delete it (kubectl delete <resource-type> <resource-name>) or investigate why it wasn’t cleaned up. - Why it works: Kubernetes prevents the creation of duplicate resources with the same name in the same namespace.
- Diagnosis:
-
Image Pull Issues on the Cluster:
- Diagnosis: Pods stuck in
ImagePullBackOfforErrImagePullstatus.kubectl describe pod <pod-name>will show specific error messages likeFailed to pull image "your-registry/your-image:tag": rpc error: code = Unknown desc = Error response from daemon: manifest unknown: manifest unknown. - Fix:
- Registry Authentication: Ensure your Kubernetes cluster has the correct
imagePullSecretsconfigured for private registries. - Image Tag: Verify that the image tag Skaffold is deploying actually exists in your registry and matches what your deployment expects. Double-check
skaffold.yamland your image tags. - Registry Reachability: Ensure your cluster’s nodes can reach your container registry.
- Registry Authentication: Ensure your Kubernetes cluster has the correct
- Why it works: Kubernetes nodes need to pull your application image to run the containers. If they can’t find the image, can’t authenticate to the registry, or the registry is unreachable, the pods will fail to start.
- Diagnosis: Pods stuck in
-
When you run skaffold dev -v debug, you’ll see Skaffold rebuild your image, push it, and then apply the updated manifest to your cluster, all while showing you the detailed output of each step.
The next thing you’ll likely encounter after debugging build and deploy issues is a problem with your application’s runtime behavior within the cluster, which might require diving into Kubernetes logs (kubectl logs) or using Skaffold’s port-forwarding features.