Skaffold’s verbose logging isn’t just about seeing more output; it’s about unlocking a real-time, step-by-step execution trace of Skaffold’s entire build and deploy lifecycle, revealing exactly where and why your containerized application is failing.

Let’s see this in action. Imagine you’re trying to build your Docker image with Skaffold, and it’s failing. You’ve tried skaffold build and it just hangs or gives a cryptic error.

skaffold build

Now, let’s add the verbose flag:

skaffold build -v debug

Suddenly, your terminal explodes with information. You’re not just seeing "build failed." You’re seeing Skaffold communicate with your Docker daemon, execute docker build, and critically, you’ll see the exact output from the docker build command itself, including any build errors from your Dockerfile. This level of detail is what lets you pinpoint if the issue is with your Dockerfile syntax, a missing dependency, or a misconfigured Docker build context.

The mental model Skaffold operates on is a pipeline: Build -> (Test) -> Deploy. Verbose logging lets you inspect each stage.

Build Stage:

When you run skaffold build, Skaffold first looks at your skaffold.yaml to determine your build strategy (e.g., Docker, Jib, Bazel). If it’s Docker, it generates the docker build command.

  • What to look for in verbose logs:

    • The exact docker build command Skaffold is attempting to run. This includes the context directory, the Dockerfile path, and the tags.
    • The output stream from the docker build command. This is where you’ll see the line-by-line progress of your image build and any specific errors.
  • Common Issues and Fixes:

    1. Docker Daemon Not Running or Unreachable:

      • Diagnosis: You’ll see messages like Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? or similar network errors.
      • Fix: Ensure your Docker daemon is running. On Linux, sudo systemctl status docker and sudo systemctl start docker. On macOS or Windows, check the Docker Desktop application.
      • Why it works: Skaffold (or the underlying build tool like docker build) needs to communicate with the Docker daemon to build images. If it can’t reach it, the build will fail.
    2. Incorrect Docker Build Context:

      • Diagnosis: Errors in the docker build output like open /path/to/your/app/Dockerfile: no such file or directory or COPY failed: no such file or directory.
      • Fix: In your skaffold.yaml, ensure the context field for your Docker builder points to the correct directory containing your Dockerfile and application code. For example:
        build:
          docker:
            context: ./backend # This should be the directory with your Dockerfile
        
      • Why it works: The context tells Docker where to look for the Dockerfile and what directory to consider the root for COPY and ADD commands.
    3. Dockerfile Syntax Errors:

      • Diagnosis: The docker build output will explicitly state syntax errors, e.g., Error parsing reference: "my-image:latest" not a valid repository name or unknown instruction: RUNN echo hello.
      • Fix: Correct the syntax errors directly in your Dockerfile.
      • Why it works: Dockerfile instructions have specific formats. Incorrect syntax prevents the image from being built.
    4. Missing Build Dependencies (e.g., RUN apt-get update failed):

      • Diagnosis: The docker build output will show a failed RUN command, often with an error code from the package manager (e.g., E: Unable to locate package ... or The command '/bin/sh -c apt-get update && apt-get install -y some-package' returned a non-zero code: 100).
      • Fix: Ensure your RUN commands that install dependencies are correct and that the package repositories are accessible. Often, a RUN apt-get update or RUN apk update before installing packages resolves this.
        RUN apt-get update && apt-get install -y --no-install-recommends \
            some-package \
            another-package \
            && rm -rf /var/lib/apt/lists/*
        
      • Why it works: The RUN command executes commands within the image build environment. If those commands fail, the build stops.
    5. Tagging Issues:

      • Diagnosis: Errors related to image tagging, such as Error: Invalid tag name "my-app:latest.1" or tagging failed: invalid reference format.
      • Fix: Ensure your image tags conform to Docker’s naming conventions (lowercase alphanumeric characters, periods, underscores, and hyphens). Skaffold can also auto-tag for you; if you’re manually specifying tags in skaffold.yaml or via CLI flags, double-check their validity.
        build:
          artifacts:
          - image: my-app
            tagPolicy:
              git: {} # Skaffold will use git commit SHA
        
      • Why it works: Docker requires specific formats for image names and tags.

Deploy Stage:

After a successful build, Skaffold moves to deployment. Verbose logging here shows Skaffold interacting with your Kubernetes cluster.

  • What to look for in verbose logs:

    • The Kubernetes manifests Skaffold is applying.
    • The kubectl apply or equivalent commands being executed.
    • The responses from the Kubernetes API server.
  • Common Issues and Fixes:

    1. Kubernetes Context Not Set or Incorrect:

      • Diagnosis: Error: no configuration found at ... or Unable to connect to the server: dial tcp ...: connection refused when Skaffold tries to communicate with the cluster.
      • Fix: Ensure your kubectl context is correctly set to the cluster you intend to deploy to. Run kubectl config current-context to check and kubectl config use-context <your-context-name> to switch.
      • Why it works: Skaffold uses kubectl (or its own internal client) to interact with the Kubernetes API. If kubectl isn’t configured to point to a valid, running cluster, Skaffold can’t deploy.
    2. RBAC Permissions Issues:

      • Diagnosis: Error from server (Forbidden): ... messages from the Kubernetes API server, indicating the service account Skaffold is using (or your current kubectl user) lacks necessary permissions to create, update, or patch resources.
      • Fix: Grant the appropriate Role-Based Access Control (RBAC) permissions. This typically involves creating a Role or ClusterRole and binding it to the user or service account using a RoleBinding or ClusterRoleBinding. For example, to allow creating Deployments and Services:
        apiVersion: rbac.authorization.k8s.io/v1
        kind: Role
        metadata:
          namespace: default
          name: skaffold-deployer
        rules:
        - apiGroups: ["apps"]
          resources: ["deployments"]
          verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
        - apiGroups: [""]
          resources: ["services"]
          verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
        ---
        apiVersion: rbac.authorization.k8s.io/v1
        kind: RoleBinding
        metadata:
          name: skaffold-deploy-binding
          namespace: default
        subjects:
        - kind: User # Or ServiceAccount if using one
          name: your-user-name # Or service account name
          apiGroup: rbac.authorization.k8s.io
        roleRef:
          kind: Role
          name: skaffold-deployer
          apiGroup: rbac.authorization.k8s.io
        
      • Why it works: Kubernetes secures its API with RBAC. If the identity Skaffold uses doesn’t have permission to perform the requested operation (like creating a Deployment), the API server will reject it with a Forbidden error.
    3. Invalid Kubernetes Manifests:

      • Diagnosis: Skaffold will report that kubectl apply failed, and the detailed error from Kubernetes will indicate invalid YAML syntax, missing required fields (like apiVersion or kind), or incorrect field values (e.g., invalid labels).
      • Fix: Correct your Kubernetes manifest files (.yaml or .json). Use kubectl apply --dry-run=client -f your-manifest.yaml to validate syntax before Skaffold runs.
      • Why it works: Kubernetes expects manifests to adhere to a strict schema. Errors in the manifest prevent the API server from creating or updating the resources.
    4. Resource Conflicts (e.g., Service with same name):

      • Diagnosis: Error from server (AlreadyExists): ... when trying to create a resource that already exists (and isn’t being updated).
      • Fix: If you intend to re-deploy, ensure your manifests are idempotent (Skaffold’s kubectl build and deploy usually handle this well by default using apply). If a resource truly shouldn’t exist, you might need to manually delete it (kubectl delete <resource-type> <resource-name>) or investigate why it wasn’t cleaned up.
      • Why it works: Kubernetes prevents the creation of duplicate resources with the same name in the same namespace.
    5. Image Pull Issues on the Cluster:

      • Diagnosis: Pods stuck in ImagePullBackOff or ErrImagePull status. kubectl describe pod <pod-name> will show specific error messages like Failed to pull image "your-registry/your-image:tag": rpc error: code = Unknown desc = Error response from daemon: manifest unknown: manifest unknown.
      • Fix:
        • Registry Authentication: Ensure your Kubernetes cluster has the correct imagePullSecrets configured for private registries.
        • Image Tag: Verify that the image tag Skaffold is deploying actually exists in your registry and matches what your deployment expects. Double-check skaffold.yaml and your image tags.
        • Registry Reachability: Ensure your cluster’s nodes can reach your container registry.
      • Why it works: Kubernetes nodes need to pull your application image to run the containers. If they can’t find the image, can’t authenticate to the registry, or the registry is unreachable, the pods will fail to start.

When you run skaffold dev -v debug, you’ll see Skaffold rebuild your image, push it, and then apply the updated manifest to your cluster, all while showing you the detailed output of each step.

The next thing you’ll likely encounter after debugging build and deploy issues is a problem with your application’s runtime behavior within the cluster, which might require diving into Kubernetes logs (kubectl logs) or using Skaffold’s port-forwarding features.

Want structured learning?

Take the full Skaffold course →