Building a Real Tekton CI/CD Pipeline on AWS EKS

When I started this project, my goal was not simply to get an application running in Kubernetes.

I wanted to understand the entire delivery chain.

I have spent enough time around technology to know that there is a big difference between being able to say, “Tekton builds containers and Kubernetes runs them,” and actually understanding what happens between a developer committing code and a running application appearing in a production-like environment.

So I deliberately built this project in small steps.

The final application is not particularly complicated. It is a Streamlit application that sends prompts to OpenRouter, which then routes the request to an AI model.

The interesting part is everything that happens before the application runs.

The final architecture looks roughly like this:

Developer
   |
   | git push
   v
GitHub
   |
   | clone source
   v
Tekton on EKS
   |
   | BuildKit
   v
Container Image
   |
   | push
   v
Amazon ECR
   |
   | immutable image digest
   v
Kubernetes Deployment
   |
   v
EKS Pod
   |
   v
Streamlit Application
   |
   | HTTPS API call
   v
OpenRouter
   |
   v
AI Model

What made this project valuable to me was learning what every arrow in that diagram actually means.

Starting Simple

The project began with an intentionally trivial Flask application.

It returned:

Hello from Tekton on EKS!

That simplicity was useful. Eventually it got replaced with a Streamlit application I had written previously and had done a previous post on.

When learning an infrastructure system, I do not want application complexity mixed into infrastructure troubleshooting. If the application consists of a few lines of Python, then problems involving containers, IAM, storage, networking, Kubernetes, or Tekton are easier to isolate.

The Flask application was first tested locally.

Then it was packaged into a Docker container.

Then the container was tested locally.

Only after those pieces worked did I move into AWS.

That became an important rule for the project:

Never automate something until I understand what is being automated.

Creating the Kubernetes Environment

The Kubernetes environment was an Amazon EKS cluster with two managed EC2 worker nodes.

EKS provides the Kubernetes control plane, while the EC2 instances provide the compute resources where Pods actually run.

Conceptually:

Amazon EKS Control Plane
        |
        | schedules workloads
        v
+-------------------+
| EC2 Worker Node 1 |
+-------------------+

+-------------------+
| EC2 Worker Node 2 |
+-------------------+

Once the cluster existed, I could use  from my local workstation to communicate with the Kubernetes API.

One lesson that became very clear was that there are multiple identities involved in a system like this.

My local administrative identity uses AWS IAM Identity Center and temporary AWS credentials.

Tekton workloads use a completely different identity mechanism.

And Kubernetes deployment permissions are controlled through Kubernetes RBAC.

Those identities should not be casually mixed together.

Installing Tekton

Tekton runs inside Kubernetes.

That sounds obvious now, but it is an important mental model.

Tekton is not some separate CI server sitting beside Kubernetes.

A Tekton Task ultimately becomes Kubernetes Pods running containers that perform work.

A simplified view is:

Tekton Pipeline
      |
      v
Tekton Task
      |
      v
TaskRun
      |
      v
Kubernetes Pod
      |
      v
Container executes command

The fundamental Tekton objects I worked with were:

Task — a reusable definition of work.

TaskRun — one execution of a Task.

Pipeline — a set of Tasks connected into a workflow.

PipelineRun — one execution of a Pipeline.

I initially created very simple Tasks that printed messages. That may sound trivial, but it made the Tekton object model much easier to understand before introducing Git, container builds, IAM, and Kubernetes deployments.

Sharing Files Between Tekton Tasks

One of the first interesting infrastructure problems was this:

The Git clone Task downloads source code.

The build Task needs that same source code.

But those Tasks execute in different Pods.

Pods are ephemeral. A file created inside one Pod does not automatically appear inside another Pod.

Tekton solves this using Workspaces.

In my implementation, the Workspace was backed by a Kubernetes PersistentVolumeClaim.

The storage path became:

Git Clone Task
      |
      | writes source
      v
Tekton Workspace
      |
      v
PersistentVolumeClaim
      |
      v
Amazon EBS Volume
      |
      | source reused
      v
Build Task

I installed the AWS EBS CSI driver and created a gp3 StorageClass so Kubernetes could dynamically provision EBS storage.

This was one of the points where several abstractions finally connected for me.

Tekton calls it a Workspace.

Kubernetes sees a PersistentVolumeClaim.

AWS ultimately provides an EBS block-storage volume.

Different layers, same underlying storage path.

Cloning a Private GitHub Repository

My GitHub repository is private, so Tekton needed a way to authenticate.

I created a dedicated read-only GitHub deploy key.

The public key was registered with GitHub.

The private key was stored in a Kubernetes Secret.

The clone Task received that Secret through a Tekton Workspace.

The authentication path became:

Tekton Clone Task
       |
       v
Kubernetes Secret
       |
       | SSH private key
       v
GitHub
       |
       | authenticated read-only access
       v
Private Repository

I also pinned GitHub's SSH host key using known_hosts, rather than simply disabling SSH host verification.

That distinction is important.

Authentication answers:

Who am I connecting as?

Host verification answers:

Am I really connecting to GitHub?

Both matter.

Building the Container With BuildKit

After cloning the repository, the next Task builds the container.

I chose BuildKit, specifically its rootless mode.

Tekton passes the cloned repository to BuildKit through the shared Workspace.

BuildKit reads the application's Dockerfile and produces an OCI-compatible container image.

The flow becomes:

GitHub Repository
       |
       v
Tekton Clone Task
       |
       v
Shared Workspace
       |
       v
BuildKit Task
       |
       v
Container Image

One particularly educational failure happened here.

The clone Task initially used a restrictive umask because I wanted to protect the SSH private key.

Unfortunately, that restrictive permission setting also affected the cloned source files.

The rootless BuildKit process could no longer read the Dockerfile.

The resulting error looked like a BuildKit problem, but the actual cause was upstream file permissions.

The fix was to keep the restrictive permissions while handling the SSH credentials, then reset the umask before cloning the repository.

That failure taught me something much more useful than merely getting another successful run:

In a pipeline, an error often appears one stage later than the configuration mistake that caused it.

Giving Tekton Permission to Push to ECR

The build Task needed to push container images into Amazon Elastic Container Registry.

I did not want AWS credentials stored in Kubernetes Secrets.

Instead, I used EKS Pod Identity.

A Kubernetes ServiceAccount named tekton-build is associated with an AWS IAM role.

That role has narrowly scoped permissions allowing it to push images only to approved ECR repositories.

The trust chain is approximately:

Tekton Build Pod
       |
       v
Kubernetes ServiceAccount
tekton-build
       |
       v
EKS Pod Identity
       |
       v
AWS IAM Role
       |
       v
ECR permissions

This means the Pod obtains temporary AWS credentials automatically.

There is no permanent AWS access key embedded in Tekton.

That is a much better security model.

Connecting Source Code to an Artifact

Initially, I could build and push an image, but a question immediately appeared:

How do I know which source code created that particular image?

The answer was to use the Git commit SHA.

The clone Task now calculates the exact Git commit it checked out and returns two Tekton Results:

Full SHA:
7055f590b3bd60a98d20cf3f200b6b1ae5317079

Short SHA:
7055f59

The short SHA becomes part of the ECR tag:

streamlit-openrouter-lab:git-7055f59

Now I have human-readable traceability:

Git commit
7055f59
    |
    v
Container tag
git-7055f59

That is useful, but it still does not completely solve artifact identity.

Tags can move.

A container tag is a label.

It is not the strongest possible statement about exactly which artifact Kubernetes should execute.

Deploying by Immutable Image Digest

This led to one of my favorite improvements in the project.

BuildKit produces an image digest after pushing the image to ECR.

A digest looks like:

sha256:d69c2270ea601fde64342dc7b89e823018182e48...

That digest identifies the exact OCI image manifest.

The build Task returns the digest as another Tekton Result.

The Pipeline then gives the deployment Task an image reference in this form:

repository@sha256:...

instead of:

repository:tag

The result is:

Git SHA
   |
   | human traceability
   v
git-7055f59
   |
   v
BuildKit
   |
   | exact artifact identity
   v
sha256:d69c2270...
   |
   v
Kubernetes Deployment

The tag tells me where the image came from.

The digest tells Kubernetes exactly which image to run.

That distinction is subtle but extremely important in software supply-chain security.

Deploying to Kubernetes

The final Tekton Task updates the Kubernetes Deployment.

This Task does not need AWS permissions.

Instead, it runs under a separate Kubernetes ServiceAccount called tekton-deploy.

That ServiceAccount is controlled by Kubernetes RBAC.

I restricted it so that it can patch the specific application Deployment but cannot delete it.

The permissions model becomes:

Build Task
  |
  +--> AWS identity
       ECR push permissions


Deploy Task
  |
  +--> Kubernetes identity
       Deployment patch permission

I like this design because the two Tasks have different responsibilities and therefore different identities.

The build Task does not need Kubernetes deployment privileges.

The deploy Task does not need AWS ECR credentials.

That is least privilege expressed directly in the pipeline architecture.

Replacing the Demo With a Real Application

Once the entire pipeline worked using the Flask application, I replaced the demo workload with something real.

I already had a small Streamlit application that used OpenRouter.

The application allows a user to enter a prompt, sends it to OpenRouter, and displays the AI response.

Instead of rebuilding the pipeline, I reused it.

The source repository changed.

The ECR repository changed.

The application port changed from Flask's port 5000 to Streamlit's port 8501.

But the underlying delivery architecture remained the same.

That was an important moment because the pipeline began to feel like reusable infrastructure rather than something built specifically for one application.

Separating Configuration From Code

The Streamlit application needs an OpenRouter API key.

Obviously, I did not want that key stored in GitHub or baked into the container image.

Locally, the application reads the key from a .env file.

In Kubernetes, the key comes from a Kubernetes Secret.

The Python code simply asks the environment for:

OPENROUTER_API_KEY

The application does not really care how that variable got there.

That gives me:

Local Development

.env
  |
  v
Environment Variable
  |
  v
Application

and:

AWS / EKS

Kubernetes Secret
  |
  v
Environment Variable
  |
  v
Application

The same application code works in both environments.

This is a simple example of externalizing configuration rather than hard-coding environment-specific values into the application.

Exposing the Application

Inside Kubernetes, Pods are disposable.

Their IP addresses can change whenever they are replaced.

A Kubernetes Service gives the application a stable network endpoint.

The Service forwards traffic to Pods based on labels.

For the Streamlit application:

Kubernetes Service
port 80
    |
    | targetPort
    v
Streamlit Pod
port 8501

A second Service of type LoadBalancer caused AWS to provision an Internet-facing load balancer.

The complete traffic path became:

Browser
   |
   | HTTP port 80
   v
AWS Load Balancer
   |
   v
Kubernetes Service
   |
   | targetPort 8501
   v
Streamlit Pod

I managed to prove this architecture by breaking it accidentally.

The public Service was still forwarding traffic to the old Flask port, 5000.

The Streamlit Pod was listening on 8501.

The application was running.

The Pod was healthy.

The load balancer existed.

But the browser received an empty response.

Changing the Service targetPort from 5000 to 8501 immediately fixed the application.

That was a great troubleshooting exercise because each layer was technically working while the connection between two layers was wrong.

Restricting Public Access

Initially the public load balancer could potentially be reached from anywhere on the Internet.

For a learning environment, I did not want an unauthenticated Streamlit application using my OpenRouter API account exposed indefinitely.

Kubernetes supports loadBalancerSourceRanges.

I restricted the LoadBalancer Service to the public IP addresses associated with my Internet connections (I have a primary and a backup Internet connection).

That immediately produced another useful test.

I happened to be connected to a VPN.

The application stopped working.

When I disconnected the VPN, my source IP changed back to one of the permitted addresses and the application worked again.

That proved the access restriction was actually being enforced.

The public endpoint still uses HTTP rather than HTTPS, so TLS remains intentionally unfinished.

That belongs in the security-hardening phase of this project.

Troubleshooting Was Half the Project

The successful architecture diagram looks wonderfully clean:

GitHub
   ↓
Tekton
   ↓
BuildKit
   ↓
ECR
   ↓
EKS
   ↓
Streamlit
   ↓
OpenRouter

Reality looked more like this:

wrong file permissions
wrong Git deploy key
expired AWS SSO session
wrong Service port
local YAML different from live Kubernetes object
image tag pointing at the wrong ECR repository
missing container image
shell PATH problems

And that was probably the most valuable part of the project.

At one point I edited the Tekton Pipeline YAML to point to the new ECR repository but forgot to apply the changed manifest to Kubernetes.

The PipelineRun successfully built the Streamlit application — and pushed it into the old repository.

Nothing was “broken.”

The system simply executed the configuration that actually existed in Kubernetes rather than the configuration sitting on my workstation.

That was an excellent reminder of the difference between:

Desired configuration in a local file

and:

Actual configuration running in the cluster

The Finished Pipeline

At the end of this phase, I had moved from a few lines of Flask to a real AI-backed web application deployed through a working CI/CD pipeline.

The resulting flow is:

Developer commits code
        |
        v
Private GitHub repository
        |
        v
Tekton clones exact Git commit
        |
        v
Git SHA captured
        |
        v
BuildKit creates container image
        |
        v
Image pushed to Amazon ECR
        |
        v
Image digest captured
        |
        v
Tekton deploys exact digest
        |
        v
Kubernetes creates Pod on EKS
        |
        v
Kubernetes injects OpenRouter secret
        |
        v
Service exposes Streamlit
        |
        v
User submits prompt
        |
        v
OpenRouter returns AI response

And I tested the entire path.

The application successfully returned an original AI response while running inside EKS.

At that point I considered the functional pipeline complete.

But functional and secure are not the same thing.

There are still significant security questions:

How do I scan the source code?

How do I scan dependencies?

How do I detect committed secrets?

How do I produce an SBOM?

How do I sign the image?

How do I verify provenance?

How do I prevent Kubernetes from running an untrusted image?

How should network traffic be restricted?

How should TLS be implemented?

What should be logged and audited?

Where should security gates stop a deployment?

Those questions are substantial enough that I do not want to bury them at the end of this article.

They deserve their own project phase — and their own article.

Part 1 was about making the pipeline work.

Part 2 will be about deciding whether I should trust it and the work done to lock it down completely will be in the next post on this site.

• AI