Connect GitHub Actions with AWS VPC

Connect GitHub Actions with AWS VPC

GitHub Actions is my preferred CI/CD solution. I’m using GitHub Actions to build and deploy applications on AWS. However, GitHub Actions does not have access to private subnets, which is required in the following scenarios:

  • Execute database migrations for RDS (Relational Database Service).
  • Run load or integration tests against internal ALBs or NLBs (Elastic Load Balancing).
  • Seed ElasticSearch, OpenSearch, or ElastiCache with data.

[wpcc-element _tag=”source” type=”image/webp” srcset=”/images/2024/01/connect-github-actions-with-aws-vpc-title@730w.webp 730w, /images/2024/01/connect-github-actions-with-aws-vpc-title@730w2x.webp 1460w, /images/2024/01/connect-github-actions-with-aws-vpc-title@610w.webp 610w, /images/2024/01/connect-github-actions-with-aws-vpc-title@610w2x.webp 1220w, /images/2024/01/connect-github-actions-with-aws-vpc-title@450w.webp 450w, /images/2024/01/connect-github-actions-with-aws-vpc-title@450w2x.webp 900w, /images/2024/01/connect-github-actions-with-aws-vpc-title@330w.webp 330w, /images/2024/01/connect-github-actions-with-aws-vpc-title@330w2x.webp 660w, /images/2024/01/connect-github-actions-with-aws-vpc-title@545w.webp 545w, /images/2024/01/connect-github-actions-with-aws-vpc-title@545w2x.webp 1090w” sizes=”(min-width: 1200px) 730px, (min-width: 992px) 610px, (min-width: 768px) 450px, (min-width: 576px) 330px, 545px” _close=”0″]

In the following, I will demonstrate how to access a VPC (Virtual Private Cloud) from GitHub Actions with the help of HyperEnv for GitHub Actions Runner, a solution I built recently.

What are GitHub-hosted runners?

By default, GitHub Actions executes jobs on machines provided by GitHub, so-called GitHub-hosted runners. Each GitHub-hosted runner comes with the runner application and other preinstalled tools and is available with Ubuntu Linux, Windows, or macOS operating systems.

A GitHub-hosted runner is connected to the Internet but cannot access any private networks, like a VPC on AWS. Therefore, reaching resources like an RDS database, an internal ALB, an ElasticSearch/OpenSearch domain, or an ElastiCache instance from within a running job is impossible.

[wpcc-element _tag=”source” type=”image/webp” srcset=”/images/2024/01/github-hosted-runner@730w.webp 730w, /images/2024/01/github-hosted-runner@730w2x.webp 1460w, /images/2024/01/github-hosted-runner@610w.webp 610w, /images/2024/01/github-hosted-runner@610w2x.webp 1220w, /images/2024/01/github-hosted-runner@450w.webp 450w, /images/2024/01/github-hosted-runner@450w2x.webp 900w, /images/2024/01/github-hosted-runner@330w.webp 330w, /images/2024/01/github-hosted-runner@330w2x.webp 660w, /images/2024/01/github-hosted-runner@545w.webp 545w, /images/2024/01/github-hosted-runner@545w2x.webp 1090w” sizes=”(min-width: 1200px) 730px, (min-width: 992px) 610px, (min-width: 768px) 450px, (min-width: 576px) 330px, 545px” _close=”0″]

How to access private networks from GitHub Actions?

There are two options to access private networks from GitHub Actions.

First, establish a tunnel between the GitHub-hosted runner and the private network, for example, by using VPN or SSH.

Second, run GitHub Actions inside your VPC.

How is that possible? GitHub Actions not only provides GitHub-hosted runners but also supports self-hosted runners. As illustrated in the following figure, an EC2 instance acting as a self-hosted runner launched in a VPC allows all jobs running on the machine to connect to resources in private subnets such as RDS, ElasticSearch, OpenSearch, ElastiCache, and more.

[wpcc-element _tag=”source” type=”image/webp” srcset=”/images/2024/01/self-hosted-runner@730w.webp 730w, /images/2024/01/self-hosted-runner@730w2x.webp 1460w, /images/2024/01/self-hosted-runner@610w.webp 610w, /images/2024/01/self-hosted-runner@610w2x.webp 1220w, /images/2024/01/self-hosted-runner@450w.webp 450w, /images/2024/01/self-hosted-runner@450w2x.webp 900w, /images/2024/01/self-hosted-runner@330w.webp 330w, /images/2024/01/self-hosted-runner@330w2x.webp 660w, /images/2024/01/self-hosted-runner@545w.webp 545w, /images/2024/01/self-hosted-runner@545w2x.webp 1090w” sizes=”(min-width: 1200px) 730px, (min-width: 992px) 610px, (min-width: 768px) 450px, (min-width: 576px) 330px, 545px” _close=”0″]

How to deploy a self-hosted GitHub Actions runner on AWS?

But how do you deploy a self-hosted runner on AWS? I’ve previously written about self-hosted GitHub runners on AWS. The tricky part is to come up with a scalable and cost-efficient solution.

That’s why I built HyperEnv for GitHub Actions Runner. The following figure illustrates the solution.

  1. GitHub sends a webhook event when starting a job.
  2. The API Gateway receives the event.
  3. A Lambda function validates the event and sends a message to SQS.
  4. Another Lambda function reads the message from SQS and launches an EC2 instance in an VPC of your choice.
  5. The EC2 instance starts and registers the GitHub runner.
  6. The GitHub runner executes the job.
  7. The EC2 instance terminates itself.

[wpcc-element _tag=”source” type=”image/webp” srcset=”/images/2024/01/hyperenv@730w.webp 730w, /images/2024/01/hyperenv@730w2x.webp 1460w, /images/2024/01/hyperenv@610w.webp 610w, /images/2024/01/hyperenv@610w2x.webp 1220w, /images/2024/01/hyperenv@450w.webp 450w, /images/2024/01/hyperenv@450w2x.webp 900w, /images/2024/01/hyperenv@330w.webp 330w, /images/2024/01/hyperenv@330w2x.webp 660w, /images/2024/01/hyperenv@545w.webp 545w, /images/2024/01/hyperenv@545w2x.webp 1090w” sizes=”(min-width: 1200px) 730px, (min-width: 992px) 610px, (min-width: 768px) 450px, (min-width: 576px) 330px, 545px” _close=”0″]

Deploy HyperEnv for GitHub Actions to your AWS account to enable GitHub Actions jobs to connect with RDS, ElasticSearch, OpenSearch, ElastiCache running in your VPC.

[wpcc-iframe class=”embed-responsive-item lozad” data-src=”https://www.youtube-nocookie.com/embed/w65m4cIAUkc” allow=”accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture” allowfullscreen=””]