Setting Up a Private CI/CD Pipeline in Azure—Part 2

By /
Listen to this article:
0:00
0:00

Note: This article was adapted from content originally written on October 19th, 2017, titled “Setting up a Private CI/CD Solution in Azure.” It has been simplified and split into four parts for easier reading.

Part 2: Azure Infrastructure and Networking Setup

Key Takeaways

  • This article discusses setting up Azure infrastructure for a private CI/CD solution, including resource groups and networking.
  • It details creating resource groups like Spacely-Engineering-Network and configuring a virtual network with specific settings.
  • The guide includes steps for configuring subnets, network security groups, and setting up a VPN gateway.
  • Additionally, it covers the creation of Docker Swarm manager and worker VMs for hosting CI/CD services.
  • The next part focuses on configuring core services for the CI/CD pipeline.

In this part, we’ll walk through the detailed steps of setting up the Azure infrastructure that forms the foundation of our private CI/CD solution. This includes creating resource groups, configuring virtual networks, setting up network security groups, and establishing VPN connectivity.

Azure Setup

Step 1: Create Resource Groups

Resource groups in Azure provide a logical container for managing related resources. We’ll create multiple resource groups to organize our infrastructure effectively.

  1. Navigate to Resource Groups in the Azure portal and click Add.
  2. Create the following resource groups:
Resource Group NamePurposeRegion
Spacely-Engineering-NetworkNetwork infrastructure resourcesEast US 2
Spacely-Engineering-VPNVPN gateway and related resourcesEast US 2
Spacely-Engineering-VMVirtual machines and compute resourcesEast US 2
Spacely-Engineering-Load-BalancersLoad balancing infrastructureEast US 2
Tip
Choose a region close to your primary users for optimal performance. Ensure all resources in a resource group are in the same region to minimize latency and data transfer costs.

Step 2: Create the Virtual Network

  1. Navigate to Virtual Networks and click Add.
  2. Configure the virtual network with these settings:
    • Name: Spacely-Engineering-Virtual-Network
    • Address Space: 10.0.0.0/20
    • Resource Group: Spacely-Engineering-Network
    • Location: East US 2
  3. Click Create to provision the virtual network.

Step 3: Configure Subnets

After creating the virtual network, we need to configure multiple subnets for different purposes:

  1. Open your virtual network and navigate to Subnets.
  2. Create the following subnets:
Subnet NameAddress RangePurpose
GatewaySubnet10.0.255.224/27VPN Gateway (required name)
DMZ10.0.250.0/24Load balancers and exposed services
Private-Network-110.0.0.0/24Docker Swarm managers
Private-Network-210.0.1.0/24Docker Swarm workers
Important
The GatewaySubnet must be named exactly as shown — Azure requires this specific name for VPN gateway functionality.

Step 4: Configure Network Security Groups

Network Security Groups (NSGs) act as virtual firewalls, controlling inbound and outbound traffic to resources in your virtual network.

Create the Main NSG:

  1. Navigate to Network Security Groups and click Add.
  2. Configure with:
    • Name: Spacely-Engineering-NSG
    • Resource Group: Spacely-Engineering-Network
    • Location: East US 2
  3. After creation, configure the following inbound security rules:
PriorityNamePortProtocolSourceDestinationAction
100Allow-VPN-ClientsAnyAny172.16.0.0/24VirtualNetworkAllow
110Allow-Internal-CommunicationAnyAnyVirtualNetworkVirtualNetworkAllow
120Allow-Azure-LoadBalancerAnyAnyAzureLoadBalancerAnyAllow
65000Deny-All-InboundAnyAnyAnyAnyDeny

VPN Gateway Configuration

The VPN gateway enables secure connection to your private network from external locations.

Step 1: Create Public IP Address

  1. Navigate to Public IP addresses and click Add.
  2. Configure with:
    • Name: Spacely-Engineering-VPN-Public-IP
    • SKU: Basic
    • Assignment: Dynamic
    • Resource Group: Spacely-Engineering-VPN

Step 2: Create the VPN Gateway

  1. Navigate to Virtual Network Gateways and click Add.
  2. Configure the gateway:
    • Name: Spacely-Engineering-Private-Gateway
    • Gateway Type: VPN
    • VPN Type: Route-based
    • SKU: VpnGw1
    • Virtual Network: Spacely-Engineering-Virtual-Network
    • Public IP Address: Spacely-Engineering-VPN-Public-IP
    • Resource Group: Spacely-Engineering-VPN
  3. Click Create (this can take 30-45 minutes to provision).
Note
VPN Gateway provisioning is a lengthy process. You can continue with VM creation while waiting for the gateway to complete.

Step 3: Configure Point-to-Site Connection

After the gateway is created, configure point-to-site connectivity:

  1. Open your VPN gateway and navigate to Point-to-site configuration.
  2. Configure the following:
    • Address Pool: 172.16.0.0/24 (for VPN clients)
    • Tunnel Type: SSTP & IKEv2
    • Authentication Type: Azure certificate
  3. Generate and configure certificates for authentication (refer to Azure documentation for detailed certificate steps).
  4. Download the VPN client configuration package for distribution to users.

Virtual Machine Configuration

Now let’s create the virtual machines that will host our CI/CD services.

Creating Docker Swarm Manager VMs

For each Docker Swarm manager (create 3 for high availability):

  1. Navigate to Virtual Machines and click Add.
  2. Configure the basic settings:
    • Name: Spacely-Engineering-VM-00X (where X is 1, 2, or 3)
    • VM Disk Type: SSD
    • Username: spacely-eng-admin
    • Authentication: SSH public key (recommended) or password
    • Resource Group: Spacely-Engineering-VM
  3. Select VM size:
    • Size: Standard DS2 v2 (2 vCPUs, 7 GB RAM)
  4. Configure settings:
    • Storage: Use managed disks
    • Virtual Network: Spacely-Engineering-Virtual-Network
    • Subnet: Private-Network-1
    • Public IP: None
    • Network Security Group: Spacely-Engineering-NSG
  5. Enable boot diagnostics for troubleshooting.
  6. Review and create the VM.

Creating Docker Swarm Worker VMs

For the Docker Swarm workers (create 2 for build capacity):

  1. Follow the same process as managers with these differences:
    • Name: Spacely-Engineering-VM-00X (where X is 4 or 5)
    • Size: Standard DS3 v2 (4 vCPUs, 14 GB RAM) — workers need more resources for builds
    • Subnet: Private-Network-2 (for VM-005)

Load Balancer Configuration

We’ll create internal load balancers for our services to ensure high availability.

GitLab Load Balancer

  1. Navigate to Load Balancers and click Add.
  2. Configure:
    • Name: Spacely-Engineering-GitLab-LB
    • Type: Internal
    • SKU: Standard
    • Virtual Network: Spacely-Engineering-Virtual-Network
    • Subnet: DMZ
    • IP Address Assignment: Static (10.0.250.10)
    • Resource Group: Spacely-Engineering-Load-Balancers
  3. Configure backend pool:
    • Add all three Docker Swarm manager VMs
  4. Configure health probe:
    • Protocol: HTTP
    • Port: 10080
    • Path: /
  5. Configure load balancing rule:
    • Frontend Port: 80
    • Backend Port: 10080
    • Protocol: TCP

Repeat this process for:

  • Jenkins Load Balancer: IP 10.0.250.11, ports 80→18080
  • Docker Registry Load Balancer: IP 10.0.250.12, ports 443→5000

VM Initial Configuration

Once your VMs are running, perform initial configuration on each:

Next Steps

With the Azure infrastructure and networking foundation in place, we’re ready to move on to configuring the core services that will power our CI/CD pipeline.

Continue to Part 3: Docker Swarm and Core Services Configuration, where we’ll set up Docker Swarm, configure GitLab, and establish our private Docker Registry.

This is Part 2 of a 4-part series on setting up a private CI/CD solution in Azure.

Start typing and press enter to search

Table of Contents

→ Table of Contents
Scroll to Top