Introduction

Using an Azure Container Registry, you can upload and manage individual images of an OCI compliant client (for example, Docker). These can be private and customized images that can be used within Azure App Services or container instances. However, the goal should also be to make these container registry accessible only within the application context and prohibit access from the public network. However, there is usually the problem in the corporate context that only one container registry is used for several logical environments such as the development and the productive environment. Here the ACR must be made accessible over several ways.

In this short post, I want to give you a sample implementation on how to use Terraform to make an ACR accessible across multiple virtual networks. Microsoft already provides detailed instructions on how to implement this via the Azure CLI or the UI. I have used this to write my instructions for Terraform.

Example

In my example scenario there are two different environments that work independently across different subscriptions within Azure - development and production. In each of these environments there is a virtual network dev-vnet and prod-vnet. Docker images are built for web applications thatare stored within ACR. These will then be used in the production environment, but will also be able to be tested beforehand with App Services in the development environment. Therefore, access from both virtual networks to the ACR is necessary.

For my implementation I use Azure Private Link. With this, private connections between different resources can be built and allowed. Recently, the feature has become available for Azure Container Registry as well and can be used to establish private connections to internal virtual networks.

Terraform

Azure Container Registry

To begin, let’s first create the Azure Container Registry itself:

resource "azurerm_container_registry" "acr" {
  name                = var.resource_name
  resource_group_name = var.rg_name
  location            = var.location
  sku                 = "Premium"
  admin_enabled       = true

  network_rule_set {
    default_action = "Deny"
  }

  tags = {
    Terraform   = "true"
    Environment = var.environment
  }
}

By default, we disable any public access to the ACR. The firewall configuration and the connection of a private endpoint is currently only available in the “Premium” tariff.

In each environment, we now need two sets of resources: a private DNS zone and a private endpoint.

Private Endpoint

Let’s start with Private Endpoint:

resource "azurerm_private_endpoint" "pep" {
  name                = format("%s-pep", var.container_registry_name)
  location            = var.location
  resource_group_name = var.rg_name
  subnet_id           = var.pep_subnet_id

  private_service_connection {
    name                           = format("%s-pep-connection", var.container_registry_name)
    private_connection_resource_id = var.container_registry_id
    subresource_names              = ["registry"]
    is_manual_connection           = false
  }
}

A Private Endpoint is used to establish a Private Service Connection to a resource, in our case the Azure Container Registry. The private endpoint automatically creates a network interface in the specified subnet, which we can read as a data element:

data "azurerm_network_interface" "nic" {
  name                = azurerm_private_endpoint.pep.network_interface[0].name
  resource_group_name = var.rg_name

  depends_on = [
    azurerm_private_endpoint.pep
  ]
}

We need the information from this network interface later to be able to enter the DNS entries in the created DNS zone, since the private IP addresses of the ACR within the selected subnet are configured here.

Private DNS zone

resource "azurerm_private_dns_zone" "acr" {
  name                = "privatelink.azurecr.io"
  resource_group_name = var.rg_name
}

The Private DNS zone must be named “privatelink.azurecr.io” to guarantee the functionality and correct forwarding of the Private Link service. The zone names are specified in the Azure documentation.

DNS entries

You can configure DNS settings for the registry’s private endpoints, so that the settings resolve to the registry’s allocated private IP address within the respective virtual network. With DNS configuration, clients and services in the network can continue to access the registry at the registry’s fully qualified domain name, such as myregistry.azurecr.io.

The Azure Container Registry publishes two different entries in the network interface. Once the normal endpoint and the data endpoint. In order to be able to use the FQDN in the internal DNS, these DNS entries are added to the created private dns zone.

resource "azurerm_private_dns_a_record" "pep_dns_record_data" {
  name                = lower(format("%s.%s.data", var.container_registry_name, var.location))
  zone_name           = var.pep_dns_zone_name
  resource_group_name = var.rg_dnszone_name
  ttl                 = 3600
  records             = [data.azurerm_network_interface.nic.private_ip_addresses[0]]
}

resource "azurerm_private_dns_a_record" "pep_dns_record" {
  name                = lower(var.container_registry_name)
  zone_name           = var.pep_dns_zone_name
  resource_group_name = var.rg_dnszone_name
  ttl                 = 3600
  records             = [data.azurerm_network_interface.nic.private_ip_addresses[1]]
}

The network interface returns a list of the private IP addresses of the ACR in its private_ip_addresses attribute. The first is that of the data endpoint and the second is that of the normal endpoint. The IP addresses from the array are then used to create the two new DNS entries for the DNS zone.

Multi environment

All the previous resources would already be sufficient to make an Azure Container Registry available within a virtual network (in my example, the prod-vnet) and without public access to the outside. All created resources are now located in the resources group, where the prod-vnet is also located. This group is called prod-rg in my example However, since I also want to access the container registry within the dev-vnet, further steps are necessary:

Create another Private DNS zone in the dev-rg resource group.
Create another Private Endpoint in the dev-rg connected to a subnet of the dev-vnet.
Creating the two DNS entries for the Private DNS zone in dev-rg.

To simplify this, I created both process steps as a Terraform module. The first module is only used to create the Azure Container Registry (container-registry). This is only created in the environment in which the container registry is to be located. However, since the Private DNS zone (dns-zone) and the Private Endpoint (container-registry-pep) are required in every other environment, I have also written two different modules for this.

The structure should then look like this:

dev-rg
- dns-zone
- container-registry-pep
prod-rg
- dns-zone
- container-registry
- container-registry-pep

After the Private Endpoints are created, it is now possible to access the desired Container Registry via internal networks. This could be easily tested with a virtual machine within the two virtual networks. Using dig or nslookup, myregistry.azurecr.io can then be queried.

dig myregistry.azurecr.io

Outside the virtual networks, the DNS resolution looks like this:

[...]
;; ANSWER SECTION:
myregistry.azurecr.io. 2881 IN CNAME myregistry.privatelink.azurecr.io.
myregistry.privatelink.azurecr.io. 2881 IN CNAME xxxx.xx.azcr.io.
xxxx.xx.azcr.io. 300 IN CNAME xxxx-xxx-reg.trafficmanager.net.
xxxx-xxx-reg.trafficmanager.net. 300 IN CNAME xxxx.westeurope.cloudapp.azure.com.
xxxx.westeurope.cloudapp.azure.com. 10 IN A 20.45.122.144

[...]

The external IP address of the container registry is returned.

Inside the virtual network, the internal IP of the ACR is referenced directly:

[...]
; <<>> DiG 9.11.3-1ubuntu1.13-Ubuntu <<>> myregistry.azurecr.io
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52155
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;myregistry.azurecr.io.         IN      A

;; ANSWER SECTION:
myregistry.azurecr.io.  1783    IN      CNAME   myregistry.privatelink.azurecr.io.
myregistry.privatelink.azurecr.io. 10 IN A      10.0.0.7

[...]

If I could help you with this little tutorial I would be very happy about your support! If you have any questions or suggestions, please feel free to contact me in Matrix. The link is provided on the start page.

Introduction#

Example#

Terraform#

Azure Container Registry#

Private Endpoint#

Private DNS zone#

DNS entries#

Multi environment#