Skip to main content

Networks (Stage 3)

Repository: gcp-foundations-networks

Purpose​

The networks stage creates the full network infrastructure: a centralized DNS hub, dual Shared VPCs (base and restricted) for each environment, hierarchical firewall policies, Cloud NAT, Private Service Connect, VPC Service Controls perimeters, and a JSON-driven tenant subnet system. Like the environments stage, it uses a branch-per-environment deployment model.

DNS Hub Architecture​

The DNS hub provides centralized DNS resolution across all environments. It is deployed in the shared (common) configuration and lives in prj-c-dns-hub.

DNS Hub VPC​

SubnetRegionCIDR
sb-c-dns-hub-northamerica-northeast1northamerica-northeast1172.16.0.0/25
sb-c-dns-hub-us-west1us-west1172.16.0.128/25
  • Inbound forwarding is enabled via DNS policy for on-premises resolution
  • Cloud Routers (4 total) advertise the DNS proxy range 35.199.192.0/19
  • Forwarding zone routes queries for the organization domain to on-premises name servers
  • A route to 199.36.153.8/30 enables Private Google API access

Dual Shared VPC Model​

Each environment has two Shared VPC host projects, creating a clear separation between standard and sensitive workloads:

VPC TypePurposeVPC-SCInternet Access
BaseStandard workloadsNoCloud NAT egress
RestrictedSensitive/regulated workloadsYes (perimeter)Private Google Access only

Per-Environment VPC Pair​

Complete IP Addressing Scheme​

Base Shared VPC​

EnvironmentRegionPrimary CIDRGKE Pod CIDRGKE Service CIDRPrivate ServiceProxy
Developmentnorthamerica-northeast110.0.64.0/18100.64.64.0/18100.65.64.0/1810.16.8.0/2110.18.2.0/23
Developmentus-west110.1.64.0/18------10.19.2.0/23
Non-Productionnorthamerica-northeast110.0.128.0/18100.64.128.0/18100.65.128.0/1810.16.16.0/2110.18.4.0/23
Non-Productionus-west110.1.128.0/18------10.19.4.0/23
Productionnorthamerica-northeast110.0.192.0/18100.64.192.0/18100.65.192.0/1810.16.24.0/2110.18.6.0/23
Productionus-west110.1.192.0/18------10.19.6.0/23

Restricted Shared VPC​

EnvironmentRegionPrimary CIDRGKE Pod CIDRGKE Service CIDRPrivate ServiceProxy
Developmentnorthamerica-northeast110.8.64.0/18100.72.64.0/18100.73.64.0/1810.16.40.0/2110.26.2.0/23
Developmentus-west110.9.64.0/18------10.27.2.0/23
Non-Productionnorthamerica-northeast110.8.128.0/18100.72.128.0/18100.73.128.0/1810.16.48.0/2110.26.4.0/23
Non-Productionus-west110.9.128.0/18------10.27.4.0/23
Productionnorthamerica-northeast110.8.192.0/18100.72.192.0/18100.73.192.0/1810.16.56.0/2110.26.6.0/23
Productionus-west110.9.192.0/18------10.27.6.0/23

Private Service Connect​

EnvironmentBase PSC IPRestricted PSC IP
Development10.17.0.210.17.0.6
Non-Production10.17.0.310.17.0.7
Production10.17.0.410.17.0.8

IP Address Space Summary​

RangePurpose
10.0.0.0/8Base VPC primary subnets
10.8.0.0/8Restricted VPC primary subnets
10.16.0.0/12Private Service Access ranges
10.17.0.0/24Private Service Connect IPs
10.18-19.0.0/16Base VPC proxy subnets
10.26-27.0.0/16Restricted VPC proxy subnets
100.64.0.0/10GKE Pod ranges (base)
100.65.0.0/10GKE Service ranges (base)
100.72.0.0/10GKE Pod ranges (restricted)
100.73.0.0/10GKE Service ranges (restricted)
172.16.0.0/24DNS Hub subnets

VPC Service Controls​

VPC Service Controls are applied to the restricted Shared VPC only, creating a security perimeter around sensitive workloads.

Perimeter Configuration​

Each environment's restricted VPC has its own service perimeter:

module "regular_service_perimeter" {
  source = "terraform-google-modules/vpc-service-controls/google//modules/regular_service_perimeter"

  policy         = var.access_context_manager_policy_id
  perimeter_name = "sp_${env}_shared_restricted_default_perimeter_*"
  resources      = [var.project_number]  # Restricted VPC host project
  access_levels  = [module.access_level_members.name]

  restricted_services     = var.restricted_services
  vpc_accessible_services = ["RESTRICTED-SERVICES"]

  ingress_policies = var.ingress_policies
  egress_policies  = var.egress_policies
}
  • Access levels control which identities can access resources inside the perimeter
  • Ingress/egress policies are configurable per environment (e.g., Databricks control plane IPs)
  • A 60-second propagation delay is built in after VPC creation

Hierarchical Firewall Policies​

A hierarchical firewall policy is created and associated with all major folders:

Associations: common, network, bootstrap, development, production, non-production folders

Firewall Rules​

RuleDirectionActionPrioritySource/Dest RangesPorts
Delegate RFC1918 ingressINGRESSgoto_next50010.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16all
Delegate RFC1918 egressEGRESSgoto_next51010.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16all
Windows KMS activationEGRESSallow510035.190.247.13/32tcp:1688
Google LB + Health ChecksINGRESSallow520035.191.0.0/16, 130.211.0.0/22, 209.85.152.0/22, 209.85.204.0/22tcp:80,443

The RFC1918 delegation rules use goto_next action, which passes matching traffic to VPC-level firewall rules for more granular control. This allows each VPC to define its own rules for internal traffic.

Tenant Subnet System​

Tenants can request subnets in the Shared VPCs using a JSON-driven configuration system. Subnet definitions are stored in JSON files under tenant_subnets/base/ and tenant_subnets/restricted/ directories.

JSON Subnet Format​

{
  "development": {
    "environment": "development",
    "tags": {
      "business-unit": "data"
    },
    "subnets": {
      "primary": {
        "region": "northamerica-northeast1",
        "primary_cidr": "10.4.0.0/24",
        "secondary_ranges": [
          {
            "range_prefix": "gke-pod",
            "ip_cidr_range": "100.68.0.0/18"
          },
          {
            "range_prefix": "gke-svc",
            "ip_cidr_range": "100.69.0.0/18"
          }
        ]
      }
    }
  },
  "non-production": { ... },
  "production": { ... }
}

How Tenant Subnets Work​

  1. JSON files are placed in tenant_subnets/base/ or tenant_subnets/restricted/ (one file per tenant, named {tenant}_subnets.json)
  2. The tenant_subnets.tf module reads all JSON files using fileset() and jsondecode()
  3. Subnets are created per environment using the environment key in the JSON
  4. Tag bindings are automatically applied to subnets based on the tenant's environment and custom tags
  5. Proxy subnets (for load balancers) are automatically created for additional regions beyond the defaults
  6. Subnet naming follows the pattern: sb-{env_code}-{vpc_type}-{tenant_name}-{region}

Tag-Based Subnet Access​

Tenant subnets use GCP resource tags for access control. Tags are resolved from the organization's tag hierarchy and bound to each subnet:

resource "google_tags_location_tag_binding" "subnet_tags" {
  parent    = "//compute.googleapis.com/projects/{project_num}/regions/{region}/subnetworks/{subnet_id}"
  tag_value = each.value.tag_value_id
  location  = each.value.region
}

This enables tag-based IAM conditions, allowing tenant service accounts to use only their designated subnets.

Cloud NAT​

Each Shared VPC has Cloud NAT configured for outbound internet access:

  • 2 NAT routers per region (region1 and region2) for each VPC
  • Static external IP addresses allocated per router
  • NAT is enabled by default for base VPCs (base_vpc_nat_enabled = true)
  • Restricted VPCs can optionally enable NAT

Private Service Connect​

Private Service Connect (PSC) endpoints are configured in each VPC to provide private access to Google APIs:

  • Each environment and VPC type gets a dedicated PSC IP address
  • Enables private connectivity to Google services without traversing the public internet
  • Configured via the private_service_connect.tf files in each VPC module

Interconnect Options​

The networks stage includes modules for both dedicated and partner interconnect:

Dedicated Interconnect​

Module: modules/dedicated_interconnect/

For organizations with direct physical connections to Google's network. Provides the highest bandwidth (10/100 Gbps) and lowest latency.

Partner Interconnect​

Module: modules/partner_interconnect/

For organizations connecting through a supported service provider. Available in more locations with flexible bandwidth options.

Both interconnect types use the prj-c-interconnect project created in Stage 1.

Directory Structure​

gcp-foundations-networks/
+-- envs/
|   +-- shared/
|   |   +-- dns-hub.tf              # DNS Hub VPC + routing
|   |   +-- hierarchical_firewall.tf # Org-wide firewall policies
|   |   +-- main.tf                 # Shared/common network config
|   |   +-- remote.tf               # Remote state data sources
|   +-- development/
|   |   +-- main.tf                 # Dev IP ranges + base_env call
|   |   +-- outputs_tenant_subnets.tf
|   +-- non-production/
|   |   +-- main.tf                 # Non-prod IP ranges
|   +-- production/
|       +-- main.tf                 # Prod IP ranges
+-- modules/
|   +-- base_env/                   # Per-env orchestrator
|   |   +-- main.tf
|   |   +-- tenant_subnets.tf       # JSON-driven subnet system
|   |   +-- tenant_subnets_output.tf
|   +-- base_shared_vpc/            # Base VPC resources
|   |   +-- main.tf, dns.tf, firewall.tf, nat.tf
|   |   +-- private_service_connect.tf
|   +-- restricted_shared_vpc/      # Restricted VPC resources
|   |   +-- main.tf, dns.tf, firewall.tf, nat.tf
|   |   +-- service_control.tf      # VPC-SC perimeter
|   |   +-- private_service_connect.tf
|   +-- hierarchical_firewall_policy/
|   +-- dedicated_interconnect/
|   +-- partner_interconnect/
|   +-- vpn-ha/
+-- tenant_subnets/
|   +-- base/                       # Base VPC tenant JSON files
|   +-- restricted/                 # Restricted VPC tenant JSON files
+-- proxy_subnets/
    +-- shared_base.json            # Proxy subnet CIDRs per env
    +-- shared_restricted.json