AWS PrivateLink: Secure Cross-Account Service Connectivity for Consulting Firms

Introduction

In today's cloud-first world, consulting firms and their clients often need to share services across AWS accounts. Whether you're a consulting company providing managed services to your customers, or a client consuming specialized tools from your consulting partner, the challenge remains the same: how do you securely connect services across AWS accounts without exposing traffic to the public internet?

Enter AWS PrivateLink with VPC Endpoint Services—a powerful networking solution that enables private connectivity between VPCs across different AWS accounts. In this article, we'll explore how PrivateLink works and walk through real-world scenarios where consulting firms and their clients can leverage this technology.

What is AWS PrivateLink?

AWS PrivateLink is a networking technology that provides private connectivity between VPCs, AWS services, and on-premises networks. Unlike VPC peering or internet gateways, PrivateLink keeps all traffic within the AWS network backbone, ensuring:

• Enhanced Security: Traffic never traverses the public internet
• Simplified Network Architecture: No need for VPC peering, NAT gateways, or internet gateways
• Scalability: Supports thousands of connections without IP address conflicts
• Granular Access Control: Service providers control who can access their services

The key components of PrivateLink are:

• VPC Endpoint Service (Producer): The service provider creates this to expose their application
• VPC Endpoint (Consumer): The service consumer creates this to access the provider's service
• Network Load Balancer: Sits in front of the producer's application to distribute traffic

Real-World Scenario 1: Consulting Firm Providing Services to Clients

Imagine a consulting firm called CloudExperts Inc. has built a proprietary cost optimization platform that runs in their AWS account. One of their clients, RetailCorp, wants to access this platform from RetailCorp's own AWS account—but without exposing the platform to the public internet.

In this scenario:

• CloudExperts Inc. is the service provider (producer) - they own and operate the cost optimization platform
• RetailCorp is the service consumer - they want to use CloudExperts' platform from their own AWS account

Architecture Overview

Here's how CloudExperts and RetailCorp would set this up:

1. CloudExperts (Producer Account):
   • Deploy the cost optimization API behind a Network Load Balancer (NLB)
   • Create a VPC Endpoint Service pointing to the NLB
   • Configure allowed principals (RetailCorp's AWS account ID)
2. RetailCorp (Consumer Account):
   • Create a VPC Endpoint in their VPC
   • Specify CloudExperts' VPC Endpoint Service name
   • Request connection (requires CloudExperts' approval)

Infrastructure as Code Templates

To make deployment easier, we've created ready-to-use CloudFormation and Terraform templates. Download and customize them for your environment:

Note: All templates include a comprehensive README with deployment instructions, prerequisites, and troubleshooting tips. View README

Step-by-Step Implementation (AWS CLI)

Producer Side (CloudExperts Inc.)

# 1. Create Network Load Balancer (via AWS Console or CLI)
aws elbv2 create-load-balancer \
  --name cost-optimization-nlb \
  --type network \
  --subnets subnet-12345678 subnet-87654321 \
  --scheme internal

# 2. Create Target Group and register your application instances
aws elbv2 create-target-group \
  --name cost-opt-targets \
  --protocol TCP \
  --port 443 \
  --vpc-id vpc-producer123

# 3. Create VPC Endpoint Service
aws ec2 create-vpc-endpoint-service-configuration \
  --network-load-balancer-arns arn:aws:elasticloadbalancing:us-east-1:111122223333:loadbalancer/net/cost-optimization-nlb/abc123 \
  --acceptance-required

# 4. Note the Service Name (you'll share this with RetailCorp)
# Example: com.amazonaws.vpce.us-east-1.vpce-svc-0123456789abcdef0

# 5. Add allowed principals (RetailCorp's account)
aws ec2 modify-vpc-endpoint-service-permissions \
  --service-id vpce-svc-0123456789abcdef0 \
  --add-allowed-principals arn:aws:iam::444455556666:root

Consumer Side (RetailCorp)

# 1. Create VPC Endpoint
aws ec2 create-vpc-endpoint \
  --vpc-id vpc-consumer456 \
  --vpc-endpoint-type Interface \
  --service-name com.amazonaws.vpce.us-east-1.vpce-svc-0123456789abcdef0 \
  --subnet-ids subnet-aabbccdd subnet-eeffgghh \
  --security-group-ids sg-consumer123

# 2. Wait for CloudExperts to accept the connection request

# 3. Once accepted, retrieve the endpoint DNS name
aws ec2 describe-vpc-endpoints \
  --vpc-endpoint-ids vpce-0987654321fedcba0

# 4. Your applications can now access the service via the endpoint DNS
# Example: vpce-0987654321fedcba0-abcd1234.vpce-svc-0123456789abcdef0.us-east-1.vpce.amazonaws.com

How Does RetailCorp's Data Get Analyzed?

An important question: How does CloudExperts' platform access RetailCorp's AWS data?

PrivateLink only provides the network connectivity between accounts. The actual data access happens through one of these methods:

Option 1: API-Based Access (Most Common)

RetailCorp's applications call CloudExperts' cost optimization API via the PrivateLink endpoint, sending their AWS cost data as API requests:

• RetailCorp exports their AWS Cost and Usage Reports to S3
• RetailCorp's application reads the data and sends it to CloudExperts' API via PrivateLink
• CloudExperts' platform processes the data and returns optimization recommendations
• Key benefit: RetailCorp controls exactly what data is shared

Option 2: Cross-Account IAM Roles

CloudExperts' platform assumes an IAM role in RetailCorp's account to read data directly:

• RetailCorp creates an IAM role with read-only access to Cost Explorer and billing data
• RetailCorp grants CloudExperts' AWS account permission to assume this role
• CloudExperts' platform (accessed via PrivateLink) assumes the role to fetch data
• Key benefit: Automated data collection without manual exports

Option 3: Hybrid Approach (Scenario 1 + Scenario 2)

Combine both scenarios for bidirectional communication:

• Scenario 1 (this section): RetailCorp connects to CloudExperts' platform via PrivateLink
• Scenario 2 (next section): CloudExperts connects to RetailCorp's data API via a separate PrivateLink connection
• Result: Fully private, bidirectional communication between both accounts

Important: PrivateLink provides the secure network path, but you still need to implement proper authentication (API keys, IAM roles, OAuth) and authorization at the application level.

Security Considerations

• Security Groups: Configure security groups on both sides to allow only necessary traffic
• Endpoint Policies: Use VPC endpoint policies to restrict which actions consumers can perform
• Connection Approval: Enable acceptance-required to manually approve each connection request
• Application Authentication: Always implement authentication (API keys, IAM, OAuth) at the application layer
• Monitoring: Use CloudWatch to monitor endpoint connections and traffic patterns

Real-World Scenario 2: Client Providing Data to Consulting Firm

Now let's flip the scenario. RetailCorp has a data lake in their AWS account, and they want CloudExperts Inc. to access it for analytics and reporting—without exposing the data to the public internet.

In this reversed scenario:

• RetailCorp is now the service provider (producer) - they own the data lake and control access to it
• CloudExperts Inc. is now the service consumer - they need to access RetailCorp's data for analysis

Architecture Overview

Here's how the roles reverse:

1. RetailCorp (Producer Account):
   • Deploy an API Gateway or application server that provides access to the data lake
   • Place a Network Load Balancer in front of the API
   • Create a VPC Endpoint Service
   • Whitelist CloudExperts' AWS account
2. CloudExperts (Consumer Account):
   • Create a VPC Endpoint to RetailCorp's service
   • Configure analytics tools to query data via the private endpoint

Benefits of This Approach

• Data Sovereignty: RetailCorp maintains complete control over their data
• Compliance: Meets regulatory requirements for data privacy (HIPAA, GDPR, etc.)
• Performance: Low-latency access within AWS network
• Cost Efficiency: No data transfer charges for traffic within the same region

Advanced Use Cases

Multi-Client Architecture

If you're a consulting firm serving multiple clients, you can create a single VPC Endpoint Service and allow multiple consumer accounts to connect. Each client gets their own VPC Endpoint, and you can:

• Use different target groups for client isolation
• Implement application-level authentication to distinguish between clients
• Monitor usage per client using CloudWatch metrics

Hybrid Cloud Connectivity

PrivateLink can also work with on-premises networks via AWS Direct Connect or VPN:

• Client's on-premises data center connects to their AWS VPC via Direct Connect
• VPC Endpoint in the client's VPC connects to your service
• On-premises applications access your AWS service privately

Cost Considerations

Understanding the cost structure of PrivateLink is essential for budgeting:

• VPC Endpoint (Consumer): $0.01 per hour per AZ (~$7.30/month per AZ)
• Data Processing: $0.01 per GB processed through the endpoint
• Network Load Balancer (Producer): $0.0225 per hour (~$16.43/month) + LCU charges
• No Data Transfer Charges: Within the same region, data transfer is free

Example Monthly Cost: For a consulting firm serving 5 clients with moderate traffic (100 GB/month per client):

• NLB: $16.43
• VPC Endpoint Service: Free
• Client-side costs (per client): $14.60 (2 AZs) + $1.00 (data processing) = $15.60
• Total for consulting firm: $16.43/month
• Total per client: $15.60/month

Best Practices

1. Use Multiple Availability Zones: Deploy your NLB and VPC Endpoints across multiple AZs for high availability
2. Implement Connection Approval: Always require manual approval for new connections to maintain security
3. Monitor with CloudWatch: Set up alarms for connection failures, high latency, or unusual traffic patterns
4. Document Service Names: Maintain a registry of VPC Endpoint Service names for your clients
5. Use Private DNS: Enable private DNS names for easier application configuration
6. Implement TLS: Always use TLS/SSL for encrypted communication, even though traffic is private
7. Test Failover: Regularly test your multi-AZ setup to ensure seamless failover

Common Pitfalls to Avoid

• Forgetting Security Groups: Both the NLB and VPC Endpoint need properly configured security groups
• Not Enabling Private DNS: This can make application configuration more complex
• Overlooking Endpoint Policies: Use endpoint policies to restrict access to specific actions
• Single AZ Deployment: Always deploy across multiple AZs for resilience
• Ignoring Monitoring: Set up CloudWatch dashboards and alarms from day one

Troubleshooting Tips

If your PrivateLink connection isn't working:

1. Check Connection Status: Ensure the connection request has been accepted by the producer
2. Verify Security Groups: Confirm that security groups allow traffic on the required ports
3. Test DNS Resolution: Use nslookup or dig to verify the endpoint DNS resolves correctly
4. Check NLB Health: Ensure target instances are healthy in the NLB target group
5. Review VPC Route Tables: Verify that route tables don't have conflicting routes
6. Enable VPC Flow Logs: Use Flow Logs to diagnose connectivity issues

Conclusion

AWS PrivateLink with VPC Endpoint Services provides a secure, scalable, and cost-effective solution for cross-account connectivity between consulting firms and their clients. Whether you're a consulting company providing managed services or a client sharing data with your consulting partner, PrivateLink eliminates the security risks and complexity of internet-based connectivity.

Key takeaways:

• PrivateLink keeps all traffic within the AWS network, never touching the public internet
• The producer creates a VPC Endpoint Service; the consumer creates a VPC Endpoint
• Both parties maintain complete control over their network security
• The solution scales to support multiple clients without IP address conflicts
• Costs are predictable and reasonable for most use cases

By implementing PrivateLink, you're not just improving security—you're building a foundation for trusted, long-term partnerships between consulting firms and their clients in the cloud.