Using CloudFront + WAF

Problem Statement

1. Operator requires WAF (Web Application Firewall) in front of their workloads running on OpenShift (ROSA)

2. Operator does not want WAF running on OpenShift to ensure that OCP resources do not experience Denial of Service through handling the WAF

Proposed Solution

1. Add a CustomDomain resource to the cluster using a wildcard DNS and TLS certificate.

2. Set the Wildcard DNS CNAME’s to CloudFront and enable the CloudFront + WAF services to reverse proxy and inspect the traffic before sending it to the cluster.

Preparation

1. Create a cluster

   rosa create cluster --cluster-name poc-waf --multi-az \
     --region us-east-2 --version 4.7.9 --compute-nodes 3 \
     --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 \
     --pod-cidr 10.128.0.0/14 --host-prefix 23
   

2. When its ready create a admin user and follow the instructions to log in

   rosa create admin -c poc-waf
   

3. Set some environment variables

   EMAIL=username.taken@gmail.com
   DOMAIN=waf.mobb.ninja
   

Certificate and DNS

1. Use certbot to create a wildcard cert

   certbot certonly --manual \
     --preferred-challenges=dns \
     --email $EMAIL \
     --server https://acme-v02.api.letsencrypt.org/directory \
     --agree-tos \
     --manual-public-ip-logging-ok \
     -d "*.$DOMAIN"
   

2. Follow Certbot’s instructions to create a DNS TXT record. certificate records will be saved on your system, in my case in /etc/letsencrypt/live/waf.mobb.ninja/. set that as an enviroment variable.

   CERTS=/etc/letsencrypt/live/waf.mobb.ninja
   

Custom OpenShift Domain

1. Create a project and add the certs as a secret

   oc new-project my-custom-route
   oc create secret tls acme-tls --cert=$CERTS/fullchain1.pem --key=$CERTS/privkey1.pem
   

2. Create a Custom Domain resource. (You can change the load balancer from the default Classic by adding spec.loadBalancerType: NLB to the following YAML)

   cat << EOF | oc apply -f -
   apiVersion: managed.openshift.io/v1alpha1
   kind: CustomDomain
   metadata:
     name: acme
   spec:
     domain: $DOMAIN
     loadBalancerType: Classic
     certificate:
       name: acme-tls
       namespace: my-custom-route
   EOF
   

3. Wait until your Custom Domain has an Endpoint

   watch  oc get customdomains
   

AWS WAF + CloudFront

1. Create a WAF rule here https://console.aws.amazon.com/wafv2/homev2/web-acls/new?region=us-east-2 and use the Core and SQL Injection rules and set it as a CloudFront distribution resource type.

2. View your WAF

   aws wafv2 list-web-acls --scope REGIONAL --region us-east-2 | jq .
   

3. Add a certificate to ACM - https://us-east-2.console.aws.amazon.com/acm/home?region=us-east-1#/importwizard/ , Paste in the cert, key, certchain from the files certbot game you.

   Make sure you create it in the US-EAST-1 region (otherwise cloud front can’t use it)

4. Log into the AWS console and Create a Cloud Front distribution

   • Origin Domain Name: < Endpoint from oc get manageddomains command >
   • Origin Protocol Policy: HTTPS only
   • Viewer Protocol Policy: Redirect HTTP to HTTPS
   • Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE
   • AWS WAF Web ACL: demo-waf-acl
   • Alternate Domain Names: *.< domain >
   • Custom SSL Certificate: < the one you just imported >
   • Origin Request Policy: create a new policy whitelist: Origin, user-agent, referer, host (IMPORTANT)

5. Hit Create then wait until the Status is Ready.

DNS CNAME

1. Create a CNAME in your DNS provider for *.<$DOMAIN> that points at the endpoint from the above status page. It should look something like d1vm7mfs9sc24l.cloudfront.net.

Deploy an Application

1. Create a new application

   oc new-app --docker-image=docker.io/openshift/hello-openshift
   

2. Create a route for the application

   oc create route edge --service=hello-openshift hello-openshift-tls \
       --hostname hello.waf.mobb.ninja
   

Test the WAF

1. Make sure you can access your application with curl

   curl https://hello.waf.mobb.ninja
   

   You should get a simple hello response

   Hello OpenShift!
   

2. Try do a XSS injection

   curl -X POST https://hello.waf.mobb.ninja \
     -F "user='<script><alert>Hello></alert></script>'"
   

   you should see this

   <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
   <HTML><HEAD><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
   <TITLE>ERROR: The request could not be satisfied</TITLE>
   </HEAD><BODY>
   <H1>403 ERROR</H1>