Restoring original visitor ip with Cloudflare and Nginx

Who doesn’t know Cloudflare yet? Cloudflare is one of the most used CDN (Content Delivery Network) providers in the world.

Using a CDN in front of your tipical web server (For example nginx) provides a lot of advantages, like lower response times, increased security by hiding your real server ip address, and in the case of Cloudflare a lot of different features. On the other hand since Cloudflare acts as a reverse proxy some of your apps (Log analyzers, or any other kind of app that requires your visitors ip addresses) might stop working at all since your server won’t be able to get the real ip addresses of your visitors anymore. Cloudflare provides a solution for this in the form of HTTP headers and a list of ip addresses that you will have to update manually.

This can be tedious, and the worse part is that you might forget of doing it regularly, getting less accurate results in your logs. But fortunately, there is another way of doing this. Yeah, that’s right you won’t have to catch those damned birds anymore, err I mean ip addresses.

Do you want to know how? Keep reading.

Step 1. Setting up the realip module (Nginx)

The starting point this time is a properly configured Nginx server (I’m using CentOS 7 this time) and a site that it is actually served through Cloudflare. If you have been keeping an eye on your access.log, you should have noticed that IPs aren’t from your visitors anymore (Just check it if you haven’t checked it yet).

The way we are going to solve this is using the realip module for nginx, and the X-Forwarded-For HTTP header (Or alternatively, CF-Connecting-IP). After copying and paste the list of ip/networks provided by Cloudflare you should end with a configuration file like the following (let’s call it /etc/nginx/cloudflare.conf) that can be included (Using the include directive) in /etc/nginx/nginx.conf:

#Cloudflare Stuff:
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 104.16.0.0/12;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 131.0.72.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2c0f:f248::/32;
set_real_ip_from 2a06:98c0::/29;
real_ip_header X-Forwarded-For;

Don’t worry about this file yet, instead we are going to create an empty file (You can use whatever filename you prefer but remember to change it in the script later):

# touch /etc/nginx/cloudflare.conf

Now let’s modify our nginx.conf to include the previous empty file, in my case it looks like:

...
    # Load modular configuration files from the /etc/nginx/conf.d directory.
    # See http://nginx.org/en/docs/ngx_core_module.html#include
    # for more information.
    include /etc/nginx/cloudflare.conf;
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/default.d/*.conf;
...

Up to this point we have not done any change yet, but we have learned how to manually configure the ipreal module properly for working with Cloudflare. If you have read the official Cloudflare documentation they state that the list of ip/networks can change frequently and it should be updated.We are going to solve this with a bash script in the next step.

Step 2. Letting our server do the work

Now it’s time to use our little script to automate the task of getting the configuration in place with an up to date list of ip addresses. Let’s take a look at the code before executing anything:

#!/bin/bash

###############################################################
# cloudflare ip/network update for nginx real ip module
###############################################################
#
# This script auto-updates the list of ip/networks from cloudflare
# and reloads nginx in case of any change.
#
# First of all create an empty file wherever NGINX_CFG is pointing, and
# launch the script. Don't forget to add a line including the config in
# your nginx config: include /etc/nginx/cloudflare.conf;
#
# Sources:
# https://support.cloudflare.com/hc/en-us/articles/200170706-How-do-I-restore-original-visitor-IP-with-Nginx-
# https://www.cloudflare.com/ips/
#
###############################################################


URL="https://www.cloudflare.com"
IPS="ips-v4 ips-v6"
TMP_CFG="/tmp/cloudflare.conf"
NGINX_CFG="/etc/nginx/cloudflare.conf"
USE_X_FORWARDED_FOR="true"

check_nginx(){
        2>&1 nginx -V | xargs -n1 | grep with-http_realip_module >/dev/null || abort "Please recompile nginx with realip module support"
}

abort() {
        echo "$1";
        exit 1;
}

get() {
        rm -f "$2"
        /usr/bin/wget "$1" -O "$2" >/dev/null 2>&1 || abort "Error downloading $2"
}

replace(){
        cp -f "$1" "$2"
        nginx -t >/dev/null 2>&1 || abort "Something went wrong, please review nginx -t output"
        systemctl reload nginx
}

# Check if nginx has realip module support
check_nginx

# Get the ip lists from cloudflare, fail if something goes wrong
for x in $IPS
do
        get "$URL/$x" "/tmp/$x"
done

# remove the previous tmp file if exists
rm -f $TMP_CFG

# Recreate the file starting with this comment
echo "#Cloudflare Stuff:" >> "$TMP_CFG"

# For each ip/network in each file create a line for the real ip module.
for x in $IPS
do
        for y in `cat "/tmp/$x"`
        do
                echo "set_real_ip_from $y;" >> "$TMP_CFG"
        done
done

# Use the X-Forwarded-For: header, alternatively you can use:
# real_ip_header CF-Connecting-IP;
# use just one.
if [ "$USE_X_FORWARDED_FOR" == "true" ]
then
        echo "real_ip_header X-Forwarded-For;" >> "$TMP_CFG"
else
        echo "real_ip_header CF-Connecting-IP;" >> "$TMP_CFG"
fi

# Compare the generated file with the previous file, if needed replace and reload nginx.
/usr/bin/diff "$NGINX_CFG" "$TMP_CFG" >/dev/null 2>&1 || replace "$TMP_CFG" "$NGINX_CFG"

# Remove the new temporal file
rm -f "$TMP_CFG"

Now you can get the script using git, or just copy and paste it (and chmod +x it):

$ git clone https://github.com/sys-dev-cat/cloudflare-nginx-realip-updater.git

When ready launch the script, and the previously empty file should instead be populated with a list of ip/network addresses and nginx directives, isn’t it beatiful? :). The script also takes care of reloading nginx if there have been changes in the Cloudflare configuration file. When you feel comfortable enough with the script, just add it to your favourite cron daemon and let your server take care of everything.

And that’s it! Let’s get back to more interesting things like why you should swat with your left paw if you are a male cat.