Setting reverse_proxy and reverse_proxy_addresses behind AWS ELBs

Torenware's picture

I've recently launched a Drupal 7 site using auto-scaling on AWS. The basic architecture is 2+ ELB that handle SSL, talking to multiple auto-scaled instances running nginx + php-fpm. Mostly, it's working.

I have noticed that the IP addresses logged for clients are in the 10.0.0.0/8 private network that Amazon uses. At first, I thought that HTTP_X_FORWARDED_FOR wasn't set, but it turns out that it is both set and correct. The problem is that settings.php wasn't set up.

I've now read the notes in setting.php, and am a bit confused.

It looks a little scary:

<?php
/<strong>
*
Reverse Proxy Configuration:
*
*
Reverse proxy servers are often used to enhance the performance
* of heavily visited sites and may also provide other site caching,
*
security, or encryption benefits. In an environment where Drupal
* is behind a reverse proxy, the real IP address of the client should
* be determined such that the correct client IP address is available
* to Drupal's logging, statistics, and access management systems. In
* the most simple scenario, the proxy server will add an
* X-Forwarded-For header to the request that contains the client IP
* address. However, HTTP headers are vulnerable to spoofing, where a
* malicious client could bypass restrictions by setting the
* X-Forwarded-For header directly. Therefore, Drupal'
s proxy
* configuration requires the IP addresses of all remote proxies to be
* specified in $conf['reverse_proxy_addresses'] to work correctly.
*
*
Enable this setting to get Drupal to determine the client IP from
* the X-Forwarded-For header (or $conf['reverse_proxy_header'] if set).
* If
you are unsure about this setting, do not have a reverse proxy,
* or
Drupal operates in a shared hosting environment, this setting
* should remain commented out.
*
*
In order for this setting to be used you must specify every possible
* reverse proxy IP address in $conf['reverse_proxy_addresses'].
* If
a complete list of reverse proxies is not available in your
* environment (for example, if you use a CDN) you may set the
* $_SERVER['REMOTE_ADDR'] variable directly in settings.php.
*
Be aware, however, that it is likely that this would allow IP
* address spoofing unless more advanced precautions are taken.
*/
# $conf['reverse_proxy'] = TRUE;

/</strong>
*
Specify every reverse proxy IP address in your environment.
*
This setting is required if $conf['reverse_proxy'] is TRUE.
*/
# $conf['reverse_proxy_addresses'] = array('a.b.c.d', ...);
?>

My first question is: how scared should I really be if executing inside of an AWS security group?

I'm not completely sure if I can do what's asked for here. I'm assuming that for reverse_proxy_addresses, I need the 10.0.0.0/8 IP addresses of my ELBs (although that isn't completely clear to me from the notes in settings.php). In general, can I even depend on these remaining stable?

What's best practice for getting ip_address() to return the right address for clients on AWS?

Comments

You can do that at the server level

perusio's picture

without touching the Drupal config.

See Nginx realip modules. It allows to define trusted hosts for obtaining the client address.

If I understood correctly your setup. This should do it:

set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;

Note that to use

omega8cc's picture

Note that to use set_real_ip_from trick you would have to compile Nginx with non-default option --with-http_realip_module. Alternatively, to have full chain of IPs/proxies logged in the Nginx access log and sent to all backends, make sure that you have proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; set in the front-end Nginx proxy and then in all backends you are using $proxy_add_x_forwarded_for in the log_format. As for the Drupal part, just allow it to use the last IP from the X-Forwarded-For chain, as explained in the settings.php comments. Since you control the front-end proxy, it is safe to define it as trusted.

Peter Bowey's picture

According to this AWS page, the 'private IP range' [per RFC1918], can be: 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16

The HttpRealipModule is useful if nginx works behind some proxy of load balancer, and the request comes from a local IP, but proxy add request header with client's IP.

Example:

set_real_ip_from   192.168.1.0/24;
set_real_ip_from   192.168.2.1;
real_ip_header     X-Real-IP;

You can check if Nginx was built with the http_realip_module by using:

nginx -V

--
Linux: Web Developer
Peter Bowey Computer Solutions
Australia: GMT+9:30
(¯`·..·[ Peter ]·..·´¯)

Simple solution

sonnykt's picture

Let's have a look at the function ip_address() in bootstrap.inc

<?php
function ip_address() {
...
   
$ip_address = $_SERVER['REMOTE_ADDR'];

...
       
// Turn XFF header into an array.
       
$forwarded = explode(',', $_SERVER[$reverse_proxy_header]);
...

       
// Tack direct client IP onto end of forwarded array.
       
$forwarded[] = $ip_address;

       
// Eliminate all trusted IPs.
       
$untrusted = array_diff($forwarded, $reverse_proxy_addresses);

       
// The right-most IP is the most specific we can trust.
       
$ip_address = array_pop($untrusted);
...

  return
$ip_address;
}
?>

Drupal appends the REMOTE_ADDR (the IP of the ELB) to the end of the forwarded IPs, then pop the right-most IP. Because we don't have the list of trusted IPs of ELB set in $reverse_proxy_addresses, Drupal always returns the ELB address instead of the forwarded one. The simple solution for this issue is just to set the REMOTE_ADDR to the array reverse_proxy_addresses in settings.php
<?php
$conf
['reverse_proxy_addresses'] = array($_SERVER['REMOTE_ADDR']);
?>

Its really a simple one and good solution.

jawad.shah's picture

Thanks sonnykt, Its really a simple one and good solution.

Won't this blindly accept

Driskell's picture

Won't this blindly accept forwarded addresses from other people's EC2 instances?

From what I can gather, although the addresses 10.0, 172.16 etc are internal to AWS, if any other AWS customer creates a machine and accesses your site, it will use that internal address.

So this only protests from spoofing outside of AWS.

The 10. and 172. ones are

michaelraasch's picture

The 10. and 172. ones are only accessible from within your own VPC, so it is fine.

Ah I see now. If you have a

Driskell's picture

Ah I see now. If you have a new AWS account it will be VPC only so the internal IP are yours.
However, I was working on an old environment in EC2-Classic and those are shared subnet with other customers so I guess it's unsafe there, but generally nobody should be using that anymore... Thanks.

Thanks sonnykt, it helped me

jm_drupal's picture

Thanks sonnykt, it helped me to resolve the issue. In my case i had to change the 'reverse_proxy_header' as below.

<?php
$conf
['reverse_proxy'] = TRUE;
$conf['reverse_proxy_addresses'] = array($_SERVER['REMOTE_ADDR']);
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
?>