Drupal Security Team response about insecure update process

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
Drupal Security Team's picture

Recently, a security researcher reported some vulnerabilities to the Drupal Security Team. The Security Team and researcher worked together to understand the risks and decided that the potential impact was small enough that the reported problems could be fixed in public and that the researcher would write a blog post with their perspective on the situation.

Below are some quotes of the critical issues from the blog post and the Drupal Security Team’s analysis of the risks.

Issue #1: Whenever the Drupal update process fails, Drupal states that everything is up to date instead of giving a warning.

This is not ideal and should be corrected, but the impact is limited to only one page of the Drupal administrative interface. All other pages in the admin interface warn about failures correctly. Also, the Drupal Security Team publishes advisories in many ways (html, email, rss, Twitter, and this update mechanism), precisely because modern security awareness requires that administrators rely on multiple mechanisms.

Help fix the missing warning at https://www.drupal.org/node/2646306

Issue #2: An attacker may force an admin to check for updates due to a CSRF vulnerability on the update functionality

The cross site request forgery (CSRF) vulnerability allows an attacker to achieve two goals: To control the time that an update is triggered, which may provide value if they are able to sniff traffic or poison dns for a brief period. To use a large amount of resources from Drupal.org. As it is, there are millions of sites making multiple requests per day to Drupal.org, so an attacker would have to engage in a rather aggressive campaign to make any noticeable impact because the update service is cached and behind a CDN. The value to an attacker from straining resources on Drupal.org via this indirect method is not a strong motivation.

Help fix this at https://www.drupal.org/node/2646328

Issue #3: Drupal security updates are transferred unencrypted without checking the authenticity, which could lead to code execution and database access.

This is accurate and in the past few days we have been working to switch infrastructure and update processes to use secure channels. Furthermore, we’ve investigated and concluded the following download channels are or were affected.

  • The core update status module did not use SSL. Fixing is in progress
  • Downloads via drush, all versions, did not use SSL. This is now fixed for supported versions of Drush.
  • Download links on project pages did not use SSL by default, but a user could change to use SSL if they chose. This is now fixed.
  • Our default version control URLs shown to users do not use SSL and SSL is not supported for anonymous copying via revision control, but we are working to to fix this.
  • File checksums were removed from the release nodes. You can still access them by clicking on view all releases from a project page.

Unencrypted file transfer makes it harder than it should be for users to ensure they obtained the intended files. DNS poisoning or man-in-the-middle attacks between Drupal.org and the person or program performing the download could result in a download of unverified code. Both of these would require a bad actor to be able to have control of the network at some point between your site and Drupal.org’s servers.

We have switched the most common download methods to use https by default and are working to add SSL to anonymous downloads via version control (git). The next step is to release an updated Drupal core.

While we are working to secure all remaining download channels, you can safely obtain code over SSL in the following ways:

  • Manually download release archives from their project pages.
  • Use a supported version of drush (7 or higher) to obtain downloads or update information. This will be fully secure after approximately 21:00 UTC on January 8th when rebuilding all the release XML files has been finished

Help fix Drupal core on https://www.drupal.org/node/1538118 and https://www.drupal.org/node/1081192


If you have any questions, please feel free to reach out to security@drupal.org. If you are a member of the press, please use security-press@drupal.org.



Comments

updates.drupal.org HTTPS redirection fails over proxy

magunz's picture

Hi there,
My production sites are hosted in servers where all http requests are trough proxy.
I recently noticed that, all request to http://updates.drupal.org are redirected to https://updates.drupal.org (HTTPS)

Unfortunately the answer 301 (redirect to https) is not handled by drupal_http_request() over proxy and the site is failing checking for updates.

I am just wondering if anyone noticed that issue?

Thank you in advance

P.s. I found a temporary solution using the "cURL HTTP Request" https://www.drupal.org/project/chr which override the default drupal_http_request() with a CURL based one.

Additional: these are (a

dman's picture

Additional: these are (a large number of different) Drupal 7.41 sites that have been maintained for years on homogenous but not identical architecture.
Something in the d.o changes has made them all stop receiving updates and complain about that this week.

It feels like (has been reported to make me believe) that a certain combination of outgoing proxy (enabled in settings.php) + new 301 redirect to HTTPS over drupal_http_request as initiated by the update status on cron ... has started to fail.

As this security advisory implies (but does not clearly state) that there is some change at the infrastructure end that coincides with our observed issue - can we get more info to troubleshoot? Is it true that a change has been applied to UPDATE_DEFAULT_URL http://updates.drupal.org/release-history
( related to http://blog.ioactive.com/2016/01/drupal-insecure-update-process.html ? )

Is it possible that this is a cause of the regression we are seeing?

Being behind a proxy prevents updating

raj45's picture

I can confirm this: I have a few web sites that are also behind a proxy, and no longer receive updates from drupal.org on admin/reports/updates.

Thanks for the confirmation

dman's picture

Thanks for the confirmation (and glad you found this issue report here)
We have been stuck in a small is-it-down-for-everyone-or-just-me? loop yesterday,

Seems like it is a real thing then...

Seems like it may have been nicer if an earlier D7 release had started to prefer the https version (in the coded constant) a while before the server started to hard-redirect to the new endpoint. But yeah, maybe we could not have anticipated this side effect.

Still - I'd like to know where I could have subscribed to get a notification warning about this. It lost our team and our contracted sysadmins half a day yesterday scratching heads.

thank you for confirming

magunz's picture

thank you for confirming

As this security advisory

greggles's picture

As this security advisory implies (but does not clearly state) that there is some change at the infrastructure end that coincides with our observed issue - can we get more info to troubleshoot?

I think all the infrastructure changes are linked from the original article. Here they are again:

  • The core update status module did not use SSL. Fixing is in progress
  • Downloads via drush, all versions, did not use SSL. This is now fixed for supported versions of Drush.
  • Download links on project pages did not use SSL by default, but a user could change to use SSL if they chose. This is now fixed.
  • Our default version control URLs shown to users do not use SSL and SSL is not supported for anonymous copying via revision control, but we are working to to fix this.
  • File checksums were removed from the release nodes. You can still access them by clicking on view all releases from a project page.

Thanks for the details. I did

dman's picture

Thanks for the details. I did see that the drush updates and download link things and other bits were related, but not affecting our exact concern.

The only point that bit us was in the "core update status module did not use SSL. Fixing is in progress" described above.
I didn't see under the "in progress" issue any explicit mention that (or when or if) the d.o response to requests to the URL http://updates.drupal.org/ has changed.

From the outside, it seems like "something" has happened. Today, we now receive a 301 redirect at that address.
AFAIK, that is different from what was happening last year.
Yet I cant find any explicit change notice in the relevant issue ticket https://www.drupal.org/node/1538118 I have looked at. It does NOT describe any committed change made so far - though has hints that changes could be made .

Questions:

  • Is it true that the 301 redirect to 'https' was applied within the last week or so?
  • was there a place I could have looked or been subscribed to to find a change notice about that?
  • is anyone else aware of the potential for (edge case) regression errors we are now seeing as a (seemingly) result of this change?

I'm usually pretty good at researching these sort of things, and giving the benefit of the doubt, but I cannot yet find the actual announcement that http://updates.drupal.org/ had switched to https-only in plaintext anywhere.

If this has happened, please just let us know when or if it did, so we can do a better job of isolating or troubleshooting the problems we seem to be having that seem to be related to this infrastructure change.

It looks like the redirect

David_Rothstein's picture

It looks like the redirect happened via https://www.drupal.org/node/2646894 (a Drupal.org infrastructure issue which I don't think was previous linked here).

That issue has already been reopened by someone and perhaps it should be rolled back if it's causing problems? To the best of my understanding it doesn't really provide any security - to improve security we'd need core to contact the HTTPS URL directly and to verify the cert (and there are core issues open for both of those, with the first obviously being simpler than the second).

I'll comment there too with a link back here.

Thanks for the link to

dman's picture

Thanks for the link to #2646894
That's probably the notification I was unable to identify in my searches up until now.

Recommend chr?

DamienMcKenna's picture

Based upon magunz's comment above it might be worth recommending people try the 'chr' module if their servers are having problems checking for updates, it uses an alternative mechanism for HTTPS connections that says supports proxies.

We can confirm that enabling

dman's picture

We can confirm that enabling the 'chr' module fixes the issue. For now.

I'd suggest that a (one-letter) patch to update the UPDATE_DEFAULT_URL constant in core may also be a reasonable fix.

@dman - no need to patch even

pwolanin's picture

@dman - no need to patch even - there is a Drupal variable you can set to override the default:

variable_get('update_fetch_url', UPDATE_DEFAULT_URL);

Thanks. I'll pass that

dman's picture

Thanks. I'll pass that suggestion back to the maintenance team that may look at applying this (as a $conf via settings.php) to the affected sites.
That looks like an acceptable fix to roll out.

Would it look something like

raj45's picture

Would it look something like this in settings.php?

Drupal 7: $conf['update_fetch_url'] = 'UPDATE_DEFAULT_URL';
Drupal 8: $settings['update_fetch_url'] = 'UPDATE_DEFAULT_URL';

@raj45: I think what you want

greggles's picture

@raj45: I think what you want is

<?php
$conf
['update_fetch_url'] = 'https://updates.drupal.org/release-history';
?>

Or you could also do

<?php
variable_set
('update_fetch_url', 'https://updates.drupal.org/release-history');
?>
in a hook_update or you could do
<?php
drush vset update_fetch_url https
://updates.drupal.org/release-history
?>
.

For what it's worth, I'm using PHP 5.5 on Ubuntu 14.04 and I did some debugging to confirm that the update checks are following the 301 without changing that.

I can confirm that enabling

giupenni's picture

I can confirm that enabling the chr module fixes the issue.

Upgrading to Drush 8.0.1 fixed it

raj45's picture

Thanks @greggles, I tried your suggestions as well as the chr module, but it didn't fix the problem. The solution was for me to upgrade to Drush 8.0.1. I can now access the updates again via Drush, but not through the GUI, though.

CHR didn't fix it for me

veg-o-matic's picture

I noticed last week or so that we couldn't get updates for any of our sites. Stumbled across this thread and tried installing CHR, but that didn't fix the problem for us. we're on D7.41 and not behind a proxy.

Did you try any of the

greggles's picture

Did you try any of the changes in my comment about update_fetch_url?

I did not, because I'm not

veg-o-matic's picture

I did not, because I'm not sure where to make any of those changes. I tried adding the first section to settings.php, but that only gave me a 500 error, so I quickly undid it.

The first change should not

David_Rothstein's picture

The first change should not cause a 500 error... Just to make sure, you should be pasting in the line of code only (not the <?php ?> tags which were just there for formatting in the code comment). In other words:

$conf['update_fetch_url'] = 'https://updates.drupal.org/release-history';

I eventually did do that, but

veg-o-matic's picture

I eventually did do that, but it didn't solve the problem. Just for the heck of it, I bounced Apache too. Still no go. Does the snippet need to be in a specific place in settings.php?

Anywhere in settings.php

David_Rothstein's picture

Anywhere in settings.php should be OK.

One other possibility that comes to mind is if the site is running an old version of PHP or OpenSSL, it could be running into https://www.drupal.org/node/1879970. I am not sure though.

I am going to bump the infrastructure issue at https://www.drupal.org/node/2646894 since I think it would be reasonable for drupal.org to undo that particular change for now.

Fail to even attempt to contact updates.d.o server

dervinfs's picture

I have been working for the past week on trying to see why two drupal 7.41 sites that I manage the servers for fail to get available updata data. I have made the change to the settings.php mentioned above by greggles, and have verified through a test php script that the change is made. My problem seems to be that my server is not even trying to reach out to the d.o server. I have ran a tcp packet capture while trying to run the "check for updates" link on the drupal site, and there are no packets captured for updates.drupal.org. As far as I know, the update process was working prior to January 2016.

It appears that I didn't have the openssl module compiled into my PHP version. It would be helpful if the error messages had more information. Openssl is required to handle https requests.

That's interesting that

David_Rothstein's picture

That's interesting that openssl might be required. I'll add a comment to https://www.drupal.org/node/1538118 about that.

And suddenly it's working

veg-o-matic's picture

And suddenly it's working again, with no effort on my part.

I love when that happens.

Yup,

David_Rothstein's picture

Yup, https://www.drupal.org/node/2646894 (the drupal.org infrastructure change) was rolled back today, so it should be working again for everyone :)

Failing to get updates

crystalgrafix's picture

We have web sites (Drupal 7.59) that are also behind a proxy, and no longer receive updates from drupal.org. Our sys admins have done the below changes and still can't get it to work.

Can someone please let us know what we are missing?

Firewall = Palo Alto PA820: Security Policy in place for the following IP addresses (these IPs seems to be changing with every request):

108.177.97.156
151.101.1.175
151.101.65.175
151.101.129.175
151.101.193.175
172.217.167.74
172.217.167.78

Have also added “*.drupal.org “to the firewall url whitelist.

Web Proxy = Forcepoint Appliance: No SSL Decryption and scanning exceptions are is in place for *.drupal.org

Why not develop locally and push to remote?

bburg's picture

What you are describing is a bit unusual from the workflows I've seen around Drupal. Typically we develop locally, and push to remote dev/live environments. This could be as simple as having remote sites site on a git repo, and just checkout new tags. Or using some sort of Continuous integration tool that rsyncs a buid to the server.

Managed hosting companies typically provide tools that aid this.

Security

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: