Seemed to recall a conversation about three months back about using the front end in drupal to download/install modules
I did some searches and can't find anything relevant. I'm putting the workflow below, if someone knows of a module, let me know, otherwise I'll write one
these steps are all done through the front end in drupal
1. enter url of module.tar.gz file(or other compressed file type) you want to install
2. select installation directory (i.e. in the sites/all/themes, sites/default/themes, sites/all/modules, sites/mycoolsite/modules)
3. click submit
on the back end module does this
1. uses curl/wget to download appropriate file
2. executes tar xvfz filename (or other appropriate extraction) to unzip the file
of course you'll need appropriate drupal permissions to access the module, and drupal will need permissions to execute the appropriate binaries on the server
thanks for the help
-Dave
Comments
Hey..!! Its very nice idea..
hi,
i have not found such module/code for doing such type of installation.But i think it will be very nice to provide such functionality using CURL.I will also give a try for it if i will get time..
Chandigarh
http://www.indersingh.com/
http://northwebstudio.com
India
Make sure it's secure
I don't know enough about it to know if that method is secure but see http://www.drupal4hu.com/node/115
Michelle
See my Drupal articles and tutorials or come check out the Coulee Region
Unholy trinity
Hey Dave,
This is not a recommended method for updating sites, since there are many variables in which you could lose control and open yourself to severe security risks.
But...
It should be possible with one of the following plus some extra precautions on how you manage your access:
http://drupal.org/project/upload_package
http://drupal.org/project/module_installer
You should also be using Update Status to notify you of outdated packages and security releases:
http://drupal.org/project/update_status
Good luck and lock it down tightly...
thanx for all the feedback
thanx for all the feedback.
i am, of course, aware of the security implications but I'm not convinced that having such a module is any more dangerous than having the ability to write custom php blocks, through the UI.
If the user(in this case me) can be trusted to write
<?phpdb_query(drop all from user);
?>
BTW, Ihaven't tried it but does anyone know what would happen if I put this
<?php
$ch = curl_init("http://www.example.com/");
$fp = fopen("example_homepage.txt", "w");
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);
?>
in a php block? Does drupal filter it out?
-dave
I've got cancer, so kidney failure as well isn't a big deal
This is the "I've got cancer, so it doesn't matter if I'm also drinking myself into kidney failure" argument.
The paranoid answer is "well, fix your cancer AND stop drinking" or "turn off that php input format AND don't build this updater module".
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
Virtual modules?
Perhaps it would be possible to have a virtual module facilitator through the database. PHP code of modules would be stored in the database, and then evaluated by the module itself. This is, of course, a huge performance drain, so I don't know... if people think it's worth it, something like this could definitely be written.
FTP
A "safe" way to do it without opening up the permissions would be to give Drupal an FTP account to use when uploading the modules. Drupal would FTP into the account, and then upload the files through the FTP permission set instead of opening up the permissions on sites/*.
Good idea...
...but would that have to be php 5 only?
why?
I don't know for certain, but Joomla! has been doing ftp for years so I don't think so.
knaddison blog | Morris Animal Foundation
No advantage.
I know Joomla does this for "security reasons" but this provides essentially no additional security benefit. If an attacker can subvert Drupal to exploit a module-upload module to write to the filesystem, there is no reason to think that the attacker cannot subvert Drupal to exploit its FTP access to localhost.
only if you store the password
I think you are right only if the module stores the password. Otherwise this is "safe" for editing files, right?
The only way an attacker could take advantage of the ftp feature when that ftp feature doesn't store the password is if they know your ftp password. In which case...why would they use this interface?
Joomla's solution (afaik) still lacks the digital signatures and ssl to guarantee that the file comes from the real project and not somewhere else.
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
pardon the follow up to myself, but...
You'd have to
1) store the password
2) have an attack that focuses specifically on Drupal
If the webserver also owns all of the files in the webroot then it could be an attack targetted at any piece of software that enables the attacker to do what they need. If you use the FTP system and store the password then the attacker has to find a vulnerability that allows them to take advantage of (or read the password to) this theoretical FTP storage/writing mechanism in order for them to have a valid attack. Right?
I mean, I wouldn't store this password regardless, but I think that it actually does gain security at both levels.
The ideal scenario in terms of security is something based upon a user initiated shell script, but then you guarantee that nobody on shared hosting will ever be able to run it...which isn't as useful to the Drupal world in general.
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
target market
And thats the real "target market", if you will, for a module like this.
Those on shared hosting who may not have access/permission to run any shell commands.
The rest of us can just do a wget and be done with it.
Cron?
You can do quite a lot with cron jobs on some shared hosting accounts. Making symlinks etc. Maybe an additional, regular cronjob hitting something in /scripts and this finds stuff queued through the interface and does the relevant downloading?
When the attacker doesn't
When the attacker doesn't know your password then they will probably put a load onto the server, either taking it down, or racking up your bandwidth, making you pay lotsalolly.
Even if you install a bandwidth module (theres one for apache) to stop the amount of hits/queries from a partiular ip, you can be sure that the attacker will then use a large amount of other ip's to target their attack on your server.
relevance?
What is the relevance of a DOS or DDOS attack to a discussion about filesystem access, permissions, and automated update systems?
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
I thought you were sposed to be on holiday!!
Did I say anything about DOS'ing?
No I didn't. You did.
Were you questioning my comment or were you adding something else, because I dont see the relevance (either).
a rose by any other name
You said:
Wikipedia says about DDOS:
Whether you want to call it a DDOS or not, what is the relevance of "putting a load onto the server" to a conversation about file system permissions and automated update systems?
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
Smile
Sorry if I'm taking this away and up a level.
Anywhere there are security implications, a breach can allow a person, trusted or not, to do stuff.
For a user to do:
"curl drupal.org/projects/coolnewmodule.tar.gz"
merely means they are fetching some file from drupal, and putting it into the pwd.
Same goes with wget.
Having a module that will spam other servers, could lead to exhausting bandwidth, for example.
Allowing a user other than user1 to do stuff to the frontpage is providing more ways of getting in to an installation.
There are many exploits that have come out including XSS. Can they be used to drill down into a server for dangerous exploitation?
The relevance of a DOS or DDOS attack to a discussion about filesystem access, permissions, and automated update systems, is when new exploits come out, when pages have not been tested at securityspace.org for instance, or some other such site, or with a cetain firefox xpi.
Any user can wget your whole site, and if you even put up a diary that links to all the days even without events, it can get quite exhausting for the server. Say that might be a kid who has just found the usefulness or playfulness of wget, with his new ultrafast broadband. What happens when this kid finds out he can use a compromised server elsewhere to have a go too.
Its certainly an interesting idea, but also a sensitive one, I think. I'd say consider the risks, and even try to make the admin module uploader installer harder. So how many times will you use it? Is it just for developers to try out their latest and greatest?
Go for it. Roses smell nice to all those given them, unless they aren't looked after.
Have I made myself clearer or dumber?
either you trust drupal or you don't
Interesting idea w/ the FTP, but again (and someone please tell me if I'm being thick here).
1)Either you trust your drupal installation with access to your files (you must because you installed it) or you don't.
2)Then either you trust some user(s), with access to your Drupal administrative pages, or you don't.
if the answer to either of the questions above is no, bearing in mind that "some user" could be user/1, with power to do anything, then why even have a drupal installation in the first place, cause you can't do anything with it.
Now I envision the only users that would have access to this module are your administrative users. With the right permissions, through the admin pages I can drop all my users, drop all my content. When install.php is run, you give drupal keys to the database, and it has enough permissions to change permission on the file system (it changes write status of settings.php after install).
Anyways, thanks for all the input, I think i'm going to go ahead and work out in detail the use cases (including where things could go horribly terribly wrong).
-dave
webserver permissions
You should not trust any web-facing software installed on your server with access to itself. You obviously allow it access to the "files" directory - yes. But, for example, the modules/ directory and index.php it should not have access.
Drupal does change the settings.php during and after install, but it then warns you if it can't make it read only because that is a security vulnerability. Also, the install instructions only ask you to make that one file editable by the web server - they don't say to make ALL the files on your server editable by the web server.
You may have read about a fair number of site defacements recently. Really they happen quite often, but basically these rely on
1) arbitrary php execution vulnerabilities in some code somewhere (even Drupal contrib has had some of these)
2) weberservers where the user that the webserver process runs as have broad file access
If you have 1 but not 2 - then there is only limited damage that attacker can do. If you have them both together, the attacker can do whatever they want to any file on your machine. So, the attacker can alter your front page. They can send all visitors of your site an executable with a virus executable that is hosted on your site. They can host viruses/blackmail of papa smurf/porn in hidden directories and then silently use your computer as a host for other attacks. They can upload spam relays. Etc.
If you look at similar systems for this (Firefox's auto-update feature, yum, apt, up2date) they are based on packages that are digitally signed with keys, download hosts that are verified using SSL certificates, and probably a couple of other measures. And even then the updating software generally requires a "root" user to run the process and do the updating - it doesn't do the updating from within the processes of the software itself. In order to provide a secure update system you would need not only the FTP feature that RobLoach mentions above but also the SSL and file-signature system on both client and server. Those are both complicated and somewhat expensive in addition to being time consuming to implement. Which is basically why Drupal doesn't have them yet.
This access separation is certainly part of (all of?) why the apache processes runs as www-data or "httpd" instead of running as root.
Does that make sense?
Sorry I was so brief earlier - this is a serious question that deserves a serious answer.
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
Good Answer
Thanks for taking the time Greggles,
that was a good answer.
I guess if I want to really know if my module(yet to be written) is secure, i can put it up on a honeypot and see if I get hacked.
:)
-Dave
honeypot != secure
I think better than that would be to implement the features I described (and also an md5sum checker) and also do as Karoly mentioned with requiring the user's ftp/sftp/ssh password every time. Then you can be at least more certain that the upgrade process is safe.
There are thousands of holes in the internet. At OSCMSCon in Sunnyvale Rasmus' keynote he mentioned that he wrote an XSS sniffer and found holes in the websites of over 80% of the banking sites that he tested it on. Banking sites. XSS. As in, "steal your users cookies and their bank account information." So, I'd say that a single installation on a honeypot is not a proof that it is secure.
Regards,
Greg
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
How many times
do I and others need to reiterate the same thing? Greg is right: your webserver must never, ever write files it then executes through PHP.
If you write code that opens an FTP-SFTP-SSH process back to the webserver through PHP5 streams and asks for a password every time it does so then the possible vulnerability still can't write.
bit o' humor
chx,
i'm not sure if you got the humor/light hearted tone in my last post. Security is a serious issue, but the point of a honeypot is to get hacked so that you can capture the actions of your hacker.
-dave
so your giving invitations
to hackers to have a go at your site?
When they know your a target, fine yeah, but when you want them to stop, will it be enough to tell them to stop. Depends on your type of hacker and their ethic, if any.
What if they hack thru your honeypot?
wget & curl are command line and thats easy for hackers to carry on and put a backdoor or 2001 into your system, even if you have a shared account.
Its a lovely idea, although why do you want to have more than one way to do something?
well..
I agree that a honeypot isn't a great idea, but maybe that was a joke.
About whether there is a need for this tool or not - there have been 3 new modules released in the last 6 months on D.o to do this. I would say that this makes a pretty clear case that there is demand for a tool like this. The idea is that people who don't have shell access to their server to be able to use wget/curl would use this tool. Frankly, if I could click in the interface to update my modules I'd probably go for it as well - the convenience and value are obvious.
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
People without shell access
People without shell access may have access to cron, and then theres php scripting that can do jobs too. Yes.
A honeypot isn't really a drupal thang, I dont think, scripting and process wise. Its an environment which drupal can work with. See:
http://www.google.gr/search?q=honeypot+drupal
+1
I've been tracking this discussion - +1 on the demand for it. I would definitely use it, and I think such a tool would make drupal more compelling for Kabissa members to use. Not only do they not usually have shell access but connectivity in Africa is such a problem that managing websites can be an enormous headache as it is. The more that can be (safely) handled through the web browser, the better.
Such a tool could also help with verifying that modules are up to date as well which would be great.
I don't know how it's done technically or if it's safe, but I like dokuwiki's plugin installer.
Cheers,
Tobias
--
Tobias Eigen
Executive Director
Kabissa - Space for Change in Africa
http://www.kabissa.org
download to /files automatically but require manual final step?
I wrote about a possible compromise approach – download modules and dependencies to the files directory, and then the user (or the entering of an extra password to access shell or FTP or SVN or...) does the final move – in my prediction (well, hope) for 2008.
benjamin, Agaric Design Collective
benjamin, agaric
Not a module but webapplication
Update statue is nice for monitoring :).
But for first install and update module, why don't create a simple web interface like http://mootools.net/download for dowload all module in one zip. After the dowload extract the zip in the /site/all/module or other destination is easy.
For update, we can imagine a link in update statue to open this page with the correct module pre-selected.
Enabling many modules at once makes figuring out errors harder
I'm all for making things easier but comments I've come across in the near past from more experienced Drupal users clearly state that enabling a bunch of contributed modules all at once makes it very difficult for the average user to then figure out what went wrong when the inevitable error messages appear.
The more experienced Drupal users' rational is that if you only enable a few modules at a time, diagnosing errors is easier. Their rational makes sense to me so my practice is to usually only enable one or two contributed modules at a time and to try testing them a bit to ensure all is going well. Afterwards, I then repeat the process as necessary.
Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing
Walt Esquivel, MBA; MA; President, Wellness Corps; Captain, USMC (Veteran)
$50 Hosting Discount Helps Projects Needing Financing
think of an idea and the community often already has it ;)
Here you are http://drupal.org/project/package_builder
It's not on D.o (I believe d.o would prefer to use a shell packaging script that creates the profiles rather than this package_builder module which I believe would have performance problems if it were subjected to widespread use).
I don't really see the value in this personally, but it's there if anyone wants to build a site that uses it they can and then promote it (perhaps charge for the use of it, or put ads on it...).
--
Open Prediction Markets | Drupal Dashboard
knaddison blog | Morris Animal Foundation
I second the web interface concept
I personally would like to see a web interface that presents a complete list of all modules withing a selected drupal version (one liners with version numbers) with a checkbox next to it for "grab this one" and another check next to it for "automatically enable". I imagine a setting for latest stable release or the latest dev version would be nice too.
Once submit was pressed, everything would be downloaded to the local machine (running the browser) either one at a time and then uploaded to the drupal site, all download and then one at a time upload. Or for a semiautomated upload of an extraction file that the user can import from the drupal site. This would allow a single ftp file upload of modules in a group and with the enable status.
I use a cpanel based provider without a shell account and a 16meg limit on php uploads.
I hope this suggestion helps.
Rich
multi-tenancy issues
Not all modules should be enabled for all sites; therefore, this capability would have to take care of multiple sites on one drupal installation.
Move to an ISP that gives shell/ssh access.
how does drupal install modules from a URL without FTp access
I want to know how Drupal installs modules through a URL without using FTP...
I recently installed a module in word press, then the next day all my files were erased for the entire site.
Anyhow, any information would be great...
Louisville Web Group, Account Executive
Installing modules without FTP client
I assume, for some technical reason, you can't use an FTP client or a Telnet/SSH client.
If you are using Drupal 5-6:
Your web host may provide an online web tool to manage directories and files, upload files and expand compressed files. Ask your host support service.
If you are using Drupal 7:
Module installation is done via Module Administration
http://drupal.org/documentation/install/modules-themes/modules-7
For documentation:
http://drupal.org/documentation/install/modules-themes