Posted by matt2000 on April 18, 2008 at 1:42am
Hey all,
I don't know if anything like this already exists, but I've written a script to add to install profiles that will go out and fetch via FTP the latest version of any missing modules from the Drupal projects repository.
I'd consider this a development version of the script, but it seems to be working fairly well, with some cool features, like preferring files that are NOT marked 'alpha', 'beta', or 'dev', but it will use these if it can't find something better.
To use, place the attached file in the folder with your profile (rename with proper .php extension only), and make your _profile_modules() look something like this:
<?php
function myProfileName_profile_modules() {
$modules_list = array(
// Enable required core modules first.
'block', 'filter', 'node', 'system', 'user', 'watchdog',
// Enable optional core modules next.
'color', 'help', 'menu', 'path', 'taxonomy', 'upload',
// Then, enable any contributed modules here.
'views', 'views_ui', 'content', 'content_access', 'devel', 'admin_menu', 'date_api', 'date_timezone', 'calendar', 'calendar_ical', 'image', 'imagefield', 'gmap', 'location', 'user_status', 'loginmenu',
);
// Additional module packages to download, because the package name might not match the module name; or we might want to have a module available, but not enabled right away.
$other_packages = array ('cck', 'date', 'image_gallery', 'send', 'mimemail', 'simplenews' );
require_once('getmods.php');
profile_download_modules(array_merge($modules_list, $other_packages), 5); // Second argument is the Drupal version to look for.
return $modules_list;
}
?>Please try this out & make comments & submit improvements.
Thanks!
-Matt
Comments
The script
Apparently file attachments don't really work here, so here is the contents of getmods.php :
<?php
function profile_download_modules($module_list = array(), $drupal_version = 6) {
//Make sure this is only called once during the install process.
global $profile_download_modules_complete;
if ($profile_download_modules_complete) {return;}
$count = 0;
$unpack = 0;
include_once './includes/common.inc';
$present_modules = array();
foreach (drupal_system_listing('.module$', 'modules', 'name', 0) as $present_module) {
$present_modules[] = $present_module->name;
}
$mods = array_diff($module_list, $present_modules);
$connection = ftp_connect("ftp.drupal.org");
$login = ftp_login($connection,"anonymous","");
ftp_chdir($connection, "pub/drupal/files/projects/") ;
foreach ($mods as $mod) {
$mod = $mod ."-". $drupal_version;
$files = ftp_nlist($connection, "./$mod"."*");
$offset = strlen($mod);
foreach ($files as $key => $file) {
if (stripos($file, "alpha", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "beta", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "dev", $offset) && count($files) > 1) {unset($files[$key]);}
}
$files = array_values($files);
$files = array_reverse($files);
$mods_dir = './sites/all/modules/';
mkdir_recursive($mods_dir);
if ($files[0] && !is_file($mods_dir . $files[0])) {
if (ftp_get($connection, $mods_dir. $files[0], $files[0], FTP_BINARY)) {
$count++;
}
else {
drupal_set_message("Could not download $mod module.");
}
}
if(is_file($mods_dir ."/". $files[0])) {
$cmd = shell_exec("cd $mods_dir; tar -zxf ". $files[0] );
if ($cmd) { $unpack++; drupal_set_message($cmd);}
}
}
ftp_close($connection);
drupal_set_message($count ." module packages successfully downloaded.");
drupal_set_message($unpack ." module packages successfully unpacked.");
$profile_download_modules_complete = TRUE;
}
function mkdir_recursive($pathname, $mode = 0777 ) {
is_dir(dirname($pathname)) || mkdir_recursive(dirname($pathname), $mode);
return is_dir($pathname) || @mkdir($pathname, $mode);
}
?>
Drupal.org user profile
Drupal Micro-blogging: http://twitter.com/matt2000
Matt--
Matt--
I'm just looking this over, and I am excited by the idea. But I have a few questions/comments. Please excuse me if I misunderstand; my understanding of things Drupal/PHP are rudimentary at best, and I haven't yet tried the code out.
If I see this correctly, your profile code is going to download everything, including core modules. But I should already have those core files, so really you only need to download the contrib modules. You've got to have core to get this far--or at least that's how I see it. So, I would think to separate the contrib modules out and only ftp and expand those. Does that make sense?
Next, perhaps it would be good to have the getmods code return an array of successful and failed downloads, so that the profile could then act only on the modules that got downloaded correctly, and make note of the modules that didn't. Here I'm thinking about the person who wants to use Drupal 6, but not all the modules are ported through. This part of it is complicated by the fact that you wisely consider the idea of downloading (but not activating) some modules. To enable this to still happen, I imagine you'd have to invoke profile_download_modules twice--once for the active modules, and once for the inactive modules. Then, merging the good active with the core modules could be returned by the original routine, and data regarding the failed and inactive modules could be saved or displayed (this is about where my PHP/Drupal knowledge gives out).
I think it would be fabulous to be able to set up a profile simply by saying what modules you want, and then just have the installation take care of getting the damn things and putting them in the right place. So I hope this will work out. I attach my variation (untested!) of the _profile_modules code.
David
function personal_profile_modules() {
//Based on Matt2000's version, mods by sunfish62
$base_modules = array(
// Enable required core modules first.
'block', 'filter', 'node', 'system', 'user', 'watchdog',
// Enable optional core modules next.
'color', 'help', 'menu', 'path', 'taxonomy', 'upload', 'dblog',
);
$contrib_mods = array(
// Establish contributed modules list.
'views', 'views_ui', 'content', 'content_access', 'devel', 'admin_menu', 'image', 'quotes', 'recipe', 'lightbox2',
/* Modules not yet in Drupal 6
'video', 'weight', 'bookreview', 'acidfree',
*/
);
// Additional module packages to download, because the package name might not match the module name; or we might want to have a module available, but not enabled right away.
$other_pkgs = array ('cck', 'date', 'image_gallery', 'send', 'mimemail', 'simplenews' );
require_once('getmods.php');
// Note: profile_download_modules gets modified to return an array of good and bad download arrays
list($c_good, $c_bad) = profile_download_modules($contrib_mods, 6); // Second argument is the Drupal version to look for.
list($o_good, $o_bad) = profile_download_modules($other_pkgs, 6); // Second argument is the Drupal version to look for.
$modules_list = array_merge($base_modules, $c_good);
//Do something to notify user about bad modules and inactive ones...
return $modules_list;
}
More on this
I've been trying this out on my test bed at home, and ran into a quick problem, which is that my test bed is running Apache on Windows, and there ain't no tar. Besides moving everything over to nix, I'm not sure where to go with this. I *can say that the code successfully gets the gz files from ftp.drupal.org, so that's a big step.
I suspect (having dug around a little more) that the getmods code can handle noting which modules downloaded, which expanded, and which failed. So all that it would return there is the array of installed modules (which for me right now is none, heh heh!).
Oh, and I see now that you have a filtering system in getmods that eliminates all the core modules. I didn't get that.
More later.
David
thanks for trying it out
I work strictly in *nix environments, so I have no idea how to make this windows friendly. And I don't really have any interest in being windows friendly, so I'll leave that task to someone else. :-)
I'm not sure what you're getting at about returning an array of missing modules. The Drupal Installer will already stop you if modules are missing when it comes time to enabled them (which comes after the download process). But it would be easy enough to implement. I already count the number of downloaded & expanded modules.
Also, if a file is downloaded, but can't be expanded, the code will not try to download it again. It will keep the same copy in sites/all/modules . But it will try to expand it again next time the script is run.
Drupal.org user profile
Drupal Micro-blogging: http://twitter.com/matt2000
My latest take
Yeah, the Windows stuff is a challenge...
What I was getting at with the missing modules is that if a module might not have a current version, one could still proceed with the installation, just not with those modules. IMO, it is better to get the installation completed, even incompletely, than to hang it up unfinished. Take a look at what I did below.
I also changed the code so that the source files get put into the profile directory, and the expanded modules get put into sites/all/modules. I did that based on something I saw in common.inc about keeping source archives in the profile folder.
I changed some of the array handling; it may or may not be appropriate.
Finally, I did test this on a MAMP install, and it all went well.
My version of getmods.php:
<?php
/<strong>
* Gets module archive files from ftp.drupal.org, save to profile folder.
* Extracts each to ./sites/all/modules
<em>
* @param $profile
* Name of installation profile
* @param $module_list
* Array of all modules in profile
* @param $drupal_version
* Number of Drupal Major version for download compatibility
* returns $mod_inst
* Array of modules placed in ./sites/all/modules
*/
function profile_download_modules($profile, $module_list = array(), $drupal_version = 6) {
//Make sure this is only called once during the install process.
global $profile_download_modules_complete;
if ($profile_download_modules_complete) {return;}
include_once './includes/common.inc';
$count = $unpack = 0;
$p_dir = "./profiles/$profile/";
$core_modules = array();
foreach (drupal_system_listing('.module$', 'modules', 'name', 0) as $core_module) {
array_push($core_modules, $core_module->name)
}
$mods = array_diff($module_list, $core_modules);
$connection = ftp_connect("ftp.drupal.org");
$login = ftp_login($connection,"anonymous","");
ftp_chdir($connection, "pub/drupal/files/projects/") ;
foreach ($mods as $mod) {
$nmod = $mod . "-" . $drupal_version;
$files = ftp_nlist($connection, "./$nmod"."</em>");
if (!$files) { drupal_set_message("$mod module not available for download in this Drupal version."); }
$offset = strlen($nmod);
foreach ($files as $key => $file) {
if (stripos($file, "alpha", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "beta", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "dev", $offset) && count($files) > 1) {unset($files[$key]);}
}
$dfile = array_pop($files);
if ($dfile && !is_file($p_dir . $dfile)) {
if (ftp_get($connection, $p_dir . $dfile, $dfile, FTP_BINARY)) {
$count++;
}
else {
drupal_set_message("Could not download $mod module.");
}
}
}
ftp_close($connection);
drupal_set_message($count ." module packages successfully downloaded.");
// Now to expand archive files into ./sites/all/modules
$mods_inst = array();
$p_dir = "./profiles/$profile";
$profile_modfiles = list_profile_modfiles($p_dir);
$mods_dir = './sites/all/modules/';
mkdir_recursive($mods_dir); // creating the destination folder for site modules
$ospos = stripos(php_uname('s'), "Windows");
foreach ($profile_modfiles as $modfile) {
$m_src = $p_dir . "/" . $modfile;
if ($ospos !== FALSE ) {
// Windows command next line NOT working!
$cmd = shell_exec("c:\WINNT\system32\cmd.exe /c tar -zxf " . $m_src . " -C " . $mods_dir);
}
else {
// *nix command
$cmd = shell_exec("tar -zxf " . $m_src . " -C " . $mods_dir);
}
if ($cmd) {
$unpack++;
array_push($mods_inst, array_shift(split("-", $modfile))); // builds valid modules
drupal_set_message($cmd);
}
}
drupal_set_message($unpack ." module packages successfully unpacked.");
$profile_download_modules_complete = TRUE;
return $mods_inst;
}
/</strong>
* Return list of all gz files in profile folder.
*
* @param $dir
* Directory of profile folder (e.g., './profiles/MyProfile')
*
* returns $mod_list
* Array of module archive files.
*/
function list_profile_modfiles($dir){
$mod_list = array();
if ($handle = opendir($dir)) {
while (false !== ($file = readdir($handle))) {
if (strpos($file, '.gz',1)) { array_push($mod_list, $file); }
}
closedir($handle);
}
return $mod_list;
}
function mkdir_recursive($pathname, $mode = 0777 ) {
is_dir(dirname($pathname)) || mkdir_recursive(dirname($pathname), $mode);
return is_dir($pathname) || @mkdir($pathname, $mode);
}
?>
and finally, my version of the profile, where the main change is to filter out unsuccessful downloads, enabling installation to continue:
function personal_profile_modules() {//Based on Matt2000's version, mods by sunfish62
$profile = "personal";
$base_mods = array(
// Enable required core modules first.
'block', 'filter', 'node', 'system', 'user',
// Enable optional core modules next.
'color', 'help', 'menu', 'path', 'taxonomy', 'upload', 'dblog',
);
$contrib_mods = array(
// Contributed modules list.
'views', 'views_ui', 'content', 'content_access', 'devel', 'admin_menu',
'image', 'quotes', 'recipe', 'lightbox2', 'video', 'weight', 'bookreview', 'acidfree',
);
// Additional module packages to download, because the package name might not match the module name; or we might want to have a module available, but not enabled right away.
$other_pkgs = array ('cck', 'date', 'image_gallery', 'send', 'mimemail', 'simplenews' );
require_once('getmods.php');
$inst_mods = profile_download_modules($profile, array_merge($base_mods, $contrib_mods, $other_pkgs), 6);
$final_mods_list = array_intersect($inst_mods, $contrib_mods); // failed modules pulled from requirements
$modules_list = array_merge($base_mods, $final_mods);
return $modules_list;
}
I hope this helps.
David
Be careful with Shell_Exec
Many hosting companies don't allow its use.
Use instead:
<?phprequire_once "Archive/Tar.php";
$tar = new Archive_Tar($m_src);
$tar->extract($mods_dir);
?>
Archive_Tar is at the PEAR repository and usually installed with PHP5.
Best,
Lopo
And even more again
Further exploration yields a glitch in my approach. For some reason, the profile_modules code runs twice between the first and second screens. On the second time through, my filtering code strips all the contrib modules from the array, since they aren't expanded the second go-around. That wasn't my goal, so it would be better just to let the regular installation requirements routine handle this. That's too bad; I would like to see an installation that goes through every time, even if it has to skip over some elements.
David
Yes, that second call is why
Yes, that second call is why I had to include the check at the top of getmods, to keep form trying to download or unpack everything twice, etc.
Anyway, I consider it a good thing that it won't finish if it can't find a module. Becuase then I can go out and get it manually, and re-run and everything is fine.
Just a matter of opinion, I guess. I think an incomplete installation is worse than a stopped installation. Also, and incomplete installation is likely to generate errors form my SQL script that is loaded in my_profile_final().
I guess the best approach for user-friendliness would be to prompt the user, "Could not find or download module X, module Y. Do you want to continue anyway?"
Drupal.org user profile
Drupal Micro-blogging: http://twitter.com/matt2000
A thought for Windows
Could you get command line un-tar using cygwin ?
Drupal.org user profile
Drupal Micro-blogging: http://twitter.com/matt2000
Second Calls and Tar
WRT incomplete downloads, ultimately, I guess it is the responsibility of the profile developer to ensure that modules exist for an installation. I think it would be nice to give the end user the kind of option you describe.
As for tar under windows, I downloaded the gnuwin32 packages and tried to get them working. Ultimately, though, even if I could get them working on my machine (I couldn't), there's no guarantee that the user's system will have tar, or if it did, where that would be. One could include tar (and its necessary windows dlls) in the profile folder, but I was not able to get it to work reliably. Stuff with paths and all.
David
View comment upper
http://groups.drupal.org/node/10810#comment-40917
Thanks
Thanks for the heads up. I will try to modify my version of the code to handle this.
Some New Issues
I have been using my version of this code now to test it out, and I have encountered a couple of problems.
1) Problems downloading the latest versions. This code does not handle the transition from 1 to 2 digit versions correctly. For example, when there is a views-6.1.10 release, this code still retrieves views-6.1.9 (since 9 is greater than 1). I haven't had a chance to examine how to fix this.
2) Problems with the way that CCK and its submodules behave on install have forced me to re-organize the different arrays to accommodate the idiosyncracies. Specifically, CCK downloads as cck-6.x.x.tar.gz, but activates as 'content', and submodules require that module. I haven't been able to test out the final functionality, but will post when I get further along.
David
My original version handled
My original version handled point (2) just fine. But I never had need to try out the later version developed here. maybe issues were introduced.
Point (1) shouldn't be too difficult to fix
However, due to the unstable nature of much drupal contrib development, I'm finding that this script is not as useful as I'd hoped. e.g., I have to maintain my own versions of date & calendar modules, because there was no official release, and the RC versions had many problems.
Drupal.org user profile
Drupal Micro-blogging: http://twitter.com/matt2000
my version of getmods.php
This doesn't Fix 1 or 2, but does clean up the code a little, make messages errors when appropriate, also sets FTP to passive mode (caused me no end of headache figuring that one out) and instead of installing modules to /sites/all/module installs them to /profiles//modules and lets the installer put it where it wants them.
Example usage (mod for http://drupal.org/project/innovationnewsprofile )
<?php
/**
* Implementation of hook_profile_modules().
*/
function innovationnewsprofile_profile_modules() {
$core_modules = array(
// Required core modules.
'block', 'filter', 'node', 'system', 'user',
// Optional core modules.
'dblog', 'help', 'menu', 'search', 'taxonomy', 'trigger', 'update', 'path',
'php',
);
$contributed_modules = array(
// OPL modules.
'innovationnews', 'editnews', 'editionmanager', 'editionviewer', 'editnews',
'xmltokml',
// Other contributed modules.
'contemplate', 'image', 'image_attach', 'image_gallery', 'token',
'pathauto', 'captcha', 'views', 'views_ui',
// CCK modules.
'content', 'content_permissions', 'number', 'text', 'optionwidgets'
);
// Additional module packages to download, because the package name might not match the module name; or we might want to have a module available, but not enabled right away.
$other_packages = array('cck');
require_once('getmods.php');
$inst_mods = profile_download_modules('innovationnewsprofile', array_merge($core_modules, $contributed_modules, $other_packages), 6);
$modules_list = array_merge($core_modules, $contributed_modules);
return $modules_list;
} // function innovationnewsprofile_profile_modules
?>
and the code itself.
<?php
<?php
/**
* Gets module archive files from ftp.drupal.org, save to profile folder.
* Extracts each to ./sites/all/modules
<em>
</em> @param $profile
* Name of installation profile
* @param $module_list
* Array of all modules in profile
* @param $drupal_version
* Number of Drupal Major version for download compatibility
* returns $mod_inst
* Array of modules placed in ./sites/all/modules
<em>/
function profile_download_modules($profile, $module_list = array(), $drupal_version = 6) {
//Make sure this is only called once during the install process.
global $profile_download_modules_complete;
if ($profile_download_modules_complete) {return;}
include_once
'./includes/common.inc';
$count = $unpack = 0;
$p_dir = "./profiles/$profile/modules/";
if (!is_writeable($p_dir)) {
drupal_set_message($p_dir.' is not writeable', 'error');
return false;
}
$core_modules = array();
foreach ( drupal_system_listing('.module$', 'modules', 'name', 0) as $core_module) {
array_push($core_modules, $core_module->name);
}
$mods = array_diff($module_list, $core_modules);
$ftp_server = "ftp.drupal.org";
$connection = ftp_connect($ftp_server) or drupal_set_message("Couldn't connect to $ftp_server to download missing modules", 'error');
$login = ftp_login($connection,"anonymous","");
ftp_pasv($connection, true); // turn passive mode on
ftp_chdir($connection, "pub/drupal/files/projects/");
foreach ($mods as $mod) {
$nmod = $mod . "-" . $drupal_version;
$files = ftp_nlist($connection, "./$nmod"."</em>");
if (!$files) {
drupal_set_message("$mod module not available for download in this Drupal version.", 'error');
}
$offset = strlen($nmod);
foreach ($files as $key => $file) {
if (stripos($file, "alpha", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "beta", $offset) && count($files) > 1) {unset($files[$key]);}
if (stripos($file, "dev", $offset) && count($files) > 1) {unset($files[$key]);}
}
$dfile = array_pop($files);
if ($dfile && !is_file($p_dir . $dfile)) {
if (ftp_get($connection, $p_dir . $dfile, $dfile, FTP_BINARY)) {
$count++;
}
else {
drupal_set_message("Could not download $mod module.", 'error');
}
}
}
ftp_close($connection);
drupal_set_message($count ." module packages successfully downloaded.");
// Now to expand archive files
$mods_inst = array();
$profile_modfiles = glob($p_dir.'*.gz');
$ospos = stripos(php_uname('s'), "Windows");
foreach ($profile_modfiles as $modfile) {
if ($ospos !== FALSE ) {
// Windows command next line NOT working!
$cmd = shell_exec("c:\WINNT\system32\cmd.exe /c tar -zxf " . $modfile . " -C " . $p_dir);
}
else {
// *nix command
$cmd = shell_exec("tar -zxf " . $modfile . " -C " . $p_dir);
}
if (!$cmd) {
$unpack++;
array_push($mods_inst, array_shift(split("-", $modfile))); // builds valid modules
drupal_set_message($cmd);
}
}
drupal_set_message($unpack ." module packages successfully unpacked.");
$profile_download_modules_complete = TRUE;
return $mods_inst;
}
?>
thanks to matt2000 and sunfish62.
Windows
Even though I'm a firm supporter of Linux in general - I get pangs of guilt when I use open source and don't contribute. To get it to extract on Windows, install 7-zip and use:
$cmd = shell_exec("c:\Program Files\7-Zip\7z.exe x -o". $p_dir . " " . $modfile);view #comment-526359
http://groups.drupal.org/node/10810#comment-526359
Oops - no windows
That command works well from command line but not so well from PHP. a combination of gzip (*.tar.gz -> *.tar) and 7-zip (*.tar -> job done) can do the job, but by that point it is easier to do it by hand.
Incidentally, Drupal must have modified the way it organises it's ftp. Here is my version of the original script:
<?php
/**
* Gets module archive files from ftp.drupal.org, save to profile folder.
* Extracts each to ./sites/all/modules
@param $profile
* Name of installation profile
* @param $module_list
* Array of all modules in profile
* @param $drupal_version
* Number of Drupal Major version for download compatibility
* returns $mod_inst
* Array of modules placed in ./sites/all/modules
*/
function profile_download_modules($profile, $module_list = array(), $drupal_version = 6) {
//Make sure this is only called once during the install process.
global $profile_download_modules_complete;
if ($profile_download_modules_complete) {return;}
include_once
'./includes/common.inc';
$count = $unpack = 0;
//$p_dir = "./profiles/$profile/modules/";
$p_dir = "./sites/all/modules/";
if (!is_writeable($p_dir)) {
drupal_set_message($p_dir.' is not writeable', 'error');
return false;
}
$core_modules = array();
foreach ( drupal_system_listing('.module$', 'modules', 'name', 0) as $core_module) {
array_push($core_modules, $core_module->name);
}
$mods = array_diff($module_list, $core_modules);
$ftp_server = "ftp.drupal.org";
$connection = ftp_connect($ftp_server) or drupal_set_message("Couldn't connect to $ftp_server to download missing modules", 'error');
$login = ftp_login($connection,"anonymous","");
ftp_pasv($connection, true); // turn passive mode on
ftp_chdir($connection, "pub/drupal/files/projects/");
$files = ftp_nlist($connection, '.'); // get file list from current directory
// for each module needed
foreach ($mods as $mod) {
$nmod = $mod . "-" . $drupal_version;
$offset = strlen($nmod);
// continue checking only the files that include the name of the module we are after
foreach ($files as $key => $file) {
if (strstr($files[$key],$nmod)){
$filelist[$key]= $files[$key];
}
}
// avoid alpha, beta, dev versions, in that order
foreach($filelist as $key => $file){
$tester = $filelist[$key]; // stripos gets confused if this doesn't happen
if (stripos($tester, "alpha", $offset) && count($filelist) > 1) {
unset($filelist[$key]);
}
if (stripos($tester, "beta", $offset) && count($filelist) > 1) {
unset($filelist[$key]);
}
if (stripos($tester, "dev", $offset) && count($filelist) > 1) {
unset($filelist[$key]);
}
}
$dfile = array_pop($filelist); // use the most recent version that satisfies our criteria
if ($dfile && !is_file($p_dir . $dfile)) {
if (ftp_get($connection, $p_dir . $dfile, $dfile, FTP_BINARY)) {
$count++;
}
else {
drupal_set_message("Could not download $mod module.", 'error');
}
}
}
ftp_close($connection);
drupal_set_message($count ." module packages successfully downloaded.");
// Now to expand archive files
$mods_inst = array();
$profile_modfiles = glob($p_dir.'*.gz');
$ospos = stripos(php_uname('s'), "Windows");
foreach ($profile_modfiles as $modfile) {
if ($ospos == FALSE ) {
$cmd = shell_exec("tar -zxf " . $modfile . " -C " . $p_dir);
}
if (!$cmd) {
$unpack++;
array_push($mods_inst, array_shift(split("-", $modfile))); // builds valid modules
drupal_set_message($cmd);
}
}
if ($ospos !== FALSE ) {
drupal_set_message('Please check '.$p_dir.', your modules should be there in *.tar.gz format.'.
' 7-Zip is a good program for uncompressing them. In future please use a *nix web server, they just work better');
}
else { drupal_set_message($unpack .'module packages sucessfully unpacked to '.$p_dir);}
$profile_download_modules_complete = TRUE;
return $mods_inst;
}
?>
Last but not least, I don't know why, but it works when I use $key and not when I use $file in the foreach loops
automated profile modules download for windows
I have written Windows enabled versions at: http://groups.drupal.org/node/10810#comment-526359
so uh .. is this code ready
so uh .. is this code ready for production use?
marafa-- insofar as these
marafa--
insofar as these various scripts go, they work pretty well. Unfortunately, none of my attempts succeeded in a complete end-to-end profile that would download, install, and activate the third-party modules for a site. There were numerous hitches in my own routines that made the scripting less than useful, as I kept finding I had to tinker with the end result anyway. Matt's last comment supports this, I think.
Since I am mostly interested in this from a philosophical perspective (i.e., I 'ought' to be able to do this), and my own personal website is set up, I never strived to work out these hitches.
FYI, the biggest troubles I ran into were the cross-platform extraction issue (especially Windows, where extraction software cannot be assumed) and some technical glitches with CCK and its submodules. Since that was in the dark days of last summer, it's quite possible that the CCK problems have been remedied.
If you're trying to build a generic profile for some Unix-based operating system (i.e., just not Windows) that isn't too heavily into CCK, one version of these scripts should work pretty well for you. And if you have improvements, send 'em on in!
David
Subscribing, what is the best
Subscribing, what is the best script to get all the dev. modules installed automatically on my ftp, on my drupal instance with /httpdocs/sites/modules/all?
greetings, Martijn
drush make
http://drupal.org/project/drush_make
Drush requires shell access
jonhattan--
As far as I understand, Drush and Drush make require shell access, which many site hosts (e.g. 1And1) don't offer without additional fees. One reason to use installation profiles is to work around that rather substantial limitation.
David
yes, those are command line
yes, those are command line tools. You can use them locally to prepare the code layout you want and then redistribute a tgz, upload via ftp or whatelse.
drush make is designed as a
@sunfish62: drush make is designed as a packaging automation tool so that you can build distributions with it; the end-user of your distribution doesn't need drush make
this implies the "compiled" distribution should be hosted somewhere so end users can download the complete package; if it's to be hosted on drupal.org, you just have to make sure your drush make script conforms to a few parameters, then drupal.org will "compile" it for you
see http://drupal.org/node/642116
ftp_login($connection,"anonymous","") does not work
it is strange but the code does not connect on the server but works well on localhost.
<?phpftp_login($connection,"anonymous","");
?>
It gives this error on the server:
Warning: ftp_login() [function.ftp-login]: Login authentication failed in /home/mysite/public_html/getmod.php on line 17while works on localhost
have u got any idea?
For those who are interested in profile installation in windows
For those who are interested in profile installation in windows, I have modified and added windows download features and it works. So you can now fully automate the download in both enviroment:
<?php
/<em>
* @file
* Defines function for downloading latest modules from drupal.org.
*/
function profile_download_modules($module_list = array(), $drupal_version = 6) {
//make sure that we are ABLE to enable/load zip libraries
zip_extension_warning();
//Make sure this is only called once during the install process.
global $profile_download_modules_complete;
if ($profile_download_modules_complete) {
return;
}
$count = 0;
$unpack = 0;
$win = array();
include_once './includes/common.inc';
$present_modules = array();
foreach (drupal_system_listing('.module$', 'modules', 'name', 0) as $present_module) {
$present_modules[] = $present_module->name;
}
$mods = array_diff($module_list, $present_modules);
$connection = ftp_connect("ftp.drupal.org");
$login = ftp_login($connection, "anonymous", "");
ftp_chdir($connection, "pub/drupal/files/projects/");
foreach ($mods as $mod) {
$mod = $mod . "-" . $drupal_version;
$files = ftp_nlist($connection, "./$mod" . "</em>");
$offset = strlen($mod);
foreach ($files as $key => $file) {
if (stripos($file, "alpha", $offset) && count($files) > 1) {
unset($files[$key]);
}
if (stripos($file, "beta", $offset) && count($files) > 1) {
unset($files[$key]);
}
if (stripos($file, "dev", $offset) && count($files) > 1) {
unset($files[$key]);
}
}
$files = array_values($files);
$files = array_reverse($files);
$mods_dir = './sites/all/modules/';
$bas = mkdir_recursive($mods_dir);
if ($files[0] && !is_file($mods_dir . $files[0])) {
if (ftp_get($connection, $mods_dir . $files[0], $files[0], FTP_BINARY)) {
$count++;
} else {
drupal_set_message("Could not download $mod module.");
}
}
$mod_file = $mods_dir . $files[0];
if (is_file($mod_file)) {
if (!strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {
$cmd = shell_exec("cd $mods_dir; tar -zxf " . $files[0]);
} else {
$win = win_unzip_modules($files[0]);
if ($win['status']) {
$cmd = $win['msg'];
}
}
if ($cmd) {
$unpack++;
drupal_set_message($cmd);
}
}
}
ftp_close($connection);
drupal_set_message($count . " module packages successfully downloaded.");
drupal_set_message($unpack . " module packages successfully unpacked.");
$profile_download_modules_complete = TRUE;
}
function mkdir_recursive($pathname, $mode = 0777) {
is_dir(dirname($pathname)) || mkdir_recursive(dirname($pathname), $mode);
return is_dir($pathname) || @mkdir($pathname, $mode);
}
function win_unzip_modules($mod, $mods_dir = "./sites/all/modules/", $zip_delete=true) {
$result = array('status' => true, 'msg' => '');
$unzip = 0;
$mods_dir = realpath($mods_dir) . "/";
$mod_zip_file = $mods_dir . $mod;
$mod_zip_open_fp = @zip_open($mod_zip_file);
if (is_resource($mod_zip_open_fp)) {
while ($zip_entry = zip_read($mod_zip_open_fp)) {
$zip_entry_dir = zip_entry_name($zip_entry);
if (!$result['msg']) {
$result['msg'] = $zip_entry_dir;
}
if (strpos($zip_entry_dir, '.')) {
$zip_filename = $mods_dir . $zip_entry_dir;
$zip_fp = fopen($zip_filename, 'w+');
if (zip_entry_open($mod_zip_open_fp, $zip_entry, "r")) {
$buf = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
zip_entry_close($zip_entry);
}
if ($success = @fwrite($zip_fp, $buf)) {
$unzip++;
} else {
$result['msg'] .= '\n' . $success;
$result['status'] = false;
}
@fclose($zip_fp);
} else {
@mkdir($mods_dir . $zip_entry_dir);
}
}
@zip_close($mod_zip_open_fp);
if ($zip_delete && $result['status']) {
$do = unlink($mod_zip_file);
}
}
return $result;
}
function load_extension($extension = 'zip') {
if (!extension_loaded($extension)) {
if (strtoupper(substr(PHP_OS, 0, 3)) === 'WIN') {
dl($extension . '.dll');
} else {
dl($extension . '.so');
}
if (!in_array($extension, get_loaded_extensions())) {
return false;
}
}
return true;
}
function zip_extension_warning($profile = 'my_profile_name', $extension='zip') {
if (!extension_loaded($extension)) {
if (!(bool) ini_get("enable_dl") || (bool) ini_get("safe_mode")) {
die("Please you need to enable the " . $extension . " extension in order to install " . $profile . " profile\n");
}
}
return true;
}
?>
Note that you need to use
Note that you need to use hook_install_tasks_alter() to create a task that runs before the main install of the modules in D7 now.
See https://drupal.org/node/509392