Experiences upgrading Drupal 5/CiviCRM 2.0 to Drupal 6/CiviCRM 2.2

I just completed an upgrade from Drupal 5 with CiviCRM 2.0 to Drupal 6 with CiviCRM 2.2 on a site with multisite hosting. Perhaps my experiences will help someone else.

I lack sysadmin skills but can use basic UNIX commands and copy and paste MySQL and php. In most cases, Google helped when I got stuck.

I have two installations:

(1) test.nccommunities.org where folks learn how things work, test upgrades, and try out new modules. The site also offers CiviCRM with sample data for people to explore.

(2) nccommunities.org which has seven additional sites running under it.

We created a third site, drupal6.nccommunities.org, installed and tested Drupal 6 on that site and chose to use the following contributed modules:

Some of these modules will not be used right away. On both Drupal 5 sites, we changed from the event module to calendar, date, and views, following the instructions in the O'Reilly "Using Drupal" book. We changed from TinyMCE to FCKeditor and added IMCE. Only test.nccommunities is currently using Messaging and Notifications. On the other sites, we're still using Notify, which is configured to subscribe new users by default. I have not thoroughly explored how to subscribe new users by default with Notifications.

The sites are all VERY INactive, so I was able to upgrade the databases, one at a time, over several days without losing data. This would not work for active sites. See afreeman's post at http://groups.drupal.org/node/26269#comment-90197 for a better way to do upgrades. I don't have privileges to change a site's virtual host settings. We'll re-evaluate the value of multisite hosting. (See www.centurionwebdev.com/content/drupal-multi-site-it-worth-it and posts on http://groups.drupal.org/node/26269).

I removed modules we weren't using from test.nccommunities and nccommunities ending up with:

All modules and core were at the latest version. (CiviCRM 2.1 and beyond require Drupal 6.)

I checked out instructions at drupal.org/upgrade and used the Drupal 6 file UPGRADE.txt. I also went to the directory for each module and looked for an upgrade file and checked the readme file. I also looked at drupal.org/project for each module for upgrade instructions.

I copied /home/drupal6/public_html/sites/all/modules to /home/drupal6/sites/all/modules/ (from public_html to the main directory. I probably should have chosen a different name. I emptied the wrong directory a couple of times, but it was easy to copy back.)

I also made a copy of the Drupal 5 modules and the Drupal 6 modules in my personal directory as backup.

NOTES:
(1) Modules in the modules directory are automatically updated in Drupal 6, even if they are not enabled. If you have modules in the directory that you aren't use, they can cause errors.
(2) The order of upgrading modules is important -- cck MUST be upgraded first, after core.
(3) Do NOT enable CiviCRM. It needs special handling after everything else is done.
(4) I had problems with caching and had to delete browser cookies, go to admin/settings/performance and "Clear cached data," go to admin/build/views/tools and "Clear Views' cache."
(5) For each site I created a file of notes -- what I did and what happened. And I created a file of update.php messages for each site.

MISTAKES:
(1) Not following the instructions and trying to upgrade core and all contributed modules at once.
(2) Not following the instructions at drupal.org/upgrade/copying-your-live-site-via-command-line. I just copied databases to the drupal6 site, so I got some errors based on not having the files for the database.
(3) Not emptying the database, resulting in tables from one database being inherited by the next.

The process for each of the nine sites was essentially as follows:

Put the site in maintenance.

Switch theme to Garland.

Disable contributed modules.

mysqldump -u user -ppassword database > xxx-nccommunities5-date.sql.txt

Re-enable contributed modules.

Put back any special themes.

Put site back online.

Login on drupal6 as user1.

Put drupal6 site in maintenance.

At /home/drupal6/public_html/sites/all/modules
rm -fr *
get permission denied messages for fckeditor
chmod -R 777 fckeditor
rm -fr fckeditor

Drop all tables from drupal6-database (I used phpmyadmin, select all, drop)

If the database is for a civicrm site, not all the tables are dropped.

mysql -u user -ppassword drupal6-database

Delete FireFox cookies

mysql -u user -ppassword drupal6-database < xxx-nccommunities5-date.sql.txt

mysql -u user -ppassword drupal6-database

In public_html/sites/default
chmod 777 settings.php
find $update_free_access = FALSE; and change to TRUE;

Run update.php (core modules only) -- mine had 60 updates.
Rerun until 0 updates.
Get LOTS of warnings but no failed messages.

In settings.php
find $update_free_access = TRUE; and change to FALSE;
chmod 444 settings.php

Go to drupal6.nccommunities.org/?q=user and login

On some sites, themes were gone. I had changed the color of garland. To fix, go to drupal6.nccommunities.org/admin/build/themes/settings/garland and save, which saves color/garland in files. (I hadn't copied the files directory to the Drupal 6 site.)

At drupal6.nccommunities.org/admin/settings/performance (bottom of the page)
"Clear cached data."

Site looked OK, though Book Navigation was turned on and some menu items were out of order. When there are duplicate primary menu items and/or duplicate navigation menu items, I had cache problems. Go back and reload the database again.

Gradually copy contributed modules from /home/drupal6/sites/all/modules
to /home/drupal6/public_html/sites/all/modules/
Order is important.

cd /home/drupal6/sites/all/modules

cp -R cck /home/drupal6/public_html/sites/all/modules/.
Enable cck content and text, if they were used on this site.
ONLY CCK CORE FILES (the ones in the tarball on the CCK project page).
run update.php (if cck was not used on the site there will be 0 updates)
Ignore:
user warning: Duplicate key name 'nid' query: ALTER TABLE content_type_event ADD INDEX nid (nid) in /home/drupal6/public_html/includes/database.mysql-common.inc on line 434.
Failed: ALTER TABLE {content_type_event} ADD INDEX nid (nid)

cp -R date /home/drupal6/public_html/sites/all/modules/.
enable date, date api, date popup, date timezone
* user warning: Table 'drupal6.date_format_types' doesn't exist query: SELECT dft.type, dft.title, dft.locked FROM date_format_types dft ORDER BY dft.title in /home/drupal6/public_html/sites/all/modules/date/date_api.module on line 1974.
* user warning: Table 'drupal6.date_format_types' doesn't exist query: SELECT dft.type, dft.title, dft.locked FROM date_format_types dft ORDER BY dft.title in /home/drupal6/public_html/sites/all/modules/date/date_api.module on line 1974.
* user warning: Table 'drupal6.date_format_types' doesn't exist query: SELECT dft.type, dft.title, dft.locked FROM date_format_types dft ORDER BY dft.title in /home/drupal6/public_html/sites/all/modules/date/date_api.module on line 1974.

Run update.php and run again and one more time, which should have 0 updates.
If you don't get 0 updates at some point:
In /public_html/sites/all/modules/date/date/date.install change
if (!db_table_exists('date_format') || !db_table_exists('date_format_types')) {
to
if (!db_table_exists('date_formats') || !db_table_exists('date_format_types')) {
per instructions at drupal.org/node/496874, "Never ending update on #6005"
update.php again. Completes this time.

cp -R views /home/drupal6/public_html/sites/all/modules/.
enable all three views modules
run update.php
* user warning: Table 'drupal6.views_view' doesn't exist query: SELECT DISTINCT v.* FROM views_view v in /home/drupal6/public_html/sites/all/modules/views/includes/view.inc on line 1295.
* user warning: Table 'drupal6.views_view' doesn't exist query: SELECT * FROM views_view WHERE name = 'upcoming_events' in /home/drupal6/public_html/sites/all/modules/views/includes/view.inc on line 1247.
run update.php
* user warning: Duplicate key name 'name' query: ALTER TABLE views_view ADD UNIQUE KEY name (name) in /home/drupal6/public_html/includes/database.mysql-common.inc on line 403.
* user warning: Table 'cache_views_data' already exists query: CREATE TABLE cache_views_data ( `cid` VARCHAR(255) NOT NULL DEFAULT '', `data` LONGBLOB DEFAULT NULL, `expire` INT NOT NULL DEFAULT 0, `created` INT NOT NULL DEFAULT 0, `headers` TEXT DEFAULT NULL, `serialized` SMALLINT NOT NULL DEFAULT 1, PRIMARY KEY (cid), INDEX expire (expire) ) /*!40100 DEFAULT CHARACTER SET UTF8 */ in /home/drupal6/public_html/includes/database.inc on line 529.
Failed: ALTER TABLE {views_view} ADD UNIQUE KEY name (name)
Failed: CREATE TABLE {cache_views_data} ( `cid` VARCHAR(255) NOT NULL DEFAULT

cp -R calendar /home/drupal6/public_html/sites/all/modules/.
enable calendar, calendar ical
run update.php

cp -R * /home/drupal6/public_html/sites/all/modules/.

Enable the remaining modules that go with this site.
update.php

og gave error messages.
On some sites, Organic groups enabled itself even if I wasn't using it.
admin/build/modules - disable og and og views integration

Calendar and upcoming events list aren't in menus.
References: "Calendar gone after upgrade," drupal.org/node/457588, and drupal.org/project/calendar, particularly Troubleshooting section.
admin/build/views/tools/convert
Calendar has been converted. Convert upcoming_events. View needs work.
upcoming_events: change style from unformatted to HTML list, unformatted list.
change row style to fields.
Change from Page to Block
Save. OK. Block appears and drupal6.nccommunities.org/events-list works.

I had a lot of trouble getting the calendar to work.

drupal6.nccommunities.org/admin/build/views/tools
Clear Views' cache

drupal6.nccommunities.org/admin/build/views, enable calendar view, edit calendar

Follow the instructions on pp. 329-331 of the O'Reilly "Using Drupal" book. I think order is important.
Change Arguments, "Date: Date (node)" under Date fields, uncheck "Node: Updated date" and check "Content: Time (field_time)"
Add Field, "Content: Time (field_time)
Click Fields "Node: Updated date" and Remove this field.
Change "field Content: Time (field_time)" Label to None
Filters: Add Node type, Calendar event
Save View
Events Calendar now appears in the primary menu and drupal6.nccommunities.org/calendar works.

Check content types. Name and description for book have reverted back so need to edit them -- for example, change Book page to Resource item and Description to "Resource items are handouts, brochures, documents, presentation slides, and other materials that you use and are willing to share with others. The Resource collections are a collaborative writing effort - you can add new items, edit items that you entered, and change the placement of items. So when you have some information to share, go ahead and add it. It is easiest to create a new item while viewing a related item in a collection, but you can also create one here. You'll need to chose a parent item, from the dropdown menu. See the "How-to" instructions for more information."

admin/build/block, change book navigation to none.

admin/build/menu-customize/navigation, enable Add information (usually known as Create Content).
In one case, it showed as enabled but wasn't in the menu, so I disabled it and then enabled again and it now shows in the menu.

Compare navigation menu items to drupal 5 and reorder as needed.

Front_page now requires setting a permission for anonymous users -- gave all roles permission.

Put online and check what authenticated and anonymous users see.

Put site offline.

mysqldump -u user -ppassword drupal6-database > xxx-nccommunities6-date.sql.txt

put site onlnine

-------------------------------------------------------------------

Change all nccommunities sites to use Drupal 6.

Log into each site -- hallman, huntresses, nccommunities, ncla, ncwu, oclc, pininc, scrapexchange.

Put each site in maintenance.

Back up sites and files.
cd /home/drupal
delete the old sites and files
cp -R public_html/sites sites and public_html/files files

Remove all old files and directories from the Drupal 5 installation directory.
cd public_html
rm -fr *
chmod -R 777 sites/all/modules/fckeditor
rm -fr *
rm .htaccess
public_html empty

Copy Drupal 6 into the installation directory.
cd
cp -R /home/drupal6/public_html/* /home/drupal6/public_html/.htaccess public_html

Copy backed up "files" and "sites" directories to the installation directory.
cd public_html
rm -fr sites
rm -fr files
cd
cp -R sites public_html
cp -R files public_html

Remove all Drupal 5 contributed modules from the sites directory.
cd public_html/sites/all/modules
rm -fr *

Copy Drupal 6 contributed modules to the sites directory.
cd
cp -R /home/drupal6/public_html/sites/all/modules/* public_html/sites/all/modules

Replace all Drupal 5 databases with their Drupal 6 versions.

mysql -u user -ppassword xxx-database < /home/drupal6/xxx-nccommunities6-date.sql.txt

-------------------------------------------------------------

CiviCRM

Upgrade CiviCRM from 2.0 to 2.1
Instructions for upgrading civicrm 2.0/drupal 5 to civicrm 2.1/drupal 6 are at
wiki.civicrm.org/confluence/display/CRMDOC21/Upgrade+Drupal+Sites+to+2.1

cd
tar -xzf civicrm-2.1.6-drupal.tar.gz
cd public_html/sites/all/modules/
rm -fr civicrm (remove version 2.0)
cd
mv civicrm public_html/sites/all/modules/

cd public_html
chmod a+rwx -R files

Go to:
nccommunities.org/sites/all/modules/civicrm/install/upgrade.php?reset=1

I had problems. Firefox wouldn't run upgrade.php. Wanted to download it. Posted at forum.civicrm.org/index.php/topic,10384.0.html
Lobo says: "This is probably a .htaccess issue (i suspect). can u check if there are any .htaccess files for each directory in that path and rename them to test things"

The .htaccess in public_html looked OK. But there was also one in the sites directory. Renamed it. Upgrade still didn't work. Cleared cookies. Success! nccommunities.org/sites/all/modules/civicrm/install/upgrade.php?reset=1
worked.

For each site that uses CiviCRM:
xxx.nccommunities.org/?q=user
clear cookies.
http://xxx.nccommunities.org/sites/all/modules/civicrm/install/upgrade.p...
CiviCRM upgrade successful

For each CiviCRM site:
mysql -u user -ppassword xxx-database
TRUNCATE TABLE sessions;

Upgrade from 2.1 to 2.2
NOTE: You MUST enable CiviCRM 2.1 on each site that uses it!!!
"CiviCRM upgrade is part of civicrm functionality and works only if you have civicrm module enabled." See http://forum.civicrm.org/index.php/topic,9809.msg42370.html#msg42370

When I enabled civicrm, I got a blank screen.
Created php.ini file in public_html containing:
memory_limit = 64M
CiviCRM works on all sites that use it.

Replace 2.1 with 2.2.
cd public_html/sites/all/modules/
rm -fr civicrm
cp -R /home/my-test-site/public_html/sites/all/modules/civicrm public_html/sites/all/modules

http://nccommunities.org/civicrm/upgrade?reset=1
Upgrade CiviCRM to Version 2.2.2
CiviCRM upgrade was successful.

Run for each site that uses CiviCRM.

Put each site online checking what each one looks like to anonymous and authenticated users.