While we still have lots of work to do, Kate Gorman, Josh King and I made a lot of progress at UPTV this week. http://www.urbanapublictelevision.org/ is UPTV's first Drupal site. In some ways that was helpful, but launching a basic Drupal site can be a lot for someone to take in... let alone all of new hardware, networking changes, file transfers, encoding, projects, reservations, error logging, etc, etc. I'm sure Kate was over whelmed at times, but she persevered all week and the small test group she brought in yesterday was blown away when they saw the 5-10 minute videos they pulled from DVD with Handbrake show up as in Flash on the server a few minutes later.
The UPTV staff and users are excited about the new site and functionality they now have access to. The folks at Acorn Active Media are excited to start customizing the TVFrame Zen sub theme and truly making the site their own.
In addition to the UPTV staff, the City of Urbana's IT group, and some of UPTV's users, I got a chance to meet with Jack Brighton (who runs http://www.pbcoreresources.org/) and Colleen Cook (a Digital Media Coordinator at the University of Illinois). Jack had some really great advice about how to move forward with trying to improve the PBCore genres to be more like ESCORT's hierarchical genre definitions. I'll post more about that conversation and the changes we are making because of it next week. Colleen was really interested in making sure we were taking accessibility into consideration. I hadn't been, but I was glad to discover that there has been a lot of accessibility work put into JW Player (http://www.longtailvideo.com/support/tutorials/Making-Video-Accessible).
I'm sure with the support of everyone I met while I was in Urbana, UPTV's site will continue to improve over the next 30 days until they officially launch to the site publicly at the end of February.
A few technical issues we encountered this week that I think we need to note somewhere before I forget them....
- Use of symlink for files dir
We need to make sure files dir is mapped to the RAID before we start chewing on video. I filled /usr up instead of the 1.5TB of RAID. This creates some really odd error messages in Media Mover when it can't process video due to lack of space that seem look permission errors.
Could not save file: sites/default/files/media_mover/ffmpeg/converted/1/MagicOnMainStreet.mpeg, to: sites/default/files/archive_new/Test - vhost changes for handling large files through a symlink
<VirtualHost *:80>
ServerAdmin user@domain.com
EnableSendfile Off
EnableMMAP Off - IMCE changes
Unless the video related dirs are ignored, IMCE take forever to add images because it tries to check the user's upload quota and to do that it has to calculate the dir sizes with gigs of video.
page.inc
898 $exclude = array('.' => 1, '..' => 1, 'CVS' => 1, '.svn' => 1, '.htaccess' => 1, 'ingest' => 1, 'ingest2' => 1, 'media_mover' => 1, 'archive_new' => 1, 'imagecache' => 1, 'css' => 1, 'js' => 1); - changes to ffmpeg_wrapper.module.
We could only get Leightronix to recognize a .mpg file. Media Mover pulls the 'support' file suffixes from FFMPEG. FFMPEG doesn't think .mpg is valid.
413 //HACK, add .mpg extension support
414 $formats['mpg'] = 'mpg'; - Issue with test videos that are less that 1 sec
The error says...
The selected file me7.mpeg.thumb3.jpg could not be saved. Only JPEG, PNG and GIF images are allowed.The issue is actually that I was uploading 1 seconds of video and telling Media Mover to grab a screen from 2 seconds into the video.
- Need to have domain name configured to register for Analytics, GMap, and reCaptcha
- Issue with enclosure in RSS...
There is still an issue with function media_mover_api_rss($node). The enclosure is added the Project's RSS feed, but the file isn't included...enclosure url="http://www.urbanapublictelevision.org/" length=" bytes" type="application/octet-stream"
UPDATE: Brian spent a lovely 3 hours looking at this in more depth today. The issue turns out not necessarily to be with Media Mover. Brian's 95% certain that the media_mover_cache table was corrupted at some point, resulting in the data on the video nodes not being updated to include the completed file url needed by the RSS enclosure. The resolution to this was to empty the media_mover_cache table which seems to have fixed all the Media RSS enclosures, and new videos are now being updated properly. There is a slight possibility that the issue may also be related to Drupal 6's possible TEMPORARY_CACHE flushing issues http://drupal.org/node/226728 but for now we're sticking with the belief that it was a corrupt cache table that needed manually cleared.
Comments
Collecting Patches
In the process of deploying at Urbana & Denver Open Media, we've had to make several small but critical patches to core & contrib. We're constantly working to avoid this, but in some cases we need to move faster than than the available fixes. I'm going to post those patches here, so we can start collecting them (and I'm also hoping someone else might be able to suggest better solutions!)
Patch #1, the ugliest of them all: file_symbolic_links.patch against includes/file.inc
Since we're working with a lot of gigantic files being accessed by multiple servers (encoding servers, web servers, video ingest stations, etc) -- we have to keep the terabytes of data on a central RAID. The RAID is mounted on all of these servers, but in the case of Drupal, we need to be able to write and load files from several specific directories on the RAID -- which means we need these directories to be inside of the files directory (we could use the private file system, but this imposes numerous limitations and configuration challenges that we can't handle at all of the stations). Symlinking the directories is extremely easy -- unfortunately Drupal's file.inc uses PHP's realpath function which resolves symbolic links to their actual path, which results in Drupal failing the file operation as it identifies it as outside the files directory. This patch changes the file_check_location logic to first see if the filepath is a symbolic link, and then process it differently.
Patch #2: filefield-ignore-filesize.patch against sites/all/modules/filefield/filefield.moduleThis was actually an easy fix at Open Media Camp, the global PHP check ONLY applies when you haven't defined an expicit max filesize for the CCK field itself. Thanks to drewish for helping me resolve this one.Since filefield is primarily used for uploading files as opposed to selecting them directly from a local folder on the server, it implements a global filesize validation check against PHP's upload_max_filesize value. Since we are frequently dealing with 2GB+ file sizes, and we don't really want to set php.ini to allow infinite file size uploads, we commented out this portion of filefield.module. It's a terrible, hackish short term solution -- but works until we can figure out the appropriate way to resolve this.
Open Media Foundation
no kittens were killed
Is the file.inc patch going to make its way into core, and/or, is there a discussion about this?
It seems straightforward enough, and harmless enough, I don't see why core wouldn't want the logic to check for symlinks, esp. as the data sets Drupal is handlng grow ever larger.
The main issue for this is
The main issue for this is here: http://drupal.org/node/155, I believe it's been marked as fixed as file_check_location has been removed in D7.
Open Media Foundation