I'm in the tail end of integrating our OMP install with our Cablecast Server. In light of the conversation about this on IRC and the recent release of the new Cablecast API I'd like to let you all know where things currently lie on our end.
Open Media Show (http://drupal.org/project/om_show):
To make this work without requiring a mpeg2 file I need to have a CCK field to put in the length of the show. Currently the length is calculated by ffmpeg using the attached mpeg2 file. I'm currently using a field (field_om_show_duration_estimate) to put in the length of the time for shows which are not mpeg2 files (VHS/SVHS/Umatic/Satellite feeds).
I did have a field to store the cablecast show id in the om_show node but now that cablecast has an API I don't need that anymore. I have yet to migrate away from using the field. After a lot of discussion with Kevin it was decided om_show can't hold the external id because the relationship between show nodes and external shows is one to many not one to one.
I got rid of filefield_local. It allowed you to select from a list a uploaded file to insert into a show node. Being a very impatient person I didn't like the workflow of having to wait the two hours for a 1 gig file to upload then remember I did that and turn around and add the show. My current implementation uses a media mover configuration to harvest from a upload directory and pull the files into their respective node. So you can create the show node then start the upload the file and you are done. The same workflow works for shows automatically recorded by the cablecast system (like satellite feeds.)
FileField Paths: I hate this module. After hours of frustration I realized this module had a "feature" of deleting file paths if the directory had no files. So, I got rid of it too. I have media mover configurations which move the files to their proper directory with the proper naming convention.
Flash (field_om_show_flash): I'm moving away from this field to using a Embedded Video field. Some of our shows are up on blip, some on vimeo and the rest on our local server (lighttpd with the H264 Streaming Module http://h264.code-shop.com/trac/wiki/Mod-H264-Streaming-Lighttpd-Version2). emvideo works fine with using you're own url (i.e. http://davismedia.org/video.mp4). I got some hacks to fix some bugs in emvideo and also to get jwplayer to utilize the h264 streaming.
ffmpeg_wrapper_remote: I completely rewrote this (http://drupal.org/node/374175). I made it work asynchronously. Given how long it takes to encode a two hour show (usually about two hours) it would have been the only thing my cron process was doing. So I made it so cron just starts the encoding. When the encoding is done the media mover config is notified so the rest of the configuration can happen on the next cron run (adding the encoded file to the CCK field).
Media Mover: I copied the mm_cck module and completely hacked it. The missing features for us were having control over which nodes CCK fields were harvested from (I enabled the option to harvest from a view) and also control over how the files were moved around (move?,copy?,link?,do nothing just add the filepath to the field?) and where to move the file too (as I got rid of filefield_paths).
I also needed media mover to be able to harvest from a directory and extract the show id from the filename so it knew which node to add the file too (as I got rid of filefield_local). Which works great when you already have 500 mpeg files on the cablecast server and you just want to shove the location into the om_show_original_file CCK field.
Airings: Using the feed_mapper didn't work for me. The cablecast RSS feed just doesn't have enough information. It also is missing a unique guid. So, I wrote a function to call the cablecast API during cron to sync the airings. I'm confused as to why airings need a end time since you can easily calculate it by using the start time plus the length of the show. Right now I just put the start time in the node created field and don't use field_om_airing_date at all.
Those are most of the hacks I've done off the top of my head. I would like to contribute as much as possible and I've been posting bug fixes and feature requests as I come across them.
Comments
patches, please?
patches, please?
=)
Aaron Winborn
Drupal Multimedia (my book, available now!)
AaronWinborn.com
Advomatic
I do have one for the bugs
http://drupal.org/node/573730
Many of these changes make
Many of these changes make sense for your workflow, but there are only two that I'm concerned about. Changing the way MPEGs get attached to om_show is great. I've been looking at the option of mounting a Princeton's vol1 into Drupal's default/files dir and altering filefield_local to only show "unlocked" files. On a Princeton, files are locked once the metadata is added. This would allow stations that only have a single Princeton to use om_timeslot_scheduler without needing FFMPEG or Media Mover.
The areas we're concerned with are...
om_shows without MPEGs - We've talked about this on IRC in the past. Let's look at self scheduling first. With the current self scheduling workflow, what do you plan to do for shows with no MPEG? The current code looks to see what channel the Theme or Project timeslot the user is trying to schedule into and uses the appropriate playback server submodule to handle the check to see if the file is already on the playback server, transfer it if it isn't, and then insert the scheduling information. Does Cablecast allow you to add a VHS tape to the schedule?
Autoscheduling is more problematic as I'm grabbing 100 om_shows based on the Timeslot Rule, ruling out any that have aired too recently, and scheduling the rest before moving on to the next Timeslot Rule for that Timeslot Event. Timeslot Rules can be anything you can query from om_shows with the most votes to om_shows with videos within a specific duration. I keep working through the Rules until the Timeslot Event is full.
I wasn't planning on treating the servers or shows differently. In fact, I was hoping to be able push the shows and schedule to multiple servers from different vendors to help stations migrate between platforms. In the current code, once you have a show node and a channel, it's simply a matter of looking up what submodule function to call to transfer ando/or scheduler that show.
When scheduling, it's easy enough to ignore om_shows that have a value for field_om_show_duration_estimate in the SQL of our Timeslot Rules, stations supporting om_show's without MPEGs would need to know how to write that query or we need to add something Timeslot Server to either enable servers to "allow non-digital Shows" or limit them to "digital only". If the server only works with digital files like the Princeton, the field_om_show_duration_estimate NOT NULL will be added to the SQL WHERE.
Other than scheduling, Views, theming, RSS, and yet to written sharing functions will all have to determine what an om_show actually is every time we handle one is going to add a lot of overhead... and I don't really understand the benefit. For BAVC we have already added an embedded_show content type. While an embedded_show could created from a local om_show, an om_show cannot be created from an embedded_show because we lack the broadcast quality MPEG.
I've also been working on adding om_ext_resource content type for overlays, switches, and interstitial that has no real duration. Im using this to register content like our flash ticker that is already registered on a Princeton.
Airings - The main reason for using the start and end dates is they were available from the Princeton when we started this. Now that we've been using Date CCK fields, I think it makes a lot of sense to keep them. While they don't need to be populated from RSS, om_airings are used to create the grid of start and end times Timeslot Scheduler is working around. om_timeslot_scheduler_find_openings() essentially reverses the start and end times of the airings to show openings. Using an end date also allows us query events that started before the time period we're looking at, but haven't ended. It just makes sense to store the start and end time if you calculate it once? The code to do a node save with a date is in om_timeslot_scheduler_add_airing. It is more complicated, but has advantages. What are the advantages of not storing the end time?
I believe we can converge.
I believe we can converge. Cablecast does allow us to schedule a playback video or any other device we have a controller for. We can also record to mpeg when the tape gets played back. We have over 7000 tapes in our library going back to 1985. It is a very valuable asset we want to get archived online.
Easy enough to add to the server configuration.
I have yet to have to do this. I'm using the same theming, views etc for om_show regardless of whether it has a mpeg file. However, I am looking into using a default screen capture for shows with no mpeg.
Well this functionality is useless if the station can't migrate over their non-mpeg shows. I know DOM decided all of the content produced by the community through the last PEG facility was worthless and threw it in the trash but I doubt most other centers will agree with that. See http://en.wikipedia.org/wiki/Book_burning for how I feel about that.
I also doubt anyone is going to be moving to adding new vhs tapes. With the exception of satellite content all new content is already encoded. So having om_show support non-mpeg content is only so we can start migrating from our cablecast system to Drupal.
Most of the issues you talk about are easily handled. The sql queries should be pulled from user defined views as opposed to being hard coded where ever possible. There might be a few fields only used for non-mpeg content required in om_show. And there might be a couple server specific admin settings added (i.e. keep drupal/cablecast shows in sync, allow non-mpeg scheduling.)
I think it all should work and I'm trying to document the best I can what I'm doing so I don't end up reinventing the wheel or creating code that is too site specific.
First, no one in our
First, no one in our organization burned down the old Denver Community Television or their content. I wasn't around, but my understanding is that the original organization burned itself down. When the new organization started they didn't have the space we have today, so there was a lot of content that was lost primarily because there was no one with the space to store it. That's unfortunate and something we're working to prevent from ever happening again by moving digital content into collections on Archive.org.
Working to avoid reinventing the wheel or creating code that is too site specific is what we're all after. It's the only way we really benefit from sharing our code. You, more than anyone, know that I'm still guilty of hacking in Denver specific code to get something done with hopes of having the time to convert that to a generalized solution at some point in the future.
After talking this through with Brian today, I think we have a solution that will work for everyone and move us one step closer to offloading some of our digital content to Archive.org.
We should add 2 fields to om_show. A checkbox for om_show_requires_ext_resource and a nodereference to the resource node. In Cablecast case, the reference would be to a om_analog_media content type. As much as I hate it, it would probably make the most sense to add that as a submodule to om_show. In our case, we'll reference an om_archive record. I think the majority of our Timeslot Rules will ignore om_show that requires an external resource, but if a few Timeslots will look for old shows that have becomes popular again. We'll write something to get the MPEG back from Archive.org.
The nodereference should allow multiple references so an om_show could...
As I stated several times... we're not going to write a tape management system, but wouldn't stop you from doing that in a way that works with the reset of the Open Media configuration. You can add whatever fields you need to om_analog_media and pass those values to your cablecast module as needed.
I really think this is the best way to move forward, but look forward to your feedback.
Sorry about the quip about
Sorry about the quip about burning tapes. I meant no offense and highly respect you and everyone else who works there.
I think the solution sounds great. Having nodereference to the resource node is very much like the Cablecast schema. It has a show record which link to reels. So om_show linking to resources is similar. Plus, it allows for site specific implementation of the resources.
Cablecast at DOM
In case no one has mentioned it, we received a demo box from Tightrope this week.
We hope/plan to move one of our channels off the princeton and onto the Cablecast to help support OMP integration with both. The team is pretty overwhelmed right now with the work for BAVC, but we've set aside a full 3-months to focus on DOM and internal house-keeping beginning in May and at that time, we should be able to contribute more to the Cablecast/OMP integration concerns. I ts great that you are paving the road for us and others in that sense.
We're encouraged by Tightrope's commitment to open-source, and they may be a strong ally in making the OMP a viable automation system that can integrate with any number of broadcast-servers... like an open-source, modular version of something like Crispin.
Tony
Whatever your first issue of concern, media had better be your second, because without change in the media, the chances of progress in your primary area are far less likely. http://denveropenmedia.org
Glad its moving along
Tony,
I'm glad we finally got you a box. I don't work out of the MN office yet, so I'm not sure, but I imagine that the box you got isn't running the beta code with the new Web Service. Shoot me an email when your ready to upgrade it: ray.tiley(at)trms(dot)com
-ray
Nerd at work
So, I'm close to having a
So, I'm close to having a working solution. I added a om_show_external_media node reference to om_show and created a om_cablecast_show with the following fields:
Show ID
field_cablecast_showid Integer
Location ID
field_cablecast_locationid Integer
Local ID
field_cablecast_localid Text
DVD Title
field_cablecast_dvd_title Integer
DVD Chapter
field_cablecast_dvd_chapter Integer
Cue Time
field_cablecast_cuetime Integer
Duration
field_cablecast_length Text
I still need to get rid of using the show id in drupal and move to looking up the nid in the cablecast database.
The cablecast plugin has the following admin settings:
Default Channel:
Default channel drupal show nodes and airings are synced to and from.
Sync Cablecast Schedule
Sync Cablecast Schedule with Drupal Airings.
Sync Cablecast Shows
Create new om shows when show records are created in Cablecast system of default channel. Create/Update shows on Cablecast system when new om show records are created or updated in Drupal.
Support OM Shows with no Mpeg file (i.e. DVD, VHS)
Support OM Shows with no Mpeg file.
I first create a new om_show in Drupal. On creation a om_cablecast_show node is created and added to the om_show. The show is also added to the cablecast system (only if Sync Cablecast Shows is checked). I then update the om_cablecast_show record with the length, cue, title, chapter and format (format is a taxonomy term like DVD, SVHS, MPEG, etc.).
om_show nodes are also created on cron when new shows are added to the cablecast system (only if Sync Cablecast Shows is checked).
I also took the time to import all of our shows in our cablecast system into drupal. But I had to use sybase calls to do this. I mentioned to ray having an option to export to csv in Cablecast might help other stations with migration.
Our program manager can then schedule the show in cablecast.
On cron the schedule from cablecast is imported into drupal (only if Sync Cablecast Schedule is checked.)
I then have a number of media mover configs:
Harvest Edit Suite Dropbox to Originals
Harvest Edit Suite Dropbox to CCK Originals file field.
Users upload files with cablecast show_id at the begining of the name. Via ftp or a file share in our edit suites.
Harvest Cablecast to Originals
Harvest files from the cablecast/MPEG2 directory where Cablecast automatically records to and add them to the CCK originals om_show file field.
I am using my own media mover module to do this. During harvesting the cablecast show id is extracted from the filename and the node that show id belongs to is looked up. While saving to the CCK field it merely add the filename to the field without moving or copying the file. mm_cck only copies the file and adds the filename to the field which wastes a bunch of time.
Harvest CCK originals to Mpeg2
Harvest file in CCK originals show field and convert to mpeg2 and add them to the CCK mpeg2 om_show file field.
Harvest CC licensed shows to Thumbnails
Harvest files in CCK original show fields and create thumbnails.
Harvest CC licensed shows to Mp4
Harvest files in CCK original show fields and create mp4 and add to CCK emfield.
I have a patch in for media mover to harvest using a view http://drupal.org/node/706720. For the conversion I have a few issues regarding my implementation here: http://drupal.org/node/374175 and here: http://drupal.org/node/647082. Suffice to say I have my own versions of mm_ffmpeg and ffmpeg_wrapper_remote. I also still need to work on the code to sort out what CCK original files do not need to be encoded (as they are already in the proper format.)
Instead of using om_show_flash CCK I created a new CCK emfield. That way some shows can be streamed from vimeo, some from bliptv, some from archive.org and some from our local server via lighttpd with moov file streaming patch. I am working on mm_bliptv to add support for vimeo and archive.org and also upgrade it to Drupal v6. If DOM wants to upload old shows to archive.org we could have a media mover config to harvest shows not aired recently, upload to archive.org and then create the om_archive.org_show media record and add that to the show node.
All the show nodes are updated when their CCK file fields are updated via media mover. When a show node gets a mpeg file added the cablecast show record is updated to use that mpeg file instead of the analog resource. This allows for a automated way to ingest our shows (program manager or timeslot scheduler schedules analog show for playback and marks it to be recorded, show plays back, gets recorded. Media mover runs and adds the mpeg file to drupal for the show node. Cablecast record gets updated with mpeg file. Next time show is played back mpeg file is used. Tape gets thrown in box for safe keeping.)
I do not know how to use the timeslot scheduler so I have played with that yet. But the plugin provides the same hooks the princeton plugin provides (i.e. add_show and add_airing).