OMP Documentation Related to Synergy Integration
This morning Rich Vasquez visited channelAustin and we discussed the remaining steps needed to integrate the work he has been doing on the Open Media Project. As background, he has been writing open source code (Ruby) to write to and read from Synergy Broadcast System's database (Microsoft Access). He achieved the part of reading from Synergy's db several months ago, and recently completed writing to it. Tackling Microsoft Access has not been easy, but Rich has basically solved the problem.
Where we are at now with the project is the following:
-
Need to make sure we understand exactly which fields in the Events and Title tables in the the Synergy db need to be populated. We're pretty sure that we know this.
-
Need to make sure we understand exactly which fields from either OM Show or OM Timeslot Scheduler correspond to the Synergy db fields. We are less sure, but think we know what these are.
-
Need to understand better the functionality of the Open Media Timeslot Scheduler and where exactly in the code we can find the answers related to the previous point, i.e. the OM field names vis a vis the Synergy field names
We're assuming that the Submit button on the Create Show page is the trigger to send the data from the Create Show page to Drupal db and then to the corresponding fields in the Synergy db.
However, digging deeper, it is not so apparent what is exactly happening. What follows is a list of a few external links to what is on the openmediaproject.org site and links to relevant discussions that have appeared on this GDO. Previously there had been more documentation on the openmediaproject.org site -- some wikis -- but they're not there now. I've gathered this all in one place for Rich's benefit and for anyone else wanting to look at this.
In the notes on the Open Media Timeslot Scheduler, linked below, it says "the current release only supports scheduling to Telvue Princeton playback servers." We understand that, and understand the array of issues that has made integration with the other playback systems difficult, with vendor reluctance to move rapidly toward an API approach being key.
We are also aware that DOM/CivicPixel intends to not focus on playback issues again until February. We're hoping though, that before February, we can make more headway with this.
Rich and I will continue to use this space to work on the OMP - Synergy Integration. And after he scans through the discussion links below and other information that I'm sending him, he'll post questions here.
Rule-Based Scheduling and Programming
http://www.openmediaproject.org/handbooks/project-documentation/rule-bas...
Open Media Timeslot Scheduler
http://drupal.org/project/om_timeslot_scheduler
The current release only supports scheduling to Telvue Princeton playback servers. Tightrope and Leightronix have stated they will add the type of APIs required to get data in and out of their servers, but to date we can only get data out using the Broadcast Synchonization module.
Open Media Show
http://drupal.org/project/om_show
Group.Drupal -- Open Media Project
Synergy Integration - Related Discussions
Modifying om_show content type to work with offline media
http://groups.drupal.org/node/25871 2009-08-30
Synergy Required Fields
http://groups.drupal.org/node/23745 2009-06-29
channelAustin Open Media System Integration
http://groups.drupal.org/node/23711 2009-06-29
Data In / Data Out - Part 2
http://groups.drupal.org/node/22021 2009-05-07
Data In / Data Out - Part 1
http://groups.drupal.org/node/21970 2009-05-06
Does Televue (Princeton) Play Nice?
http://groups.drupal.org/node/21710 2009-04-27
Access Humboldt's archive.org automation
http://groups.drupal.org/node/16547 2008-11-06
Digital Workflow
http://groups.drupal.org/node/16559 2008-11-06
First Draft of Open Media System's Video Schemas
http://groups.drupal.org/node/13803 2008-08-07
CCTV's Main Programming Tables
http://groups.drupal.org/node/14277 2008-08-26
We're assuming that the
For the Cablecast intergration I am working on I use hook_nodeapi:
http://api.drupal.org/api/function/hook_nodeapi
function cablecast_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
if (isset($node->type) && $node->type == 'om_show') {
$callback = 'cablecast_show_'. str_replace(' ', '_', $op);
if (function_exists($callback)) {
$callback($node, $a3, $a4);
}
}
}
function cablecast_show_presave(&$node){
// update/create show record in Cablecast.
}
Close...
Close, but transfers to playback shouldn't be part of the save process of om_show.
In most cases, the playback servers aren't big enough to hold all the video in a station's archive. I realize this isn't the case for Synergy or Tightrope where you can pull from external share points, but to make this a solution that will work for servers that don't support that (or in some cases different servers for different channels) we need to trigger the transfer on schedule not save. Another reason to transfer as part of schedule is that the file you want to transfer is the MPEG2 referenced in field_om_show_mpeg2_fid, not the file that is originally attached. In most cases that takes longer than the node save process... though the way DOM's web server been performing lately it might be close for short videos.
I've been adding functionality to the dev release of om_timeslot_scheduler as well as adding a sub module to timeslot to handle pairings. I add more documentation about pairings once we finalize how Ann wants it to work in Denver, but the quick description is om_pairings is an optional module designed to give a portion of a Timeslot Event's time to the Theme that corresponds to the main theme based on the amount of content that has been added to the archive associated to that theme over X amount of time. So if you enable om_timeslot_pairing and assign a Timeslot Event 4 hours to your most popular Theme, the last 30 minutes would be given to the Theme with the least about of content. At Denver, Faith is paired with African American based on the shows added in the last 6 months.
Most stations have only done the first step to enabling user scheduling which is create a few Timeslot Events that are User Schedulable. If you install the dev release of om_timeslot_scheduler you will find this interface at admin/settings/om-timeslot-scheduler that looks like this...
When om_timeslot_pairing is enabled and user scheduling is enabled, users will be presented with X number of upcoming times their show could air that are either directly associated with the Theme or associated because of the pairing. When that step is completed, om_timeslot_scheduler will attempt to transfer the video, metadata, and scheduling information to the playback server associated with the channel identified for primary airings.
If the MPEG2 doesn't exist, the transfer attempt will be saved to try again next time cron is run. Future attempts to find open times to air another program will see this pending scheduled event as an unavailable time. Station directors will be able to see pending scheduling to take some action if there is something wrong with the encoder or the specific files users are trying to encode. While FFMPEG has been working well in Denver, we still run into files that can't be converted from the original codec to an MPEG2.
I think the next step is to move the SOAP logic for the Princetons we've been using in Denver and PCM into another om_timeslot_scheduler sub module and add the channel/server registration UI to om_timeslot_scheduler. Based on the server sub modules a station registers, that will determine what server types are available to associate with each channel.
This reverses the workflow from where I think Darrick was going, but with this approach every time a scheduling function is asked to add a show it can be handled the same way.
Darrick can add his Tighrope logic to a similar sub module. If Tightrope ever adds a supported API, we can update that sub module. This will hopefully give Rich enough structure to move forward with a Synergy sub module as well.
I know this is the first time some of you have ever read about pairings. That was in large part because when we started digging into this we realized that what people thought was happening and what the Python code that drove the Drupal 5 version of Denver Open Media was doing were not the same. Pairings were hard coded into the Python logic and didn't take into account reoccurring slots given to producers within a Theme when determining the how the time should be allocated based on the pairing. By adding several reoccurring faith based shows before a larger block of time allocated to the faith theme, Denver was actually inadvertently skewing their programing towards faith since the autoscheduling logic had no concept of what was scheduled before and after the start and end time it was looking at.
We've adjusted our approach to Timeslot Events Associated to Project. Those are now being placed within larger Theme Events and being treated as if a show is already scheduled. At some point in the autoscheduling process, if a Project hasn't added a show to their Project Timeslot Events that Event will be filled with other programming based on the Theme it is in. We've been talking about 3 days in Denver, but that will be something each station can configure themselves.
Hopefully this gives everyone a better idea of where things are at and where we are headed.
If your Princeton server does
If your Princeton server does not support holding all of the show metadata and files you have in Drupal then hook_presave would not be implemented for that server. Our cablecast server does. So, I've implemented it. Cablecast also supports creating show records with no associated media record (mpeg file, tape, etc.). So I've implemented that to. Also, what about the case if someone just wants to use OM show without OM Airing or the Timeslot scheduler (i.e. to sync with archive.org.)
Syncing the mpeg file is a different hook. When the mpeg file is updated in Drupal it is synced with the server. In your case you wouldn't implement that either.
When om timeslot scheduler creates a schedule it is synced with the playback server. In the cablecast case we just have to send the schedule to the server. Then our programming manager has to review it in Cablecast and run Auto Pilot there to activate it. This spits out a report which tells us what tapes to load where and if any mpeg files are missing.
In the Princeton case when you send the schedule you will have to write code to make sure the Princeton has all the show data and mpeg files it needs to playback that schedule.
Syncing the airings seems to work the same for all servers. Granted some have lacking rss feeds. So the plugins will have to write a feedapi_mapper which is server dependent.
So workflow wise:
Cablecast/Synergy:
Add/update show info in Drupal -> show record added/updated in Cablecast (hook_presave).
Add/update mpeg2 file in Drupal -> mpeg2 file added/updated in Cablecast (hook_cck_update).
Add/update schedule in Drupal -> schedule added/updated in Cablecast (hook_schedule).
Princeton:
Add/update show info in Drupal -> Check if show is currently scheduled in Princeton (hook_nodeapi)
Add/update mpeg2 file in Drupal -> Check if show is currently scheduled in Princeton (hook_cck_update)
Add/update schedule in Drupal -> schedule added/updated in Cablecast and show metadata and mpeg files synced (hook_schedule).
archive.org/youtube/blip:
Add/update show info in Drupal -> show record added/updated in archive.org (hook_presave).
Add/update mpeg2 file in Drupal -> mpeg2 file added/updated in archive.org (hook_cck_update).
Add/update schedule in Drupal -> create playlist for on-demand channel.
All servers:
Sync airings via feedapi.
My feeling is to maximize the flexibility of the OM modules we should provide the opportunity to create modules which sync OM show metadata and files with third party stores (be they playback servers, streaming services, etc.) Allow OM show to be linked to a number of configurable optional media sources as opposed to requiring they are linked to a mpeg2 file.
The Princeton can store the
The Princeton can store the metadata, but when the MPEG2 are added they are locked on the specific server. The only way to delete them is through Princeton's API or the web interface.
I've committed om_timeslot_princeton to CVS. It's still very Denver specific, but I'm adding the variables to om_timeslot_scheduler. Since there are 2 types of transfer on Princeton (one with video and metadata and another without the metadata) I used om_timeslot_princeton_transfer($nid, $channel, $type_of_transfer), but that can be changed.
There is already functionality in Media Mover to move or copy files to Amazon using their S3 interface. Now that Archive has implemented and documented their S3-like interface the logical place to implement transfers is there.
The breakdown of functions makes sense, the only difference is really where/when they are called and adding om_shows with no digital files.
I think the compromise is going to be to add a 'transfer on Show creation' option to om_show's configuration (which doesn't exist yet) that calls the functions, but that will only work in locations that are skipping the original attachment step adding MPEG2s directly to the om_show (Austin and Portland).