This are working drafts, so PLEASE offer feedback. The naming of the fields hasn't been finalized either. I didn't change many of the prefixes to make it easier to track where fields have been move to. Much of the naming will be determined by CCK's naming conventions. Other field names may change based on the modules we end up building on (like Media Mover), but if you think there is a better/more logical term we should be using please suggest alternatives BEFORE we write a bunch of code referencing these field names.
We are going to define as much of these content types as possible/logical with CCK. That will allow us to leverage all of the great work being contributed to CCK as well as allow each location to extend the schema. The updated schema reflects the core fields required by the Open Media System modules.
These are the current video related tables from both Denver Open Media and the Open Flows/MNN installs.
.jpg)
This is the first draft at a new schema that would provide an upgrade path for both stations and incorporate what we've learned from running a Drupal driven public access station with the current codebase.
Big changes between these schemas:
- Project content type has been expanded to easily generate more attractive project pages as well as populate the default values for several shows and videos form fields.
- Projects have always been able to have multiple shows/episodes at both DOM and MNN, but now we're separating the show and the video. At DOM, shows have always had at least 2 videos (MPEG2 and FLV). Now we are going to allow x number of videos attached to a show. The videos can vary in format as well as language.
- Both Project and Shows can be scheduled for playback. A Project can have series of rules that are applied to determine which show from the project airs like "show the most recent, unaired episode, if no unaired episode then show the most popular". This can also be used to schedule an English version of a show on one channel and the Spanish version on another.
- The themed blocks currently used by DOM are going to change as well. Projects and shows will be tagged with pbcore genres. To make this easier, each location will be able to tag those genres with their own taxonomy terms. That tagging will be displayed in the UI making it easier for users to select the terms. Shows will automatically be tagged with the same genre tags if nothing is changed. This change is being made to facilitate sharing. When videos are shared, they will include their "official" genre that is the same at every location as well as the free tagging that the specific locations have added to the video. The free tagging is going to be much harder to do anything useful with. The "offical" genre tags will allow the video to be processed at some level since both the Time Blocks and Project/Shows will be tagged with the same taxonomy
- Licensing info will be handled by an updated version of the Creative Commons module
- We've replaced the creator and executive producer fields with a contributor field. This field will use some UI magic to manage a comma separated list of everyone involved in the Project/Show. That will likely be expanded in the future, but should give us what we need to show more information about projects each user has contributed to in their profile page.
- CiviCRM, User Point budgeting and Organic Group fields will be included, but as optional dependencies.
I'm sure there are more changes that I'm forgetting to mention and issues with the schema diagrams, but this should give everyone an idea where we are heading.
| Attachment | Size |
|---|---|
| Open_Media_Schemas_Current(2).jpg | 289.27 KB |
| Open_Media_Schemas_Updated.jpg | 256.46 KB |
.jpg){kind=link}
Comments
Looks nice! A few comments
Looks nice!
A few comments to the last part (others are better at looking at the database schema):
The more I think about this though, there is enough flexibility to set it up any way we'd want. The viewer, for example, would only have to know what theme/time block they are interested in and that theme block is tagged with the genres that only the producer may know about (at least when thinking cable, the web is of course more diverse).
I also understand from the above that each location will be able to choose what genres be made available for selection?
Well, at least those are some ideas to start with. :)
And btw, you're doing some amazing work here!!
My Responses to Daniel's Questions
I responded in separate thread
I can add a separate field for producer, but I think using the JQuery magic we developed to modify a CCK textarea into a semi-structured one to many relationship will handle this. I will post a demo of that technique soon.
No, but a Project can have just one show in the case of a documentary.
At DOM, we air exactly what is submitted (see FAQ #1). I understand the history of running the tape for several seconds before laying anything on it and count downs for tape ops, but is there a reason to include a 10 second lead in a digital file?
This is something you could add with CCK, but I don't think we'll include it as part of the core schema unless there is a need at several locations for this. Aaron Winborn just added a dynamic logo option to his Drupal media player, it could be used in the web playback as well.
Node ids (nid) and version ids (vid) are auto incrementing. Again, fields for location specific tracking information can be added with CCK.
This could be a useful feature and not too complicated to add to the show, but that content would be included in the file that was shared so I don't think we'll include this in the initial release.
We're only concerned with digital playback. If something is on DVD you'd either need to rip it and ingest it or play it during a manually programmed block.
That is included as field_video_sharing, but should be moved up to the project and show. The sharing options with be dependent on selecting a CC license.
Also included as field_video_website_use
That type of information will be included in the Playback Rules which is part of the Timeslot Scheduler. I expect there will be a lot more discussion about that schema as well. Currently the only rules that exist are in the Python script.
This is handled by the aspect ratio and frame size fields in the video table. This should actually support having both a PAL, NTSC, and HD version of a show in addition to multiple languages in those formats.
Thanks for the response!
Thanks for the response! Hmm, there should be an easy way to quote, rather than copy, paste and reformat. :)
3. Can a show not be connected to a project?
No, but a Project can have just one show in the case of a documentary.
Ok. So in our case, currently with organizational membership, project could actually be used as member? I mean, we'd need to easily track shows by member. Although I suppose this could be done with CCK too.
4. What about offset/trimming for a video? If an mpeg-2 file is to start 10 seconds into the video for example.
At DOM, we air exactly what is submitted (see FAQ #1). I understand the history of running the tape for several seconds before laying anything on it and count downs for tape ops, but is there a reason to include a 10 second lead in a digital file?
This is for live recordings. That is, we will feed the video server with a live signal in case of broadcasting live over IP for example, and have that encoded on-the-run. The start and end of the live event might not be easily synced, so rather than re-encoding the file it would be easier to offset, or trim the start and the end. The same goes for studio recordings where no other additional editing is required apart from trimming, if the show is to air shortly after recording.
5. What about a "Logo" field for each project?
This is something you could add with CCK, but I don't think we'll include it as part of the core schema unless there is a need at several locations for this. Aaron Winborn just added a dynamic logo option to his Drupal media player, it could be used in the web playback as well.
Ok. Well, for web playback it's not as important, as info about the show is easily accessible anyway. It's rather for TV playout, when the viewer can't know what show that is running. However, this could probably be set with an RSS feed somehow.
7. What about playlists? For example if one show may be made up of multiple files. This can for example be the case if there is not enough time to export a huge file, but we rather split it into several. Or maybe a project always could have a "project intro" being played as a separate file. Or a separately recorded "presentation" of the show. Of course this could be made to always be part of the exported file, but in some cases time might not allow, and all stations may not be able to combine multiple mpeg-2 files without recompressing.
This could be a useful feature and not too complicated to add to the show, but that content would be included in the file that was shared so I don't think we'll include this in the initial release.
Ok. What about coming up next being automagically added like 60 seconds before the show airs?
8. In "x number of videos", is formats other than digital files possible too? For example if something is to be played from a DVD, or a live show to control the switch. I mean, in case all that is to be included in this database as well. I understand that this feature also can be used to track multiple copies of the same file?
We're only concerned with digital playback. If something is on DVD you'd either need to rip it and ingest it or play it during a manually programmed block.
Ok, I saw this more as a complete database to easily track where the original might be stored, if not in a digital format. But sure, most of the time those tapes or DVDs wouldn't be stored at the station anyway (although we currently do DVD backup too).
:)
Feedback
I have a few points of feedback. Here goes:
In content_type_om_video is field_video_duration_seconds_value just the entire duration of the video expressed in seconds? And then field_video_duration_value is in HH:MM:SS? Or is field_video_duration_value just HH:MM? Was a little confused on that, so maybe either field_video_duration_total_value or field_video_duration_hhmm_value or something a little clearer.
I am slightly confused by the shows/videos explanation you gave. So the "show" is the program and the "video" is the actual media (VHS, FLV, MPEG-2)? If so, I think this is a very good idea and will help account for integration with systems like Synergy (is there a plan to integrate with systems like Synergy? And if so, have you looked clearly at how Synergy schedules and accounted for it?).
In content_type_om_video I think there should be clearer names for field_video_value and field_video_data. I can't clearly understand what they are at present.
If you are separating out shows and videos, then the fields in content_type_om_show should probably also reflect that and not be called field_video_* but instead field_show_*....is that because of the upgrade? I'm not saying it's really misleading, but it just might make sense to stay consistent (and then I would assume that the same fields in content_type_om_project would also switch over to be clearer).
I noticed you have a field_msn_0_value in the airing type. What does that reflect? Is that a reference to content_type_om_video to determine what media type is playing? If not, have you considered the need for that (some systems will want that)
Is field_start_time_value in content_type_om_airing an offset? If not, have you considered the need for an offset?
I like what you've done with the idea of a contributor field, but I also wonder if it's still necessary to allow for singling out one person as, say, the "executive producer" (user ref i would assume). and then also a field for contributors beyond that. Similarly, it may be useful to have a separate field for "staff" contributors vs. volunteer/producer contributors.
if field_notes_0_value of the project type for a project description? If not, a project description (optional) might be good. That way there can be a general description for the overarching series/project as well as individual descriptions by episode.
how are thumbnails dealt with? If i remember correctly, dom has a selection form for thumbnails? So wouldn't a value be stored somewhere for the selected thumbnail? At cctv we also doing something similar to this for selections, and we store a thumbnail number (1-6) depending on which one is chosen. then the rest of the path is generated automatically.
I am surprised to not see production/acquisition date, production time as fields for content_type_om_show. Most access centers I have encountered would find these fields very useful, so while I know you could add them in later, they would be well utilized if they were built in.
I think it would be helpful in these schemas to see what the field types are as well
hope some of this is helpful. thanks for putting these up; great work!
Producer
I generally agree with all these suggestions.
Just a comment about this paragraph:
It would indeed be good to single out who has actually produced a show vs who is the "originator". For example a station/staff may produce a show on behalf of an organization, as opposed to self-produced shows by an organization or individual, or shows produced by volunteers.
Responses to Emily's Feedback
Yes. The values in those fields look like 00:01:13.6 / 73, 00:27:03.7 / 1623.7, 00:58:14.0 / 3494. I'm going to have to deffer to Brian as to why we are currently storing 2 values vs. reformatting them as needed.
No, but we are willing to look at any system we get access to. Some of what is included will be based on who we end up partnering with as part of the Knight grant.
The video file used when a show is aired will be determined by the Timeblock Scheduler. We have just started looking at how that is going to work.
We aren't currently using offsets at DOM. See my response to Daniel. If there are reasons to include an offset in an all digital workflow, please let me know what those reasons are.
These are all CCK field that extend the standard node table. We will post a revised draft of the schema with the field types and possibly shells of the modules that just generate the CCK content types early next week.
Thanks for the explanation.
Thanks for the explanation. Just a few quick points:
First, one of the stations I work for uses Synergy and I have had to integrate it numerous times with their website, so if/when you are ready to start looking at other systems I am more than willing to provide participation from the Synergy end (and they would be, too). There are a lot of folks using Synergy still who are feeling a little stuck, and I think counting them in on this process would be greatly appreciated.
Second, I agree with Daniel about offset/trim. I think in some cases there is a need for a digital offset. For example, encoding a satellite feed for MPEG-2 that will play immediately after. Rather than trimming the program, it is sometimes easier and more efficient to just provide an offset. In other scenarios, there are lots of PEG programs that are not exactly 30/60/90/etc minutes long. In this case, sometimes the program starting next will use an offset in order to get the schedule back on time (using offset to start after initial logo or starting credits). In that case, they don't want to trim the program as they normally would want to show the whole thing.
Lastly, can you better explain the idea behind the manually programmed block?
Based on Daniel's #8 and the responses, I think that even if you guys are only concerned with digital playback, you are going to loose a large base of people who could work with this. I don't mean that this work needs to run a flatman or anything crazy like that. Your stuff doesn't have to be a switcher, but it can integrate a little. It will kind of suck to do all the scheduling in this system, then have it export to mc (which will be the case for many), and THEN have to use MC to schedule all the other ins/switches, etc. Is it incredibly difficult to add in other inputs like many master control systems do? This would just be another cck type and it would have to be integrated into the scheduling component. I am just thinking very generally at this time about it; it doesn't need to do anything fancy, but it would be good to allow it in some capacity...maybe when I better understand your manually programmed block some of this will be answered.
Airings
The Airing table is unchanged from Open Flows/MNN. DOM only has a single field for tracking last aired... and AFAIK, even that isn't used now.
Other metadata standards than PBCore?
Just to complicate matters more I came to think of one more thing. PBCore is afaik used for the public broadcasting community in the US. In Europe the Euopean Broadcasting Union (EBU) recommends another set of metadata standards: http://www.ebu.ch/en/technical/metadata/specifications/index.php. If this is to be used globally, would there be a way to use more than one set of metadata standards? I mean, PBCore could be the default, while EBU could be optional?
I have no idea how the discussions go around this whole issue in community media circles in Europe. I will try to find that out. And possibly PBCore could be used as the metadata standard over here too, as I doubt there is a common standard being used in community media in Europe. Just wanted to open up for this possibility.
I'm open to any advice on
I'm open to any advice on how to actually support both standards. Here's the use case I'm working from...
Let's say a station other than DOM has produced a show about Methamphetamine and tagged it with the PBCore genre tags of Documentary, Interview, Crime, Excerpts and Self improvement. They can include additional tags specific to their location as well as text in the description that will travel along with the video, but the PBCore tags are the only tags the locations have in common... so they are really the only thing the system can act on without some understand of the content and possibly even the context.
These are the current themes of DOM's programming blocks...
Academic/Informational
African American
Arts and Entertainment
Faith Based
General Youth
GLBT
International Issues
Latino
Mile High Perspective
Music
News, Business and Government
PSAs, Promos and Shorts
Well-Being & Lifestyle
These will become a taxonomy vocabulary in the new system. PBCore genres as generic as Documentary, Interview, and Experts could be "mapped" to all of these vocabulary terms, but Self help will likely only be associated to Academic/Informational and Well-Being & Lifestyle. Crime might only be associated with Academic/Informational.
So when DOM gets the new show, the system will automatically include it in potential programming available for the Academic/Informational block.
The way EBU handles genre type terms is much more useful because it includes Escort classification system ( pdf) includes levels. The Methamphetamine show would be tagged as NON-FICTION / INFORMATION > Human interest > Personal problems... and that's probably the only way it would make sense to tag it.
I think the Escort classification system is better than PBCore, but I've been told that the new ACM metadata standard is subset of PBCore so I've been working from that. Hopefully the people in the ACM working group are aware of Escort and have reasons for sticking with PBCore's genres.
ESCORT and PBCore
Yeah, I agree. I like ESCORT better because it's much easier to locate a genre for a program. PBCore is just a long, messy list and I fear that many producers will not really consider enough in what genre to put a program because they are difficult to overview.
I don't have any advice on how to support both standards at the moment, will think about it. And indeed it would be interesting to hear more about the discussions in the ACM working group about the use of PBCore for metadata, there have probably been much good thinking that we could find useful.
Rules for scheduling time blocks
I'm pretty curious about how the rules for scheduling a time block will work. Specifically I'm interested in how rules for projects and shows will work together. Will a slot first be offered to a project and the rules decide what show within that project that is aired? Or will certain time slots be for projects and others for any show, regardless of project?
Or maybe the intention is to set up very advanced rules like:
Well, then the rules can be so complicated that they are actually quite difficult to understand... Although the above actually does make some sense. Old shows will then also be repeated, and at around the same time of the year when it first was aired (good for holiday-specific shows for example), but with less frequency for every year.
Could rules be setup to for example "preserve" the last five minutes before every hour, if a show doesn't happen to be longer than 55 minutes? Then weather and upcoming events could for example be scheduled to automatically run five minutes to every hourr, if there is no show running then. Or would this kind of customization rather be made in the code itself?
Maybe you could you give some more examples of an automatic scheduling setup using this new set of rules?
I'm also keen to know how to set an end date for a particular show, after which it will never repeat (although other shows that are older may still repeat). This is pretty important, as some shows are interesting to air for a certain period, but after that they will always be out-of-date and should not be scheduled for repeat.
With the genre tagging I understand it as a show can appear in any theme block that has the same tag? That is, if a show has 3 genre tags it may be possible to see that show appear in up to 3 theme blocks, right?
I've looked at the Python script currently used at Denver Open Media and am quite excited about the possibilities of filling slots with shows automagically, based on rules, as well as adding block-specific files before and/or after each show, or automatically adding "Coming up next" or "This show will also air on" after a show. Which I suppose also will be pretty easily done?
Also, when this feature will be available there could for example be a wiki with example rules for others to use and get ideas from.
Edit: Maybe this should be in its own discussion thread? But I can't delete it...
I will be posting more about
I will be posting more about the Time Block Scheduler this weekend, but I need to finish the shell module that generates the video fields first.
We've had a few meetings these week to address the feedback on the video schema as well as map out how the Timeblock Scheduler will work. We are adding start and end dates to both shows and project with an option about enforcing the end date. Videos with enforced end date will not be sharable since this goes against the very nature of Creative Commons licensing and Archive.org. Locally, you will be able to use the unenforced end date in your Playback Rules however you want.
Playback Rules are simply a title and a SQL statement. Initially the system will include the Rules we develop here, but each location will be able to create their own Rules. The hope is that the locations using the system will share the Rules they develop. The basic SQL model will also support adding additional modules. Obviously we will include Rules with SQL that use data from Voting and Karma, but another location could add a Rule that was tied to donations/fund raising with an updated version of ChipIn (http://drupal.org/project/chipin) or anything other module they find useful. There is no end to the interesting configurations Playback Rules will support!
Time Block Scheduler + Creative Commons and file sharing
The weekend is saved! :)
Sounds great with the addition of start and end dates (although end dates probably will be more useful)! My thinking was along what I think you mean by "unenforced end date". That is, an end date that is set more because a show may no longer be in the viewers' interest because of it's outdated content. But maybe "enforced end date" may have a role somewhere too...
I will wait with any further questions or suggestions until I've seen more details about the Time Block Scheduler.
About Creative Commons and sharing. Will it at all be possible to share shows with other licenses than CC? I understand you will begin with file sharing based only on CC. But laws may differ in different countries about how CC translates into local practice, which means also other licenses may come into play. In Sweden the moral rights (rights of creators of copyrighted works) is part of copyright law for example, which CC covers only partly.
File sharing between public access channels only, but not necessarily "available for public download" at archive.org, may also be something that many producers would allow, while still not giving their works a Creative Commons license. Film festivals is often an example of this in our little country in the north. Will those films initially not be possible to share with the Open Media tools?
Submitting the same video twice
I like the idea of adding as much info about the user who is submitting the content, and its origins, as possible. This way, we can track programs a little bit more intelligently. One of the core problems that I have seen is users submitting the same original content with multiple descriptions. Over time, some users forget that they submitted a particular video, and then resubmit it with new title data. Then we have multiples of the same video in the system. Or alternately, multiple users upload the same content.
A way around this seems to be requesting and recalling some info, such as duration in seconds, requesting info about the orgin of the program etc.
As an example, let's say a user submits a PETA video featuring Gillian Anderson, and calls it "Rabbits Screaming". Another user submits it as "Gillian Anderson Talks About Giorgio Armani". Same video, different descriptors.
If they are asked additional questions about the origin of the video, and the content, we may be able to remove or restrict duplicates. Some automatic lookups and suggestions (such as keywords, duration, etc.) may also help this. It will require some AJAX and probably a little more load on the server, but should eliminate what I dread as a content curator - like the zillions of Dane Cook videos on YouTube.
I think this is especially relevant as we look at exchanging data across multiple PEG stations. Multiple copies across multiple PEG providers? Yikes!
Also, would like to put in a plug against offsets or "faked" durations. The content should be closely cropped and we should provide some tools for viewers to crop their own GOPs. I went through this using Virage with multiple uploaders, and close cropping was totally key.