Drupalcon Boston 2008 Video Encoding Project

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
jlmeredith's picture

UPDATE

If you would like to be involved in the encoding of DV content, please post the following information and we will contact you with login information for the content.

I am hoping that we will be getting started on video transfer this week. If that is the case, encoding should be ready to begin sometime end of next week. Please post the following information in this thread:

  • Your machine(s) specs - processor/memory/OS
  • Your internet connection specs - up and down
  • How many hours per week you can devote to encoding
  • Do you have the ability to edit DV content prior to encoding - this is primarily to trim beginning and end of video along with boosting gain if needed.

ORIGINAL POST

First I would like to say thank you to all of the volunteers that made recording audio and video at Drupalcon Boston possible. Without you we would not have captured the 150+ hours of audio video that we did. Thank you!

This post will outline the game plan for getting all of the raw audio and video footage moved from computers and tapes to the Drupal community. If you have questions, please feel free to contact me. I am not a professional A/V person, so I am very interested in suggestions and comments about how we can do things better or different.

TO ALL OF THOSE WHO RECORDED AUDIO and VIDEO BUT DID NOT LEAVE IT WITH THE CONFERENCE ORGANIZERS

There were several people at Drupalcon who were recording audio and video from the soundboards and various A/V drops. Please post in this thread, what sessions you recorded, the approximate length and if you have posted the raw files to a site somewhere where can the files be found. We would like to arrange for a copy of the RAW files to remain with the Drupal community for future use if possible.

PLANS FOR RECORDED VIDEO FOOTAGE
Below is the step-by-step plan that has been put in place to deal with the video shot during Drupalcon Boston.

1) All of the raw video footage (90+ Hours) is currently on MiniDV tapes and will be transfered over the next couple of weeks to several SATA drives.

2) Once that footage is transfered to the drives, those drives will be made available to the internet.

3) Individual volunteers will be solicited to edit and encode the video to a standardized format for republishing to Archive.org.

The following are issues that need to be addressed and ideas from the community would be appreciated.

1) What would be the best way to distribute the raw DV footage? Due to the large size of the files, it has been suggested that a torrent system might be a viable solution because of the ease in resuming broken connections. Or a download manager of some kind could be used along with just raw single person downloads. What do you all think?

2) Once each volunteer has downloaded their session for encoding, what would be the best standard format for encoding the video. Keep in mind that all video will be uploaded to Archive.org, so their workflow and systems should be taken into account.

Thanks again to everyone who helped!

Jamie

(tmg-studio)

Comments

For Compressor which comes

mfb's picture

For Compressor which comes with Final Cut Studio 2, you can set up batch conversion and just let it run. It has some presets that work fine, e.g. for iPod/iPhone. Just make sure you use a preset with appropriate aspect ratio (4x3 or 16x9). Custom settings are saved as .setting XML files in ~/Library/Application Support/Compressor, so we could set up the same settings on multiple machines.

On the F(L)OSS side of things, I used a Kino (easy to install on Ubuntu) H.264 export preset for compressing Dries' keynote.

If I help with encoding, I'd rather have DV tapes or hard drive shipped to me than download raw DV. The 90 minute sessions will be 20GB each... Fedex is faster :)

Your Conversion of Dries Keynote looked great

jlmeredith's picture

Can you provide any additional details about the bitrate used along with quality settings for audio and the video as well.

Unfortunately, the directive handed to me by the Drupalcon organizing team was to get the video to a position where it can be downloaded from the internet. While I agree physically having the hard drive on hand would be better, it was decided that the option of download would be better and I agree with this decision completely. Keep in mind that it is our goal to have one individual transcoding each session. Although it will take a bit of time for each person to download, overall having many people work on things at one time will be faster.

I would like to see us settle on a universally available application for conversion if possible. Is there a program that can be used on windows, mac and nix systems?

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Kino ffmpeg scripts

mfb's picture

There are universal apps that can do conversion on the command-line (like ffmpeg) but it seems that most people use platform-specific GUI apps for video encoding.

In Kino I used: H.264 MP4 Dual Pass (FFMPEG) Broadband Quality (medium size, progressive, 564 kb/s). The script is here: http://kino.cvs.sourceforge.net/kino/kino/scripts/exports/ffmpeg_h264_du... and http://kino.cvs.sourceforge.net/kino/kino/scripts/exports/ffmpeg_utils.s... From this we could extract the actual ffmpeg commandline.

Help Converting

Brian@brianpuccio.net's picture

I'm more than willing to help convert. I've got access to a Mac Pro (8x 2.8GHz). I don't have any video software installed, aside from handbrake and ffmpeg. If someone wants to give me the ffmpeg arguments (maybe if that's the software that is chosen) then I'll be more than glad to download and convert (or have some mailed to me).

FedEx

mlhess's picture

If you have people that are willing to convert, and have the hardware and knowhow to do it, make a backup of the drive, and just ship the drive, they can convert it from RAW into something easier to deal with maybe flv, and ship it back.

Happy to

my mum's picture

convert the footage - download or DV fedex i don't mind - I am based in the UK but have a few spare hands who can help out.

ALSO VERY IMPORTANT
Are we able to get presentation details / powerpoints / URLS for the slides? - obviously some presentation demos would be hard to recreate, but mixing good quality screenshots into the presentations will enhance them no end. I'll be using final cut for this but it could be done reasonably by any free video editor.

Great Question!

jlmeredith's picture

Thanks for the support. We will need all we can get from anywhere we can get it :)

I am inclined to say that the goal here is to get the video converted in its raw form. I think in many cases the screen was the main focus of the camera and in those instances where it was not, the slides will be available from the session threads for users to follow along with in an additional application.

Mashups and special mixes are welcome, but I think the consensus is that little time should be spent editing and more time converting. While it would be ideal to try and make the presentations more dynamic, the shear volume of video to be addressed makes extensive editing less desirable to having the video out there for people to consume. I hope that makes sense. Let me know if I am off base or missing something in the consideration of what we should do.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

I'm willing to help...

Alex UA's picture

I'm fine with downloading large files off the internet, though realistically we're talking about many gigs here, so depending on your hopes for a time frame for getting this done, sending out DVDs via priority mail might be the easiest, fastest, and possibly cheapest (if you're not getting free bandwidth from someone) way to get this done.

I also think that this is something that other members of the Dojo would be interested in-- as we could definitely use some of the conference talks as starting points for Dojo lessons.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Count me in

pauldeden's picture

I can help as well. All I need is a quick description of the decided upon format, conversion tools and procedure (command-line arguments if a command-line tool) to convert, and a link to download from the sata drives and I can start.

My answers to the original questions:
1) +1 for bittorrent. It will ease the downloading burden.
2) I don't have enough video experience to talk intelligently regarding which format to use. From my perspective anything easy to view with freely available tools would be best.

One suggestion. It would be nice to have a link to each session's video on archive.org from that particular session's page on http://boston2008.drupalcon.org/sessions. That way it would be easy to find a particular session's video.

Thank you to everyone who recorded! It was nice to know that I could go to one session and still see another one later on.

...And a little looking out for the other guy too. -- Mr. Smith

...And a little looking out for the other guy too. -- Mr. Smith

What kind of codec used for compression?

butterfi's picture

What's the target format for these clips? I'd imagine some flavor of mpeg-4, 2 pass, variable bit rate, etc. But it would useful to have a target spec identified.

In our pre-drupalcon meeting

mfb's picture

In our pre-drupalcon meeting I think we were unanimous on using H.264 .mp4 format. The Kino export preset I used is called "H.264 MP4 Dual Pass (FFMPEG) Broadband Quality (medium size, progressive, 564 kb/s)" . There are also 1152 kb/s and 2240 kb/s presets but you have diminishing returns, how many people really want to download files that large (aside from someone using the archive.org files as raw material from which to edit). See the Kino scripts above if you want all the ffmpeg command-line flags.

H.264 sounds like the way to go

jlmeredith's picture

I like the quality of the video you posted for the Keynote. Lets go with those settings (the medium quality). Can someone post the equivalent for Handbrake, Quicktime, Final Cut, Adobe Premier and any other common conversion tools?

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Here are some sample

mfb's picture

Here are some sample quicktime settings:
File -> Export.
Movie to MPEG-4
Options... ->
Video Format H.264
Data Rate 500kbps
Optimized for Download
Image Size 320x240 (standard) or 384x216 (widescreen)
Frame rate 24
Key frame every 240 frames
Video Options... Best Quality
Audio AAC-LC
Data rate 64kbps
Channels Mono
Output sample rate 44.1khz
Encoding quality best
Streaming disabled

UPDATE

jlmeredith's picture

I am hoping that we will be getting started on video transfer this week. If that is the case, encoding should be ready to begin sometime end of next week. Please post the following information in this thread:

  • Your machine(s) specs - processor/memory/OS
  • Your internet connection specs - up and down
  • How many hours per week you can devote to encoding
  • Do you have the ability to edit DV content prior to encoding - this is primarily to trim beginning and end of video along with boosting gain if needed.

Thanks to everyone for their help.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

My Specs

Brian@brianpuccio.net's picture

Mac Pro 8x 2.8GHz (though only 2GB of RAM, looking to upgrade that once the funds are available)
10 mbit down, 1 mbit up (in practice a little less than that)
Overnight, 7 nights a week, so at least 50 hours a week, if not more
Depends, do iMovie do this? Or some sort of OSS that runs on OS X?

Computer specs

mflage's picture

Since I'm a sysadm I have quite a few computers at my disposal. The main ones I'll be using are a Xeon at 2.33 GHz (2 GB ram) and a P4 at 2.26 GHz (2 GB ram). I'm also working at setting up some sort of encoding cluster using all the different computers I've got in the network. But this all depends on the multithreading capabilities of the codec we go for.

Theoretically I have access to a internet connection in the Gbit/s, but in practice I'll say 100 Mbit/s both ways, so that should be more than enough :)

Since easter is coming up, I'll be physically separated (sob) from this network and its resources, so I won't be able to do any editing the first couple of weeks. But as long as the videos are edited beforehand I can just write up a script that will encode continously. So I'd say 24/7 as long as the movies are sent to me edited.

Machine available

lyricnz's picture

I'm happy to run some encoding, if you can provide the magic spell to do it. Machine is dual cpu dual cure AMD 2214 HE with 4GB of ram, and 500GB of disk. 100mbit up and down. Fedora 8.

Simon Roberts
Taniwha Solutions

Small suggestion/request

rcross's picture

Would it also be possible to include in the encoding/editing process a package of just the audio files for each presentation? For those of us without unlimited bandwidth (or slower connections) this would really be beneficial for most presentations to be able to see the slides and listen to the audio.

--Ryan

This is definately planned.

jlmeredith's picture

There was a coordinated effort to record all of the audio at the con and I am working on contacting those who did the recording. Hopefully we will have that shortly as it will take much less time to get post processed. Once we know what raw audio we have, we will then be able to look supplementing that with audio pulls from the video. I know there were some sessions that we got audio but no video and visa versa. Keep looking here for updates.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

OK

nitsunto's picture

I have a 3.0ghz x 8 mac pro 10GB ram machine and a 2.8 x 4 4GB ram mac pro. Osx 10.5.2 on both
If we can come up with batch settings for compressor/squeeze i can dedicate a nearly unlimited amount of processing time as long as it is batch.
10 megabit down (more like 4 really) 1.5 up (ish)
yes, i can edit.

I have four machines all

Steven Jones's picture

I have four machines all fairly quick, though none massively quick all with plenty of RAM.
All are fairly non busy, so would happily let them churn for as long as is needed.
Network, I have access to a university network that blocks bit torrent, but is stunningly fast, or if you were going to use bit torrent, then 8Mbit download, 1.5Mbit upload.
Of course I can edit!

Let me know how I can help!

Also

my mum's picture

Can we get hold of the slides/notes for each presentation? It would be benificial for all if these were dropped into the timeline.

Machine-wise I have 4 available, with 16m up and down - I will probably use compressor to batch the files overnight after capture (can run pretty much 24hr a day), then edit the smaller file but if any alterations are needed on the master, they can be done at the initial capture.

The presenter slides should

jlmeredith's picture

The presenter slides should be available at each of the session tracks on Drupalcon.org. I will be doing some contact work first of next week to make sure presenters get all of their assets attached to the sessions.

It would be great to integrate the slides into the encoded files, though we should consider the extra time that will require and the delay that it will present to the overall encoding process. The primary goal expressed to me was to get the RAW video encoded as fast as possible with only minimal editing (trimming the beginning and ends of the video). Given this goal, how do you think that slide integration will affect delivery time?

One thing to also consider is that many of the videos are shot with the camera pointed at the screen during the presentation. I am open to anything. Just want to make sure we all are on the same track so we can insure consistency in the content.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

it would effectively

my mum's picture

mean the prep time for each hour long session would be incresed by 60-80 mins as the correct time to insert a slide would have to be judged - I can usually get this down to 20mins or so by scrubbing the playhead (particularly if a camera image of the slide remains on screen - great for sync), but it can get complex for some 'demo' presentations.

I am still in favour of getting these presentations as raw footage 'topped and tailed' with both video and audio downloads, then going throught the archive and performing any post-production. Using the archive.org h264 versions should also speed up and editing/processing time and also allow us to see when the slides will just not render well at our chosen format settings (a necessary compromise). It also means those that just can't wait / just don't want the slides, can view the raw items sooner.

Just to make sure we are on

jlmeredith's picture

Just to make sure we are on the same page, you are in favor of going with the 'topped and tailed' method and then reworking with slides if needed? Would it be beneficial to assess the encoding format based on what post processing applications are being used. Are there any certain compressed formats for say Final Cut that can be edited in the timeline without having to be re-rendered when the slides are inserted?

I worry about the degrading the video by reprocessing the compressed form. I am not opposed to inserting the slides during the first post production, I just want to make sure it does not add significant time to the process and that we have a unified effort for getting it done the same for each video.

Right now it looks as though we will have plenty of volunteers, which is great! We are figuring 20-30 hours per drive with some of those being 1.5 hours sessions so 20 volunteers should be enough. If everyone can get their video segment downloaded in the first 3 days after the drives are loaded at OSLOSU then that gives about 4 days for post and re-uploading before the next round of raw footage comes around.

Thoughts? Keep in mind, nothing is set in stone and we have 4-6 days to work this out. Everyones opinion is needed.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Having watched videos from

Steven Jones's picture

Having watched videos from past DrupalCons, I can say it's MUCH better if the slides are in there, even if the camera is pointed at the screen most of the time you can hardly read it.

It will be extra effort, but it will be so worth it. Let's just put them in from the start.

Did we get any replies for the slides?

my mum's picture

I can't immeddiately see any on the tracks - am happy to chase people down but wondered if there was any response from the initial contact?

EC2 might be just the ticket...

ngiarratana's picture

Sorry to come late to the game but I was wondering if Amazon's EC2 cloud computing service had been considered for this purpose. It's designed specifically for this type of surged processing need. It basically lets you provision as many linux boxes as necessary for whatever period of time you need. There's no up front cost and it only charges you for the amount of time you use the boxes + bandwidth to/from. If people are interested, I can provide more information.

I think this was considered

jlmeredith's picture

I think this was considered at one point in the discussions. The only major issue that arose was how to get the raw DV to the machines for processing. Uploading, even on a fast connection would not be realistic since we are talking about over 1 terabyte of data (1.135 to be exact). If there were a way to move the drives to the EC2 system, that might be a good option, though we would need to look seriously at the cost implications.

My understanding is that using a system like EC2 would result in a significant expense. Can you give an idea of what the cost and time frame would be to encode raw DV to Medium quality MPEG-4 H.264? We are always open to options.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

S3/EC2

matt@antinomia's picture

S3/EC2 would cost $0.10 USD for each GB transferred in, $0.10/instance-hour for each of the instances/machines (~$150.00), plus $0.18 per GB transferred out for the first 10 TB. This works out to $113.50 for the data transfer in. 20 instances @ 24 hours a day for 3 days (not sure if this is a proper estimate) works out to $144. Hard to estimate data-transfer out, depends on how many people would be grabbing the files from Amazon. Amazon does support bittorrent swarms for downloading which can reduce transfer costs.

--
Matt Koglin, Antinomia Solutions

An added side note: Somone

jlmeredith's picture

An added side note:

Somone has loaned me a Elgato Turbo264 USB devices to help this process along. It is a hardware based encoding device that speeds h.264 encoding to near real time vs the traditional 3:1 or 5:1 ratio.

Price: $99.95 from Elgato or $89.99 from Amazon

http://www.elgato.com/elgato/na/mainmenu/products/Accessories/Turbo264/p...

Not that anyone needs to purchase one, but it is a darn handy looking device that I am sure would help things along.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Your machine(s) specs -

butterfi's picture

Your machine(s) specs - processor/memory/OS:

Various flavors of Mac's -- I've done a lot of compression over the years so have a variety of machines that can be used

Your internet connection specs - up and down:

my work has an OC-3

How many hours per week you can devote to encoding:

I can devote a few hours for trimming, post, etc. My encoding machine can devote a lot more ;-)

Do you have the ability to edit DV content prior to encoding - this is primarily to trim beginning and end of video along with boosting gain if needed.

Yes

setvik's picture

Hi,

I just discovered that I took one of the session DV tapes (Drupal Association Panel discussion) I recorded home with me to Japan by mistake.

I've dumped the DV data to my macbook pro and will encode it this evening using the quicktime settings Mark posted above.

Once it's encoded, where should i send it?

(I noticed the place holders on archive.org; can i upload the video to the appropriate place holder? or is that something the placeholder's creator will need to do?)

Thanks a ton!,

Stein

That would be great! Can

jlmeredith's picture

That would be great! Can you also send Acquia the actual tape? They are planning to keep all of the tapes in an archive. Shoot me a contact message with your email so I can get with about the details.
tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

will do

setvik's picture

No prob. i'll send it on to Acquia.

How do I upload the encoded video to archive.org?

  • Stein

Just thought I would give

jlmeredith's picture

Just thought I would give everyone a heads up! The tapes and hard drives have been delivered to the conversion company and should be in the process of being converted starting today. I do not have a firm date for the first set completion, but would expect it in the next couple of days.

Once that is completed, the drive will be shipped to me here in TN where I will do some organizational work and then send the video on to OSLOSU.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Post on drupalcon.org

toursheet's picture

I just saw this thread - might be helpful to post on http://boston2008.drupalcon.org/? Many people are waiting for updates and check the drupalcon.org not g.d.o/boston

The First Drive is Complete!

jlmeredith's picture

The first drive of video will be delivered to me here in TN later this week. I will do some quick quality checking, cataloging and organization and then the drive will be on its way to OSLOSU! If all goes well, the drive should be at OSLOSU by this weekend. More info to follow.

tmg-studio (Jamie)

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

Any news on these drives?

my mum's picture

We seem to be stalling a little here - let me know if theres anything i can do

thanks

Help track down audio

Amazon's picture

Hello, we are still in need of tracking down the audio.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

any update?

Matt V.'s picture

any update?

Four rooms with Audio

Amazon's picture

Audio was being recorded in four rooms.

Mark Burdett was one of the people. Who were the other three?

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

hmm

Matt V.'s picture

I may be wrong, but I got the impression that the audio was being covered in a more hit-or-miss fashion.

Did you record any audio?

Amazon's picture

Did you record any audio?

I know there were people recording audio in each room at some point.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

only what's on the video

Matt V.'s picture

If you're asking me personally, I only recorded using a video camera, but the camera had a shotgun mic, so the audio shouldn't have been too bad. I believe there was at least one other camera with a similar mic.

Is the audio the bottleneck? I definitely don't mean to be a pest, especially since I know a lot of people volunteered a lot of time and effort to make the taping happen...but it seems like the videos become less relevant as time goes by. I'd gladly settle for whatever the camera picked up, at this point. I missed some of the panels I really wanted to see, so that there would be more complete video coverage.

Amazon's picture

One last drive has been sent to capture the data from the final tape. That drive will sent to Tennessee where Jamie will organize the raw digital files.

Jamie is sending the existing files and Drives to a fast a linux cluster running ProXchange video transcoding software that will transcode 90 GBs of raw video data to web video files. Then those drives will be sent to OSUOSL where the files will be available in a final format.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Cool, thanks for the update.

Matt V.'s picture

Cool, thanks for the update.

The video has moved on to

jlmeredith's picture

The video has moved on to the next step in the process. We have completed the conversion of the video to DV format and it is now being transcoded to more manageable sizes. For the most part, the audio is ok, though there are several instances where the boom mic was pointed with the camera at the projection screen which caused an issue with hearing the speaker. But in most cases a bit of extra volume helps to make it listenable.

I can assure you that the video has not become "less relevant" as I have spent many hours tyring to keep the process moving. There was a small delay of about 6 days as I have recently taken on a new job and had to table the video work for a few days. Overall we are not doing too bad considering the volume of video that is being addressed.

I apologize that we have not been able to move things along quicker. Hopefully the next Drupalcon we will have a better system for making sure that things get done quicker.

Jamie Meredith

--
Jamie Meredith
Technical Account Manager
Acquia, Inc.

thanks

Matt V.'s picture

Thanks for the update. And thanks for all your work on it!

Any news or date?

hansrossel's picture

Hi,

Thanks for all the work. Is there any news or date when the video's will be available? I missed some very interesting sessions...

Also:
- Maybe it is a good idea to put news about the video's on the homepage of Drupalcon Boston http://boston2008.drupalcon.org/, it's a bit hard to find this thread and I thought that all videos were lost.
- Where will the video's be published? Will it be the Drupalcon site or archive.org or both?

Hans

archive.org

mr.scales's picture

Everyone will pretty much be able to embed items from there. That is, when get the files. I'm getting alot of emails asking what is happening with the placeholders on archive.org, it is a bit embarrassing to be honest as I can't provide any timescales - can anyone tell me / make a commitment to a date when these will be available? We are looking like not completing this inside 3 MONTHS of the event, which is shocking.

Videos sent to New York to be post processed

Amazon's picture

The videos were converted to digital format and mailed to Tennessee to be organized together. They were then sent to a video processing cluster in New York, where they were supposed to be processed in a single day, that was a couple of weeks ago.

First, no Drupalcon has ever recorded all of the video and kept them together, so that's been a lot of effort.

Second, I have tried on at least six occasions to request help tracking down the audio recordings and basically the ball has been dropped every single time. If you want to be shocked, be shocked by the fact that the 4-6 people who spent four days recording audio can't remember each others names and aren't willing to step forward in a coherent manner and say, "Yes, I have X Gigabytes of Audio from Room X and I've not yet shared it".

I'll go berate Jamie for not volunteering enough.....

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

"I'll go berate Jamie for

mr.scales's picture

"I'll go berate Jamie for not volunteering enough....."

A little unfair - I haven't pointed the finger at anyone. The length of time is shocking, undeniably, and shows this needs to be rethought next time, perhaps allowing for more processing points and a drip feed of content. As far as inferring blame (if anyone needs to go that far) - I certainly don't have enough information available on what-is-being-done-when-and-by-who to make that sort of call.

I too have tried tracking down audio, with similar results to yourself (and yes, we can tag this with 'shocking' too) - however there seems to be little or no problem with the dv audio so is this still an issue? Do we still have 'audio-only' sessions that are outstanding? I have some tools to improve the sound, but will need and have requested a sample of the problem to assess.

Great news below, looking forward to helping get the content out there.

Improving process

Amazon's picture

I was expressing frustration and putting a tongue firmly in my cheek.

One of the things you learn when dealing with open source communities, is that closed is bad, and open is good. While a valiant effort was being made to get the job done, it was a little too closed, and so I've taken some steps to open up this video transcoding process.

I've got a phone call into Narayan, about 5 emails to Heidi, Jamie, Mark, Forest, Sooz, Narayan this morning and we should see the last disk with 5% of the RAW video go from Mark to Forest at MNN in NYC Monday. If I don't get a OSUOSL storage capacity by Monday I'll take some action.

The key lesson for future efforts is that we should probably avoid the logistical details of shipping physical media, and instead go with much more expensive, but more transparent shipping of media through a hosted media service.

Documenting the process in the open, and making sure everyone has the contact information of the people involved will make it easier for people who want to help reach out and do so. If you don't see an announcement by Monday that we've moved the video to a place the community can access, I'll authorize a online storage purchase out of the Drupalcon budget as a stop gap, so the community can step in and manage the completion of video processing.

Cheers,
Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Yes, lets have some transparency

mr.scales's picture

Yep - I get that you weren't actually going to shout at Jamie, but it read as though I had attacked someone - no harm done. Funny old thing, text.

If we can take a look at the process I'd like to offer guidance - it would be a good start to document what happened this time in full and identify problems. I'm happy to help and advise on this - my company just turned around 35 multimedia conference sessions in 2 days from a 3 camera shoot - so I've got some valid ideas to contribute!

There will I think need to be a group in an executive position though, as in the build up to Boston, there were many contributors, with many suggestions, not all of them workable, interoperable, best practice etc (but of course their input and effort is invaluable). One thing I'm confused about currently is that we seem to have gone from DV/video tape to DV quality encodes and then to mp4 - did any circumstances make this unavoidable?

Also, thanks for stepping in on this - you obviously have the numbers and contacts/relationships to get this going and it is much appreciated.

video encoding at Drupalcon Szeged

kvantomme's picture

I'm very much interested in any suggestions that we can take into account for the video recording/encoding at Drupalcon Szeged.

I was wondering if it wouldn't be possible to have on-site encoding, so that a session can be posted still the same day. How long does it take to encode a video? What is the 'film length/time needed to encode' ratio?

If we would have an encoding room/desk with 4 PC's could we pull the trick?

Right now we are also researching the options for having an external dedicated video team at the conference. But that might be to expensive.

--

Check out more of my writing on our blog and my Twitter account.

I'd say ...

my mum's picture

You can encode directly to just about any laptop using windows media encoder - it is free and(much as I hate to say it) a great tool for this kind of archiving. As long as your camera has a DV out to firewire and you can input your audio mix direct into the camera (not the computer, as you want to back up the signal to DV tape as well in cas the encoder falls over) then I would recommend it as the encde wil be ready as soon as you press 'stop encoding'. The DV tape is then there purely as a back-up, as it would need to be captured again by the machine and converted (either directly into WMEncoder or FInal Cut) - which could take a few hours transfer/process/conversion. You would need one encoder per camera unless you employed some kind of live vision mixer (SONY anycast etc).

The thing to remember is that AUDIO IS KING - when you distribute the content, realise that more people will want to download it and listen on an audio device, rather than be tied to watching a presenter occaisionally waved his/her hands about. If you really want the video, it would be good to include the slides too, and embed them into the video timeline - though bear in mind that anything under 11pt or detailed graphs are going to degrade massively when converted to a format you could upload to say, youtube.

If you are going to spend on this, get in an audio engineer, that means at least your bottom line is people can enjoy the recordings in a basic high qualiy format rather than a patchy effort at three elements (video/slides/audio) . You could then make the slides available seperately - or use slideshare.

One soundguy with a mixer, portable recorder, microphones for your presenters (depending how many) and 2 roving mics to be passed around by your organisers would easily cover it. An engineer to cover this would be looking at around £300 per day - though some venues have specialist contractors that they may insist on supplying. Feel free to post some more details - I'm happy to contribute.

parallel media

kvantomme's picture

that's a great point you are making. Would it be a problem do you think if we don't try to mix the 3 media: just separate video, audio and slides provided in parallel (e.g. not spend the time on mixing it into 1 video)?

--

Check out more of my writing on our blog and my Twitter account.

If you are sticking to PCs

my mum's picture

You could save some time by recording the slide timings with MSProducer (awful program, but this part is useful) fro the presenting machine, then lining up the slides and audio. If you are taking video, you will probably capture the audio with the same device to avoid syncing issues - there are loads of movie to audio converters to seperate it afterwards (usually taking 2-5mins for 1/2 hour footage) - so you have two of those formats right off the bat! As long as you are capturing the video sensibly (some good options for mac in the post below, which I'll add 2 cents to), to a lightweight format, you don't have to spend time converting it or re-rendering it if you aren't mixing it. Windows Media File editor will allow you to quickly 'top and tail' a movie with a fade in and out, without needing to re-encode the file.

You can also line up slides and audio at slideshare now.

Archive.org is a great place to host this mix of files and will handle some conversion into various formats too.

we are going to try something like this...

RockSoup's picture

at DrupalCamp Seattle.

Have a look: http://groups.drupal.org/node/11677

-jared

-jared

Amazon's picture

I just got off the phone with Forrest Mars and chatting with super volunteer Jamie Meredith (who is learning when to ask for help) and 95% of the video is converted to MP4. I have the FTP server with the videos. I'd like to try to get it to OSUOSL, and uploaded to video services after volunteers have tagged the videos.

The 95% of the videos that are done take up 18GB and are sitting on a 1 MB unmetered connection on a OC3 backbone. My preference is to get them moved first to OSUOSL, so the community has access and we can divide up this effort instead of trying to break Jamie :-(

Cheers,
Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

What tags

toursheet's picture

Thanks for the updates~! How about we use 'drupalconboston2008' for the folksonomy tag on the posts?

Last drive sent for processing, Video at OSUOSL,

Amazon's picture

Mark from SafeAndSound has shipped the last drive to Forest at MNN, it should take a couple of days to process the last 5% of the video.
Narayan is copying the 19GB of MP4 video from Jamie Meredith's server to Web 3 at OSUOSL, where it needs to be categorized and tagged before being posted to http://archive.org
I've had one person come forward indicating that they have audio for 6 sessions. I know Mark Burdett has audio for his room. I believe there's some audio already loaded up to http://archive.org

If you are interested in categorizing the video and tagging it, or willing to help track down the audio please let me know.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Amazon's picture

We've got 90 hours of video that needs to be uploaded to archive.org.

Anyone available to categorize it and upload?

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

I can help next week

Matt V.'s picture

I'll be on vacation for the holiday weekend, but if there's still stuff left to do on Wednesday, I'd be glad to help.

I can help

rcross's picture

I can help with the uploading to archive.org - what are the details? and any procedures/policies to follow?

--Ryan

Slides?

rcross's picture

Good to hear some news/progress about the video - what about any of the slides? I haven't really seen any slides from any of the presentations except Angie's and Dries'

(not pointing fingers, just asking)

--Ryan

Still looking for media files...

rcross's picture

Any updates on the media files for DrupalCon Boston? Its great to see some discussion about things for Szeged, but I'm still waiting to see Boston's content.

It sounds like the video is done/ready - but it doesn't seem to be uploaded to archive.org where all the placeholders are. Is there another location where the files are available for download?

What about slides? This is usually useful even with video available. I think I noticed some random few on drupalcon.org, but nothing comprehensive or easy to find.

Any audio recordings? Mentioned above, this is also useful.

Thanks guys!
--Ryan

Could help uploading

hansrossel's picture

Hi,

If the video is done and it just has to be uploaded to archive.org I can help.

I have about 15 GB (per month) available to upload (at 1 Mbps), just send me the files and I will start immediately.

Hans

archive.org placeholders, description & tags

hansrossel's picture

Hi,

I replaced the placeholder message of all uploaded/ready Drupalcon Boston videos on archive.org with the description of the session, so it becomes clear which videos are available yet. I also added tags so all uploaded videos have tags drupalconboston2008, drupalcon and drupal, so the tag drupalconboston2008 should give you the yet uploaded videos: http://www.archive.org/search.php?query=drupalconboston2008%20AND%20subj...

I'm currently busy uploading the remaining files; placeholder message still has to be removed of the latest uploads + tags added.

update uploading continued

hansrossel's picture

OK, I got yesterday night almost halfway uploading the remaining videos, will continue today and hope to have finished uploading them by the end of the day. I'm slowed down a bit because archive.org is quite often offline.
Placeholders still have to be replaced by the session description and tags have to be added. If you're impatient you can follow the proceedings of the upload at http://groups.drupal.org/node/11950

Hans

Upload ready

hansrossel's picture

OK, I've finished uploading videos, tagging, removing the placeholder text and adding the descriptions of the sessions.

All ready videos that I could find are available for download on archive.org now; Let me know if anybody of you can map or find the missing files for the missing sessions, then I will upload them too once they are ready, see http://groups.drupal.org/node/11950

Thanks to all of you that helped making the videos! I'm allready relaxed sitting down watching the sessions I missed right now!

Hans
www.koba.be

2 more to be done

Amazon's picture

This video seems to be fine when I put this URL in Quicktime: http://david's server/16-45M-wide.mp4
Design on the Edge of Drupal
http://www.archive.org/details/DrupalconBoston2008-DesignOnTheEdgeOfDrupal

This one too:
http://david's server/70-60M-wide-pt1.mp4
http://www.archive.org/details/DrupalconBoston2008-JqueryDrupalAMatchMad...
http://boston2008.drupalcon.org/session/jquery-and-drupal-match-made-heaven

If anyone gets a chance to upload those two to Archive.org that would be great.

There's still some detective work to be done to figure which videos might be missing and why: bad recording, mislabeled, tape went home in camera, video's still need to be transferred from post processing. If detective work sounds interesting, please let me know. It will be really great to have as thorough as possible a set of videos.

Cheers,
Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

uploaded the last 3 items

hansrossel's picture

I have uploaded the last 2 items.
Because of the permission problem I am experiencing only for the two videos here above on archive.org I cannot tag those or add the description (and remove the placeholder info), but if nobody checks them in and adds those details, they will be made available by archive.org in 48 hours.

Thank you to all of you!

dccircuit's picture

A big thank you to all of you who have been involved in this!... (Didn't realize until Hans most recent message on boston2008.drupalcon.org forum what was going on behind the scenes with these videos!).

I'm going to be looking through these pretty carefully, myself, and will try to reply with any that don't seem to be there.

Meanwhile, one in particular that I was looking for and didn't find was: http://boston2008.drupalcon.org/session/site-recipe-tracking-event-resul...

Any news on that one?
Thanks very much.

Three more videos to be released

Amazon's picture

Hi, there's three move videos to be released. We are merging the 6 halves and will be 3 more videos that need to be watched, categorized and uploaded to Archive.org containers.

Kieran

Drupal community adventure guide, Acquia Inc.
Drupal events, Drupal.org redesign

Drupal Event Organization

Group organizers

Group categories

Event type

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: