How close do you have to get the lenses for this to work? ie Do you have to closely match a 50mm video lens to the digital SLR equivalent?

well slap my mamma....

i'm stunned, when can i buy it?
;-)

simple, original and useful. so this is an invent on cinema. this idea and project is brillant. bravo.

This is insane. Genius work.

Freakin' Amazing!

this work... is just...amazing. You guys are doing things that i think that was just not possible...

When the software will be released? :B

Saw this a week or two ago on 1000dollarfilm. RIDICULOUSLY exciting for the future of low-budget videography!

I wanna be a beta-tester!! :)

Unfortunately, all the software written for this project is research quality code. There is no UI, no error checking, and it's slow. I need to put in the engineering effort to create a user-friendly system around the research software and release it as an open-source beta. But that is going to be hard to do with the new research projects I'm working on.

However, the details on how to implement the software/algorithms have been published in an EGSR paper which is available from the project website. Ideally, a product team like Adobe would carry the torch from here on and create a professional quality implementation of the ideas presented in this work.

Excellent work! I'm dazzled.

would you give the blender foundation the permission to use your algoriths ?

we as a community can make things happen.

Very impressive.

This is so mind-bending it's creepy!
Does it take long to implement the effects in it's current state?

That is an awesome use of technology.

This is unbelievable! I'd really use one in my projects. The amounts of time it would save.... even if it is slow.

Hey Pravin! great to see your work highlighted here. I can't wait to see it implemented in some sort of product as well... Best, Jay

Is this a joke... ? Sounds unbelievable...

This is truly amazing. This gives extreme possibilities. What plans do you have for this project? Feature in After Effects? Will it be a plug-in? A stand-alone suite?

How is it with color-matching/color-space? For example, can different SLR's be used, say download hi-res photos online for enhancing own video? What about shadows. Can the system extrapolate details if the lighting differs between pictures?

What about complex geometry, say the Eiffel tower?

This is amazing stuff. Wow, just wow. When is this going to be released to the general public? :p

Very nice! I am a pro photographer -- still photography, color digital and B&W medium format -- and would like to produce a "video" to share here. What software did you use, or is it a proprietary application? Thank you!

*ties his point and shoot to his camcorder and sets them both to 'auto'*

«running man» style «digital matte tracking» as in tinyurl.com/at2m15 has finally arrived. OK it's still static though but it's damn close.

So, where do I buy it?! :)

Genius stuff... It'd be nice if the Blender Foundation does pick it up.

Wow, this is magic. Will you marry my daughter and bring your video spells over to my house for holidays?

Vraiment impressionnant
Impatient de voir la suite de ces recherches et leur application dans un software .

please allow me to give you money for this product.
please...

Yeah, enhance the Zapruder film and then restore the newly discovered footage of Fritz Lang's Meropolis.

HDR for video has amazing potential!

That was amazing how the sign was able to be removed.

Adobe will love this stuff, at least for 3 more versions of After Effects. You really need to develop a UI.

Congratulations !

The best use of these techniques I've ever seen.

100% incredible. I give much money for super math.

Wow.

Have fun working at Apple when they call you to add this to Final Cut!

Brilliant. Can't wait until this research finds its way into accessible editing suites.

It's going to be a godsend for budget film makers, and further blur the lines between the pros and the amateurs. It'll put film-resolution quality into the hands of people with editing and directing skills, but who lack access to thousands of dollars in equipment.

Particularly if it's upgraded to support dynamic movement in a scene.

Sincere thanks for developing this, and a sincere plea to not completely move on from the project before algorithms that support dynamic movement are created (though I appreciate your time constraints).

Maths FTW.

-Torrey

that is freakin' amazing. I love it.

Holy ****, this is amazing.

I'd totally buy this software. ;-)

wow, this is awesome stuff, but I'm sure it requires a significant amount of computational power...

Pleeeeeaaasssseee release it as open source code so someone can cook up an UI and we can all use it this is MASSIVE for low budget producing! F*ck Adobe Open Source is the way to go(d)!

You guys are incredible. The future looks fantastic

Pro, do you have any interest in pursuing a contact at Apple about this concept? Considering they have a lot more capital to burn than Avid and have a vested interest in video I would think they'd find your demo very intriguing.

Wow... very impressive even at 5min/frame. Manually making these changes would take much longer .

Holy hot cakes, Batman! That's incredible!

Boo apple. Yes Zapruder Film.

Yes I hope to see this in (better) working state someday (soon), I have already thought of a few tests to try.

I just got back from the SIGGRAPH show where I saw some of the best industry software for post production effects work, and this video still blows me away. I can't wait till this is in some sort of product. looks to be the basis of a great tool.

Your video stabilization seems to add a new aspect to post-stabilization of camera movement... if I looked at it correctly... it stablized in the z-axis (not rotation only) and takes out movements toward and away from the subject matter. That's pretty impressive...

This blows my dome. I really hope that someone brings this to market soon.

This is awesome.

WOW! I would pay you to start developing this for Blender... If only I hadn't bust all my money on a new computer...

Wow, really amazing!

Yeah, that was great! Truly revolutionary software!

This is amazing stuff! Anybody reminded of Blade Runner?

That this is not available immediately on all platforms is exceedingly CRUEL.
Thanks for the ToRmENt!!!!!!

Geektastic!!

The illusion of reality brought by video eventually will come to an end: the perception of our world thru the media will change and the viewer will have to be much more aware of other forms of information in order to validate the veracity of anything shown on tv.

If this is the way of the future post processing and computing power will generate the need for huge investments in unusual places ( you will pay more for your renderfarm that for your actors or shooting equipment) You may even rent the renderfarm in a far away land or from a zombie network built out of trojan infected machines.

The quality of the documentaries and reality shows is about to become even with big budget series like "lost". How weird!.

WOW! This is sweet! Features you could add, using web images, ie. use image reconition and get pictures from google image search (think microsoft made a concept of using other pictures based on geotagging to make a super high res image of the Notre Dame). Would be something to concider. Requires that a lot of pictures use geotagging.

Moving objects would be solid gold. Today there are some software creating 3d models from video. You draw the geometry on the object in your video (here's a link: acvt.com.au/research/videotrace/ ). If you could use the depth algorithm to accomplish something like it you would be very very close to enhance moving objects as well!

If you could extract the depth data and project the video on it. Then it would be fantastic if you could import it to programs like combustion to make composites with video and 3d objects that you could manipulate in 3D space. Imagine filming a room and taking photographs, then run your algorithms to create a super detailed 3D room.

Add feats like these and you guys will be the saviors of filmmakers budgets and the liberator of their imaginative minds! :P AWSOME!

WOW!
This is just amazing! thought the next steps for CG field of software developments are appearing! ;)
Btw, I'm curious what the hell you're workin on right now! :D

mim,

Brilliant stuff, but I notice all your videos were deep focus, I wonder how does it fare in shallow depth of field situations, would it be able to estimate the depth as efficiently as it did here?

Impressive!
Really looking forward for the final results of this research!

This is just brilliant! Hope someone will get a UI made for it.

Hi Pro !
Nice work !

Quick question. In the video you are presenting, your footage got a large amount of parallax. That make easier the depth computation. what happen if you got try your algorithm on footage that contain less parallax ?

Do you use your still images to introduce parallax ?
Can you apply this with non static scene ?

Usually we are doing the same kind of work by hand ( 3d space reconstruction and image based modelling and texturing ) Having a software doing that automatically will be a time saver !

I can think of so many uses for this right now. That is unbelievable!

WTF!!???

where can i buy one

Amazing!!

And so ends the current "reliability" (not the there is much as most of this can already be accomplished albeit slowly) of video in news and media.

*mind implodes* bravo guys, this is amazing

This still seems like a "time" costly task to perform, when doing things like green screening, using a higher resolution camera, better optics, and lighting, etc. would provide equal or better results as you scale up, with minimal time effort. I love the other video where the "space time" curve is drawn on... amazing stuff really!
Kudos for the alex grey vimetar too!

really neat work -- HDR will have a lot of applications, and the photo effects applied to a scene could be an easy way to get nice effects with time in post rendering vs. lower cost in initial production. Great work!

5min a frame would make this a cluster-only application - still brilliant though!

What'd be interesting is how much of this processing can be done on the GPU(s) - If Adobe for instance gets this into After Effects - perhaps with some more optimizations and GPU utilization (1 TFlop per card is feasable) - things could be sped up.

Pro, do you think this is possible?

That's amazing! I don't really work with video... it's just hobby for me... and I simply love these effects!

I'd never have thought something this powerful could be post-processed like this. We're entering a new world of FX with this stuff - before too long it'll be realtime!

i want a beta on osx

looks amazing. It seems like the stuff you're working on has a lot in common with what the Mokey/Monet guys are doing. You should give them a holler, maybe get a job there. imagineersystems.com

That stabilized shot looked GREAT!

Kudos to the team, I just hope we see this technology in professional programs soon.

The HDR video was a bit disappointing. i think I saw more window detail in the bracketed images than in the output video. But this is pioneering stuff here!

Simply speechless!

awesome !

I saw this video in a computer graphics/computer vision class last semester, and it still amazes me. The "space/time fusion" algorithm seems to be the key here -- any other paper would have stopped just before that and presented the proof of concept.

If you run this on non-static scenes, what happens? Does the depth approximation/structure from motion just give up? If it still chugs through I'd be curious to see the artifacts.

at first i didnt notice it, but then looking at the video for a second time i realized how great this will be. the meir fact that i can take digital stills with differnt exposure time, then feed them in will allow the guys with camcorders the ability to go up against the high contrast level of film(almost).

great work.

Oh... my... amazing. Wow.

Happened upon this by accident and had to watch it a couple of times. It truly does seem groundbreaking from the ironic opposing potentials of super realism and complete fiction. Like most everyone here I would love to have this to play with. I suspect most of us won't. However Pro has stated that he doesn't have time to move forward on it presently and is willing to release under GPL. Is that correct? In fact it looks that the code is available as indicated minus Noah's portion. Get moving!

Coincident with the "invisibility shield" light bending tech that is emerging this may well signal the end of our eyes being our most trusted sensor as well as video being a truly useful reporting document without far better vetting. Already video can be manipulated frame by frame with relatively high realism; it is, after all a series of still images. On the other hand most humans are willing to suspend disbelief for any trusted source and swallow the narrative anyway. Net change is probably a continuing upward curve of cynicism.

If this code isn't snapped up immediately in a bidding war then my name's not Romario ...or other interested parties have something written that is very similar. Sure would love to see this remain in the open source domain though.

Hell, this has been posted for a week already. Any updates from Pro on the interest generated back-channel or is it all NDA-ed now?

Very cool stuff. Thanks for sharing it here.

bad. ass.

object removal looked really easy.

impressive work!

That is a very promising system! Any ideas to release a public beta? :D

this is great and thanks for making your research and codes publicly available!