Bundler - the Photosynth core algorithms GPLed

[update- the output of bundler is less misaligned looking than this, I was incorrectly displaying the results here and in the video]

Bundler (http://phototour.cs.washington.edu/bundler) takes photographs and can create 3D point clouds and camera positions derived from them similar to what Photosynth does- this is called structure from motion. It's hard to believe this has been out as long as the publically available Photosynth but I haven't heard about it- it seems to be in stealth mode.


Bundler - GPLed Photosynth - Car from binarymillenium on Vimeo.

From that video it is apparent that highly textured flat surfaces do best. The car is reflective and dull grey and so generates few correspondences, but the hubcaps, license plate, parking strip lines, and grass and trees work well. I wonder if this could be combined with a space carving technique to get a better car out of it.

It's a lot rougher around the edges lacking the Microsoft Live Labs contribution, a few sets I've tried have crashed with messages like "RunBundler.sh: line 60: 2404 Segmentation fault (core dumped) $MATCHKEYS list_keys.txt matches.init.txt" or sometimes individual images throw it with "This application has requested the Runtime to terminate it..." but it appears to plow through (until it reaches that former error).

Images without good EXIF data trip it up, the other day I was trying to search flickr and find only images that have EXIF data and allow full view, but am not successful so far. Some strings supposed limit search results by focal length, which seems like would limit results only to EXIF, but that wasn't the case.

Bundler outputs ply files, which can be read in Meshlab with the modification that these two lines be added to ply header:

element face 0
property list uchar int vertex_index

Without this Meshlab will give an error about there being no faces, and give up.

Also I have some Processing software that is a little less user friendly but doesn't require the editing:

http://code.google.com/p/binarymillenium/source/browse/trunk/processing/bundler/

Bundler can't handle filenames with spaces right now, I think I can fix this myself without too much work, it's mostly a matter of making sure names are passed everywhere with quotes around them.

Multi-megapixel files load up sift significantly until it crashes after taking a couple of gigabytes of memory (and probably not able to get more from windows):

...
[Found in EXIF tags]
[CCD width = 5.720mm]
[Resolution = 3072 x 2304]
[Focal length (pixels) = 3114.965
[Found 18 good images]
[- Extracting keypoints -]

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.


Resizing them to 1600x1200 worked without crashing and took only a few hundred megabytes of memory per image, so more megapixels may work as well.

The most intriguing feature is the incremental option, I haven't tested it yet but it promises to be able to take new images and incorporate them into existing bundles. Unfortunately each new image has a matching time proportional to the number of previous images- maybe it would be possible to incrementally remove images also, or remove found points that are in regions that already have high point densities?

Makeavi

Discovered a neat windows (and vista) tool for turning image sequences into videos: http://makeavi.sourceforge.net/


1280x720 in the 'Microsoft Video 1' format worked well, though 57 MB of pngs turned into 135 MB of video. 'Uncompressed' didn't produce a video just a small 23kb file. 'Intel IYUV' sort of produced a video but not correctly. 'Cinepak' only output a single frame. 'VP60 Simple profile' and 'VP61 Advanced Profile' with the default settings worked, and actually produces video smaller than the source images, though quicktime player didn't like those files. Vimeo seems to think VP61 is okay:


More Velodyne Lidar - overhead view from binarymillenium on Vimeo.

This new video is similar to the animated gifs I was producing earlier, but using a new set of data. Vimeo seems to be acting up this morning, I got 75% through an upload of the entire file (the above is just a subset) and it locked up. I may try to produce a shorter test video to see if it works.

I have around 10 gigs of lidar data from Velodyne, and of course no way to host it.

My process for taking pcap files and exporting the raw data has run into a hitch- wireshark crashes when trying to 'follow udp stream' for pcap files larger than a couple hundred megabytes. Maybe there is another tool that can do the conversion to raw?

Next Year

http://flickr.com/photos/released/2159331315/

I and a few others from Hackerbot put a bunch of work into getting the visualization running, and it worked mostly and was decent, but I didn't put much work into providing variation on the effects, I just let them go as is for hours. But the live video input was broken, mplayer on a pvrusb2 would work for a few minutes and then lock up so hard a kill -9 wouldn't even kill it. And the audio would work after it was futzed with but then stop upon trying it into a new graph.











http://www.flickr.com/photos/foist/2158862880/
http://www.flickr.com/photos/foist/2158071625/

Harrier from binarymillenium on Vimeo.

Gephex graph examples

I used to just keep my graphs to myself and only show videos of results occasionally, but from now on I'll be backing up all the good ones online suitable for distribution under the GPL.

Here's a couple:

http://code.google.com/p/binarymillenium/wiki/GephexGraphs



This one is cool:


galaxy from binarymillenium on Vimeo.

This is okay:


BW Plasm demo from binarymillenium on Vimeo.

Opticlash 2

This was an interesting event, I didn't go to the first one in 2005 and I had my doubts about the format going in- and I still have my doubts, but overall it was a success in terms of promoting VJing- and also VJ Scobot won, and I think he did do the best VJing there.

There were three sets of screens at the front of the room, the center had camera views of the two competing VJs and the outer ones had the video they were outputting. This seemed less than ideal because of difficulty in viewing both simultaneously- you could either watch one or the other except from the most distant points or oblique angles.

It's possible the VJs at some points were glimpsing what the other guy was doing and responded in some way, but I think mainly they were just concentrating on their own stuff- which is unfortunate, because the most crowd-pleasing aspect of a competition is any kind of interaction and drama that can be generated between the contestants. It would be great to have one contestant go for a minute or two and the the other goes, trying to outdo the other, maybe playing a similar sort of clip, or mocking them somehow, or anything like that- and they would go back and forth a few times.


Opticlash 2 from binarymillenium on Vimeo

I didn't actually see the last set, since the show was going on a bit longer than advertised- the 15 minute sets for the later rounds should have been at most 10 minutes on schedule or not.

The judges didn't add a lot besides the rote rating judgements they offered, the MC initially wanted some kind of vocal rationale or something out of the judges but they were microphone shy.

Here's a writeup from one of the judges, and he posted some video (Pixelflip vs. ?) on dailymotion.

VJing Toorcon

Post-mortem thingy:



My setup was to use gephex on a Linux laptop and have a video camera and dvd player as video sources. Switching between them would have been nicer with a switch box, I was just reconnecting cables while having a graph loaded on gephex that wasn't dependent on external video input.

I had a couple of newly burned DVDs, one was lots of video game imagery. Biohazard Battle on the Sega Genesis worked decently, but old Atari games with flatter coloring and graphics could have been more interesting. Unfortunately my Atari 2600 didn't work after I pulled it out of storage. One of those $20 battery powered video game joysticks would work well for sourcing the imagery live, but I'd need someone else to play (I recall somebody brought one to Open Lab a few months ago but didn't try it out).

The other dvd had videos of a computer animation contest, one where the executable creating the animation had to be less than 32K. I had to run the dvd player output through a anything-to-anything box from Canopus with firewire in/out on it.

Four projectors were going with cloned video, and I had an LCD monitor to see what I was doing. None of the projection surfaces in the main room were that ideal but they worked. The better surfaces were out in the hall but I couldn't see them.

Gephex (or underlying Linux graphics software) is finicky about driving full-screen, sometimes there would be a sliver of the desktop underneath it. Playing around with it usually got rid of it. I spent a long time going through all my graphs and deleting ones that didn't work and adjusting output settings for ones to keep. It would be nice if each individual graph didn't have it's own output settings.

Kino plus my screengrab frei0r module was my way of getting 1394 video into gephex. I could probably write my own 1394 gephex/frei0r input module using source from Kino but I haven't gotten around to it yet.

Ideally there would be a way to get video into linux so that it is seen as a webcam- a video-to-usb device sounds ideal- I could get video from a video camera or a dvd player without using firewire at all. Supposedly there is one from X10 with linux drivers, and another from Hauppage, but neither is sold on Amazon or other sites that feel respectable enough to purchase from.






Toorcon 2007 from binarymillenium on Vimeo

Another option would be to run Gephex on the windows side of my laptop, but unfortunately it came with Vista preinstalled, and gephex doesn't run so well there.

I would have liked to be able to record the whole show or portions of it. Another RGB-to-video plus another video camera with video input recording (like my old Canon DV camera) would have worked, but added a lot of wires. Also I didn't have a spare VGA splitter output to drive it.

911 Media Open Lab - April 15

911 Media Arts Center Open Lab - 2007.04.15 on Vimeo

I bought a new laptop somewhat recently, and am currently dual-booting between Vista & Ubuntu. Most of my custom software was only set up to run in Ubuntu, but I forgot to figure out getting the S-Video output to work when I brought it to Open Lab- despite all the other ease-of-use advances of Ubuntu something as simple as configuring an external output is still a pain. So instead I went back into Vista and just messed around with Wings3D, while others manipulated that source video with some hardware video mixers.

The slower bits are edited out, and overall I like the most of the scenes in there- even if they don't match up to the (also live generated) music, I think there's a few underlying ideas that could be developed into more interesting clips:
-Flying over alien landscapes, manipulating them, and a kind of 80s CG flat shading look
-Simple shapes that generate fractals

Wings isn't really meant for live performance. With a little more work I could set up a lot more keyboard shortcuts so the context menus don't show up as much. A more intensive effort would be to make models in Wings, export them to objs or something and a have a custom app running on the external monitor that can be triggered to load the model.

Bones Animation - Re-Acting

Bones - Re-Acting from binarymillenium on Vimeo.

I think I could have done a better job with this video, edited it a little more heavy, but I don't like to get to bogged down with it. I sort of think as these as visual notes to myself, I can refer back to them and recreate the effect I captured in a bigger and more meaningful work.

The main thing making editing more difficult was that I was using image sequences in Premiere- my computer isn't fastest enough to actual play back unrendered image sequences (and I was too lazy to render it), so it was hard to get the edits and feel right.

The source imagery is from a code.google project called 'bones' http://code.google.com/p/binarymillenium/wiki/Bones. It's a very simplistic bones animation implementation, using osg::nodes and with randomly generated hierarchy and animation. Every vertex in the object loaded for a bone has a weight that mixes (using quaternion slerp) the positions of the parent osg::node and the child. The weights are automatically generated based on the distance from the vertex from the root of the object where it joins with the parent.

Render-to-texture feedback with OSG

This actually doesn't use any of the screen capture code from the last post ( I'm still working out how to best use that) - it just does a good old fashioned glReadPixels to current the current opengl screen.


OSG Feedback from binarymillenium on Vimeo.

Gephex

I created a couple of custom gephex modules (for 0.4.3):

Average

Find the average color or brightness of a framebuffer. (maybe add HSV next)

Slow motion

Play back snippets of framebuffer input in slow motion. The music is from ccmixter, William Berry's Time To Take Out The Trash.

The source code (GPLed of course) and windows dlls are provided in those zip files.
More custom effects are on the way.



New video showing off basic gephex effects:




...but none of those I just created. Maybe next time.

Phoeeb Deform

Brand new video, made almost from scratch (a stock photograph from deviantart was used as a referenced and applied as a texture).





Phoeeb Deform from binarymillenium on Vimeo.




The music is from ccMixter (http://ccmixter.org/media/files/vincent_vega11/6966), which has supplied me with several good soundtracks for recent videos, although it can be difficult to browse the music- where's the sort by rating button? I recall another CC music site less focused on remixing and more on original works that had more movie like soundtracks I'd like to use in the future but I forget the site.

Gravity

Gravity from binarymillenium on Vimeo.