Depth buffer to 3d coordinates?

I'm having trouble transforming screen coordinates back to 3d, which this post describes- can anyone help me?





---
Update - I've got it figured out now, I should have been using gluUnProject:



FloatBuffer fb;

fb = BufferUtil.newFloatBuffer(width*height);

gl.glReadPixels(0, 0, width, height, GL.GL_DEPTH_COMPONENT, GL.GL_FLOAT, fb);
fb.rewind();

int viewport[] = new int[4];
double[] proj=new double[16];
double[] model=new double[16];
gl.glGetIntegerv(GL.GL_VIEWPORT, viewport, 0);
gl.glGetDoublev(GL.GL_PROJECTION_MATRIX,proj,0);
gl.glGetDoublev(GL.GL_MODELVIEW_MATRIX,model,0);

...
for(int i...
for (int j...
...
glu.gluUnProject(i,height-j,rawd, model,0,proj,0,viewport,0,pos,0);
float d = (float)-pos[2];


After all that depth d will be linear and in proper world coordinates.

Artoolkit + rangefinder

Since my relatively inexpensive purely visual depth map approach wasn't that successful, I've tried it out using a rangefinder instead of a visible laser. This means I can point the video camera straight at the marker (which is attached to the rangefinder), and it can point at anything provided I don't tilt it so the camera can't see the marker/fiducial.

This is the result:


Artoolkit with a Rangefinder from binarymillenium on Vimeo.

The following plots show the tracked attitude of the rangefinder as measured by ARToolkit:






My left to right bottom to top scanning approach is very apparent.


And here is the tracked attitude (as a 3-component vector) plus the range vs. time:



You can see how cyclical it is, as I scan the floor in front of me the range doesn't change much until I reach one end and tilt the tripod up a little, and then later on I start to capture the two wheels of the car.

University of Washington BioRobotics Lab

I took a tour of the UW BioRobotics Lab, where an old professor of mine works on telerobotics with haptic interfaces.



This is a surgery robot called 'The Raven'. It's mostly camouflaged due to the large amounts of detail and contrast in the robot itself and in the background. The DV camera is going to be replaced by a pair of HD cameras that will provide stereo vision.



Multiple motors pull on cables seen in a later photo that control the manipulator end of the arm.










Blake Hannaford shows the arms that will replace the manually positioned arms seen in the previous photos.