http://getsatisfaction.com/livelabs/topics/pointcloud_exporter
What to do
Get Wireshark http://www.wireshark.org/
Allow it to install the special software to intercept packets.
Start Wireshark. Put
http.request
into the filter field.
Quit any unnecessary network activity like playing youtube videos- this will dump in a lot of data to wireshark that will making finding the bin files harder.
*** Update ***
Some users have found the bin files stored locally in %temp%/photosynther, which makes finding them much easier than using Wireshark, but the for me on Vista the directory exists for one user but not others- but the bin files have to be stored locally somewhere right?
***
Open the photosynth site in a browser. Find a synth with a good point cloud, it will probably be one with several hundred photos and a synthiness of > 70%. There are some synths that are 100% synthy but have point clouds that are flat billboards rather than cool 3D features- you don't want those. Press p or hold ctrl to see the underlying point cloud.
*** Update ***
Use the direct 3d viewer option to view the synth, otherwise you won't get the synth files. (thanks losap)
***
Start a capture in Wireshark - the upper left butter and then click the proper interface (experiment if necessary).
Hit reload on the browser window showing the synth. Wireshark should then start show ing what files are being sent to your computer. Stop the capture once the browser has finished reloading. There may be a couple screen fulls but near the bottom should be a few listings of bin files.
Select one of the lines that shows a bin file request, and right-click and hit Copy | Summary (text). Then in a new browser window paste that into the address field. Delete the parts before and after /d8/348345348.../points_0_0.bin. Look back in Wireshark to discover what http address to use prior the that- it should be http://mslabs-nnn.vo.llnwd.net, but where nnn is any three digit number. TBD- is there a way to cut and paste the fully formed url less manually?
If done correctly hit return and make the browser load the file- a dialog will pop up, save it to disk. If there were many points bin files increment the 0 in the file name and get them all. If you have cygwin a bash script works well:
for i in `seq 0 23`
do
wget http://someurl/points_0_$i.bin
done
Python
Install python. If you have cygwin installed the cygwin python with setup.exe, otherwise http://www.python.org/download/ and download the windows installer version.
*** UPDATE *** It appears the 2.5.2 windows python doesn't work correctly, which I'll look into- the best solution is to use Linux or Cygwin with the python that can be installed with Linux ***
Currently the script http://binarymillenium.googlecode.com/svn/trunk/processing/psynth/bin_to_csv.py works like this from the command line:
python bin_to_csv.py somefile.bin > output.csv
But I think the '>' will only work with cygwin and not the windows command prompt. I'll update the script to optionally take a second argument that is the output file.
If there are multiple points bin files it's easy to do another bash loop to process them all in a row, otherwise manually do the command above and create n different csvs for n bin files, and then cut and paste the contents of each into one complete csv file.
The output will be file with a long listing of numbers, each one looks like this:
-4.17390823, -1.38746762, 0.832364499, 24, 21, 16
-4.07660007, -1.83771312, 1.971277475, 17, 14, 9
-4.13320493, -2.56310105, 2.301105737, 10, 6, 0
-2.97198987, -1.44950056, 0.194522276, 15, 12, 8
-2.96658635, -1.45545017, 0.181564241, 15, 13, 10
-4.20609378, -2.08472299, 1.701148629, 25, 22, 18
The first three numbers are the xyz coordinates of a point, and the last three is the red, green, and blue components of the color. In order to get a convention 0-255 number for each color channel red and blue would have to be multiplied by 8, and green by 4. The python script could be easily changed to do that, or even convert the color channels to 0.0-1.0 floating point numbers.
Point Clouds - What Next?
The processing files here can use the point clouds:
http://binarymillenium.googlecode.com/svn/trunk/processing/psynth/
Also programs like Meshlab can use them with some modification- I haven't experimented with it much but I'll look into that and make a post about it.
104 comments:
hi!
I've updated the bin_to_csv file to loop through a number of files, so if you have 50 bin files you can do this:
python bin_to_csv.py points_0_ 50 > test.csv
And it will loop from 0 to 50. It also still works the old way.
Correct me if I am wrong, but does each bin file only represent a single point? If so, i would expect there to be thousands of .bin files extracted from photosynth. My first few tries I only got about 11.
No, each bin file contains hundreds of points, 11 sounds like reasonable number of bin files for a decent synth.
I had the same problem. The project I am trying to pull points from clearly has 10s of thousands of points but I only get one per bin file.
Binary, Are you also getting camera positions?
has anyone figured out how to get multiple points in a single output file yet?
Greg, you can email me at the name of this blog at gmail.com, and send a bin or a link to the photosynth.
I have kind of a bad hack where I throw out all point positions that either seem too small or too big, that might be messing up in your case.
The image positions might be in the json file, I might get around to investigating that (though I'm much more interested in just the point clouds)- anyone else is welcome to look into it also. Also there is a lot of data in some of the bin files that aren't point positions.
Replace fraw = open(filename) with fraw = open(filename,'rb') to make it work.
is this to make it work in windows or a separate problem?
Yeah that should fix the windows problem.
a few tips:
1)in wireshark after you capture you can use the menu - File,Export,Objects,HTTP to export all the .bin files in one go without reconstructing the urls. you get some other random stuff as well.
2) when you create a photosynth a .bin file gets sent to microsoft you can capture this with wireshark or capture the file from the %temp%/photosynther directory. this file is a zipped copy of all the .bin files.
3)there is another file 0.json its a bit cryptic but the camera position and orientation info is in there.
im not up on JSON but using the code from here http://www.codeproject.com/KB/recipes/JSON.aspx
and some basic c# (below) i could get out the camera data.
JsonObject jsonOb;
JsonObject Joimage;
JsonObject Jor;
JsonObject Jogroup;
string json = File.ReadAllText(@"0.json");
using (JsonParser parser = new JsonParser(new StringReader(json), true))
jsonOb = parser.ParseObject();
JsonObject Jo1 = (JsonObject)jsonOb["l"];
JsonObject Jo2 = (JsonObject)Jo1[""];
JsonObject Jox = (JsonObject)Jo2["x"];
string CSVoutput = "group,image,p0,p1,p2,j0,j1,j2,j3,j4,j5,j6,j7,j8\n";
foreach (string groupKey in Jox.Keys)
{
Jogroup = (JsonObject)Jox[groupKey];
Jor = (JsonObject)Jogroup["r"];
foreach (string ObKey in Jor.Keys)
{
CSVoutput += groupKey + "," + ObKey;
Joimage = (JsonObject)Jor[ObKey];
foreach (JsonNumber coord in (JsonArray)Joimage["p"])
CSVoutput += "," + coord.Value.ToString();
foreach (JsonNumber coord in (JsonArray)Joimage["j"])
CSVoutput += "," + coord.Value.ToString();
CSVoutput += "\n";
}
}
File.WriteAllText(@"json.csv", CSVoutput);
if you want to capture the bin/json files when creating the synth without using wireshark make a c# console app out of this code and leave it running while the synth is processing - it seemed to work ok for me - hope someone finds it useful.
static void Main(string[] args)
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = Path.Combine(Path.GetTempPath(), "Photosynther");
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "*.bin";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
Console.WriteLine("watching photosynth temp files");
Console.WriteLine(watcher.Path);
Console.WriteLine("Press \'q\' to quit.");
while (Console.Read() != 'q') ;
}
private static void OnChanged(object source, FileSystemEventArgs e)
{
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
File.Copy(e.FullPath, e.FullPath + "_copy.zip", true);
}
Thanks for the tips and the code art79!
Just managed to load it in Meshlab, without color for now but I'm working on it...
Before we can start the python script needs a little adjustment, the commas (",") and tabs ("\t" i think) need to be replaced by simple spaces (" ").
After running the script, we need to determine the number of lines in the csv file. In my example, its 100000 lines = 100000 points.
Now create an empty .ply with the following header:
ply
format ascii 1.0
element vertex 100000
property float x
property float y
property float z
property uchar red
property uchar green
property uchar blue
element face 0
property list uchar int vertex_index
end_header
Append the content from your csv and add the following line:
0 0 0 0
Without this MeshLab will output an error.
I'm not really sure about the colors yet, but wanted to give feedback for everyone interested.
Great work so far binarymillenium, I've had the idea to use photosynth to create 3d models since the fist presentation I saw, thanks alot for your findings!
got it working in meshlab!
in the py script, add:
red = red * 255 / 31
green = green * 255 / 63
blue = blue * 255 / 31
after the byteswap code to correct the colors, and the ply header finally has to look like this:
ply
format ascii 1.0
comment made by Knele
element vertex 55000
property float x
property float y
property float z
property uchar red
property uchar green
property uchar blue
element face 0
property list uchar int vertex_index
end_header
then the clean csv data (without commata & tabs) and
0 0 0 0
where 55000 = n points. add a .ply extension and tada - a Polygon File Format 1.0 compatible point cloud ;-)
Carl Lipo of Cal State is using the binarymillenium code to reconstruct the terrain of easter island for a science application http://blogs.msdn.com/photosynth/archive/2008/11/24/creative-uses-of-photosynth-in-science.aspx
I apologize in advance for the noob programming question:
I tried making a visual studio C# console app in ms visual studio using art79's code. I used the following code but I get an error, "The namespace name 'FileSystemEventArgs' could not be found (are you missing a using directive or an assembly reference?)"
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace rip_psynth
{
class Program
{
static void Main(string[] args)
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = Path.Combine(Path.GetTempPath(), "Photosynther");
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "*.bin";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
Console.WriteLine("watching photosynth temp files");
Console.WriteLine(watcher.Path);
Console.WriteLine("Press \'q\' to quit.");
while (Console.Read() != 'q') ;
}
private static void OnChanged(object source, FileSystemEventArgs e)
{
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
File.Copy(e.FullPath, e.FullPath + "_copy.zip", true);
}
}
}
try adding this at the top of your code.
using System.IO
also it wont work until the photosynther folder is created in your temp directory by photosynth. Either modify the code to watch for that to happen or just start a synth before starting your console app.
Thanks Art79!!
hi binary,
sorry for my unexperience but this is my first time approach with a program like python.
im an italian student and i need to export the point cloud from my synth and following your excellent tutorial i've got the bin files. then i need to treat these with python but... i dont know how! where should i save the .py script and the .bin files? i use windows XP, which script should i write in python?
it would be great if you can explain in details.
thank you
Carlo
Costobenzina, the py and bin files all go in the same directory- other than that I'm not sure I understand your last question.
thank you binary,
i mean, once i open python what shuld i write in? or i just have to open your .py file? i've tried this but nothing appened.
charlo82@gmail.com
Hi binarymillenium,
managed to get the process working in windows, I would like to batch process the bin files to speed things up and also to create one .csv file for import to a 3d programme.
re
python bin_to_csv.py points_0_ 50 > test.csv
I have tried this but can't get it to loop through the files. for example if I have 7 bin files called
points_0_0.bin to points_0_7.bin
what would I write?
Thanks
Drummond, it should be:
python bin_to_csv.py points_0_ 7 > test.csv
Make sure you have downloaded the latest bin_to_csv.py.
Hi everyone, I've done my best to port the Python script to C# a Console Application.
You have to manually download all the .bin files to the same folder, and pass the first (points_0_0.bin) as the argument to the EXE. It should loop through and parse all the files into a single .ply file for import to Meshlab.
Example blogged here. I don't think I've got the colors right yet, though.
Hi there... wow this looks really promising. Although I wouldn't really know where to begin... I would like to get all the points and import them into Cinema4D or something like that.
Would that be feasible?
Hi, thanks for the step-by-step process. Using your python script as a starting point, I've written a rhinoscriptto import the csv points into rhino 3d.
However, I'm still working to isolate the colors. Perhaps a couple tests with pure red, blue, and green synths to determine how they are broken down?
This is Spectacular. Just did a fun post about this:
Computer Vision Gurus and a Cool Technique to Use PhotoSynth as a 3D Scanner of Any Object
at:
http://isontech.blogspot.com/2009/04/cool-technique-to-use-photosynth-as-3d.html
You all might be interested in the php front end that my friend and I've been working on:
http://antiverse.isa-geek.net/bintoply/
-Mark
Well, I was searching a bit a script to import csv file to max ... Nowhere :oD Ok, well I need it to school so as I did it... Color correction works, point positions works.
at the begining, you can change value of unit to increase or decrease size of the point cloud.
PS: Is it possible somehow to export bigger amount of points? I read you skip too small or too big points... So I would like to increase amount of points in the scene a bit... :o) Would be helpful for reconstructing the mesh...
And Here The Script Is> (change "filename.csv" into your filename)
================================
csvfile = OpenFile "c:\downloads\cosi.csv"
unit = 100
try
while not eof csvfile do
(
x = (readDelimitedString csvfile "," as float)*unit
y = (readDelimitedString csvfile "," as float)*unit
z = (readDelimitedString csvfile "," as float)*unit
r = (readDelimitedString csvfile "," as float)
g = (readDelimitedString csvfile "," as float)
b = 0
try
while (not eof csvfile) do
(
b = b*10 + (readChar csvfile as float)
)
catch
r = r * 255 / 31
g = g * 255 / 63
b = b * 255 / 31
Point transform:(matrix3 [1,0,0] [0,0,1] [0,-1,0] [x,y,z]) isSelected:on
$.wirecolor = color r g b
$.size = 1
)
catch
close csvfile
EDIT ======
And Here The Script Is> (change "cosi.csv" into your filename)
I've made a little video of what I did with point clouds and photosynth after working with the processes you worked on here. Here it is: http://www.youtube.com/watch?v=TuHJUS2olyc
-Mark
Mike, I realy appreciate your work.
Could you tell us (me!) how to convert your point cloud (.ply file) into a mesh?
Thanks
Sorry Mark for renamimg you Mike!
Hi,
I love what your tutorial is trying to accomplish. This opens the possibility for all kinds of new and interesting applications of photosynth available without (too much) trouble to may people.
I am working through your tutorial and I am having trouble locating the bin files in wireshark. I searched through all the entries returned in the capture and I could not find any .bin files. Any suggestions?
Zachary, send me an email and I'll send you some screen shots. -M
mark...your email address?
Zachary, my email address is willis[DOT]arch[AT]gmail[dot]com
I'd also like to re-state that I'm just using the concepts and ideas generated here at binarymillenium and not the creator ;)
-Mark
Zachary, I had the same problem. Try viewing your synth with the direct 3d viewer and than you'll be able to find and download the bin files with wireshark.
I've been working with a very large photosynth (http://photosynth.net/view.aspx?cid=a5fc4527-07e7-49ab-8ca4-d5690eb1b947 ) it has 118 bin files associated with it was made from 350 photographs.
It seems that Photosynth handles the point cloud in blocks of 500,000 points. So, the first 500,000 points can be found in files points_0_0.bin through points_0_99.bin. Then the remaining points are held (or next 500,000) points are in points_1_0.bin through whatever.
I've noticed that in the Direct3D Viewer all the point cloud data lines up nicely BUT when you use the python script on points from the first 500,000 set (points_0_0.bin to points_0_99.bin ) with the points from the second set (points_1_0.bin+) they do not. The points from the second set fly off at an odd angle (see a graphic here that shows the problem http://70.114.146.89/~mwillis/photosynth_problem.jpg )
Does anyone know what's going on here? Can you suggest a fix?
If you're curious the 3D point cloud for the synth (after cleaning) can be found here:
http://www.youtube.com/watch?v=pdJPHb5xmUs
Thanks for any help or insight you can give.
-Mark
Mark,
It's really impressive you've got 500K+ points out of photosynth.
There's got to be a transform (3x4 or 4x4 matrix maybe) stored somewhere that relates the coordinate system of the second dataset to the first. It might be easy to find in the xml files, but if it is stored in the bin files it would be very difficult.
Mark,
My understanding of the numbering of the BIN files is the first digit represents the part of the synth you are in. If your synth is 100% synthy you only got a 0, less then 100% and some parts of your synth cant be related to the other parts so they get separated out into a new set of bin files. Ive had up to 155 files in the first part of a synth which was more the 600k points.
art79 is right, in the link you provided I see up to points_0_125 on in that Incan Mountain photosynth, and also different sets up to point_6_0 (since it wasn't 100% synthy). Or was that the wrong link?
Each of the points_# groups will come in with a different world space.
Sometimes you can group, translate rotate and scale each additional point cloud to about where they should go in your first point cloud and merge them. Occasionally there is overlap in the clouds which can help quite a bit, depending on how and why your synth was broken into peices. It is not very accurate but sometimes it is good enough. I haven't tried with meshlab or GSI but I understand they have tools for quilting point clouds together.
Hi ,
I've found a little noob trick to get all the bin simply when u're a noob in Python or C like me, use downthemall, it could use some batch download , here is an example :
http://mslabs-xxx.vo.llnwd.net/d2/photosynth/M6/collections/2c/d3/d3/2cd3d3f7-2139-4d99-bf2a-d2930c3c33da.synth_files/points_[0:40]_[0:40].bin
Just use Add Url and type this whith the good URL it's very easy !! if some body could help me to run de c# version of the script , I never manage to get the exe, is anybody can make it ??? please , it would be Super !!!! thx a lots
Well, at the challange of my friend I went through and made a Java program for all of this. Its in the Alpha version but if anyone is doing much with photosynth and downloading the .bins this will make everything SO much easier.
http://antiverse.isa-geek.net/bintoply/java
Grab the latest version thats windows only and it will allow you to do everything. It will sniff out the packet information like wireshark, and then import the .bin's into the program, and even give you a 2D view of the point cloud, from any of the axis you choose.
Just please remember this is my alpha version of it. But all the basics are there, and a little more.
Thx Lot Vinuen !!!!! Thats rock !!! it only need to load more than 100 bin and it will be perfect
!!!
Dauzet, you can upload for than 100 but you need to break it into several uploads. What I do is zip my bin files in groups of 20, upload the zip files one at a time (browse for one, and hit upload, wait for it to upload, browse for the next one, and hit upload again). Once I get to the last zipped bin file I check "Convert" and then hit upload.
It is true that it will not allow unlimited numbers of bin files. I just uploaded 363 bin files. I ended up breaking it into 6 seperate PLYs but they can be merged in Meshlab to a single mesh.
-M
Daz, the Java program will allow up to as many as you want to load into it limited only by the memory of the computer. You may need to specify the file numbers differently though if it misses something which it can. I have personally loaded over 600 into it at one time using the packet sniffer.
The Web bases system is basically as Mark replied. It is limited only by the size of upload (which you can do multiple of), and the amount of processing time on the computer. If I have the config set to where Im doing dev on it, it is less than the runtime. And honostly I dont keep track most of the time which it is on.
I thought you all might like to see my latest 3D from this process:
http://www.youtube.com/watch?v=9-sjaUjrTnw&feature=channel_page
The 3D part is toward the end of the animation.
Mark,
That's really impressive.
If you can document some of the steps and any tricks you had to do to go through the point cloud to mesh process a lot of people would really appreciate it.
Well, it's not that simple of a process but here is the basic workflow:
Basic Work Flow:
1) Gather low altitude photos of project area
2) GPS several points on the ground that can be seen in the aerial photos
3) Sort out blurry photos and those that aren't part of the subject matter
4) Upload photos to Photosynth
5) Grab bin files and covert to ply
6) Edit ply files for stray points and noise
7) Import ascii ply into an MS Access (or other database software capable of handling millions of cells)
8) Rescale the Z values so that ArcGIS can handle the precision
9) Save the data as a dbase file
10) Import the dbase file into ArcGIS as a shapefile
11) Convert the shapefile into a 3D shapefile using 3d Analyst (optional)
12) Use Spatial Adjustment tool to align shapefile with GPS data collected in the field
13) Create a 3D surface from the 3D shape in Imagine using the Create Surface tool (must be floating double integer)
14) Clip 3D surface with Mosaic tool in Imagine
15) Open 3D surface in ArcGIS and remove all values of zero and replace with NODATA using Raster Calculator
16) Export to Google Earth KMZ file
I think the open source GRASS GIS app could replace ArcGIS but I haven't used it for almost 10 years and might be wrong.
The process isn't exactly simple...
Hope that helps,
Mark
Mark,
But is that how you made the rock mesh that is shown towards the end of the video?
I have some projects for scanned objects like that (topographical data would be intersesting, but not as much as more complex objects)- email me at the name of this blog at gmail.com if you want.
binarymillenium, I'm so sorry about that. I had been distracted with a new synth project and didn't stop to realize that I hadn't posted that work here. Anyway, I was confused.
The work flow that I describe above is for THIS work -
Plan map:
http://70.114.146.89/~mwillis/SynthDEM.jpg
Video of Digital Elevation Model:
http://www.youtube.com/watch?v=j_Iqs8go75k&feature=channel
Google Earth KMZ:
http://70.114.146.89/~mwillis/Synth_DEM_by_M_Willis.kmz
Sorry for the confusion. I'll email you to discuss your question.
Here is the KMZ link again. Just noticed it was cut off:
http://70.114.146.89/~mwillis/Synth_DEM_by_M_Willis.kmz
For some reason the forum is chopping off the letters ".kmz" from the end of the link. So, if you want to view the KMZ, add those letters in lowercase to the end of: http://70.114.146.89/~mwillis/Synth_DEM_by_M_Willis
after copying it into your browser. Sorry for the hassle.
Hi Mark, Ive been using a similar method for processing DEMs and thought i would point out a few things that you should consider.
1) the spatial adjustment tool in arcgis is as far as i know only a 2d adjustment. Photosynth outputs point clouds can have an arbitatary rotation in 3d so you should find a way to apply a 3d adjustment.
2) you can use spatial or 3d analyst to interpolate to a raster if you want to simplify things and remove the need for imagine.
Hello everyone,
Could somebody help me please?
I've downloaded and installed Python2.6 running on windows xp. Sorry I don't have linux (yes-I'm aware that this is a sin)
The last time I had to use programming was at University with MatLab, and I've consumed copious amounts of alcohol to erase the experience.
So what I've done so far. I've got the bin files up to points_0_9.bin using wireshark.
I think I've got the latest bin_to_csv.py file (thanks binarymillenium)
I've placed all the files in the directory C:\Python26\Lib\site-packages - I'm hoping this is a default search path for Python
Ok now I've tried using the IDLE Python (GUI)
>>> bin_to_csv.py points_0_9.bin > test.csv
which gives:
SyntaxError: invalid syntax (points_0_9 is highlighted!)
and with
Python (command line) I get the same error.
If I type
>>>python bin_to_csv.py points_0_9.bin > test.csv The syntax error indicator moves to the "python bin_to_csv" part.
I'm not familiar with the python method of inputting arguments. Should I be using strings or commas or brackets? I'm hoping it's not windows incompatibility issues. I recall binarymillenium saying something about things not working in windows and having to use sygwin in Linux.
Please throw me a lifeline!?
Ok so if nobody wants to help I guess its sink or swim time...
Right I can confirm that running python2.6 for windows does not allow you to run the bin_to_csv.py script.
So do yourself a favor and download/install cygwin from http://cygwin.com/ Make sure that you get as many of the modules that go with it because one of them is python, (although I wouldn't be surprised if there are co-dependent modules)
Also do yourself another favor and read the pdf from http://www.cs.rpi.edu/~magdon/
courses/cs1/labs/lab1/cygwin.pdf
(thank you to the author)
Now if you're a newbie like me and cygwin and python are a means to an end, then follow these steps.
1. Go to Your C: drive and create a folder, - call it what you like, I'm naming mine "Py"
2. Now paste the Points bin files and the bin_to_csv.py script into the folder you've just created
3. Run Cygwin (just double click the icon on your desktop)
4. In the command window type "cd C:" and hit return
5. Now type "cd Py" or whatever name you used for your folder.
The cygwin environment has now been pointed at your file and can "see" it
6. now type
"python bin_to_csv.py points_0_0 > output.csv"
and hit return
7. Go check your folder ("Py" in my example) you should find a file called output.csv which can be opened with Microsoft excel.
I should point out that I'm using the older bin_to_csv.py as supplied by binarymillenium which means that I have to convert several separate bin files, and copy and paste the values from output.csv into a master spreadsheet.
There is a bash loop .py file also available, but I haven't tried it out yet.
I'm going to see if I can get these points into SolidWorks. I'll let you all know how it went.
Thank you binarymillenium, you've made my day!
I realize it's not very clear, but the latest bin_to_csv requires a space between points_0_ and the index of the last bin file:
python bin_to_csv.py points_0_ 50 > test.csv
(there is a space before the 50, it's a different parameter that points_0_)
Very new to coding, but extremely excited about the potentialities of PhotoSynth. I'm having a few problems. I am running Windows XP and was not able to get Python to work correctly. Every time I entered in the command line, there would be a syntax error, with an arrow underneath the v in csv.ply. Any pointers to how to get Python to work in Windows. Someone earlier said they had got it to work but didn't explain how.
Today is literally the first time I have ever looked seriously at scripting, so some of the language on this blog is a bit hard to follow. I am currently downloading Cygwin to try it with that method. Also, I was successful with extracting the .ply file through http://antiverse.isa-geek.net/bintoply/, but the file keeps crashing each time I open it in the program, though the point cloud does load and become visible before crashing the program.
Any suggestions or advice to the above ramble would be most appreciated.
binarymillenium right above your post tells how to get the python script to work.
For my java program if you can email me a with the errors it is getting I can probably fix it.
To do this, assuming I don't make a type mistake here:
---------
1) Start a Dos window
2) go into the directory of bintoply - cd c:/bintoply - or whatever directory it is.
3) type 'java -jar -Xms32m -Xmx256m bintoply.java > btop.log'
3a) You may need to provide the directory of java, usually 'c:/program files/java/jre6/bin/java'
4) Recreate the error.
5) Email me (vinuuen@yahoo.com) the btop.log, and a link to your photosynth.
Sorry for the confusion. The Java program is not crashing, Meshlab is crashing. I am now able to get the point cloud into an earlier version of Meshlab without crashing but the poisson reconstruction is not working. Any trouble shooting advice? The .ply file I am now using in Meshlab came from http://antiverse.isa-geek.net/bintoply/ without any problems.
Email me a link to the photosynth so I can see it and if I get time I will figure out whats wrong.
But I imagine its probably the end of line stuff.
This might solve your problem.
http://pspcexporter.codeplex.com/
The application was extremely helpful in streamlining the process. Thanks for the link. I and a few others in my program are already using it regularly with our work.
Hi!
Thanks to you all for the great work. I managed to reconstruct several also relatively complex objects.
One thing however takes really long: texturing the final mesh.
All the information is already in the synth, so I was wondering if there was a way to quickly project the images onto the mesh for UV mapping?
Any ideas?
Thanks!
froland
froland,
What software did you use to reconstruct the mesh? I'd love to see the output and a link to the original synth.
I think the photo information is in the xml file (maybe in the form of a 4x4 projection matrix), though I haven't devoted any time to figuring it out. Bundler's output might be easier to parse (maybe even documented).
Other than that the 'snap image' function in wings 3d could texture at least one of the images onto the object pretty quickly.
binarymillenium,
Thanks for your help. I'll see if I can get bundler running some time soon.
As a test, I had tried the first reconstruction on a good public synth:
http://photosynth.net/view.aspx?cid=19483106-a1af-4e66-9dac-2615ee4427c6
I used VRmesh, which has a comfortable wizard to generate the mesh from a point cloud. You can see the result here:
http://www3.picturepush.com/photo/a/2882841/img/2882841.jpg
http://www4.picturepush.com/photo/a/2882842/img/2882842.jpg
The face worked out quite well, however, ear and ponytail are not as good. Meanwhile, I did some similar synths myself, which actually made nice 3D prints from shapeways.com. Unfortunately, my VRmesh trial is over now and so I'll see what I get with meshlab. Any tutorials on this?
Regarding UV texturing: Both blender and photoshop CS4 extended can paint texture from a photo onto a mesh, like a projection onto the UV map. Here's a tutorial for blender if anybody's interested:
http://www.youtube.com/watch?v=LkzLzRIv3XE
This face was textured that way it seems, but the mesh could be better: http://www.youtube.com/watch?v=bfgvzFzM9hU
I found it quite hard to get the projection angle right when it comes to organic shapes, eg. heads. Thus, it would be good to get the relative coordinates of the original image and 3D mesh. If I get it right they are in the .json file. Ideally, a blender script could texture the mesh using the json and a few selected images. At least that's my rough idea on how it could work.
Any thoughts on that? Is it feasable?
Cheers!
froland,
That triangulation of a synth is really impressive. The other day I gave VRmesh a try on some laser scan data and it left a lot of holes I don't know how to seal up (I'd know how in Wings 3D but it dies above a few hundred thousands polygons)- the holes were where the laser scan left a lot of shadowed areas, maybe a multi-pov scan would work better.
The synths I've tried to triangulate with meshlab have not turned out well, mainly due to noise- but that face synth is extremely un-noisy so synths of that quality might work.
I'm working on a not-very-robust triangulation method that should work well for producing watertight meshes out of the right kind of point cloud data. If you want to collaborate on something send me an email at the name of this blog at gmail.
What you say about texturing should be possible- though it seems like if you can manually align say a head point cloud with xyz axes, and you have photos of the head along those axes and that are more telephoto (to be more orthogonal), lining up those reference photos should be easy in a 3D program.
A script to automatically align a symmetrical object like a head wouldn't be that hard either, I forget the name of the technique but in Matlab I think it is easy to pass in a bunch of points and find a line or plane that bisects them.
Hi,
first I must say that the tutorial and the comments are great! This is amazing :)
But I've got some problems.
The wireshark, 'downthemall' and phyton routine works fine. Then I put the header and footer of the ply file in manually and manage to open the point set in Meshlab, but when I try to make a mesh out of it Meshlab crashes every time.
Any suggestions on how to do it in Meshlab or other software that might be workable? I'm an architect and would like to be able to combine these synth models with some C4D and ArchiCAD models and stuff.
I'm on a Macbook Pro with 2.8 GHz Intel Core 2 Duo and 4 GB of RAM, running OS X 10.6.2.
Thanx!
Tor,
VRmesh looks like the least expensive software that can turn the point clouds into meshes- it took several hours for a few hundred thousand points when I tried it (it was a lidar dataset not photosynth)- also it has a trial edition that will last a few weeks.
There are much more expensive ones like Geomagic and maybe Pointools, I haven't tried them out.
If your points are are in roughly spherical or cylindrical shape there are some more naive but quicker methods I've been using with my own (unpublished) scripts in matlab. If you have a link to your synths I could evaluate it for that approach.
binarymillenium,
thank you for your quick respons!
I checked out the website of VRMesh, Pointools and Geomagic - it says it only runs on xp/vista, but I'm not running windows on my machine as of now - big cleaning job needed for that to work...
I have only downloaded some synths from the photosynth website to experiment with - can't upload my own, being on a mac you know...
Maybe I'm just in over my head here... but it would have been so cool if Meshlab worked ok. Anyone else gotten Meshlab to work without crashes on these photosynth point sets?
Thanx again! :)
Just wanted to share a recently completed project from a visual studies course I am taking at Columbia University. My partner and I used some of the methodolgies posted here with Rhino and Grasshopper scripting to scan my partners face and create and manipulated a surface modeled from PS point cloud. Here's our final project presentation:
http://www.vimeo.com/9901360
I'll post a full description of the project with a link to imagery in a week or so. Enjoy the video!
You guys have done some amazing stuff here. I've learned more about point cloud processing and photosynth here than any other place on the net. I have a question...has anybody tried aligning a point cloud to any gps or real world ground control? If so, would you mind sharing how you did it? If not, any thoughts on where to find such info? Thanks.
Tracy, both Nathan Craig and I have done this. Nathan has written an excellent page detailing his methods. You can find it here: http://www.personal.psu.edu/nmc15/blogs/anthspace/2010/02/structure-from-motion-point-clouds-to-real-world-coordinates.html
My work flow is a little different and is avilable here:
http://photosynth.net/discussion.aspx?cat=6b63cb81-8b57-4d5d-a978-41d5509bf59a&dis=7b771e05-8a31-4109-8258-e97e4a9f41ae
Hope that helps,
Mark
@Simon Love the video. Great work.
Mark,
Your response helps a lot, thank you so much. We have access to the full suite of ArcMap (3d Analyst, Spatial Analyst, etc.) Do you or does anybody else here know how to get the point cloud registered using ArcMap? Or what steps in your process do you think lend themselves to ArcMap?
We have gotten through the steps where we export the ply file from VRMesh as a .dxf, then import it into ArcMap via Data Interoperability/Quick Import. We used a .dxf because we were unable to get an ascii file to work. Either way, when we bring the point file in and create a TIN, our elevations categorized by color do not follow the terrain as we know it to be. It does not "elevate" to north...the color bands runs east-west, if that makes any sense. When we manipulate the orientation of the point cloud in VRMesh, this does give us some favorable results, but not the accuracy we need. Any help is greatly apppreciated. My coworker also adds that he can't believe how much we've learned here in these forums from you guys...way more than any tutorials from software sites.
--Tracy
Tracy,
There are tools in arcGIS to do georeferencing (these are mentioned in Marks second link). These tools only give you a 2d georeference which may be good enough for some applications but photosynth point clouds really need a 3d georeference that will correct for issues such as a tilt in the surface.
For this you are probably better off using Java Graticule 3D that is mentioned in Marks first link (I haven't tried it but it sounds good... or find something similar).
ArcGIS does have the capacity to do a 3d georeference but it seems to only be available through arcobjects code.
Tracy, if you're using VRMesh Studio, you should still save the file as ascii PLY or as a comma delimited file. Open that file in notepad, delete the header info, select all the data (ctrl-a), copy (ctrl-c) and paste (ctrl-v) that into an empty Excel or Access Database. You'll end up with six columns (X,Y,Z,R,G,B). Because ArcGIS has issues with floating points, you'll need to scale the data. What I do is create three new columns (X1,Y1,Y2) where X1=(X+100)*1000, Y1=(Y+100)*1000, and Z1=(Z+100)*1000. By adding 100 to XYZ I'm removing all the negative numbers (This is just my personal preference). By multiplying by 1000 I get rid of the floating point problem. The next step is to save the database. Open ArcMap. In ArcMap go to TOOLS>ADD XY DATA>Pick the database file you just made> Choose your X and Y Field. Leave Coordinate System as Unknown. Click "ok". This will give you a bunch of pointsin the map view. Right click the new file under layers, go to Data and then EXPORT. This will allow you to save the 2D data as shapefile. ArcMAP will ask you if you want to add the newly exported file. Click Yes. Ok, now we have a 2D point shapefile. To create a TIN or similar data you need a 3D shapefile. To do this open 3D Analyst toolbar, click 3D Analyst>Convert>Features to 3D. That will open a dialog box, make sure the 2D shapefile is selected under Input Features, then select the radio button next to "Input feature attribute". In the drop down box next to that pick your Z2 column. This will associate the 2D points with a Z value when you export it. Export a new 3D shapefile.
This 3D shapefile is what you need to generate TIN, Grids, contours, etc with. You do that in 3D or Spatial Analysts.
Easy, huh? =)
Art79 is correct about the georeferencing. Most people use "Georeferencing " under Tools>Customize>Tool Bars. Note that this assumes your synth's ground plane is aligned with the real world ground plane.
I guess I should get around to actually writing a tutorial...
Mark,
THAT's what we've been wracking our poor mushy brains on!!! We knew we needed to scale the xyz's to something more usable but had no idea what kind of multiplier to use. We have z's at 0.81xxxx and 1.25xxxx and that relates to nothing in the real world that we could figure.
We printed out Nathan's step by step guide and were going to try to use Java Graticule 3d (thanks to you too, Art79, we will try that approach as well; better for us to try several avenues and compare the results).
Why'd you pick "+ 100 (* 1000)"?
Thanks again, you guys are the best. Probably saved our Arc software from getting whacked with a baseball bat tomorrow...
Eh, I forgot this...we're using the demo of VRMesh to clean the point cloud. My coworker found that the way the points imported into Arc is dependent upon how the point cloud was oriented in VRMesh when exported to a .dxf.
He spun the pt. cloud around and saved several times and we got different results -- once it came into Arc -- every time. He spent a lot of time with the Widget Transformation tool in VRMesh tilting the point cloud with top towards us, away from us, tilted flat away from us, upside down, etc., but we never got an understanding of orientation in VRMesh. It seems friendlier than Meshlab but the "help files" haven't been so helpful. Does anybody here understand how orientation works in VRMesh, as far as saving out the point cloud? Does the Widget actually do anything or is it something just for reference of some kind?
Thank you all again and good night...;)
Doing most of the extra programming for Mark, I can say that some of the programs have problems with negative numbers. So the +100 always moves any negatives to positives. Though a +10 should work as well from my experience. The *1000 is so you always get a 3 place precision. Most points in the point cloud wont match a x/y/z of 999/999/999 so you get a larger varied cloud of all positive numbers. Depending on the scale you use of +SS you then get a SS,xxx/SS,yyy/SS,zzz point for every point in your cloud. You always scale first so that you end up with positives. If you multiply first you get negatives unless you use an absurdly large scaler. The scaler is just moving the point cloud's origin to the scaler before multiplying. Once you multiply it you can just get rid of the the numbers after the decimal. And if you need more precision just Change your multiplier to a higher number. The actual picking of +100 and *1000 was just testing to see what worked best.
And no I dont know much about VRMesh or any of the others.
Hope this helps.
- vinuuen, The other Mark
So helpful, thanks again, your suggestions and comments rock! Lots of different things to try today...
So...another GIS-related question, wondering if anyone else has noticed weirdness of this sort: we got some lovely photos of a valley, bottom to top and terrain on either side and ran them through Photosynth.
Then we exported the point cloud and put it into the demo version of VRMesh, cleaned up the few stray points and exported as a .dxf. We used Quick Import through Data Interoperability to get it into ArcMap. Then we made it into a 2d shapefile, then a 3d shapefile from which we can render TINS.
The weird thing is, when we render TINS, we get different colored bands that represent changes in elevation, which is expected, BUT they're running vertically -- up and down the valley -- rather than horizontally as one would expect (and hope for!) across the valley (I am over-explaining, but if you're looking into the valley, the bands of elevation run north/south if you pretend north is "up" when they should be running east/west..side to side).
Anybody have any thoughts on why this is occuring and what can be done about it? I have some suspicions about the .dxf export process from VRMesh...we get different resulting data in ArcMap, depending on the position the point cloud is in when exported from VRMesh, as in how it's tilted, spun, oriented when we save it as a .dxf. But it seems no matter how we tilt, spin and export, we get elevation bands going the wrong way in our Arc TINS.
Any help or musings is/are appreciated.
--Tracy
I know from some other software I have used in the past for CnC Operations that one program may export a DXF as x,y,z. Then the importing program reads, x,y,z, and another reads y,x,y. So that may be your issue there.
Another thing to look at is to make sure you are only using the first set of points. points_0*.bin and not the points_1*, 2* and so such. Those have different orientations and are unmatched points, so that can also screw things up.
Bleh.. err, the other one reads in y,x,z. dern tyops.
Thanks...more stuff to ponder and much appreciated.
--Tracy
Tracy, sounds to be like you model's Z axis isn't aligned with the real world Z axis. Which is basically what Vinuuen is getting at and sounds like what you've been experimenting with.
Can you show us the Synth you're starting with?
-Mark
Mark,
Do you want the URL address of the synth or the point cloud we've extracted, which is HUGE.
Thanks,
--Tracy
Tracy, just the URL for now. I won't be able to look at it until early next week tho. Heading out for the weekend. -Mark
http://photosynth.net/view.aspx?cid=0dcf06d3-8dc7-4c35-b5e7-ff810bd5489a
Thanks! Have a good weekend.
Hi All,
I'm very interested in using PhotoSynth output to plug into dense point-cloud reconstruction (such as PMVS2), as PhotoSynth seems to get better results than the offline reconstruction code.
SynthExport has just added exporting of the camera parameters to CSV, and I'm trying to write a quick javascript form to generate the next stage of input for PMVS, but I am having problems with the format of the parameters.
The parameters output by Bundler (which is based on the same code as PhotoSynth) are listed in the documentation, and I have applied calculation to convert the X, Y, Z rotation into a matrix, only to find that the rotation values are almost definitely in radians and not degrees.
I need some sanity checking on my conversion, as I'm not fantastic with matrices.
Kind Regards,
Josh
I was able to export cameras from Photosynth to Maya. You can see the results here, http://www.xrez.com/blog/photogrammetric-gigapixel-images/ and get the code (thanks to help from art79 & Simeon Basset) on the new photogrammetry forum, http://www.pgrammetry.com/forum/viewtopic.php?f=14&t=85
Greg, that is awesome work! VERY VERY NICE! It is great to see our work evolve like this.
Thanks Mark!
Bit behind the times here, but I'm working on producing a 3D model of a hillside that is very susceptible to landslides for my final year project at University. I've managed to get as far as downloading the .bin files but don't understand how to progress and covert the files so they're suitable to be used on meshlab. Any help would be seriously appreciated!
Ross, my tutorial may help:
http://palentier.blogspot.com/2010/12/how-to-create-digital-elevation-model.html
Post a Comment