PCL library and binary format

[Dimitri]

Hi All,

as any of you used the PCL library :
http://www.pointclouds.org/

What'd you think of their file format ?
http://pointclouds.org/documentation/tu ... ile-format
With the work we're doing with Nicolas Brodu, we're now generating points clouds with a lot of scalars attached to each point and we're looking at a binary file format to speed up file loading, and reduce file size.
Their implementation seems quite simple to read and write, and extremely flexible (binary or ascii with systematically an ascii header which is quite useful)

Would it be more interesting than the .bin format of CC ? I like that each scalar can have its name attached to it so that it could be loaded directly into CC for easier management of the various scalar fields.

The only thing it does not really cater for is the case of an archive of multiple scans (generated from segmentation, or corresponding to different scanning positions) while the .bin format of CC can do that (or alternative open source format such as .ptx,.ptg or.ptz).

Note also that they have developed a lot of functions in their libraries that could be useful in qCC. It is quite oriented towards robotics, and it remains to see how it operates on very large point clouds.

Cheers

Dimitri

[cnnewton]

PCL is base on VTK. CC is base on OpenGL. I think is hard to put PCL into CC.

[daniel]

Hi,

i'm not sure to understand why VTK (which is a multi-purpose library to load/process/display all kinds of 3D data) would not be compatible with OpenGL. See vtkOpenGLRenderer for instance.

However that's not really the point as what Dimitri propose is not to use PCL to display entities, but only to load/save point clouds?

Even simpler and less "contaminating": we only have to support their file format in CloudCompare. In this case, I totally agree with that (I'm only lacking of time to do the job myself!)

[cnnewton]

Hi,
You are right, Dimitri is interested in PCL format, not combine PCL to CC.

[luke_penn]

PCL is base on VTK. CC is base on OpenGL. I think is hard to put PCL into CC.

I would like to make the point on PCL library, it seems it is not clear what is...

Note that pcl uses VTK ONLY for its visualization classes and not for the algorithms.
PCL can be compiled without VTK support if needed.
PCL includes many freshly developed algorithms for point cloud processing:
- various features descriptor computations: http://docs.pointclouds.org/trunk/group__features.html
- some generic filtering algorithms: http://docs.pointclouds.org/trunk/group__filters.html
- kdtree searches using FLANN
- keypoints extraction, as Harris, NARF, ad SIFT
- registration with inital alignement computation based on keypoints and sample consensus methods
- a lot of standard-use filters as normal estimation, principal curvatures estimation, segmentation, sample consensus model fitting, etc etc... they are also introducing CUDA and many functions can works on multi-cored pcs

I think PCL is growing really fast with the help of a really big community (see http://pointclouds.org/about.html), maybe the use of this library for CC should be taken in consideration :-) The possibility to have access to PCL features from a really nice GUI as CloudCompare could, I think, give better visibility to CC.

I think developing IO support for PCD format should be pretty simple, but what about beginning to use the algorithms?

Only question... they use BSD license... is this a problem?

Luca Penasa

[daniel]

Indeed, it would be nice to "open up" CC to such an interesting library.

I guess we should be able to create rather easily "converters" to automatically convert a ccPointCloud to a PCL one (and vice versa, even if it might be incomplete in this way).

However, CC is not just a GUI front-end. It has its own database and structures (such as the octree). And everything is made around it. Moreover, each PCL algorithm would need its specific dialog, menu entry, etc. It would become a bit of a mess in the menus and I fear for the stability of the whole (I already fear for the stability of CC alone :D).

The safest thing would be to port each PCL algorithm (or set of algorithms) through CC plugins. Each time there would be a conversion back and forth of course between ccPointCloud to a PCL format.

And for the serious part: who wants to give it a try? (I can help for the converters and building an example plugin)

P.S.: BSD license is compatible with GPL (at least if I believe wikipedia)

[luke_penn]

Indeed, it would be nice to "open up" CC to such an interesting library.

And for the serious part: who wants to give it a try? (I can help for the converters and building an example plugin)

I could help in writing a reader for pcd files. I think the easiest way to get this is to write the support as a plugin, that could be compilable once one have PCL installed.
I think the reader part is not problematic but the writer give me some problems: pcl is highly templated so the point clouds are normally initialized as:

pcl::PointCloud<PointType> cloud;

where PointType is one of the types provided by the library itself as PointXYZ, PointXYZRGB PointXYZI (intensity) etc...
Custom types can also be provided as user defined strucures but for writing a CC point cloud in which we can have any number of fields one would need to initialize a custom point type each time... Given I am not a really skilled programmer I don't know how to face with this problem. Maybe we can talk about this problem in a further moment.

We could start with a plugin that:
- enable PCD extension opening in the file chooser
- read that pcd file using PCL library (In this sense one would need to have PCL installed for compiling the code - I think is not the best option but it does make sense to me - one would need to work with PCD files if he/she uses PCL)
- convert the pcd file to CC internal format (keeping sensor position)

PCL supports also dense clouds (rows x lines of scan) so when possible also this type of data storage can be considered if CC have its own type for dense clouds.

If someone could provide me an example plugin that
- enable file format in file chooser
- create a simple CC cloud and populate some of the fields
- send the converted point cloud to the active CC 3D view
I could have a smart insight on how CC works and i could complete the plugin with the conversion part

I think that as first attempt we not need any dialog for opening pcd files (for selecting fields to be imported etc...).

Also pointing me to the right lines of codes that does the things above could be enough but i don't know anything about CC so...

[daniel]

Indeed the plugin seems to be the best solution.

I have a few remarks:
- plugins in CC are totally independent. By design, they can't have any influence on the application. So for instance a plugin can't change the behavior of the file chooser. However, a plugin can spawn its own file loading dialog (which is simply a Qt's QFileDialog instance) and then create the corresponding ccPointCloud which will passed be eventually to CloudCompare as any other 'I/O filter' do.
- once the plugin is compiled, it will be shipped with the CloudCompare official release as the other plugins (at least for the Windows version). It may have to be shipped alongside a PCL Dll (if we can't use static linking) but anyway it should work for anybody. Or do I miss something? (I didn't had the time to look at PCL in depth yet).

The easiest plugin to mimic is 'qKinect'. It connects to an external device (through a 'third party' library - libfreenect), spawns a small dialog, acquire a cloud and send it to the main application. However, if you don't want to bother with this too much, you can first write the pieces of code to convert a PCL cloud to a ccPointCloud (and the inverse conversion?). I can then build up the plugin and you'll use it as a base for your next developments (applying PCL algorithms I guess?).

For the multiple scalar-fields issue, we can assume first that only the active scalar field will be considered/exported (as most of CC algorithms do actually).

[luke_penn]

I managed to write my first plugin for cloudcompare, but here are some problems:
- I am really not able to write a self-contained reader for PCD files (I am not a really skilled programmer so it would take me too long). I think there are two possibilities: (first) re-write all the IO classes from PCL but for me is pretty difficult, given they support binary, ascii and compressed file formats. One would need to read their IO source files and reimplement everything (they also makes use of some externel libraries as Eigen and Boost also for the IO part). (second) take a subset of their headers/implementations files and use them for loading their file format (I dont know if there are any license issues - also the external library problem remain). We could also ship a basic version of pcl with CloudCompare given they use a modular approach we can reduce the compile time of pcl only to the IO parts, without compiling all the modules (filters etc...), but some dependencies problems would remain.
- Given that I am interested in producing a toolbar for PCL algorithms for me loading a PCD files is useful only if I have a complete version of PCL installed on the system.

PCL developers provide a nice all-in-one installer also for windows. So i think it would be enough that who will eventually prepare the windows release of the plugin simply add a check for ensure the library is installed. If not a simple error message could be given to the user and no PCL functionality will be enabled. Maybe CC will automatically not load the plugin if cannot find PCL libraries? I down know exactly how plugin support works.

So I would like to separate our aim in two different things:
- PCD support for CC. That need to be coded into CC itself as for other formats. What solution have been used for LAS files?
- PCL plugin that can be used against any cloud loaded in CC. This will provide a common interface to several filters/algorithm from PCL (so PCL is really needed) from filtering to registration.

For now I can give PCD reading capabilities to this plugin but I am not able to afford coding PCD format inside CC, that would require someone who can decipher how they pack data in binary format and how to read it. If someone else can do this it would be great (so also my plugin will use that support).
At now my plugin add a toolbar and eventually a menu dedicated to PCL algorithms, is this a problem for CC's plugin policy?

If this plugin goes outside of the normal plugin policy I could keep its development as a "third part" plugin, maybe on GIT or somewhere else.

Any suggestion would be really appreciated!

Now i have some question
- PCL algorithms does not have any support (as far as i know) for progressbar. what plugin could i see for an example? Conversion from PCL point cloud and CC is made simply copying the data so i think i could use a progressbar here.
- Is there a generic progressbar with the bar moving continuously left to right (for example) so to give the user the impression something is happening?
- Is there any native support for start computations in a separate thread, so that the user could eventually press a STOP button? Every time I will start a PCL algorithm on a CC cloud a copy of needed data will be made so I think that leaving the user interact with main window will be safe. How CC filters/plugins normally solve this problem?

Thank you for CC. Is a really great tool! This will permit me to make my algorithms available on all platforms :-)
and sorry for the long mail!

Luca Penasa

Indeed the plugin seems to be the best solution.

I have a few remarks:
- plugins in CC are totally independent. By design, they can't have any influence on the application. So for instance a plugin can't change the behavior of the file chooser. However, a plugin can spawn its own file loading dialog (which is simply a Qt's QFileDialog instance) and then create the corresponding ccPointCloud which will passed be eventually to CloudCompare as any other 'I/O filter' do.
- once the plugin is compiled, it will be shipped with the CloudCompare official release as the other plugins (at least for the Windows version). It may have to be shipped alongside a PCL Dll (if we can't use static linking) but anyway it should work for anybody. Or do I miss something? (I didn't had the time to look at PCL in depth yet).

The easiest plugin to mimic is 'qKinect'. It connects to an external device (through a 'third party' library - libfreenect), spawns a small dialog, acquire a cloud and send it to the main application. However, if you don't want to bother with this too much, you can first write the pieces of code to convert a PCL cloud to a ccPointCloud (and the inverse conversion?). I can then build up the plugin and you'll use it as a base for your next developments (applying PCL algorithms I guess?).

For the multiple scalar-fields issue, we can assume first that only the active scalar field will be considered/exported (as most of CC algorithms do actually).

[daniel]

Hello,

LAS file is typically the example of what shouldn't be done in CC ;). It's quite a hassle to compile liblas (because of the boost dependency mainly) and we should definitely have opted for a purely plugin-based solution at that time (instead of putting it directly in the main CC executable as an I/O filter, bringing a lots of dependencies to third-party libraries). It is now deactivated by default and I only keep a working version of liblas to compile the official (Windows) release. As soon as the Cmake project for CC is ready (soon ?), it will be much easier to handle such cases (with check-boxes to enable or not liblas support, and the library installation path to setup if it's enabled).

But meanwhile, we must only rely on plugins for importing/exporting PCD files (so only the corresponding projects will depend on PCL libraries). And as you said, if the PCL libraries are not present on the system, the plugin simply fails to load and CC will start without loading it. By the way, the "Windows" PCL binaries are only for MSVC (which is not the official compiler of CC). This bound to Code::Blocks will also be removed with the future Cmake version of CC.

So: plugins, plugins and more plugins. But as you can't modify CC behavior via plugins, you can't add a toolbar "as is".

You have too options:
- either you create a collection of small plugins, each corresponding to a single operation (import from PCD, export to PCD, Processing 1, Processing 2, etc.). The advantage from your point of view will be that each one will automatically be added to the plugin toolbar. If you name them with the same suffix (PCL[something]) and set them icons with a coherent style, you'll get a nice 'PCL' toolbar in CC. The main problem is of course the hassle of managing multiple projects (but it will be a simple duplication of a unique project with very few customizations each time).
- otherwise you create one big plugin that spawns a toolbar like dialog each time you click on it. It will then apply the user selected action (same list as above) on the currently selected cloud (as plugins have automatically access to the selected entities. And they can also output new clouds that are automatically added to CC main database). You just have one plugin project to handle and you can share a lot of code. I think it's definitely the solution to start with (waiting for a better integration, thanks to Cmake). It will also be easier to re-use the code as a separate class/library once a proper integration could be realized.

I looked at PCL code and tutorial and I couldn't see what is the problem with loading PCD files? If you check either the 'test' folder, or the tutorial (http://pointclouds.org/documentation/tu ... eading-pcd), you'll see that you first have to load the undefined PCD cloud in a "blob" structure:

Code: Select all

sensor_msgs::PointCloud2 cloud_blob;
pcl::io::loadPCDFile ("test_pcd.pcd", cloud_blob);

Then convert it to the type of cloud you are able to handle:

Code: Select all

pcl::fromROSMsg (cloud_blob, *cloud);

In the case of CC, 'cloud' will typically be of type pcl::PointCloud<pcl::PointXYZRGBNormal>::Ptr. I'm not sure how to handle scalar fields (I guess that PCL offers an equivalent concept?)

And eventually, for your questions:
- the ccProgressDialog is the default progress bar class. It implements the CCLib::GenericProgressCallback interface that gives all necessary accessors to 'play' with it (there are many examples of how to use both).
- the qPoissonRecon plugin uses a "never ending" looping toolbar (as we have the same problem). This is just the way you initiate the ccProgressDialog that make it so.
- Qt's QtConcurrent framework offers a very nice and simple way to launch pieces of code in separate threads (see QtConcurrent::run for instance). In CC plugins (which are QPlugins) you have a direct access to this framework.

Don't hesitate to share your plugin project with me in order to work on all those points as most efficiently as possible (and contact me directly by mail also).