Segment anything

[PablerasBCN]

segment anything implementation idea.

May be we could rasterize current view, run segment anything on it, then project the colors into the cloud.

I know the color projecting may not be easy but has been a forever requested feature, so here another excuse to implement it :D

this is a segmentation from a screen capture of a point cloud



to make it perfect, may be the ideal scenariow ould be to move camera over a path and every nth frames segment anything, this way if a point accumulates a class n number of times you would get a more robbust segmentation (I think).

Just an idea. Cheers.

[daniel]

Does "segment anything" means something in particular? Or is it a manual process? (sorry, I don't understand)

[PablerasBCN]

Apologies!

I thought this would be more known. Not too long ago facebook released a segmentation model that seems to work quite nicely. I tested on an screen capture of a point cloud and certainly idientifies objects properly.

Segment Anything
https://github.com/facebookresearch/segment-anything

So simply as an small chat idea said that it would be cool to have the hability to screen capture, or render current view at XY resolutions, run the model and classify points in viewport based on the segmentation by projecting the segmented image over the cloud as SF.

[daniel]

Ah, interesting. Sadly it's a Python repository. I won't be able to do anything with it myself. But maybe via CloudCompy or the Python plugin, someone skilled enough could come up with something indeed.

[SD_cc]

I've been testing Segment Anything on rasterized images of point clouds. the results are super encouraging! In my particular case, the resulting polygon is used to crop the point cloud (using CC CLI). Could be interesting to transpose into a plugin...

[jason3d]

Ive been following looking for pointcloud implementations, saw a bunch of GIS projects adding it or something similar, also came across https://github.com/Pointcept/SegmentAnything3D

[Oxyum1]

I've look a bit into segment anything and I have found this api : https://slaice.ai/
with this api I can use SAM without too much code. But I don't know if it's the best one... do you guys know of it or have some advise ?

[Ciroo1]

I've look a bit into segment anything and I have found this api : https://slaice.ai/
with this api I can use SAM without too much code. But I don't know if it's the best one... do you guys know of it or have some advise ?

Yeah, Slaice.ai looks pretty handy if you want to use SAM with minimal setup. It’s good for quick integrations, but if you need more control or want to avoid API limits, you might consider running SAM locally. Depends on your project—API for convenience, local setup for flexibility.

[PablerasBCN]

Hi, so, this allowed me to test exactly why on the original post I did suggest

https://github.com/Yarroudh/segment-lid ... me-ov-file

I has an option to in a free perspective set the view for the segmentation to be run, so it is "usefull"for non top down scans.

YEt, since you can only get "one shot" is not usable in practice but serves as a ground for testing the concept.

I had to install twice, I would say it was bcause I had to install pytorch specific to my cudatoolkit version.

---------------------

the python commands, this one is with the topview I did just for testing but if you use the free view the resolution param seems not to be needed as resolution seems to be the size of the 3d windows. Also it would be welcom point size control etc... but still for giving it a quick test, it works.

---------

from segment_lidar import samlidar, view
import os

# Usar vista interactiva (Pinhole)
viewpoint = view.TopView()

# Crear la instancia de SamLidar con la ruta del modelo
model = samlidar.SamLidarmodel = samlidar.SamLidar(ckpt_path="C:/Users/pablo/anaconda3/envs/samlidar/modelos/sam_vit_h_4b8939.pth",
resolution=0.05) # Aumenta la resolución de la imagen raster


# Leer la nube de puntos desde la nueva ruta
points = model.read("E:/LIDAR_JAPAN/MMS11_Triple_Lane_Touge/MMS/SAM_TESTING/Input_DATA_INTENSITY/08NF6208.las")

# Segmentar la nube de puntos
labels, *_ = model.segment(points=points, view=viewpoint,
image_path="E:/LIDAR_JAPAN/MMS11_Triple_Lane_Touge/MMS/SAM_TESTING/raster.tif",
labels_path="E:/LIDAR_JAPAN/MMS11_Triple_Lane_Touge/MMS/SAM_TESTING/labeled.tif")

# Guardar los resultados en la misma carpeta
model.write(points=points, segment_ids=labels,
save_path="E:/LIDAR_JAPAN/MMS11_Triple_Lane_Touge/MMS/SAM_TESTING/segmented.las")