Handling large areas

[Cambraceres]

Hi, I am a brand new user needing to integrate lidar data into a workflow currently using photogrammetry only (PIX4d).

I am changing our data gathering technique because we have a project to complete that is too large to process in that manner.
The data will be gathered using Emesent's Hovermap software. My question has to do with the overall size, the methodology to deal with large datasets, and the likelihood my computer will need to be upgraded. This is a project of (2) 40 acre parcels.

Are there any solid workflows I can rely on to deal with projects of this size?

My system exceeds system requirements but seems to be lagging a bit with a sample dataset from another project of only 77 million points.
Specs:
Processor: AMD Ryzen 7 1700X Eight-Core Processor, 3400 Mhz, 8 Core(s), 16 Logical Processor(s)
Installed Physical Memory (RAM) = 32.0 GB
Total Physical Memory = 32.0 GB
Total Virtual Memory = 89.9 GB
Graphics Card: GeForce GTX 1080 TI

What is the recommendation here?

TY

[daniel]

Hum, with such a configuration, you should be able to handle much larger point clouds indeed (i.e. 300 M. to 500M. points at least, while keeping the GUI version responsive). Also, if you wait a little bit, you should see the LoD (level of details) structure being prepared and finalized. It should help then to keep things responsive, even with larger clouds.

Also, make sure you don't display the normals (if you have any) as this is always super slow to display.

Then regarding your workflow, it depends on what you want to do exactly? What's your expected output?

[Cambraceres]

Daniel, thanks so much, I appreciate the insight. It seems that cloudcompare has several tools I need to use.

I prepare topographic maps using PIX4d. I have been doing so for years, but now we see the need with certain jobs to use LiDAR to minimize areas lacking data from vegetation or other obstructions. I will be receiving in a day or two point clouds gathered using Emesent's "hovermap" system, and these will need to be utilized to produce topo maps of (2) 40 Acre sites.

I need to be able to perform several functions.

1. Georeference the point cloud (match it to points surveyed in field, and I have tested that process with small Pix4D generated clouds of 50m or so points, it works well with those.

2. I need to apply the CSF filter plugin to clean off vegetation, to whatever degree that is feasible, such that only ground points remain

3. I need to be able to import the resulting .las file into Civil 3D to actually prepare the survey documents. There may or may not be an orthomosaic underlay from Pix 4D to allow structures and features to be traced. ( I have had zero luck with this and desperately need to figure out why the las files are giving me an error when I try to import them to CAD)


Importing the .LAS file into C3d is critically important. I also tried to import .shp, which I do all the time using pix 4D created files, and it also did not work.

I have also one beginner question. My system performance display appears to suggest that my graphics card is barely active. When I read support articles they all say to go to the NVIDIA control panel, select the program, and then use the dropdown to tell it to use the GPU. That control panel is missing that option. I suspect my GPU to be inactive, which may explain why even a 30m point cloud gives me a display buffer.

[daniel]

1. You can do that with the 'Align' tool

2. The CSF filter is quite greedy in terms of resources. But if you have a lot of memory and time, hopefully it should work?

3. You can definitely export LAS files (and there's 2 different LAS export libraries on Windows, so at least one of them should do the job properly ;)

When you start CloudCompare, you should see in the Console (just after the message 'CloudCompare started!') the Graphics card that is used. If it's not NVIDIA, then yes, there's an issue. But that would be surprising (what else could be used?)

[Cambraceres]

Daniel, thank you much.

It appears that the software is indeed using the proper graphics card. Can you think of any reason it would never go above 2 percent usage? My CPU, on the other hand, goes straight to 100 when applying the CSF filter, for instance. The performance is an issue as I cannot see the points well enough to clean the ground of vegetation. The CSF filter does quite a bit of the work, but I need to be able to manually clean some of the rest if possible.

Should I expand RAM to 64 GB considering my purposes?

I still cannot view even small clouds without a buffer and lag showing up every ten seconds or so.

Do you have any best practice documentation on exporting the .las files? That is critically important. What do you mean by "Export libraries"?

I appreciate your quick responses, the flights are happening now, so I need to be ready to move with a workflow to get the proper info into Civil 3d to allow map creation to proceed.

Thank you much

[Cambraceres]

I cannot figure out why the .las files I save from cloud compare will not load in c3d.

[daniel]

CloudCompare doesn't use the GPU apart for displaying the 3D entities (I mean it doesn't do computations on the GPU). Then it can store the points on the GPU memory (if you select the right option in the Display options). But only the clouds (not the meshes). This should make the display much faster.

Increasing your computer memory should help. But if your clouds have only 50M. points, then the issue might be somewhere else (you can look at the memory usage live during a computation to see if you reach the limit and Windows starts to swap?).

Then, on Windows, you have to different 'filters' that you can select when exporting LAS files. You have the default 'LAS cloud' filter (which relies on the PDAL library), and you also have the LAS 1.3 or 1.4' filter (which relies on LASlib). The two filters should output 'standard' LAS files. You might want to ask Civil 3D why their software refuses to load it?