LiDAR and Drone Automation for Detailed Photogrammetry
The R&D team is improving our ability to capture and process 3D models of objects and environments. We’ve previously published an article on how we reconstruct journalistic scenes in 3D, and created an end-to-end guide for photogrammetry with mobile devices. We’ve also experimented with using NeRF for high-fidelity captures of people, as well as for scenes not usually fit for photogrammetry.
Recently, we began testing what it would be like to incorporate some advanced capture techniques into our workflow with an eye to improving both quality and efficiency. As 3D modeling for journalism becomes increasingly ubiquitous, we believe that speed and accuracy will set apart the work done by The Times. LiDAR (Light Detection and Ranging) and automated drone capture are two technological areas we identified that held the most promise for our experimenting.
We generated this sparse monochromatic point cloud with LiDAR. While it doesn’t provide the photorealistic reconstruction of photogrammetry, it does generate more accurate measurement data.
LiDAR
A LiDAR scanner allows us to measure distances in 360-degrees with accuracy down to the millimeter. This makes it an essential tool for efficiently reproducing complex environments, providing the ability to create highly detailed measurements when capturing small objects, or entire city blocks.
LiDAR can also capture color data for individual points of measurement, allowing us to create this dense color point cloud.
Using a terrestrial LiDAR scanner, we took 68 scans across the field, often while foot-deep in muddy water. Each scan captures an area with a radius of approximately 100ft, but many scans are needed to fill in the areas between the sculptures.
This was an ideal test site due to both its ambitious scale and granular details: 124,021sqft made it our largest capture, and chipping paint and crumbling concrete made a suitable challenge for the fidelity of our LiDAR sensors. The private property also allowed us the ability to test out our automated drone workflow for 3D capture in a safe setting.
Lincoln’s bust fell during relocation, exposing the inside. Surprisingly, LiDAR was able to measure the internal dimensions through a small hole in the sculpture.
LiDAR also aids in the swift creation of ambient occluded models. These renders are useful for highlighting important shape and detail without photorealistic textures applied. This approach is often preferable for stories where removing distracting visual elements is necessary to convey key information related to shape and size.
We can, of course, go further than using LiDAR to improve textureless models. Combining the scans with traditional still photogrammetry and our automated drone photogrammetry workflow yields incredibly detailed 3D models in a fraction of the time of traditional captures.
Our drone executing its automated path.
Automated Drone Capture
As drone technology improves, even consumer level drones have begun to incorporate pre-planned flight paths. Using predetermined paths helps us expedite the capture, mathematically ensuring we have enough images with enough overlap for a highly detailed capture, without extraneous or duplicate frames. In addition to automating the path, we also automate the number of images captured during the flight, and from what angles.
The first pass captures orthographic, or top-down images. Each successive pass traces a similar path over the same area and captures images from four different perspectives, ensuring proper scene coverage to generate a highly detailed model. Previously, photographers would manually capture each image, and carefully annotate by hand what images they had and what images they still needed to capture. The preproduction for automated drone capture negates that laborious process.
This model was created by combining high resolution automated drone capture with measurement from LiDAR scans.
The Big Picture
Adding LiDAR to our workflow allows us to make models with a significant increase in accuracy. Automated drone captures are faster and more efficient. Combining these technologies allows us to deliver the most detailed models we’ve ever created, faster and more reliably than we have before. This means that our journalists can spend more time reporting stories and less time solving technical issues.
- Efficiency. Automating our drone captures saves time by cutting down a labor-intensive manual process. LiDAR enables us to quickly measure complex environments.
- Accuracy. Not only is LiDAR fast, it’s accurate, allowing us to take accurate spatial measurements quickly and reliably. Automated drone capture means less duplicative and extraneous image data, and a more accurate reconstruction.
- Flexibility. Combining these capture capabilities allows us to model much larger and more complex spaces than we have before with a smaller footprint, enabling 3D capture for many more scenarios and journalists.
Looking Ahead
The available hardware and capabilities for photogrammetric capture are evolving quickly, and R&D plans to continue to explore this space.
Want to learn more about our experiments with photogrammetry? Read our End-to-End Guide to Mobile Photogrammetry or follow @NYTimesRD for updates on future explorations.