Name: a field primer for monitoring benthic ecosystems using structure-from-motion photogrammetry
Uploaded: 2021-04-15T12:30:00
Duration: 6 min 36 s
Description: Watch this Scientific Journal Video about A Field Primer for Monitoring Benthic Ecosystems Using Structure-From-Motion Photogrammetry at JoVE.com

We thank the Paul G. Allen Family Foundation for funding this research and are grateful to Ruth Gates for the inspiration to use technology to help conserve reefs. We also thank NOAA and other collaborators for thoughtful discussion concerning these methods. Lastly, we thank Catie Foley and Patrick Nichols for providing the drone and underwater video of these methods.
We acknowledge the National Fish and Wildlife Foundation as a funding partner in this work.

1. Materials
<ol>
	<li>Camera
	<ol>
		<li>Ensure minimum specifications of durability and waterproof nature (or a waterproof housing) and a minimum frame rate of 2 frames/s (fps). 
		NOTE: A minimum frame rate of ~4 fps was used in this example.</li>
		<li>Digital Single Lens Reflex (DSLR) camera
		<ol>
			<li>Set the camera to shoot continuously at a photo capture rate between 2 fps and 5 fps.</li>
			<li>To reproduce the protocol described for this example, use a camera in an underwater housing (see Table of Materials) with the following settings: Manual Mode (M); f10, 18 mm; shutter speed = 1/320; exposure compensation = -1/3; image quality = highest, no RAW; drive mode = continuous; autofocus = AI SERVO; ISO = Auto, max3200; file numbering = Auto reset; image auto rotate = Off; time/date = UTC.</li>
		</ol>
		</li>
		<li>Action camera
		<ol>
			<li>Set to video mode or continuous shooting mode at the highest resolution and frame rate possible. 
			NOTE: The action camera can also be used in continuous mode as long as the frame rate is 2 images per second or greater.</li>
			<li>To reproduce the protocol in this example (see Table of Materials), use a waterproof action camera with the following settings: Video resolution = 4K (4:3 aspect ratio); frame rate = 30 fps. 
			NOTE: For action cameras, it may be easier to attach the line from the spool to the swimmer rather than to the camera. In this example, the line was attached to the swimmer&#39;s wrist via a small lanyard.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Spool rig (Figure 1A)
	<ol>
		<li>Ensure that the spool is of the appropriate size to hold the length of line needed for the survey site radius. 
		NOTE: The circumference of the spool controls the spacing of the spiral swim lines, and the length of the line determines the sample area. In this example, an ~8 inch (~20 cm) diameter spool was used for ~50 inch (~1.3 m) spacing of swim lines. See 9 for details.</li>
		<li>Select a spool rig with a flanged edge (for smoothly guiding the line on and off the spool) and attachment points for a handle and pole (to control height from substrate). Ensure that the spool rig is inherently negatively buoyant or made so with the addition of weights. 
		NOTE: In this example, polyvinyl chloride (PVC) pipes for the handle and pole were used, and the spool was 3D printed in polylactic acid plastic. However, the spool can be as simple as a large PVC pipe or any other round object with the desired diameter.
		<ol>
			<li>For frequent use and/or challenging field conditions, select a spool made of a more durable&#160;material such as aluminum.</li>
			<li>Make sure that the spool does not rotate on the pole or spin when in use.</li>
		</ol>
		</li>
		<li>Fix the line to the spool at one end and to a detachable clip at the other for connecting to the camera. 
		NOTE: The length of the line defines the radius of the site. Here, 6 m of line was used for sites of 12 m in diameter.</li>
	</ol>
	</li>
	<li>Calibration tiles
	<ol>
		<li>Although specialized calibration tiles are not necessary, ensure that negatively buoyant, recognizable objects of known size are included in the model for scale. Consider surge and current conditions to ensure that suitable materials are used, so that tiles remain stationary during photo collection. 
		NOTE: Here, scale marker templates available as part of certain software programs were printed on waterproof paper, which was attached to 1-inch-thick PVC tiles.</li>
		<li>Divers will need a means to measure the depth of the tile. In our example, we use an electronic depth gauge (see Table of Materials).</li>
	</ol>
	</li>
	<li>Color correction
	<ol>
		<li>Set white balance on the camera to custom. Take a photo of an 18% grey card or white dive slate underwater before the start of every SfM survey. Do this every time a new site is started. 
		NOTE: The photo will allow for color correction and will also help to separate the downloaded images from different sites when conducting multiple surveys on the same day.</li>
	</ol>
	</li>
</ol>
2. Detailed methods
<ol>
	<li>Site selection
	<ol>
		<li>Select a site that has enough room to swim the entirety of the spiral pattern (~113 m2 in this example). In addition to the area being surveyed, incorporate a small buffer area to ensure that the entire survey area is sufficiently photographed to yield high-quality data.</li>
		<li>Consider the ability and equipment of the two-person team. Shallow sites (&#60; ~2 m) can be surveyed on snorkel, whereas deeper sites may require SCUBA.</li>
	</ol>
	</li>
	<li>If planning to repeatedly survey the site regularly, mark the center point, where the spool rig will be placed, with a tag or a permanent structure (e.g., rebar or cinder block). At the very least, take a global positioning system coordinate so that the site can be relocated with assistance from a printout of the orthomosaic. 
	NOTE: Permanent underwater structures typically require a permit.</li>
	<li>Prepare the site.
	<ol>
		<li>Set the spool in the middle of the site.</li>
		<li>Set out calibration tiles and record their depths. Place calibration tiles face-up, ~2 m away from the center. 
		NOTE: In this example, 3 calibration tiles were placed in a triangle around the center of the site. Calibration tiles should be appropriately weighted and positioned to ensure minimal movement during the collection of the photos.</li>
	</ol>
	</li>
	<li>Instruct the swimmer to swim with the camera while the assistant tends the spool.
	<ol>
		<li>The assistant sets the pole and the attached spool upright in the center of the selected site and holds the spool rig upright and stationary.</li>
		<li>Ensure that the swimmer attaches the side of the camera closest to the spool to the line and holds the camera facing straight down ~1 m from the benthos. 
		NOTE: If the swimmer must tilt the camera, try to make sure that it is tilted slightly forward rather than backwards to avoid collecting images in the swimmer&#39;s shadow. Tilting the camera slightly forward for both the outward spiral and the return spiral may also capture better angles of the benthos and produce better models, especially when there are overhangs and holes.</li>
		<li>Once the camera is properly positioned, the swimmer begins taking continuous images of the benthos while swimming forward and maintaining tension on the line.</li>
		<li>Ensure that the swimmer continues to swim in a spiral at a consistent speed while taking photographs until the line is completely unwound from the spool. 
		NOTE: The swimmer should try to stay a constant distance of ~ 1 m above the benthos and swim the spiral at a moderate pace to ensure sufficient overlap between images. When in doubt, slower is better.</li>
		<li>In highly rugose environments (e.g., coral reefs), include a third worker (second assistant) who can prevent line entanglement by hovering above the center of the line and gently lifting it over obstacles.</li>
		<li>When the line is completely unspooled, the swimmer reverses directions, reattaching the camera if necessary, and swims the camera in the opposite direction to begin re-winding the line back onto the spool while taking pictures. NOTE: Swimming the reverse spiral is not absolutely necessary, but will typically produce better models.</li>
		<li>If a single spiral method is desirable to save time, then the swimmer would detach the line from the camera and skip to step 2.4.12 while the assistant winds the line and removes the spool rig from the site.</li>
		<li>As soon as the swimmer begins to swim in the opposite direction the assistant rotates the spool to wind the line in &#189; of a turn (180&#176;) against the new swimming direction. This &#189; turn ensures that the swimmer&#8217;s return path is offset from the original path to yield greater photo coverage of the site.</li>
		<li>Ensure that the swimmer continues to take pictures and swim the reverse spiral until the line is almost completely rewound around the spool.</li>
		<li>When the swimmer&#8217;s and assistant&#8217;s spacing prevents further progress, the swimmer will then stop taking pictures to detach the camera from the line and allow the assistant to remove the spool rig from the center of the site.</li>
		<li>Once the spool is removed from the site, the swimmer images the center of the site by holding the camera facing straight down and moving the camera in a small spiral pattern over the center of the site.</li>
	</ol>
	</li>
</ol>
3. Clean up the site.
<ol>
	<li>Pick up calibration tiles and any other equipment before departing the site. 
	NOTE: Never leave trash or equipment at a site. Always leave a site cleaner than you found it.</li>
</ol>

<ol>
	<li>Levy, J., Hunter, C., Lukacazyk, T., Franklin, E. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessing+the+spatial+distribution+of+coral+bleaching+using+small+unmanned+aerial+systems.">Assessing the spatial distribution of coral bleaching using small unmanned aerial systems.</a> Coral Reefs. 37 (2), 373-387 (2018).</li><li>Muller-Karger, F. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Satellite+sensor+requirements+for+monitoring+essential+biodiversity+variables+of+coastal+ecosystems.">Satellite sensor requirements for monitoring essential biodiversity variables of coastal ecosystems.</a> Ecological Applications. 28 (3), 749-760 (2018).</li><li>Dornelas, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Towards+a+macroscope:+Leveraging+technology+to+transform+the+breadth,+scale+and+resolution+of+macroecological+data.">Towards a macroscope: Leveraging technology to transform the breadth, scale and resolution of macroecological data.</a> Global Ecology and Biogeography. 28, 1937-1948 (2019).</li><li>Burns, J. H. R., Delparte, D., Gates, R. D., Takabayashi, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Integrating+structure-from-motion+photogrammetry+with+geospatial+software+as+a+novel+technique+for+quantifying+3D+ecological+characteristics+of+coral+reefs.">Integrating structure-from-motion photogrammetry with geospatial software as a novel technique for quantifying 3D ecological characteristics of coral reefs.</a> PeerJ. 2015 (3), 1077 (2015).</li><li>House, J. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Moving+to+3D:+Relationships+between+coral+planar+area,+surface+area+and+volume.">Moving to 3D: Relationships between coral planar area, surface area and volume.</a> PeerJ. 2018 (6), 4280 (2018).</li><li>Carlot, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Community+composition+predicts+photogrammetry-based+structural+complexity+on+coral+reefs.">Community composition predicts photogrammetry-based structural complexity on coral reefs.</a> Coral Reefs. , 1-9 (2020).</li><li>Young, G. C., Dey, S., Rogers, A. D., Exton, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cost+and+time-effective+method+for+multi-scale+measures+of+rugosity,+fractal+dimension,+and+vector+dispersion+from+coral+reef+3D+models.">Cost and time-effective method for multi-scale measures of rugosity, fractal dimension, and vector dispersion from coral reef 3D models.</a> PloS ONE. 12 (4), 0175341 (2017).</li><li>Wilson, S. K., Robinson, J. P. W., Chong-Seng, K., Robinson, J., Graham, N. A. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Boom+and+bust+of+keystone+structure+on+coral+reefs.">Boom and bust of keystone structure on coral reefs.</a> Coral Reefs. 38 (4), 625-635 (2019).</li><li>Pizarro, O., Friedman, A., Bryson, M., Williams, S. B., Madin, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+simple,+fast,+and+repeatable+survey+method+for+underwater+visual+3D+benthic+mapping+and+monitoring.">A simple, fast, and repeatable survey method for underwater visual 3D benthic mapping and monitoring.</a> Ecology and Evolution. 7 (6), 1770-1782 (2017).</li><li>Torres-Pulliza, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+geometric+basis+for+surface+habitat+complexity+and+biodiversity.">A geometric basis for surface habitat complexity and biodiversity.</a> Nature Ecology &amp; Evolution. 4, 1495-1501 (2020).</li><li>Bayley, D., Mogg, A., Koldewey, H., Purvis, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Capturing+complexity:+field-testing+the+use+of+'structure+from+motion'derived+virtual+models+to+replicate+standard+measures+of+reef+physical+structure.">Capturing complexity: field-testing the use of 'structure from motion'derived virtual models to replicate standard measures of reef physical structure.</a> PeerJ. 2019 (7), 6540 (2019).</li><li>Leon, J. X., Roelfsema, C. M., Saunders, M. I., Phinn, S. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Measuring+coral+reef+terrain+roughness+using+'Structure-from-Motion'+close-range+photogrammetry.">Measuring coral reef terrain roughness using 'Structure-from-Motion' close-range photogrammetry.</a> Geomorphology. 242, 21-28 (2015).</li><li>Burns, J. H. R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessing+the+impact+of+acute+disturbances+on+the+structure+and+composition+of+a+coral+community+using+innovative+3D+reconstruction+techniques.">Assessing the impact of acute disturbances on the structure and composition of a coral community using innovative 3D reconstruction techniques.</a> Methods in Oceanography. 15-16, 49-59 (2016).</li><li>Edwards, C. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Large-area+imaging+reveals+biologically+driven+non-random+spatial+patterns+of+corals+at+a+remote+reef.">Large-area imaging reveals biologically driven non-random spatial patterns of corals at a remote reef.</a> Coral Reefs. 36, 1291-1305 (2017).</li><li>Piazza, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Underwater+photogrammetry+in+Antarctica:+long-term+observations+in+benthic+ecosystems+and+legacy+data+rescue.">Underwater photogrammetry in Antarctica: long-term observations in benthic ecosystems and legacy data rescue.</a> Polar Biology. 42 (6), 1061-1079 (2019).</li><li>Bonney, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Citizen+science:+A+developing+tool+for+expanding+science+knowledge+and+scientific+literacy.">Citizen science: A developing tool for expanding science knowledge and scientific literacy.</a> BioScience. 59 (11), 977-984 (2009).</li><li>Dickinson, J. L., Zuckerberg, B., Bonter, D. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Citizen+science+as+an+ecological+research+tool:+Challenges+and+benefits.">Citizen science as an ecological research tool: Challenges and benefits.</a> Annual Review of Ecology, Evolution, and Systematics. 41 (1), 149-172 (2010).</li><li>Storlazzi, C. D., Dartnell, P., Hatcher, G., Gibbs, A. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=End+of+the+chain?+Rugosity+and+fine-scale+bathymetry+from+existing+underwater+digital+imagery+using+structure-from-motion+(SfM)+technology.">End of the chain? Rugosity and fine-scale bathymetry from existing underwater digital imagery using structure-from-motion (SfM) technology.</a> Coral Reefs. 35 (3), 889-894 (2016).</li><li>Ventura, D., Lasinio, G. J., Belluscio, A., Ardizzone, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+low-cost+drone+based+application+for+identifying+and+mapping+of+coastal+fish+nursery+grounds+Feeding+ecology+View+project+Habitat+use+of+juvenile+Diplodus+species+View+project.">A low-cost drone based application for identifying and mapping of coastal fish nursery grounds Feeding ecology View project Habitat use of juvenile Diplodus species View project.</a> Estuarine Coastal and Shelf Science. 171, 85-98 (2016).</li><li>Ventura, D., Bonifazi, A., Gravina, M. F., Belluscio, A., Ardizzone, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mapping+and+classification+of+ecologically+sensitive+marine+habitats+using+unmanned+aerial+vehicle+(UAV)+imagery+and+object-based+image+analysis+(OBIA).">Mapping and classification of ecologically sensitive marine habitats using unmanned aerial vehicle (UAV) imagery and object-based image analysis (OBIA).</a> Remote Sensing. 10 (9), 1331 (2018).</li><li>George, E. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Relevance+of+coral+geometry+in+the+outcomes+of+the+coral-algal+benthic+war.">Relevance of coral geometry in the outcomes of the coral-algal benthic war.</a> bioRxiv. , (2018).</li><li>Anelli, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Towards+new+applications+of+underwater+photogrammetry+for+investigating+coral+reef+morphology+and+habitat+complexity+in+the+Myeik+Archipelago,+Myanmar.">Towards new applications of underwater photogrammetry for investigating coral reef morphology and habitat complexity in the Myeik Archipelago, Myanmar.</a> Geocarto International. 34 (5), 459-472 (2017).</li><li>Suka, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Processing+photomosaic+imagery+of+coral+reefs+using+structure-from-motion+standard+operating+procedures.">Processing photomosaic imagery of coral reefs using structure-from-motion standard operating procedures.</a> U.S. Dept. of Commerce, NOAA Technical Memorandum NOAA-TM-NMFS-PIFSC-93. , (2019).</li><li>Galtier d'Auriac, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Before+platelets:+the+production+of+platelet-activating+factor+during+growth+and+stress+in+a+basal+marine+organism.">Before platelets: the production of platelet-activating factor during growth and stress in a basal marine organism.</a> Proceedings of the Royal Society B: Biological Sciences. 285 (1884), 20181307 (2018).</li><li>Bouslimani, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Molecular+cartography+of+the+human+skin+surface+in+3D.">Molecular cartography of the human skin surface in 3D.</a> Proceedings of the National Academy of Sciences of the United States of America. 112 (17), 2120-2129 (2015).</li><li>Protsyuk, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3D+molecular+cartography+using+LC-MS+facilitated+by+Optimus+and+'ili+software.">3D molecular cartography using LC-MS facilitated by Optimus and 'ili software.</a> Nature Protocols. 13 (1), 134-154 (2018).</li></ol>

The authors have no competing financial interests or other conflicts of interest.

This study demonstrates that both the DSLR camera and the action camera produce models with better than 0.5 mm/pixel resolution in less than 10 h of processing time on a standard desktop computer. The major tradeoff between the DSLR and action camera, aside from cost, is finer resolution versus faster processing time, respectively. However, the reported processing times only include the computational processing. Thus, although the computational time is less for the action camera, there is a significant amount of time (10-20 min) invested in image extraction from the videos that is not required with the DSLR. An alternative is to use the action camera in continuous shooting mode to avoid image extraction. Continuous shooting mode was not used in this example, as the action camera can only shoot at 2 fps, which requires a significantly slower swim-rate to collect enough images to build a complete model. In this regard, there is a tradeoff between longer time in the field using the continuous shooting mode versus longer time on the computer, extracting images, when using video mode.
Advantages of the action camera include affordability and ease of transport and operation underwater. The main advantage of the DSLR is that it produces higher resolution images; hence, DSLR cameras are recommended over action cameras when the former is not cost-prohibitive. The kinds of questions a study seeks to address will also be important in determining the method used. For instance, an action camera might be preferable in environments that are relatively homogenous (e.g., seagrass beds, dead coral/rubble habitats), or where only broad community metrics (such as abundance, diversity) are being assessed over large spatial scales. However, a DSLR camera might be deployed in cases where tracking fine-scale changes in individual organisms or substrates is of interest.
As this is a field method, the model outputs will depend on various environmental factors such as lighting, water clarity, surface conditions, amount of surge, and movement of fish or non-stationary benthic structures (e.g., sea grass). Although there are no absolute thresholds of when it is appropriate to use this method, slightly overcast days with high water clarity, calm surface conditions, and little surge typically produce the best models. Moreover, there is a limit to the minimum depth required for these methods. These methods do not work well under conditions where there is less than 0.5 m of water because of the low overlap between photos and fewer distinguishing features per photo. However, this does highlight another advantage of the action camera, i.e., they are smaller and thus are easier for use at shallower depths. Furthermore, a smaller diameter spool and higher frame rate (or wider-angle lens) can improve image overlap in very shallow conditions9.
Many other data types can be integrated with this approach. For example, orthomosaics have been used to show the spatial density of molecular data (e.g., genes and metabolites) on corals24 and humans25 using the open source software &#39;ili&#39;26. The same platform could also be used to map the spatial densities of animals, microorganisms, viruses, and/or chemicals in the environment. Other examples have used SfM for annotating benthic species spatially onto orthomosaics using geographic information system software10. Furthermore, the 3D models generated by SfM can be used to estimate habitat characteristics such as rugosity and fractal dimension. Indeed, the methods outlined here were recently used to derive a new geometric theory for habitat surfaces10. Finally, orthomosaics are being used as input surfaces for spatially explicit computational models, allowing for dynamical simulations to be overlaid on the model&#39;s 3D surface. Being able to easily generate large images and 3D representations of benthic habitats has allowed marine scientists to address hitherto unimagined questions3.
Overall, here is a detailed protocol for conducting underwater SfM photogrammetry with either DSLR cameras or more cost-effective action cameras. These methods can be used by scientists for a broad range of purposes, from extracting data about benthic ecosystems to developing 3D input surfaces for in silico simulations. However, these protocols can also be used by non-scientists as part of citizen science efforts to gather valuable information on patterns of biodiversity, habitat complexity, community structure, and other ecological metrics.

Ecosystem processes are naturally dynamic and can be difficult to quantify. The past decade has seen a surge in new technologies for capturing ecosystems and their dynamics in a range of scales from 3D laser scanning of individual ecosystem features to satellite remote sensing of large areas1,2,3. In benthic habitats, structure is intimately connected with ecosystem function8, making tools that simultaneously allow for monitoring geometry and community structure especially valuable for understanding ecological dynamics. However, many modern approaches cannot be used in aquatic systems due to the physical properties of water (e.g., refraction, distortion, turbidity). Techniques, such as LiDAR (Light Detection and Ranging) and some aerial survey methods, may be appropriate on large spatial scales, but cannot acquire the resolution needed to assess fine scale changes in benthic habitats. Structure-from-Motion (SfM) photogrammetry methods have recently been adapted to produce large-scale, high-resolution orthomosaics and 3D surface models of underwater habitats4,5,6,7.
SfM photogrammetry is a relatively low-cost, simple, non-invasive, and repeatable method that allows for the generation of large-scale, high-resolution records of the benthic environment in aquatic ecosystems9. SfM uses a sequence of 2D images to generate 3D model reconstructions. The models generated from SfM can be used to collect data on the structural complexity (e.g., rugosity, dimensionality)4,5,10,11,12 and community structure (e.g., species composition, population demography)13,14,15 of benthic ecosystems. Furthermore, as this method is relatively inexpensive, quick, and repeatable, it can be used by both scientists and non-scientists to gather valuable, objective information on these ecosystems. Therefore, this method is a viable technique for use in citizen science projects where standardization of sampling effort, minimization of bias, engagement of participants, and ease of training are vital to the quality of data and overall success16,17.
This article provides a detailed protocol for conducting underwater SfM surveys. Simultaneously, the use of a DSLR camera has been compared with that of a more cost-effective &#39;action camera&#39;, and the relative advantages and disadvantages of each are outlined. The overall objective is to familiarize scientists and non-scientists with benthic SfM survey methods as rapidly as possible by providing a simple, commonly used protocol, in turn, promoting the use of this method more widely. &#160;For examples of studies that have applied variations of this method to study underwater ecological communities, see Burns et al. (2015)4, Storlazzi et al. (2016)18, Ventura et al (2016 and 2018)19,20, Edwards et al. (2017)14, George et al. (2018)21, Anelli et al. (2019)22, and Torres-Pulliza et al. (2020)10.
The method described here requires a two-person snorkel or SCUBA team. After the survey site is selected, a spool of line (Figure 1A) is placed at the center of the site, and calibration tiles (Figure 1B) are distributed ~2 m from the center. One person (the swimmer) swims with the camera and captures images of the site, while the second person (the assistant) tends the spool in the center of the plot (Figure 1C). First, the swimmer connects the camera to the spool via the line and then begins to take continuous pictures of the benthos while swimming face-down and forward to unwind the line off the spool. The swimmer should maintain a vertical distance of ~1 m above the substrate at all times, adjusting their position to match that of the topography as they swim. Importantly, the line connecting the spool and camera should remain taut at all times to create even spacing in the spiral as the swimmer surveys the plot. The assistant maintains the spool in a stable, upright position and ensures that the spool does not rotate, and that the line does not become tangled.
Once the line has been completely unwound, the swimmer stops, turns, and swims in the opposite direction to recoil the line around the spool. As the swimmer switches directions, the assistant turns the spool to wind the line in, exactly 180&#176; to prevent exact overlap of the outgoing path. Once the swimmer is as close to the center as possible, the camera is detached from the line, and the assistant takes the spool and line and swims away from the central portion of the site. The swimmer then finishes imaging the center of the plot by moving the camera in a small spiral over the center. While there are several ways to image an area effectively, the spool-and-line method described here is robust in even non-ideal environmental conditions where choppy surface waters, swell, or low visibility might otherwise impede data collection. In these scenarios, this method keeps snorkelers/divers attached and ensures high overlap of images by keeping the swimmer on a controlled path.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Action camera (GoPro Hero7 Black)</td>
			<td>GoPro</td>
			<td></td>
			<td>Could be any waterproof action camera</td>
		</tr>
		<tr>
			<td>Adobe Lightroom</td>
			<td>Adobe</td>
			<td></td>
			<td>Color correction</td>
		</tr>
		<tr>
			<td>Calibration tiles ( flat PVC board cut to size for Agisoft targets. Attach a dive weight underneath if expecting waves)</td>
			<td></td>
			<td></td>
			<td>Any negatively buoyant object of known size and color. We recommend using the scale marker templates available from Agisoft Metashape software (v.1.6.0).</td>
		</tr>
		<tr>
			<td>DSLR camera (Canon EOS Rebel SL3 )</td>
			<td>Canon</td>
			<td>3453C002AA</td>
			<td>Could be any DSLR camera in a underwater housing</td>
		</tr>
		<tr>
			<td>Line (plastic clothes line filament)</td>
			<td></td>
			<td></td>
			<td>Any negatively buoyant line that is strong enough to withstand field use</td>
		</tr>
		<tr>
			<td>Micro SDXC memory card (for GoPro)</td>
			<td></td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Oceanic Veo 2.0</td>
			<td>Oceanic</td>
			<td></td>
			<td>Digital depth gauge</td>
		</tr>
		<tr>
			<td>SDXC memory card (for DSLR)</td>
			<td></td>
			<td></td>
			<td>Any SDXC memory card should work, so long as there is enough space to hold all the pictures necessary to build the model&#160;</td>
		</tr>
		<tr>
			<td>Spool (2 inch-long section of 8 inch diameter PVC pipe which was attached to a 3 feet section of 1 inch PVC pipe to form the stem</td>
			<td></td>
			<td></td>
			<td>Any negatively buoyant, round object of the desired diameter</td>
		</tr>
		<tr>
			<td>Underwater camera housing for DSLR (Ikelite 200DLM/C Underwater TTL Housing)</td>
			<td>Ikelite</td>
			<td>6970.09</td>
			<td>Should be the specific water housing for the DSLR make and model</td>
		</tr>
		<tr>
			<td>Windows 10 desktop computer with an Intel i9-9900K 8-core CPU, two Nvidia GeForceRTX 2070 SUPER GPUs, and 128 GB of RAM.&#160;</td>
			<td></td>
			<td></td>
			<td>Processing&#160;</td>
		</tr>
	</tbody>
</table>

a field primer for monitoring benthic ecosystems using structure-from-motion photogrammetry

Structure-from-motion (SfM) photogrammetry is a technique used to generate three-dimensional (3D) reconstructions from a sequence of two-dimensional (2D) images. SfM methods are becoming increasingly popular as a noninvasive way to monitor many systems, including anthropogenic and natural landscapes, geologic structures, and both terrestrial and aquatic ecosystems. Here, a detailed protocol is provided for collecting SfM imagery to generate 3D models of benthic habitats. Additionally, the cost, time efficiency, and output quality of employing a Digital Single Lens Reflex (DSLR) camera versus a less expensive action camera have been compared. A tradeoff between computational time and resolution was observed, with the DSLR camera producing models with more than twice the resolution, but taking approximately 1.4-times longer to produce than the action camera. This primer aims to provide a thorough description of the steps necessary to collect SfM data in benthic habitats for those who are unfamiliar with the technique as well as for those already using similar methods.

In this example, Reef Site 2&#173;&#173;_7 located on Patch Reef 13 in K&#257;ne&#699;ohe Bay, O&#699;ahu, Hawai&#699;i, was imaged, and 3,125 JPEG photos from the DSLR and 3,125 JPEG frame captures from the action camera video (Table 1) were used as input to create the orthomosaics and 3D models. The general workflow consisted of 5 stages: 1) alignment of photos to generate the sparse point cloud, 2) scaling the sparse point cloud and optimizing cameras, 3) building the dense point cloud (depth maps were also generated during this stage), 4) building the digital elevation model (DEM) and orthomosaic, and 5) generating the 3D model and texture. Note that stages 4 and 5 do not necessarily need to be done in that order, but they must be performed after processing the dense point cloud and depth maps. Georeferencing the models should occur prior to generating the orthomosaic and DEM. The settings used for these stages and processing details are outlined in Table 2 and Table of Materials, respectively.
For more detailed methods of how to generate 3D models and orthomosaics see the Supplementary Material and Suka et al.23. Processing time was shorter for the action camera-derived model for every step including sparse point cloud generation, dense point cloud generation, mesh model rendering, and textured model rendering. This led to a significantly faster overall processing time for the action camera model (6 h 39 min) than the DSLR model (9 h 14 min). The exact time for model processing will vary with computational power and specific hardware configurations.
The model generated using images from the DSLR camera contained 2,848,358 sparse cloud points and 787,450,347 dense cloud points while the model generated from the action camera images contained only 2,630,543 sparse cloud points and 225,835,648 dense cloud points. This led to the DSLR models having more than 2x the resolution than the action camera models with orthomosaics resolutions of 0.442 and 0.208 mm/pixel for the DSLR- and action camera-derived models, respectively (Table 1). Despite the better resolution of the DSLR model relative to the action camera model, both methods were able to produce high-quality models with little difference in visual representation when the ~113 m2 reef area was represented as a 20 cm2 digital elevation model (Figure 2 top panels) or 2D orthomosaic projection (Figure 2 middle panels).
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/61815/61815fig01.jpg" /> 
Figure 1: Structure-from-Motion photogrammetry. (A) Example of a spool rig for controlling swimmer distance with an attached handle and pole for precise positioning and handling. (B) Calibration tiles. (C) A schematic of the swim path with relative positions of the swimmer (green) and the assistant (orange). <a href="https://www.jove.com/files/ftp_upload/61815/61815fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/61815/61815fig02.jpg" /> 
Figure 2: Visual comparison of digital elevation models and orthomosaics. Digital elevation models (top) and orthomosaics (middle) constructed from DSLR (left) and action camera (right) images. The bottom panel is a zoom of the areas in the white boxes in the orthomosaics. The heatmap scales in the top panel represent distance from the surface of the water in meters (m). <a href="https://www.jove.com/files/ftp_upload/61815/61815fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td></td>
			<td>Canon EOS Rebel SL3</td>
			<td>GoPro Hero 7</td>
		</tr>
		<tr>
			<td>Cost</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Camera</td>
			<td>~$600.00</td>
			<td>~$220.00</td>
		</tr>
		<tr>
			<td>Underwater housing</td>
			<td>~$1,700.00</td>
			<td>NA</td>
		</tr>
		<tr>
			<td>Total Cost</td>
			<td>~$2,300.00</td>
			<td>~$220.00</td>
		</tr>
		<tr>
			<td>Photos</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Photo file format</td>
			<td>jpeg</td>
			<td>jpeg</td>
		</tr>
		<tr>
			<td>Photo resolution</td>
			<td>24 Megapixels</td>
			<td>12 Megapixels (from 4K video)</td>
		</tr>
		<tr>
			<td>Aligned photos / total photos</td>
			<td>3125 / 3125</td>
			<td>3125 / 3125</td>
		</tr>
		<tr>
			<td>Photogrammetry metrics</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Sparse cloud points</td>
			<td>2,848,358</td>
			<td>2,630,543</td>
		</tr>
		<tr>
			<td>Dense cloud points</td>
			<td>787,450,347</td>
			<td>225,835,648</td>
		</tr>
		<tr>
			<td>Faces (3D model)</td>
			<td>11,919,451</td>
			<td>3,834,651</td>
		</tr>
		<tr>
			<td>Digital elevation model (DEM) resolution</td>
			<td>0.831 mm/pixel</td>
			<td>1.77 mm/pixel</td>
		</tr>
		<tr>
			<td>Orthomosaic resolution</td>
			<td>0.208 mm/pixel</td>
			<td>0.442 mm/pixel</td>
		</tr>
		<tr>
			<td>Processing times</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Sparse cloud generation</td>
			<td>1 h 23 min</td>
			<td>1 h 27 min</td>
		</tr>
		<tr>
			<td>Dense cloud generation</td>
			<td>4 h</td>
			<td>3 h 11 min</td>
		</tr>
		<tr>
			<td>Mesh model rendering</td>
			<td>3 h 32 min</td>
			<td>1 h 49 min</td>
		</tr>
		<tr>
			<td>Texture rendering</td>
			<td>19 min</td>
			<td>12 min</td>
		</tr>
		<tr>
			<td>Total computer processing time</td>
			<td>9 h 14 min</td>
			<td>6 h 39 min</td>
		</tr>
	</tbody>
</table>
Table 1: Detailed information about setup cost, photos used to construct the models, photogrammetry metrics, and processing time. Processing was done using the same settings for both models. Note that processing time does not include time for various steps such as photo editing, extracting images from video, re-aligning photos, and editing and scaling the models.
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td></td>
			<td>Canon EOS Rebel SL3</td>
			<td>GoPro Hero 7</td>
		</tr>
		<tr>
			<td>Images</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Average file size</td>
			<td>~ 8.3 MB</td>
			<td>~ 4.7 MB</td>
		</tr>
		<tr>
			<td>Photo acquisition</td>
			<td>Continuous mode</td>
			<td>Extracted from 4K video</td>
		</tr>
		<tr>
			<td>Color correction</td>
			<td>Manual</td>
			<td>Manual</td>
		</tr>
		<tr>
			<td>Lens correction</td>
			<td>No</td>
			<td>Yes</td>
		</tr>
		<tr>
			<td>Photogrammetry Process Settings</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Sparse cloud generation</td>
			<td>Accuracy: High</td>
			<td>Accuracy: High</td>
		</tr>
		<tr>
			<td></td>
			<td>Key Point: 40,000</td>
			<td>Key Point: 40,000</td>
		</tr>
		<tr>
			<td></td>
			<td>Tie Point: 4,000</td>
			<td>Tie Point: 4,000</td>
		</tr>
		<tr>
			<td></td>
			<td>Generic Preselection: Yes</td>
			<td>Generic Preselection: Yes</td>
		</tr>
		<tr>
			<td>Dense cloud generation</td>
			<td>Medium Quality</td>
			<td>Medium Quality</td>
		</tr>
		<tr>
			<td>3D mesh model generation</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Source data:</td>
			<td>Depth Maps</td>
			<td>Depth Maps</td>
		</tr>
		<tr>
			<td>Quality:</td>
			<td>Medium</td>
			<td>Medium</td>
		</tr>
		<tr>
			<td>Face count:</td>
			<td>Low</td>
			<td>Low</td>
		</tr>
		<tr>
			<td>Interpolation:</td>
			<td>Enabled</td>
			<td>Enabled</td>
		</tr>
		<tr>
			<td>Calculate vertex colors:</td>
			<td>Yes</td>
			<td>Yes</td>
		</tr>
		<tr>
			<td>3D texture generation</td>
			<td rowspan="1"></td>
			<td rowspan="1"></td>
		</tr>
		<tr>
			<td>Texture type:</td>
			<td>Diffuse Map</td>
			<td>Diffuse Map</td>
		</tr>
		<tr>
			<td>Source data:&#160;</td>
			<td>Images</td>
			<td>Images</td>
		</tr>
		<tr>
			<td>Mapping mode:</td>
			<td>Generic</td>
			<td>Generic</td>
		</tr>
		<tr>
			<td>Blending mode:</td>
			<td>Mosaic</td>
			<td>Mosaic</td>
		</tr>
		<tr>
			<td>Texture size/count:</td>
			<td>4096 / 1</td>
			<td>4096 / 1</td>
		</tr>
		<tr>
			<td>Digital elevation model (DEM)</td>
			<td>From Dense cloud</td>
			<td>From Dense Cloud</td>
		</tr>
		<tr>
			<td>Orthomosaic</td>
			<td>From DEM</td>
			<td>From DEM</td>
		</tr>
	</tbody>
</table>
Table 2: Detailed information on collected images and photogrammetric processing. Processing was done using the same settings for both models.
Supplementary Material.&#160;&#160;<a href="https://www.jove.com/files/ftp_upload/61815/JoVE_SI_Roach.docx" target="_blank">Please click here to download this file.</a>

Watch this Scientific Journal Video about A Field Primer for Monitoring Benthic Ecosystems Using Structure-From-Motion Photogrammetry at JoVE.com

A Field Primer for Monitoring Benthic Ecosystems Using Structure-From-Motion Photogrammetry

We provide a detailed protocol for conducting underwater structure-from-motion photogrammetry surveys to generate 3D models and orthomosaics.

Structure-from-motion (SfM) photogrammetry is a technique used to generate three-dimensional (3D) reconstructions from a sequence of two-dimensional (2D) ...

Research

JoVE Journal

Environment

The Benthic Exchange of O2, N2 and Dissolved Nutrients Using Small Core Incubations

The Benthic Exchange of O2, N2 and Dissolved Nutrients Using Small Core Incubations

The measurement of sediment-water exchange of gases and solutes in aquatic sediments provides data valuable for understanding the role of sediments in ...

Modification and Application of a Leaf Blower-vac for Field Sampling of Arthropods

Rice fields host a large diversity of arthropods, but investigating their population dynamics and interactions is challenging. Here we describe the ...

Reliable Method for Assessing Seed Germination, Dormancy, and Mortality under Field Conditions

We describe techniques for approximating seed bank dynamics over time using Helianthus annuus as an example study species. Strips of permeable polyester ...

Field Experiments of Pollination Ecology: The Case of Lycoris sanguinea var. sanguinea

Field Experiments of Pollination Ecology: The Case of Lycoris sanguinea var. sanguinea

Plant-pollinator interactions have been studied for approximately one hundred years. During that time, many field methods have been developed to clarify ...

Methodology for Developing Life Tables for Sessile Insects in the Field Using the Whitefly, Bemisia tabaci, in Cotton As a Model System

Methodology for Developing Life Tables for Sessile Insects in the Field Using the Whitefly, Bemisia tabaci, in Cotton As a Model System

Life tables provide a means of measuring the schedules of birth and death from populations over time. They also can be used to quantify the sources and ...

Ecosystem Fabrication (EcoFAB) Protocols for The Construction of Laboratory Ecosystems Designed to Study Plant-microbe Interactions

Ecosystem Fabrication EcoFAB Protocols for The Construction of Laboratory Ecosystems Designed to Study Plant-microbe Interactions

Beneficial plant-microbe interactions offer a sustainable biological solution with the potential to boost low-input food and bioenergy production. A ...

The Hawaii Protocol for Scientific Monitoring of Coffee Berry Borer: a Model for Coffee Agroecosystems Worldwide

Coffee berry borer (CBB) is the most devastating insect pest for coffee crops worldwide. We developed a scientific monitoring protocol that is aimed at ...

Methods for Image-based Surveys of Benthic Macroinvertebrates and Their Habitat Exemplified by the Drop Camera Survey for the Atlantic Sea Scallop

Underwater imaging has long been used in the field of marine ecology but decreasing costs of high-resolution cameras and data storage have made the ...

Construction of a Low-cost Mobile Incubator for Field and Laboratory Use

Incubators are essential for a range of culture-based microbial methods, such as membrane filtration followed by cultivation for assessing drinking water ...