The Evaluation of Photogrammetry-Based DSM from Low-Cost UAV by LiDAR-Based DSM

(1) University of Zagreb, Faculty of Geodesy, Chair of Photogrammetry and Remote Sensing, Kačićeva 26, HR-10000 Zagreb, Croatia; (2) University of Zagreb, Faculty of Forestry, Department of Forest Inventory and Management, Svetošimunska 25, HR-10000 Zagreb, Croatia; (3) Oikon Ltd. Institute of Applied Ecology, Department of Natural Resources Management, Trg Senjskih Uskoka 1-2, HR-10000 Zagreb, Croatia; (4) Croatian Forest Research Institute, Division for Forest Management and Forestry Economics, Trnjanska cesta 35, HR-10000 Zagreb, Croatia

of the tasks. Furthermore, UAVs have a capability of an autonomous recording, and hence they are becoming independent devices for gathering a large number of highquality data of the field or object with appropriate accuracy.
Recently, comprehensive reviews on applications of UAVs in forestry have been provided by Tang and Shao [2] and Torresan et al. [3]. In general, the common UAVs applications in forestry are related to monitoring of forest health and disturbances [4][5][6], forest inventory [7,8], forest cover mapping [9], etc. Digital surface model (DSM), which is one of the main photogrammetric products of UAV surveys, has great application in forest inventory. By subtracting available digital terrain model (DTM), which presents terrain surface, from DSM, which presents forest surface, a canopy height model (CHM) is generated. DTMs are nowadays commonly generated using airborne laser scanning (ALS) technology based on light detection and ranging (LiDAR) or airborne digital photogrammetry [10]. From CHMs various metrics can be derived which are then used for estimation of various tree [11] and stand variables [7,12]. The Structure from Motion (SfM) algorithm has been suggested for DSM generation by many authors [13][14][15]. Camera calibration and image phototriangulation process are initially performed to generate accurate DSM or digital terrain model (DTM) [16]. Camera calibration method and the algorithm for the precise elimination of lens distortion on digital cameras was developed by Gašparović and Gajski [17]. Continuing the research Gašparović and Gajski [18] presented a new method of two-step camera calibration for micro UAVs.
Methods for producing photogrammetric DSMs without using ground control points (GCPs) were presented in several studies [19][20][21]. To obtain external orientation parameters, Chikhradze [19] used single-frequency Global Navigation Satellite Systems (GNSS) receivers, while Vander et al. [20] and Fazeli et al. [21] used dual-frequency differential GNSS. Furthermore, Gimbal influence on the stability of exterior orientation parameters of UAV images was examined in study by Gašparović and Jurjević [22].
The DSMs generated from airborne digital stereo images were evaluated in many studies [e.g. [23][24][25] which revealed that many factors may influence on their quality, especially in complex forest structure. The research on DSM quality obtained from UAV images are still lacking (especially in South-east European region), but it can be assumed that apart from technical characteristics related to UAV (e.g. camera quality, GNSS precision) similar factors (e.g. image quality, algorithm for image processing, weather conditions, forest structure, etc.) are present.
The main goal of this research is to evaluate the quality of photogrammetry-based DSM from low-cost UAV's images collected in non-optimal weather (windy and cloudy weather) and environmental (inaccessibility for regular spatial distribution of GCPs) conditions. Namely, urgent cases (e.g. detection and estimation of forest damage) sometimes require rapid and immediate reaction when data acquisitions have to be done in non-optimal weather conditions during the survey. Furthermore, in dense forests it is very difficult to find a place for GCPs, especially to obtain the regular spatial distribution of GCPs which will provide the most accurate orientation of images. Therefore, vertical agreement assessment of UAV-based DSMs generated without and with using GCPs was evaluated with LiDARbased DSM in this study.

Study Area
The research was conducted in the lowland forest complex of Pokupsko Basin located 35 km southwest of Zagreb, Central Croatia ( Figure 1). The study area (77.39 ha) encompasses two 45-year-old mixed forest stands (subcompartments 36a and 37a, management unit "Jastrebarski lugovi") dominated by pedunculate oak (Quercus robur L.) accompanied by black alder (Alnus glutinosa (L.) Geartn.), common hornbeam (Carpinus betulus L.), and narrow-leaved ash (Fraxinus angustifolia Vahl.), and with the Corylus avellana L. and Crataegus monogyna Jacq. in the understorey. The study area is flat, with ground elevations ranging from 108 to 113 m a.s.l.

UAV-Based Canopy Digital Surface Models
The UAV images were acquired using the DJI Phantom 4 Pro UAV with FC6310 camera (Table 1) on 14 September 2017. The average flying height was 200 m above ground level. The study area was covered by 488 RGB images with the ground sampling distance (GSD) of approximately 5 cm. The images were collected in 11 flight lines with endlap of 90% and sidelap of 80%. Weather conditions during UAV survey were not suitable (non-optimal) due to windy and cloudy weather.
used. Tie-points on all images, as in the previous case, were automatically determined using SfM algorithm. Phototriangulation with self-calibration was based on image coordinates of tie-points and GCPs, and GCPs' coordinates in the terrestrial coordinate system. A priori EOPs were not used in this case. A raster DSM (hereinafter referred to as DSM P-GCP ) with a spatial resolution of 0.5 m was generated from the point cloud obtained by automatic correlation of oriented images.
The whole procedure of image orientation and DSMs generation was performed using Agisoft PhotoScan software (version 1.2.6, 64 bit).

LiDAR-Based Canopy Digital Surface Model
A raster LiDAR-based DSM (hereinafter referred to as DSM L ) with a spatial resolution of 0.5 m was provided by Hrvatske vode Ltd. (Zagreb, Croatia). Table 2 provides an overview of LiDAR sensor and data characteristics used for DSM L generation. The resulting point densities (11.59 points·m -2 ) and the stated horizontal (0.15 m) and vertical (0.08 m) accuracies were based on a considerably larger area (which included and non-forested areas as well) than the one considered in this study. DSM L was generated from returns classified as "first return" and "only return". DSM L was used as reference data for vertical agreement assessment of UAV-based DSMs (DSM P and DSM P-GCP ). Due to its high accuracy, the LiDAR data were often used as reference data for evaluation of UAV data [27][28][29].
Before the UAV survey, seven ground control points (GCPs) were placed and measured in the study area ( Figure  1). The GCPs' positions (x, y, z coordinates) were measured using the Trimble GNSS receiver connected with the Croatian Positioning System (CROPOS) which enables to obtain both horizontal and vertical positional accuracy from 2 to 5 cm (CROPOS -Users' Manual). Due to dense forest and mostly invisible ground from the air, it was not possible to provide (set up) the regular spatial distribution of GCPs over the entire study area which enables the most accurate orientation of images [26]. Therefore, GCPs were set up and measured on the forest roads from where they can be easily detected on UAV images (Figure 1).
From the collected UAV images, two DSMs were generated. First DSM was generated without using GCPs. This means that DSM was generated from UAV images whose orientation was based on a priori exterior orientation parameters (EOPs) only. A priori EOPs were measured during flight in metadata files of each image by GNSS. Firstly, tie-points on all images were automatically determined using the Structure from Motion (SfM) algorithm. Image coordinates of tie-points and a priori EOPs were then used for photo-triangulation with self-calibration. By automatic correlation of oriented images, the point cloud was obtained and then used to generate raster DSM (hereinafter referred to as DSM P ) with a spatial resolution of 0.5 m.
To generate the second DSM, the classic image phototriangulation method based on tie-points and GCPs was

Vertical Agreement Assessment
The vertical agreement assessment of the UAVbased DSMs was conducted by comparing elevations of 60 checkpoints of a regular 100 m grid obtained from DSM L with the elevations of planimetrically corresponding points obtained from DSM P and DSM P-GCP . Prior to defining measures for agreement assessment, the normality of residuals (vertical errors between UAV-and LiDAR-based DSMs) distribution was analyzed using: (a) histograms with

Digital camera fC6310
Sensor type CMOS  [30,31], and (c) normal Q-Q plots ( Figure  2). All performed tests revealed non-normal distribution of vertical errors for both UAV-based DSMs. Consequently, the following robust measures suggested by Höhle and Höhle [10] were used for vertical agreement assessment: median, normalised median absolute deviation (NMAD), 68.3% quantile and 95% quantile. Additionally, root mean square errors before (RMSE) and after removing outliers (RMSE * ) were calculated. The equations for all measures, as well as for the threshold for outliers can be found in Höhle and Höhle [10]. The statistical analyses were performed using the STATISTICA software (version 11) [32] and R programming language (version 3.

3.3) [33].
To support statistical analyses, the visual assessment of UAV-and LiDAR-based DSMs, as well as the visual assessment of difference raster models was performed. Difference raster models were generated by subtracting LiDAR-based from UAV-based DSMs. Both, difference raster model generation and its visualization were conducted using Global Mapper (version 19) [34] and QGIS (version 2.18) [35] software.

RESULTS AnD DiSCUSSiOn
According to the described methods, DSM P ( Figure  3a) and DSM P-GCP (Figure 3b) were generated. Detailed information on DSMs processing is presented in Table 3. It can be seen that computer processing time for both DSMs is almost the same, whereas the time spent on manual work is considerably greater for DSM P-GCP generation (30 min) than for DSM P generation (10 min). Namely, during the DSM P-GCP generation, most of the time (≈20 min) was spent on the manual detection of the GCPs on images, while for the DSM P generation the UAV images were orientated without using GCPs.
The results of the vertical agreement assessment of the UAV-based DSMs (DSM P and DSM P-GCP ) with DSM L conducted on 60 checkpoints of the regular 100 m sample grid are shown in Table 4. When comparing UAV-based DSMs with DSM L , it is necessary to have in mind that between the acquisition of LiDAR and UAV data is a time gap of one year which corresponds with one vegetation and subsequently with annual height increment. According to the internal database (unpublished material) of Croatian Forest Research Institute, annual height increment for the forest of the study area ranges from 0.2 m to 0.45 m depending on tree species. As expected, DSM P-GCP shows higher accuracy, i.e. higher vertical agreement with DSM L than DSM P . Namely, the horizontal accuracy (RMSE XY ) of DSM P assessed with 7 GCPs (which were not used in its generation) is 5.67 m (Table 3). Since such horizontal errors may produce greater vertical errors [36], especially for surfaces with great variations in height on a small area (e.g. forest) [25,37], the lower vertical agreement of DSM P with  DSM L is understandable. This is especially evident in Figure  3c, which shows a comparison of DSMs' profiles through the exemplary area. By observing profiles at greater peaks, it can be seen that DSM P-GCP profile follows the DSM L profile, whereas for DSM P profile the horizontal displacement of 5-10 m compared to DSM L profile can be observed. The improvement in vertical agreement of UAV-based DSMs with DSM L when GCPs are used can be observed visually on difference models (Figure 4). Similarly, when comparing two DSMs derived from WorldView-2 images, Hobi and Ginzler [38] found clear improvement of the DSM's vertical accuracy when GCPs were used. Furthermore, Figure 3c shows that DSM L provides the highest discrimination of vertical forest structure clearly describing very steep variations in height (e.g. small gaps in the forest canopy, forest road). On the contrary, the profiles of both UAV-based DSMs are considerably smoother. Only bigger gaps in the forest canopy and a forest road ( Figure  3a and 3b) can be detected, but in both cases, the vertical profiles of UAV-based DSMs do not reach the ground elevations. This is reasonable because LiDAR is an active sensor whose beams can penetrate through smaller gaps in the forest canopy and reach the ground, whereas the digital camera of UAV system used in this research (Table 1) is a passive optical sensor whose signal can characterize only the canopy surface [39].
Besides the technical limitations of low-cost UAV (e.g. camera quality, GNSS precision) used in this study and nonregular spatial distribution of GCPs, it can be suggested that the weather conditions (windy and cloudy weather) during UAV survey influenced image quality to a certain extent and consequently DSMs quality. The uncertainties are larger due to the complexity of the forest environment (e.g. moving trees, occlusions, shadows, images radiometric quality, etc.), which seriously affect the image matching procedure, and thus DSM quality [23,25,40,41].

COnCLUSiOnS
This research confirmed great potential of images obtained by low-cost UAV for forestry applications, even if they are surveyed in non-optimal weather (windy and cloudy weather) and environmental (inaccessibility for regular spatial distribution of GCPs) conditions. This could be of importance for cases when urgent UAV surveys are needed (e.g. detection and estimation of forest damage) which do not allow careful and longer survey planning.    The vertical agreement assessment of UAV-based DSMs with LiDAR-based DSM confirmed the importance of GCPs for image orientation and DSM generation. Namely, a considerable improvement in vertical accuracy of UAVbased DSMs was observed when GCPs were used. While DSMs generated without GCPs can be used for visualisation and monitoring purposes, DSMs generated with GCPs have potential to be used in forest inventory. To confirm this, further research should focus on estimating the accuracy of tree and stand attributes.