2021 Virtual Undergraduate Research Symposium
2021 Virtual Undergraduate Research Symposium
Classifying Lithofacies of the Entrada Sandstone using Drone Photogrammetry and Machine Learning
Classifying Lithofacies of the Entrada Sandstone using Drone Photogrammetry and Machine Learning
HONORABLE MENTION
PROJECT NUMBER: 53 | AUTHOR: Michael Field, Geophysical Engineering
MENTOR: Zane Jobe, Geology and Geological Engineering
ABSTRACT
Developments in drone photogrammetry and structure from motion allow us to create high quality 3-dimensinal point clouds from sequences of drone photos. Point clouds have effectively classified features of the built environment, rock slope features, and many other targets. This research aims to continue this line of research by classifying the lithologies present in the Entrada sandstone using drone photogrammetry. The Entrada sandstone includes a clean, cross bedded sandstone we refer to as the dune facies, a laminated or featureless silty sandstone we refer as the interdune facies, and small bushes and shrubs. We used drone photos collected over the Entrada Sandstone formation to create a 3D point cloud, labeled dune and interdune facies and vegetation, and created multi-scale features for use in supervised machine learning. These multi-scale features included a variety of geometric, slope, texture, and absolute color features. We showed that these features can accurately classify the dune and interdune facies and vegetation in the Entrada sandstone with a 95% confidence interval of the mean accuracy falling between 0.967 and 0.971. We also showed that the standard deviation and mean of the LAB are the most important features in the Entrada, with a 95% confidence interval of the mean accuracy of models using those features falling between 0.945 and 0.95.
PRESENTATION
AUTHOR BIOGRAPHY
Michael is a junior studying Geophysical Engineering and minoring in Data Science. He has been conducting research regarding automated prediction of lithologies in outcrop using drone photogrammetry and machine learning. During this effort Michael showed that mapping of lithologies in outcrops using drone photogrammetry is viable and that texture and spectral features are the most important.
Nice job Michael!
I wonder if you considered machine learning clustering algorithms to help categorize the point cloud data? It would be interesting to see if they could be helpful in auto-labeling features to reduce the manual labor involved in that process.
Thank you for the comment. I did do a k-means clustering with the full feature set with k=3, corresponding to the 3 classes. One of the greatest challenges in this work is producing a fully generalizable model that can be used to rapidly classify any novel outcrop point cloud. It is likely that it will be too difficult to train a supervised model on enough formations to produce a model that can predict any arbitrary facies, but k-means or another unsupervised machine learning method could provide a semi-automated classification approach that could generalize to any outcrop.
Hi Michael, fantastic work. I’m really glad to see a successful demonstration of this analysis in another area.
I have a few questions:
What was the criteria that made you decide that the mean and standard deviation features were the most important?
How did you split the dataset into training and testing portions? This can have a major impact on the accuracy values, because you could have training and testing points with overlapping neighborhoods.
What’s the difference between the “1MS” and “MS” feature sets?
Did you try applying the RF to another area of the same outcrop and seeing (qualitatively) how it performs?
Do you know approximately how the k-means clustering results compared to the supervised results?
Did you try only using subsets of the color components, like only A and B? Sometimes that seems to improve results a bit.
Hi Luke, thank you for the great questions and for your advice along the way.
1) I should have mentioned that it was my initial hypothesis that the mean LAB color would be the most “important” feature. This was because the color of the two facies and the vegetation are all distinct from each other, while the surface of the outcrop has a pretty consistent slope and smoothness except for vegetation. I concluded that the mean and standard deviation of the LAB color (MS features) were the most important features from results like those shown in Figure 5. I chose permutations of feature combinations and trained models with those feature combinations and compared the performances of the models. My first interpretation from Figure 5 was that models trained only on mean and standard deviation of LAB without the geometric and slope features perform nearly as well as those trained on all the features. For better comparison I should have included a model trained only geometric and slope features in Figure 5, which produced mean accuracies around 0.6, significanlty lower than models trained on MS features. Also not shown in the poster were the Random Forest features importances extracted from the model, which similarly showed MS features as the most important and scale 3 (0.5 m) consistently higher than the smaller two scales. I did not include these feature importances because many of the features are highly correlated with each other which skew the results of permuting features.
2) First, I labelled about 40 million points in Cloud Compare and randomly sampled 3000 points from each class. I generated features for those 9000 points, creating my dataset. I then shuffled the dataset and split 2/3 into training and 1/3 into testing using sklearns train_test_split utility
3) Feature sets with a number at the beginning indicate that only the 1st, 2nd, or 3rd scale features are used to train the model. Feature sets with no number at the beginning means the model was trained with those features at all 3 scales. Scale 1 , 2, and 3 refer to 0.1 m, 0.3 m, and 0.5 m radius for neighbor searching, respectively.
4) Figure 7 shows bulk “test area” for qualitative assessment. I chose a portion of the outcrop and generated features for all the points shown. I used the 3GDMS model, or the geometric, slope, mean and stdev of color features at 0.5 m scale model trained on the random 9000 points to classify the test area. I did not export the locations of my randomly sampled points I used to train the model for visualization so I don’t know if there were any training points in this test area, but so few points were sampled out of the 40 million that it is unlikely there were training points. Qualitatively we can see that model mostly works, but with a lower accuracy than what is represented in Figure 5. This is because the model was trained and tested on my labelling, which was only done in high confidence areas. If I chose not to label the lower confidence labels according the model, we would likely see the lower right hand corner of Figure 7 unlabelled.
5) I did not finish my analysis of the k-means due to confusion about matching clusters to labels. I think this may have been because the three classes are only really separable in the color features as described above. I would like to investigate this further.
6) I did not try only using a subset of the LAB color. L and A were consistently more important features across all scales.
Great poster, Michael! Two questions:
(1) Fig. 3 shows a lot of unlabeled areas. Did you look into what is driving the label failure in these regions?
(2) The biggest impediments you mention are the time to build a ball tree and calculate features. Have you considered options for speeding those steps up? Even if you haven’t, where do you think the time gains will be for these steps?
Thank you for the questions, Matt!
1) Figures 2 and 3 show my manual labeling for the purpose of supervised learning. A large proportion of the outcrop remained unlabeled because of time constraints and only labeling the highest confidence points. I did not assign confidences to my labels. Figure 7 shows a classified test region where I should have left low confidence classifications from the model unlabeled. This probably would have left much of the lower right around the bush unlabeled. The lower right portion of the test area is an area where the dune and interdune facies mix. Deposition of the dune facies on top of the interdune facies can cause the inderdune facies to squish up into the dune facies, causing a phenomena known as injectite. Areas with injectite were left unlabeled by me and would likely be the lowest confidence classified points by the model.
2) The ball tree neighbor searching is a large step up from my first iteration, which did a brute force neighbor search that was very computationally expensive. To search for neighbors the tree must be built on the entire point cloud, including the unlabeled areas. This certainly takes time, I believe 30 minutes to an hour in this case. If you are planning on computing features for many points, this is a good price to pay to then be able to query the tree for neighbors. This step could have some speed up from tuning some of the parameters, or by using the KDtree instead of the Ball Tree. These can be found in the sklearn.neighbors module. The feature calculation could benefit greatly from parallelization where many threads could be used to compute features at different points. Brodu and Lague, 2012 pioneered the use of multi-scale geometric features I used. They recognized that this is computationally expensive and proposed sampling “core points” similar to how I sampled my points, and only computing the features for those core points. Then other points can search for the nearest core point and take the features from it. This is a why by which features can be propagated to more points while limiting computation.
Hello Michael,
I really enjoyed hearing about how computational modeling can be applied to field geology. I was wondering, what was the motivation behind studying the Entrada Sandstone specifically for this project? Is this location particularly friendly to drone captures, or useful to other geological studies?
Hello Michael,
I really enjoyed your presentation on how computational modeling can be applied to field geology. I was wondering, what was the motivation behind studying the Entrada Sandstone specifically? Is this region particularly friendly to good drone captures, or are these results applicable to other geological studies?
Hi Jack, I didn’t really have any personal motives behind studying the Entrada. The reasons for studying it were because the drone photos were available to me, and Mary Carr has expertise in the Entrada and eolian systems so she willing to help me understand the facies. The Entrada and other prominent formations in Utah outcrop nicely making them easy targets for drone photogrammetry. This outcrop in particular is very clean with little vegetation or soil, and relatively distinct facies. This outcrop is a good outcrop to demonstrate this technology for lithology prediction, and Nataly Chacon’s master’s thesis will explore more difficult geologic settings. You ask a good question about how is this actually useful for other geologic studies. This technology could improve access to outcrops and improve the speed of geologic mapping. This could play a role in mapping for resources or carbon sequestration. If you collect photogrammetry or LiDAR data, analysis of the outcrop can continue back in the computer lab where quantitative analyses of cracks, bedding, change detection, and other things can be done. Beyond those reasons, point cloud datasets provide great educational tools. Once the point clouds are generated they can be shared amongst researchers and educators to provide every geology student access to outcrops.
Hi Michael,
Great job on your poster. I think you did an awesome job explaining your research and I loved how colorful you made your poster.
Why did you select a random forest for this project? Since there are many different kinds of classification models, I’m curious why you selected RF specifically over logistic regression, etc. Was it to reduce overfitting from decision trees?
I chose RF mainly for the reason you mentioned. They have been proven in so many different ways and are designed to limit overfitting. I was confident that I could compare the performance of models with 10 features and with 60 features because RF is so robust.