Multivariate Multi-Step Long Short-Term Memory Neural Network for Simultaneous Stream-Water Variable Prediction
Abstract
:1. Introduction
2. Data and Methods
2.1. Study Area
2.2. Multivariate Exploratory Data Analysis (EDA)
2.3. Feature Engineering (FE)
2.4. Long Short-Term Memory (LSTM) Recurrent Neural
2.5. Model Evaluation and Improvement
3. Results and Discussion
3.1. Predicted and Observed SW Variables
3.2. Model Evaluation Matrices
3.3. Hyperparameters Optimization
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Uddin, M.G.; Nash, S.; Olbert, A.I. A Review of Water Quality Index Models and Their Use for Assessing Surface Water Quality. Ecol. Indic. 2021, 122, 107218. [Google Scholar] [CrossRef]
- Ni, X.; Parajuli, P.B.; Ouyang, Y.; Dash, P.; Siegert, C. Assessing Land Use Change Impact on Stream Discharge and Stream Water Quality in an Agricultural Watershed. Catena 2021, 198, 105055. [Google Scholar] [CrossRef]
- Simeonov, V.; Stratis, J.A.; Samara, C.; Zachariadis, G.; Voutsa, D.; Anthemidis, A.; Sofoniou, M.; Kouimtzis, T. Assessment of the Surface Water Quality in Northern Greece. Water Res. 2003, 37, 4119–4124. [Google Scholar] [CrossRef]
- Alam, R.Q.; Duti, B.M.; Hossain, M.M. Potential of water footprint assessment as a decision support tool for sustainable water allocation option in Dhaka city. In Proceedings of the 4th Annual Paper Meet and 1st Civil Engineering Congress, Dhaka, Bangladesh, 22–24 December 2011; Institution of Engineers (IEB): Dhaka, Bangladesh, 2011. [Google Scholar]
- Alam, R.Q.; Duti, B.M.; Monowar, M.M. Climate Change Impacts on Water Related Sectors of Dhaka City and Assessing the Plausible Adaptation Options. In Proceedings of the 2nd International Conference on Environmental Technology & Construction Engineering for Sustainable Development ICETCESD, Sylhet, Bangladesh, 10–12 March 2012; Shahjalal University of Science and Technology: Sylhet, Bangladesh, 2012. [Google Scholar]
- Duti, B.M.; Aurin, F.H.; Rahman, M.D.M. Changes Observed in the Historical Trend of Local Rainfall as a Climatic Factor and Its Effect on the Streamflow of the Turag and Buriganga River System. In Proceedings of the International Conference on Environmental Technology & Construction Engineering for Sustainable Deve, ICETCESD-2011, Rhodes, Greece, 8–10 October 2011; Shahjalal University of Science and Technology: Sylhet, Bangladesh, 2011. [Google Scholar]
- Islam, M.T.; Duti, B.M.; Hasan, M.R.; Rahman, M.A.; Sahabuddin, S.M. Groundwater Salinity Modelling in the South West Region of Bangladesh Considering Climate Change. J. Indian Soc. Coast. Agric. Res. 2020, 38, 76–84. [Google Scholar]
- Li, K.; Wang, L.; Li, Z.; ** by Integrating Surface and Subsurface Hydrology with River Hydrodynamics. J. Hydrol. 2019, 575, 1155–1177. [Google Scholar] [CrossRef]
- Belayneh, A.; Duti, B.M.; Mekuanent, F.; Biniyam, S.; Gebrehiwot, T.; Buruk, W. Effect of Different HRU Definition on Catchment Runoff Prediction and Climate Change Impact Investigation Using the SWAT Model in the Kleine Nete Basin, Belgium, 3rd ed.; Open Water Symposium: Addis Ababa, Ethiopia, 2015. [Google Scholar]
- Subramanya, K. Engineering Hydrology, 3rd ed.; Tata McGraw-Hill Publishing: New Delhi, India, 2008. [Google Scholar]
- Woznicki, S.A.; Baynes, J.; Panlasigui, S.; Mehaffey, M.; Neale, A. Development of a Spatially Complete Floodplain Map of the Conterminous United States Using Random Forest. Sci. Total Environ. 2019, 647, 942–953. [Google Scholar] [CrossRef]
- Horritt, M.S.; Bates, P.D. Effects of Spatial Resolution on a Raster Based Model of Flood Flow. J. Hydrol. 2001, 253, 239–249. [Google Scholar] [CrossRef]
- Duti, B.M.; Khan, M.; Tamanna, T.; Mukherjee, N.; Rashid, M.A. Vulnerability Assessment and Adaptation Technique for Climate Change Induced Drought in Bangladesh. In Proceedings of the 5th International Conference on Water & Flood Management (ICWFM-2015), Dhaka, Bangladesh, 6–8 March 2015. [Google Scholar]
- Barzegar, R.; Aalami, M.T.; Adamowski, J. Short-Term Water Quality Variable Prediction Using a Hybrid CNN–LSTM Deep Learning Model. Stoch. Environ. Res. Risk Assess. 2020, 34, 415–433. [Google Scholar] [CrossRef]
- Zhu, X.; Khosravi, M.; Vaferi, B.; Nait Amar, M.; Ghriga, M.A.; Mohammed, A.H. Application of Machine Learning Methods for Estimating and Comparing the Sulfur Dioxide Absorption Capacity of a Variety of Deep Eutectic Solvents. J. Clean. Prod. 2022, 363, 132465. [Google Scholar] [CrossRef]
- Sinshaw, T.A.; Surbeck, C.Q.; Yasarer, H.; Najjar, Y. Artificial Neural Network for Prediction of Total Nitrogen and Phosphorus in US Lakes. J. Environ. Eng. 2019, 145, 04019032. [Google Scholar] [CrossRef]
- Khosravi, M.; Arif, S.B.; Ghaseminejad, A.; Tohidi, H.; Shabanian, H. Performance Evaluation of Machine Learning Regressors for Estimating Real Estate House Prices. Preprints 2022. [Google Scholar] [CrossRef]
- Yazdan, M.M.S.; Khosravi, M.; Saki, S.; Mehedi, M.A.A. Forecasting Energy Consumption Time Series Using Recurrent Neural Network in Tensorflow. Preprints 2022. [Google Scholar] [CrossRef]
- Zhang, J.; Zhu, Y.; Zhang, X.; Ye, M.; Yang, J. Develo** a Long Short-Term Memory (LSTM) Based Model for Predicting Water Table Depth in Agricultural Areas. J. Hydrol. 2018, 561, 918–929. [Google Scholar] [CrossRef]
- Cannas, B.; Fanni, A.; See, L.; Sias, G. Data Preprocessing for River Flow Forecasting Using Neural Networks: Wavelet Transforms and Data Partitioning. Phys. Chem. Earth Parts ABC 2006, 31, 1164–1171. [Google Scholar] [CrossRef]
- Paoli, C.; Voyant, C.; Muselli, M.; Nivet, M.-L. Forecasting of Preprocessed Daily Solar Radiation Time Series Using Neural Networks. Sol. Energy 2010, 84, 2146–2160. [Google Scholar] [CrossRef] [Green Version]
- Khosravi, M.; Tabasi, S.; Hossam Eldien, H.; Motahari, M.R.; Alizadeh, S.M. Evaluation and Prediction of the Rock Static and Dynamic Parameters. J. Appl. Geophys. 2022, 199, 104581. [Google Scholar] [CrossRef]
- Abdollahzadeh, M.; Khosravi, M.; Hajipour Khire Masjidi, B.; Samimi Behbahan, A.; Bagherzadeh, A.; Shahkar, A.; Tat Shahdost, F. Estimating the Density of Deep Eutectic Solvents Applying Supervised Machine Learning Techniques. Sci. Rep. 2022, 12, 4954. [Google Scholar] [CrossRef]
- Kumar, R.; Yazdan, M.M.S.; Mehedi, M.A.A. Demystifying the Preventive Measures for Flooding from Groundwater Triggered by the Rise in Adjacent River Stage. Preprints 2022. [Google Scholar] [CrossRef]
- Sahoo, S.; Russo, T.A.; Elliott, J.; Foster, I. Machine Learning Algorithms for Modeling Groundwater Level Changes in Agricultural Regions of the U.S. Water Resour. Res. 2017, 53, 3878–3895. [Google Scholar] [CrossRef] [Green Version]
- Predicting Residential Energy Consumption Using CNN-LSTM Neural Networks—ScienceDirect. Available online: https://www.sciencedirect.com/science/article/pii/S0360544219311223 (accessed on 23 May 2022).
- Khosravi, M.; Ghoochani, S.; Nazemi, N. Deep Learning-Based Modeling of Daily Suspended Sediment Concentration and Discharge in Esopus Creek. Preprints 2023, 2023051167. [Google Scholar] [CrossRef]
- Sutskever, I.; Vinyals, O.; Le, Q.V. Sequence to Sequence Learning with Neural Networks. In Advances in Neural Information Processing Systems; Curran Associates, Inc.: Dutchess County, NY, USA, 2014; Volume 27. [Google Scholar]
- Sundermeyer, M.; Schlüter, R.; Ney, H. LSTM Neural Networks for Language Modeling. In Proceedings of the Interspeech 2012, ISCA’s 13th Annual Conference, Portland, OR, USA, 9–13 September 2012; pp. 194–197. [Google Scholar]
- Mikolov, T. Recurrent Neural Network Based Language Model. Available online: http://www.fit.vutbr.cz/research/groups/speech/servite/2010/rnnlm_mikolov.pdf (accessed on 16 April 2022).
- Akatu, W.; Khosravi, M.; Mehedi, M.A.A.; Mantey, J.; Tohidi, H.; Shabanian, H. Demystifying the Relationship Between River Discharge and Suspended Sediment Using Exploratory Analysis and Deep Neural Network Algorithms. Preprints 2022. [Google Scholar] [CrossRef]
- Khosravi, M.; Mehedi, M.A.A.; Baghalian, S.; Burns, M.; Welker, A.L.; Golub, M. Using Machine Learning to Improve Performance of a Low-Cost Real-Time Stormwater Control Measure. Preprints 2022. [Google Scholar] [CrossRef]
- USGS. Current Conditions for USGS 01463500 Delaware River at Trenton NJ. Available online: https://waterdata.usgs.gov/nwis/uv?01463500 (accessed on 28 May 2022).
- Beretta, L.; Santaniello, A. Nearest Neighbor Imputation Algorithms: A Critical Evaluation. BMC Med. Inform. Decis. Mak. 2016, 16, 74. [Google Scholar] [CrossRef] [Green Version]
- Feature Engineering and Deep Learning-Based Intrusion Detection Framework for Securing Edge IoT|SpringerLink. Available online: https://link.springer.com/article/10.1007/s11227-021-04250-0 (accessed on 17 May 2022).
- Zheng, A.; Casari, A. Feature Engineering for Machine Learning: Principles and Techniques for Data Scientists; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2018; ISBN 978-1-4919-5319-8. [Google Scholar]
- Talbot, C.; Thrane, E. Flexible and Accurate Evaluation of Gravitational-Wave Malmquist Bias with Machine Learning. Astrophys. J. 2022, 927, 76. [Google Scholar] [CrossRef]
- Karimi, M.; Khosravi, M.; Fathollahi, R.; Khandakar, A.; Vaferi, B. Determination of the Heat Capacity of Cellulosic Biosamples Employing Diverse Machine Learning Approaches. Energy Sci. Eng. 2022, 10, 1925–1939. [Google Scholar] [CrossRef]
- Rebala, G.; Ravi, A.; Churiwala, S. Machine Learning Definition and Basics. In An Introduction to Machine Learning; Rebala, G., Ravi, A., Churiwala, S., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 1–17. ISBN 978-3-030-15729-6. [Google Scholar]
- Bengio, Y.; Simard, P.; Frasconi, P. Learning Long-Term Dependencies with Gradient Descent Is Difficult. IEEE Trans. Neural Netw. 1994, 5, 157–166. [Google Scholar] [CrossRef]
- Kilinc, H.C.; Haznedar, B. A Hybrid Model for Streamflow Forecasting in the Basin of Euphrates. Water 2022, 14, 80. [Google Scholar] [CrossRef]
- Song, X.; Liu, Y.; Xue, L.; Wang, J.; Zhang, J.; Wang, J.; Jiang, L.; Cheng, Z. Time-Series Well Performance Prediction Based on Long Short-Term Memory (LSTM) Neural Network Model. J. Pet. Sci. Eng. 2020, 186, 106682. [Google Scholar] [CrossRef]
- Younger, A.S.; Hochreiter, S.; Conwell, P.R. Meta-Learning with Backpropagation. In Proceedings of the IJCNN’01, International Joint Conference on Neural Networks, Washington, DC, USA, 15–19 July 2001; Proceedings (Cat. No.01CH37222). Volume 3, pp. 2001–2006. [Google Scholar]
- Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997, 9, 1735–1780. [Google Scholar] [CrossRef]
- Staudemeyer, R.C.; Morris, E.R. Understanding LSTM—A Tutorial into Long Short-Term Memory Recurrent Neural Networks. ar**v 2019, ar**v:1909.09586. [Google Scholar]
- Tsang, G.; Deng, J.; **e, X. Recurrent Neural Networks for Financial Time-Series Modelling. In Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Bei**g, China, 20–24 August 2018; pp. 892–897. [Google Scholar]
- Maulik, R.; Egele, R.; Lusch, B.; Balaprakash, P. Recurrent Neural Network Architecture Search for Geophysical Emulation. ar**v 2020, ar**v:2004.10928. [Google Scholar]
- Keras: The Python Deep Learning API. Available online: https://keras.io/ (accessed on 28 May 2022).
- Mehedi, M.A.A.; Khosravi, M.; Yazdan, M.M.S.; Shabanian, H. Exploring Temporal Dynamics of River Discharge Using Univariate Long Short-Term Memory (LSTM) Recurrent Neural Network at East Branch of Delaware River. Hydrology 2022, 9, 202. [Google Scholar] [CrossRef]
- Ahmad, M.; Al Mehedi, M.A.; Yazdan, M.M.S.; Kumar, R. Development of Machine Learning Flood Model Using Artificial Neural Network (ANN) at Var River. Liquids 2022, 2, 147–160. [Google Scholar] [CrossRef]
- Gupta, H.V.; Kling, H. On Typical Range, Sensitivity, and Normalization of Mean Squared Error and Nash-Sutcliffe Efficiency Type Metrics. Water Resour. Res. 2011, 47, W10601. [Google Scholar] [CrossRef]
- Willmott, C.J.; Robeson, S.M.; Matsuura, K. A Refined Index of Model Performance. Int. J. Climatol. 2012, 32, 2088–2094. [Google Scholar] [CrossRef]
SW Parameters | Unit | Descriptions |
---|---|---|
Discharge | ft3/s | Quantity of stream flow |
Water Level | ft | Stream-water height/level at the gage location |
Temperature | °C | Sensor-recorded temperature in °C at the gage |
Dissolved Oxygen (DO) | mg/L | The amount oxygen dissolved in the SW. |
Turbidity | FNU | Measure of turbidity in Formazin Nephelometric Unit (FNU) |
pH | - | the acidity or alkalinity of a solution on a logarithmic scale |
Specific Conductance (SC) | μS/cm | Measure of the collective concentration of dissolved ions in solution |
Count | Mean | Std | Min | 25% | 50% | 75% | Max | |
---|---|---|---|---|---|---|---|---|
Discharge (ft3/s) | 255,066 | 13,265.43 | 10,657.91 | 2150 | 6240 | 10,800 | 16,100 | 150,000 |
Water Level (ft) | 255,066 | 9.98 | 1.47 | 7.8 | 8.89 | 9.73 | 10.73 | 20.76 |
Temperature (°C) | 255,066 | 13.35 | 4.43 | 0 | 12.02 | 13.58 | 15.01 | 31.30 |
pH | 255,066 | 7.90 | 0.208 | 6.6 | 7.00 | 8.23 | 9.16 | 9.71 |
SC (μS/cm) | 255,066 | 208.19 | 22.23 | 49 | 201.11 | 208.64 | 221.09 | 453 |
Turbidity (FNU) | 255,066 | 6.44 | 6.54 | 0.2 | 5.61 | 6.44 | 7.29 | 469 |
DO (mg/L) | 255,066 | 11.02 | 1.11 | 6 | 11.02 | 11.07 | 12.67 | 16.90 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Khosravi, M.; Duti, B.M.; Yazdan, M.M.S.; Ghoochani, S.; Nazemi, N.; Shabanian, H. Multivariate Multi-Step Long Short-Term Memory Neural Network for Simultaneous Stream-Water Variable Prediction. Eng 2023, 4, 1933-1950. https://doi.org/10.3390/eng4030109
Khosravi M, Duti BM, Yazdan MMS, Ghoochani S, Nazemi N, Shabanian H. Multivariate Multi-Step Long Short-Term Memory Neural Network for Simultaneous Stream-Water Variable Prediction. Eng. 2023; 4(3):1933-1950. https://doi.org/10.3390/eng4030109
Chicago/Turabian StyleKhosravi, Marzieh, Bushra Monowar Duti, Munshi Md Shafwat Yazdan, Shima Ghoochani, Neda Nazemi, and Hanieh Shabanian. 2023. "Multivariate Multi-Step Long Short-Term Memory Neural Network for Simultaneous Stream-Water Variable Prediction" Eng 4, no. 3: 1933-1950. https://doi.org/10.3390/eng4030109