Enhancing Weather Monitoring for Agriculture with Deep Learning: Anomaly Detection in East Java Using LSTM Autoencoder and OCSVM

Maulana Ahsan Fadillah; Yenni Angraini; Rahma Anisa

doi:10.15575/join.v10i1.1571

Authors

Maulana Ahsan Fadillah Study Program of Statistics and Data Science, IPB University, Indonesia
Yenni Angraini Study Program of Statistics and Data Science, IPB University, Indonesia
Rahma Anisa Study Program of Statistics and Data Science, IPB University, Indonesia

DOI:

https://doi.org/10.15575/join.v10i1.1571

Keywords:

Anomaly Detection, LSTM autoencoder, LSTM, OCSVM, Weather

Abstract

Agricultural productivity in East Java is under threat from unpredictable and harsh weather patterns, particularly rapid variations in sunlight length and rainfall intensity. These abnormalities can interrupt agricultural cycles, lower yields, and make farming communities more vulnerable to climatic calamities. However, current weather monitoring systems frequently fall short of detecting small anomalies in time series weather data that could serve as early warning signs of such disasters. This study seeks to close this gap by creating a robust anomaly detection methodology adapted to time-dependent weather variables important to agriculture. In this study, a hybrid model combining Long Short-Term Memory (LSTM) autoencoder and One-Class Support Vector Machine (OCSVM) is proposed. The LSTM autoencoder's structure reconstructs time series data and signifies anomalies through reconstruction errors (MSE), while OCSVM validates these anomalies to reduce false positives. The model was applied to daily weather data from East Java spanning 2015–2024. The results showed that the model effectively detected 11 anomalies in sunlight duration and 7 in rainfall, with F1-scores of 0.71 and 0.82, respectively. Several of these anomalies corresponded to actual disaster events such as floods, landslides, and droughts. This research contributed to the field by demonstrating the effectiveness of combining deep learning and machine learning for weather anomaly detection. The proposed framework offers valuable insights for early warning systems and can support local governments and farmers in improving disaster preparedness and enhancing agricultural resilience in East Java.

References

[1] M. R. Kamal and M. A. Setiawan, "Deteksi Anomali dengan Security Information and Event Management (SIEM) Splunk pada Jaringan UII," J. Ajang Unjuk Tugas Akhir oleh Mhs Inform., vol. 4, pp. 1–6, 2021.

[2] H. Shi, J. Guo, Y. Deng, and Z. Qin, "Machine learning-based anomaly detection of groundwater microdynamics: case study of Chengdu, China," Sci. Rep., vol. 13, no. 1, pp. 1–19, 2023. doi: 10.1038/s41598-023-38447-5.

[3] Z. Z. Darban, G. I. Webb, S. Pan, C. Aggarwal, and M. Salehi, “Deep Learning for Time Series Anomaly Detection: A Survey,” ACM Comput. Surv., vol. 57, no. 1, 2024.

[4] D. T. Ha, N. X. Hoang, N. V. Hoang, N. H. Du, T. T. Huong, and K. P. Tran, "Explainable Anomaly Detection for Industrial Control System Cybersecurity," IFAC-PapersOnLine., vol. 55, no.10, pp. 1183-1188, 2022.

[5] P. Mobtahej, X. Zhang, M. Hamidi, and J. Zhang, "An LSTM-Autoencoder Architecture for Anomaly Detection Applied on Compressors Audio Data," Comput. Math. Methods., vol. 32, pp. 1–22, 2022. doi: 10.1155/2022/3622426.

[6] P. Saeipourdizaj, P. Sarbakhsh, and A. Gholampour,” Application of Imputation Methods for Missing Values of PM10 and O3 data: Interpolation, Moving Average, and K-Nearest Neighbor Methods,” Environmental Health Engineering and Management Journal., vol. 8, no.3, pp. 215-226, 2021.

[7] Y. Wei, J. Jang-Jaccard, W. Xu, F. Sabrina, S. Camtepe, and M. Boulic, "LSTM-Autoencoder-Based Anomaly Detection for Indoor Air Quality Time-Series Data," IEEE Sens. J., vol. 23, no. 4, pp. 3787–3800, 2023. doi: 10.1109/JSEN.2022.3230361.

[8] Z. Liu, Z. Xu, J. Jin, Z. Shen, and T. Darrell, “Dropout Reduces Underfitting,” Proceedings of Machine Learning Research, Vol.202, pp. 22233-22248, 2023.

[9] Z. Farhadi, H. Bevrani, and M. R. Feizi-Derakhshi, "Combining Regularization and Dropout Techniques for Deep Convolutional Neural Network," in Proc. IEEE Glob. Energy Conf. (GEC), pp. 335–339, 2022. doi: 10.1109/GEC55014.2022.9986657.

[10] M. K. Anam, S. Defit, Haviluddin, L. Efrizoni, M. B. Firdaus, “Early Stopping on CNN-LSTM Development to Improve Classification Performance,” Journal of Applied Data Sciences., vol. 5, no. 3, pp. 1175-1188, 2024.

[11] D. Chicco, M. J. Warrens, and G. Jurman, "The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation," PeerJ Comput. Sci., vol. 7, pp. 1–24, 2021.

[12] L. B. V. Amorim, G. D. C. Cavalcanti, and R. M. O. Cruz, "The choice of scaling technique matters for classification performance," Appl. Soft Comput., vol. 133, pp. 1–37, 2023. doi: 10.1016/j.asoc.2022.109924.

[13] S. Maleki, S. Maleki, N. R. Jennings, “Unsupervised Anomaly Detection with LSTM Autoencoders using Statistical Data-Filtering,” Applied Soft Computing., vol. 108, 2021.

[14] Y. Qiao, K. Wu, and P. Jin, “Efficient Anomaly Detection for High-Dimensional Sensing Data with One-Class Support Vector Machine,” IEEE Transactions on Knowledge and Data Engineering., vol. 35, no. 1, pp. 404-417, 2021.

[15] Ş. K. Çorbacıoğlu, and G. Aksel, “Receiver operating characteristic curve analysis in diagnostic accuracy studies: A guide to interpreting the area under the curve value,” Turkish journal of emergency medicine., vol. 23, no. 4, pp. 195-198, 2023.

[16] Z. Xu, “Factors Influencing the Predictive Performance of the LSTM Model on Stock Prices and Its Application in Forecasting US Technology Sector Stock Prices—Sample Time Span, Time Window Length, Feature Selection, and Prediction Days.,” In Proceedings of the International Conference on Image Processing, Machine Learning and Pattern Recognition., pp. 370-375, 2024.

[17] L. R. E. Malau, K. R. Rambe, N. A. Ulya, and A. G. Purba, “Dampak Perubahan Iklim terhadap Produksi Tanaman Pangan di Indonesia,” Jurnal Penelitian Pertanian Terapan., vol. 23, no. 1, pp. 34-36, 2024.

[18] D. Auliya, A. H. Rosandi, and W. T. Subroto, "Analisis Perubahan Iklim terhadap Produktivitas Padi di Jawa Timur," Diponegoro Journal of Economics., vol. 13, no. 3, pp. 55-65, 2024. https://doi.org/10.14710/djoe.47595.

[19] F. Angiulli, F. Fassetti, and L. Ferragina, "Reconstruction Error-based Anomaly Detection with Few Outlying Examples," arXiv preprint arXiv:2305.10464, 2023. doi: 10.48550/arXiv.2305.10464.

[20] T. O. Hodson, "Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not," Geosci. Model Dev., vol. 15, pp. 5481–5487, 2022. Doi: https://doi.org/10.5194/gmd-15-5481-2022.