Keywords
Reinforcement Learning
SAC Algorithm
Policy Optimization
Environmental Adaptation
Intelligent Control
Abstract
With the rapid development of artificial intelligence and robotic engineering, traditional pre-programmed robot control systems can no longer adapt to complex, dynamic, and unstructured working environments. Robot autonomous learning has become a core research direction to realize intelligent robot evolution and environmental adaptive optimization. Autonomous learning robots can independently perceive environmental information, summarize task experience, optimize control strategies, and complete unknown tasks without manual intervention and repeated parameter debugging. This paper systematically studies the core framework, key algorithms, and optimization mechanisms of robot autonomous learning. Focusing on reinforcement learning and deep adaptive learning paradigms, it constructs an improved soft actor-critic (SAC) autonomous learning model, optimizes the policy gradient update mechanism and reward function setting strategy, and solves the problems of low exploration efficiency and slow convergence speed of traditional robot learning algorithms in dynamic environments. A series of comparative experiments are carried out based on robot navigation and obstacle avoidance task scenarios. The experimental results show that the improved autonomous learning algorithm has significant advantages in convergence speed, task completion accuracy, and environmental robustness. The average task success rate is increased by 12.7%, and the learning convergence time is shortened by 28.3% compared with the traditional Q-learning algorithm. This study provides a theoretical basis and technical reference for the engineering application of high-efficiency autonomous learning robots in complex scenarios.
References
Que, H. H., Jin, Y., Wang, T., Liu, M. K., Yang, X. H., & Qiao, F. (2023). A Survey of Approximate Computing: From Arithmetic Units Design to High-Level Applications. Journal of Computer Science and Technology, 38(2), 251-272.
Zhu, H., Huang, X., Gao, H., Jiang, M., Que, H., & Mu, L. (2025, July). A Wireless Collaborated Inference Acceleration Framework for Plant Disease Recognition. In International Conference on Intelligent Computing (pp. 331-341). Singapore: Springer Nature Singapore.
Liu, W., Zhao, R., Sha, Z., Cui, Q., & Zhang, Y. (2026). Mapping the City Through the Lens of Language Models. arXiv preprint arXiv:2608.02971.
Zhao, R., Liu, W., Sha, Z., Su, N., Zhang, Y., & Long, Y. (2026). Culturally uneven urban perception in large language models [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2604.20048v3
Lyu, Y., & Ding, R. (2025). Discovering Self-Regulated Learning Patterns in Chatbot-Powered Education Environment. arXiv preprint arXiv:2510.01275.
Lyu, Y., Armas-Cervantes, A., & Mendoza, A. (2025, December). Uncovering self-regulated learning patterns in Chatbot-powered environment through a process mining lens. In 36th Australasian Association for Engineering Education Annual Conference (AAEE2025) (pp. 371-379). Brisbane, QLD: Engineers Australia.
Lyu, Y. (2025). Enhancing Approximate Conformance Checking Accuracy with Hierarchical Clustering Model Behaviour Sampling. International Journal of Advanced Computer Science & Applications, 16(8).
Han, Z., Chen, W., Han, Y., Mao, R., & Qin, J. (2026). Fast Diversified Top-k Rule Discovery via User-Guided Embeddings. IEEE Transactions on Knowledge and Data Engineering.
Chaoyong Ma, Nan Si, Xuewei Song, Chen Liang and Haoming Luo, A Bearing Fault Diagnosis Method Based on Concise Empirical Wavelet Transform, International Journal of Comprehensive Engineering，2023, 12(1), 1-6
Yuyan Li, Lin Chen, Mengyu Yang, Qiaomei Li and Taifu Li, Polarization Converters Composed of Metasurfaces with Independent Manipulation of Polarization and Phase, International Journal of Comprehensive Engineering，2023, 12(1), 7-14
Jiaxun Du, Zhenbao Fu, Jiachen Liu, Huaqing Wang, Zhentao Ge and Liuyang Song, Bearing Fault Diagnosis Method Based on Attention Residual Network Under Small Sample Condition, International Journal of Comprehensive Engineering，2023, 12(1), 15-24
Jiawei Lin, Changkun Han, Liuyang Song, Wei Lu and Huaqing Wang, Time Domain Feature Enhancement Model Based on Correlated B-spline Wavelet Base Convolutional Sparse, International Journal of Comprehensive Engineering，2023, 12(1), 25-33
Bing Wang, Haihong Tang, Xiaojia Zu and Wuwei Feng, Application of Parameter Adaptive VMD in Bearing Fault Diagnosis, International Journal of Comprehensive Engineering，2023, 12(1), 34-42
Xiaojia Zu, Haihong Tang and Bing Wang, Research on Intelligent Diagnosis Based on Support Vector Machine for Structural Fault, International Journal of Comprehensive Engineering，2024, 13(1), 1-12.
Xi Qiao, Di Zhao, Nan Si, Jifeng Sui and Yonggang Xu, Weighting Spectrum Editing Based on Amplitude Mapping and Its Application in Composite Fault Diagnosis of Rolling Bearings, International Journal of Comprehensive Engineering，2024, 13(1), 13-21
Xixin Chen, Ming Tu, Lei Su and Ke Li, Rolling Bearing Fault Diagnosis Based on Digital Twin Method, International Journal of Comprehensive Engineering，2024, 13(1), 22-39
Jianheng Liang, Yang Yang, Ran Zhang, Guican Wu, Yuanyuan Yang, TaiFu Li, Honglin Duan and Yuyan Li, Research on Perfume User Experience Evaluation Based on Human Machine Hybrid Generation Adversarial Network, International Journal of Comprehensive Engineering，2024, 13(1), 40-54
Han Feng, Xia Qin and Hongtao Xue, A Novel BN and DST Fusion Framework for Fault Identification of In-wheel Motor, International Journal of Comprehensive Engineering，2024, 13(1), 55-62
Li, P., Zhang, H., Li, W., Huang, D., Guo, Z., Chen, J., ... & Koshizuka, N. (2025). GeoAvatar: A big mobile phone positioning data-driven method for individualized pseudo personal mobility data generation. Computers, Environment and Urban Systems, 119, 102252.
C. Huang, X. Chen, G. Chen, P. Xiao, G. Ye Li and W. Huang, "Deep Reinforcement Learning-Based Resource Allocation for Hybrid Bit and Generative Semantic Communications in Space-Air-Ground Integrated Networks," in IEEE Journal on Selected Areas in Communications, vol. 43, no. 12, pp. 3942-3954, Dec. 2025, doi: 10.1109/JSAC.2025.3623157.
Feng Sun, Zhenguo Song, Dinghao Dong, Shuai Shen, Jie Cai, Heng Luo and Jianhui Zhou, A Variational Time-domain Decomposition Method for Vibration Signals Based on Exponential Adaptive Weighted Moving Average Optimization, International Journal of Comprehensive Engineering，2025, 14(1), 1-12
Zheng, X., Hu, S., Dwyer, V., Barrett, L., & Derakhshani, M. (2026). Joint attention mechanism learning to facilitate opto-physiological monitoring during physical activity. Biomedical Signal Processing and Control, 113, 108949.
Zhichao Qiu, Zhijia Yan, Jinhua Huang, Yewen Chen, Yaotiao Ren, Shimin Yu and Xuewei Song,Bearing Fault Diagnosis Based on Adaptive Multiple Synchronous Compressed Transform and Twin Network, International Journal of Comprehensive Engineering，2025, 14(1), 13-22
Xuefei Ming, Zhenyu Pan, Gang Wang, Yong Ji, Lei Su and Ke Li, Sparse Wavelet Neural Network with Frequency-Domain Gating for Ultrasonic Vibration-Based Flip-Chip Defect Detection, International Journal of Comprehensive Engineering，2025, 14(1), 23-39
Wang, Z., Yu, J., Liu, H., Zheng, Z., Jin, Y., Li, S., ... & Zhang, K. (2025, July). Generative Music Models’ Alignment with Professional and Amateur Users’ Expectations. In Findings of the Association for Computational Linguistics: ACL 2025 (pp. 6909-6920).
Xu, Y., Sun, Y., Zhai, B., Li, M., Liang, W., Li, Y., & Du, S. (2025, April). Zero-shot video moment retrieval via off-the-shelf multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 9, pp. 8978-8986).
Minggang Tong, Xin Wang, Wenhua Zhang, Qi Hu and Zengwei Lyu, Intelligent Fault Diagnosis of Dynamic Equipment in Public Auxiliary Systems Based on Multimodal Deep Q-Networks, International Journal of Comprehensive Engineering，2025, 14(1), 40-54
Lijun Yan, Chaoyong Ma, Heng Zhang, Mengjie Xie and Rongbin Zhang, Multi-objective Optimization of Solenoid Valves Integrating AHP Decision-making and Orthogonal Design, International Journal of Comprehensive Engineering，2026, 15(1), 1-12
Jiaqi Guan, Jiachen Liu, Huaqing Wang, Liuyang Song, Hassan Sajjad and Zhilong Gao, Research on Rolling Bearing Remaining Useful Life Prediction Method Based on Knowledge Distillationnternational, Journal of Comprehensive Engineering，2026, 15(1), 13-23
Cheng, X., Qin, Y., Tan, Y., Li, Z., Wang, Y., Xiao, H., & Zhang, Y. (2026). Psymem: Fine-grained psychological alignment and explicit memory control for advanced role-playing llms. Transactions of the Association for Computational Linguistics, 14, 510-529.
Wang, T., Yu, Y., Wang, Q., & Qian, J. (2025). Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation. arXiv preprint arXiv:2508.01394.
Mini Han Wang , " AI-Powered Innovations in Ophthalmic Diagnosis and Treatment ", Bentham Science Publishers (2025). https://doi.org/10.2174/97988988122871250101
Tian Teng and Xiao Zhang, Design Principles and Feasibility Assessment of an Intelligent Spare-Parts Shelving System for Shipboard Storage Environments, Journal of Comprehensive Engineering，2026, 15(1), 24-30
Su, J., Cai, C., Zhu, F., He, C., Xu, X., Guan, D., & Si, C. (2024, September). Momentum auxiliary network for supervised local learning. In European Conference on Computer Vision (pp. 276-292). Cham: Springer Nature Switzerland.
Su, J., Zhu, F., Shi, H., Han, T., Qiu, Y., Luo, J., ... & Gao, J. (2025). MAN++: Scaling Momentum Auxiliary Network for Supervised Local Learning in Vision Tasks. arXiv preprint arXiv:2507.16279.
Quan, Q., Gao, Y., & Wang, Q. (2025). The impact of teacher emotional support on students' engagement in AI-mediated English learning environments: The mediating role of resilience and self-efficacy. Acta Psychologica, 260, 105766.
Xu, Z., Zhang, X., Li, R., Tang, Z., Huang, Q., & Zhang, J. (2024). Fakeshield: Explainable image forgery detection and localization via multi-modal large language models. arXiv preprint arXiv:2410.02761.
Xu, Z., Zhang, X., Zhou, X., & Zhang, J. (2025). AvatarShield: Visual Reinforcement Learning for Human-Centric Video Forgery Detection. arXiv preprint arXiv:2505.15173.
Huang, Q., Xu, Z., Zhang, X., & Zhang, J. (2025). UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization. arXiv preprint arXiv:2510.03161.
Cui, J., Zhang, G., Chen, Z., & Yu, N. (2022). Multi-homed abnormal behavior detection algorithm based on fuzzy particle swarm cluster in user and entity behavior analytics. Scientific Reports, 12(1), 22349.
Liu, T., He, J., Zhou, R., Song, J., Liu, H., & Liang, Y. (2025). Revolutionizing clinical decision making through deep learning and topic modeling for pathway optimization. Scientific Reports, 15(1), Article 28787. https://doi.org/10.1038/s41598-025-12679-z
Wu, X., Xing, X., Yao, J., Qian, Q., & Song, J. (2025). Scnet: spectral convolutional networks for multivariate time series classification. Applied Intelligence, 55(6), Article 456. https://doi.org/10.1007/s10489-025-06352-1
Ma, C., Song, J., Xu, Y., Fan, H., Wu, X., & Sun, T. (2024). Vehicle-Based Machine Vision Approaches in Intelligent Connected System. IEEE Transactions on Intelligent Transportation Systems, 25(3), 2827-2836. https://doi.org/10.1109/TITS.2023.3276325
Song, J., Wu, X., Yao, J., Zhang, Q., Shang, C., Qian, Q., & Song, J. (2026). SPC: Self-supervised point cloud completion. Neural Networks, 194, Article 108107. Advance online publication. https://doi.org/10.1016/j.neunet.2025.108107
Yu, C., Li, P., Wu, H., Wen, Y., Deng, B., & Xiong, H. (2024). USM: Unbiased Survey Modeling for Limiting Negative User Experiences in Recommendation Systems. arXiv preprint arXiv:2412.10674.