Засићена линеарна јединица као универзална симетрична активациона функција за дубинско учење

  • Maja Lutovac Banduka English
  • Vladimir Milićević Faculty of Mechanical and Civil Engineering, University of Kragujevac
  • Igor Franc English
  • Nemanja Zdravković English
  • Nikola Dimitrijević English
DOI: https://doi.org/10.5937/sjm21-57891
Ključne reči: вештачке неуронске мреже, решења у затвореном облику, одлука о класификацији, екстракција карактеристика, алгоритми машинског учења, дубинско учење

Sažetak


Постоји више симетричних активационих функција које се користе у вештачким неуронским мрежама за дубинско учење. У овом раду предлажемо универзалну активациону функцију засновану на засићеној линеарној јединици (SaLU), која се може користити као замена за било коју познату симетричну активациону функцију. За задатке класификације није неопходно прецизно израчунавање вероватноће детектованих класа. Одлука о класификацији доноси се на основу највеће вероватноће за дате улазне вредности. Као доказ концепта, показујемо да се две најчешће коришћене активационе функције — хиперболички тангенс и алгебарски сигмоид — могу ефикасно заменити функцијом SaLU избором једног параметра. Штавише, теоријска степ-функција такође може бити замењена функцијом SaLU, уз шири опсег прелазне области. Сви изводи заснивају се на симболичкој обради. Такође је приказана визуелизација опсега улазних вредности које доводе до одговарајуће класификације. Ово може помоћи научницима и програмерима у пројектовању сложених алгоритама машинског учења и у разумевању начина рада алгоритама дубинског учења.

Reference

Khan, S., Rahmani, H., Shah, S., & Bennamoun, M. (2018). A Guide to Convolutional Neural Networks for Computer Vision, Switzerland, Springer Nature, 2018, 54-56.

Bernard, E. (2022). Introduction to Machine Learning, Champaign, IL, USA: Wolfram Media

Wolfram, S. (2023). An Elementary Introduction to the Wolfram Language, 3rd ed., Champaign, IL, USA: Wolfram Media

Lutovac, M.D., & Tosic, D. (2014). Schematic Solver, Mathematica application package, symbolic signal processing, software implementation, mouse driven interactive drawing tool, https://library.wolfram.com/infocenter/TechNotes/4814/

Milićević, V., Franc, I., Lutovac Banduka, M., Zdravković, N., & Dimitrijević, N. (2025). Symbolic analysis of classical neural networks for deep learning, International Journal for Quality Research, 19 (1), 85-100.

Lutovac Banduka, M., & Lutovac, M. (2024). Multiplierless neural networks for deep learning, in Proc. 13th Mediterranean Conference on Embedded Computing, Budva, Montenegro, Jun. 11-14, 2024, 262-265.

Zhao, F., & Huang, S. L. (2022). On the universally optimal activation function for a class of residual neural networks, AppliedMath, 2 (4), 574-584.

Kamienny, P.A., d’Ascoli, S., Lample, G., & Charton, F. (2022). End-to-end symbolic regression with transformers,” presented at the 36th Conference on Neural Information Processing Systems (NeurIPS 2022), New Orleans, USA, Nov. 28th - Dec. 9th 2022, Adv. Neural Inf. Proc. Syst., 35, 10269-10281.

Goodfellow, I., Bengio, Y., & Courville, A. (2016), Deep Learning, Cambridge, MA, USA: MIT Press.

Setzu, M., Guidotti, R., Monreale, A., Turini, F., Pedreschi, D., & Giannotti, F. (2021), GLocalX - from local to global explanations of black box AI models, Artificial Intelligence, 103457, 294, 1-15

Wiratsin, O., & Ragkhitwetsagul, C. (2025). Effectiveness of Explainable Artificial Intelligence (XAI) Techniques for Improving Human Trust in Machine Learning Models: A Systematic Literature Review, IEEE Access, 13, 121326-121350, 2025, doi: 10.1109/ACCESS.2025.3575022

Lăzăroiu, G., Andronie, M., Iatagan, M., Geamanu, M., Stefanescu, R., & Diamarescu, I. (2022). Review deep learning-assisted smart process planning, robotic wireless sensor networks, and geospatial big data management algorithms in the internet of manufacturing things, ISPRS International Journal of Geo-Information, 11 (5), 277.

Dubey, S.R., Singh S.K., & Chauduri, B.B. (2021). Activation functions in deep learning: A comprehensive survey and benchmark, arXiv preprint arXiv:2109.14545.

Stursa, D. & Dolezel P. (2019). Comparison of ReLU and linear saturated activation functions in neural network for universal approximation, 22nd International Conference on Process Control (PC19), 11-14 June 2019, Strbske Pleso, Slovakia

Wang, J.H., Cao, J., Li, W., Yu, P., & Huang, K. (2019). A novel parallel accelerated CRPF algorithm," Applied Intelligence, 50 (3), 849-859.

Chen, J., & Pan. Z. (2023). Saturated Non-Monotonic Activation Functions, arXiv preprint arXiv:2305.07537.

Lau, M.M., & Hann Lim, K. (2018). Review of Adaptive Activation Function in Deep Neural Network," 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES). 03-06 December 2018, Sarawak, Malaysia

Yang, X., Gao, Z., & Zhang, J. (2020). Event-driven H∞ control with critic learning for nonlinear systems, Neural Networks, 132, 30-42.

Lee, Y., Park, C., & Kang, S. (2023). Deep Embedded Clustering Framework for Mixed Data, IEEE Access, 11, 33-40.

Zhang, N., Zhao, J., Ma. L., Kong, H., & Li, H. (2022). Tool Wear Monitoring Based on Transfer Learning and Improved Deep Residual Network, IEEE Access, 10, 119546-119557.

Ren, Y.M., Alhajeri, M.S., Luo, J., & Chen, S. (2022). A tutorial review of neural network modeling approaches for model predictive control, Computers & Chemical Engineering, 165 (3), 107956.

Objavljeno
2026/06/09
Broj časopisa
Vol. 21 Br. 1 (2026): Serbian Journal of Management
Rubrika
Originalni naučni članak

The Author wishes to submit the Work to SJM for publication. To enable SJM to publish the Work and to give effect to the parties’ intention set forth herein, they have agreed to cede the first right to publication and republication in the SJM Journal.

 

Cession

 

The Author hereby cedes to SJM, who accepts the cession, to the copyright in and to the paper.

 

The purpose of the cession is to enable SJM to publish the Work, as first publisher world-wide, and for republication in the SJM Journal, and to grant the right to others to publish the Work world-wide, for so long as such copyright subsists;

 

SJM shall be entitled to edit the work before publication, as it deems fit, subject to the Authors approval

 

The Author warrants to SJM that:

 

- the Author is the owner of the copyright in the Work, whether as author or as reassigned from the Author’s employee and that the Author is entitled to cede the copyright to SJM;

 

- the paper (or any of its part) is not submitted or accepted for publication in any other Journal;

 

- the Work is an original work created by the Author;

 

- the Author has not transferred, ceded, or assigned the copyright, or any part thereof, to any third party; or granted any third party a licence or other right to the copyright, which may affect or detract from the rights granted to SJM in terms of this agreement.

 

The Author hereby indemnifies the SJM as a body and its individual members, to the fullest extent permitted in law, against all or any claims which may arise consequent to the warranties set forth.

 

No monetary consideration shall be payable by SJM to the Author for the cession, but SJM shall clearly identify the Author as having produced the Work and ensure that due recognition is given to the Author in any publication of the Work.

 

Should SJM, in its sole discretion, elect not to publish the Work within 1 year after the date of this agreement, the cession shall lapse and be of no further effect. In such event the copyright shall revert to the Author and SJM shall not publish the Work, or any part thereof, without the Author’s prior written consent.