User evaluation of the interfaces for the remote control of manufacturing systems

  • Vesna K Spasojević-Brkić Univerzitet u Beogradu, Mašinski fakultet
  • Goran Punik



This paper is a contribution to the new direction for enhancement in remote controlling functionality in manufacturing systems and presents a methodology for evaluating user interfaces for the remote control of a Ubiquitous Manufacturing System. Survey is based on the experiment that involved 68 subjects, students at the Faculty of Mechanical Engineering, the University of Belgrade, Serbia, as future users of the interface for the remote collaborative control of manufacturing systems. The results show that the user evaluation instrument consists of usability and collaboration as correlated measures. Usability comprises effectiveness, efficiency and representational fidelity measures, while collaboration includes collaboration effort, awareness/ involvement and co-presence. The results of this survey will help in future design decisions and measuring its quality of use.


Keywords: interface, evaluation, remote control, manufacturing system




Rad predstavlja doprinos novom pravcu unapredjenja daljinskog upravljanja funkcionalnostima i prezentuje metodologiju za procenu korisničkih interfejsa za daljinsko upravljanje ubikvitnim proizvodnim sistemima. Istraživanje je zasnovano na eksperimentu, koji je uključio 68 ispitanika, studenata Mašinskog fakulteta Univerziteta u Beogradu, koji predstavljaju buduće korisnike interfejsa za daljinsko kolaborativno upravljanje proizvodnim sistemima. Rezultati pokazuju da se instrument za ocenu sastoji od koreliranih mera korisnosti i kolaboracije. Korisnost sadrži mere efikasnosti, efektivnosti i reprezentativne tačnosti, dok kolaboracija uključuje napor, uključenost i koprisutnost. Rezultati rada treba da pomognu pri odlučivanju u budućem dizajniranju i pri merenju kvaliteta upotrebe.


Ključne reči: interfejs, evaluacija, daljinsko upravljanje, proizvodni sistem

Author Biography

Vesna K Spasojević-Brkić, Univerzitet u Beogradu, Mašinski fakultet
Prof. dr Vesna Spasojević-Brkić, Katedra za Industrijsko inženjerstvo, Mašinski fakultet Univerziteta u Beogradu


Abdinnour‐Helm, S. F., Chaparro, B. S., & Farmer, S. M. (2005). Using the end‐user computing satisfaction (EUCS) instrument to measure satisfaction with a web site. Decision Sciences, 36(2), 341-364.

Aladwani, A. M., &Palvia, P. C. (2002). Developing and validating an instrument for measuring user-perceived web quality. Information & Management, 39(6), 467-476.

Barnum, C. (2002). Usability testing and research. New York: Longman.

Bevis, K.J. & Henke, K.A. 2008. Evaluating usability in an information product. Proc. of the IEEE InternationalProfessional Communication Conference (IPCC‟08), 1-5.

Dalgarno, B., & Lee, M. J. (2010). What are the learning affordances of 3‐D virtual environments?.British Journal of Educational Technology, 41(1), 10-32.

DeLone, W. & McLean, E., (1992). Information Systems Success: The Quest for the Dependent Variable, Information SystemsResearch, (3:1), 60–95.

Doll, W. J., &Torkzadeh, G. (1991). The measurement of end-user computing satisfaction: theoretical and methodological issues. MIS quarterly, 5-10.

Doll, W. J., Xia, W., &Torkzadeh, G. (1994). A confirmatory factor analysis of the end-user computing satisfaction instrument. Mis Quarterly, 453-461.

Grandzol J., & Gershon M. (1998). A survey for standardizing TQMmodeling research. Int. J. Qual. Sci., 3(1): 80-105.

Grudin, J., (1988). Evaluation of user interfaces for remote collaborative control of manufacturing systems, Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces, in Proceedings of the ACM CSCW'88 Conference on Computer-Supported Cooperative Work, Perspectives on Evaluation, pp. 85-93.

Hair J, Anderson R, Tatham R, Black W (1998). Multivariate Data Analysis, 5th edition, Prentice Hall, London.

Hartson, H., Andre, T. & Williges, R. (2001). Criteria for evaluating usability evaluation methods,International Journal of Human-Computer Interaction, Vol. 13, Issue 4, 373-410.

Holzinger, A. (2005). Usability engineering methods for software developers.Commununications of the ACM, 48(1), 71–74.

Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International journal of human-computer studies, 64(2), 79-102.

Hrimech, H., Alem, L., &Merienne, F. (2011). How 3D interaction metaphors affect user experience in collaborative virtual environments. Advances in Human-Computer Interaction, 2011: 8.

Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: guidelines for determining model fit. Articles, 2.

IEC 9126-1: 2001 Software Engineering-Product Quality-Part 1: Quality Model. Geneva, Switzerland: International Organization for Standardization.

ISO 9241-11:1998 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability, International Organization for Standardization.

Ives, B. and M. Olson. (1984)User Involvement and MIS Success; A Review of Research,Management Science. 30 (5), 586-603.

Joreskog, K. G., & Sorbom,, D., (1984). LISREL VI: Analysis of Linear Structural Relationships by the Method of Maximum Likelihood, National Educational Resources, Chicago, Ill.

Liu, Q., & Shi, Y. J. (2008). Grid manufacturing for cross-enterprise. Int J AdvManufTechnol, 36, 205–212.

Lu, S. Y., ElMaraghy, W., Schuh, G., & Wilhelm, R. (2007). A scientific foundation of collaborative engineering. CIRP Annals-Manufacturing Technology, 56(2), 605-634.

Pappas, M., Karabatsou, V., Mavrikios, D., &Chryssolouris, G. (2006). Development of a web-based collaboration platform for manufacturing product and process design evaluation using virtual reality techniques. International Journal of Computer Integrated Manufacturing, 19(8), 805-814.

Putnik, G.D., Shah, V., Castro, H. & Cunha, (2011) M.M.: The Functionality of Remote Controlling in Ubiquitous Manufacturing Systems Demonstrator for I*PROMS ‘efm’ Showcase, Tekhné - Revista de EstudosPolitécnicos/Polytechnical Studies Review, 9(15), 169-189.

Redish, G. (2007). Expanding usability testing to evaluate complex systems. Journal ofUsability Studies, 2(3), 102–111.

Rezayat, M., (2000) The enterprise - web portal for life cycle support. Computer Aided Design, 32(2), 85–96.

Shah, V., Putnik, G. D., & Castro, H. (2012). Prototype Multiplex Communication System for Remote Control of Machine Tools. In Virtual and Networked Organizations, Emergent Technologies and Tools. Springer Berlin Heidelberg, 241-252.

Straub, D. W., Hoffman, D. L., Weber, B. W., &Steinfield, C. (2002). Measuringe-commerce in net-enabled organizations: An introduction to the special issue.Information Systems Research, 13(2), 115–124.

Todd, S., 1992, Collective Action: Theory andApplication, University of Michigan Press.

Vilana, J. R., & Monroy, C. R. (2010). Influence of cultural mechanisms on horizontal inter-firm collaborations. Journal of Industrial Engineering and Management, 3(1), 138-175.

Wang, L., Orban, P., Cunningham, A., & Lang, S. (2004). Remote real-time CNC machining for web-based manufacturing. Robotics and Computer-Integrated Manufacturing, 20(6), 563-571.

Supplementary files
Original Scientific Paper