A Redefined Approach to Student Assessment Based on Essay Writing in ESP
Abstract
Fierce competitiveness of admission to highly respected universities and companies has created the need to evaluate and profile successful candidates. Traditional evaluation of candidates involving their grades is no longer satisfactory and does not provide a clear image of a candidate. There should be a way to profile a candidate and predict one’s success ranging from a success in a course to a working position. Our research aimed to study potential correlations of students’ writings and their revision exam grades acquired in English for Specific Purposes courses. 203 university engineering students submitted their introductory essays for the computational (LIWC) and statistical (SPSS) analysis. The results showed that the higher achievers used more scholarly vocabulary and verbose sentences. They were task-oriented thinking in a categorical way. On the other hand, the lower achievers used less complex sentences and were other-oriented, demonstrating a verbal dynamic style. However, stereotypical behaviour was only exhibited by the male students, i.e. due to societal pressure, the female students put more effort into their task completion, compensated for the lack of knowledge attempting to establish themselves in the male-dominated field.
References
Atkinson, R. (2001). Standardized tests and access to American universities. American Council on Education. http://works.bepress.com/richard_atkinson/36/.
Attali, Y. & Burstein, J. (2006). Automated essay scoring with e-rater V.2. Journal of Technology, Learning, and Assessment, 4, (3).
Božić Lenard, D. (2016). Gender differences in the political speeches from the 113th United States Congress (unpublished doctoral thesis). The Faculty of Humanities and Social Sciences Osijek, Croatia.
Božić Lenard, D., Ferčec, I. & Liermann-Zeljak, Y. (2018). Grammar or vocabulary – students’ friends or foes? In: N. Stojković and N. Burkšaitienė (Eds.), Establishing Predominance of English for Specific Purposes within Adult English Language Teaching (1-26). Cambridge Scholars Publishing.
Brown, R. M. (2004). Self-composed: Rhetoric in psychology personal statements. Written Communication, 21, 242-260.
Burstein, J., Chodorow, M. & Leacock, C. (2004). Automated essay evaluation: The Criterion online writing system. AI Magazine, 25, 27-36.
Burstein, J., Marcu, D. & Knight, K. (2003). Finding the WRITE Stuff: Automatic Identification of Discourse Structure in Student Essays. IEEE Intelligent Systems: Special Issue on Natural Language Processing, 18 (1), 32-39.
Chung, C.K. & Pennebaker, J.W. (2007). The psychological function of function words. In K. Fiedler (Ed.), Social communication: Frontiers of social psychology (343-359), Psychology Press, New York.
Crossley, S. A. & McNamara, D. S. (2010). Cohesion, coherence, and expert evaluations of writing proficiency. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of the 32nd Annual Conference of the Cognitive Science Society (984-989). Austin, TX: Cognitive Science Society.
Crossley, S. A. & McNamara, D. S. (2011). Understanding expert ratings of essay quality: Coh- Metrix analyses of first and second language writing. International Journal of Continuing Engineering Education and Life-Long Learning, 21, 170-191.
Crossley, S. A. & McNamara, D. S. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35, 115-136.
Crossley, S. A., Roscoe, R. & McNamara, D. S. (2014). What is successful writing? An investigation into the multi ways writers can write successful essays. Written Communication, 31 (2), 184-214.
Crossley, S. A., Weston, J., McLain Sullivan, S. T., & McNamara, D. S. (2011). The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28 (3), 282-311.
Deane, P. (2013). On the relation between automated essay scoring and modern views of the writing construct. Assessing Writing, 18, 7-24.
Elliott, S. (2003). Intellimetric: From Here to Validity. In Shermis, M. & Burstein, J. (eds.) Automated essay scoring: A cross-disciplinary perspective (71-86). Hillsdale, NJ: Lawrence Erlbaum Associates.
Ferguson, E., Sanders, A., O’Hehir, F., & James, D. (2000). Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. Journal of Occupational and Organizational Psychology, 73, 321-344.
Fredrickson, B.L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218-226.
GlenMaye, L., & Oakes, M. (2002). Assessing suitability of MSW applicants through objective scoring of personal statements. Journal of Social Work Education, 39, 67-82.
Graesser, A. C., McNamara, D. S., Louwerse, M. M. & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, and Computers, 36, 193-202.
Hamel, G. (2011). First, let’s fire all the managers. Harvard Business Review, 89, 48-60.
Haswell, R. H. (2000). Documenting improvement in college writing: A longitudinal approach. Written Communication, 17, 307-352.
Heylighen F. & Dewaele, J.M. (2002). Variation in the contextuality of language: an empirical measure. Foundations of Science, 6, 293–340.
Jarvis, S., Grant, L., Bikowski, D. & Ferris, D. (2003). Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing, 12 (4), 377-403.
Jurafsky D., Ranganath R. & McFarland, R.D. (2009). Extracting social meaning: identifying interactional style in spoken conversation. In Proceedings of NAACL, 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Human Language Technologies: 638–646.
Kellogg, R. & Whiteford, A. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44, 250-266.
McNamara, D. S., Crossley, S. A., & Roscoe, R. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45 (2), 499-515.
McNamara, D. S., Louwerse, M. M., McCarthy, P. M., & Graesser, A. C. (2010). Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47, 292-330.
Nisbett, R.E., Peng, K., Choi, I. & Norenzayan, A. (2001). Culture and systems of thought: Holistic versus analytic cognition. Psychological Review, 108, 91–310.
Pennebaker, J.W. (2013). The Secret Life of Pronouns: What Our Words Say About Us. New York: Bloomsbury Press
Pennebaker, J.W. & Chung, C.K. (2007). Expressive writing, emotional upheavals, and health. In H. Friedman & R. Silver (Eds.), Handbook of health psychology (263-284), Oxford University Press, New York.
Pennebaker, J., Chung, C., Ireland, M., Gonzales, A., Booth, R. (2007). The Development and Psychometric Properties of LIWC2007. Austin, Texas: LIWC.net
Pennebaker, J.W, Chung, C.K., Frazee, J., Lavergne, G.M. & Beaver, D.I. (2014). When Small Words Foretell Academic Success: The Case of College Admissions Essays. PloS ONE, 9 (12), doi: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4281205/
Pennebaker, J.W. & King, L.A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77, 1296–1312.
Rabow, M. W., Wrubel, J. R., & Rachel, N. (2009). Promise of professionalism: Personal mission statements among a national cohort of medical students. Annals of Family Medicine, 7, 336-342.
Robinson, R.L., Navea, R. & Ickes, W. (2013). Predicting final course performance from students’ written self-introductions; A LIWC analysis. Journal of Language and Social Psychology, 32, 481-491.
Turner, R., & Nicholson, S. (2011). Reasons selectors give for accepting and rejecting medical applicants before interview. Medical Education, 45, 298-307.
Varner, L.K., Roscoe, R.D. & McNamara, D.S. (2013). Evaluative misalignment of 10th-grade student and teacher criteria for essay quality: An automated textual analysis. Journal of Writing Research, 5 (1), 35-59.
Walfish, S., & Moreira, J. (2005). Relative weighting of admission variables in marriage and family therapy graduate programs. American Journal of Family Therapy, 33, 395-402.
Waugh, C.E. & Fredrickson, B.L. (2006). Nice to know you: Positive emotions, self-other overlap, and complex understanding in the formation of new relationships. The Journal of Positive Psychology, 1, 93-106.
The details about the publication policy, including copyright and licensing, are available at: