Past research projects

New Methods for New Methods: Designing new curricular opportunities for quantitative methodology at the doctoral level

Research team: Prof. Gavin Brown

The challenges of educational data, being as complex as they are, is that graduate students (esp. at doctoral level) need advanced skills in a wide variety of complex and sophisticated data analysis techniques. This project involves research into new methods of teaching the new and powerful methods of research. The project so far has developed a new doctoral level course (Educ706AB) and has been seeking grant support for a variety of studies. Read more at: https://dx.doi.org/10.17608/k6.auckland.c.3469797.v1

Graduate Student Attributes

Research team: Professor Gavin Brown, Dr Makayla Grays, Dr Helena Cooper-Thomas, Sally Warrender

This multi-part project examines the outcomes of getting a bachelor degree in light of The University of Auckland Graduate Student Profile.

Part 1 (funded by the Vice-Chancellor’s Strategic Development Fund). This completed part of the project (Grays & Brown) surveyed incoming students in 2014 and 2015 to establish whether there were statistically significant changes in self-reported attitudes and beliefs around curiosity, openness, and love of learning. As part of that project a literature review on the purpose of higher education was prepared and presented at AERA 2014 (Chan, Brown, & Ludlow).

Part 2. Currently, the second strand (Warrender, Cooper-Thomas, & Brown) involves multi-method evaluation of how students develop the various attributes as they go through their degrees.

Part 3. Professor Brown is a collaborator with an AKO Aotearoa funded project (2017-2019), Enhancing future employability through tertiary education, AKO National Project Fund. The project is led by Prof. Susan Geertshuis (PI), Dr. Rob Wass (Otago), Tim Watts (NZAGE), Dr. Barbara Kensington-Miller & Dr. Sean Sturm (UoA).
Collaborators interested in further aspects of this project are invited to approach Prof. Brown (gt.brown@auckland.ac.nz).

Papers from Part 1 and 2 have been presented at various conferences. You can view research reports and conference papers here.

Conference publications

• Brown, G. T. L., Zlatkin-Troitschanskaia, O., Pant, H. A., Molerov, D., Lautenbach, C., Cascallar, E. C., Musso, M. F., Kyndt, E., Grays, M. P., & Boud, D. (2016, August). Higher Education Learning Outcomes: Assessment & Measurement Challenges. Symposium presented at the biennial conference of the EARLI SIG 1 Assessment & Evaluation, Munich, Germany, August 24-26, 2016.
• Brown, G. T. L., & Grays, M. P. (2016, August). Evaluating stability of self-reported personal dispositions: A repeated measures study of undergraduate students. Paper presented at the biennial conference of the SIG 1 Assessment & Evaluation EARLI, Munich, Germany.
• Warrender, S., Cooper-Thomas, H., & Brown, G. T. L. (2016, July). Graduate attributes – Do students really develop these at university? Paper presented the Higher Education Conference 2016, Amsterdam, NL.
• Grays, M. P., & Brown, G. T. L. (2015, May). Measuring openness to diversity as a tertiary student outcome. Poster presented at the annual convention of the Association for Psychological Science, New York, NY.
• Grays, M., & Brown, G. T. L. (2015, May). Investigating Senior Management Views of the Graduate Profile: Are we Assessing these Attributes? Paper presented at the Assessment for Learning in Higher Education 2015 conference, Hong Kong.
• Warrender, S., Cooper-Thomas, H., & Brown, G. T. L. (2014, November). Of Course Students Develop Graduate Attributes While at University—Graduate Recruiter Perspectives. Paper presented at the 4th Aotearoa New Zealand Organisational Psychology and Organisational Behaviour Conference, Auckland, NZ.
• Warrender, S., Brown, G. T. L., & Cooper-Thomas, H. (2014, November). Graduate attributes: Graduate recruiter perspectives. Paper presented at the 7th Education Psychology Forum, Dunedin, NZ.
• Chan, R. Y., Brown, G. T. L., & Ludlow, L. H. (2014, April). A concerning misalignment?: Comparing institutional and student perspectives on the purposes and goals of completing a U.S. bachelor's degree. Paper presented at the annual meeting of the American Educational Research Association, Philadelphia, PA.

Technical Reports

• Grays, M., & Brown, G. T. L. (2015, March). Understanding undergraduate attributes: A pre-post test survey of student self-reports during academic year 2014 (Graduate Profile Outcomes Research Project—Quant-DARE Tech. Rep. #8). Auckland, NZ: The University of Auckland, Faculty of Education.
• Grays, M., & Brown, G. T. L. (2014, December). Understanding undergraduate attributes: A survey of student self-reported interest in and acceptance of diversity at the start of academic year 2014 (Graduate Profile Outcomes Research Project—Quant-DARE Tech. Rep. #6). Auckland, NZ: The University of Auckland, Faculty of Education.
• Grays, M., & Brown, G. T. L. (2014, December). Understanding undergraduate attributes: The Views of Senior Leadership at one Research-Intensive University (Graduate Profile Outcomes Research Project—Quant-DARE Tech. Rep. #5). Auckland, NZ: The University of Auckland, Faculty of Education.
• Grays, M., & Brown, G. T. L. (2014, November). Understanding undergraduate attributes: A survey of student self-reported intellectual openness and love of learning at the start of academic year 2014. (Graduate Profile Outcomes Research Project—Quant-DARE Tech. Rep. #3). Auckland, NZ: The University of Auckland, Faculty of Education.
• Grays, M., & Brown, G. T. L. (2014, May). Structural Challenges in Evaluating Graduate Profile Outcomes at The University of Auckland (Graduate Profile Outcomes Research Project—Quant-DARE Tech. Rep. #1). Auckland, NZ: The University of Auckland, Faculty of Education.]

Comparing OECD PISA reading in English to other languages

Research team: Dr Mustafa Asil, Associate Professor Gavin Brown

The OECD Programme for International Student Assessment (PISA) provides international comparability of student performance under the assumption that the tests are equivalent measures across languages and countries. In this project we conducted measurement invariance/equivalence testing of Book 11 of the PISA 2009 reading literacy study relative to Australian English speaking students. Potential sources of non-invariance were investigated. We concluded that while statistically significant non-invariance was found, for most of the 54 comparison countries the effects were relatively small. The study has been published as:

• Asil, M., & Brown, G. T. L. (2016). Comparing OECD PISA reading in English to other languages: Identifying potential sources of non-invariance. International Journal of Testing, 16(1), 71-93. doi: 10.1080/15305058.2015.1064431

Understanding Spanish Teachers’ Beliefs and Values about Self- and Peer Assessment

Research team: Associate Professor Gavin Brown and Matthew Courtney (PhD student)

This project, undertaken for Dr Ernesto Panadero, U. Oulu, Finland, has managed a complex survey data set and carried out EFA, CFA, and SEM using WLSMV estimation (all in Mplus) to determine factors impinging on teacher self-reported use of self-assessment for learning practices. Initial results were presented at the 2013 EARLI conference and final results were published in Assessment in Education in 2014.

A follow-up analysis around peer-assessment utilised conventional EFA and CFA approaches using AMOS and was published in 2016 in European Journal of Psychology of Education.

• Panadero, E., Brown, G. T. L., & Courtney, M. G. (2014). Teachers’ reasons for using self-assessment: A survey self-report of Spanish teachers. Assessment in Education: Principles, Policy and Practice, 21(4), 365-383. doi: 10.1080/0969594X.2014.919247
• Panadero, E., & Brown, G. T. L. (2016). Teachers' reasons for using peer assessment: Positive experience predicts use. European Journal of Psychology of Education. Advanced online publication. doi: 10.1007/s10212-015-0282-5

Starpath Project

Dr Earl Irving as part of the Starpath Project

The Starpath Project was established to increase the entry and retention of Maori and Pasifika students, and students from low socio-economic backgrounds, into degree-level study. We carefully track the progress of students in 39 secondary schools from low socio-economic areas in Auckland and Northland, and provide a range of school-wide strategies for improving their results on NCEA. These include target setting using longitudinal achievement data (using a modified form of Data Envelopment Analysis), tracking and monitoring, academic counselling and parent-student-teacher conferencing. Currently we are tracking over 90,000 students, and have one of the largest longitudinal databases of student achievement and tertiary enrolment in the country.

International Baccalaureate-Primary Years Programme Evaluation

Quant-DARE assisted in a multi-disciplinary contract evaluation for the IB Research Office Singapore on the status of the International Baccalaureate-Primary Years Programme (IB PYP) in New Zealand schools. Dr Matthew Courtney, RF in Quant-DARE, analysed PYP student performance on New Zealand standardised test data (i.e., e-asTTle and PATs) relative to appropriate norms, which had to be derived from available data. Dr Claire Sinnema compared the Primary Years Programme curriculum with the New Zealand curriculum. Professor Saville Kushner and Dr Acacia Cochise completed case study analyses into how the PYP is implemented in two New Zealand schools. The project report is available at: http://www.ibo.org/globalassets/publications/ib-research/pyp/evaluation-of-the-pyp-in-new-zealand.pdf.