Approaches to learning as predictors of academic achievement: Results from a large scale, multi-level analysis


Artikel

Approaches to learning as predictors of academic achievement: Results from a large scale, multi-level analysis

Kim J. Herrmann*,a, Velda McCuneb, Anna Bager-Elsborga

aCentre for Teaching and Learning, Aarhus UniversitybInstitute for Academic Development, University of Edinburgh

The relationships between university students’ academic achievement and their approaches to learning and studying continuously attract scholarly attention. We report the results of an analysis in which multilevel linear modelling was used to analyse data from 3,626 Danish university students. Controlling for the effects of age, gender, and progression, we found that the students’ end-of-semester grade point averages were related negatively to a surface approach and positively to organised effort. Interestingly, the effect of the surface approach on academic achievement varied across programmes. While there has been considerable interest in the ways in which academic programmes shape learning and teaching, the effects of these contexts on the relationship between approaches to learning and academic outcomes is under-researched. The results are discussed in relation to findings from recent meta-analyses in the field.

Key words: Higher education, approaches to learning, academic achievement, academic programmes

Published: July 2017

* Författarkontakt: Kim J. Herrmann, kh@au.dk

Artiklar og reflektioner är kollegialt granskade. Övriga bidragstyper granskas av redaktionen. Se www.hogreutbildning.se ISSN 2000-7558

©2017 Kim J. Herrmann et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), allowing third parties to share their work (copy, distribute, transmit) and to adapt it, under the condition that the authors are given credit, that the work is not used for commercial purposes, and that in the event of reuse or distribution, the terms of this license are made clear.

Citation: Kim J. Herrmann et al. (2017) «Approaches to learning as predictors of academic achievement: Results from a large scale, multi-level analysis», Högre utbildning 7, 29–42. http://dx.doi.org/10.23865/hu.v7.905

INTRODUCTION

This study considers how the relationships between approaches to learning and academic achievement are moderated by programmes of study when age, gender, and year of study were taken into account. Programmes of study refer to the particular sets of modules which a student completes to achieve his or her academic degree. The study’s context was 41 programmes offered by seven departments situated within one faculty in a Danish university. Given the ongoing attention paid to the relationships between approaches to learning and academic outcomes in the literature, it is important to analyse these connections across diverse contexts, instruments and methods of analysis. The research presented in this paper uses multi-level modelling (MLM) to avoid some of the potential pitfalls in research in which multi-programme samples have been analysed (Diseth & Kobbelvedt, 2010; Heikkilä & Lonka, 2006). For example, because MLM takes into account that students within programmes are interconnected (Hox, 2010), results in the present paper are less prone to being biased by students being nested within study programmes.

Students’ approaches to learning and studying

The results which are reported in this paper build on research into students’ approaches to learning and studying which began when Marton and Säljö (1976, 1997) identified the crucial distinction between deep and surface approaches to learning (originally levels of processing). These studies explored students’ experiences of reading academic articles. Through their seminal work on students’ accounts of their learning experiences, Marton and Säljö identified qualitative differences in the students’ understanding of the learning material explained in terms of qualitative differences in how students approached their learning. Some students applied a deep approach to learning which involved looking for and working to understand the meaning of what they were learning. Students adopting a surface approach to learning, on the other hand, focused on reproducing features of the learning materials without seeking understanding.

Since then, the distinction between deep and surface approaches to learning has been repeatedly identified in quantitative and qualitative studies in diverse countries and academic disciplines using different research methods (Biggs 2012; Entwistle, 2009; Entwistle & McCune, 2004). As research within this area began to focus more broadly on students’ day-to-day study activities, a third approach to learning was identified. This approach was labelled a strategic (or achieving) approach and described a focus on achieving high grades in combination with well-organised studying and attention to assessment requirements (Biggs, 1987; Entwistle & Ramsden, 1983). However, as research regarding students’ approaches to learning developed further, the emphasis in recent questionnaires designed to measure approaches to learning and studying has shifted away from achievement motivation and towards students’ self-reported ability to manage and organise their learning (Entwistle & McCune, 2004). Hence, the concept of organised effort has replaced the strategic approach to learning in recently developed inventories (Entwistle, McCune & Hounsell, 2003; Parpala & Lindblom-Ylänne, 2012).

Approaches to learning as predictors of academic achievement

Marton and Säljö (1976) found that qualitative differences in the students’ levels of understanding of an academic text were related to qualitative differences in the students’ ways of processing information while reading the text. Marton and Säljö (1984) later interpreted their findings as representing a functional relationship between qualitative differences in approaches to learning and qualitative differences in the outcome of learning, which led them to propose that a deep approach to learning was preferable if the desired learning outcome was to understand concepts rather than merely memorising facts. The qualitative research by Marton and Säljö was soon followed by quantitative research by Biggs (1987) and Entwistle and Ramsden (1983) who developed questionnaires to measure students’ approaches to learning and relate them to academic achievement in form of students’ grades.

Within the past 15 years, two systematic reviews have explored the association between students’ approaches to learning and academic achievement. Watkins (2001) found that for Western university students, the average correlation between academic achievement and the deep (r = .18), surface (r = −.12), and achieving approaches to learning (r = .20), respectively, were found to be in the expected direction albeit of modest magnitude. In a more recent and extensive meta-analysis, Richardson, Abraham, and Bond (2012) found a negative but weak relation between a surface approach to learning and grade-point average (GPA) (r = −.18). A positive and weak relation was found between GPA and deep (r = .14) and strategic approaches (r = .23) to learning respectively.

A possible explanation for the rather modest strengths of association between approaches to learning and examination grades has been that assessment systems in higher education might not always reward high quality learning outcomes, especially when assessment relies on examination by means of multiple-choice test (Biggs, 1987). Working within a Dutch university context, Vermunt (2005) concluded that aspects of students’ learning strategies such as critical, analytical, and concrete thinking seemed to only be rewarded to a lesser extent in exam results. More recently, Asikainen et al. (2013) also emphasised that assessment grades in higher education may not fully reflect the quality of students’ learning.

How might programmes of study moderate the relationships between approaches to learning and achievement?

The academic programme is the nexus of a range of influences on teaching and on students’ learning. Trowler and Cooper (2002) explain how the norms and practice of particular academic departments are important in explaining the choices academics make about teaching and assessment. There is an interplay between academic disciplines, departmental teaching and learning practices and institutional culture and policy (Kreber, 2009; Trowler, 2014). All of these layers of social practice shape the choices made about teaching and assessment within each programme of study. Then the teaching - and particularly the assessment practices - of programmes are likely to shape the relationships between approaches to learning and academic achievement. The relationship between the deep approach and academic achievement, for example, is likely to be strongest in programmes which reward understanding and critical analysis in their assessments. Vermunt (2005) comments that of all of the indicators used in his study of students’ learning patterns, academic discipline had the strongest relationships with how students learned. Following Vermunt, we suggest that the typical teaching and assessment patterns in diverse programmes and disciplines will likely shape how students’ learn and how this relates to achievement.

The present study

The meta-analyses by Watkins (2001) and Richardson et al. (2012) in combination seem to offer positive, albeit not strong, evidence for the claims put forward by researchers such as Marton and Säljö (1984), Biggs and Tang (2011) and Entwistle (2009) that approaches to learning can be significant for students’ academic performance. More recently, Trigwell, Ashwin, and Millan (2013) noted that when more sophisticated multi-variable (structural equation modelling) approaches were used with their data set from a university in the UK, a relationship between the deep approach and achievement was not apparent. Interestingly, the lack of relationship between the deep approach and achievement was shown even though the university where the data were collected had a particular emphasis on examinations which tested understanding. The authors note that their findings differ from other studies using similar analytic approaches, which did find a positive relationship between a deep approach and achievement. Recent path analysis research by Karagiannopoulou and Milienos (2015), however, also failed to find a relationship between the deep approach and academic outcomes. Trigwell et al. (2013) suggest the importance of using more sophisticated analytic approaches and more diverse variables when considering the relationships between approaches to learning and achievement.

Overall, this complex picture suggests the value of taking forward further more nuanced investigations of the interplay between diverse influences on students’ academic outcomes, including approaches to learning. The aim of the present study was to further explore the relation between students’ approaches to learning and academic achievement in a sample of Danish university students and with the use of a more sophisticated analytical approach (i.e., multi-level linear modelling). We investigated the following questions:

  1. To what extent do approaches to learning predict academic achievement for the participants in this study?
  2. To what extent is the relationship between approaches to learning and academic achievement moderated by the particular programmes of study experienced by these participants when age, gender and year of study are taken into account?

By applying recently developed inventories of students’ approaches to learning and analysing the data by means of multi-level linear modelling, the study extends previous research in the following ways. First, multilevel linear modelling allowed us to analyse a large sample of students representing a multitude of higher education programmes without violating the assumption of independence of errors. This statistical approach also allowed us to take into account variability in the strength of the relation between approaches to learning and academic achievement. Thus, the analyses could potentially provide clues about academic programmes as moderators of the association between approaches to learning and academic achievement. Our analytic approach also allowed us to take into account the additional variables such as age, gender and year of study. Finally, the inventory adopted allowed us to explore the relation between academic achievement and the organised effort scale recently developed by Entwistle and McCune (2004) which has been less researched.

METHODS

Instrument and measures

Results were based on data collected by use of three scales from the Finnish Learn questionnaire (Parpala & Lindblom-Ylänne, 2012) which was a further development of the British Experience of Teaching and Learning Questionnaire (ETLQ) (Entwistle et al., 2003). The three scales were based on items from the short version of the Approaches to Learning and Studying Inventory (ALSI, see Entwistle & McCune, 2004). The ALSI was developed based on the ASSIST questionnaire (Tait, Entwistle & McCune, 1998) which in turn built on the original Approaches to Studying Inventory developed by Entwistle and Ramsden (1983). The intention in creating the ALSI was to enhance and update the measurement of high quality learning in higher education based on previous research and a broader review of the literature. The Finnish researchers, however, changed the ALSI by modifying existing items and including three items from the Learning and Studying Questionnaire (LSQ) (Entwistle et al., 2003) and two items from the Revised Learning Process Questionnaire (R-LPQ) (Kember, Biggs, & Leung, 2004). The first section of the Learn, thus, includes 12 items measuring three aspects of students’ approaches: Deep approach (‘I look at evidence carefully to reach my own conclusion about what I’m studying’), surface approach (‘Even though I study some things over and over again to remember them, they do not make sense to me’), and organised effort (‘I am generally systematic and organized in my studies’). The 12 items were translated into Danish with minor modifications, and confirmatory factor analysis suggested that the three scales were applicable in the context of Danish higher education (Herrmann, Bager-Elsborg & Parpala, 2016).

Regarding the measurement of academic achievement, the Danish grading scale ranges from −3 (lowest, equivalent to an ‘F’) to 12 (highest, equivalent to an ‘A’) and is comparable with the European Credit Transfer and Accumulation Scheme (ECTS) grading system (Danish Ministry of Education, and Science, 2015). Data about students’ grade point average at the summer 2014 end-semester examinations were collected in the fall of 2014 and subsequently linked to the students’ responses to the survey. Grades inevitably have their limitations as measures of students’ learning. Grades may, for example, combine assessments which address very different aspects of learning. Grades may also be influenced by anxiety about assessment and therefore not fully show the quality of students’ learning. Nonetheless, students and other stakeholders are strongly concerned about the grades that students achieve and, thus, it is worthwhile to explore what might affect these grades. Data about students’ age, gender and study programme was received from the universities study administrative office.

Participants

The study was conducted at as part of a larger triennial survey assessing students’ experiences of their studies and teaching-learning environment at a large social sciences faculty in a large and research intensive Scandinavian University. The students in the target group represent 18 undergraduate and 23 graduate programmes which are offered by seven departments. While the format of teaching and assessment may vary between departments and even individual study programmes, teaching at the undergraduate level typically involves a combination of lectures and discussion classes. At the graduate level, students typically attend seminars. The students’ academic achievement is assessed by end-of-semester examinations. Assessment formats vary between the individual courses; however, written examinations (both on-site and take-home) are the most common followed by oral examinations. By contrast, multiple-choice examinations are very rare at the faculty.

All 12,721 full-time students were invited to participate in the survey and 4,377 students responded, equalling a response rate of 34%. In the course of analysis, 38 cases were deleted due to missing responses on more than 25 percent of the survey’s items. Also, for this study we only included students for whom we could obtain information about their end-of-semester grade-point average. This reduced the sample size by a further 713. The final sample thus consisted of 3,626 students (2,519 undergraduate and 1,107 graduate students). The mean age was 23.5 (SD = 3.4) and the sample consisted of 47% male and 53% female students. An independent-samples t-test showed that the mean age was lower in the sample (M = 23.5, SD = 3.41) than in the group of non-responders (M = 25.1, SD = 5.1, t[9943.8] = 20.2, p < .001). Likewise, the proportion of female student was slightly higher in the sample (M = 53.1, SD = .50) than in the group of non-responders (M = .49, SD = .50, t[6704.2] = −4.35, p < .001). Table 1 shows the descriptive statistics of the study programmes within the departments.


Tab1e 1. Description of study programmes (number of students, proportion of sample, number of study programmes, average programme size, and range between smallest and largest programme) within seven departments.
Departments n % Programmes Average size Range
Business Communication 469 13% 9 52 1–97
Law 549 15% 4 137 31–263
Political Science 475 13% 4 119 11–341
Psychology 402 11% 2 201 123–279
Management 66 2% 1 66
Economics 1,505 42% 17 89 6–824
Business Technology 160 4% 4 40 14–56
Total 3,626 100% 41 88  
Notes: n = number of students in the sample; % = proportion of sample total; Programmes = number of study programmes within the department; Average size = average number of students per programme; Range = smallest program to largest programme within the department.

Statistical analysis

Concerning regression analysis, Dorman (2009) recently stressed the importance of taking into account the hierarchical nature of data when conducting research in educational environments. Multilevel linear modelling (MLM) takes the grouping structure into account by estimating variance associated with group (i.e. the programme) differences in average response (intercepts) and group differences in associations (slopes) between predictors and the dependent variable (academic achievement). This is accomplished by declaring intercepts and/or slopes to be random effects (Hox, 2010).

Concerning the order in which variables were included in the multilevel linear model, Hox (2010, p. 56) proposes the following bottom-up strategy. Initially, a model with only the dependent variable and the grouping variable is analysed. This null-model is useful because it provides an estimate of the intra-class correlation and because it provides a measure of deviance which serves as a benchmark for evaluating subsequent models. Next, explanatory variables at the individual level are entered into the model as fixed effects. In this particular study, the theoretically important variables (the three approaches to learning scales) were entered before entering control variables (age, gender, and study year). Finally, it was tested whether the correlations between approaches to learning and academic achievement varied significantly between groups (programmes). Hox (2010, p. 58) suggested that this is best done on a variable-by-variable basis. Thus, the variability of the three approaches to learning scales was tested one-by-one.

The following guidelines were observed when interpreting results. Regarding internal reliability, Cronbach’s alpha (α) values above .70 are generally considered acceptable (Abell et al., 2009). Concerning statistical correlation, Cohen (1988) suggested that Pearson’s r correlation coefficients greater than .1 but less than .3 be interpreted as weak correlation, correlation greater than .3 but less than .5 be interpreted as moderate correlation, and correlation greater than .5 as strong correlation. Finally, the advice from Abell et al. (2009) was followed to examine variables for extreme skewness being defined as skewness numerically larger than 3.0.

Multilevel linear modelling was implemented through STATA 13.

RESULTS

Before calculating students’ individual scores on each of the three approaches to learning scales, the scales’ 12 items were examined with regard to missing values. The proportion of missing values ranged between 0.2 and 1.3%. Missing values were substituted with the individual’s mean on the scale’s remaining items. According to Schafer and Graham (2002), this procedure is a reasonable choice when each group of items to be averaged forms a single and well-defined construct and when the reliability is high (a > .70). Since approaches to learning are well-defined constructs (Herrmann et al., 2016; Parpala & Lindblom-Ylänne, 2012) and the scales’ internal reliabilities (Cronbach’s a) ranged from 0.71 to 0.78 (see Table 2), the conditions for mean substitution were satisfied. Also, no variables with extreme skewness (larger than numerical 3.0) were observed.


Table 2. Descriptive statistics (Cronbach’s alpha, range, mean, standard deviation, skewness)
  α Range Mean SD Skewness
Deep approach scale .71 1–5 3.57 .69 −.32
Surface approach scale .76 1–5 2.59 .77 .45
Organised effort scale .78 1–5 3.53 .83 −.42
Examination grade(s) −3–12 7.29 2.97 −.58
N= 3,626

Table 3 shows a correlation matrix. End-of-semester examination grade was significantly correlated with deep approach (r = .077, p < .05), surface approach (r = −.199, p < .05), and organised effort (r = .137, p < .05). Table 3 also shows correlations between the three scales measuring students’ approaches to learning. The deep approach scale was negatively correlated with the surface approach scale (r = −.223, p < .05) and positively correlated with the organized effort scale (r = −.244, p < .05). The correlation between the organized effort scale and the surface approach scale was statistically significant albeit weak. (r = −.74, p < .05).


Table 3. Correlation matrix (Pearson’s r)
  1 2 3 4 5
1. Examination grade(s) 1.000        
2. Deep approach .077* 1.000      
3. Surface approach −.199* −.223* 1.000    
4. Organised effort .137* .244* .074* 1.000  
5. Age .048* .077* −.070* −.011 1.000
Note: * p < 0.05
N = 3,626

Multilevel model of academic achievement

A two-level hierarchical model assessed the effect of deep approach, surface approach, and organised effort on academic achievement. Age, gender and study year were entered as control variables. First-level units were 3,626 students while 41 academic programmes (18 undergraduate and 23 graduate programmes, respectively) were second-level units. The null-model (see Table 4) included the programme variable only. The intra-class correlation of .09 (.810/ [.810 + 7.913]) indicated the value of taking into account variation that can be attributed to the programme level.


Table 4. Multilevel linear model (MLM) of academic achievement predicted by approaches to learning (unstandardised correlations coefficients)
  Model 0 Model 1 Model 2 Model 3
(Constant) 7.475 7.976 8.170 8.189
Deep approach   −.126 (n.s.) −.084 (n.s.) −.073 (n.s.)
Surface approach   −.707*** −.617*** −.623***
Organised effort   .497*** .517*** .517***
Age     −.096*** −.097***
Gender (ref: male)     −.072 (n.s.) −.076 (n.s.)
First year     Ref. Ref.
Second year     1.307*** 1.302***
Third year     1.914*** 1.910***
Fourth year     1.798*** 1.770***
Fifth year     2.795.*** 2.778***
χ2 (programme) .810 .694 .528 .106
χ2 (student) 7.913 7.540 7.060 7.041
χ2 (slope, surface approach)       .039
Log likelihood −8933.1 −8844.0 −8721.4 −8717.8
Nprogamme 41      
Nstudents 3,626      
Note: Significance levels: * p < 0.05; ** p < 0.01; *** p < 0.001.

In Model 1 (see Table 4), the deep approach, surface approach, and organised effort scales were entered as fixed effects. Two of the three predictors were significantly associated with academic achievement; however, the deep approach scale was not. The model in which students’ approaches were included as explanatory variables (Model 1) was significantly better than the null model in which only the intercepts were included (Model 0), χ2(3, N = 3,626) = −2*(−8933.1 – −8844.0) = 178.2, p < .05. Thus, the explanatory variables as a group improved the model beyond the model produced by considering only the variability within programmes.

In Model 2, the control variables age, gender and study year were entered as fixed effects. Age was negatively correlated with academic achievement while the effect of gender on academic achievement was statistically insignificant. Students’ grade point average also tended to increase as they progress in their studies. The overall results from Model 1 did not change substantially when the control variables were entered, that is, Model 2 still showed that academic achievement was negatively related to the surface approach scale and positively related to organised effort scale while the association with the deep approach scale was statistically non-significant. The full model (Model 2) as a whole was significantly better than Model 1, χ2 (6, N = 3,626) = −2*(−8844.0– −8721.4) = 245.3, p < .05.

Because previous research has indicated that the effect of approaches to learning on academic achievement might vary across academic programmes, we explored various solutions for a further model by conducting a series of multilevel analyses in which the deep approach, surface approach, and organised effort scales, respectively and one at a time, were specified as random effects. Such analyses would provide indication of the extent to which the effects of deep approach, surface approach, and organised effort on academic achievement differ across programmes. First, Model 2 was tested against a model in which the deep approach scale was specified as a random effect; however, this alternative model did not differ significantly from the full model, Model 2, χ2(2, N = 3,626) = −2*(−8721.4 – −8719.4) = 4.0, p > .05. Next, a model was tested in which we allowed the effect of the surface approach scale on academic achievement to vary across programmes (see Model 3 in Table 4). The chi square-test showed that this alternative model was a significant improvement on Model 2, χ2 (2, N = 3,626) = −2*(−8721.4 – −8717.8) = 7.2, p < .05. Finally, a model showed that the effect of the organised effort on academic achievement did not vary significantly across programmes, χ2 (2, N = 3,626) = −2*(−8721.4 – −8720.0) = 2.7, p > .05. Thus, this series of analyses indicated that the effect of the surface approach on academic achievement varies across individual study programmes.

Summary of results

In summary, controlling for the effects of age, gender and study year, the full multi-level model showed a positive correlation between organised effort and academic achievement; however, we found no statistically significant correlation between academic achievement and the deep approach scale. The full model showed a significant negative association between the surface approach scale and academic achievement. Subsequent analyses suggested that the association between the surface approach scale and academic achievement varies across programmes, that is, Model 3 suggested that the (negative) effect of a surface approach was stronger in some programmes than in others.

DISCUSSION

The aim of the present study was to achieve a nuanced analysis of the relationships between students’ approaches to learning and academic achievement and how these connections might be moderated by the academic programme. We used multi-level modelling to investigate how the relationships between approaches to learning and academic achievement were moderated by programmes of study when age, gender, and year of study were taken into account. This is an important contribution to research in this area as we find that previous studies have often not taken moderating variables such as specific contexts into account.

We found a significant negative relationship between a surface approach and academic achievement and a significant positive relationship between organised effort and academic achievement. The relationship between the surface approach and achievement varied depending on the programme when age, gender, and year of study were taken into account.

Approaches to learning, organised effort and academic achievement

Our initial, descriptive analysis showed a positive but weak correlation between the deep approach to learning scale and academic achievement; however, when taking into account the hierarchical nature of the data as well as controlling for variables such as age, gender, and year of study, the correlation became statistically non-significant. While this result is seemingly out of line with the findings from the meta-analyses by Watkins (2001) and Richardson et al. (2012) - who found a statistically significant, positive but weak association between a deep approach to learning and academic achievement - it nevertheless corresponds with findings from other studies which also failed to identify statistically significant correlations between academic achievement and the deep approach scale (Diseth 2007a, 2007b; Diseth & Martinsen, 2003; Gjibels et al., 2005; Richardson, 1995; 2003; Trigwell et al., 2013).

As pointed out by other researchers (e.g., Entwistle et al., 2000; Vermunt, 2005), the non-significant correlation between deep approach and academic achievement may also be attributed to difficulties connected to measuring the qualitative aspects of academic achievement in quantitative research, especially when relying on examinations results as the measure of academic achievement. While multiple choice examinations are very rare in Danish higher education and while the intended learning outcomes at the institution in question generally stress the importance of understanding, it cannot be ruled out that the exam marking may not have fully rewarded deep understanding. Asikinen et al. (2013) discuss how assessments in higher education may not fully measure all of students’ qualitative learning outcomes and may not always reward a deep approach. This being the case, some of the deep learning taken forward by participants in the present study may not have been reflected in their grades. Further, a deep approach can result in complex and subtle understandings which students may struggle to express well under examination conditions or in the face of multiple coursework deadlines. For some students, anxiety about assessment might also limit their capacity to express fully the extent of their deep understanding.

It is also possible that the lack of relationship between the deep approach and achievement may be explained by the difficulty of accurately measuring the strength of students’ approaches to learning, particularly the deep approach. It is unlikely that different students mean exactly the same thing when they give a positive response to items such as ‘I look at evidence carefully to reach my own conclusion about what I’m studying’. It is possible to imagine students engaging in more or less sophisticated and skilled evaluation of evidence, which these questionnaires would not detect. Thus, caution should be exhibited about the apparent lack of association between the deep approach to learning and academic achievement. Future research should trial different quantitative measures and different methodologies for evaluating students’ approaches to learning, such as interviews or observation.

The results of the multilevel analysis indicated a clear negative relationship between academic achievement and a surface approach to learning even when moderating variables were taken into account. This finding is consistent with the meta-analyses by Watkins (2001) and Richardson et al. (2012) as well as recent studies that were not part of these meta-analyses (e.g., Lizzio et al., 2002). It is notable that the negative impact of the surface approach is significant even though the positive impact of a deep approach was not. This may be because the items representing the surface approach in our questionnaire focus on a strong and clear lack of understanding. The items for the deep approach relate to a range of strategies for seeking understanding thus present a more nuanced picture of students’ learning and understanding which may be more difficult to assess in examinations.

The relationship between organised effort and academic achievement was significant and positive in the multi-level model. This is consonant with the findings of Watkins (2001) and Richardson et al. (2012) but those reviews focused on the strategic approach to learning as defined in earlier research, which tended to place more emphasis on achievement motivation. Richardson et al. (2012) did, however, also consider effort regulation and time and study management from other instruments in the North American tradition (as opposed to the European roots of approaches to learning and studying). These measures both showed positive relationships with academic achievement which is a good fit with the findings of the present study.

It is likely that approaches to learning and studying exert their effects on students’ academic achievement within a complex web of other influences. Despite the increasing emphasis on the constructive alignment of the multiple facets of teaching-learning environments to encourage deep learning (Biggs, 1996; Biggs & Tang, 2011) different students will inevitably still have diverse experiences of the same teaching-learning environment (Hounsell & Hounsell, 2007). The work on creating more constructively aligned curricula has focused mainly on assessments which would reward a deep approach, whereas organised effort is more likely to be of value in any learning situation. Approaches to learning may mediate some relationships, such as those between self-efficacy and performance (Musso et al., 2013). Intelligence, working memory capacity, attention, anxiety management and cultural capital are also likely to form part of the picture of what influences academic achievement (Musso et al., 2013). Given this complexity – and the ways in which grades conflate various aspects of achievement - the modest relationships found between approaches and academic outcomes in the present study are to be expected.

The effect of programmes on the relationships between approaches and academic achievement

The relationships between the surface approach and academic achievement did vary significantly between different programmes of study. This was the case even though the programmes included in this study were all led from departments located within the same faculty. Programme effects on the relationships between the surface approach and academic outcomes are likely to be the result of a rich interplay between: the academic discipline; the nature of the topics under study; the particular student cohorts and the teaching and assessment practices developed in these contexts. Thus we would argue that the impacts arise in a nuanced interplay between multiple aspects of a particular setting. In future, qualitative research could be used to follow up on such quantitative differences in findings between programmes to give a richer understanding of what might have brought about these differences.

Limitations

The present study focused on one faculty in one particular university. Whilst detailed attention to particular settings is valuable, as it allows closer consideration of relevant context, it will be important to repeat the multi-level analyses with other samples. It would have been ideal for the present study if we had been able to access disaggregated data on students’ achievement on different types of assessment as this would likely have made the relationships with approaches to learning and studying clearer. It may be necessary to get to a level of detail where the particular kinds of questions asked in examinations and coursework are considered in order to fully make sense of the relationships between students’ approaches to learning and their academic achievement. Longitudinal data of this kind would be particularly powerful.

Missing data is a well-known problem in educational and social science research and the present study is no exception. While it was possible to meaningfully substitute missing values on the item level for the three approaches to learning scales, it was not possible to substitute for the large proportion of subjects missing due to either non-participation or lack of necessary background data from the university’s study administrative system. Hence, results from this study do not necessarily extend beyond the current sample and multiple studies representing multiple samples should be taken into account before generalising findings about the relation between approaches to learning, academic achievement, and study programmes.

The analysis described in the present study considers the impact of each of the approaches to learning on achievement in general terms across this particular population. This gives less insight into the impact of the particular combinations of approaches to learning chosen by each student at different points in time. It is possible, for example, that the same student might combine deep and organised studying for one assessment but for another assessment put in less effort and be less organised while still attempting a deep approach. A possible direction for future research would therefore be longitudinal consideration of individual students’ patterns of approaches to learning and studying over time and how these relate to the outcomes of particular assessments.

CONCLUSIONS AND IMPLICATIONS

The relationships between approaches to learning and academic achievement are important as they contribute to an understanding of how students can be supported to learn more effectively in higher education. Previous research shows that students’ choice of approaches to learning and studying can change (Biggs & Tang, 2011; Scouller, 1998) and that they can be taught to apply their approaches more successfully (Masui & DeCorte, 2005; Vermunt, 1995). While it can be challenging to influence students’ approaches to learning (McCune & Entwistle, 2011), well developed and consistent deep approaches are key if students are to be prepared for the supercomplex challenges they will face in their later lives (Barnett, 2007; McCune & Entwistle, 2011).

The lack of a significant relationship between a deep approach and academic achievement found in this and other research is therefore of concern and should be investigated in new ways. If assessment in higher education is not rewarding engaged critical thinking and students’ efforts to build connections between ideas then this is highly problematic. We suggest that future research should investigate this relationship more closely at the level of particular academic programmes. This would take into account the interacting effects of particular academic disciplines, faculties and teaching and assessment practices, as discussed in this paper. It would make sense to design extended scales measuring the deep approach which are tailored to what would make for high quality critically reflective learning in a particular context (Anderson & Hounsell, 2007). This would make it more likely that any relationships which existed between a deep approach and academic achievement would successfully be identified.

AUTHOR PRESENTATION

Kim J. Herrmann, Ph.D., is an associate lecturer at Aarhus University and editor-in-chief of the Journal of Danish University Education. His research interests include students’ approaches to learning and teacher-student interaction in small-class teaching. His recent work includes the book ‘Effektiv holdundervisning – en håndbog for nye undervisere på universitetsniveau’.

Dr Velda McCune is a Senior Lecturer at the University of Edinburgh. For the last 16 years her research has focused on understanding the learning experiences of students and academics in higher education.

Anna Bager-Elsborg, Ph.D., is a postdoctoral researcher at Aarhus University. Her areas of interest are disciplinary differences in university teaching, departmental cultures for change, and more recently big data about assessment and student learning.

REFERENCES

Abell, N., Springer, D. W. & Kamata, A. (2009). Developing and validating rapid assessment instruments. Oxford: Oxford University Press.

Anderson, C. & Hounsell, D. (2007). Knowledge practices: ‘doing the subject’ in undergraduate courses. Curriculum Journal, 18(4), 463–478.

Asikainen, H., Parpala, A., Virtanen, V. & Lindblom-Ylänne, S. (2013). The relationship between student learning process, study success and the nature of assessment: a qualitative study. Studies in Educational Evaluation, 39, 211–217.

Barnett, R. (2007). A will to learn. Buckingham, UK: Open University Press.

Biggs, J. & Tang, C. (2011). Teaching for Quality Learning at University (4. ed.), Maidenhead: Open University Press.

Biggs, J. (1987). Student Approaches to Learning and Studying. Hawthorn, Victoria: Australian Council for Educational Research.

Biggs, J. (1996). Enhancing teaching through constructive alignment. Higher Education, 32, 1−18.

Biggs, J. (2012). What the Student Does: Teaching for Enhanced Learning. Higher Education Research and Development, 31, 39–55.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

Danish Ministry of Education and Science. (2015). Bekendtgørelse om karakterskala og anden bedømmelse ved uddannelser på Uddannelses- og Forskningsministeriets område (karakterbekendtgørelsen) [Ministerial Order on University Examinations and Grading (the Examination Order)]. Retrieved from https://www.retsinformation.dk/Forms/R0710.aspx?id=167998

Diseth, Å. & Kobbeltvedt, T. (2010). A mediation analysis of achievement motives, goals, learning strategies, and academic achievement. British Journal of Educational Psychology, 80(4), 671–687. doi: 10.1348/000709910X492432

Diseth, Å. & Martinsen, Ø. (2003). Approaches to Learning, Cognitive Style, and Motives as Predictors of Academic Achievement. Educational Psychology, 23(2), 195–207.

Diseth, Å. (2007a). Approaches to learning, course experience and examination grade among undergraduate psychology students: testing of mediator effects and construct validity, Studies in Higher Education, 32(3), 373–388.

Diseth, Å. (2007b). Students’ Evaluation of Teaching, Approaches to Learning, and Academic Achievement. Scandinavian Journal of Educational Research, 51(2), 185–204.

Dorman, J. P. (2009). Statistical tests conducted with school environment data: The effect of teachers being clustered in schools. Learning Environment Research, 12, 85–99.

Entwistle, N. (2009). Teaching for Understanding at University: Deep Approaches and Distinctive Ways of Thinking. Basingstoke: Palgrave Macmillan.

Entwistle, N., McCune, V. & Hounsell, J. (2003). Investigating ways of enhancing university teaching-learning environments: Measuring students’ approaches to studying and perceptions of teaching. In E. De Corte, L. Verschaffel, N. Entwistle & J. van Merrienboer (eds.), Unravelling basic components and dimensions of powerful learning environments (pp. 89–107). Oxford: Elsevier Science.

Entwistle, N. & McCune, V. (2004). The Conceptual Bases of Study Strategy Inventories. Educational Psychology Review, 16, 325–345.

Entwistle, N., Tait, H. & McCune, V. (2000). Patterns of response to an approaches to studying inventory across contrasting groups and contexts, European Journal of Psychology of Education, 15(1), 33–48.

Entwistle, N. & Ramsden, P. (1983). Understanding student learning. London: Croom Helm.

Gjibels, D., Van de Watering, G., Dochy, F. & Van den Bossche, P. (2005). The relationship between students’ approaches to learning and the assessment of learning outcomes. European Journal of Psychology of Education, 20(4), 327–341.

Heikkilä, A. & Lonka, K. (2006). Studying in higher education: students' approaches to learning, self‐regulation, and cognitive strategies. Studies in Higher Education, 31(1), 99–117. doi: 10.1080/03075070500392433

Herrmann, K. J., Bager-Elsborg, A. & Parpala, A. (2016). Measuring perceptions of the learning environment and approaches to learning: validation of the learn questionnaire. Scandinavian Journal of Educational Research, 1–14. doi: 10.1080/00313831.2016.1172497

Hounsell, D. & Hounsell, J. (2007). Teaching–learning environments in contemporary mass higher education. In N. J. Entwistle & P. D. Tomlinson (eds.), Student Learning and University Teaching. British Journal of Educational Psychology Monograph Series II, Number 4 (pp. 91−111). Leicester, UK: British Psychological Society.

Hox, J. (2010). Multilevel Analysis: Techniques and Applications (2. ed.). New York: Routledge.

Karagiannopoulou, E. & Milienos, F. (2015). Testing two path models to explore relationships between students’ experiences of the teaching-learning environment, approaches to learning and academic achievement. Educational Psychology, 35(1), 26–52.

Kember, D., Biggs, J. & Leung, D. Y. P. (2004). Examining the multidimensionality of approaches to learning through the development of a revised version of the Learning Process Questionnaire. British Journal of Educational Psychology, 74, 261–280.

Kreber, C., ed. (2009). The university and its disciplines: Teaching and learning within and beyond disciplinary boundaries. New York: Routledge.

Lizzio, A., Wilson, K. & Simons, R. (2002). University students’ perceptions of the learning environment and academic outcomes: implications for theory and practice, Studies in Higher Education, 27(1), 27–51.

Marton, F. & Säljö, R. (1976). On qualitative differences in learning. I. Outcome and process, British Journal of Educational Psychology, 46, 4–11.

Marton, F. & Säljö, R. (1984). Approaches to Learning. In F. Marton, D. Hounsell & N. Entwistle (eds.), The Experience of Learning (pp. 36–55). Edinburgh: Scottish Academic Press.

Marton, F. & Säljö, R. (1997). Approaches to learning. In F. Marton, D. J. Hounsell & N. J. Entwistle (eds.), The experience of learning (pp. 39–58). Edinburgh: Scottish Academic Press.

Masui, C. & De Corte, E. (2005). Learning to reflect and to attribute constructively as basic components of self-regulated learning. British Journal of Educational Psychology, 75, 351–372.

McCune, V. & Entwistle, N. (2011). Cultivating the disposition to understand in 21st century university education. Learning and Individual Differences, 21, 303–310.

Musso, M., Kyndt, E., Cascallar, E. & Dochy, F. (2013). Predicting general academic performance and identifying the differential contribution of participating variables using artificial neural networks. Frontline Learning Research, 1, 42–71.

Parpala, A. & Lindblom-Ylänne, S. (2012). Using a research instrument for developing quality at the university, Quality in Higher Education, 18, 313–328.

Richardson, J. T. E. (1995). Mature Students in Higher Education: II. An investigation of approaches to studying and academic performance. Studies in Higher Education, 20(1), 5–17.

Richardson, J. T. E. (2003). Approaches to studying and perceptions of academic quality in a short web-based course, British Journal of Educational Technology, 34, 433–442.

Richardson, M., Abraham, C. & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin, 138, 353–387.

Schafer, J. L. & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.

Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35, 453–452.

Tait, H., Entwistle, N. J. & McCune, V. (1998). ASSIST: a reconceptualisation of the approaches to studying inventory. In C. Rust (eds.), Improving student learning: improving students as learners. Oxford: Oxford Centre for Staff and Learning Development.

Trigwell, K., Ashwin, P. & Millan, E. (2013). Evoked prior learning experience and approach to learning as predictors of academic achievement. British Journal of Educational Psychology, 83, 363–378.

Trowler, P. (2014). Depicting and researching disciplines: Strong and moderate essentialist approaches. Studies in Higher Education, 39(10), 1720–1731.

Trowler, P. & Cooper, A. (2002). Teaching and Learning Regimes: Implicit theories and recurrent practices in the enhancement of teaching and learning through educational development programmes. Higher Education Research and Development, 21(3), 221–240.

Vermunt, J. (1995). Process-oriented instruction in learning and thinking strategies. European Journal of Psychology of Education, 10, 325–349.

Vermunt, J. (2005). Relations between student learning patterns and personal and contextual factors and academic performance, Higher Education, 49, 205–234.

Watkins, D. (2001). Correlates of approaches to learning: a cross-cultural meta-analysis. In R. J. Sternberg & L. F. Zang (eds.), Perspective on thinking, learning, and cognitive (pp. 165–195). Mahwah: Lawrence Erlbaum.