higher learning

Abstract

Learners who `strategically’ engage with information systems and computing curricula are becoming more prevalent in Higher Education institutions. Increasingly they demand more prescriptive advice from teaching staff to achieve particular grades and often fail to demonstrate the learning outcomes that the curriculum originally intended to deliver. The use of Biggs’ Constructive Alignment (Biggs 1999) to revise the curriculum has presented new opportunities to reflect upon the learning activities that need to be undertaken, and can result in a focus on teaching process rather than how to achieve a product. This paper presents a case study of how a business process modelling module has been developed over six years to progressively align the assessment tasks with learning activities and Intended Learning Outcomes (ILO). Early indications were that the explicit declaration of ILO made the learners appear more strategic. This shift provided new challenges for the teaching staff, particularly in terms of designing appropriate summative assessment, and presenting the ILO to learners in a meaningful and justifiable way. The use of form templates is described to assist the communication of ILO, as well as supporting the thinking necessary to build a constructively aligned curriculum. Such a curriculum places more emphasis upon the dynamics of learning interactions, and should explore what learners achieve outside of the timetabled curriculum. Finally this work is evaluated in the context of qualitative responses to open questions asked at the end of each iteration of the module.

Final publication can be found here.

1. Introduction

For some time now UK Higher Education (HE) institutions have wrestled with the pressure to demonstrate economic viability in an increasingly diverse ‘marketplace’ of students. From a teaching practitioner’s perspective, new cohorts of students present a vast array of challenges; students with ‘non-traditional’ academic backgrounds are welcomed, whilst the increase in mangerialism present within universities also promotes concepts such as ‘customer’, ’supplier’ and ’service’, that inevitably impacts upon how students perceive their educational experience. More students see HE as a route to improved employment prospects and proactively take a strategic stance towards their learning. Unfortunately the strategic approach does not rest easily with the ‘traditional’ university learning experience, and as a result the primary means of imparting knowledge (in the ‘academic’ sense) tends to favour students who approach HE from an academic, rather than strategic standpoint. Since the academic students appear to be an ever-decreasing minority, there is a need to address many of the undesirable symptoms of strategic learning; pre-occupation with assessment, little or no preparation before scheduled classes, lack of engagement with learning activities and a ‘minimum effort, maximum marks’ mentality towards the whole HE experience.

Two categories of learner are lucidly typified in Biggs (2003:p8-9) and also through a short video (’Teaching Teaching & Understanding Understanding‘, http://www.daimi.au.dk/~brabrand/short-film/index-gv.html). ‘Constructive alignment’ is proposed by Biggs (1996, 1999, 2003) as a means of designing and delivering curricula that supports students’ successful achievement of learning outcomes. It is based upon the premise that the students will take more responsibility for their own learning if they understand what the intended learning outcomes (ILO) are. Thus the ‘alignment’ comes from the communication of ILO, together with learning and assessment activities that are congruent with the intentions of the tutor, so that the students have greater confidence in managing their own learning.

“If students are to learn desired outcomes in a reasonably effective manner, then the teacher’s fundamental task is to get students to engage in learning activities that are likely to result in their achieving those outcomes”

(Shuell, 1986: 429)

Practitioners recognise that very often the assessment vehicle becomes the focus of the learning, and students will speculate as to how they will be assessed and use this strategically in their approach to the curriculum (Biggs 2002). Using the assessment itself to increase engagement in the learning is an approach offered by Gibbs (1999) though this could simply be addressed by the use of an examination, albeit with deficiencies of testing recall argued by Elton and Johnstone (2002).

2.   A case study in information systems

This case study relates to the teaching of a Business Process Modelling (BPM) module to final year undergraduate students. Students choose this module as an option and come from a variety of BSc routes: Computing (generic), Networking, Software Engineering, Web Information Systems and Services and Business Information Systems. As a result the skills of the students are wide-ranging, especially in terms of technical ability. This is often compounded by the expertise gained by students who elect to spend the third year of their degree on industrial placement, prior to the final year.

Discussions with employers confirmed that the University had a good track-record of producing technically able students. What appeared to be lacking however, was the ‘business aware’ student; somebody who had sufficient awareness of business concerns to be able to understand the needs of business and the potential impact of IT, making appropriate judgements based upon generic business analysis skills. Since the majority of the students had limited or no work experience this was understandable, but it was felt that the opportunity to explore simple business analysis practices would give the students a valuable set of transferable skills and therefore a significant advantage as they searched for employment.

The first incarnation of the module in 2002 followed a well-trodden path; two semesters, one hour lecture and one hour tutorial per week. Students were introduced to business modelling and analysis techniques through the lectures (or more often than not the lecture slides were downloaded from the University’s VLE, Blackboard), and tutorial activities were presented to practice and develop these new skills. Attendance at lectures was generally poor (20 students from a cohort of 75, 27%) though tutorials were better at 45% (24 students).

Assessment of the students was also traditional, with one piece of group work accounting for 40% of the marks of the module, and an individual submission for the remaining 60%. The group work required the students to collectively research and present an online resource from a selection of topics provided by academic staff. The individual assignment required the students to apply their skills to a case study, demonstrating the impact that their plans would have on the needs of the business.

The final marks were generally predictable, if a little disappointing. Out of 75 students only 3 or 4 appeared to understand and apply their new learning appropriately. The rest of the submissions failed to grasp the problems that were presented in the case study and mostly presented a list of hardware and software (with associated ‘costings’), followed by ‘recommendations’ that had no justification. Somehow they had missed the point.

3.   Aligning the curriculum to industry

The first step was to identify what characteristics an IT Business Analyst would have to demonstrate in their employment. Following discussions with employers, together with prior experience from within the teaching team, the following abilities were deemed to satisfy this need:

1. An ability to capture existing processes;
2. An ability to identify wasteful process interactions;
3. An ability to communicate process revisions and new processes;
4. An ability to use processes as building blocks for new processes;
5. An ability to make justifiable financial assessments of existing processes and financial projections for savings.

Reflecting back over the first year of delivery all of the materials made reference to these attributes and even included relevant exercises in the tutorials. What was lacking though was the opportunity to practice the skills in a variety of circumstances, so that the students could experience the decisions to be made when a ‘real-life’ situation presents itself.

Using Biggs’ approach to Intended Learning Outcomes (ILO) the attributes described above were revised and expressed as shown in Table 1.

After defining the learning outcomes that were intended to be conveyed, the approach to delivery was reconsidered. Attendance at lectures had been in steady decline for some time, and did not apply to this module exclusively. Feedback from some students included:

“Lectures are pointless. You can get the stuff from Blackboard anyway”.

A minority of students requested:

“More lectures please. I like to hear the stories from industry”.

In part this supported Biggs’ hypothetical scenario of two student ‘types’; Robert was very strategic and wanted to do the minimum work necessary to pass the module. Susan, on the other hand was a naturally academic, autonomous learner who required little guidance.

 

Table 1. Intended Learning Outcomes for a final year undergraduate Business Process Modelling curriculum.

It was considered important to attempt to replicate as much of the industrial situation as possible in the teaching and since much BPM activity is either conducted on a one-to-one basis or with small groups, the large group lecture had little to offer. The weekly lecture slot was replaced with a one hour surgery session, where students could bring specific issues they wanted to discuss. The tutorials then became the focus of the learning activity, and exercises from the previous year were combined with more practical activities to create a very participative curriculum.

At the beginning of the module the revised ILO were presented along with the scheduled activities, as part of the normal introduction. In prior years the ‘learning outcomes’ had always appeared rather abstract and so there was a tendency to pay little attention to them. This time there was some impetus to make explicit reference to them, and also make a continued reference to them during subsequent conversations.

As the semester progressed it was clear that after an apparent increase in student engagement, some of the strategic learning behaviours started to re-appear. Attendance at the surgeries was more or less the same as the previous year at 43% (23 students from a cohort of 54), but what was noticeable was that there was a greater variety of students attending than before. Clearly the students saw the benefit of having sessions where they could bring their own issues, however the issues they discussed were predominantly assessment-focused. Students who already demonstrated ‘academic’ ability, with high overall average marks in all of their modules displayed no less enthusiasm for the module and their attendance was predictably high.

Whilst the ILO were better aligned, the students still found the sessions fragmented until the end, and struggled to see the whole picture. After conducting the annual module review it was concluded that the students needed to be more aware of the curriculum in a holistic sense; the pre-occupation with alignment of learning activities to ILO had produced a disjointed curriculum.

4.   Communicating the intentions

It became apparent that while the revised ILO were much more explicit, there was still a significant ‘intellectual distance’ between the ILO statements and the actual learning activities. This distance was within the reach of the academic students, but still challenging for the ‘Roberts’.

In an attempt to re-dress the decomposition of learning, and subsequent fragmentation that had occurred in the curriculum, the ILO were mapped to scenarios that we expected a Business Analyst to find themselves in. Each scenario was presented simply as a description of the activity and two salient questions: “Why do I need to learn this?” and “How will I know when I understand it?”, as illustrated in Table 2. Although the questions are simple they have considerable significance. They make explicit the relevance of the learning by aligning it to an employment scenario, upon which students can project their own thoughts and begin to understand some of the issues that they will face. One student remarked:

“I thought that accountants worked with numbers in an office somewhere. I didn’t realise that I would have to sell an idea to them.”

Iteration 3 of the module during the 2004/5 academic year concentrated specifically upon the presentation of the learning activities; since the rationale for the ILO made the outcome more relevant to the learner, it seemed also applicable to make the presentation of this information a means for the students to navigate the actual learning materials. In common with many HE institutions the content was delivered via a virtual learning environment (VLE) and the ILO were presented as per the example in Table 2, in an HTML document.

Subsequently Table 2 was appended with target ‘questions’ for the students to consider as the basis of their learning. Each question was intended to focus the learning activity so that the students could relate what they were doing to the overall ILO, whilst also ’sign-posting’ them to much deeper activities.

Table 2. Presenting the learning activities to students.

The base questions for each activity were judged to be the lowest depth of achievement to obtain a pass mark; subsequent questions progressively would stretch the students, with the potential reward of a higher mark. Each of the base questions was also a hyperlink to some learning activity. The ‘deeper’ questions had no materials associated with them. Table 3 illustrates how this was presented within the VLE.

The teaching team debated this particular modification to the presentation, since it seemed to be reverting back to supporting a strategic learning approach, particularly since the ‘depth’ was actually being referred to as ‘Mandatory’, ‘Honours’ and ‘First Class’. Finally it was judged to be a high-risk experiment that should be performed anyway, since we were already satisfied that the learning activities were considerably better aligned with the ILO.

An unexpected outcome for the teaching staff was that the process of presenting this information to the students raised our awareness of the relevance of the task and thus the ILO could often be refined further. As such the tables have now been adopted as form templates during the planning stage as a means of assisting the curriculum development cycle.

5.   Assessing the learning

One of the challenges of assessment was to again provide suitable alignment with the learning activities and the ILO. Since students are now familiar with the prospect of receiving an explanation of how their work is going to be assessed, it was important that this particular artefact was seen to be congruent with the novelties described so far.

We were faced with the dilemma of charting a path through the learning that would enable less able students to develop their abilities so that they could achieve at least a pass grade, whilst also providing the guidance to stretch high achievers. We also wanted to assess in a way that would recognise unintended but significant learning, without having to specifically declare it as part of the grading criteria.

 

Table 3. Revised presentation of learning activities incorporating hyperlinks to actual materials for each of the ‘Mandatory’ questions.

Previously the assessment criteria had indicated what tasks needed to be completed to achieve a pass, and the individual honours classifications for higher marks. We wanted to move away from the prescriptive description of ‘things to show’ and get the students to engage with a qualitative approach; in the same way that teaching staff would assess what they had learned, the students needed to be able to demonstrate what they had learned both in terms of breadth and depth. Often a student who is dissatisfied with their mark genuinely believes that they have had a significant learning experience, and is disappointed when the specific criteria of an assessment grid illustrates the explicit criteria that they have not met. In many instances these criteria serve only to encourage strategic learning and the staff were conscious that the emphasis upon ILOs should not be wasted by a clumsy assessment rubric.

Reviewing the ILO confirmed that the intentions were correct, and the activities illustrated that we had explicated a rationale for the learning. The last piece of the jigsaw was to establish how learning could be demonstrated, without prescribing or constraining its achievement. The teaching team has found this aspect a particular challenge for a number of reasons:

1. Students, through their experiences of HE, are used to, and therefore most comfortable with, prescriptive assessment matrices;
2. Students can find open-ended assessments overwhelming with disastrous consequences;
3. Staff (including the teaching team of this module) were also practised at producing prescriptive assessments, in part to manage excessive marking workloads, but also to satisfy the audit trails of moderation for Quality Assurance.

Basing the assessment criteria upon the module ILO at one level seemed straightforward, but it is the articulation of the qualities that have to be demonstrated that is difficult to express. This process proved enlightening in that it forced the teaching team to think about how each of the ILO, and then the learning activities, would prepare a student to satisfactorily demonstrate their learning.

In keeping with our desire to explicate the rationale behind the assessment criteria, it was decided that we would attempt to describe how the assignment would be marked in relation to each of the ILO. A first attempt was in fact little different from the prescriptive marking grids that had been produced many times before; “illustrate three ways of…”, “describe how you relate X to Y…” and so on. At the other extreme we anticipated many focused, probing, questions if the explanation was too vague. In the end we settled for some elements of description, relying upon the relative potential success of the aligned learning materials, therefore attempting to relate to the assessment of professional problem solving skills better.

Already the students were coping with more open-ended learning activities, as a result of adopting the principles of constructive alignment for the first 3 years. Table 4 illustrates the assessment grid for the individual assignment, which provides a brief explanation of how each of the ILO will be assessed.

 

(Intended) Learning Outcomes		How will my submission be marked?
Apply a modelling notation to construct a representation of an existing business process		The key word here is apply; can you demonstrate that you have used recognised methods to capture a business process from a case study? You will need to understand the merits of particular modelling approaches and be able to reason in favour of your submission. Remember to articulate your thinking and don’t describe the history or theory of modelling.
Analyse business interactions and apply business process patterns to identify wasteful interactions		Can you evidence how you have discovered potential areas of waste in the case study? How do you illustrate your choice of process patterns? You will need to communicate your understanding of how business processes interact and how you have selected particular processes for consideration over other processes.
Apply a modelling notation to construct and explain a representation of a business process and argue a case for revision		Can you show somebody how an existing business process can be transformed in a non-technical way? You will need to illustrate the argument and demonstrate a valid case for revision. Have you considered how the process will affect the business?
Re-use business process components to organise and compose new business processes		Do you understand the feasibility of making your own generic process patterns, and how these can be used as building blocks for new processes?
Review a business process and argue a financial case for adoption.		Can you present a ‘business case’ for business process revision? Do you understand the need for this understanding to be re-used for different processes? Can you argue a robust financial case?
General learning outcome		You have presented your submission in a logical, structured way to a non-technical audience. There is an evident ‘route’ through the assignment and it is easy to find your evidence for your arguments. The submission is lucidly and concisely summarised before being explained in greater detail. Your writing style is factual and professional and relates to what is expected of the industry. You have ensured that you have only included evidence if it supports your case and there is no ‘waffle’. You have practised academic integrity and honesty.
Fail	Pass		Honours	First Class
Less than Pass, plagiarism evident or work not submitted.	The Learning Outcomes have been addressed satisfactorily with some areas of weakness.		All the Learning Outcomes are satisfied comprehensively.	All of the Learning Outcomes have been met in a way that demonstrates a deep level of understanding of both theory and application to the case study, together with creativity.

Table 4. Assessment criteria for the individual assignment.

At the bottom of the table is the same grading scale that is presented with the learning materials. As is evident from the whole approach of constructive alignment, each artefact demands constant refinement. Statements such as “Remember to articulate your thinking and don’t describe the history or theory of modelling” and “You have ensured that you have only included evidence if it supports your case and there is no ‘waffle’.” come from the experience of receiving too many assignments with redundant material.

6.   Reflections

The module is structured around a single case study. All of the learning activities are applied to that case study in and out of scheduled classes. The final ‘capstone’ individual assessment is different case study, which the teaching team publish at the beginning of the module, but will not discuss with the students. Thus students that can apply their learning to a new case study can therefore demonstrate many of the ILO.

Applying the learning to the case study gives something concrete to practice the new skills on, and contains enough real-life ambiguities and difficulties to discuss in class (and in an online discussion forum). Students are then faced with making assumptions, and justifying them based upon what they had already discovered about the system to be improved.

In one session a particularly quiet student said:

“You could always give the flip chart pen to the person who is contributing least to the discussion.”

Feedback like this has generally been scarce in the past, particularly with the technically-focused students. Often there is a pre-occupation with technology, and a wholesale rejection of the issues surrounding technology, which are of vital importance in the workplace.

From a delivery perspective there is also less prescriptive teaching and much more room for emergent learning. The alignment of the learning activities to the qualities that will be assessed has encouraged much more enquiry on the part of the students, and many more opportunities for students to reflect in and on their actions.

The introduction of an aligned curriculum clearly presents a significant benefit for the students, and the depth of learning that they have demonstrated in the final assignment is considerably better than with prior cohorts. In particular students like the collegial way of working with teaching staff to solve problems and apply their learning to new situations:

“I didn’t feel as if [the tutor] was lecturing us. He just worked with us and we learned from his experience.”

“The time went really quickly in class because everyone was busy sorting out the exercises. At one point I got stuck and [the tutor] helped me with my spreadsheet. Then I helped him with Facebook!”

Teaching staff find the formative feedback in-class much easier and far more stimulating; a wider range of topics are discussed and there is much more opportunity to discuss a hypothesis and propose strategies to resolve issues.

Undoubtedly there has been an inevitable ‘novelty’ attached to this module as it is the only curriculum in the course that has adopted a significantly different approach to delivering a learning experience. However the reaction was unanimously positive:

“The material was really refreshing and I didn’t miss the [lack of] lectures at all.”

“I really liked the tutorials because it all seemed relevant and would have been really useful on my placement.”

“I could relate what we did in class to what I’ll need to do. I’m sure this will be useful for interviews.”

One aspect that we were keen to convey was an appreciation of the value of the students’ work and how they could use this to select appropriate improvements and argue their case. Past experience of teaching finances on this module was generally poor and most of the students chose to exclude it from their submissions. The better communication of our intentions, and the subsequent relation of this topic to the scenarios has reinforced the relevance of this learning:

“I feel that I have achieved a lot. I can now see ways in which I can put my computing knowledge to use and save a company money.”

“I can see now how IT can save money, but more importantly I can see when NOT to spend money on new hardware if the numbers don’t add up.”

“The tutor advised us all to keep a journal or blog and I’ve never seen the point of it for technical subjects. When we did a role-play to interview the client I realised that there was so much going on I wouldn’t remember it all afterwards. I’ve kept a private blog for the last 6 weeks and it’s showing me all sorts of things that I wasn’t aware of. I’ve learned so much about myself. Thankyou.”

Overall feedback from the students has been much more forthcoming with this approach to delivery, which we feel is attributable to the extent to which open discussion and personal reflection is experienced in the module, and subsequently used as the basis of facilitating learning.

7.   Conclusions 

One conclusion is that the students need to be made aware of how they learn and why they need to learn. More open-ended, exploratory assessments such as applying learning to a case study only cause confusion if the students cannot see that it is the engagement with the process of learning itself that makes them ‘better’. From the student learning experience perspective, this education would be easier if it started earlier, rather than leaving it to the final year.

Making the learning activities relevant to the ILO is difficult and it is necessary to communicate explicitly the alignment to students. Presenting this information with a scenario-based rationale in simple language has helped students interpret the intentions. The use of scenarios has also reinforced the understanding of what the teaching staff were attempting to achieve, with the added benefit of refining the ILO for each successive delivery.

Investing the time in producing an aligned curriculum and its subsequent presentation with rationale produces noticeable improvements for module delivery – students are on task much quicker and are better prepared to question their learning rather than the curriculum design itself.

Constructive alignment has the potential to bring many positive aspects to a course but it is not until the whole curriculum approaches alignment that the significant benefits become apparent. As we approach the delivery for 2009/10 academic year we shall continue to amend and refine in pursuit of the elusive aligned curriculum.

References

Biggs, J., (1987). Student Approaches to Learning and Studying Hawthorn, Vic: Australian Council for Educational Research

Biggs, J., (1993). “What do inventories of students’ learning process really measure? A theoretical review and clarification” Brit. J. Ed. Psych. vol 83 pp 3-19

Biggs, J., (1996). Enhancing teaching through constructive alignment, Higher Education, 32, 347-364.

Biggs, J., (2002). Aligning the curriculum to promote good learning, Constructive Alignment in Action: Imaginative Curriculum Symposium, LTSN Generic Centre, November 2002. Available at: http://www.palatine.ac.uk/files/1023.pdf

Biggs, J., (2003). Teaching for Quality Learning at University. The Society for Research into Higher Education, Open University Press, McGraw-Hill Education.

Elton, L., and Johnston, B., (2002). Assessment in universities: a critical review of research. LTSN Generic Centre. Available at: http://eprints.soton.ac.uk/59244/01/59244.pdf

Entwistle, N., (1981). Styles of Learning and Teaching; an integrated outline of educational psychology for students, teachers and lecturers, Chichester: John Wiley.

Gibbs, G. (1999). Using assessment strategically to change the way students learn, in: S. Brown and A. Glasner (Eds), Assessment Matters in Higher Education, Buckingham, the Society for Research into Higher Education, Open University Press.

Marton, F., and Säljö (1976). “On Qualitative Differences in Learning - 1: Outcome and Process” Brit. J. Educ. Psych. 46, 4-11

Ramsden, P., (1992). Learning to Teach in Higher Education London. Routledge.

Shuell, T. J., (1986). Cognitive conceptions of learning. Review of Educational Research, 56: 411-36.

Slavin, R., (2000). Educational Psychology: Theory and Practice, 6th Edition, Englewood Cliffs, New Jersey , Allyn and Bacon.

Tags: BPM, constructive alignment, curriculum design, information systems

This entry was posted on Monday, February 23rd, 2009 at 11:28 am and is filed under Teaching. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.