Measuring Business Value of Learning Technology Implementation in HE Setting

(1)

www.bsc.es

Measuring Business Value of Learning Technology Implementation in HE

Setting

Nia Alexandrov, Education and Training Team, BSC

Computational Science and Skills Gap in HE Context

The skills considered hardest to find:

– scientists with HPC capabilities, – parallel programmers,

– algorithm developers,

– system administrator with high-end computing

experience.

(2)

Nia Alexandrov

3

Aims

Intersecting influences: pedagogical approaches, technological affordances, and the management of the processes at programs, departments and University strategy levels

– Decades technological developments have strong impact on teaching particularly when combined with a virtual learning environment putting technology in the position of an enabler and enhancer

– Quality teaching using collaborative learning is the bases for building of a set of research skills

– Management practices affect the success or failure of courses and impact motivation to learn

Intersecting Influences Model [1/2]

Nia Alexandrov

⁴

Legend:

M – motivator

H – hygiene factor

Factors standing

in overlapping

areas are under

intersecting

influences

(3)

Contributing Areas

Nia Alexandrov

5

Area Influence Impact Affected Change

Pedagogy: Ways of teaching Understanding the way we learn informs the design of the format of individual work and subject/unit assessment

Collaborative work

& tools bring additional learning outcomes: acquiring of interpersonal skills, social values, and professional attitudes

Technology Tools and their affordances &

constrains

Technological advances bring changes in the lecturing format

Management Motivational theory and business value assessment

Staff training, student – lecturer interaction,

Financial and structural

implication of ILT on organizational level

PEDAGOGY AND

IT IMPLEMENTATION

(4)

Integrated Learning Processes Model consists of four distinct stages and conforms to Kolb’s natural learning cycle and the work of James

Zull

Nia Alexandrov

7

Underlying Pedagogy

Construction

•

Classify

•

Associate

•

Analyze

•

Recall

•

Reflection

Abstraction

•

Visualize

•

Synthesize

•

Plan

•

Conjecture

•

Hypothesize

Concrete Experience

•

Navigate

•

Explore

•

Sense

•

Review

Action

•

Experiment

•

Discuss

•

Debate

•

Create

•

Practice

Tools Supporting Increased Complexity of Interaction

Type of teaching materials

Media forms Learning activities

Complexity of interaction

Elements of Collaborative work

Class of Web 2.0 tools

Facilitating the corresponding level

Level 5:

Discursive

Productive Project work Negotiation of learning objectives

Clearly Perceived Positive Interdependence/

Individual Accountability and Personal Responsibility/

Group Processing / Small Group Skills/ Interaction

Wikis/ Social Networking/

Syndication and notification/

Collaborative editing

Level 4:

Adaptive (by the tutor)

Adaptive

Practical session/

Laboratory work

Based on previous communication, teacher may refocus the dialogue and the student would re-conceptualize as result

Group Processing / Small Group Skills/ Interaction

Internal and interpersonal dialogue

Syndication and notification/

Collaborative editing/

Media Sharing/

Simulations/ Immersive Technologies

Level 3:

Interactive (at the level of actions)

Communicative Seminar/

team discussion

Teacher functions as moderator supporting students discussions and reflections on their work

Small Group Skills/ Interaction Two cycles of interaction/

dialogue are possible:

Peer-to-peer within the team and Tutor-student(s)

Social Bookmarking/

Media Sharing/ Forums / Quizzes & Tests

Level 2:

Reflective

Interactive Tutorial

There is a provision of feedback on student´s work

Interaction

One to one, tutor - student

interaction

Blogs/ Forums

Level 1:

Descriptive

Narrative Lecture Makes concepts

available

No collaboration needed

Mashups / Static pages

Nia Alexandrov

8

(5)

Gains of Implementation of IT

Teaching Methods and Curricula Design

Management Processes

Subject Specific

Use o f IT

Support of team work and individual accountability

Common timetable and progression path Access to teaching and administrative

resources irrespective of location

Remote access to relevant teaching resources including HPC resources

Gain

Development of problem solving, research and collaboration skills through implementation of CCCP activities

Streamlined administration Decentralised control over material update and student feedback

HPC and domain specific applications;

Visualisation and simulation environments;

Mathematical Modelling capabilities

Over multidisciplinary subject area

Nia Alexandrov

9

BUSINESS VALUE OF IT

IN HE CONTEXT

(6)

Measuring the Organisational Level of Performance Using the CMF

Business Value for IT - mainly industry driven emerging area

– Intel Corporation - M. Curley (2005) and D. Sward (2006) and

Innovation Value Institute - the National University of Ireland, Maynooth

The approach:

– introducing certain metrics for estimating the Business Value of IT – Linking the metrics with defining the baseline,

– defining the IT innovations to be introduced,

– defining the expected productivity gain and measuring Return On Investment within the given timeline

Nia Alexandrov 11

Reasoning for using IT-CMF in HE context

Measuring the level of implementation of IT as a teaching tool and in the management of the teaching/ learning process allows us to determine the organisational level of performance

The principles for good practice in HE relate motivation to study context

In management theory Herzberg links dissatisfies (hygiene factors) with job context.

Nia Alexandrov ₁₂

(7)

5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return of investment-new

understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected benefits based on strong alignment of IT investment objectives with the business objectives of the organisation

Nia Alexandrov

13

5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return of investment-new

understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected

benefits based on strong alignment of IT investment objectives with

the business objectives of the organisation

(8)

5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return on investment-new

understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected benefits based on strong alignment of IT investment objectives with the business objectives of the organisation

Nia Alexandrov

15

5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return of investment-new

understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected benefits based on strong alignment of IT investment objectives with the business objectives of the organisation

Nia Alexandrov ₁₆

(9)

5 stages of maturity of the organizational attitudes towards spending for IT

Ad Hoc – no defined processes and no future-proof investment decisions

Repeatable – focus on total cost ownership - seeking to control cost and quality of services

Defined – shift from TCO to return of investment-new

understanding that the business value of IT investment lies outside of the IT department

Managed – portfolio management, sophisticated approach to managing IT investments attempting to optimise investments based on a defined criteria and taking into account the organisational business strategy

Optimised – measurement and tracking of actual and projected benefits based on strong alignment of IT investment objectives with the business objectives of the organisation

Nia Alexandrov

17

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1 Ad hoc

No defined processes

Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2

Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

3

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4

Portfolio management Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.

Research publications increase based on achieving results impossible without IT investment and so on

(10)

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1

No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2 Repeata

ble

Cost control and services focus

Cost effective investments in providing services – centrally supported VLE , central DB , conferencing tools, mail service for all students,

3

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4

Portfolio management Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.

Research publications increase based on achieving results impossible without IT

investment and so on Nia Alexandrov

19

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1

No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2

Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

3 Defined

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials;

simulation and visualisation tools

4

Portfolio management Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.

Research publications increase based on achieving results impossible without IT investment and so on

Nia Alexandrov

20

(11)

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1

No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2

Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

3

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4 Managed

Portfolio management

Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.

Research publications increase based on achieving results impossible without IT investment and so on

Nia Alexandrov

21

Understanding of Business Value in the Higher Education Setting

CMF Level

Description CMF in the Higher Education Context

1 No defined processes Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

2 Cost control and services focus Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

3

Investment governance based on business cases

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks; availability of collaborative tools and digital learning materials; simulation and visualisation tools

4

Portfolio management Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

5 Optimised

Alignment with organisational business objectives

Strategies accounting for the benefits with great revenue distance – students taught in

technology rich environment with state-of-the- art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability.

Research publications increase based on

(12)

CMF

Level CMF in the Higher Education Context ACET CMF level in the UoR context

1 Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

Developing the E-LANE Integrated Metacognitive Processes Model (IMPM)

2 Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

Transition to BlackBoard based course support and non-paper office for ACET MSc programs and creation of student forum

3

Management or educational reasons for investment – centralised administrative systems for submission and management of work and marks;

availability of collaborative tools and digital learning materials; simulation and visualisation tools

Management and teaching using BlackBoard and RISIS at UoR as the VLE for local MSc courses and transnationally for the EM MSc in NeBCC; ACET student applications and alumnae forum integration

4

Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

Integration of ReaCTor (Immersive Virtual System)and IBM Themes Blue JS21 based Supercomputer into multidisciplinary teaching and research under the Computational Science Research Theme

5

Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability. Research publications increase based on achieving results impossible without IT investment and so on

Nia Alexandrov

23

CMF

Level CMF in the Higher Education Context ACET CMF level in the UoR context

1 Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

Developing the E-LANE Integrated Metacognitive Processes Model (IMPM)

2 Cost effective investments in providing services – centrally supported VLE , conferencing tools, mail service for all students, central DB

Transition to BlackBoard based course support and non-paper office for ACET MSc programs and creation of student forum

3

4

5

Nia Alexandrov

24

(13)

CMF

Level CMF in the Higher Education Context ACET CMF level in the UoR context

1 Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

Developing the E-LANE Integrated Metacognitive Processes Model (IMPM)

3 Management or educational reasons for investment – centralised

administrative systems for submission and management of work and marks;

availability of collaborative tools and digital learning materials; simulation and visualisation tools

Management and teaching using BlackBoard and RISIS at UoR as the VLE for local MSc courses and transnationally for the EM MSc in NeBCC; ACET student applications and alumnae forum integration

4

5

Nia Alexandrov

25

CMF

Level CMF in the Higher Education Context ACET CMF level in the UoR context

1

Ad Hoc decision resolving particular current needs – isolated cases of use of VLE, conferencing or web tools by some courses or departments

Developing the E-LANE Integrated Metacognitive Processes Model (IMPM)

3

4 Sustained investment in advanced technologies and tools facilitating research and collaboration and staff training for implementing the

technology in the teaching and learning process coupled with creation of cross disciplinary bodies for managing the developments

Integration of ReaCTor (Immersive Virtual System)and IBM Themes Blue JS21 based Supercomputer into multidisciplinary teaching and research under the Computational Science Research Theme

Strategies accounting for the benefits with great revenue distance – students

(14)

VLE, conferencing or web tools by some courses or departments (IMPM)

3

4

5 Strategies accounting for the benefits with great revenue distance – students taught in technology rich environment with state-of-the-art tools gain

advantage in terms of future proof skills and knowledge acquired, which in turn would reflect into better employability. Research publications increase based on achieving results impossible without IT investment and

so on…

Nia Alexandrov

27

HERZBERG THEORY AND

STUDENTS’ MOTIVATION

(15)

Relation between the principles of good practice in HE and Herzberg´s motivators

Principles of good practice in HE Herzberg´s Motivators in work environment

Student faculty interaction Recognition, Interpersonal relations - superior

Student collaboration Recognition, Interpersonal relations - peers

Encouragement of active learning Work itself

Prompt feedback Recognition, Company policy and administration

Emphasize on time-on-task Company policy and administration Communication of high expectations Achievement, Possibility of growth

Respect of diverse learning styles Recognition, Work itself

Nia Alexandrov

29

Inferring the Motivation Factors in HE

The case study was the MSc students of ACET from 3 consecutive intakes:

– 80 overall 61 returned the survey

The questionnaire is divided in three parts:

– questions on identifying the satisfying factors using IT, – questions on identifying dissatisfying factors using IT – questions on general technology use.

A posteriori approach to study design:

– Herzberg’s methodology

– categories were derived from the data and compared with the already

derived by Herzberg in the Motivation to Work case

(16)

Motivators and Hygiene factors for Students

Motivators Hygiene factors

Achievements IT Support

Recognition Content and Assessment

Student Involvement and Participation Quality of Teaching Teaching Methods Teaching and Learning

Resources

Feedback Administrative Support

Technology Features Facilitating the learning

Organization and Student Support

Interaction with Convenors and Professors

Visibility

Nia Alexandrov

31

Intersecting Influences Model [1/2]

Nia Alexandrov

³²

Legend:

M – motivator

H – hygiene factor

Factors standing

in overlapping

areas are under

intersecting

influences

(17)

Nia Alexandrov

33

Intersecting Influences Model [2/2]

Organising the Motivation Factors into sets with predominant influence of one or several areas – pedagogy, technology and management:

– Single area of influence:

• IT support (HF) and Technological features supporting learning (M) - Tech

• Quality of Teaching (HF), Content and Assessment (HF), Interaction with Convenors and Professors (M), and Teaching Methods (M) - Ped – Dual areas of influences

• The administrative support (HF) Man or Tech.

• Teaching and learning resources (HF) Ped or Tech.

– Triple (complex) areas of influences

• Achievements (M) and Recognition (M) - interlinked factors in Long-Range sequences are in the triple intersection and to be successfully facilitated require adequately functioning technology, appropriate pedagogical environment and supportive management processes.

Conclusions:

The capacity created by management practices was

identified by determining the Business Value of IT and of the organizational level of performance on IT-CMF in HE setting.

the position of technology as means to resolve or aggravate issues as facilitator was determined by inferring the Motivation and Hygiene Factors.

Pedagogy sets the principles of teaching that create the

needed skills set and as such has leading position but is

unable to resolve all issues on its own.

(18)

References

[1] N.Alexandrov, R. Ramirez-Velarde, and V. Alexandrov, ed., “Technological Advances in Interactive Collaborative Learning”, Taylor and Francis, 250 pages,Dec, 2012, ISBN-9781466502086.

[2] N. Alexandrova, V. Alexandrov and Raul Ramirez “The Role of Computational Science and Emerging Technologies in the Natural Sciences Education at University Level”, Procedia Computer Science, Elsevier, Volume 9, pp. 1789-1798, 2012.

[3] Martin Curley, “Managing IT for Business Value”, Intel Press, 2005.

[4] David Sward, “Measuring Business Value of IT”, Intel Press, 2006.

[5] IVI, Innovation Value Institute, http://ivi.nuim.ie/, Maynooth, Ireland.

[6] F. Herzberg, B. Mausner, B.B. Snyderman, “The Motivation to Work”, Transaction Publishers, Sixth Edition, 2003.

[7] Chickering, A.W., and Gamson, ZF 1991, Applying the Seven Principles for Good Practice in Undergraduate Education, New Directions for Teaching and Learning. Number 47, Fall 1991. San Francisco: Jossey-Bass Inc.

[8]Walker R, J. Voce and J. Ahmed, 2012, UCISA Report: 2012 Survey of Technology Enhanced Learning for Higher Education in UK, http://www.ucisa.ac.uk/tel

Nia Alexandrov

35