Pushing the Frontiers of New Assessments – EDGE2011

My talk at EDGE2011 in Delhi was part of a panel that was presenting different thoughts on cutting edge developments in Assessments. I specifically focused on the tracking data, metrics and corresponding analytics that could be found by using games and simulations (or blends of the two).

When I talk of Games and Simulations, I typically classify and differentiate between various types in the following sense.

For me, leaving aside gaming for entertainment genres, games and simulations are a rich source of tracking virtually any kind of learning activity, experiential or intellectual. Some domains may be extremely abstract, of course, and not lend themselves to any clear ways of assessing learning. There is also the argument that games may not lend themselves to clear linkages with performance on the job. But, in essence, games and simulations allow learning and assessment solution designers to build rich reflective environments from which we can make informed judgments of performance.

For simulations and games, as also for Alternate Reality Games, the real complexity is in the design of the environment – the complex of objects and their changing relationships with each other – which by itself is also a dynamic emergent phenomenon. Take for example this hospital simulation developed by Indusgeeks and IIL.

This simulation is built up upon a complex environment of objects and their relationships within the added affordances of a Virtual World environment. There are some key advantages of these types of simulations.

Firstly, since the platform is that of a virtual world, players can visually observe the behaviour of other participants in the same scene. This lends itself to a way in which player behavior can be assessed and feedback provided. This is critical to solve many infrastructural challenges. LABs are expensive to build and maintain in a physical world and there are space-time limits to intervention by teachers. In a virtual world, actions can be recorded – thereby not only breaking the time challenge, but also by enhancing the teacher’s ability to capture and display best practices. Not only that, it allows teachers to personalize feedback by being part, directly or indirectly, of multiple virtual spaces concurrently. Imagine having a set of consoles at the command of a teacher – each monitoring a specific LAB – that would show indicators when a student is stuck or making a serious mistake!

Secondly, the scenario can be manufactured. Often, scenarios can be constructed that are difficult to replicate in real life. But artificially manufactured scenarios, provided they are sufficiently hi-fidelity, can provide an intense learning/assessment experience. By virtue of this manufacturing activity, the domain knowledge is exposed in a substantial manner, thereby supervening the need for elaborate teaching artifacts and curricular structures.

Thirdly, by being visually (and otherwise) immersive, these types of simulations provide a first-hand account of future real life experiences. This gives much more comfort than traditional assessments, specially to the potential employer, because she knows the learner has experienced the job situation even before she has been hired.

Fourthly, simulations lend themselves to new forms of collaborative construction. For example, we built a prototype with Indusgeeks, that showcased how virtual props could be used in SecondLife ( a virtual world platform) so learners could collaborate and perform.

Students building a model of a network

Fifthly, the environment can throw up rich data sets for subsequent analysis. Not only can we track behavior, but also compliance, knowledge, collaboration skills and a host of other competencies. Not only can we do it for one learner, we can do it across learners. We don’t any longer need to build top and bottom performer reports, but we can get insight that is far more fine-grained than that. This allows us to design various means of remediation as well by seeding simulators with specific conditions to train and test learners on.

In summary, simulation based training and simulation based assessments, are both key innovations, that must be broad-based into education. This has the power to transcend traditional curricula and assessment structures if used in a relevant and maximally effective way.

Read Full Post »

EDGE2011 New Delhi

The Emerging Directions in Global Education (EDGE) conference in Delhi was a power packed event. It saw a coming together of government, academia and private players in the education sector. It was an intriguing experience. There were some key ideas that I took away from the conference.

I believe now that there is full realization that no one player/stakeholder can meet the challenges which has had several implications – deregulation, shared governance, public-private partnership, increased dialogue, increased allocations in government spending, higher accountability – but most of all, a willingness to break out of the shackles of a dilapidated educational system and mindset.

I felt that this time around, as opposed to what I heard across conferences last year, there is a visible tension to make things happen in a participative manner. We have two very erudite and skilled people in Kapil Sibal, Minister, HRD and D Purandeswari, Minister for State, HRD. Listening to them speak with passion on their understanding on what ails education today in India, and their initiatives to make a change, I felt vastly more comfortable that a certain maturity has stepped in that can only do wonderful things if allowed to flourish.

It is still disconcerting that we are using old solutions to combat current and future problems, though. For me atleast, centralized governance as an overarching one-size-fits-all strategy, doesn’t cut any ice. Scale must meet scale.  Education has to be seeded locally, not centrally.

Secondly, we are throwing outdated educational technology and content formats/pedagogical techniques at the problems. Witness the NMOEICT initiative which will end up creating the world’s largest open courseware repository (of course, at this rate, anything we do will be the largest just given the kind of scale we will end up addressing), but is based on page turning WBTs and non-interactive videos. There seems to be little consensus on revisiting or creating anew, modes of addressing quality content/network provision.

Thirdly, I must say that the quality of educational administrators may need significant investment of time and effort to upgrade and enable to meet the new challenges. I found that a lot of very basic questions on people handling, capability building, government liasoning, operational problem-solving, research, infrastructure etc. were being asked that administrators should have already figured out with experience or formal training/facilitation.

Fourthly, I found a distinct trend, in the workshop topics, to promote ends over means. Excellence is being counted as an end and the definition of excellence seems to be making it to the top 100 institutions in the world, or educational governance (Governance Issues in Educational Institutions, Mahadevan, IIM-Bangalore) along the lines of possibly a Six Sigma for education, or best ways to partner with a foreign university (Making International Partnerships Work), or getting the right financing (Funding Education in the Emerging Market) for education.

There seems to be a strident market propaganda creating hype and fear. It starts with the references to scale (40 mn Higher Ed students by 2020). Then it goes into how inadequate the infrastructure is, how ill-paid the teachers are and how poor the students are to afford quality education. Then it segues into how the West has solved the problems using quality systems, educational financing and de-regulation (which it hasn’t really, but who argues with an E&Y or Parthenon Group report, right?).

And finally, it makes the case that if we do not push reforms for privatization (de-regulation, accountability, quality), open up the market for foreign partners (increasing student mobility through credit systems, influencing students by marketing and advertising, giving access to rural markets etc.) and increase the flow of private funds (financing of education, so that the poor 70% of India can add debt to their woes), then we are going to be unable to reap the demographic dividend, may need to import labour, lose our competitive edge, create internal strife and increase poverty.

This is an eminently laughable and indefensible thesis, a case of starting with conclusions and then gathering supporting facts. I promise to present a full case analysis for the E&Y and Parthenon reports when I get the time. But let me make three overarching statements:

1. The market knows no social justice
2. Industrial models of education have failed
3. Diversity and scale are the biggest challenges facing any educational system

One surprising omission was lack of any talk whatsoever on educational data – from collection to analytics. This is a disturbing piece because without completeness and accuracy of data, we are not in any position to make decisions. Look at the disclaimers on the DISE reports from NIEPA and the disclaimers from the Central Statistical Organization to know that there is something seriously lacking even in basic reporting of educational information.

Another surprising omission was a lack of discussion on new innovations (except for a brief session on New Frontiers in Assessments, which I thought was good and not just because I was one of the speakers) that could prove to be critical in a country this size. I would have thought that if 3-4 innovations capable of reaching mass scale were discussed, it would be invaluable for their evolution and for mass early acceptance.

The good thing in the Assessments panel, moderated by Madan Padaki (from Manipal who incidentally was given a well deserved award for his contribution to the education sector), was that I heard something which was music to my years. The speakers put together accounted for perhaps over 90% of the assessments being undertaken by private assessment companies in India. And a couple of them voiced an interesting dream – that perhaps, very soon, there would be no assessments (atleast not the kind we have subjected our children to so far). I liked that very much.

I liked S B Mujumdar (Chancellor, Symbiosis) when he talked about the triangle of excellence, inclusion and expansion, though the way he put it made me feel that he has no hope of a unifying strategy for all three to grow symbiotically (pardon the pun).

Shashi Tharoor did give an entertaining (and if may say so, rather vague) talk on why Liberal Arts are essential. He felt that the utilitarian approach of professional education may destroy liberal arts and that a Well formed mind is better than a well filled mind.

Workshop Tweets

Active engagement with knowledge. Tharoor #edge2011

Tharoor a liberal arts edu will help you enlarge minds, to make for ordered minds, to develop good thinking habits #edge2011

Well formed mind is better than a well filled mind. Tharoor. #edge2011

Tharoor utilitarian approach of prof education may destroy liberal arts…may not be so #edge2011

Creativity is key Rao #edge2011

Exam system has exhausted and destroyed young minds #edge2011

Liberate education from bureaucracy. This is the biggest obstacle Rao #edge2011

#edge2011 now for Rao’s vision lecture..

#edge2011 university at your doorstep Content can only be provided by institutions of excellence..invest in intellectual development Sibal

#edge2011 meta universities..sit in front of your laptop and decide your level of excellence..Sibal

#edge2011 leverage software capability. Teacher is no longer totally central…content becomes important..

#edge2011 community should take responsibility…

#edge2011 make freedom an integral part of the system..financing education is a commitment..a duty not a power to control Sibal

#edge2011 Sibal speak. Collaborate not seek conquest. We need a change of mindset..govt cannot drive, can only facilitate.

#edge2011 CNR Rao speaks… Real role of a teacher is to give..1500 papers…60 years of scientific research

#edge2011 shashi tharoor will be talking about why liberal arts education is important. Wish this was webcasted.

#edge2011 pillai quoting PM India is an emerging idea?

#edge2011 majumdar symbiosis exploit triangle of excellence, inclusion and expansion

#edge2011 interesting…sibal asked rao to precede him in lighting the inaugural lamp

#edge2011 Kapil Sibal just walked in..Minister HRD

At #edge2011 day 2 in Delhi. CNR Rao is going to be speaking today. Expect some cool insights.

#edge2011 increase participation in governance. Mahadevan

#edge2011 peer pressure and peer review is main driver of excellence. Mahadevan.

#edge2011 interesting…people are saying that public funded colleges could take a leaf out of the book of best practice institutions…

#edge2011 interesting debate…an IIM B perspective on acad perf etc is not universal. Mahadevan

#edge2011 its Mahadevan…not madhavan…oops

#edge2011 internal accruals of public colleges can fund performance incentives. Madhavan

#edge2011 first implementers of acad perf mechanisms should adopt both quantitative and qualitative measures. Madhavan

#edge2011 following Minzberg? Treat students as citizens not clients or customers

#edge2011 academic performance – how do we measure, how do we implement, how do we design metrics..Mahadevan

#edge2011 professors have a half life decay. How to accommodate them. Mahadevan.

#edge2011 teaching as a profession is the last career choice Mahadevan

#edge2011 education is a sellers market for India. Watch out for game changing regulatory changes. Mahadevan

#edge2011 self regulation is particularly important. Don’t depend on the govt. Mahadevan

#edge2011 three issues of the Internet – privacy, trust and security.

B Mahadevan IIM-B what are the core issues in higher edu today? #edge2011 same old stuff

At the edge2011 conference in Delhi

Read Full Post »