Can Teaching be Measured?

4175299981_614e7d9dc5 (1)

(image)

By Justine Rogers

Last week UNSW had its second ‘Great Debate’, introduced last year as a fun, accessible way for the UNSW community to explore a serious and stirring topic. (For a post on last year’s, click here)

Each team: professor-manager, non-prof academic, and student.

The topic: Of Course Teaching Can be Measured (it’s a 5.3!).

I was on the affirmative (which I knew going in would be tough).

Given it was a private event for staff and students, I’ve written this assuming some version of the Chatham House Rule applies.

The affirmative’s arguments were:

  1. Teaching can be measured, albeit imperfectly, and certainly better and more reliably than it is now.
  2. Teaching needs to be measured to enhance the quality, rewards and status of teaching.

The negative’s arguments were:

  1. Teaching cannot be measured, only learning experiences and learning outcomes can. 
  2. Teaching measures are flawed and unreliable.

The negative committed to the empirical questions, whereas I tried (unsuccessfully in the 4 or so mins we had) to engage both sides in the wider empirical and normative argument suggested in affirmative point 2: whether there is some positive correlation between measurement, and motivation, quality and status, and therefore whether a more robust measurement of teaching is worthwhile.

I wish we’d had the format and time to examine this: whether this is true, or whether, using research measures as example, such measures have too many biases, perverse incentives, and inefficient and/or demoralising effects to be of real value (even if it entails superficial value). 

I will share my main arguments here, some of which I am fairly convinced, many posed as part of my role on the affirmative side, and some raised in the spirit of fun and provocation. Above all, I think the topic raised several questions left that need to be contemplated, many of which I’ve posted below – so please share your thoughts!

Continue reading

Managing the demand and supply of lawyers (pre and post law school obstacles)

After reading an interesting post over at the Faculty Lounge on Israel’s recent extension of their article/trainee period to two years from one year, a few thoughts came to mind. The first is about the Israeli legal education supply of lawyers context in comparison to that of the US, and the second is about the Australian (and other) legal system’s management of the supply and demand for lawyers.

1. Israeli Legal Education Context.  Consideration of the Israeli supply of lawyers context in comparison to that of the US raises a few issues: the fact that: the US has 40 times the population of Israel; that there are over 200 US law schools versus 10 in Israel; and that there are roughly 40000 US law grads versus 1800 in Israel.  In other words  – in comparison to the US, Israel is producing significantly more lawyers per population (almost twice as many)  from significantly more law schools per population (twice as many).  And most of us think there is excess legal education capacity in the US.  This suggests a legal education excess capacity problem in Israel.

One solution to such an excess capacity problem is, as the post discusses,  to make it harder to become a lawyer – scaring off new entrants and demanding greater commitments from those that manage to make it.  The US does this through a few devices, including the demand for a first degree before starting law school (tuition and lost income during those undergrad years means a student investment up front of at least $100,000 if not many times that) and a post law school bar exam (that is passable by any conscientious law grad after putting in the four months needed alongside attending a bar prep program).  Arguably the Israeli two year internship costs the future lawyer less and may perhaps show what practice may be like, hence winnowing out those for whom it would be unsuitable.  But, as the Faculty lounge post suggests, neither is a perfect solution.

2. Applicability to Australia and beyond. The second thought that came to mind from the post was a result of my concern over a growing excess capacity problem in Australian legal education (and in other systems as well).  I thus wondered whether Israel’s proposed solution to their excess legal graduate capacity problem might have any relevance for Australia and others.  Australia, like Israel, also has too many law schools and too many law graduates (see my earlier post).  But, like all systems Australia also has barriers that might winnow out those insufficiently committed to the career or for whom the cost-benefit or risk analysis of securing that law job does not work.  Those barriers may include a double degree for LLB students or a prior degree for JD students followed by a relatively short trainee/study post law school requirement (e.g. College of Law plus some work experience).  Some LLB programs do not require the double degree, but the short post law school trainee/education period is still required.

There are many different approaches around the world to manage demand and supply of lawyers (another is the the strict limitations on the number of law schools).  But the diversity of approaches all have their own pros and cons.  Managing the demand for and supply of lawyers is complex and difficult.  Accordingly, it would seem to be a task for all of those involved in the process (the law schools, the different professional bodies, the regulators, and the courts) to work together.  It is also a task which should be informed through comparative consideration of foreign approaches (comparative consideration means treading carefully – considering the different contexts and functions and avoiding the usual pitfalls associated with transplantation).

Colin Picker

Cheating at University

4732885512_9bf97a8838Photo Credit

Justine Rogers

Last week I was asking students in my ethics class to discuss legal values and what ones they’d picked up from law school. They raised a range of things, from compassion to competition. But one student said, “No plagiarising, no cheating, being honest in your work!”. “It’s drummed into us from Day 1”, one added.

I was rather chuffed to hear this, but I am not sure I can or should be too pleased. The research shows that these are problems affecting all Australian universities, though unevenly across them and within the disciplines. Sydney University has just released part 1 of report, ‘an approach to minimising academic misconduct and plagiarism at the University of Sydney. Its focus is detection and prevention. The Report shows that most instances of misconduct were categories of negligence (lack of understanding or carelessness about how to cite and reference). The rest, the active fraud, is where it gets disturbing.

There’s the less-straight-forward (as far as severity of categorisation goes) collusion and recycling, but most of it is outright dishonest plagiarism and ghost writing, or getting someone else to do the work and submitting it as your own. Ghost writing, in particular, is becoming more prevalent and difficult to regulate. Students are taking advantage of sophisticated and therefore hard-to-detect online services, marketed to them, ones like MyMaster. These fraudster strategies affect most directly take-home assessments, but now students are adapting the technology available to cheat in exams. They are using their phones and watches to bring in material, using loo breaks to quickly check the internet, taking photos of confidential papers, and one I hadn’t thought of in my old cheating (paper-based) days:* paying impersonators to come and take the exam on their behalf. Other categories found in the report were fraudulent medical certificates or other bad faith uses of special consideration.

A summary of the Report’s recommendations (produced by the Academic Misconduct and Plagiarism Taskforce, Sydney University, 2015: 2): Continue reading

Pressure to publish in select journals results in distortions

Randy Schekman, Nobel prize winner, says his lab will no longer send papers to top scientific journals Nature, Cell and Science because of his belief that the pressure to publish in such journals distorts science. More and more, legal academics are coming under the same kids of pressure to publish in particular “prestigious” journals. As is the case in science, such pressure distorts the kinds of legal scholarship that academics choose to pursue. Since prestigious journals only rarely overlap with practitioner journals, for example, there is a risk that engagement between academia and the profession will decline. Is this simply a game we all accept we have to play for career progression, do we believe that it is ultimately of benefit to the legal academy to go along with this, or is it something on which we should, like Schekman, take a stand?

By Lyria Bennet Moses

What is the SRA getting the UK into?

By Alex Steel

John Flood’s post below highlights the radical nature of the SRA’s proposal.  To be fair to LETR it was primarily a research paper rather than an options paper, and there’s a lot in there that’s of interest to Australian academics – including an overview of the current state of UK education and practice, and useful literature reviews.  A couple of cautionary notes from LETR are apposite to the proposals to move to competency based outcomes: Continue reading

One Size Doesn’t Fit All in Legal Education

By John Flood

After digesting the Legal Education and Training Review report (LETR) for three months, England’s largest legal regulator, the Solicitors Regulation Authority (SRA), has delivered its response. This is important because the big legal regulators—SRA, Bar Standards Board, Ilex Professional Services—shape the structure of the qualifying law degree, and they commissioned LETR.

Without revisiting the LETR report in detail,[1] it recommended no radical change. Instead the SRA, according to its chief executive, Anthony Townsend, is to propose a radical programme of reform.

It has three elements: Continue reading

ALTA Conference, Teaching and Instructor Freedoms

This week’s ALTA conference was an interesting mix of papers. A number of people are talking about the ‘flipped’ classroom. This ‘new’ approach involves getting your students to read or do some other activity before they come to class.  It was interesting how new and unusual many people seemed to think this was. There was real concern about the possibility of getting students to do the work, and it became clear that some universities keep their teachers on a very tight rein – e.g., do not allow them to have class participation marks, do not allow a teacher to say to a class that they will not teach the class that day if they had not done the reading, and there was clearly a sense that many universities have so many rules that it makes it impossible for teachers to teach classes in the way they think fit.

There is always a tension, I guess, between too much and not enough regulation. The thing regulators of any kind (including teachers) need to keep in mind is the extent to which people will perform to expectation. If you regulate people as if they will behave badly they probably will. The challenge is to regulate in a way which suggests to people that they will do well so that they respond to those expectations and rise to meet them. Having some faith in teaching staff to do the right thing may be beneficial from that point of view alone.  The same applies to the classroom where students often perform extremely well when it is clear to them that they are expected to. 

By Prue Vines